The Studio Museum in Harlem announced Thursday that it had taken the significant step of parting ways with David Adjaye, the charismatic Ghanaian British architect who is building its new home in Manhattan. A library project in Portland, Ore., is moving forward without him. A sculpture park in Lincoln, Mass., canceled a show of his work planned for fall. And other cultural institutions from Princeton, N.J., to Liverpool, England, expressed serious concerns in response to the allegations of sexual harassment and misconduct against Mr. Adjaye that surfaced this week.

“The actions being alleged are counter to the founding principles and values of the Studio Museum,” Raymond J. McGuire, chairman of the museum’s board, wrote in a statement to The New York Times.

Mr. Adjaye himself in a statement said that he was stepping away from completion of the Studio Museum project “with the heaviest heart,” adding that “the prospect of the accusations against me tarnishing the museum and creating a distraction is too much to bear.”

On Tuesday, The Financial Times reported that three women, who were not named in the article, had accused Mr. Adjaye, 56, “and his firm of different forms of exploitation — from alleged sexual assault and sexual harassment by him to a toxic work culture — that have gone unchecked for years.”

For arguably the world’s first Black “starchitect,” the accusations have led to troubled speculation as to what sort of impact the matter would have on his business and future projects, including many already in the works. Mr. Adjaye has denied the accusations.

“I absolutely reject any claims of sexual misconduct, abuse or criminal wrongdoing,” Mr. Adjaye said in a statement released Tuesday by Kendal Advisory, a communications and crisis management firm. “These allegations are untrue, distressing for me and my family and run counter to everything I stand for.”

Mr. Adjaye added that he was “ashamed to say that I entered into relationships which though entirely consensual, blurred the boundaries between my professional and personal lives.” He said he “will be immediately seeking professional help in order to learn from these mistakes to ensure that they never happen again.”

Strict policies against executives dating subordinates are generally standard, given the intrinsic power imbalance. In 2016, for example, the former president of Lincoln Center, Jed Bernstein, was forced to resign because of his relationship with a staff member.

Mr. Adjaye, who has offices in London, New York and Accra, Ghana, and was knighted by Queen Elizabeth in 2017, has relinquished a number of roles in the wake of the allegations. These include being an architectural adviser to the mayor of London, and being part of the team working on a British Holocaust memorial next to the Palace of Westminster. Mr. Adjaye said in his statement that he did not want the allegations to “become a distraction” to those projects. He has also agreed to step away from a major Chicago housing project, The Chicago Sun-Times reported Thursday.

Mr. Adjaye’s globe-spanning architectural commissions include 130 William, a recently completed residential tower in Lower Manhattan, and a much-lauded charter school in a former Bronx ice warehouse.

Among the firm’s eagerly anticipated U.S. projects is Princeton’s new University Art Museum, in Princeton, N.J., currently under construction

“We find the nature of the accusations enormously troubling,” said James Steward, the director of the Princeton University museum. “It’s fair to say that most of our work with Adjaye is behind us. We have an obligation to all the people involved in this project to see it to completion.”

The Multnomah County Library, in Portland, Ore., said in a statement Wednesday that “Adjaye Associates is no longer associated with Multnomah County and the East County Library project,” but that Holst Architecture, a local partner, “will continue through design and construction as planned.”

Bedrock, a Detroit-based developer that hired Adjaye Associates for a major Cleveland waterfront revitalization, said Thursday that “in light of” the serious allegations, “we are evaluating the business associations as we continue to move the project forward.”

Mr. Adjaye recently made a foray into sculpture. An immense spiral made of crushed New York City limestone was featured at the Gagosian Gallery in 2021.

The Trustees, an organization that runs the de Cordova Sculpture Park and Museum in Lincoln, Mass., which planned to show an earthen sculpture by Mr. Adjaye this fall, said in an email Thursday that it had placed his exhibition “on indefinite hold.”

Because the accusations have only just surfaced, those professionally tied to Mr. Adjaye said they would only speak on condition of anonymity until they had more information.

But some people who have worked closely with Mr. Adjaye, and in many cases consider themselves his friends, suggested that the architect hand off work to associates, or step back from day-to-day operations, as the prominent architect Richard Meier was forced to do after sexual harassment accusations in 2018.

Whether Mr. Adjaye will remain involved in his commissions depends in part, people in the field said, on a project’s stage of development. Those well under construction, like the Princeton art museum, can proceed without him, though they will miss out on the architect’s previous star power. (The Studio Museum said the building will be completed by Adjaye Associates’ New York-based team and Cooper Robertson, the architect of record.) People in the architectural world also wondered aloud how those in Mr. Adjaye’s firm would react, including considering whether to resign.

Mr. Adjaye has stepped down as a trustee of the Serpentine Gallery, an acclaimed contemporary art museum in London. “We have accepted his resignation with immediate effect,” a spokesman for the museum said in an email. Mr. Adjaye has also resigned from the Serpentine committee that each year makes the high-profile commission for a summer pavilion (past recipients include Rem Koolhaas and Theaster Gates).

Several clients of Mr. Adjaye said that the accusations had jeopardized their projects — potentially halting progress, inhibiting fund-raising and damaging public perception — and that Mr. Adjaye had failed to give them a heads up on the Financial Times piece so that they could prepare their own damage control.

Whether the accusations against Mr. Adjaye will affect his completed or almost-completed projects remains to be seen. The Smithsonian’s National Museum of African American History and Culture in Washington, D.C., which opened in 2016, has been closely identified with Mr. Adjaye and its success is widely considered responsible for the architect’s celebrity status.

Lonnie G. Bunch III, the Smithsonian’s secretary and the founding director of the African American museum, declined to comment.

While some members of the public have decided to swear off the creative output of cultural figures accused of wrongdoing — such as Chuck Close’s paintings, Woody Allen’s films or Michael Jackson’s music — buildings in their solidity and permanence are more difficult to cancel. It is harder to tear them down or turn away.

Mr. Adjaye’s fall from grace is particularly stark in light of how high he had ascended. In 2021, when Mr. Adjaye received the Royal Institute of British Architects’ Royal Gold Medal, one of the world’s top honors, President Barack Obama in a video message described the architect’s work as “genius — pure and simple.”

In a statement Wednesday, RIBA said that if the organization’s “standards are shown not to have been met, we will not hesitate to take the appropriate action.”

A representative for National Museums Liverpool, for which Mr. Adjaye is redeveloping its International Slavery Museum and Maritime Museum, said in an email that it takes “the allegations described very seriously.”

It is unclear how other countries where Mr. Adjaye has projects will respond to the accusations. In many parts of Europe and Africa the reaction to the #MeToo movement has been muted. Several cultural figures who lost work in the United States following allegations of sexual harassment, including the singer Plácido Domingo and the ballet master Peter Martins, found work in other countries.

In recent years, Mr. Adjaye has won a host of high-profile commissions in the Middle East, Africa and Asia, including Ghana’s national cathedral (a personal project of the country’s president); part of a mega-city in Saudi Arabia; and a museum in Nigeria to showcase African art. He’s also designed the forthcoming Kiran Nadar Museum of Art, a vast contemporary art museum to be built in Delhi.

In Ghana, where Mr. Adjaye’s success is a source of great pride, the news media has widely reported the allegations — often emphasizing the architect’s vehement denials.

But Sam Okudzeto Ablakwa, a Ghanaian opposition politician, said in a telephone interview that Mr. Adjaye should step away from the cathedral project and “be told to clear his name before having any more involvement.”

“This is supposed to be a symbol of holiness and virtuousness,” Mr. Ablakwa said. “These allegations do not fit with that.”