The governing body announced a review last week but has decided to "remove any doubt of the review's integrity or independence" by moving aside.
Several gymnasts, including Olympians Becky and Ellie Downie and Amy Tinkler, have come forward to highlight the abuse they have suffered in training, saying the mistreatment of athletes had become "ingrained".
Tokyo 2020
History-maker Kinsella backing fervent Olympic fever to unite the nation
10/08/2021 AT 16:19
The Downies said they had been afraid to speak out previously, while Rio 2016 bronze medallist Tinkler said she made a formal complaint in December 2019 and is still waiting for a response.
A confidential helpline is being set up for British gymnasts who say they have suffered bullying or abuse.
"It is vital the Review is unequivocally independent with full resources to effectively deal with concerns raised by gymnasts," said British Gymnastics CEO Jane Allen.
"In the past week, the complexities have increased, and it is clear to retain the trust of the gymnastics community we have decided to recuse ourselves from any management of the Review.
"Our priority is to learn the lessons and ensure the welfare of all those within gymnastics. By stepping aside, we hope the Review can now proceed unimpeded."
Gymnastics
Page swapping Olympic heroics for Humberside after bagging brilliant Tokyo bronze
10/08/2021 AT 12:08
Gymnastics
Brave Biles bounces back from adversity to battle to bronze on final day of gymnastics at Tokyo 2020
04/08/2021 AT 15:08