The governing body announced a review last week but has decided to "remove any doubt of the review's integrity or independence" by moving aside.
Several gymnasts, including Olympians Becky and Ellie Downie and Amy Tinkler, have come forward to highlight the abuse they have suffered in training, saying the mistreatment of athletes had become "ingrained".
- 'Heartbroken' Amy Tinkler says negative experiences led to gymnastics retirement
- 'Beaten, starved and screamed at' – Shocking abuse claims from British gymnasts
History-maker Kinsella backing fervent Olympic fever to unite the nation
The Downies said they had been afraid to speak out previously, while Rio 2016 bronze medallist Tinkler said she made a formal complaint in December 2019 and is still waiting for a response.
A confidential helpline is being set up for British gymnasts who say they have suffered bullying or abuse.
"It is vital the Review is unequivocally independent with full resources to effectively deal with concerns raised by gymnasts," said British Gymnastics CEO Jane Allen.
"In the past week, the complexities have increased, and it is clear to retain the trust of the gymnastics community we have decided to recuse ourselves from any management of the Review.
"Our priority is to learn the lessons and ensure the welfare of all those within gymnastics. By stepping aside, we hope the Review can now proceed unimpeded."
Page swapping Olympic heroics for Humberside after bagging brilliant Tokyo bronze
Brave Biles bounces back from adversity to battle to bronze on final day of gymnastics at Tokyo 2020