The number one seed won the first set but her young countrywoman fought back and forced a decider which went to an enthralling tie-break.
And the Charleston native secured her place in the semi-final with a 1-6, 6-4, 7-6(4) triumph.
Serena had beaten older sister Venus in another three-set clash on Thursday - and her first-round match against another American, Bernarda Pera, had also gone the distance.
Coco Gauff falls to Jennifer Brady in Lexington
Serena Williams pleased with tough test on WTA return
Rogers will now face Switzerland's Jil Teichmann, who beat Cici Bellis 6-2, 6-4.
Earlier in the day, Jennifer Brady defeated the conqueror of Johanna Konta, Marie Bouzkova, 6-1, 6-2.
Coco Gauff, the 16-year-old sensation, plays Ons Jabeur in the day's final match.
Serena Williams stunned by Shelby Rogers at Top Seed Open
Serena Williams survives scare to defeat sister Venus in epic comeback