Andrezieux's ground was so small that they moved the game to nearby Saint-Etienne but even without home advantage they were too good for their more illustrious opponents and won with a goal in each half.
The defeat extended Marseille's winless run to seven matches in all competitions but they were not the only Ligue 1 side who failed to advance to the last 32.
On Saturday, Montpellier, currently riding high in the league in fourth place, conceded in stoppage time to lose 1-0 at third-tier side Entente SSG.
Angers also lost to an amateur side, beaten 1-0 by a Viry-Chatillon, while Nimes Olympique were hammered 3-0 by Lyon Duchere.
Among the Ligue 1 clubs who did manage to qualify for the next round were Olympique Lyonnais, who beat lower league Bourges 2-0, and Stade de Reims, who beat Lens by the same scoreline.
Bottom club Guingamp won 4-2 away at Stade Pontivyen, while second-bottom AS Monaco also got a rare win, knocking out Canet Roussillon 1-0 away from home.
League leaders Paris Saint-Germain head to fourth-tier Pontivy later on Sunday.