It was the German's first victory at the Hungaroring and second of the season, bringing his career tally to 41 to equal three-time world champion Ayrton Senna's mark.
It was also the first time since the 2013 Brazilian Grand Prix, a run of 29 races, where Mercedes failed to score a podium finish with either car with Hamilton sixth and Rosberg eighth.
Hamilton was on course to lose his championship lead, first after running wide at the chicane on the first lap when he felt Rosberg crossed his line and then losing time when he pitted for a new wing and then having to a drive-through penalty after contact with Red Bull's Daniel Ricciardo.
But in defending second with a few laps to go, Rosberg clashed with Ricciardo at Turn 1, giving the Mercedes a puncture and damaging Ricciardo's front wing.
Rosberg was forced to pit and rejoined eighth, two places behind Hamilton who had fought his way back through the field to increase his championship lead to 21 points.
Hamilton and Rosberg both bogged down from the front row at the start, allowing Vettel and Kimi Raikkonen to get ahead, with Rosberg running third ahead of Hamilton.
Ferrari looked on course for its first one-two since the 2010 German Grand Prix before a MGU-K problem cost Raikkonen power. He eventually retired after the team were unable to fix it.
The virtual safety car was called into action when Nico Hulkenberg's Force India suffered a front wing failure, pitching the German head-on into the tyre barrier at Turn 1.
When it became clear there was too much debris on the track, the real safety car was sent out, bunching the field up to cost Vettel his sizeable lead - setting up a thrilling finale.
Daniil Kvyat scored a career-best second, to become the youngest driver to score a podium since Vettel in Italy 2008, the Russian benefitting from Ricciardo's clash with Rosberg late on.
Ricciardo survived the contact, which the stewards decided to take no further action on, with Rosberg to give Red Bull a double podium with third.
Toro Rosso's Max Verstappen, who had a drive-through penalty for making contact with Williams' Valtteri Bottas, battled his way up to a career-best fourth place.
The drama in the second-half of the race allowed Fernando Alonso to finish fifth, scoring McLaren-Honda's best result of the season, with Jenson Button finishing ninth.
Romain Grosjean was seventh in the Lotus while Marcus Ericsson gave Sauber an unlikely point with 10th, a few seconds clear of team-mate Felipe Nasr.
Williams failed to score for only the second time this season, with Massa - who was given a time-penalty for being out of position on the grid and forcing an aborted start - 12th and Bottas 13th.