Argentina appeared to have won the Cup when substitute Cesar Delgado put them 2-1 ahead in the 87th minute. But Adriano levelled with the last kick of the game and his seventh goal of the competition.
The goal sparked a brawl after Brazil celebrated in front of the Argentina bench and referee Carlos Amarilla called for police intervention.
Argentina's players squirted water at their opponents while Adriano was booked for taking his shirt off as he celebrated.
Copa America 2004
Brazil defender Luisao taken to hospital
26/07/2004 AT 08:11
Andres D'Alessandro and Gabriel Heinze missed Argentina's first two penalties in the shootout as Brazil, who had rested their top players for the tournament, celebrated an unlikely triumph, converting all four of their efforts.
Brazil had also drawn level at 1-1 with the last kick of the first half when central defender Luisao glanced in Alex's free kick. Luisao had earlier given away the penalty from which Cristian Gonzalez opened the scoring in the 20th minute for Argentina.
Argentina, inspired by an excellent performance from Carlos Tevez, dominated nearly the entire game.
They quickly took a stranglehold on the game, winning almost every ball in the midfield and putting the Brazil defence under constant pressure.
Zanetti gave Julio Cesar an early test with a long shot which the Brazil goalkeeper just managed to turn around the post.
Mauro Rosales shot across the face of the goal from a difficult position before Cristian Gonzalez was tripped by Luisao in the penalty area.
Gonzalez took the kick himself and sent Julio Cesar the wrong way as he converted his third penalty of the competition.
As Argentina continued to pour forward, Luis Gonzalez burst through the middle and unleashed a low shot which Julio Cesar did well to turn around the post.
But Brazil equalised out of the blue in first-half injury time. Alex, almost anonymous until then, took a free kick on the left and Luisao atoned for the penalty by rising above the defence to score with a glancing header.
Argentina's dominance continued after halftime and they should have gone ahead in the 50th minute when Rosales crossed from the right and the ball found Tevez in an excellent position but he failed to connect.
Rosales had a shot blocked by a defender in the 62nd minute after a brilliantly worked opening by Tevez. Substitute Cesar Delgado fired over and Tevez was twice denied by Julio Cesar.
The game came alight in the 87th minute when Delgado scored from a rebound after the Brazil defence had failed to clear his own cross.
But in injury time, Adriano scored on the turn to send the game straight to penalties, as extra time is not played at the Copa America.
Copa America 2004
Adriano giving Ronaldo food for thought
26/07/2004 AT 07:47
Copa America 2004
Brazil knock spots off Uruguay
22/07/2004 AT 09:24