Djokovic has faced fierce criticism after he, Grigor Dimitrov, Borna Coric and Viktor Troicki contracted Covid-19 following the charity event in Serbia and Croatia.
Amritraj told Reuters that while the world number one's intentions were "absolutely correct", and that he had followed local government protocols, he should have went the extra mile to ensure the health and safety of everyone involved.
"In hindsight, yes, obviously he could have taken a lot more precautions before running the event," the 66-year-old television pundit told Reuters by telephone from Los Angeles.
Nick Kyrgios hits back at Borna Coric on Adria Tour - 'Do you have rocks in your head?'
"In spite of certain local government mandates, I think it is critical to follow some common sense."
Djokovic has apologised and admitted it was probably "too soon" to run the event, which was played in front of packed stands in Belgrade.
Amritraj, who reached a career-high ranking of 18 and beat Bjorn Borg, Rod Laver and Jimmy Connors in his playing career, thought the tournament would have been good for tennis as it would bring the sport back on television.
Djokovic must use ATP presidency to make amends for Adria Tour debacle - Tennis Legends
"As days progressed, I realised that they were going to put about 1,000 people in a stadium and I really didn't know how big the stadium was," he added.
"And later on...after finding out that the event itself had pretty much a full house from the perspective of fans and none of this social distancing, masks, all of that were even in play, it became very much more of a concern."
Djokovic also heads up the players council on the ATP Tour and Amritraj, who held the same post for four terms from 1989, said it came with a "huge" responsibility.
"Same for Roger (Federer) and Rafa (Nadal), whatever they say or do is going to have a great degree of influence over the sport worldwide and what people think," he added.
Amritraj said the Adria Tour debacle showed there should be no crowds if, as planned, the main circuits restart in August after a gap of five months.
"The crowd usually makes a huge difference," he said.
"Unfortunately you will lose that element of the match. It'll be pretty straightforward to see a spectacular point and there'll be no applause in the way.
"So that certainly will affect the players. But again, it's something that we have to get used to, and it's better than not having it at all."
Dominic Thiem defends Novak Djokovic: 'He didn't break any law'
Novak Djokovic: Criticism after Adria Tour a witch-hunt