The 63-year-old has overseen an underwhelming World Cup qualifying campaign so far with Turkey currently third in Group I after six games.
Terim's side won just three matches, sitting behind joint-leaders Croatia and Iceland, but off-field problems have also played their part in his exit.
The former Galatasaray manager was allegedly involved in a brawl with a restaurant owner in the Turkish town of Alacati earlier this month, according to reports.
The Turkish Football Federation said in a statement on their website: "Some non-football issues have been wearing out our football director Fatih Terim and the management of (the) Turkish Football Federation.
"The two sides agreed that it would be more healthy for both parties to part ways."
Terim previously managed Turkey between 1993 and 1996 before taking over again between 2005 and 2009, with his latest stint beginning in 2013.