The former Manchester City midfielder joined the Superleague club in September on a free transfer, but the Ivorian played just five times during his short stint in Greece.
Toure, who won the Africa Cup of Nations with his country and was named African Footballer of the Year four times, attended training on Tuesday to say his goodbyes after his second spell with the club ended in disappointment.
The three-time Premier League winner had a spell with Olympiacos in 2005 before moving to Monaco, but the club have released a statement confirming his departure.
Arsenal through to Europa League last eight despite limp second-leg defeat
Yaya Toure is a free agent again
Image credit: Getty Images
It read: "Olympiacos FC and Yaya Toure would like to announce to the fans that they have mutually agreed to end their cooperation.
"Yaya is and remains an important member of the Olympiacos family, who has marked a central part of the history of the club. Olympiacos would like to thank him for everything he has offered to the Club.
Yaya will remain deeply devoted to the club and wishes Olympiacos lots of success.
"Yaya is leaving as a friend and the doors at Olympiacos will always be open for him. Our paths will surely cross again."
Greek radio station Sport24 reported that Toure had fallen short of manager Pedro Martin's expectations, and the player is now free to join another club.
Aubameyang faces a test of his mettle in Arsenal's blockbuster season finale
Arsenal 'have to stop making errors costing goals' - Arteta