Ba, who lit up the Premier League with his powerful style and goalscoring ability, hopes the Blues do not end up in a long rebuild cycle this season and beyond.
Chelsea's ownership changes have seen co-owner Todd Boehly take charge of their transfer strategy this summer, to mixed success.
"I'm scared that Chelsea will go through a difficult period of time, even more difficult than what Manchester United have gone through or Arsenal went through when their manager left," Ba told i News.
"We're not talking about a manager going, we're talking about the owner, the one who set the tone.
"It was unbelievable what [former owner Roman] Abramovich did and they are changing the whole structure. Marina Granovskaia will be a big blow to lose because the work she did at Chelsea was incredible.
"It's the one that I am most pessimistic about. I hope I'm wrong."
