Chelsea fan here. I absolutely feel your pain - I refused to go when FSW was our interim manager (it was helped by me living in Stoke and being skint then, though, tbf)
He didn't apologise for what he said about us (“We don’t need to give away stupid plastic flags to our fans to wave, our supporters are always there with their hearts and that is all we need. It’s the passion of the fans that helps us to win matches, not flags. Chelsea fans lack passion.”) so I wouldn't hold out for an apology if I were you.
The atmosphere at the ground was awful, apparently and, as others have mentioned, had match going fans at loggerheads, some of which ended up with them fighting. It was far from ideal.
Luckily for us, he was only interim - hopefully you'll be rid of him soon enough. Our games will be a laugh, though, he'll be being booed from all sides!