Most coaches avoid it if at all possible. In Jackson's case, I can understand the knee jerk reaction of the fans wanting to see him benched. The thing is though not a lot of good can come out of it beyond in the immediate moment.
Most coaches (especially a playoff team) will ride it out with the horse that got them there. I mean maybe Flacco comes in and is effective, maybe not. What is his likelihood of pulling out a win?
Let's say he does pull out the win, now you have yourself a full blown QB controversy entering the next round (and perhaps the offseason). It's a Pandora's Box for the coach and the team.
The QB position is unique like that.