Is there an NFL playoff scenario where the Patriots would play the Colts in round 2?

The Colts are the #1 seed in the AFC, and the Patriots are the #3 seed. So doesn't that mean, no matter what, if the Patriots win against the Ravens, they would have to play the Chargers in the second round?