It could be that they wanted the seat they had...like maybe they wanted *that* window...or they might have expected you to move, or they might have honestly not cared. I know some folks just don't mind either way, so they will sit regardless and won't move. And some might figure that the bus will fill up again anyway, and the person next to them is safer to keep as a seatmate.

There isn't any rule against moving, though. If you want to move, you move. On the subway in NYC and buses here, it is totally usual to see people relocate when they can have more space to themselves. It is not offensive at all so you shouldn't have to worry about hurting anyone's feelings.