If two cars travelling the same direction contact each other such that the front axle of one car bumps the rear axle of the other (the car in front), it's usually the car in front that has the safety problem. This is the same principle behind the PIT maneuver.
Otherwise (changing lanes, left turn across traffic), I think it’s more “not ahead of ME!” than about the signal itself.