In a hypothetical perfect lock, they would fall back down. But in real life, every lock has slight imperfections, like not all of the pins being exactly the same diameter, and not all of the pin chambers being perfectly lined up. It's these imperfections that let previous pins bind and stay up when you move on to picking other ones.