Yeah and some people use them to their advantage. For instance, notice that President Obama uses "umms" to ensure that he's never not making a sound. Makes it harder for an interlocutor to interrupt.
Huh? That's funny--I would have said the opposite. It was from watching Obama's speeches that I learned that silent pauses can be much less annoying than "umms". If you want to see someone using a lot of "umms", watch Justin Trudeau.
