You sound like you might have CAPD. A common coping mechanism is lip reading to help fill in missing auditory information, often without consciously realizing it.

I don't know what your situation is, but my sons just don't use the phone. I handle all phone calls.

I'm so used to us just side stepping such issues that I have trouble imagining feeling obligated to connect a certain way. Some people prefer email. Some people prefer texting. Etc.