Exactly. It takes just one person who knows your phone number to upload their entire contact list and it's saved. The likelihood that Facebook doesn't have your phone number and name is 0% unless you don't have a phone number.
It's worse than that. If you gave them your phone number, you can ask to see it and have them to erase it. If one of your contacts gave them your number, it's _their_ data, you cannot get Facebook to confirm they have it and you can't ask them to remove it ...
