It's context dependent. The situation is different if it's, say, a colleague instead of your spouse. I think people are reading way too much into that part of the comment.