Generally we think things are far away when it takes a longer time to get to them. We have some reasonable assurance that the speed of light is immutable and so we can measure the distance in our frame of reference by bouncing light off of Pluto. Are you nerd sniping sir?
I am making the distinction of what we perceive to be reality to actual reality.
Distance is a human concept. The moment we stop thinking distance does not exist. It may be a limitation that we perceive distance as something to be overcome through rocket ships and not through other methods.
Time is also in the same category. If you want to read a good book on the topic read “the end of Time quote by Jason Barbour.