The issue is that it's easier logistically to go to the gym in the morning than going to the gym at noon.

So, it depends on how social are you and what other activities you schedule around the fasting.