I think it depends on person. Also Cal Newport, though advocating the concept of "stopping the work day at 5 p.m.", also strongly advocates that one should not mindlessly spend the time "after work", but should engage in other types of productive activities such as reading books. He quotes Bennett, who said that the human mind doesn't tire except for when sleeping, and all it needs is switching to a different type of intellectual activity, which I find to be very reasonable.

My personal experience has been that whenever I'm varying the type of productive activities throughout the day, I get a lot done, feel satisfied, stay sharp and productive. Whenever I do too much mindless entertainment such as playing too much of one video game or browsing reddit, I end up not feeling very well the next day whatsoever.

So in the case of Wolfram maybe he checks email or does some type of activity that is not the same as the type of activity he does while at the desk, which I think can be totally conceivable and productive.