In the daytime, you're supposed to work, whether you're in the flow or not. At night, if you're coding, it's because you're in the flow. So the phenomenom is explained by a biased sample.
Order people to work at 2 am and see what remains of this productivity boost.
I've noticed the night effect for a long time too, and while the lack of interruptions might have something to do with it, I don't think it nearly explains my 3x boost in productivity.
I think there's something biological going on as well. Around 1 or 2 am, I sometimes get a bang of energy and creativity that I never get during the day. It's the "second wind," the injection (possibly) of cortisol and other stress hormones that cause my body to go into stress-protection mode. I become hypervigilant, which causes fewer bugs. I think more quickly, writing code faster and coming up with amazing design ideas instantly.
If you think about this from an evolutionary perspective, it makes perfect sense-- in the animal world, when you become stressed (i.e., a tiger comes up behind you, or you get exhausted looking for food) the outcome is binary: either you die or you live. So the body does extraordinary things to keep you alive, like make you temporarily smarter and more focused.
Over hundreds of thousands of years, perhaps, the body has developed in such a way that the lack of sleep for some number of hours (for me, almost exactly sixteen or seventeen hours) triggers the release of stress hormones which sacrifice the body's current state for mental and physical agility.
When will someone figure out how to hack my brain to make it like that all the time, without the nasty side effects (sleep deprivation) the next day?
Sleep is cyclic; it comes in phases of varying depth. (http://www.helpguide.org/life/sleeping.htm) I find the difference between seven and a half hours of sleep and six hours isn't bad, but the difference between six hours of sleep and four hours of sleep is much more difficult to deal with. Looking at the diagram on the linked page, it's obvious why: sleep stages three and four are important and if you sleep less than six hours they get screwed up. More REM sleep gets messed up too, which is also important.
Day-to-day energy levels are cyclic too. Every day I want to take a nap at 3:00 in the afternoon. I'm also most energized from around 10:00 to 11:00 in the morning. It's a pattern.
A boost of energy at a certain time at night happens likely for the same reason it does during the day: levels of consciousness are cyclic.
As far as nighttime being more productive: there are fewer interruptions which means you can concentrate. It's easier when your cell phone/RSS/Twitter/blog/text/email isn't blowing up.
Interesting. So maybe people who call themselves "night owls" are maybe just hacking their body's cycle to their temporary advantage... Most of my family consider themselves night owls, but maybe we've just been habituated to staying up late and abusing this.
Still, I look forward to the alone time when I code at night too. I even structure my day around it a lot of the time.
I've come up with my best ideas at 2am but I've learned from experience that's the worst time to try implementing them.
In the wee hours typos, syntax errors and simple transposition bugs seem to abound and those take an annoying amount of time to track down and fix. Best to just write the plan out in detail and leave the implementation until morning, unless you want to peck at the keyboard for much longer than you would have when awake.
I start to have problems implementing ideas at night as well, but that only starts happening around 5am. On the other hand I'm pretty useless for at least 2 hours after I wake up.
I think this is because your inner voice is doing this at late night:
"Dude, what the hell are you doing at this ungodly hour?"
"I am trying to get _____ done"
"Oh okay"
So, you are continuously conscious and aware. Naturally you are more productive. I create the same phenomenon during the day too by using a simple tool that I wrote for this exact reason: Smacklet http://www.smacklet.com
An significant factor might also be that when you are tired, you want to get the job done quickly and you spend less time thinking about the options and more time just going ahead.
There are times I need to get a lot of boring stuff done and I can just power through that in a night session. I find I can just run through mindless work at a really fast pace when I am tired and not thinking about it too much, or thinking about all the other exciting things I could be doing.
I've noticed coding after consuming (a small amount of) alcohol has the same effect for me - I tend to go off on fewer tangents and stick to what I need to do.
Funny, this has never worked or made much sense for me. I have always been able to kick butt from 6 a.m. until as late as midnight, but after that, forget it.
Then when I read the author's reasons for being so productive at night, I realized that you can do all of that during the day. I never IM, text, or RSS, only a few people have my cell phone #, I only check email (and hacker news) every hour or two, and the door is closed. Works great.
I've always felt that I was superbly productive at night, but when I measured how much I actually got done, I was working 3-4 times more slowly than I do in the morning or just before dinner. Sleep deprivation does weird things to your perception of time; it's easy to think that you've been busily coding away, only to find that you've written about 2 functions and it's 4 hours later. So without hard data, I'm a little skeptical.
I do find that having nobody around me is a huge boon to productivity, enough that it can sometimes offset the nighttime effect. It's not just the lack of interruptions: I've found that the mere presence of people makes me concentrate on the work less, whether they're talking/IMing me or not.
I was a night person until the final year of my PhD. Something about that year made me switch to being a morning person, strange as it sounds.
Now I get my best stuff done before most people wake up. I start to slow down a little after lunch. After about 7 or 8pm I find it hard to get any good work done.
I was just trying to find a quote I used to like, about it being uncivilized to expect people to work after lunch... I think it was by a famous mathematician or philosopher, but I can't find it now - does anyone know who I'm thinking of?
Coding at night is good. But I noticed another thing: texts written at night suck. In the morning you will inevitably be ashamed of what you wrote. The best time for blogging or writing emails is the beginning of the day. I wonder if it's the same for other people, and why this is.
Since I work for myself (from home), no one sees when I work. So I code at night when they're not calling and emailing. I often just stick to communication and getting menial stuff done during the day. Sometimes people start to notice though, and some people I work with don't bother calling me until the afternoon, in case I sleep in a bit late after a solid night of coding...
On the one hand it might look less professional in some peoples eyes, but in others they think it means I must be some kind of super hacker. Either way, it's amusing for me :)
Thyroid-stimulating hormone,among others, are released prior to sleep.
The thyroid controls how quickly the body burns energy, makes proteins, and how sensitive the body should be to other hormones.
So it may be that you get a burst of energy just prior to sleep(to make you feel tired enough for a good night sleep).
Working at night is great. Trouble is, you pay the price the next day when you're downing coffees just to keep from falling asleep on your keyboard.
I'm guessing at least some here have made that trip out to the parking garage around lunchtime while simultaneously setting their cellphone alarm for +20 minutes for just a quick nap. Extra points if you keep a blanket in the back seat. :)
Night-time is best for my writing but worst for my coding - go figure. If I have a big backlog of emails, or I'm trying to write a journal paper, I don't even bother trying during the day anymore, unless there's a deadline.
Order people to work at 2 am and see what remains of this productivity boost.