I had the same problem, and it was annoying as heck. Try running a multi-process build (Nightly/Aurora/Beta) and see if your experience gets any better.
For some reason the freezes are much worse in multi-process. Certain sites freeze it up consistently; newegg.com for example I just measured 67 seconds the whole window was completely unresponsive. If I disable e10s it runs fine.
