Going up is also a shortcut: CMD+arrow up.

I assume they didn't expect users to use directory hierarchies much and thought everybody would dump their files into flat dirs and search them with spotlight.