My finding with D3 is that it chooses to give you power at the expense of ease. So you can do almost anything, but very little is simple. For most of it, you pretty much have to make all the shapes yourself with SVG, then use D3 to tie their dimensions and ___location to the data. So you have to know SVG from the get go. That's not a problem. It's a little weird with what it considers CSS stying vs node properties, but is otherwise straightforward enough. As long as you go into learning D3 with this in mind, and don't come at it as equivalent to other user friendly graphing libraries, it's amazing.
A year or two, I would have agreed with you. However, I've been working on a new project in D3 over the past month and I've realized that it now has so many mature, built-in visualization layouts that almost anything you'd want to do can be done out of the box with a little bit of tweaking. See, for example, the layouts in https://github.com/mbostock/d3/wiki/Layouts (which include a pretty tree layout) as well as the axis/timeseries handling built into d3.scale/time/svg. Granted if you're doing a completely new visualization, you're going to have to do a lot of custom stuff, but there are a LOT of helpers built in these days.
The biggest learning hump for me was understanding the way data is bound to DOM elements, and how a mismatch between the data and the selected DOM elements is handled. I never really understood until I read Mike Bostock's post "Thinking With Joins", at which point I attained d3 enlightenment: http://bost.ocks.org/mike/join/
