Dataframes aren't strongly typed (while RDDs are).

It's difficult to add strong typing to an existing API.

I certainly prefer using DataFrames generally, but sometimes you need to switch to RDDs because some operations are easier.

Hopefully Datasets bridge the gap well. I haven't used them yet, but the examples look nice.