"Old-school" Markovian language models (the vast majority of what's being used in production today) are mostly word-based but for text applications with tons of data, high-order character models are competitive with word-based models. (http://www.aclweb.org/anthology/W05-1107)