That hasn't worked well for too many people. Most OS businesses that haven't failed tend to dual license, and cripple the installation/documentation(technical or product) in some way for the OS version.
Case in point though: spring source had "community" builds + special builds for customers, and the community builds + documentation were crippled somewhat. I know a lot of commercial open source users (who weren't paying) were disappointed by some of their actions.
