They don't have to be the same people if they were the people who showed up in the primary to vote for someone like Ron Paul and then stayed home on election day when the Republicans nominated an establishment candidate instead.

But also, you might be surprised. Elections aren't decided by the bulk of partisan voters who show up to always vote for the same party and then cancel each other out. They're decided by the much smaller number of people willing to vote for a candidate instead of a party, and therefore move votes from one column to the other.