Yes, the camera would have to physically rotate or stay level. Think of a film camera: holding the camera sideways means shooting in landscape at a 90 degree angle = instant portrait mode, no fancy software rotation effects required.

Alternatively, if we had a completely square sensor... though that still wouldn't help if you rotate at less than 90 degree angles.