"Sailing upwind" here -- if you were genuinely tacking, rather than doing something analogous to tacking -- would (with a big enough solar sail) let you propel yourself into the sun more quickly than gravity alone would pull you in.
So yes, you can produce a sideways component and let gravity do the rest, but that component can never actually push you towards the sun.
Sure, the analogy is loose, but as a sailor myself, I think it fits the fundamental ethos- one works the natural forces against each other to get where they are trying to go- wind, tides, and both aerodynamic and hydrodynamic lift.
The fact that one has a number of force vectors in varying directions with both sailing and solar sailing, both let you work them together in unison get where you want to go.
Quite often when I am "sailing" the wind is light or dies, and I go to "windward" by timing the currents, and wind at best lets me position for the right currents, but below a sufficient level produces no lift to windward.
So yes, you can produce a sideways component and let gravity do the rest, but that component can never actually push you towards the sun.