That's why PulseAudio and PipeWire run as real-time. Or they should.
Back when Pulse was new and I was running Gentoo I used to help other Gentoo users get their real-time settings correct. I believe we used rtprio in limits.conf. I don't recall when RTKit became a thing.
If your sound daemon is running as real-time and still missing deadlines then there's something wrong with your system hardware. Or I suppose, the sound source feeding the pulseaudio daemon is not getting enough CPU time.
I think they indeed should run at a (low) real-time priority, but only if they are limited to a fraction of total available CPU power, with say cgroups or similar.
Otherwise they can easily lock up the system, and that should not be the default configuration.
