That extremely depends on details of your contract, understanding with clients, level of honesty, risk appetite, and legal advice.
There are any amount of large companies where being employed by multiple entities is either frowned upon to expressly verboten. Of course, there are also any number of companies that do no care as long as you deliver, and many that actively encourage / expect nerds to nerd out in our spare time :)
This entire "You're a contractor, but you can only work for us" really makes me ask "Then why are they a contractor", cause it really sounds like they are an employee.
In this situation, they probably work for a consulting firm who is hiring them as FTE. And my experience with this is that the consulting firm itself is party to the agreement on how intellectual property rights are transferred and the employee of the firm signs an agreement that the firm can negotiate such on their behalf.
It boils down to, employee transfers IP to their consulting firm, and the firm agrees to transfer ownership of the IP for which hours were billed to produce.
IME, the firm I worked for would not bill clients for work done to our shared libraries. I'd record those hours to the firm itself, but I would still get paid the same.
