E.g. some other thread that isn't yours getting timesliced onto your core. Or something in the OS interrupt handling code wiping out most of your CPU cache, etc.

Bear in mind that by default most Linux distros will a timeslice of 10 milliseconds so your thread can easily skip that much time simply because it was some other thread's turn to run. JVM pauses with ZGC and even G1 most of the time (when set to the most aggressive settings) are now so low, that your pause times will be dominated by other factors.

handwaving here, but broadly correct: modern OS' will try to arrange work/app wakeups so they happen at the same time - ex. if I schedule a 15 ms timer, and you schedule a 18 ms timer, maybe we both just get woken at 18. or 20. or the phone rang, so we had to kick up power to talk to the tower, lets grab email