Can you explain any of the technical details around this perchance? I'm super curious. I know that SO_REUSEPORT[1] exists but is that the only little trick to make this work? From what I've read with SO_REUSEPORT it can open up that port to hijacking by rogue processes, so is that fine to rely on?
You don't even need that. If the old server process exec()s the new one, it can pass on its file descriptors -- including the listening socket -- when that happens.
If an attacker is already running rogue processes on your box, the minor details surrounding SO_REUSEPORT is the least of your worries. An attacker could just restart nginx, and won't care about lost requests.
>it can open up that port to hijacking by rogue processes
That seems relevant if the process is using a non-privileged port that's >= 1024. If we're talking about privileged ports (<= 1023), though, only another root process could hijack that, and those can already hijack you many other ways.
poked around a bit of that from a previous job, here's what I remember:
1. there's a control process and worker processes
2. on upgrade, control process launches new worker processes from the new binary
3. requests are drained from old worker processes
4. most of the time nginx request handlers allocate from a per-request allocation pool, so requests mostly don't share memory
5. for the cases where there are global states, there's a separate shared memory pool that you need to allocate from (which is kind of hard to work if you are not using built-in nginx primitives)
[1] https://lwn.net/Articles/542629/