I used RabbitMQ together with python and celery quite extensively and it scales really well. One thing we had trouble with though was to find a nice mechanism to scheduled tasks. Eg. “Run this task 12 hours before departure”. Maybe AMQP is the wrong place to solve that problem.
I've been using something like this for exponential backoffs, but I think it'd work for this case as well.
Let's say you've got one exchange and one main queue for processing: jobs.exchange and jobs.queue respectively.
If you need to schedule something for later, you'd assert a new queue with a TTL for the target amount of time (scheduled-jobs-<time>.queue). Also set an expiry of some amount of time, so it'd get cleaned up if nothing had been scheduled for that particular time in a while. Finally, have its dead-letter-exchange set to jobs.exchange.
This could lead to a bunch of temporary queues, but the expiration should clean them up when they haven't been used for a bit.
You're usually stuck polling for stuff like that. I'm a big fan of using things like Advanced Python Scheduler for those sorts of tasks: https://apscheduler.readthedocs.io/en/stable/
Yeh I would argue don't use a message queue for this, they're really best processing many messages quickly, there are plenty of scheduling libraries that have various persistence layers to handle this depending on your ecosystem.
