I want to distribute tasks to worker nodes using a queue. The work depends on external resources.

The access to those needs to be rate limited across multiple dimensions (i.e. customer id, city, phone provider, there may be only 1 per customer id / second, 1000/city second and 20 / phone provider/second). The rate limit needs to apply to all worker nodes. My worker nodes share the same network.

How can I achieve that while maintaining a reasonable throughput? Any experience?

My naive attempt would be to use one leaky bucket per dimension implemented in Redis and check those prior accessing the resource. If there's no token in all the necessary buckets available, I would put the request back at the end of the queue.

Are there better ways to implement this?