Last active
March 31, 2017 09:40
-
-
Save termie/b9e1e9a751f25ae40fa6486f40cefe2b to your computer and use it in GitHub Desktop.
is there a job queue that lets me encode these semantics?
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Is there a job queue system (or a job queue system plus an approach to using it) that lets | |
me only process jobs with a maximum concurrency based on some property of the job? | |
The workload looks like | |
U1, J1 | |
U1, J2 | |
U2, J3 | |
U1, J4 | |
U1, J5 | |
U3, J6 | |
U3, J7 | |
U3, J8 | |
U2, J9 | |
The runners are | |
R1, R2, R3, R4 - runs jobs from all users not U3 | |
R5-R10 - runs jobs from U3 only | |
The user U1 and U2 can have two jobs running concurrently (so as not to denial of service each other), U3 can have 6. | |
We can set aside U3 for a little bit because they have some separate semantics and can be their | |
own topic/queue and are known ahead of time (U1 and U2 are "generic" jobs). | |
The goal is to have the runners pick up the following jobs (starting from the talk of the queue): | |
R1: U1, J1 | |
R2: U1, J2, | |
R3: U2, J3, | |
R4: U2, J9 | |
R5+: U3, J* | |
Leaving the following in the queue to get picked up once U1's jobs free up | |
U1, J4 | |
U1, J5 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment