List:General Discussion« Previous MessageNext Message »
From:Gavin Towey Date:September 8 2009 5:40pm
Subject:RE: Queue / FIFO in MySQL?
View as plain text  
You can add a LIMIT n to your update clause.

Gavin Towey

-----Original Message-----
From: Allen Fowler [mailto:allen.fowler@stripped]
Sent: Monday, September 07, 2009 5:18 PM
To: mysql@stripped
Subject: Queue / FIFO in MySQL?


I need to create a system where records are generated by a "producer" process and
processed by several "worker" processes.

I was thinking about something like:

1) Producer INSERTs new records with "state" = "new" & "worker" = "null"
2) Producer sleeps and loops back to step #1

1) Worker UPDATEs all records with "worker" = "pid" & "state" = "working" where
"state" == "new"
2) Worker SELECTs all records where "worker" = "pid" & "state" = "working"
3) For each record that is done, worker updates record with "state" = "done"
4) Worker loops back to step #1

Note: In this scheme the worker winds up with all "new" records generated since the last
worker claimed any. Not sure how else to guarantee atomicity. I would prefer "only n
records per request". Ideas?

I am sure something like this must have been before....  Can anyone point me to example
code, libraries, and/or refinements on the scheme?  (preferably using python...)

Thank you,

MySQL General Mailing List
For list archives:
To unsubscribe:

The information contained in this transmission may contain privileged and confidential
information. It is intended only for the use of the person(s) named above. If you are not
the intended recipient, you are hereby notified that any review, dissemination,
distribution or duplication of this communication is strictly prohibited. If you are not
the intended recipient, please contact the sender by reply email and destroy all copies of
the original message.
Queue / FIFO in MySQL?Allen Fowler8 Sep
  • Re: Queue / FIFO in MySQL?Hank8 Sep
  • Re: Queue / FIFO in MySQL?Perrin Harkins8 Sep
    • Re: Queue / FIFO in MySQL?Alex Arul Lurthu8 Sep
  • RE: Queue / FIFO in MySQL?Gavin Towey8 Sep
    • RE: Queue / FIFO in MySQL?Jerry Schwartz8 Sep