For the project we are working on for Teachers Without Borders we need a reliable asynchronous way to process tasks. There are a myriad of ways to do this and if you aren’t careful you’ll spend weeks second guessing yourself while investigate every possible option. Not that I would know that from experience.
We chose Starling and Workling as our solution. Starling is a message queue that uses memcache. It was built by the guys at Twitter and released as open source. I bet it scales.
Workling is a great plugin built by a couple of guys working on boomloop.com It makes using Starling a bit easier especially when it comes to consuming messages from the queue.
There are a few blog posts around that describe how to get stuff up and running. Other than that the documentation is a bit sparse. I have to believe that will change over time. I am throwing my own links and experience into the mix so I don’t forget how to do this.
First get the gems you need to run starling:
sudo gem install starling sudo gem install memcache-client </pre>Get the [working plugin][6]git clone http://github.com/purzelrakete/workling/tree/master/Changes </pre>[6]: http://github.com/purzelrakete/workling/tree/master/Changes# Get starling and working going with: # # Fire up starling - this isn't in daemon mode. Add a -d for that. # starling -P tmp/pids/starling.pid -q tmp/starling # # Next fire up the server # script/server # # Get workling going # script/workling\_starling\_client start </pre>A few more resources http://www.williambharding.com/blog/rails/starling-update-from-the-horses-mouth/ http://davedupre.com/2008/03/25/ruby-background-tasks-with-starling/