Some time ago I mentioned a project I was working on to move an application from Heroku to a virtual server running on Rackspace. One of the challenges I faced in that project was setting up some background jobs.

Given that we were using Ubuntu we decided to use Upstart to manage the jobs.

Here is a summary of the procedure I followed:

1. Create an specific user for the jobs to run (useradd worker)
2. Create a config file to make upstart manage each job. These files should be placed under /etc/init/. These files should include some generic information for upstart (like the user that will run the service) and also some specific information related to job (like how to start it). Here you can find and example to configure Clockwork (a cron replacement app built on Ruby).
3. Now to manage the job you can use this command
   sudo service clockwork {start|status|stop|restart}
4. The log file of the job will be placed under /var/log/upstart and the file name will be <job_name>.log

Hope you find it useful.