x
11224742 10207744386832061 1134591581572583218 n

Waiting for Background Jobs

over 3 years ago by Alex

Popup

There are many situations in web applications where users need to wait for a time-consuming process to finalize. There are countless examples across various domains, but the more common ones are preparing a PDF, importing data from other services or time-consuming calculations. Our high-level strategy for handling these kinds of processes consists of a few general guidelines:

Perform time-consuming processes in the background, using a background job system such as Resque or Delayed Job
Optimize the background job code as much as possible, using libraries such as Upsert, ActiveRecord Import
Maintain user comfort by providing constant feedback

Let’s walk through a more detailed example. At Handicap Server, changing league settings causes us to recalculate handicaps for the entire player roster. Here is how the user experience looks.

First, the user changes the handicap method or settings.

As a result of changing the handicap method, we start a background job that recalculates all handicaps. The user is taken to a waiting page, which we also use to maintain user comfort by pushing constant updates.

Once the process is done, we signal this to the user and they can continue their workflow.

How does this all work?

First, we create a job model and database table that stores information about each background process. This model references an owner (e.g., “BackgroundJobs::LeagueRecomputeHandicaps”), a method name and method arguments. When the job method is run, we are going to call:

def run_code
    method_args[:job_id] = self.id

    who = self.owner.constantize
    who.send(self.method_name, method_args)
end

Additionally, we save a title, an array of notifications (messages broadcasted from inside the background job to the user), a success and error url (where to redirect next) and how to redirect (automatically or following an intermediary user click).

We structure all background jobs inside a module. Each background job can be structured as necessary, but we usually have a prepare function and a perform function.

module BackgroundJobs
class LeagueRecomputeHandicaps
  # Recalculate handicaps for a league
  # args = {:job_id, :league_id}
  def self.perform(args)
    job = Job.find_by_id(args[:job_id])
    league = League.find_by_id(args[:league_id])
    
    league.players.order(:name).each do |p|
      p.compute_handicap(10.years.ago, job)
    end
    
    job.change_status("succes")  
  end
  
  # Create the job record
  def self.prepare_job(league)
    Job.create(
      owner:       self.to_s,
      method_name: "perform",
      title:       "Recomputing Handicaps",
      method_args: { league_id: league.id } ,
      succes_url:  Rails.application.routes.url_helpers.league_path(league.id),
      error_url:   Rails.application.routes.url_helpers.league_path(league.id),
      succes_type: "user_click",
      error_type:  "user_click"
    )
  end
end
end

When the league settings are updated, we simply call BackgroundJobs::LeagueRecomputeHandicaps and receive back a job object. We immediately redirect to the job show page and start polling for updates. The JavaScript polling code, poll response and calculations are very simple.

jobs.js.coffee

window.hs = window.hs or {}
((jobs, $) ->

  update_interval = 0

  jobs.stop_checking = ->
    clearInterval(update_interval);

  jobs.go_here = (url) ->
    window.location = url;
    jobs.stop_checking();

  jobs.update = (url) ->
    update_interval = setInterval(->
 $.ajax({
        type:'GET',
        url: url+'.js'})
    ,2500)

) window.hs.jobs = window.hs.jobs or {}, jQuery

JobsController#poll

def poll
  if @job.status == "prepared"
    @job.update(status: "scheduled")
    @job.delay.run
  end

  respond_to do |format|
    format.html { redirect_to job_path(@job.id) }
    format.js { }
  end
end

poll.js.haml

$("#notifications").append("#{j(render partial: 'jobs/status', locals: { job: @job } )}");
- if @job.status == "completed"
  - if @job.error_encountered == false
    - if @job.succes_type == "auto_redirect"
      window.hs.jobs.go_here("#{@job.succes_url}");
    - else
      window.hs.jobs.stop_checking();
      $("#current").hide();
      $("#my_spinner").append("#{j(render partial: 'jobs/spinner', locals: { status: "succes" } )}");
      $("#my_panel").append("#{j(render partial: 'jobs/finish', locals: { url: @job.succes_url, status: "Continue", given_class: "btn btn-success btn-block" } )}");
  - else
    - if @job.succes_type == "auto_redirect"
      window.hs.jobs.go_here("#{@job.error_url}")
    - else
      window.hs.jobs.stop_checking();
      $("#current").hide();
      $("#my_spinner").append("#{j(render partial: 'jobs/spinner', locals: { status: "error" } )}");
      $("#my_panel").append("#{j(render partial: 'jobs/finish', locals: { url: @job.error_url, status: "Back", given_class: "btn btn-danger btn-block" } )}");

While there are some other subtleties, this blog post describes the main steps that we take in order to enhance the user experience when waiting for background jobs. We would be extremely happy to hear about other techniques that are used successfully to solve this problem.

Did you like this post? Share it with your friends!