Maintenance-tasks

For purposes of data migrations or any other tasks that require long-term execution or should be run once, we use gem maintenance_task. All the main documentation for the gem you can find on their GitHub page. But we need to describe some points:

UI access

Maintenance tasks UI can be access in the /maintenance_tasks path under the admin domain (e.g. https://admin.uala.com/maintenance_tasks). It's protected with the same basic authentication credentials as Sidekiq dashboard

Task structure

All tasks are inside the folder app/tasks/maintenance in the uala-main project. Each task should be inside the Maintenance module and be a child of the MaintenanceTasks::Task class. Usually (for example for data migrations), a task has two methods inside:

• collection: return an Active Record Relation or an Array to be iterated over.
• process: do the work of your maintenance task on a single record

module Maintenance
  class DefineMostPopularVenueTreatmentGroups < MaintenanceTasks::Task
    def collection
      Venue.where.missing(:favorite_venue_treatment_groups)
    end

    def process(element)
      Venues::DefineMostPopularVenueTreatmentGroupsService.new(element.id).run
    end
  end
end

For a more "traditional" type of task you can use the #no_collection method of the Task class and write the code inside the #process method:

module Maintenance
  class TestTask < MaintenanceTasks::Task
    no_collection

    def process
      puts 'No collection process'
    end
  end
end

Considerations when writing Tasks

• Duration of Task#process: processing a single element of the collection should take less than 25 seconds, or the duration set as a timeout for Sidekiq or the queue adapter configured in your application. It allows the Task to be safely interrupted and resumed
• Idempotency of Task#process: it should be safe to run the process multiple times for the same element of the collection. Read more in this Sidekiq best practice. It’s important if the Task errors and you run it again, because the same element that errored the Task may well be processed again. It especially matters in the situation described above, when the iteration duration exceeds the timeout: if the job is re-enqueued, multiple elements may be processed again
• if you use the #collection method with ActiveRecord relation then you need to know that it's impossible to use ORDER BY or LIMIT there by the reason of cursor iteration design. It will cause an error on the gem side
• Sidekiq jobs will never fail nor you'll find any failure trace from a sidekiq perspective. All failures are rescued by the maintenance_tasks gem. Failures are just reported to Datadog via Uala::Reporter and in the task page and cannot be retried, they have to be restarted from the beginning
• You can run multiple instances of the same task (even concurrently), there's no limit on this
• Individual processing is not wrapped into a transaction, so if you write something and the job fails for whatever reason the write will persist in the database
• You won't have access to the logger out of the box, you need to use Rails.logger

Tests

Every task can be and should be tested.

RSpec.describe Maintenance::DefineMostPopularVenueTreatmentGroups do
  let!(:venue_without_favorite_venue_treatment_groups) do
    create(:venue)
  end

  describe '#collection' do
    it 'returns only venues without favorite_venue_treatment_groups' do
      # Venue with favorite_venue_treatment_groups shouldn't be processed
      venue_with_favorite_venue_treatment_groups = create(:venue)
      create(:venue_treatment_group, venue: venue_with_favorite_venue_treatment_groups, favorite: true)

      expect(described_class.collection).to eq([venue_without_favorite_venue_treatment_groups])
    end
  end

  describe '#process' do
    let(:service) { instance_double(Venues::DefineMostPopularVenueTreatmentGroupsService) }

    it 'processes a venue' do
      aggregate_failures do
        expect(Venues::DefineMostPopularVenueTreatmentGroupsService).to(
          receive(:new).with(venue_without_favorite_venue_treatment_groups.id).and_return(service)
        )
        expect(service).to receive(:run).once

        described_class.process(venue_without_favorite_venue_treatment_groups)
      end
    end
  end
end

Protected tasks

By default all tasks can be run by simply accessing tasks dashboard and clicking on the Run button. But sometimes we need to protect some tasks from being run by mistake. For this purpose we can use the Uala::ProtectedMaintenanceTask module. It will add an input for a confirmation token to run the task.

Current token is stored within our ENV variables and can be found in the ENV['PROTECTED_MAINTENANCE_TASK_TOKEN'] variable.

In order to create a task simply use that module and the token input will be added and validated before running the task. e.g.

module Maintenance
  class SuperSecureTask < MaintenanceTasks::Task
    include Uala::ProtectedMaintenanceTask

    def process
      # Do something
    end
  end
end