The calculation engine
Calculation jobs and status
Calculating thousands of records takes time, so it runs as a background job. A record that fails is kept and flagged with its reason; it is never dropped from the total.
On this page you'll learn
- What happens when you press Calculate
- The three separate statuses you'll see
- Why failures are rows, not omissions
- Runs on
- A background worker (Celery)
- Progress
- Polled every 2 seconds while running
- Failure
- A row with
errorset, shown in the table
The job flow
- 1
Start
You press Calculate. Slate creates a calculation job and returns at once with "Calculation started".
- 2
Work through the records
The worker calculates each matching record in turn and updates the job's processed count. The button's percentage reads that count.
- 3
Write a row for every record
A success writes the figures; a failure writes the reason. Both are rows.
- 4
Finish
The job is marked completed (or failed, if the batch itself broke), and the final success or error message appears.
Three statuses, three questions
| Status | Values | Answers |
|---|---|---|
| Upload status | pending → processing → processed / failed | Was the file read? |
| Calculation job status | processing → completed / failed | Did the batch run? |
| Row error | empty, or the specific reason | Did this record calculate? |
Errors surface, they don't hide
A record that can't be calculated is written as a row whose error says why, in plain words, such as "No UK_DEFRA mapping for fuel X" or "Facility has no province recorded". It shows in the Calculated Emissions table with a red status, so a total is never quietly short.
Some inputs can be missing without failing the record. Where a method can't quantify one gas (manure N₂O without the farm's manure management split, say), the row records that gas as unquantified with the reason, and says whether you can fix it by supplying data.
Tip
After adding a new background task or migration on a local install, restart the worker. A stale worker leaves jobs at "pending" forever without an error.