18 · The Complete Document-Review Workflow
We have now examined every major piece separately:
- Jinja-rendered pages
- Browser events and
fetch() - Pydantic request validation
- Authentication, authorization and CSRF
- SQLAlchemy sessions and SQLite transactions
- Background work
- Polling
- Server-Sent Events
- WebSockets
- Browser and Python event loops
Today we will assemble those pieces into one document-review workflow using production-oriented boundaries.
These are focused excerpts from the same application. They assume the existing router, Jinja setup, session middleware, DbSession, dictionary-backed CurrentUserDep, and authorized-document lookup. The ReviewOperator dependency from lesson 14 still requires a reviewer or admin role before a review can start.
The document-analysis functions and full migration files remain application-specific. We will show how their inputs and results connect without turning the lesson into a complete application. You can follow this workflow without implementing the optional WebSocket room from lesson 16.
The architecture will deliberately use different mechanisms for different jobs:
| Operation | Mechanism |
|---|---|
| Display the initial page | Jinja HTML response |
| Start a review | POST using fetch() |
| Persist the requested work | SQLAlchemy transaction |
| Perform the review | Separate worker reading durable job records |
| Inform the browser | Server-Sent Events |
| Update the visible page | Safe DOM changes |
We will not use a WebSocket.
The browser sends one command and then receives progress. That is not frequent bidirectional interaction. Ordinary HTTP plus SSE is simpler.
Where today’s code sits
Section titled “Where today’s code sits”The complete successful journey is:
Browser requests document page ↓FastAPI authenticates and authorizes ↓SQLAlchemy reads SQLite ↓Jinja renders HTML ↓Browser constructs DOM ↓User clicks Start review ↓JavaScript sends CSRF-protected POST ↓Pydantic validates request ↓Service commits durable job row ↓FastAPI returns 202 Accepted ↓Separate worker claims the job ↓Worker performs review and stores progress ↓SSE endpoint reads current state ↓Browser receives progress events ↓JavaScript updates visible statusThere are four independent lifecycles:
Page request→ begins and ends quickly
Start-review request→ begins and ends quickly
Review job→ continues independently
SSE connection→ remains open while reporting progressThe database connects them.
The database model
Section titled “The database model”Earlier examples used Document.status. Here, a reviewed Alembic migration renames that field to Document.review_status so it is clearly distinguished from ReviewJob.status.
The document field now looks like this:
# In app/models/documents.py
review_status: Mapped[str] = mapped_column( String(30), default="not_started",)This is a field excerpt inside the existing Document class. Keep its other columns. Update callers alongside the rename, and migrate existing values deliberately: the earlier waiting state pending becomes queued when represented by a durable job. A document with no requested review begins as not_started; it is not already queued.
Do not simply reset every old row. Existing active work needs an explicit migration or reconciliation decision before adopting the new workflow. We are describing the resulting schema, not supplying that data migration here.
The migration also adds a durable review_jobs table. Its foreign keys assume the existing documents and users tables are registered in the model metadata; their full definitions are outside this excerpt:
SQLite also requires PRAGMA foreign_keys=ON on each database connection to enforce those foreign keys. Declaring ForeignKey(...) in a model alone does not enable enforcement; the engine setup must enable it when connections are opened. SQLAlchemy SQLite foreign-key support
from datetime import datetime, timezone
from sqlalchemy import ForeignKey, String, Textfrom sqlalchemy.orm import Mapped, mapped_column
from ..database import Base
class ReviewJob(Base): __tablename__ = "review_jobs"
id: Mapped[int] = mapped_column( primary_key=True, )
document_id: Mapped[int] = mapped_column( ForeignKey("documents.id"), index=True, )
requested_by_user_id: Mapped[int] = mapped_column( ForeignKey("users.id"), )
depth: Mapped[str] = mapped_column( String(20), )
status: Mapped[str] = mapped_column( String(30), index=True, )
progress: Mapped[int] = mapped_column( default=0, )
result_summary: Mapped[str | None] = mapped_column( Text, nullable=True, )
error_message: Mapped[str | None] = mapped_column( Text, nullable=True, )
created_at: Mapped[datetime] = mapped_column( default=lambda: datetime.now(timezone.utc), )
started_at: Mapped[datetime | None] = mapped_column( nullable=True, )
finished_at: Mapped[datetime | None] = mapped_column( nullable=True, )The job can move through:
queued→ processing→ completedor:
queued→ processing→ failedThe row is not merely a progress display. It records work for the separate worker to attempt. Its depth column preserves the user’s choice after the HTTP request has ended.
Because the row lives in SQLite, it remains after:
- The HTTP request ends
- The browser closes
- FastAPI restarts
- The SSE connection disconnects
Alembic created the table during deployment. Alembic does not participate in any of today’s browser requests.
Storing the job survives a web-process restart. Recovering a job after its worker crashes is an additional responsibility; the recovery section below explains what this simplified worker still omits.
The Pydantic contracts
Section titled “The Pydantic contracts”The browser may choose a review depth:
{ "depth": "standard"}Pydantic defines the accepted request:
from typing import Literal
from pydantic import BaseModel, ConfigDict
ReviewDepth = Literal[ "quick", "standard", "thorough",]
class ReviewStart(BaseModel): model_config = ConfigDict( extra="forbid", )
depth: ReviewDepth = "standard"
class ReviewAccepted(BaseModel): job_id: int document_id: int status: Literal["queued"] events_url: str
class ReviewProgress(BaseModel): job_id: int document_id: int status: Literal[ "queued", "processing", "completed", "failed", ] progress: int result_summary: str | None = None error_message: str | None = NoneThese models describe three different boundaries:
ReviewStart→ What the browser may request
ReviewAccepted→ What the POST route returns immediately
ReviewProgress→ What the browser may later receiveThe POST response does not contain a completed review.
It says only:
The application accepted this operation and durably queued it.
The Jinja page
Section titled “The Jinja page”The initial page is still server rendered:
<section id="review-panel" data-start-url="{{ url_for( 'start_document_review', document_id=review_document.id ) }}" data-events-url="{{ review_events_url }}" data-csrf-token="{{ csrf_token }}" data-can-review="{{ 'true' if can_review else 'false' }}"> <h1>{{ review_document.title }}</h1>
<p> Review status: <strong id="review-status"> {{ review_document.review_status }} </strong> </p>
<p> Progress: <span id="review-progress"> {{ review_progress }}% </span> </p>
<form id="review-form"> <label for="review-depth"> Review depth </label>
<select id="review-depth" name="depth" {% if not can_review or review_document.review_status in ['queued', 'processing'] %}disabled{% endif %} > <option value="quick">Quick</option> <option value="standard" selected> Standard </option> <option value="thorough"> Thorough </option> </select>
<button id="start-review" type="submit" {% if not can_review or review_document.review_status in ['queued', 'processing'] %}disabled{% endif %} > Start review </button> </form>
<p id="review-message" role="status" aria-live="polite"> {% if latest_job and latest_job.status == 'failed' %} {{ latest_job.error_message }} {% endif %} </p>
<section id="review-result" {% if not latest_job or latest_job.status != 'completed' %}hidden{% endif %} > <h2>Review result</h2> <p id="review-summary">{{ latest_job.result_summary if latest_job and latest_job.status == 'completed' else '' }}</p> </section></section>
<script src="{{ url_for('static', path='/document-review.js') }}" defer></script>When the browser first receives this HTML, it already contains the latest known database status.
If JavaScript fails to load, the page is still meaningful:
Document titleReview statusProgressReloading the page asks the server and database for current state again.
That gives the application a reliable recovery path independent of its live JavaScript connection.
Rendering the page
Section titled “Rendering the page”The page route resembles:
@router.get( "/{document_id}", response_class=HTMLResponse,)def document_page( request: Request, document: AuthorizedDocument, current_user: CurrentUserDep, db: DbSession,): latest_job = db.scalar( select(ReviewJob) .where( ReviewJob.document_id == document.id, ) .order_by( ReviewJob.id.desc(), ) .limit(1) )
events_url = ""
if latest_job is not None: events_url = str( request.url_for( "document_review_events", document_id=document.id, ).include_query_params(job_id=latest_job.id) )
return templates.TemplateResponse( request=request, name="documents/detail.html", context={ "review_document": document, "latest_job": latest_job, "review_events_url": events_url, "can_review": current_user["role"] in { "reviewer", "admin", }, "review_progress": ( latest_job.progress if latest_job is not None else 0 ), "csrf_token": get_csrf_token(request), }, headers={ "Cache-Control": "no-store", }, )This is a normal synchronous route because it uses synchronous SQLAlchemy.
FastAPI runs the normal def route in its thread pool.
The route receives:
request→ HTTP and session information
document→ authenticated and authorized ORM object
db→ request-scoped SQLAlchemy SessionIt queries for the latest review job, passes values into Jinja and returns an HTML response.
After the response finishes, the database dependency closes the session.
No database session survives inside the browser.
The events URL includes the particular job ID. A later review of the same document must not silently replace the job this page is observing. Jinja also renders an existing completed result or failure message, so reloading preserves useful information even without a live connection.
Starting the review in JavaScript
Section titled “Starting the review in JavaScript”The browser script begins with DOM references:
const panel = document.querySelector("#review-panel");const form = document.querySelector("#review-form");const depthElement = document.querySelector("#review-depth");const startButton = document.querySelector("#start-review");const statusElement = document.querySelector("#review-status");const progressElement = document.querySelector("#review-progress");const messageElement = document.querySelector("#review-message");const resultSection = document.querySelector("#review-result");const summaryElement = document.querySelector("#review-summary");
let reviewEvents = null;
const ACTIVE_STATUSES = new Set([ "queued", "processing",]);
const canReview = panel.dataset.canReview === "true";
form.addEventListener("submit", startReview);
window.addEventListener("pagehide", () => { reviewEvents?.close(); reviewEvents = null;});
window.addEventListener("pageshow", resumeReviewDisplay);
resumeReviewDisplay();
function resumeReviewDisplay() { const isActive = ACTIVE_STATUSES.has( statusElement.textContent.trim(), );
setReviewControls(isActive);
if ( isActive && panel.dataset.eventsUrl && reviewEvents === null ) { openReviewEvents(panel.dataset.eventsUrl); }}
function setReviewControls(isActive) { startButton.disabled = isActive || !canReview; depthElement.disabled = isActive || !canReview;}The initial status came from Jinja. If a review is already queued or processing, the script immediately resumes observation instead of waiting for another submission. pageshow also handles a page restored from browser history after pagehide closed its old connection.
The server-rendered permission flag controls presentation. The POST route still enforces the real permission independently.
Submitting the form calls:
async function startReview(event) { event.preventDefault();
setReviewControls(true); resultSection.hidden = true; summaryElement.textContent = ""; messageElement.textContent = "Submitting review request…";
let outcomeUnknown = true;
try { const response = await fetch( panel.dataset.startUrl, { method: "POST", headers: { "Content-Type": "application/json", "Accept": "application/json", "X-CSRF-Token": panel.dataset.csrfToken, }, body: JSON.stringify({ depth: depthElement.value, }), }, );
if ( response.status >= 400 && response.status < 500 && response.status !== 409 ) { outcomeUnknown = false; }
if (response.status === 401) { throw new Error( "Your session expired. Please sign in again." ); }
if (response.status === 403) { throw new Error( "You are not permitted to start this review." ); }
if (response.status === 409) { throw new Error( "A review is already in progress. Reload the page to observe it." ); }
if (!response.ok) { throw new Error( "The server could not confirm the review request." ); }
const data = await response.json();
outcomeUnknown = false; panel.dataset.eventsUrl = data.events_url; statusElement.textContent = data.status; progressElement.textContent = "0%"; messageElement.textContent = "Review queued. Waiting for progress…";
openReviewEvents(data.events_url); } catch (error) { messageElement.textContent = outcomeUnknown ? "The outcome is uncertain. Reload this page before trying again." : error.message;
setReviewControls(outcomeUnknown); }}The JavaScript object:
{ depth: depthElement.value}exists only in the browser.
JSON.stringify() turns it into request-body text:
{"depth":"standard"}The browser also sends:
Cookie: app_session=...X-CSRF-Token: ...The cookie authenticates the user.
The CSRF token helps prove that this state-changing request came through a page produced by our application rather than a malicious cross-site form.
JavaScript is not trusted merely because it came from our static file. The server still verifies everything.
The outcomeUnknown flag separates a definite rejection from a missing confirmation. A network failure, a server error, or a success response whose body could not be read may all leave an accepted job behind. In that case, the controls stay disabled until the user reloads authoritative state. The next lesson adds a way to retry the same operation safely.
The start-review route
Section titled “The start-review route”The FastAPI route receives the POST:
@router.post( "/{document_id}/reviews", response_model=ReviewAccepted, status_code=status.HTTP_202_ACCEPTED, name="start_document_review",)def start_document_review( request: Request, payload: ReviewStart, document: AuthorizedDocument, operator: ReviewOperator, db: DbSession,): verify_csrf_token(request)
try: job = queue_document_review( db=db, document_id=document.id, firm_id=document.firm_id, requested_by_user_id=operator["id"], depth=payload.depth, ) except ReviewAlreadyRunningError as exc: raise HTTPException( status_code=status.HTTP_409_CONFLICT, detail=str(exc), ) from exc
return ReviewAccepted( job_id=job.id, document_id=document.id, status="queued", events_url=str( request.url_for( "document_review_events", document_id=document.id, ).include_query_params(job_id=job.id) ), )Before the route executes, FastAPI has already:
- Passed the request through session middleware and matched the URL.
- Resolved authentication and parsed the document ID for the authorization lookup.
- Created a SQLAlchemy session and loaded an authorized document.
- Required a reviewer or administrator through
ReviewOperator. - Parsed and validated the request body as
ReviewStart.
This list groups the preparations by responsibility. The precise interleaving follows the dependency graph; all of them must succeed before the service runs.
The route then verifies CSRF because this is a state-changing operation.
Authorization and CSRF answer different questions:
Authorization→ Is this authenticated user allowed to start this review?
CSRF protection→ Did the write originate through an approved application interaction?Both matter.
There are two authorization questions here. AuthorizedDocument permits access to this document. ReviewOperator permits the review action. Retaining both prevents a user who may only view a document from starting work on it.
The service receives document.firm_id, the firm belonging to the already authorized document. That also preserves the administrator policy: an administrator viewing another firm’s document does not accidentally try to update it under the administrator’s own firm ID.
get_csrf_token() and verify_csrf_token() are the small session-token helpers from the earlier CSRF pattern: create an unpredictable session value when needed, then compare the submitted X-CSRF-Token with that value using secrets.compare_digest(). They are assumed helpers in these route excerpts.
Durably queuing the work
Section titled “Durably queuing the work”The service writes the request into SQLite:
from sqlalchemy import updatefrom sqlalchemy.orm import Session
from ..models.documents import Documentfrom ..models.review_jobs import ReviewJob
class ReviewAlreadyRunningError(Exception): pass
STARTABLE_STATUSES = { "not_started", "completed", "failed",}
def queue_document_review( *, db: Session, document_id: int, firm_id: int, requested_by_user_id: int, depth: str,) -> ReviewJob: transition = db.execute( update(Document) .where( Document.id == document_id, Document.firm_id == firm_id, Document.review_status.in_( STARTABLE_STATUSES ), ) .values( review_status="queued", ) )
if transition.rowcount != 1: db.rollback()
raise ReviewAlreadyRunningError( "A review is already active." )
job = ReviewJob( document_id=document_id, requested_by_user_id=requested_by_user_id, depth=depth, status="queued", progress=0, )
db.add(job) db.commit() db.refresh(job)
return jobThe conditional update is important:
Document.review_status.in_( STARTABLE_STATUSES)SQLite changes the document to queued only if its current state permits a new review.
If two nearly simultaneous requests try to start the same document review, only one should successfully make that state transition.
The service then inserts:
review_jobs rowstatus: queuedprogress: 0depth: standardBoth changes commit before the route returns.
The job is now durable.
The document transition and job insert belong to the same database transaction. If an insert or commit fails, that unit of work must be rolled back; closing our request session also rolls back outstanding failed work. A production caller should log the original failure rather than translate an arbitrary database error into “a review is already active.”
Why the response is 202 Accepted
Section titled “Why the response is 202 Accepted”FastAPI returns:
HTTP/1.1 202 AcceptedContent-Type: application/json
{ "job_id": 410, "document_id": 101, "status": "queued", "events_url": "https://app.example.com/documents/101/review-events?job_id=410"}202 Accepted does not mean:
The document review succeeded.It means:
The server accepted responsibility for processing the request.At this moment:
Browser request→ finished
Review job→ has not necessarily begunThis is why returning:
200 OKwith:
{"status":"completed"}would be misleading.
The browser displays:
Review queued. Waiting for progress…rather than:
Review completed.The durable worker
Section titled “The durable worker”A separate worker process looks for queued jobs.
Here, “durable” describes the stored job record. The simplified loop performs one job at a time and still needs the recovery machinery discussed below to recover an interrupted attempt.
For this small SQLite application, the worker can use the same application code and database but run as a separate command:
python -m app.workers.review_workerA simplified loop looks like:
import time
from .review_tasks import ( claim_next_review_job, process_review_job,)
def run_worker() -> None: while True: job_id = claim_next_review_job()
if job_id is None: time.sleep(1) continue
process_review_job(job_id)
if __name__ == "__main__": run_worker()This is ordinary synchronous Python.
It does not need asyncio because this one worker deliberately performs one queued job at a time.
The operating system or process manager is responsible for keeping it running.
Claiming the next job
Section titled “Claiming the next job”The worker first claims a queued job:
import loggingfrom datetime import datetime, timezone
from sqlalchemy import select
from ..database import SessionLocalfrom ..models.documents import Documentfrom ..models.review_jobs import ReviewJob
logger = logging.getLogger(__name__)
def claim_next_review_job() -> int | None: with SessionLocal.begin() as db: job = db.scalar( select(ReviewJob) .where( ReviewJob.status == "queued", ) .order_by( ReviewJob.id, ) .limit(1) )
if job is None: return None
job.status = "processing" job.started_at = datetime.now(timezone.utc)
document = db.get( Document, job.document_id, )
if document is None: raise LookupError("Document no longer exists")
document.review_status = "processing"
job_id = job.id
return job_idThe session and transaction exist only while claiming the work.
When the with block exits normally, SQLAlchemy commits:
ReviewJob.statusqueued → processing
Document.review_statusqueued → processingThen the session closes.
For one SQLite worker, this simple pattern is understandable.
With several workers, job claiming requires stronger coordination so two workers cannot claim the same row. That is one reason production job systems use PostgreSQL locking, Redis-backed queues or dedicated brokers.
Processing without holding a database transaction open
Section titled “Processing without holding a database transaction open”The worker should not keep a SQLAlchemy session open throughout a slow external operation.
A bad structure would be:
with SessionLocal.begin() as db: job = db.get(ReviewJob, job_id)
result = call_external_review_service( document_text, )
job.result_summary = resultIf the external service takes 45 seconds, the database transaction may remain open for the same period.
Instead, the worker separates database operations from slow processing:
def load_review_input(job_id: int) -> dict: with SessionLocal() as db: job = db.get(ReviewJob, job_id)
if job is None: raise LookupError("Review job no longer exists")
return { "document_id": job.document_id, "depth": job.depth, }
def process_review_job(job_id: int) -> None: try: review_input = load_review_input(job_id)
save_review_progress( job_id=job_id, progress=20, )
findings = analyze_document( document_id=review_input["document_id"], depth=review_input["depth"], )
save_review_progress( job_id=job_id, progress=80, )
summary = build_review_summary( findings, )
complete_review_job( job_id=job_id, summary=summary, )
except Exception: logger.exception( "Review job failed", extra={"job_id": job_id}, )
fail_review_job( job_id=job_id, message="The review could not be completed.", )load_review_input() copies the saved document ID and depth into an ordinary dictionary before its session closes. analyze_document() and build_review_summary() stand for the application’s review algorithm; their implementations are omitted. The analysis function receives the persisted depth and owns any short reads needed to obtain document content.
Each persistence helper opens a new short session:
def save_review_progress( *, job_id: int, progress: int,) -> None: with SessionLocal.begin() as db: job = db.get(ReviewJob, job_id)
if job is None: return
job.progress = progressThe architecture becomes:
Open session→ read required data→ close session
Perform slow work→ no database transaction held open
Open session→ store progress→ close session
Perform more work
Open session→ store final result→ close sessionThe worker passes IDs and plain data between stages rather than keeping ORM objects alive indefinitely.
Completing the job
Section titled “Completing the job”When processing succeeds:
def complete_review_job( *, job_id: int, summary: str,) -> None: with SessionLocal.begin() as db: job = db.get(ReviewJob, job_id)
if job is None: return
job.status = "completed" job.progress = 100 job.result_summary = summary job.finished_at = datetime.now(timezone.utc)
document = db.get( Document, job.document_id, )
if document is None: raise LookupError("Document no longer exists")
document.review_status = "completed"If processing fails:
def fail_review_job( *, job_id: int, message: str,) -> None: with SessionLocal.begin() as db: job = db.get(ReviewJob, job_id)
if job is None: return
job.status = "failed" job.error_message = message job.finished_at = datetime.now(timezone.utc)
document = db.get( Document, job.document_id, )
if document is None: raise LookupError("Document no longer exists")
document.review_status = "failed"The database records the outcome regardless of whether a browser is still listening.
Opening the SSE connection
Section titled “Opening the SSE connection”After the POST returns 202, JavaScript calls:
openReviewEvents(data.events_url);The function contains:
function openReviewEvents(eventsUrl) { reviewEvents?.close();
const eventSource = new EventSource(eventsUrl);
reviewEvents = eventSource;
eventSource.addEventListener("open", () => { if (reviewEvents !== eventSource) { return; }
messageElement.textContent = "Listening for review progress…"; });
eventSource.addEventListener( "review_progress", (event) => { if (reviewEvents !== eventSource) { return; }
const data = JSON.parse(event.data);
updateReviewDisplay(data); }, );
eventSource.addEventListener("review_unavailable", () => { if (reviewEvents !== eventSource) { return; }
eventSource.close(); reviewEvents = null; setReviewControls(true); messageElement.textContent = "This review is no longer available. Reload the page."; });
eventSource.addEventListener("error", () => { if (reviewEvents !== eventSource) { return; }
if (eventSource.readyState === EventSource.CLOSED) { reviewEvents = null; messageElement.textContent = "Progress connection closed. Reload the page to check access."; } else { messageElement.textContent = "Progress connection interrupted. Reconnecting…"; } });}EventSource sends an authenticated GET using applicable same-origin cookies.
It does not need the CSRF header because it observes state rather than changing it.
The separation is:
POST→ authenticated write→ CSRF required
SSE GET→ authenticated read→ authorization required→ no mutationThe SSE endpoint
Section titled “The SSE endpoint”We use the native FastAPI SSE interface introduced in lesson 15. It requires FastAPI 0.135.0 or newer: the path operation yields ServerSentEvent objects and declares response_class=EventSourceResponse. Returning a raw dictionary stream inside a response object would bypass that route’s SSE encoding. FastAPI SSE documentation
The stream needs current authorization, but it must not keep the ordinary AuthorizedDocument dependency’s database session open. Follow the short-session pattern from lesson 15 and return an immutable access description:
# app/dependencies.py, alongside CurrentUserDep
from dataclasses import dataclassfrom typing import Annotated
from fastapi import Depends, HTTPExceptionfrom sqlalchemy import select
from .database import SessionLocalfrom .models.documents import Documentfrom .models.review_jobs import ReviewJobfrom .schemas.reviews import ReviewProgress
@dataclass(frozen=True)class ReviewProgressAccess: document_id: int job_id: int firm_id: int | None is_admin: bool
def load_review_progress( access: ReviewProgressAccess,) -> ReviewProgress | None: with SessionLocal() as db: statement = ( select(ReviewJob) .join(Document, Document.id == ReviewJob.document_id) .where( ReviewJob.id == access.job_id, Document.id == access.document_id, ) )
if not access.is_admin: if access.firm_id is None: return None
statement = statement.where( Document.firm_id == access.firm_id, )
job = db.scalar(statement)
if job is None: return None
return ReviewProgress( job_id=job.id, document_id=job.document_id, status=job.status, progress=job.progress, result_summary=job.result_summary, error_message=job.error_message, )
def get_review_progress_access( document_id: int, job_id: int, current_user: CurrentUserDep,) -> ReviewProgressAccess: access = ReviewProgressAccess( document_id=document_id, job_id=job_id, firm_id=current_user["firm_id"], is_admin=current_user["role"] == "admin", )
if load_review_progress(access) is None: raise HTTPException(404, "Review not found")
return access
ReviewProgressAccessDep = Annotated[ ReviewProgressAccess, Depends(get_review_progress_access),]The dependency obtains document_id from the path and job_id from the query string. It checks that the selected job belongs to this authorized document before the response begins. A missing or inaccessible combination returns 404.
ReviewProgress(...) copies the ORM attributes into a Pydantic object while the short session is still open. Notice the mapping from job.id to job_id: the database attribute and public field have different names. Afterward, the stream has ordinary access values and Pydantic progress data; it does not hold an ORM object or database session.
Here is the endpoint in the documents router:
import asyncioimport timefrom collections.abc import AsyncIterable
from fastapi import Requestfrom fastapi.sse import EventSourceResponse, ServerSentEvent
from ..dependencies import ( ReviewProgressAccessDep, load_review_progress,)
MAX_STREAM_SECONDS = 60
@router.get( "/{document_id}/review-events", name="document_review_events", response_class=EventSourceResponse,)async def document_review_events( request: Request, access: ReviewProgressAccessDep,) -> AsyncIterable[ServerSentEvent]: previous_state = None started_at = time.monotonic()
while time.monotonic() - started_at < MAX_STREAM_SECONDS: if await request.is_disconnected(): return
progress = await asyncio.to_thread( load_review_progress, access, )
if progress is None: yield ServerSentEvent( event="review_unavailable", data={"message": "Review no longer available"}, ) return
current_state = ( progress.status, progress.progress, progress.result_summary, progress.error_message, )
if current_state != previous_state: yield ServerSentEvent( event="review_progress", data=progress, retry=3000, )
previous_state = current_state
if progress.status in { "completed", "failed", }: return
await asyncio.sleep(1)load_review_progress() creates and closes its own SQLAlchemy session inside each worker-thread call. FastAPI encodes the Pydantic value in data as JSON once. The browser’s JSON.parse(event.data) therefore receives the object shape described by ReviewProgress.
The generator does not hold one session for the life of the stream.
Each iteration:
- Checks whether the browser disconnected.
- Opens a short database operation in a worker thread.
- Reads current state.
- Closes the database session.
- Sends an event only if state changed.
- Stops at a terminal status.
- Otherwise waits cooperatively for one second.
After at most about 60 seconds, this connection ends. If the job remains active, native EventSource reconnects and the authentication and access dependencies run again. A slow database call can extend that duration; it is a bounded loop, not a hard network timeout.
The document and job restriction is checked during each read. The user’s role and firm in the access description are refreshed on reconnection, so this example can take up to a connection lifetime to reflect a changed user permission. Immediate revocation would require checking current user access during the stream as well.
Updating the browser
Section titled “Updating the browser”The browser receives an object such as:
{ "job_id": 410, "document_id": 101, "status": "processing", "progress": 80, "result_summary": null, "error_message": null}It updates the DOM:
function updateReviewDisplay(data) { statusElement.textContent = data.status;
progressElement.textContent = `${data.progress}%`;
setReviewControls(ACTIVE_STATUSES.has(data.status));
if (data.status === "completed") { reviewEvents?.close(); reviewEvents = null;
summaryElement.textContent = data.result_summary;
resultSection.hidden = false; messageElement.textContent = "Review completed.";
return; }
if (data.status === "failed") { reviewEvents?.close(); reviewEvents = null;
messageElement.textContent = data.error_message ?? "The review failed.";
return; }
messageElement.textContent = `Review is ${data.status}.`;}Values are inserted through:
textContentThe result summary is treated as text, not trusted HTML.
The browser now displays:
Review status: processingProgress: 80%Review is processing.Later:
Review status: completedProgress: 100%Review completed.
Review resultThree paragraphs require additional review.The database changed first. The DOM changed afterward.
What happens if the POST response is lost?
Section titled “What happens if the POST response is lost?”Consider this sequence:
- The browser sends
POST /documents/101/reviews. - SQLite commits job
410. - FastAPI begins sending
202 Accepted. - The network disconnects before the browser receives it.
JavaScript sees a network failure.
But the review job exists.
This is why our error message says:
The outcome is uncertain. Reload this page before trying again.It does not say:
The review was not started.The browser does not know that.
Reloading the Jinja page will query SQLite and display:
Review status: queuedThe database resolves the uncertainty.
If that job is still active, the reloaded page also opens its events URL and resumes progress updates. If it finished already, Jinja displays the saved result or failure message directly.
A production API may also accept an idempotency key so a safe retry returns the existing job instead of creating a duplicate operation.
What happens if the browser closes?
Section titled “What happens if the browser closes?”Suppose Alice closes the page at 40%.
The browser closes its SSE connection.
The SSE generator detects the disconnection and ends.
But the worker continues:
Browser page→ gone
SSE stream→ closed
Review job→ still processing
SQLite→ still receiving progress updatesWhen Alice returns later:
- The browser requests the Jinja page.
- FastAPI authenticates and authorizes her again.
- SQLAlchemy reads the latest database state.
- Jinja renders
completed. - JavaScript sees that the review is already terminal.
- No live stream is necessary.
The browser observes the job. It does not own the job.
What happens if the worker crashes?
Section titled “What happens if the worker crashes?”A durable row is necessary, but it is not a complete job system.
Suppose the worker changes:
queued → processingand then crashes.
The job may remain:
processingforever unless the system has recovery logic.
A production worker normally adds:
- Attempt count
- Last heartbeat
- Lease expiration
- Retry limit
- Next retry time
- Permanent failure state
- Structured error logging
For example:
Worker claims job at 10:00Lease expires at 10:05
No heartbeat by 10:05→ another worker may retry itThe business operation should also be designed so retries do not create duplicate external effects.
Durable work means more than storing a status string. It means the system can determine what happened after interruption.
Why this workflow does not use WebSockets
Section titled “Why this workflow does not use WebSockets”Our interaction contains:
Browser → serverOne Start review command
Server → browserSeveral progress updatesThat is naturally expressed as:
POST+SSEA WebSocket would require:
- A handshake
- Connection management
- Custom message protocols
- Manual reconnection
- Origin validation
- Per-message authorization
- Multi-worker broadcast infrastructure
It would not eliminate:
- The POST’s business validation
- The durable job table
- The worker
- SQLAlchemy transactions
- Authentication
- Authorization
- Safe DOM handling
If reviewers were exchanging frequent live notes, WebSockets might be justified. For one command followed by progress, they add more moving parts than value.
Could we use polling instead?
Section titled “Could we use polling instead?”Absolutely.
After receiving 202, the browser could call:
GET /documents/101/review-statusevery three seconds.
For a small application, polling may be the best first release.
The decision is practical:
| Requirement | Polling | SSE |
|---|---|---|
| Simplest deployment | Best | More setup |
| Updates every few seconds | Good | Good |
| Immediate progress changes | Acceptable | Better |
| Automatic HTTP request visibility | Better | Less obvious |
| Open connection required | No | Yes |
| Proxy streaming configuration | No | Yes |
| Automatic browser reconnection | Application controlled | Built in |
The worker and database architecture remain the same.
Only the browser-notification mechanism changes.
That is a sign of good separation:
Business processing→ independent of transport
Transport→ replaceable browser-notification choiceWhat each component owns
Section titled “What each component owns”The final architecture has clear responsibilities.
Browser
Section titled “Browser”The browser:
- Displays Jinja HTML
- Handles the submit event
- Creates JSON text
- Sends the authenticated POST
- Opens the SSE connection
- Parses incoming JSON
- Updates the DOM
- Shows connection errors honestly
It does not decide whether the user is authorized.
Middleware
Section titled “Middleware”Middleware:
- Reconstructs the signed session
- Applies application-wide request behavior
- Participates in both the page request and API requests
It does not perform the review.
Dependencies
Section titled “Dependencies”Dependencies:
- Authenticate the current user
- Create request-scoped SQLAlchemy sessions
- Load authorized documents
- Reject unauthorized access before routes proceed
They do not contain the long-running review algorithm.
Router
Section titled “Router”The router:
- Defines public URLs
- Receives HTTP objects
- Invokes CSRF verification
- Translates service exceptions into HTTP statuses
- Chooses
202 Accepted - Returns Pydantic response objects or streaming responses
It coordinates the HTTP boundary.
Pydantic
Section titled “Pydantic”Pydantic:
- Validates the review-depth request
- Defines the accepted response shape
- Serializes progress messages
- Prevents unplanned fields from entering the service
It does not enforce database concurrency.
Service layer
Section titled “Service layer”The service:
- Decides whether a review may start
- Performs the atomic state transition
- Creates the durable job
- Commits or rolls back
- Raises application-specific errors
It does not render HTML or manipulate the DOM.
SQLAlchemy
Section titled “SQLAlchemy”SQLAlchemy:
- Maps Python objects to database rows
- Tracks changes
- Generates SQL
- Manages transactions through sessions
It does not keep the worker alive.
SQLite
Section titled “SQLite”SQLite:
- Stores documents
- Stores jobs
- Coordinates transactions
- Preserves authoritative state
It does not push DOM updates to the browser.
Worker
Section titled “Worker”The worker:
- Claims queued work
- Performs the review
- Stores progress
- Records success or failure
- Continues without an open browser page
It does not own the HTTP response.
SSE:
- Carries progress toward the browser
- Reconnects after temporary network interruption
- Stops when progress reaches a terminal state
It does not perform or persist the review.
JavaScript DOM code
Section titled “JavaScript DOM code”JavaScript:
- Turns server state into visible text
- Keeps loading and error states honest
- Closes unused connections
It does not create authoritative business state merely by changing what appears onscreen.
The complete lifecycle
Section titled “The complete lifecycle”These four sections describe overlapping lifecycles. Once the job is committed, the worker and the browser’s progress connection can advance independently; the numbering follows their responsibilities rather than promising one global execution order.
Initial page
Section titled “Initial page”- The browser requests
/documents/101. - Session middleware reconstructs authentication.
- FastAPI matches the document route.
- Dependencies load the user and database session.
- Authorization limits access to the correct firm.
- SQLAlchemy reads the document and latest job.
- Jinja renders the title, status, URLs and CSRF token.
- FastAPI sends HTML.
- The database session closes.
- The browser constructs the DOM.
- JavaScript registers its submit handler and resumes an existing active job’s events connection when needed.
Starting the review
Section titled “Starting the review”- The user submits the review form.
- JavaScript prevents page navigation.
JSON.stringify()creates the request body.fetch()sends the POST, cookie and CSRF header.- Middleware reconstructs the session again.
- Dependencies authenticate, authorize document access, and require reviewer or administrator permission again.
- FastAPI validates JSON as
ReviewStart. - The route verifies CSRF.
- The service atomically changes the document to
queued. - The service inserts a
ReviewJob, including the requested depth. - SQLite commits.
- FastAPI returns
202 Accepted. - JavaScript parses
ReviewAccepted. - The browser displays
queued.
Performing the review
Section titled “Performing the review”- The separate worker finds the queued row.
- It changes the job and document to
processing. - SQLite commits.
- The worker copies the saved document ID and depth into plain review input.
- Its database session closes.
- The worker performs slow processing.
- Short new sessions store progress.
- The final session stores the summary and
completed. - SQLite commits the terminal state.
Reporting progress
Section titled “Reporting progress”- JavaScript opens the SSE URL containing the selected job ID.
- The browser sends an authenticated GET.
- A short-lived dependency checks access to that document and job, then releases its database session.
- The SSE generator begins.
- Each status check opens a short SQLAlchemy session.
- SQLite returns current progress.
- FastAPI sends a
review_progressevent. - JavaScript parses its JSON data.
- DOM text displays the progress.
- The process repeats, reconnecting with fresh authentication after each bounded connection, until
completed,failed, or loss of access. - JavaScript closes the EventSource.
- The server generator ends.
- The final result remains durable in SQLite.
The final architectural rule
Section titled “The final architectural rule”The easiest way to understand this system is to separate four questions:
Who may request the operation?→ authentication, authorization and CSRF
What operation should occur?→ route, Pydantic model and service logic
Where does the authoritative state live?→ SQLAlchemy transactions and SQLite
How does the browser learn about changes?→ polling, SSE or WebSocketThose questions are related, but they are not interchangeable.
For this document-review workflow, the boring production-oriented choice is:
Jinja→ initial page
fetch() POST→ explicit state-changing command
Durable database job→ reliable ownership of work
Separate worker→ long-running processing
SSE→ one-way live progress
Safe DOM updates→ visible browser resultNo single part is “the application.”
The application is the controlled movement of data and responsibility across all of them.