# Using Page Leases A **lease** gives one client exclusive use of a page across many requests, so a multi-step flow cannot be interrupted by other clients between steps. Leases are **optional**. A page nobody leased behaves exactly as it did before leases existed: every client can use it, no extra header needed. You only need this guide if several clients (scripts, agents, the control panel, a human with curl) can hit the same Penelope instance at the same time. API reference for the two endpoints: [docs/reference/API.md](../reference/API.md#leases). --- ## The problem leases solve Penelope already stops two requests from acting on the same tab at the same instant. Every ops, primitive and plugin call reserves its page (state `busy`) while it runs and frees it (`idle`) when it returns. A second request for that page waits up to 30 s, then fails with `409`. That protection covers **one call**. Most useful work is a **chain of calls** where the page itself carries the state from one step to the next (see "Chainable" in [ADDING_PLUGINS.md](ADDING_PLUGINS.md#best-practices)): ``` client A: open-account @alice → page is on alice's profile client A: open-following → following popup is open client A: get-following → reads the popup client A: close-following ``` Between any two of A's calls the page is `idle`, so client B can slip in: ``` client A: open-account @alice → alice's profile client B: open-account @bob → bob's profile ← B took the idle page client A: open-following → opens BOB's following, or fails on wrong state ``` Nothing crashed, and every single call was correctly serialized, but A's flow is now wrong. A lease closes that gap: while A holds the lease on the page, B's calls on it are refused straight away with `423 Locked`. --- ## Quick start ```bash BASE=http://localhost:5000/api/v1 AUTH="Authorization: Bearer $PENELOPE_KEY" # 1. take the lease (ttl in seconds, default 300) LEASE=$(curl -s -X POST "$BASE/ops/main/lease" -H "$AUTH" \ -H "Content-Type: application/json" -d '{"ttl": 120}' | jq -r .lease_token) # 2. send the token on every call of the flow curl -s -X POST "$BASE/main/plugin/instagram/open-account" -H "$AUTH" -H "X-Lease: $LEASE" \ -H "Content-Type: application/json" -d '{"handle": "alice"}' curl -s -X POST "$BASE/main/plugin/instagram/open-following" -H "$AUTH" -H "X-Lease: $LEASE" curl -s "$BASE/main/plugin/instagram/get-following" -H "$AUTH" -H "X-Lease: $LEASE" # 3. give the page back curl -s -X DELETE "$BASE/ops/main/lease" -H "$AUTH" -H "X-Lease: $LEASE" ``` The same calls with Python `requests`: ```python import requests BASE = "http://localhost:5000/api/v1" s = requests.Session() s.headers["Authorization"] = f"Bearer {PENELOPE_KEY}" r = s.post(f"{BASE}/ops/main/lease", json={"ttl": 120}) r.raise_for_status() s.headers["X-Lease"] = r.json()["lease_token"] try: s.post(f"{BASE}/main/plugin/instagram/open-account", json={"handle": "alice"}).raise_for_status() s.post(f"{BASE}/main/plugin/instagram/open-following").raise_for_status() following = s.get(f"{BASE}/main/plugin/instagram/get-following").json() finally: s.delete(f"{BASE}/ops/main/lease") ``` Release in a `finally` block. If the client dies before releasing, the lease still expires on its own (see [Expiry](#expiry-and-renewal)). --- ## Concepts | Term | Meaning | |---|---| | **Lease** | A claim on one page, held by whoever knows its token. One lease per page at most. | | **Token** | Random opaque string (`secrets.token_urlsafe(16)`) returned by `POST /ops/{page_id}/lease`. Treat it as a secret for the length of the flow. | | **Holder** | Any request carrying the matching `X-Lease` header. Penelope doesn't know about client identity; the token *is* the identity. | | **TTL** | Seconds of inactivity after which the lease is dropped. Range `0 < ttl ≤ 600`, default `300`: a leased page idle for 10 minutes is always freed. | A lease isn't the same as a page reservation: - **Reservation (`busy`)**: automatic, one call long, always on. It stops two calls from interleaving *inside* the browser. - **Lease**: explicit, many calls long, opt-in. It stops other clients from running *between* your calls. A holder's calls still reserve the page as usual. Two parallel requests with the same token on the same page are serialized just like any other two requests (the second waits for the first, up to 30 s). A lease gives exclusivity against other clients, not parallelism within one. --- ## What a lease controls With an active lease on page `P`, a request **without** the matching token (no `X-Lease` header, a wrong one, or an old one) gets `423` for: | Route family | Examples | |---|---| | Ops on `P` | `navigate`, `extract-content`, `screenshot`, `search-object` | | Destroying `P` | `POST` / `DELETE /ops/P/destroy-page` | | Primitives on `P` | every `/primitive/P/...` route | | Plugins on `P` | every `/P/plugin/...` route, all families | | The lease itself | `POST /ops/P/lease` (can't steal), `DELETE /ops/P/lease` (can't release someone else's) | A leased page is also kept **in the foreground**: granting or renewing the lease, and every call that uses it, activates its tab (`page.bring_to_front()`), so the flow is the one visible on VNC. With two leased pages driven at the same time, the tab in front follows whichever was called last. Unleased pages never change the active tab (only `create-page`, which opens the new tab active, and `zoom-page` do). A lease does **not** affect: - **Other pages.** Leasing `main` leaves `scraper` open to everyone. - **Creating pages.** `POST /ops/create-page` is page-less and always allowed. - **Read-only system endpoints.** `GET /system/status/pages` still lists the page and reports `"leased": true`. It never shows the token. - **The shared browser session.** All pages live in one persistent browser context, so cookies, logins and localStorage are shared. Leasing a page doesn't stop another client from using the *same account* in another tab. If a site must only be driven by one flow at a time, lease the page every client uses for that site, or coordinate at the client level. Internal callers that ask for "any free page" (no `page_id`) skip pages leased by someone else. None of the HTTP routes do this today, since they all take a `page_id`, but it keeps the pool honest. --- ## Status codes | Code | When | What to do | |---|---|---| | `200` | Lease granted, renewed or released; or a call on a page you may use | continue | | `404` | `page_id` doesn't exist (lease endpoints don't create pages) | `POST /ops/create-page` first | | `409` | Page is **busy** with a running call and stayed busy past the 30 s wait | retry later; not lease-related | | `422` | Invalid `ttl` (`≤ 0` or `> 600`, or not a number) | fix the body | | `423` | Page is **leased by another client**, or your token is no longer the holder | see [Recovering from 423](#recovering-from-423) | `423` comes back **immediately**: Penelope doesn't wait for a lease to free up, because leases last minutes, not the few seconds a busy page does. Error body (standard `ErrorResponse` envelope): ```json { "success": false, "error": "Page main is leased by another client", "page_id": "main" } ``` --- ## Lifecycle ``` POST /lease call with X-Lease unleased ─────────────────▶ leased ◀──────────────────────┐ (renews) ▲ │ │ │ │ DELETE /lease │ └─────────────────────────┘ ├──────────────────────────┘ │ ttl passes with no calls, page not busy ├────────────────────────── (dropped within a minute) │ └── page destroyed / closed ─ lease disappears with the page ``` ### Taking a lease `POST /api/v1/ops/{page_id}/lease`, body optional: ```json { "ttl": 300 } ``` ```json { "success": true, "page_id": "main", "lease_token": "J152Qv0L88Ropw1ZAAC-cQ", "ttl": 300 } ``` - Works whether the page is `idle` or `busy`. Leasing a busy page doesn't interrupt the running call: that call finishes normally, and only *later* calls from other clients are refused. - If another client holds a live lease: `423`. - If an earlier lease has expired: a new lease with a **new token** is granted. ### Renewing Two ways, both keep the same token: 1. **Implicitly.** Every call that uses the page with the right `X-Lease` pushes the expiry to `now + ttl`, both when the call starts and when it finishes. A flow that keeps calling never needs to renew by hand. 2. **Explicitly.** Call `POST /ops/{page_id}/lease` again *with* your `X-Lease` header. This returns the same token and sets the new `ttl` from the body (or the default 300 if there is no body). Use it to keep a lease alive across a long client-side pause, or to change the TTL. ### Releasing `DELETE /api/v1/ops/{page_id}/lease` with your `X-Lease`: ```json { "success": true, "page_id": "main", "message": "Lease on 'main' released" } ``` - Wrong or missing token while the lease is live: `423`. - Lease already expired (nobody holds it): succeeds. Release is idempotent toward expiry, so a `finally: release()` never fails just because you were slow. - Releasing doesn't touch the page: its URL, open popups and scroll position stay as they are. The next client inherits whatever state you left. ### Expiry and renewal - The deadline is `last activity + ttl`, where activity means a holder call starting or finishing, or an explicit renew. - **A lease never expires mid-call.** While the page is `busy`, the lease is kept even if the deadline passes, and the call's completion renews it. A `get-following` that scrolls for 10 minutes under a 60 s TTL is safe. - A **ticker** drops expired leases every minute, so an idle leased page is freed at most a minute after its deadline. Any check in between (a call, a lease request, `GET /system/status/pages`) drops it at once. Until then the status may show `"leased": true, "lease_expires_in": 0.0`, with the same effect: the next check sees it expired. - Pick a TTL longer than the **longest gap between two calls** in your flow, including client-side thinking time (an LLM agent deciding its next step can take a while). Call durations don't count, since the lease can't expire mid-call. The TTL can't pass 600 s: a flow that pauses longer renews the lease explicitly during the pause. ### When leases disappear - **Server restart** (including a restart by `updater.py` after a new commit): leases live only in memory, so all of them are gone. The pages are recreated too, so your flow has to start over anyway. - **Page destroyed** by its holder (`destroy-page` with `X-Lease`), or the tab closed from the browser: the lease goes with the page. - **Expiry**, as above. --- ## Recovering from 423 A `423` means one of two things: 1. **You never had the lease.** Someone else is using the page. Pick or create another page, or retry later. `GET /system/status/pages` shows `leased` and `lease_expires_in` per page to help you decide. 2. **You had the lease and lost it.** Your gap between calls was longer than the TTL, the lease expired, and possibly another client leased the page in between. Your old token will never work again. In case 2, **don't assume the page is where you left it**. Another client may have navigated it. Take a new lease, then re-observe the page before going on. With state-machine plugin families, run their state reader first (`instagram/state`, `bank/bper/read-state`, ...) and follow its `hint`, or restart the flow from its first step. A `423` with **no** `X-Lease` header on an unleased-looking page means another client leased it a moment ago. Same as case 1. --- ## Patterns ### One page per worker The simplest safe setup is for each worker to create its own page and lease it for its whole lifetime, renewing it (`POST /ops/{page_id}/lease` with its `X-Lease`) whenever it would sit idle near 10 minutes: ```bash curl -s -X POST "$BASE/ops/create-page" -H "$AUTH" -H "Content-Type: application/json" \ -d '{"page_id": "worker-7"}' curl -s -X POST "$BASE/ops/worker-7/lease" -H "$AUTH" -H "Content-Type: application/json" \ -d '{"ttl": 600}' # ... work ... curl -s -X DELETE "$BASE/ops/worker-7/destroy-page" -H "$AUTH" -H "X-Lease: $LEASE" ``` Destroying the page as the holder ends the lease as well. The pool is capped at `MAX_PAGES` (default 5), so at most that many workers can each have their own page. ### Short critical sections on a shared page If pages are shared and only some steps depend on each other, lease just around those steps and release right after. Other clients get the page back sooner. ### Watching without interfering `GET /system/status/pages` needs no auth and doesn't reserve the page, so a dashboard can watch leased pages freely. Calls that *do* reserve the page, such as `screenshot` and `extract-content`, are refused on a page leased by someone else. Pass the token if the observer is part of the same flow. ### Human in the loop / control panel The control panel at `/` doesn't send `X-Lease`, so a page leased by a script shows `423` errors in the panel. That's intended: the script owns the page. To take over by hand, stop the script and either release its lease or wait for the TTL to run out. --- ## For plugin authors Plugins get lease enforcement for free, **as long as they acquire their page through `browser_navigator._with_page(page_id, ...)`** (or `_get_page_to_use`), as the [plugin template](ADDING_PLUGINS.md#plugin-template) does. The `X-Lease` header is read by a router dependency (`worker/lib/lease.py:bind_lease`) and stored in a context variable that page acquisition checks. Plugins never see or pass the token. Don't reach into `browser_navigator.get_page_pool().pages[...]` to use a page directly. That skips reservation *and* lease checks. Nothing changes for plugin parameters, return values or routes. --- ## Implementation notes | Piece | Location | |---|---| | `X-Lease` → context variable | `worker/lib/lease.py` (`current_lease`, `bind_lease`), attached to the ops, primitive and plugin routers in `worker/server.py` | | Lease state per page | `worker/lib/page_management.py` — `ManagedPage.lease_token / lease_ttl / lease_expires`, `is_leased`, `lease_allows`, `grant_lease`, `renew_lease`, `release_lease` | | Ticker | `IntelligentPagePool.sweep_leases`, run every minute by `store.sweep_loop` (started in `worker/server.py`) | | Check on acquire | `ManagedPage.try_acquire` (atomic, under the page lock) and `BrowserNavigator._get_page_to_use` (fast `423` before waiting) | | Check on destroy | `BrowserNavigator.destroy_page` | | Foreground | `ManagedPage.bring_to_front`, called by `BrowserNavigator.lease_page` and by `_get_page_to_use` when the page is leased; a failure is logged, never raised | | Endpoints | `worker/routers/ops.py` — `ops_lease_page`, `ops_release_lease` | | Models | `worker/models/ops.py` — `LeasePageRequest`, `LeasePageResponse`, `ReleaseLeaseResponse` | | Status fields | `ManagedPage.get_status` → `leased`, `lease_expires_in` | Tokens are compared with `secrets.compare_digest`. CORS allows the `X-Lease` header, so browser clients can send it. Known limits: - In memory only; lost on restart. - One holder per page. Leases can't be shared read-only, and there's no queue of waiters: a refused client has to retry. - No admin override. Holding the API key isn't enough to break someone else's lease; wait for the TTL (at most 10 minutes, plus up to a minute for the ticker) or restart the server.