Cookbook / Data and storage
Keyset pagination and optimistic locking
Page through large tables at constant cost, and stop two editors from silently overwriting each other.
Two things that matter once a table is big and several people use it. OFFSET pagination gets slower the deeper you go, and loses or repeats rows when data changes between pages. Keyset (cursor) pagination avoids both. Optimistic locking makes a stale save fail instead of overwriting someone else's change.
use axeon_orm::prelude::*;
use axeon_orm::Error;
#[derive(Model, Clone, Debug)]
struct Product {
id: i64,
name: String,
price: i64,
#[model(version)]
version: i64,
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let db = Db::connect("sqlite://paging.db?mode=rwc").await?;
db.create_table::<Product>().await?;
let rows: Vec<Product> = (1..=25)
.map(|n| Product { id: 0, name: format!("item {n}"), price: n * 10, version: 0 })
.collect();
Product::insert_many(&db, &rows).await?;
// Keyset pagination: no OFFSET and no COUNT(*), so page 1000 costs the same as page 1.
let mut after: Option<String> = None;
let mut page_number = 0;
loop {
let page = Product::query()
.order_by(Product::PRICE.desc())
.cursor_paginate(&db, after.as_deref(), 10)
.await?;
page_number += 1;
println!("page {}: {} rows, first = {}", page_number, page.items.len(), page.items[0].name);
match page.next_cursor {
Some(cursor) => after = Some(cursor), // send this to the client as ?after=...
None => break,
}
}
// Optimistic locking: two editors load the same row...
let mut alice = Product::find_or_fail(&db, 1).await?;
let mut bob = Product::find_or_fail(&db, 1).await?;
alice.price = 99;
alice.update(&db).await?; // saves, bumps the version, and reloads the row in place
println!("alice saved, version is now {}", alice.version);
// ...and the second save, made from stale data, is refused.
bob.price = 1;
match bob.update(&db).await {
Err(Error::StaleRecord(_)) => println!("bob's save was refused: someone else changed it first"),
other => println!("unexpected: {:?}", other),
}
Ok(())
}
cargo run
Notes#
- A cursor holds the sort values of the last row, and the next page is the rows after them. It is opaque and URL-safe: return it as
next_cursorand accept it back as?after=. - The primary key is added as a tie-breaker, so ordering is total even when the sort column has duplicates. You can mix ascending and descending columns.
- Sort columns must be plain and non-nullable. A cursor made for another sort order, or a damaged one, fails with
Error::InvalidCursor(a 400 in Axeon handlers). StaleRecordis a 409 in Axeon handlers: tell the user their copy is out of date and let them reload.Query::updateand upserts bump the version too.- Keyset pagination cannot jump to page 7 or show a total. If you need those, use
paginate.