# fenecdb

> Minimal embedded document database: one file, live queries an app's
> screens follow, a replica on the device that syncs with a server, a server
> over HTTP with a file per tenant, and search built in -- BM25 over text,
> with typos, highlights and facets, and HNSW vectors for meaning. It runs in
> the browser as WebAssembly, in an app, and as a server. Its query language
> is FenecQL, which is not SQL but reads SQL's everyday spellings.

Writing FenecQL -- the reference below spells it out in full:

- It is not SQL, but it reads SQL's everyday spellings as the statements
  they mean: `CREATE TABLE` and `CREATE INDEX`, `INSERT INTO t (a, b)
  VALUES (1, 'x')`, `UPDATE t SET a = 1 WHERE ...`, `IN (1, 2)`,
  `BETWEEN`, `LIKE` and `ILIKE`, `ON CONFLICT DO NOTHING` and `DO UPDATE
  SET`, `DISTINCT`, `EXISTS`, single quotes and `||`
  (https://fenecdb.com/docs/sql). A double-quoted word is text, never a
  column: a field is written bare, `where name = 'Ada'`. Every row has an
  `id` (an int the database makes; no `rowid`, no `_id`). A JOIN on a
  key is read as `lookup`, which FenecQL writes `get orders lookup users on
  id = user_id` -- each order with its user nested; the clauses after
  `lookup` are the looked-up collection's -- and a filter by another
  collection's rows is an uncorrelated
  `in (get <collection> select <field> where ...)`, or `not in (...)` for
  the rows it does not name. There is no interactive
  transaction: a `/batch` of statements (in the page, a `run` of several)
  lands whole or not at all, and `require <n>` after a write makes its count
  a condition that puts the whole batch back.
- The aggregates -- `count(*)`, `count(f)` (the rows where `f` is not
  null), `count(distinct f)`, `sum`, `avg`, `min`, `max`, `first(f by k)`,
  `last(f by k)` -- go in the select list, over every match or per
  `group <key>`, and `having` keeps groups. Distinct values are a group:
  `select city group city`.
- Text: `like 'a%'` and `ilike` take SQL's `%` and `_`; `~` is a
  case-insensitive *contains* (`email ~ "@example.org"`), never a regular
  expression, and `match` ranks words over an `@text` field.
- Square brackets are list literals and nothing else: `tags: ["a", "b"]`,
  `tags [text]`. Nothing here marks an optional part with them; a clause you
  do not need is simply left out.
- Strings take double or single quotes. Parameters are `$1`, `$2`, ...; a vector
  parameter is a list of numbers. A timestamp is written as text,
  `"2026-09-01T10:00:00Z"`, or `"2026-01-01"` in a comparison.
- Types: `bool int float text bytes timestamp`, lists such as `[text]`,
  `json` (any JSON value; a path such as `meta.source` reads into it),
  `geo` (a point, `[lon, lat]`), `vector<N>`, `vector<N, f16>` and
  `sparse<N>`. A typed field holds no object; a `json` field does. There is
  no decimal -- money is an `int` of cents -- and no UUID type (use
  `text @hash`).
- Indexes: `@hash` for equality, `@unique` for a value only one row may hold,
  `@sorted` for ranges and for `order` with a `limit`, `@ttl(30m)` for rows
  that expire, `@text` for `match`, `@hnsw(cosine)` (or `l2`, `dot`) for
  `near`. Only an `and` chain uses an index: `=` and `in [...]` on a `@hash`
  field or on `id`, `<`, `<=`, `>`, `>=` and `=` on a `@sorted` field;
  everything else scans. `explain get ...` runs a query and returns the path
  it took.
- Text is ordered by its bytes; `order title collate tr` orders one query's
  text as Turkish does (ICU's `tr`: `ç` after `c`, `ı` before `i`). A field
  declared `name text collate tr` compares by it wherever its text is ordered
  -- `order`, `<` and `>` in `where`, `min`, `max`, its `@sorted` index --
  while `=` compares bytes.
- A text field declared `nocase` (`email text nocase @unique`) compares by
  its case fold everywhere -- `=`, `in`, its indexes, `order`, `group`,
  `facet` -- and keeps what was written: `Alice@x.com` is `alice@X.COM`.
- `match` ranks by BM25 and `near` by distance; each decides the order, so
  neither combines with `order`, and the two together need `fuse`, which
  ranks by both. `match ... fuzzy` finds a word typed wrong and
  `match ... prefix` one half typed. Each returns at most 10 000 rows
  (`limit + offset`) and adds a `_score` column.
- After `lookup`, every clause belongs to the child collection and `limit`
  counts children per parent. `on <field>` names the child's field that
  holds the parent's `id` (`get orders lookup items on order_id`); `on <its
  field> = <this one's>` runs the other way (`get items lookup orders on
  id = order_id` attaches each item's order). `required` goes right after
  `on <field>` and keeps only the parents with a matching child -- SQL's
  `exists`; `count` goes before `lookup`.

Every statement, by example:

```fenecql
create collection articles (title text @hash, views int, tags [text], published timestamp @sorted, body text @text, embed vector<4> @hnsw(cosine))
put articles {title: "Rust", views: 10, tags: ["lang"], published: "2026-09-01T10:00:00Z", body: "Ownership and borrowing", embed: [0.1, 0.2, 0.3, 0.4]}
put articles [{title: "Zig", views: 3}, {title: "Go", views: 7}]
get articles select title, views where views < 100 and tags has "lang" order views desc limit 5 offset 5
get articles select title order title collate tr, views desc limit 20
get articles where title in ["Rust", "Go"] count
get articles select title, sum(views), max(published) where tags has "lang" group title order sum(views) desc limit 5
get articles order published desc limit 20 lookup comments on article_id where score >= 4 order published desc limit 3
get articles count lookup comments on article_id required where score = 5
get comments where score >= 4 lookup articles on id = article_id select title
insert into articles (title, views) values ('Nim', 4), ('Odin', 2)
get articles where title like 'R%' and views between 1 and 100 and title not in ["Zig"]
get articles select title, highlight(body) match body "borowing" fuzzy facet tags limit 10
get articles select title where published >= "2026-01-01" near embed $1 limit 3
get articles match body "borrowing" near embed $1 fuse limit 10
get articles match body "borrowing" rerank embed $1 candidates 200 limit 10
explain get articles where views < 100 order published desc limit 5
set articles {views: views + 1} where title = "Rust" require 1
del articles where views > 100000
create index on articles (views) @hash
drop collection articles
```

From JavaScript, `db.from('articles').where('views', '<', 100).order('published', 'desc')
.limit(10).rows()` builds the same FenecQL with every value a parameter, and
`db.live(query, draw)` runs it again after every write to what it reads.


---

Source: https://fenecdb.com/docs/

# Documentation

fenecdb is an embedded document database small enough to
run inside a browser tab. It also keeps an app's file on a phone and serves
any language over HTTP, a file per tenant; search is built in.

## What it is

Your documents live in one file, indexed as you declare, and you query
them with one language, FenecQL — in the page, in an app, from Rust, or over
HTTP from a server. A screen's live query runs again when a write changes
what it reads, a replica on the device syncs with the server, and the
server takes many clients' writes into the one file through one writer,
their disk syncs shared, and many tenants into a file each.

```fenecql
create collection articles (
  title     text  @text,
  tags      [text],
  year      int   @hash,
  published timestamp @sorted
)

get articles select title
  where year >= 2024 and tags has "rust"
  order published desc
  limit 10

get articles select title, highlight(title)
  match title $1 fuzzy
  limit 10
```

Details

fenecdb keeps its documents in one file and indexes them — hash,
unique, ordered, text, points and vectors. The indexes are held in memory
and built from the documents, the first time a query reads each, and a
vector graph is saved in the file too, so an open need not build it again.
It answers queries from inside a browser tab, in an app, or from a server
any language reaches over HTTP.

Everything in these docs describes the engine as it is built today. Where
something costs you a capability, that cost is stated next to the feature
rather than at the bottom of the page.

One file. Records are appended to immutable 8 MiB segments and the byte
sequence on disk is the byte sequence in memory, so a read decodes directly
over the arena. A `HashMap<DocId, Loc>` offset index is the
only helper structure in the whole store.

Search is not bolted on. `@text` is an index and
`match` a query clause, ranked by BM25, with `fuzzy`,
`highlight()` and `facet` beside it; for search by
meaning, `vector<N>` is a type, `@hnsw` an index
and `near` a clause. The planner knows them all, so a filter and a
search are planned together rather than stacked on each other.

## Where to go next

New here? Start with the Quickstart. The rest are guides to come back
to.

- [Quickstart](https://fenecdb.com/docs/quickstart)

Build it, open a file, write a row and query it. Five minutes from clone.

- [How it works](https://fenecdb.com/docs/concepts)

Segments, the mapped file, one writer, and why there is no page cache.

- [FenecQL](https://fenecdb.com/docs/fenecql)

The language: statements, types, operators, indexes, parameters.

- [From SQL](https://fenecdb.com/docs/sql)

The SQL it reads as written, what it refuses and what to write instead, and where the meaning is not SQL's.

- [JavaScript and TypeScript](https://fenecdb.com/docs/javascript)

The browser client, the query builder and generated TypeScript types.

- [Mobile and native apps](https://fenecdb.com/docs/mobile)

Swift, Kotlin and Flutter: a file on the device, live queries driving the UI.

- [HTTP endpoint](https://fenecdb.com/docs/http)

A REST surface derived from the schema, plus a subscription stream.

- [Sync](https://fenecdb.com/docs/sync)

A local replica that reads without the network and writes optimistically.

- [Server](https://fenecdb.com/docs/server)

Running `fenec-server`: flags, tokens, durability, limits, containers.

- [Replication](https://fenecdb.com/docs/replication)

Read replicas, promotion, archives and backups restored to a moment, every tenant of a node archived at once.

- [Tenants and sharding](https://fenecdb.com/docs/sharding)

A file per tenant behind a router that fails nodes over; regions, one URL for all of them, reads near the client, and a kit to deploy them.

- [Integrations](https://fenecdb.com/docs/integrations)

LangChain, LlamaIndex, the Vercel AI SDK and React.

- [Serverless and Cloudflare](https://fenecdb.com/docs/serverless)

A database in a Durable Object, or a function talking to a server.

- [Embedded Rust](https://fenecdb.com/docs/embedding)

`fenec-core` as a library inside your own program.

- [Benchmarks](https://fenecdb.com/docs/benchmarks)

YCSB against SQLite, PostgreSQL, MongoDB and Redis, writers beside readers, requests and search, each with its method.

- [Compare](https://fenecdb.com/docs/compare)

Beside PostgreSQL, SQLite and PGlite: when to choose each.

- [Limits](https://fenecdb.com/docs/limits)

What it does not do, and every hard-coded ceiling.

## Where it runs

The same engine ships four ways. Pick the one that matches where your data
should live.

| Target | What you get | Size |

| Browser | WebAssembly module plus a dependency-free ES client | 330 KB gzip
the wasm module |

| Command line | The `fenec` shell, `fenec import`, `fenec types`, `fenec backup` and `restore` | 1880–3013 KB |

| Server | `fenec-server`: HTTP, replication and tenants in one process | 2.99 MB image |

| Embedded | `fenec-core` as a Rust library | zero dependencies |

## What it gave up

Staying small meant leaving some things out on purpose. The main ones:

- **One writer.** Many readers, but one write or batch at a time.

- **No SQL dialect, and a join only on a key.** Every client speaks FenecQL, over HTTP or in process; it reads SQL's everyday spellings as the statements they mean, a `JOIN` on an indexed key among them ([From SQL](https://fenecdb.com/docs/sql)).

- **No decimals.** Store money as cents.

Details

The list is short, and it is made of consequences of the design rather
than a roadmap.

- **One writer, no MVCC.** A write, or a `/batch` of them,
    holds the write lock for as long as it runs and lands whole or not at
    all; readers read beside it. There are no interactive transactions held
    open across requests.

- **A join only on a key.** A query reads one collection, plus the
    children `lookup` attaches to each of its rows — and,
    chained, the children of those — and filters by another's rows only
    through an uncorrelated `in (get …)`. SQL's
    `JOIN` on an indexed key reads as one of those, answered flat;
    a join of any other condition is none.

- **No SQL dialect.** FenecQL is the one language, in the page, in
    Rust and over HTTP. It reads `CREATE TABLE`, `INSERT …
    VALUES`, `UPDATE … SET`, `JOIN`,
    `IN (…)`, `EXISTS`, `BETWEEN`,
    `LIKE` and `ON CONFLICT` as the statements they
    mean and refuses the rest saying what to write; there is no wire
    protocol for SQL drivers, and no ORM speaks it.

- **No multi-writer replication,** and no change of a field's type in
    place: it would rewrite every document under the write lock.

- **No decimal type.** `float` is binary floating point and
    unfit for money; the answer is an `int` in cents.

- **Objects only in a `json` field.** A typed field refuses
    one and a list holds one type; a `json` field holds any value
    JSON does, and a path reads into it.

The full accounting, including every hard-coded ceiling, is in
[Limits](https://fenecdb.com/docs/limits).

## How the crates fit together

The code is a stack of small Rust crates; the engine at the bottom has no
dependencies at all.

Details

Dependencies point one way. Nothing points back up.

```text
fenec-core   storage, indexes, plan executor, plugins, JSON, calendar, changes
     |       std only, zero dependencies
fenec-ql     FenecQL lexer and parser              fenec-wasm   browser ABI
     |                                                          (no wasm-bindgen)
     |
fenec-http   REST/JSON surface, SSE subscriptions, replication, tenants
     |
fenec-wire   the PostgreSQL client the importer reads with
     |
fenec-import SQLite file reader, PostgreSQL COPY source, --follow
     |
fenec-server the server: HTTP, replication, tenants, the --follow mirror
     |
fenec-cli    the `fenec` shell
```

`fenec-bench` is a measurement harness and the only crate allowed
external dependencies — that is where `rusqlite` and
`postgres` live, so the comparisons in
[Benchmarks](https://fenecdb.com/docs/benchmarks) can run against the real engines.


---

Source: https://fenecdb.com/docs/quickstart

# Quickstart

Go from a fresh clone to a collection you write rows into,
read back and search, in the browser, on a server or in the shell.

## Build

-

Clone the repository.

```bash
git clone https://github.com/fenecdb/fenec
cd fenec
```

-

Build and run the tests.

```bash
make test      # 705 Rust tests, 103 JS tests
```

**The wasm32 target comes from rustup.** Homebrew's `cargo`
  ships no wasm32 standard library. The Makefile prefers
  `~/.cargo/bin/cargo`; building by hand needs
  `rustup target add wasm32-unknown-unknown` first.

Details

That produces `target/release/fenec` (the shell) and
`target/release/fenec-server` (the server). The Rust suite runs first on
purpose: `cargo test` builds `fenec-server`, and the JS sync
tests run against that binary.

## In the browser

-

Build the module and open the console at `http://localhost:8787`.

```bash
make serve                    # or: make serve PORT=3000
```

-

For your own page, take the files it needs.

```bash
cp web/fenec.js web/builder.js web/http.js web/fenec.wasm my-page/
```

-

Open a database, make a collection and write two rows.

import { Fenec } from './fenec.js';

const db = await Fenec.open('./fenec.wasm');
db.run('create collection notes (title text @text, year int @sorted)');
db.run('put notes {title: $1, year: $2}', ['Dunes', 2021]);
db.run('put notes {title: $1, year: $2}', ['Night at the oasis', 2024]);

-

Read them back by a filter and an order, and search the words, a
    typo and all.

```js
const { rows } = db.run('get notes where year >= $1 order year desc', [2020]);
// [{ id: 2, title: 'Night at the oasis', year: 2024 }, { id: 1, title: 'Dunes', year: 2021 }]

const hits = db.rows('get notes select title match title $1 fuzzy', ['oasys']);
// [{ title: 'Night at the oasis', _score: 0.62... }]
```

-

Keep the rows in step as they change: a live query hands its
    function the rows now, and again after every write to what it
    reads.

```js
db.live(db.from('notes').where('year', '>=', 2020).order('year', 'desc'), (rows) => {
  console.log(rows.map((n) => n.title));   // draw the list here
});
db.run('put notes {title: $1, year: $2}', ['Sand and stars', 2023]);   // logged again, the new row in it
```

-

Serve the page over HTTP — it does not open from `file://`.

```bash
python3 -m http.server
```

Details

`make serve` builds the wasm and serves
`http://localhost:8787`. `web/index.html` is a full
console: a FenecQL editor, a sample data generator, live statistics and
IndexedDB persistence.

For your own page you need four files: `fenec.js`, the
`builder.js` and `http.js` it imports, and
`fenec.wasm`. No npm, no bundler, no build step. A page whose
queries run on a server needs only `client.js` and those two
([Which package](https://fenecdb.com/docs/javascript#packages)).

import { Fenec, persist, restore } from './fenec.js';

const db = await Fenec.open('./fenec.wasm');

db.run('create collection notes (title text @text, year int @sorted)');
db.run('put notes {title: $1, year: $2}', ['Night at the oasis', 2024]);

const { rows } = db.run('get notes where year >= $1 order year desc limit 10', [2020]);

// or with the builder, for dynamic filters and bound parameters
const recent = await db.from('notes').where('year', '>=', 2020).order('year', 'desc').limit(10).rows();

await persist(db);      // write the snapshot to IndexedDB
await restore(db);      // read it back next session
// or, in a worker: a file every statement is flushed into, which fenec-server opens too
// const file = await openFile(db, 'notes.fenec');

A page can declare its collections in code instead, and the open makes
them ([State in the page](https://fenecdb.com/docs/javascript#state)).

**It does not open over `file://`.**
  `WebAssembly.instantiateStreaming` and module
  `import`s both need an HTTP origin. Any static server works:
  `python3 -m http.server`, `npx serve`, nginx.

The same module runs in Node, where `fetch` cannot resolve a
relative path — so hand it the bytes:

```js
import { readFile } from 'node:fs/promises';
const db = await Fenec.open(await readFile('./fenec.wasm'));
```

## As a server

-

Start the server on a file.

```bash
./target/release/fenec-server --file data.fenec --http 127.0.0.1:8080
```

-

Make a collection and write a row. The language is FenecQL, and the
    values go as parameters.

```bash
curl -X POST http://127.0.0.1:8080/query \
  -d '{"query": "create collection notes (title text @text, year int @sorted)"}'
curl -X POST http://127.0.0.1:8080/query \
  -d '{"query": "put notes {title: $1, year: $2}", "params": ["Night at the oasis", 2024]}'
```

-

Read it back by a filter and an order, or search the words.

```bash
curl -X POST http://127.0.0.1:8080/query \
  -d '{"query": "get notes where year >= $1 order year desc limit 10", "params": [2020]}'
curl -X POST http://127.0.0.1:8080/query \
  -d '{"query": "get notes select title match title $1 fuzzy", "params": ["oasys"]}'
```

-

Or use the REST routes, and follow the rows as they change.

```bash
curl 'http://127.0.0.1:8080/notes?select=title,year&year=gte.2020&order=year.desc&limit=10'
curl -N 'http://127.0.0.1:8080/notes/changes?year=gte.2020'
```

Details

Every language reaches the server over HTTP: [Your
language](https://fenecdb.com/docs/languages) has the same requests in each, and every route is on HTTP endpoint. A write's answer is
`{"affected": 1}`, a read's an array of rows, and the stream
of `/notes/changes` sends the rows its filter holds, then each
change to them ([Subscriptions](https://fenecdb.com/docs/http#subscriptions)).

```bash
./target/release/fenec-server --ping        # 0 means up — a health check
```

Tokens, durability, limits and containers are on
[Server](https://fenecdb.com/docs/server). Past one machine, a read replica and
archives are on [Replication](https://fenecdb.com/docs/replication), a file per
tenant behind a router, in one region or several, on Tenants and sharding, and the systemd units, alert
rules and runbook to run them on machines of your own in Deploying it.

## In the shell

-

Open a file of its own in the shell — not the server's
    `data.fenec`, which two processes would corrupt.

```bash
./target/release/fenec notes.fenec
```

-

Create a collection, each field typed and the ones you filter,
    order and search by indexed.

```fenecql
create collection notes (
  body   text @text,
  topic  int @hash,
  posted timestamp @sorted
)
```

-

Write your rows. A write runs at the empty line after it, or at
    a `;`.

```fenecql
put notes {body: "a night at the oasis, under the stars", topic: 3, posted: now()}
put notes [ ... 100 000 more ... ]
```

-

Read them back, newest first.

```fenecql
get notes select body, posted where topic = 3 order posted desc limit 10
```

-

Search the words. `fuzzy` finds them typed wrong.

```fenecql
get notes select body match body "oasys stars" fuzzy limit 10
```

Details

```bash
./target/release/fenec notes.fenec              # interactive
./target/release/fenec notes.fenec -c 'get notes limit 5'
echo 'get notes' | ./target/release/fenec notes.fenec
./target/release/fenec notes.fenec < seed.fenecql   # a script
```

Inside the shell: `.help` `.tables` `.stats`
`.functions` `.save <path>` `.checkpoint`
`.quit`.

A statement may run over several lines, and the shell runs it only
once nothing after it can go on with it. Typed, a read or a schema change
written on one line runs at Enter. A write, and a read written over
several lines, wait for the next line: one that goes on with them —
`where`, `order`, `and` — joins them, and
an empty line or another statement runs them first. A `;` after
a statement runs it at once. So `del notes` with `where id
= 3` on the line under it deletes one row. A `.` command
leaves a waiting statement waiting — `.tables` to look a name
up before the `where` — and `.quit` leaves it
unrun.

Piped from a file, a statement runs once the next one begins, and a
paste is read the same way, its lines all there at once — a
`.` command in either waits behind the statement before it,
and runs once that has. `-c` reads its text whole, as
`POST /query` does, so a text that does not parse runs
nothing. A line that begins no statement — a `where` after a
read that already ran at Enter, a `wher` mistyped, a clause
written past a `;` — is refused, and so is the statement before
it, which it may have meant to go on with: `del notes; where id =
3` deletes nothing.

A paste the terminal delivers in pieces further apart than 40 ms is
read as typed lines, and a pasted statement whose last line has no
newline yet waits for Enter like a typed one.

Fields and indexes can come later too, with no row rewritten:
`alter collection notes add field mood text`, then
`create index on notes (mood) @hash` over the rows already there
([Changing a collection](https://fenecdb.com/docs/fenecql#changing-a-collection)).
`prefix` finds a word half typed (Typos and words begun), and
`highlight()` and `snippet()` mark what matched (Highlights and snippets).

## Search by meaning

A vector field adds search by meaning beside the words: `near`
ranks the rows by how close their embedding is to a query's, and
`fuse` puts that ranking together with `match`'s. The
vectors come from your embedding model. In the shell, on the notes above,
each is written out as its list of 384 numbers, since the shell binds no
parameters:

```fenecql
alter collection notes add field embed vector<384>
put notes {body: "hnsw is a navigable small world graph", topic: 3, posted: now(), embed: [0.12, -0.03, …]}
create index on notes (embed) @hnsw(cosine)
get notes select body where topic = 3 near embed [0.11, -0.02, …] limit 10
```

Over `POST /query` the statement says `near embed
$1` instead and the vector goes in `params`, as every
value does there; in the page, through `db.run`, it is a
`Float32Array` among the parameters.

Details

Load the vectors first and build the index after, not before: the
graph is then built once over all of them, on every core, rather than a
row at a time. The reason is in How it works.

When a file with a vector index is closed from an interactive or piped
session, a checkpoint is written so the graph is not rebuilt on the next
open. A one-shot `-c` call does not write one.

To serve the shell's notes, quit the shell and start
`fenec-server --file notes.fenec`: one process holds a file at a
time. In the page the builder writes the search too:
`db.from('notes').near('embed', queryVector).limit(5).rows()`.
Every clause is on [FenecQL](https://fenecdb.com/docs/fenecql#the-vector-clause), and
words and meaning together on [Hybrid
search](https://fenecdb.com/docs/fenecql#hybrid).

## Bring your own data

`fenec import` copies a SQLite file or a PostgreSQL table into a
collection, its schema and indexes included.

```bash
./target/release/fenec import data.sqlite --table docs --into articles --index "category@hash"
```

Details

`fenec import` derives the schema from the table's columns and
builds the indexes you name once the rows are in, into
`articles.fenec` unless `--file` names another.
`--follow`
keeps a PostgreSQL table's copy in step, applying each change as it
commits.

```bash
./target/release/fenec import postgres://user@host:5432/database --table docs \
    --into articles --index "category@hash" --follow
```

`--dry-run` prints the derived schema and any warnings without
writing anything. A vector column comes across too, with
`--vector embed:384` for a SQLite blob and `--index
"embed@hnsw(cosine)"` ([Vector
columns](https://fenecdb.com/docs/import#vector-columns)). Full flag list in [Import](https://fenecdb.com/docs/import).

## Generate TypeScript from the file

Generate TypeScript types from the schema already in your file, so your
code and the database cannot drift apart.

```bash
./target/release/fenec types articles.fenec > web/fenec-schema.d.ts
```

Details

The schema is already in the database, so it is not written a second time in
TypeScript — it is generated, and the two cannot drift.

```ts
import { Fenec } from './fenec.js';
import type { FenecSchema } from './fenec-schema.js';

const db = await Fenec.open<FenecSchema>('./fenec.wasm');
const rows = await db.from('articles').select('title', 'category').rows();
//    rows: { title: string | null; category: string | null }[]
```

## Make targets worth knowing

The Makefile wraps the commands you will run most.

Details

| Command | What it does |

| `make test` | Rust suite, then the JS suite |

| `make wasm` | Build for wasm32 and copy to `web/fenec.wasm` |

| `make wasm FEATURES="text sorted"` | The same with those features alone ([A smaller module](https://fenecdb.com/docs/javascript#features)) |

| `make serve` | Build the wasm and serve the browser console |

| `make bench` | Scale measurement |

| `make memory` | Memory footprint, for calibrating `--max-memory` |

| `make sweep` | The ef/recall trade-off |

| `make compare` | Against SQLite and pgvector (run `make pgvector-up` first) |

| `make small` | The 1880 KB binary: `--profile cli --no-default-features` |


---

Source: https://fenecdb.com/docs/concepts

# How it works

A handful of design decisions carry the whole engine, and
this page gives what each one bought and what it cost.

## No page cache

fenecdb keeps no cache of its own between the disk and your queries. The
bytes on disk are already in the form a query reads, so there is nothing to
copy in, evict or warm up.

- **Writes only append.** Old data is never changed in place.

- **Reads decode in place.** A read takes just the field it needs.

- **`compact` reclaims space** from deleted and overwritten
    rows. A server and an app run it on their own, beside the queries, once
    half the file is dead.

Details

Classic databases cut the disk into pages and copy those pages into a cache
in user space. The eviction policy, dirty-page tracking, checkpoints and most
of the locking cost all follow from that one structure.

fenecdb skips the layer:

- Segments are **immutable**. A write only appends to the active one,
    which is sealed at 8 MiB.

- The byte sequence on disk and the byte sequence in memory are **the same
    format**, so there is no caching stage — a read decodes directly over the
    arena slice.

- The only helper structure is the `HashMap<DocId, Loc>`
    offset index.

- Projection and filters decode a single field; the fields in front of it
    are skipped without allocating.

No eviction policy, no dirty pages, no warm-up. Dead bytes are reclaimed by
`compact`, which `fenec-server` and the native library
run on their own ([Compaction](https://fenecdb.com/docs//docs/server#compaction)): left to
grow, a file of updated rows spreads its live records thin over pages of
dead ones, and reads wait for the disk.

## The file is mapped

The operating system maps the file into memory, so the documents stay in
the file and the process holds only the indexes built from them. A file
larger than the machine's memory can be opened this way.

One file of 1 GB, 2.3 million rows with a hash and an ordered index
(`make open-bench`):

|   | read into memory | mapped |

| open | 2.26 s, 1 095 MB held | **2.13 s, 188 MB** |

| `compact` | 1.52 s, peak 2 012 MB | **2.08 s, peak 236 MB** |

| held afterwards | 969 MB | **117 MB** |

- **Replace a file by rename, never by copying over it.** A copy over
    an open file, or a disk read that fails, ends the process.

- **Opening still takes time in proportion to the file.**

- **`fenec-server --no-mmap`** reads the file into memory
    instead, for network file systems.

Details

Because the format on disk is the format in memory, the file can be the
memory: `fs::open` maps it where the target maps files, and a
document is decoded from the pages the operating system brings in — and can
drop again. What the process holds is what it derived from the documents: the
offset index, the hash, ordered and text indexes, the graph, and the
documents written since the open until they amount to 16 MB.

A 10 GB file of 23 million rows opens on an 8 GB machine this way, which
reading it cannot: that would want about 20 GB. The trade is the one every
mapped file makes: a page the disk cannot read ends the process rather than
the query, and a file copied over in place while it is open does too —
replace one by rename, as fenecdb's own rewrites do. What it does not change is the
open's cost — the records are still walked to build the offset index and the
indexes are still built from the documents, so opening is proportional to the
file either way.

A rewrite writes the new file and points the stores at it, so the file just
written is the one being read and the old one is let go of. The rewrite
knows where it put each record, so the stores work their new places out
without reading the new file back: that read cost 1.8 to 2.3 s of a 1 GB
checkpoint, under the write lock, where working the places out takes 19 ms.
A `compact` over a mapped file therefore copies no record into
memory, and rebuilds no index but a graph holding tombstones: the documents
are the same ones, and every index is keyed by id. On a server it runs beside
the database: the new file is written with no lock held, and the writers wait
only while the documents written meanwhile are added to it — 24 ms at most
over a 1.1 GB file, where writing it under the lock held them 1.22 s.

What is written after the open reaches the file as each block lands — a
`/batch`, a statement — and the stores hold those
documents in memory only until they amount to 16 MB: then they read them
from the file, where the records put them, and let the memory go. A block
that has not landed yet spills into the file at 16 MB the same way, in a
record a load takes only once the block's land names it, so a
load of any size holds no more; one of 250 000 768-dim rows
peaked at 904 MB where it had at 2 389, of which 820 are its graph. Held until a restart before, 250 000 768-dim rows loaded into a
server were held twice, in the file and in memory: the engine counted 1 592
MB after the load, 820 now and 818 for the same file started again. Nothing
of the file is read or written for it — where each record went is noted as
it lands, and the file is mapped with room past its end, which the appends
grow into — and the pause is 1 ms for 16 MB of 768-dim documents, 2 ms of
128-dim ones. Whether the memory goes back to the system is the allocator's
call: Linux's give a large block back when it is freed, so the process holds
what the engine counts, while macOS's keeps hundreds of megabytes of them in
a cache of its own.

`fenec-server --no-mmap` reads the file into memory instead — a
replicated one, and every tenant of a `--dir` node, as well — over
a network file system, where mapping is a bad bet, or to have
`--max-memory` cover the data as well as the indexes.

## Vectors are first-class

Vectors are part of the language, not a plugin: `vector<N>`
is a type, `@hnsw` an index and `near` a clause.

```fenecql
get articles where year >= 2024 near embed $1 ef 200 limit 10
get articles near embed $1 exact limit 10        -- exact scan, for verification
```

Details

- HNSW is written in-house. Every vector lives in one contiguous arena.

- Under the cosine metric vectors are normalised inside the index, so the
    query reduces to a dot product.

- Neighbour selection uses the diversity heuristic (Malkov & Yashunin,
    Algorithm 4). Taking only the `m` nearest candidates trapped the
    graph in local clusters and lowered recall.

- The read-only part of the build — descent, beam search, neighbour
    selection — runs in parallel over a batch; only writing the links is
    serial. The result is independent of thread count and
    **deterministic**.

- The graph is written to the file, but only **the links, not the
    vectors**: the vectors are already in the document records, and the
    expensive part is the build.

## A filter and a vector together

A `near` with a `where` always fills its page with
rows that pass the filter, however the filter and the vectors relate. An index
on the filtered field makes a selective filter fast.

```fenecql
get articles where year >= 2020 near embed $1 limit 10
```

- **A filter matching many rows is cheap:** 1.1 ms for 23% of
    200 000 rows.

- **A filter matching few rows is read in full** — 17.9 ms for 1% —
    unless an index narrows it: 0.16 ms over `@sorted`.

- **`explain`** says which path ran.

Details

The filter's rows are found first, but only as far as the plan needs
them. An index hands them over directly — a `@hash` equality, a
`@sorted` range. Otherwise they are probed: rows are tested in
blocks spread across the collection until more of them match than an ANN
walk would measure anyway (`ef × m0`, 3 200 at the defaults), or
until none are left. Then one of two paths runs:

- if the set stayed smaller than that, it is searched directly — cheaper,
    and exact;

- otherwise the ANN runs and each candidate is tested against the filter.
    If the result lands under the limit, the probe carries on to the end and
    the whole set is searched exactly.

The probe changes when the rows are found, never the answer: it is the
one finding the whole set first gave, row for row. What it saves is the scan.
Over 200 000 × 128, `year >= 2020`, 23% of the rows, took
16.4 ms with the set found first, 0.14 ms of it the ANN, and takes 1.1 ms
probed. A filter under the budget is still read in full unless an index
narrows it: 17.9 ms for 1% of the rows, 0.16 ms over `@sorted`.
The blocks are spread out so that a filter for the newest rows, which sit at
the end of the id order, is found as fast as one for rows everywhere: 1.7 ms
for the newest 23%.

**The fallback is not an optimisation, it is required.** The filter
  is tested after the candidates are gathered. When the filter field
  correlates with the vector, every one of the `ef` nearest
  neighbours can be eliminated and the query would return empty while
  thousands of documents match.

Deleted documents do the same without a filter. Every write that changes
or deletes a document's vector leaves a tombstone in the graph until
`compact`, and a beam full of them holds fewer live rows than the
page. A short answer is walked again with the beam wider by the tombstones,
since no more than all of them can be in it — or, where a walk that wide
would cost more than reading every vector, searched exactly. Either way the
page is full, and `explain` says which ran.

## Half precision

Store vectors in 16-bit floats to halve their size on disk and in memory,
for a slightly slower search and almost no recall loss.

```fenecql
create collection docs (embed vector<768, f16> @hnsw(cosine))
```

Same data, 100 000 × 128:

|   | vector<128> | vector<128, f16> |

| vector arena | 51.2 MB | **25.6 MB** |

| file image | 59.3 MB | **33.7 MB** |

| write, index included | **3.5 s** | 4.2 s |

| ANN p50 | **0.070 ms** | 0.096 ms |

| recall@10 | 100% | 100% |

Details

Precision is part of the type, not of the index — the same choice pgvector
makes with `halfvec`. That shrinks the record and the arena
together; as an index option only the arena would shrink, and in the browser
the file image is in memory too.

The runtime representation does not change: values are `f32`
everywhere and conversion happens only at the storage boundary.

The cost is arithmetic: distance over an f16 arena unpacks inside the loop.
Two things were measured and fixed. The expansion has to be branchless — the
first version called the exact conversion in `codec`, whose
subnormal branch killed auto-vectorisation and made the build five times slower
(52 s); the branchless version is one multiply and is bit-identical over all
65 536 values. And the fixed side of the diversity loop is unpacked once
rather than per comparison: 22.6 s down to 12.4 s.

The win is at the memory boundary. One million 768-dimension vectors take
about 3 GB as `f32` and about 1.5 GB as `f16`. In
WASM32's 4 GB address space that is the difference between fitting and not.
Embedding model output sits comfortably inside the f16 range (±65 504, roughly
three decimal digits) and the measured recall loss is 0.3 points.

## Quantized indexes

A quantized index keeps a small code for each vector instead of the vector
itself: a byte a component with `quant=int8`, a bit a component
with `quant=bit`. The index shrinks a lot, and the scores stay
exact because the final order comes from the real vectors.

```fenecql
create collection docs (embed vector<768, f16> @hnsw(cosine, quant=int8))
```

- **`int8`** halves the index and keeps the recall of full
    vectors on every set measured.

- **`bit`** makes the index about a fifteenth the size, and
    is the fastest — but how well it works depends on your vectors, so measure
    it on them.

Details

Half precision shrinks the record and the arena together. A quantized
index leaves the record alone and shrinks the arena further, by holding codes
instead of vectors.

A code only estimates a distance, so it is used for what an estimate is
good for: walking the graph to the right neighbourhood. The candidates the
beam holds, `ef` of them, are then put in order by the documents'
own vectors, read out of the store as `rerank` reads them, and the
first `limit` are the answer. The scores are exact; the codes
decide only which candidates are found.

Not every candidate has to be read. An int8 code rounds each component to
a step of its scale, so it knows how far off its estimate can be: the
candidates are read nearest estimate first, and one that can no longer make
the page is left unread. A bit code keeps no step, and the whole beam is
read. A filtered set too small for the walk to pay — fewer rows than the
distances a walk would measure, `ef` × 2`m` — is
ranked by its codes and read the same way.

A bit code holds the signs of how far a vector lies from its centre, not
the signs of the vector. Vectors crowd around the few directions they are
about, so most of a vector's own signs are its neighbours' too, and the
signs could not tell the neighbours apart. The index learns 128 centres by
k-means from its first 2 048 vectors, which it holds whole until then — a
smaller index searches exactly — and keeps with each vector the centre it
is nearest and two numbers that scale its signs into an estimate. A search
takes its query's product with each centre once, and only the query's part
away from the centre goes through the signs. The query goes through them
cut to six bits a component, a bit plane each, so a code's signs meet it in
popcounts: 72 over 768 dimensions where a float was added a component,
which took a query at 100 000 × 768 from 0.51 ms to 0.29 at a beam of 200
with the same recall. The centres are kept in the file with the graph, so a
graph restored codes its vectors as they were coded.

One million vectors of 768 dimensions, stored as `f16`, in 64
clusters each spread along 32 directions — embeddings vary along far fewer
directions than they have — with a `@hash` field of 333 groups
for the filtered rows, on an 8 GB machine that was swapping, so the
latencies are high for all three (`make quant-bench`):

|   | vectors | quant=int8 | quant=bit |

| vector arena | 1 536 MB | 772 MB | **105 MB** |

| heap, the documents included | 3 397 MB | 2 596 MB | **1 897 MB** |

| index build | 1 017 s | **758 s** | 874 s |

| recall@10, beam of 100 | 97.4% | 97.1% | **97.9%** |

| recall@10, beam of 200 | 98.4% | 98.7% | **99.7%** |

| recall@10, beam of 400 | 98.6% | 98.8% | **99.9%** |

| ANN p50, beam of 100 | 1.30 ms | 0.62 ms | **0.41 ms** |

| ANN p50, beam of 200 | 2.08 ms | 1.09 ms | **0.66 ms** |

| ANN p50, beam of 400 | 3.38 ms | 1.66 ms | **1.25 ms** |

| documents read a query, beam of 100 / 400 | — | **20 / 20** | 100 / 400 |

| filtered to 3 003 rows, p50 | 2.26 ms | 1.36 ms | **0.97 ms** |

| filtered to 3 003 rows, documents read | — | **12** | 100 |

The bit column was measured again once the query met the codes in
popcounts, in a run of bit codes alone: in that run the query whole took
0.80, 1.30 and 2.16 ms at the three beams, and 1.36 ms filtered.

int8 codes follow the vectors closely enough for the default beam of 100,
and closely enough that 20 of its hundred candidates are read, with the
recall of reading them all. What is left unread is further off than six
times the spread of its rounding, which a candidate passes less than 1.5e-8
of the time. Bit codes, taken from the centres, find as many of the true ten
at the same beam, but bound nothing, so every candidate is read; a
`quant=bit` index searches 200 unless told otherwise. They are
for the arena, a seventh of int8's, and walk the graph fastest of the three.
The rest of the heap is the documents, whose vectors the answers are ordered
by.

**How well a bit code estimates depends on the vectors.** Spread in
  every dimension instead — as many degrees of freedom as dimensions, the
  hardest case for any graph — 100 000 such vectors gave full vectors 97.8%
  at a beam of 100, int8 codes 98.0%, and bit codes 89.7%, 97.5% at their
  default of 200 and 99.6% at 400.
  The vectors' own signs, which bit codes were until they were taken from
  centres, held 36% and 75% there: when a cluster is crowded around its
  centre, most of its members' signs are the centre's, and they could not
  tell the members apart. int8 codes held with full vectors on every set
  measured; measure bit codes on the vectors they are for.

The kernels over codes are scalar on every target, in the eight-strip
order of the vector kernels: natively LLVM vectorises them, and in the
browser they add in the same order, so a graph over codes is the same graph
in both. A bit code's popcounts are whole numbers, exact everywhere, and
the few floats they are scaled by are taken in one order.

## Sparse vectors

A sparse vector weighs the words a text is about, including related words
it never uses. `sparse<N>` stores one, and
`@inverted` indexes it so that `near` returns the exact
top results.

```fenecql
create collection passages (body text @text, splade sparse<30522> @inverted)
get passages near splade $1 limit 10
```

Encoded with SPLADE++ and scored on BEIR (`make beir`,
[Benchmarks](https://fenecdb.com/docs/benchmarks#retrieval-quality)):

| nDCG@10 | SciFact | FiQA |

| `match`, BM25 | 0.662 | 0.232 |

| `near`, dense (all-MiniLM-L6-v2) | 0.645 | 0.366 |

| `match … near … fuse`, dense | 0.700 | 0.366 |

| `near`, sparse (SPLADE++) | 0.693 | 0.331 |

| `match … near … fuse`, sparse | 0.698 | 0.314 |

Details

A learned sparse model weighs a text's vocabulary: SPLADE gives each of
BERT's 30 522 word pieces a weight, and all but a couple of hundred come out
zero. It learns which words a text is about, including ones it never uses —
an abstract on damage to cerebral white matter is weighted for
`disability` as well as the `disabilities` it says.
`sparse<N>` holds such a vector by its non-zero entries, and
`@inverted` indexes it the way `@text` indexes words: a
list per dimension of the documents with a weight there.

`near` over it scores by dot product and sums the query's lists
a list at a time — or walks them with MaxScore, the pruning `match`
uses, where they hold few documents in a wide span of ids — so what comes back
is the exact top ten rather than an estimate of it; only documents sharing a
dimension with the query are ranked.

On SciFact, where a claim shares its words with the evidence, SPLADE alone
scores what BM25 and the dense vectors do fused, with one index. On FiQA,
where a question shares few words with its answer, it lifts BM25 from 0.232
to 0.331 but stays under the dense vectors, and fusing it with BM25 there
costs it: the two are both lexical, and the weaker one drags. The index holds
968 928 postings for SciFact's 5 183 documents — 186.9 a document — in
13.1 MB, built in 0.07 s; FiQA's 57 638 hold 8.8 million in 107 MB, built
in 1.2 s. Like the text index it is derived from the documents, built by
the first query that reads it after an open, and never written to the
file.

**A SPLADE query reaches most of the corpus.** Its 37 to 65
  expanded terms touch 38 262 of FiQA's 57 638 documents. MaxScore, walking
  the lists a document at a time, gave up on 99.7% of them only after reading
  their share of the lists: 2.58 ms at the median. Summed a list at a time
  into a slot a document, with nothing chosen among the lists' heads, the same
  exact top ten comes in 0.68 ms, against 0.38 ms for a dense
  `near`, which walks a graph instead, and 0.25 ms for
  `match`; on SciFact 0.10 ms, where the walk took 0.60. Bounds
  per block of a list, to let the walk skip blocks, did not pay: the stretch
  every list could skip together was short.

## Bulk load first, index after

When you load a lot of data, create the vector index after the rows are
in. The load is then a plain append, and the graph is built in one parallel
pass.

```fenecql
create collection docs (title text, embed vector<768>)
put docs [ ... 100 000 documents ... ]
create index on docs (embed) @hnsw(cosine)
```

Details

Writing the index during the load makes every insert a graph mutation.
Building it afterwards turns the write path into a pure append and builds the
graph in one parallel pass.

This is already the recommended order for SQLite and PostgreSQL. In fenecdb
`create index` exists for exactly this reason, and the comparison in
[Benchmarks](https://fenecdb.com/docs/benchmarks) uses that order for all three engines.

## Zero dependencies

The core is written against Rust's standard library alone, with no
outside packages. That keeps the WebAssembly module small and every line of
it auditable.

Details

The core uses `std` and nothing else: its own binary codec, its
own JSON, its own HNSW, its own lexer and parser, its own SCRAM, even its own
decimal-to-`f64` — `str::parse` reaches for a 12 KB table
of powers of five, which is a twelfth of the module and incompressible. The
reason is
the WebAssembly output — it has to stay small and auditable.

The rule covers `fenec-core`, `fenec-ql`,
`fenec-wasm`, `fenec-http`, `fenec-server`,
`fenec-wire`, `fenec-import` and
`fenec-shard`. `fenec-core` does dev-depend on
`fenec-ql`, which Cargo permits through a dev dependency, so tests
can be written as real queries.

**What it costs.** Every bug is ours, and so is every protocol detail:
the HTTP server, the JSON reader, the JWT checks and the PostgreSQL client the
importer reads with are all hand-written.

## Straight into the browser

The same engine runs in the page as a WebAssembly module, loaded by one
dependency-free JavaScript file. A database can be kept in IndexedDB or in a
file a server can open.

- **`persist` / `restore`** keep it in
    IndexedDB, saving only what changed.

- **`openFile`**, in a worker, keeps it as a real fenecdb
    file in the origin private file system.

Details

`fenec-wasm` exposes a C ABI and there is no
`wasm-bindgen`. The glue is about 605 lines of JavaScript inside
`web/fenec.js`, alongside persistence and the sync layer; the
query builder and the HTTP client are dependency-free modules of their own
it imports, which `@fenecdb/web/client` loads without the rest
([Which package](https://fenecdb.com/docs/javascript#packages)).

`persist(db)` and `restore(db)` move the database in
and out of IndexedDB the way a file holds it: an image, and the writes since
as chunks after it. The first `persist` stores the image; every
later one stores only what was written since the last, from a journal the
wasm side keeps once asked to. In Chrome 153, over a 32 MB database,
persisting one new row takes 0.18 ms where the image took 92 ms; in Safari
26.1, 0.98 ms and 418 ms. Once the chunks outgrow half the image, a new
image replaces them, and `compact` replaces them too.
`restore` reads the image and its chunks back as one file, so the
graph restored from the image takes the chunks' writes rather than being
rebuilt.

In a worker, `openFile(db, name)` keeps it in a file of the
origin private file system instead, which is the file itself: the bytes
`fenec-server` keeps, so `fenec` opens a page's file and a
page loads a server's. The same journal feeds it, a statement at a time:
`run` appends the writes and flushes the file before it answers,
0.56 ms a row in Chrome and 0.36 ms in Safari, statement included, and the
file opens again in half the time the chunks are restored in. A new image
is written beside the file and flushed before it is written over it, so a
crash leaves one of the two whole.

**Threads are compiled out of wasm32.** The parallel HNSW build path
  must not enter that target. `now()` errors there too —
  wasm32-unknown-unknown has no clock, so time is passed in as a parameter.

## Single writer

Many readers can work at once, but only one write runs at a time. Every
write lands whole or not at all, which is what lets fenecdb do without the
bookkeeping other databases need for concurrent writers.

- **A statement or a batch** lands as one unit: a crash
    keeps all of it or none.

- **Keep batches short:** other writes wait while one runs.

- **A long read does not hold the writer:** an aggregate, a scan or
    a read-only batch reads what it pinned and lets writes land beside
    it.

**Two processes opening the same file corrupts it.** This is why the
  HTTP endpoint is a second listener inside `fenec-server` and never its
  own binary — one process, one file, one sync policy, one shutdown signal.

Details

Reads take a shared lock (`Database::query`), writes take the
exclusive one (`execute_with`). A `/batch` is a block:
it takes the exclusive lock and holds it until it lands or is put back. One
writer is what makes that isolation: nothing reads a block's writes before
they land, because nothing else runs. There are no interactive transactions
held open across requests: one writer would wait on a client's think time.

A long read holds the shared lock only as long as it takes to pin what
it reads: on a server, an aggregate, a facet, an `order`, a
filtered `count` or a read with no `limit` over a
collection of 10 000 rows or more, and a read-only `/batch`
holding one (`Database::pin` embedded). Under the lock it copies each
collection's id index, 12 bytes a row, and shares its stored documents;
the read then runs with no lock held, on the database as it stood at the
pin, while writes land beside it. A read an index answers — an equality on
a `@hash` field, a range on a `@sorted` one,
`match`, `near`, `lookup` — is short, and
runs under the lock as before: indexes change in place with every write,
and copying one would cost as much as the read.

Every write is a block. What a statement writes — every document of a
`put` of many, of a `set` or a `del` matching
many — is held back from the file and the change feed until it ends, and lands
as one record: a crash cuts all of it off or none, a replica applies it at
once, and a subscriber is woken once for all of it. A statement that fails
half way puts back what it did: each store is cut back to where it stood, and
each document it wrote goes out of the indexes, the version before it back in.
A `/batch` and the browser module's `run` of several
statements are one block the same way. A
`create`, a `drop` and a `create index` are
put back with the rest — the collection made goes, the one dropped comes
back, the index goes — and only a `compact`, which rewrites the
file, runs on its own. Held back, a write costs within 2% of what it did: a lone
`put` 841 ns in memory against 829 before blocks.

That is what removes MVCC row headers, the WAL and the visibility map —
about 2.8 MB of headers over 100 000 rows. Most of the 2.4× between this file
and the same data under pgvector is elsewhere: pgvector's HNSW index keeps its
own copy of every vector, while this graph stores only the links
([Benchmarks](https://fenecdb.com/docs/benchmarks)).

Concurrency still exists on the read side. HNSW search buffers are kept
thread-locally so the same index can be searched in parallel; eight concurrent
full scans on eight cores measured a 4.3× speed-up.

## The replica is an endpoint, not a library

A page can keep a local copy of part of the server's data and query it with
no network round trip. The server says what changed, and the copy follows.

```ts
const db = await sync({ url, shapes: [{ collection: 'tasks', key: 'key' }] });
await db.from('tasks').where('priority', '>=', 3).rows();   // local, no network
```

Details

Because the same engine runs in the browser and on the server, the
local-copy architecture needed no new layer — only two things: the server being
able to say "these changed", and the client being able to choose where to bind
a query. `bind()` already existed, and the first was sitting in the
file format, because the file is already a change log.

It is TanStack DB's model with two differences. There is **no incremental
dataflow**: there a collection is a `Map`, and re-running a filter
on every keystroke over a large set is unacceptable — here the local side is
indexed and the query is already under a millisecond. And **a shape is
mandatory**: the whole database is in memory, so the client cannot pull all
of it. Details in [Sync](https://fenecdb.com/docs/sync).


---

Source: https://fenecdb.com/docs/languages

# Your language

fenecdb speaks HTTP and JSON, so every language reaches it
with the HTTP client it already has. No driver to install.

Every example on this page is a program that runs against a real server
on each change to fenecdb (`make languages-test`). Python,
JavaScript, Go and .NET have official clients, each its standard library
alone; in the other languages these few lines are the whole client. Swift,
Kotlin and Dart embed the database in the app, a file on the device
([Mobile and native apps](https://fenecdb.com/docs/mobile)), and reach a server with
`Fenec.connect`; their tabs run the same flow in their own
tests. Each tab ends with a full example: one app, Notes, in every
language, a folder of a few files to clone and run
([examples](https://github.com/fenecdb/fenec/tree/main/examples)),
and beside them a whole shop in Next.js
([shop](https://github.com/fenecdb/fenec/tree/main/examples/shop)),
a double-entry payments ledger on Node
([ledger](https://github.com/fenecdb/fenec/tree/main/examples/ledger))
and courier dispatch across tenants
([dispatch](https://github.com/fenecdb/fenec/tree/main/examples/dispatch)).

## Start a server

```bash
fenec-server --file app.fenec --http 127.0.0.1:8080 --http-token secret
```

Then pick your language. Each example makes a collection of a text and
a number, each indexed, writes two rows, reads them back by a filter and an
order, searches the words with a typo in them and handles a refusal. Each
sends one FenecQL statement a request to `POST /query`, its
values as `$1`, `$2`, ... in `params`, and
the token as `Authorization: Bearer`.

## Connect and search

pip install fenecdb   # the standard library alone

from fenecdb import Client, FenecError

db = Client("http://127.0.0.1:8080", token="secret")

db.query("create collection if not exists docs "
         "(title text @text, year int @sorted)")
db.query("put docs {title: $1, year: $2}", ["Dunes", 2021])
db.query("put docs {title: $1, year: $2}", ["Night at the oasis", 2024])

rows = db.query("get docs select title, year where year >= $1 order year desc limit $2", [2020, 5])
# [{"title": "Night at the oasis", "year": 2024}, {"title": "Dunes", "year": 2021}]

hits = db.query("get docs select title match title $1 fuzzy", ["oasys"])
# [{"title": "Night at the oasis", "_score": 0.62...}]

try:
    db.query("get nowhere")
except FenecError as e:
    print(e.status, e)   # 404 collection `nowhere`

Or let the [query builder](#builder) write the statement,
every value a parameter, every name checked: the read's text to the byte,
and the search as `match title $1 fuzzy 2`, two edits at most,
which is what a plain `fuzzy` allows a word anyway, so the rows
are the same.

docs = db.collection("docs")
docs.insert({"title": "Night at the oasis", "year": 2024})

rows = docs.select("title", "year").where("year", ">=", 2020).order("year", "desc").limit(5).rows()
hits = docs.select("title").match("title", "oasys", fuzzy=True).rows()
n = docs.where("title", "~", "Dunes").count()

`batch([(text, params), ...])` runs statements as one block,
all or none -- a `FenecError` that stops it says which
(`e.at`, from 0); `idempotency_key=` on
`query` and `batch`, or
`with_idempotency_key(key)` for every write of a copy, makes a
write once however often it is sent (`db.replayed`), and
`db.seq` is the change the last write left the database at.
`AsyncClient` does the same in an event loop, the endpoints of
its `collection(...)` awaited; LangChain and LlamaIndex have
stores of their own ([Integrations](https://fenecdb.com/docs/integrations)).

[Full example](https://github.com/fenecdb/fenec/tree/main/examples/python): Notes, a CLI over the server with the builder, `match`, `fuse` and the change stream.

async function query(q: string, params: unknown[] = []) {
  const res = await fetch('http://127.0.0.1:8080/query', {
    method: 'POST',
    headers: { 'content-type': 'application/json', authorization: 'Bearer secret' },
    body: JSON.stringify({ query: q, params }),
  });
  const body = await res.json();
  if (!res.ok) throw Object.assign(new Error(body.error), { status: res.status });
  return body;
}

await query('create collection if not exists docs (title text @text, year int @sorted)');
await query('put docs {title: $1, year: $2}', ['Dunes', 2021]);
await query('put docs {title: $1, year: $2}', ['Night at the oasis', 2024]);

const rows = await query('get docs select title, year where year >= $1 order year desc limit $2', [2020, 5]);
const hits = await query('get docs select title match title $1 fuzzy', ['oasys']);

Or `npm install @fenecdb/web`: its `connect` sends
the [query builder](#builder)'s statements to a server, the same
code that runs in the page.

import { connect } from '@fenecdb/web';
import type { FenecSchema } from './fenec-schema.js';   // fenec types app.fenec

const db = connect<FenecSchema>('http://127.0.0.1:8080', { token: 'secret' });
await db.from('docs').insert({ title: 'Night at the oasis', year: 2024 });
const rows = await db.from('docs').select('title', 'year').where('year', '>=', 2020).order('year', 'desc').limit(5).rows();
//    rows: { title: string | null; year: number | null }[]
const hits = await db.from('docs').select('title').match('title', 'oasys', { fuzzy: true }).rows();
//    hits: { title: string | null; _score: number }[]

The same code runs as plain JavaScript, without the types. In the
browser there is no server at all: the same database runs in the page
([JavaScript and TypeScript](https://fenecdb.com/docs/javascript)).

[Full example](https://github.com/fenecdb/fenec/tree/main/examples/node-server): Notes from Node through `@fenecdb/web/client`, its live queries run on the server; in the page, [web-local](https://github.com/fenecdb/fenec/tree/main/examples/web-local) and [react](https://github.com/fenecdb/fenec/tree/main/examples/react).

go get github.com/fenecdb/fenec/integrations/go   # the standard library alone

import fenecdb "github.com/fenecdb/fenec/integrations/go"

db := fenecdb.New("http://127.0.0.1:8080", fenecdb.WithToken("secret"))

db.Exec(ctx, "create collection if not exists docs (title text @text, year int @sorted)")
db.Exec(ctx, "put docs {title: $1, year: $2}", "Dunes", 2021)
db.Exec(ctx, "put docs {title: $1, year: $2}", "Night at the oasis", 2024)

type Doc struct {
	Title string `json:"title"`
	Year  int    `json:"year"`
}
rows, err := fenecdb.QueryAs[Doc](ctx, db, "get docs select title, year where year >= $1 order year desc limit $2", 2020, 5)
hits, err := db.Query(ctx, "get docs select title match title $1 fuzzy", "oasys")

var e *fenecdb.Error
if _, err := db.Query(ctx, "get nowhere"); errors.As(err, &e) {
	fmt.Println(e.Status, e.Code, e.Message) // 404 not_found collection `nowhere`
}

Or let the [query builder](#builder) write the statement,
immutable, each step a copy:

docs := db.From("docs")
docs.Insert(ctx, fenecdb.D("title", "Night at the oasis", "year", 2024))

query := docs.Select("title", "year").Where("year", ">=", 2020).Order("year", "desc").Limit(5)
rows, err := fenecdb.RowsAs[Doc](ctx, query)
hits, err := docs.Select("title").Match("title", "oasys", fenecdb.Fuzzy(2)).Rows(ctx)
n, err := docs.Where("title", "~", "Dunes").Count(ctx)

`Query` gives rows as `[]map[string]any`;
`Batch`, `IdempotencyKey`, `After` for a
replica, `Subscribe`, `Changes` and
`WithTenant` cover the rest of the [HTTP
endpoint](https://fenecdb.com/docs/http) ([integrations/go](https://github.com/fenecdb/fenec/tree/main/integrations/go));
a refusal is an `*fenecdb.Error`, and a batch's says which
statement stopped it (`At`, from 0).
A `[]float32` goes out as the decimals that read back as each
`f32`, so a vector round-trips to the bit.

Without the SDK, `net/http` and `encoding/json`:

// query runs one FenecQL statement and decodes its answer into out.
func query(out any, q string, params ...any) error {
	body, _ := json.Marshal(map[string]any{"query": q, "params": params})
	req, _ := http.NewRequest("POST", "http://127.0.0.1:8080/query", bytes.NewReader(body))
	req.Header.Set("Content-Type", "application/json")
	req.Header.Set("Authorization", "Bearer secret")
	res, err := http.DefaultClient.Do(req)
	if err != nil {
		return err
	}
	defer res.Body.Close()
	if res.StatusCode >= 300 {
		e := &FenecError{Status: res.StatusCode}
		json.NewDecoder(res.Body).Decode(e)
		return e
	}
	if out == nil {
		return nil
	}
	return json.NewDecoder(res.Body).Decode(out)
}

query(nil, "create collection if not exists docs (title text @text, year int @sorted)")
query(nil, "put docs {title: $1, year: $2}", "Dunes", 2021)
query(nil, "put docs {title: $1, year: $2}", "Night at the oasis", 2024)

var rows []struct {
	Title string `json:"title"`
	Year  int    `json:"year"`
}
err := query(&rows, "get docs select title, year where year >= $1 order year desc limit $2", 2020, 5)

`FenecError` is a struct of the status and the body's
`error`.

[Full example](https://github.com/fenecdb/fenec/tree/main/examples/go): Notes, a CLI over the server with the builder, typed rows and a subscription.

cargo add ureq --features json
cargo add serde_json

use serde_json::{json, Value};

fn query(q: &str, params: Value) -> Result<Value, ureq::Error> {
    Ok(ureq::post("http://127.0.0.1:8080/query")
        .set("Authorization", "Bearer secret")
        .send_json(json!({"query": q, "params": params}))?
        .into_json()?)
}

query("create collection if not exists docs (title text @text, year int @sorted)", json!([]))?;
query("put docs {title: $1, year: $2}", json!(["Dunes", 2021]))?;
query("put docs {title: $1, year: $2}", json!(["Night at the oasis", 2024]))?;

let rows = query("get docs select title, year where year >= $1 order year desc limit $2", json!([2020, 5]))?;
let hits = query("get docs select title match title $1 fuzzy", json!(["oasys"]))?;

match query("get nowhere", json!([])) {
    Err(ureq::Error::Status(404, res)) => { /* res.into_json()? holds {"error": ...} */ }
    other => { /* ... */ }
}

Or embed the engine in your process, with no server
([Embedded Rust](https://fenecdb.com/docs/embedding)).

[Full example](https://github.com/fenecdb/fenec/tree/main/examples/rust): Notes with `fenec-core` embedded, a file and no server.

dotnet add package FenecDb   # HttpClient and System.Text.Json alone

using FenecDb;

using var db = new FenecClient("http://127.0.0.1:8080", new() { Token = "secret" });

await db.ExecAsync("create collection if not exists docs (title text @text, year int @sorted)");
await db.ExecAsync("put docs {title: $1, year: $2}", ["Dunes", 2021]);
await db.ExecAsync("put docs {title: $1, year: $2}", ["Night at the oasis", 2024]);

var rows = await db.QueryAsync<Doc>("get docs select title, year where year >= $1 order year desc limit $2", [2020, 5]);
var hits = await db.QueryAsync<Hit>("get docs select title match title $1 fuzzy", ["oasys"]);

try { await db.QueryAsync("get nowhere"); }
catch (FenecException e) { Console.WriteLine($"{e.Status} {e.Code} {e.Message}"); } // 404 not_found collection `nowhere`

record Doc(string Title, int Year);
record Hit(string Title, [property: JsonPropertyName("_score")] float Score);

Or let the [query builder](#builder) write the statement,
immutable, each call a new query:

var docs = db.From("docs");
await docs.InsertAsync(new { title = "Night at the oasis", year = 2024 });

var query = docs.Select("title", "year").Where("year", ">=", 2020).Order("year", "desc").Limit(5);
var rows = await query.RowsAsync<Doc>();
var hits = await docs.Select("title").Match("title", "oasys", fuzzy: 2).RowsAsync<Hit>();
var n = await docs.Where("title", "~", "Dunes").CountAsync();

.NET 8 and newer, async throughout with a `CancellationToken`
on every call. `QueryAsync<T>` maps rows to records or
classes by property name, any case; `BatchAsync`,
`WithIdempotencyKey`, `After` for a replica,
`SubscribeAsync` as an `IAsyncEnumerable`,
`ChangesAsync` and a `Tenant` option cover the rest
of the [HTTP endpoint](https://fenecdb.com/docs/http)
([integrations/dotnet](https://github.com/fenecdb/fenec/tree/main/integrations/dotnet));
a refusal is a `FenecException`, and a batch's says which
statement stopped it (`At`, from 0).
A `float[]` or `ReadOnlyMemory<float>` goes out
as the decimals that read back as each float.

Without the SDK, `HttpClient` and `System.Text.Json`:

var http = new HttpClient { BaseAddress = new Uri("http://127.0.0.1:8080") };
http.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", "secret");

async Task<JsonElement> Query(string query, params object[] args)
{
    var json = JsonSerializer.Serialize(new { query, @params = args });
    var res = await http.PostAsync("/query", new StringContent(json, Encoding.UTF8, "application/json"));
    var body = await res.Content.ReadFromJsonAsync<JsonElement>();
    if (!res.IsSuccessStatusCode)
        throw new HttpRequestException(body.GetProperty("error").GetString(), null, res.StatusCode);
    return body;
}

await Query("create collection if not exists docs (title text @text, year int @sorted)");
await Query("put docs {title: $1, year: $2}", "Dunes", 2021);
await Query("put docs {title: $1, year: $2}", "Night at the oasis", 2024);

var rows = await Query("get docs select title, year where year >= $1 order year desc limit $2", 2020, 5);
var titles = rows.EnumerateArray().Select(r => r.GetProperty("title").GetString());

The body is serialised whole: `PostAsJsonAsync` streams it chunked,
and the server wants a `Content-Length`.

[Full example](https://github.com/fenecdb/fenec/tree/main/examples/dotnet): Notes, a console app over the server with the builder, records and `SubscribeAsync`.

static final HttpClient HTTP = HttpClient.newHttpClient();

static HttpResponse<String> query(String json) throws Exception {
    var req = HttpRequest.newBuilder(URI.create("http://127.0.0.1:8080/query"))
        .header("Content-Type", "application/json")
        .header("Authorization", "Bearer secret")
        .POST(HttpRequest.BodyPublishers.ofString(json))
        .build();
    return HTTP.send(req, HttpResponse.BodyHandlers.ofString());
}

query("""
    {"query": "create collection if not exists docs (title text @text, year int @sorted)"}""");
query("""
    {"query": "put docs {title: $1, year: $2}", "params": ["Dunes", 2021]}""");
query("""
    {"query": "put docs {title: $1, year: $2}", "params": ["Night at the oasis", 2024]}""");

String rows = query("""
    {"query": "get docs select title, year where year >= $1 order year desc limit $2", "params": [2020, 5]}""").body();
String hits = query("""
    {"query": "get docs select title match title $1 fuzzy", "params": ["oasys"]}""").body();

`java.net.http` alone, Java 11 and newer (text blocks need 15),
run as `java Search.java`. The standard library has no JSON: the
test compares the answer's text and reads the titles with a pattern, and a
project reads the rows with Jackson or Gson.

function query(string $q, array $params = []) {
    $ch = curl_init('http://127.0.0.1:8080/query');
    curl_setopt_array($ch, [
        CURLOPT_POST => true,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_HTTPHEADER => ['Content-Type: application/json', 'Authorization: Bearer secret'],
        CURLOPT_POSTFIELDS => json_encode(['query' => $q, 'params' => $params]),
    ]);
    $body = json_decode(curl_exec($ch), true);
    $status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
    if ($status >= 300) {
        throw new RuntimeException($body['error'], $status);
    }
    return $body;
}

query('create collection if not exists docs (title text @text, year int @sorted)');
query('put docs {title: $1, year: $2}', ['Dunes', 2021]);
query('put docs {title: $1, year: $2}', ['Night at the oasis', 2024]);

$rows = query('get docs select title, year where year >= $1 order year desc limit $2', [2020, 5]);
$hits = query('get docs select title match title $1 fuzzy', ['oasys']);

The curl extension and `json_encode`; Guzzle sends the same
request.

require "json"
require "net/http"

URL = URI("http://127.0.0.1:8080")

def query(q, params = [])
  req = Net::HTTP::Post.new(URI.join(URL, "/query"), "Content-Type" => "application/json")
  req["Authorization"] = "Bearer secret"
  req.body = JSON.generate(query: q, params: params)
  res = Net::HTTP.start(URL.host, URL.port) { |http| http.request(req) }
  body = JSON.parse(res.body)
  raise "#{res.code}: #{body["error"]}" unless res.is_a?(Net::HTTPSuccess)
  body
end

query("create collection if not exists docs (title text @text, year int @sorted)")
query("put docs {title: $1, year: $2}", ["Dunes", 2021])
query("put docs {title: $1, year: $2}", ["Night at the oasis", 2024])

rows = query("get docs select title, year where year >= $1 order year desc limit $2", [2020, 5])
hits = query("get docs select title match title $1 fuzzy", ["oasys"])

.package(url: "https://github.com/fenecdb/fenec", from: "0.1.7")   // product "FenecDB"

import FenecDB

// Embedded: the database is a file the app holds, no server.
let db = try await Fenec.open(path: path)

try await db.execute("create collection if not exists docs (title text @text, year int @sorted)")
try await db.execute("put docs {title: $1, year: $2}", "Dunes", 2021)
try await db.execute("put docs {title: $1, year: $2}", "Night at the oasis", 2024)

let rows = try await db.query("get docs select title, year where year >= $1 order year desc limit $2", 2020, 5)
// rows[0]["title"] == "Night at the oasis", rows[0]["year"] == 2024
let hits = try await db.query("get docs select title match title $1 fuzzy", "oasys")
// hits[0]["title"] == "Night at the oasis", hits[0]["_score"] its score

do {
    _ = try await db.query("get nowhere")
} catch let e as FenecError {
    print(e.code, e)   // notFound: collection `nowhere` ...
}

let docs = try db.from("docs")
try await docs.insert(["title": "Night at the oasis", "year": 2024] as Value)
let recent = try await docs.select("title", "year").where("year", ">=", 2020).order("year", "desc").limit(5).rows()
let found = try await docs.select("title").match("title", "oasys", fuzzy: 2).rows()
let n = try await docs.where("title", "~", "Dunes").count()

// No file: every query goes to the server, through the same builder.
let remote = Fenec.connect(url: "http://127.0.0.1:8080", token: "secret")
try await remote.execute("put docs {title: $1, year: $2}", "Dunes", 2021)
let recent = try await remote.from("docs").select("title", "year").where("year", ">=", 2020).order("year", "desc").limit(5).rows()

Rows decode as any `Decodable`, and a `[Float]`
goes to the library as its bytes. A live query for SwiftUI, where the file goes
and a replica a server keeps in step are on
[Mobile and native apps](https://fenecdb.com/docs/mobile). A device reaches a
server through TLS: `URLSession` makes the requests; over it,
`remote.batch([...], idempotencyKey:)` lands statements whole
and `withIdempotencyKey` makes a write once
([Without a file](https://fenecdb.com/docs/mobile#connect)).

[Full example](https://github.com/fenecdb/fenec/tree/main/examples/swift): Notes, a SwiftUI app over a file, its list a `LiveQuery`, and a CLI.

implementation("com.fenecdb:fenecdb-android:0.1.7")   // the AAR; the JVM: com.fenecdb:fenecdb

import com.fenecdb.*

// Embedded: the database is a file the app holds, no server.
val db = Fenec.openAsync(path)

db.execute("create collection if not exists docs (title text @text, year int @sorted)")
db.execute("put docs {title: \$1, year: \$2}", "Dunes", 2021)
db.execute("put docs {title: \$1, year: \$2}", "Night at the oasis", 2024)

val rows = db.query("get docs select title, year where year >= \$1 order year desc limit \$2", 2020, 5)
// rows[0].string("title") == "Night at the oasis", rows[0].int("year") == 2024
val hits = db.query("get docs select title match title \$1 fuzzy", "oasys")
// hits[0].string("title") == "Night at the oasis", hits[0].double("_score") its score

try {
    db.query("get nowhere")
} catch (e: FenecException) {
    println("${e.code} ${e.message}")   // NOT_FOUND collection `nowhere` ...
}

val docs = db.from("docs")
docs.insert(mapOf("title" to "Night at the oasis", "year" to 2024))
val recent = docs.select("title", "year").where("year", ">=", 2020).order("year", "desc").limit(5).rows()
val found = docs.select("title").match("title", "oasys", fuzzy = 2).rows()
val n = docs.where("title", "~", "Dunes").count()

// No file: every query goes to the server, through the same builder.
val remote = Fenec.connect("http://127.0.0.1:8080", token = "secret")
remote.execute("put docs {title: \$1, year: \$2}", "Dunes", 2021)
val recent = remote.from("docs").select("title", "year").where("year", ">=", 2020).order("year", "desc").limit(5).rows()

The calls suspend, on `Dispatchers.IO`; Java calls the
`...Blocking` ones. Live queries are `Flow`s, and a
replica a server keeps in step is `Fenec.sync`
([Mobile and native apps](https://fenecdb.com/docs/mobile)). A device reaches a
server through TLS: `HttpURLConnection` makes the requests; over
it, `remote.batch(listOf(...), idempotencyKey = id)` lands
statements whole and `withIdempotencyKey` makes a write once
([Without a file](https://fenecdb.com/docs/mobile#connect)).

[Full example](https://github.com/fenecdb/fenec/tree/main/examples/kotlin-android): Notes, a Compose app over a file, its list a live `Flow`, and a JVM CLI.

flutter pub add fenecdb_flutter   # Dart alone: dart pub add fenecdb

import 'package:fenecdb/fenecdb.dart';

// Embedded: the database is a file the app holds, no server.
final db = await Fenec.open(path);

await db.execute('create collection if not exists docs (title text @text, year int @sorted)');
await db.execute(r'put docs {title: $1, year: $2}', ['Dunes', 2021]);
await db.execute(r'put docs {title: $1, year: $2}', ['Night at the oasis', 2024]);

final rows = await db.query(r'get docs select title, year where year >= $1 order year desc limit $2', [2020, 5]);
// rows[0]['title'] == 'Night at the oasis', rows[0]['year'] == 2024
final hits = await db.query(r'get docs select title match title $1 fuzzy', ['oasys']);
// hits[0]['title'] == 'Night at the oasis', hits[0]['_score'] its score

try {
  await db.query('get nowhere');
} on FenecException catch (e) {
  print('${e.code.name} ${e.message}');   // notFound collection `nowhere` ...
}

final docs = db.from('docs');
await docs.insert({'title': 'Night at the oasis', 'year': 2024});
final recent = await docs.select(['title', 'year']).where('year', '>=', 2020).order('year', 'desc').limit(5).rows();
final found = await docs.select(['title']).match('title', 'oasys', fuzzy: true).rows();
final n = await docs.where('title', '~', 'Dunes').count();

// No file: every query goes to the server, through the same builder.
final remote = Fenec.connect('http://127.0.0.1:8080', token: 'secret');
await remote.execute(r'put docs {title: $1, year: $2}', ['Dunes', 2021]);
final recent = await remote.from('docs').select(['title', 'year']).where('year', '>=', 2020).order('year', 'desc').limit(5).rows();

Every call runs on an isolate the database keeps, so none blocks the UI.
Live queries are `Stream`s, and a replica a server keeps in step
is `Fenec.openSynced` ([Mobile and native
apps](https://fenecdb.com/docs/mobile)). A device reaches a server through TLS: `dart:io`'s
`HttpClient` makes the requests; over it,
`remote.batch([...], idempotencyKey: id)` lands statements whole
and `withIdempotencyKey` makes a write once
([Without a file](https://fenecdb.com/docs/mobile#connect)).

[Full example](https://github.com/fenecdb/fenec/tree/main/examples/flutter): Notes, a Flutter app over a file, its list a `StreamBuilder` over a live query.

## The builder

Python, JavaScript, Go, .NET, Swift, Kotlin and Dart each have a query
builder: a chain of calls that writes the FenecQL statement, every value a
parameter and every name checked against FenecQL's own rule, which is where
an injection would have to get through. All seven build the same text from
the same chain, to the byte, and refuse the same mistakes with the same
message. Python's builder starts with `collection`, since
`from` is a keyword there.

| Python | JavaScript | Go | .NET |

| `db.collection("docs")` | `db.from('docs')` | `db.From("docs")` | `db.From("docs")` |

| `.where("year", ">=", 2024)` | `.where('year', '>=', 2024)` | `.Where("year", ">=", 2024)` | `.Where("year", ">=", 2024)` |

| `.where(or_({"lang": "tr"}, {"tags": {"has": "rust"}}))` | `.where(or({ lang: 'tr' }, { tags: { has: 'rust' } }))` | `.WhereCond(fenecdb.Or(fenecdb.Fields("lang", "tr"), fenecdb.Fields("tags", fenecdb.Ops("has", "rust"))))` | `.Where(Cond.Or(Cond.Cmp("lang", "=", "tr"), Cond.Cmp("tags", "has", "rust")))` |

| `.match("title", q, fuzzy=True)` | `.match('title', q, { fuzzy: true })` | `.Match("title", q, fenecdb.Fuzzy(2))` | `.Match("title", q, fuzzy: 2)` |

| `.near("embed", v, ef=64)` | `.near('embed', v, { ef: 64 })` | `.Near("embed", v, fenecdb.Ef(64))` | `.Near("embed", v, ef: 64)` |

| `.lookup("reviews", on="product_id", limit=3)` | `.lookup('reviews', { on: 'product_id', limit: 3 })` | `.Lookup("reviews", fenecdb.On("product_id"), fenecdb.Limit(3))` | `.Lookup("reviews", on: "product_id", limit: 3)` |

| `.order("title", "desc", collate="tr")` | `.order('title', 'desc', { collate: 'tr' })` | `.Order("title", "desc", fenecdb.Collate("tr"))` | `.Order("title", "desc", collate: "tr")` |

| `.rows()`, `.first()`, `.count()` | `.rows()`, `.first()`, `.count()` | `.Rows(ctx)`, `.First(ctx)`, `.Count(ctx)`, `RowsAs[T]` | `.RowsAsync()`, `.FirstAsync()`, `.CountAsync()`, `.RowsAsync<T>()` |

| `.insert(doc)`, `.update(patch)`, `.delete()` | `.insert(doc)`, `.update(patch)`, `.delete()` | `.Insert(ctx, doc)`, `.Update(ctx, patch)`, `.Delete(ctx)` | `.InsertAsync(doc)`, `.UpdateAsync(patch)`, `.DeleteAsync()` |

| `.update({"n": inc(1)})`, `.insert(doc, if_absent=True)` | `.update({ n: inc(1) })`, `.insert(doc, { ifAbsent: true })` | `.Update(ctx, fenecdb.D("n", fenecdb.Inc(1)))`, `.Insert(ctx, doc, fenecdb.IfAbsent())` | `.UpdateAsync(new { n = Computed.Inc(1) })`, `.InsertAsync(doc, ifAbsent: true)` |

| Swift | Kotlin | Dart |

| `try db.from("docs")` | `db.from("docs")` | `db.from('docs')` |

| `.where("year", ">=", 2024)` | `.where("year", ">=", 2024)` | `.where('year', '>=', 2024)` |

| `.where(.or(["lang": "tr"], ["tags": ["has": "rust"]]))` | `.where(Cond.or(mapOf("lang" to "tr"), mapOf("tags" to mapOf("has" to "rust"))))` | `.where(Cond.or([{'lang': 'tr'}, {'tags': {'has': 'rust'}}]))` |

| `.match("title", q, fuzzy: 2)` | `.match("title", q, fuzzy = 2)` | `.match('title', q, fuzzy: true)` |

| `.near("embed", v, ef: 64)` | `.near("embed", v, ef = 64)` | `.near('embed', v, ef: 64)` |

| `.lookup("reviews", on: "product_id", limit: 3)` | `.lookup("reviews", on = "product_id", limit = 3)` | `.lookup('reviews', on: 'product_id', limit: 3)` |

| `.order("title", "desc", collate: "tr")` | `.order("title", "desc", collate = "tr")` | `.order('title', 'desc', 'tr')` |

| `.rows()`, `.first()`, `.count()`, `.rows(as: T.self)` | `.rows()`, `.first()`, `.count()`, `.rowsBlocking()` | `.rows()`, `.first()`, `.count()` |

| `.insert(doc)`, `.update(patch)`, `.delete()` | `.insert(doc)`, `.update(patch)`, `.delete()` | `.insert(doc)`, `.update(patch)`, `.delete()` |

| `.update(["n": .inc(1)])`, `.insert(doc, ifAbsent: true)` | `.update(mapOf("n" to Computed.inc(1)))`, `.insert(doc, ifAbsent = true)` | `.update({'n': Computed.inc(1)})`, `.insert(doc, ifAbsent: true)` |

Each has `select` with aggregates and `group`,
`where` with every operator, `in`, `null`,
paths into a json field and `or`, `and`,
`not` and `raw`; `match`, `near`,
`fuse` and `rerank`; `highlight`,
`snippet` and `facet`, the facets beside the rows;
`lookup` and its chains; `order` with a collation; `limit` and
`offset`; and the writes, an `update` or
`delete` with no filter refused unless it says `all`,
a value worked out over the row with `inc` and `expr`
([Writes that read the row](https://fenecdb.com/docs//docs/fenecql#writes)), and an
insert `if absent`.
A query is immutable, so a base query can be kept and branched from.

Details: one golden file

`integrations/builder-golden.json` is the contract: each case a
chain of steps (`{op, args}`, the JavaScript builder's names) and
the statement it must make, its text and parameters, or the refusal and its
message. The JavaScript builder is the reference: `make
builder-golden` runs every chain through it and writes what it made,
and its own tests run the file again, so the two cannot drift. The Python,
Go, .NET, Swift, Kotlin and Dart tests each read every case, call their own
builder, and compare the text and parameters exactly. The cases cover every method and option,
each operator and kind of value, parameter numbering across
`where`, `near`, `match` and the levels of
a lookup chain, and the names and combinations each builder refuses. Each
SDK's tests also run builder queries against a real server and hold their
answers to the same statements sent as text.

What a language has no word for is written its own way: Go's object
conditions are `Fields` and `Ops`, names and values
in turn, since a Go map has no order and the order is the text's; a Go
document keeps its order as a `fenecdb.D(...)`, and a map's
fields go in sorted. Options are keyword arguments in Python, named
arguments in C# and functional options in Go, where a struct could not tell
`limit 0` from no limit. A refusal is a `FenecError`
with status 0 in Python, a `*fenecdb.QueryError` in Go and a
`FenecQueryException` in .NET. Swift's dictionaries have no
order either, so a document or an object condition is a `Value`
or `Row` literal, which keeps the order written, or a
`Codable` type, its fields in the order it encodes them; Kotlin's
and Dart's maps keep their order. A refusal there is a
`FenecError`, a `FenecException` and a
`FenecException`, each with the code `builder`, and
every builder method throws as it is called, so a Swift chain takes one
`try`.

## From the shell

The same request from `curl`:

```bash
curl -X POST localhost:8080/query -H 'authorization: Bearer secret' \
  -d '{"query":"get docs select title, year where year >= $1 order year desc limit $2","params":[2020,5]}'
# [{"title":"Night at the oasis","year":2024},{"title":"Dunes","year":2021}]
```

The REST routes, batches, subscriptions and the change stream are on
[HTTP endpoint](https://fenecdb.com/docs/http).

Details: what the examples are held to

The programs under `integrations/languages` are the examples
above with their asserts: `make languages-test` starts a
`fenec-server` over a new file for each language and runs Python,
Java, PHP and Ruby from their official images in Docker, and JavaScript, Go,
C# and Rust with the toolchains on the machine — Go and C# both through
their SDK and without it, and each SDK's program through its query builder
too. The SDKs' own tests (`make go-test`,
`make dotnet-test`) start a primary, a replica of it and a node
of tenants and cover the rest: every type of parameter, the errors, a batch
put back whole, an idempotency key, reading a write on the replica, a
subscription, the change stream, a tenant and a scoped token. CI runs it on every change,
so a change to the HTTP endpoint that would break one of them is caught
before it ships.

The TypeScript tab's programs run there as the JavaScript they are
without their types, and the tab's code itself is type-checked under
`tsc --strict` against the client's declarations, as every
TypeScript example in these docs is (`make docs-types`).

Every answer is JSON: a read is an array of rows, a ranked search's rows
(`match`, `near`) carry their `_score`, a
write is `{"affected": n}`, and
a refusal is a status of 400 and up with `{"error": "..."}` — 401
for a missing or wrong token, 404 for a collection that is not there, 409
for an id `insert` finds taken. A vector goes in as a JSON array
of numbers and comes out as the shortest text that reads back as each
`f32`. Values always go in `params`, never into the
statement's text.

The server keeps a connection open between requests (HTTP/1.1
keep-alive) for a client that reuses it, as Go's, .NET's, Java's and
ureq's do. A body is read by its `Content-Length`; a chunked one
is refused.


---

Source: https://fenecdb.com/docs/fenecql

# FenecQL

The query language you use to create collections, write
documents and search them by vector, by text or by filter.

## Statements

FenecQL is a small language with vectors in the core. It is not SQL and
does not try to be, but it reads SQL's everyday spellings as the statements
they mean — `select … from`, `CREATE TABLE`,
`INSERT INTO … VALUES`, `UPDATE … SET a = …`,
`JOIN` as a `lookup`, `IN (…)`,
`EXISTS`, `DISTINCT`, `BETWEEN`,
`LIKE`, `ON CONFLICT` — and refuses the rest saying
what to write: see [From SQL](https://fenecdb.com/docs//docs/sql). These are all its
statements.

```fenecql
create collection [if not exists] <name> ( <field> <type> [required] [collate und|tr] [nocase] [@index], ...
                                         [, @sorted(<field> [desc], ...) | @unique(<field>, ...)] )
drop collection [if exists] <name>
create index [if not exists] on <name> (<field> | <path>) @index
create index [if not exists] on <name> (<field> [desc], <field> [desc], ...) @sorted | @unique
drop index [if exists] on <name> (<field>, <field>, ...)
alter collection <name> add field <field> <type> [collate und|tr] [nocase] [@index]
alter collection <name> drop field <field>
alter collection <name> rename field <field> to <new name>
alter collection <name> alter field <field> @ttl(<duration>) | @sorted

put <name> { field: value, ... }              -- or [ {...}, {...} ]
insert [into] <name> { field: value, ... }     -- as put, refusing an id that is taken
put <name> { field: value, ... } if absent     -- a held id or @unique value passed over, and not counted
put <name> { ... } if absent else set { n: n + new.n }   -- an upsert: the row holding it set, new.f the document's
put <name> $1 [if absent ...]                -- the documents a parameter holds: an object, or a list of them
insert into <name> (a, b) values (1, 'x'), ... [on conflict do nothing | do update set a = ...]  -- SQL's spelling
update <name> set a = <expr>, b = <expr> [where ...]   -- SQL's spelling of set
get <name> [select a, b] [where <expr>]
           [near <field> <vector> [ef N] [exact]]
           [match <field> <text> [fuzzy [N]] [prefix]] [rerank <field> <vector> [candidates N]]
           [fuse [k N] [candidates N]]             -- match and near, both ranking
           [order <key> [collate und|tr] [asc|desc], ...] [limit N] [offset N]   -- N or a parameter, $2
                                                   -- a key: a field, or an expression, distance(loc, $1)
           [facet <field> [top N | ranges [..]] [disjunctive], ...]   -- value counts over every matched row
           [lookup <name> on <child> [= <parent>] [required] <clauses...>]...
           [lookup <name> on (<child>, ...) = (<parent>, ...) ...]   -- a key over several fields
get <name> select a, highlight(f [, pre, post]), snippet(f, N [, ellipsis [, pre, post]])
           match <field> <text> ...           -- where the words matched
get <name> where <field> in (get <name> select <field> ...)  -- a subquery's list
get <name> [where <expr>] count               -- number of matching rows
get <name> select [<key>,] count(*), count(distinct e), sum(e), avg(e), min(e), max(e),
           first(e [by k]), last(e [by k]), approx_count_distinct(e),
           hll_accumulate(e), hll_combine(sketch) [as <name>], ... [where <expr>]
           [group <key>, ... [having <expr>] [order <column> [desc]] [limit N] [offset N] [count]]
get <name> select a, <expr> as <name> ...     -- a column worked out over each row
explain get <name> ...                         -- the path the query took
select a, b from <name> [where ...]           -- the classic SQL order works too
set <name> { field: value, path: value } [where <expr>]  -- a value may read the row: {n: n + 1}
del <name> [where <expr>]
<set | del ...> [order <key> [desc], ...] [limit N] [returning * | a, b]   -- the page a get answers, written and answered
<put | insert ...> [returning * | a, b]       -- the rows written, ids and all, in the documents' order
<put | insert | set | del ...> require N    -- refused, and its batch put back, unless it wrote N rows
get <name> ... require N [lookup ...]       -- refused, and its batch put back, unless it answered N rows

collections | describe <name> | compact [<name>]

-- a path, meta.source.rank, reads into a json field wherever a field is read
```

- **Every document gets an `id`** on its own. Give one in a
    `put` and it writes over the document with that id.

- **`insert` never writes over.** If an id is taken, the
    whole statement is refused; `put … if absent` passes over it
    instead, and the answer counts what was written; `put … if absent
    else set {n: n + new.n}` sets the row holding it instead, an upsert
    in one statement, `new.f` the document's field.

- **A `set` can read the row it writes.**
    `set hits {n: n + 1} where key = "ip1"` is an increment, made
    under the write lock — see [Writes that read the
    row](#writes).

- **A `set` or a `del` can pick its rows and answer
    them.** `order` and `limit` write the page a
    `get` with them would answer, and `returning` hands
    it back: a job queue's claim in one statement — see
    [Writes that pick their rows](#pick). A `put` or
    an `insert` answers the rows it wrote with
    `returning`, the ids it made among them.

- **A page's numbers take a parameter.** `limit $2 offset
    $3`, and a facet's `top $4`: every page of a list is
    one statement, which a server parses once — see [Pages
    by parameters](#paging).

- **`require N` makes a count a condition.** A write that
    does not write exactly `N` rows, or a `get` that
    does not answer exactly `N`, is refused, and the batch it is
    in is put back whole — see [Writes that must find
    their rows](#require).

- **`alter collection` rewrites no document.** A field
    added reads `null` in the documents already there; a field's
    type does not change — see [Changing a
    collection](#changing-a-collection).

- **A statement ends where the next one begins, or at a
    `;`.** No separator is needed, and a statement may run
    over several lines: the parser reads where each ends. A
    `;` ends the statement before it, so no clause goes on past
    one — `del notes; where id = 3` is refused, not read as one
    statement, and so is a `;` inside brackets, `in (get
    … ;)`. A clause that is a statement too begins one after it:
    `get a lookup b on k; select z` is two statements, the
    second a read of `z`. A query string's `select=`,
    `group=` and `facet=` take a `;` after
    their list as they always did. How the shell runs statements written
    over lines is in [In the
    shell](https://fenecdb.com/docs/quickstart#in-the-shell).

Details

The `id` field is automatic. Supplying `id` inside a
`put` turns it into an upsert. `insert` makes documents
and never writes over one: a document naming an id that is taken refuses the
statement, and nothing of it is written — `409` over HTTP, as
PostgreSQL's `INSERT` refuses a taken key. Without an `id` the
two are the same. A retried write that must not be made twice either way
sends an [`Idempotency-Key`](https://fenecdb.com/docs//docs/http#idempotency).

## Reference

Types, indexes, operators and functions at a glance.

| **Types** | `bool` `int` `float` `text` `bytes` `timestamp` `vector<N[, f16]>` `sparse<N>` `[type]` `json` `geo` (a point, `[lon, lat]` — see [Points](#geo)) |

| **Case** | `email text nocase` compares by its case fold wherever it is compared — `=`, its indexes, `order`, `group` — and keeps what was written — see [Case-insensitive text](#nocase) |

| **Paths** | `meta.source.rank` reads into a `json` field in `where`, `select`, `order` and `set`, and takes `@hash`, `@unique` and `@sorted` — see [Objects and paths](#json) |

| **Indexes** | `@sorted(owner, created desc)` and `@unique(org, slug)` over several fields (see [Indexes over several fields](#composite)), `@hash`, `@unique`, `@sorted`, `@ttl(30m)` (a `timestamp` field's rows expire — see [Rows that expire](#ttl)), `@text(k1=.., b=.., prefix=.., chars)`, `@hnsw(metric, m=.., ef_construction=.., ef_search=.., quant=int8|bit)`, `@inverted` (a `sparse<N>` field's), `@geo` (a `geo` field's) |

| **Defaults** | `@hnsw(cosine, m=16, ef_construction=200, ef_search=100)`; `ef_search=200` with `quant=bit` |

| **Metrics** | `cosine` `l2` `dot` |

| **Operators** | `=` `!=` (or `<>`) `<` `<=` `>` `>=`, `~` text contains (case-insensitive), `has` list contains, `in [..]` (or `in (..)`), `in (get …)` — see [Subqueries](#subqueries), `is null`, `between a and b`, `like` and `ilike` with `%` and `_`, `not in`, `not like`, `not between` — see [From SQL](https://fenecdb.com/docs//docs/sql) |

| **Arithmetic** | `+` `-` `*` `/` on numbers, a `timestamp` moved by milliseconds, and `+` (or `||`) joining two texts — in a `set`'s values and in `where`; see [Writes that read the row](#writes) |

| **Logic** | `and` `or` `not` — `and` binds tighter |

| **Parameters** | `$1`, `$2`, … as in PostgreSQL, wherever a value goes, and as `limit`, `offset` and a facet's `top` — see [Pages by parameters](#paging) |

| **Functions** | `lower` `upper` `len` `coalesce` `greatest` `least` `now` `expired` ([rows past their `@ttl`](#reaper)) `timestamp` `bucket(at, 15m [, zone])` `cosine` `l2` `dot` `norm` `normalize` `distance(loc, p)` `within(loc, box)` ([Points](#geo)), `case when … then … else … end`, plus plugins — see [Functions over a row](#functions) |

| **Ordering** | `order year desc, title asc` — a tie on the first key is decided by the second; `id` can be ordered too; a key may be any expression over the row, `order distance(loc, $1)`, `order price * qty desc` (see [Ordering by an expression](#order-by-expression)); `order name collate und` puts text in Unicode's order for every language and `collate tr` in Turkish order, and a field declared `name text collate und` is in it wherever it compares — see [Collation](#collation) |

| **Picking** | `set … order run_at limit 10 returning *` writes the rows a `get` with the clauses answers and hands them back, as written; `del … returning` as they were; `insert … returning id` the rows it made — see [Writes that pick their rows](#pick) |

| **Explain** | `explain get ...` runs the query and returns the path it took, one row a step — see [Explain](#explain) |

| **Typos** | `match body $1 fuzzy` matches each word within the edits its length allows, a word as typed ranking first; `prefix` matches the last word as the start of one, as a search box is typed into — see [Typos and words begun](#typos) |

| **Highlights** | `highlight(body)` and `snippet(body, 20)` in the select list mark the words a `match` found, as UTF-16 offsets or between tags you pass — see [Highlights and snippets](#highlights) |

| **Facets** | `facet brand, color top 10` counts each value over every row the query matches, beside the page — see [Facets](#facets) |

| **Counting** | `count` returns one row with one column; it does not combine with `select`, `near`, `match`, `order`, `limit` or `offset` |

| **Aggregates** | `count(*)` `count(…)` (the rows it is not null in) `count(distinct …)` `sum` `avg` `min` `max` `first(… by …)` `last(… by …)` `approx_count_distinct` `hll_accumulate` `hll_combine` over fields, paths or expressions, whole or per `group` of one key or several, the groups kept by `having` — see [Aggregates](#aggregates) and the [analytics and market-data recipes](https://fenecdb.com/docs//docs/analytics) |

| **Columns** | `select px * qty as notional`: an expression over the row, named with `as` — see [Columns worked out](#columns) |

| **Ceilings** | `near`, `match` and `fuse` at most 10 000 rows (`limit + offset`, a parameter's as a number written), `rerank` at most 10 000 candidates, `fuse` 10 000 a side; a word of a `fuzzy` or `prefix` `match` 10 000 words of the index; expression depth 512 levels; a `json` value 64 levels deep, a path 64 keys; `in (get …)` 100 000 values and 4 levels — see [Limits](https://fenecdb.com/docs/limits) |

## Pages by parameters

`limit`, `offset` and a facet's `top`
take a parameter wherever they take a number: a `get`'s, a
`lookup` level's, an inner `get`'s, and a
`set`'s or a `del`'s that picks its rows.

```fenecql
get articles where year = $1 order published desc limit $2 offset $3
get products limit $1 lookup reviews on product_id order created desc limit $2
get products match title $1 facet brand top $2 limit $3
set jobs {owner: $1} where run_at <= now() order run_at limit $2 returning *
```

- **Every page is one statement.** Written into the text, each page
    size and each page was a text of its own, which a server parses unless
    its parse cache still holds it; as parameters a list is one text
    whatever page is asked. Every builder sends them so.

- **A parameter is the number it holds.** It is checked as the
    statement runs, before the plan: `limit $1` handed 10
    answers what `limit 10` answers, row for row, through the
    same plan, and `explain` names the number.

- **It is refused as the number written would be,** in the same
    words: a count is a whole number from 0 (`20.0`, a float
    with no fraction, is 20), and `-1`, `2.5`, a text
    or `null` is ``limit` takes a whole number from 0, not
    -1`. Past a ceiling it meets the check the number written meets:
    `near` and `match` at 10 000 rows of `limit +
    offset`, a facet's `top` from 1 to 10 000. A count
    past what memory could hold is every row.

- **`require N` stays a number written.** Its count is
    what the statement guards — `require 1`, "this row and no
    other" — a constant of the code, which never makes a text of its own;
    a page's size and place are a request's data.

Details

The statement keeps the parameter, so what reads a statement by its
shape reads `limit $2` as a limit: `count` and an
aggregate without `group` refuse `offset $1` as they
refuse `offset 5`, and a sync replica refuses a synced
`set … limit $1` as it refuses `limit 1`. What reads
it by its number reads the number it is handed: a long read is pinned or not by the page it asks
for, as a number written is. `/_stats/statements` counted a page size
written in as a place already, `limit $3`; the parse cache
(`POST /query`, 16 shards of 64, a shard emptied when full)
kept a text a page.

## The vector clause

`near` finds the documents whose vector is closest to the one
you pass. Add a `where` to search only the rows that match.

```fenecql
get articles near embed $1 limit 10
get articles where year >= 2024 near embed $1 ef 200 limit 10
get articles near embed $1 exact limit 10
```

- **`ef` trades speed for accuracy** for one query.

- **`exact` checks every vector**, which is how you measure
    recall.

- **Each row gets a `_score`** column.

Details

`ef` raises the search beam for this query only, trading latency
for recall. `exact` replaces the ANN walk with a full scan, which is
how you verify recall against ground truth. A query that uses
`near` gains a `_score` column. Over a
`sparse<N>` field `near` ranks by dot product and
is exact already, so it takes no `ef` — see Sparse vectors.

A filter next to `near` is planned, not stacked: the filter's
rows are read only until it is clear whether there are more of them than the
ANN would measure anyway, and the planner picks between searching that set
exactly and running the ANN with each candidate tested against the filter.
The mechanism, and the fallback that keeps it correct, is in How it works.

## The text clause

`match` ranks documents by how well their text matches your
words. `rerank` then reorders the best of them by vector
distance.

```fenecql
get articles match body $1 limit 10
get articles where year >= 2024 match body $1 limit 10
get articles match body $1 rerank embed $2 candidates 500 limit 10
```

- **`match` needs a `@text` index** and adds a
    `_score` column.

- **`rerank` needs no vector index.** It reads the vectors
    of the matched rows directly.

- **Reranking cannot find what the words missed.** For that, use hybrid search.

Details

`match` ranks by BM25 over a `@text` index and adds a
`_score` column. Like `near` it decides the ordering, so
it combines with neither `order` nor `near` — unless
`fuse` says how the two rankings make one (below). A
`where` filter is applied while the postings are merged, so it costs
no second pass.

`rerank` reorders what `match` found by exact vector
distance. It needs `match` — there is nothing to reorder without it
— but it does **not** need an `@hnsw` index: the vectors are read
straight out of the store, so this path builds, holds and validates no graph at
all. `candidates` sets how many of the matched rows are rescored
(default 200); it is never allowed below `limit + offset`.

**The two stages answer different questions.** `match` is
  recall and knows nothing about meaning; `rerank` is precision but
  only over what the first stage handed it. Measured on BEIR: on SciFact
  50 candidates score nDCG@10 0.656 against 0.645 for a full dense scan of the
  same vectors, and on FiQA 1 000 candidates come within 0.0001 of the full scan —
  each while scoring under 2% of the corpus. If the lexical stage cannot find
  the answer at all, no amount of reranking will.

## Typos and words begun

`fuzzy` lets a word of a `match` find the words a
typo or two away from it, and `prefix` lets the last word find
the words it begins — what a search box needs as someone types into it.

```fenecql
get products match title $1 fuzzy limit 10             -- "databse" finds "database"
get products match title $1 prefix limit 10            -- "datab" finds "database", "databases"
get products match title $1 fuzzy prefix limit 10      -- both, the last word a misspelt beginning
get products match title $1 fuzzy 1 limit 10           -- one edit at most
```

- **A word's length decides its edits**: none under 5 characters,
    one from 5 to 8, two from 9. `fuzzy 1` allows one at most,
    `fuzzy 0` none.

- **A word as typed ranks first.** A document holding the word
    itself scores more from it than one holding only a word near it.

- **A word reaches 10 000 words of the index at most**, and the query
    is refused past that rather than cut — a prefix of one letter over a
    large vocabulary is.

- **`highlight()` and `snippet()` mark what was
    found**: `databse` marks `database`.

Details

An edit is a letter added, dropped, changed, or two letters next to
each other swapped — Damerau's distance in its restricted form, as Lucene,
Elasticsearch and Meilisearch count it: the four are what Damerau found
four misspellings in five to be, and a swap counted as two edits would put
a five-letter word with one out of reach. An edit that touches a word's first letter counts two, as
Meilisearch counts it: a first letter is rarely the mistyped one, and
letting it be multiplies what is found by words no one meant. The bounds by
length are Meilisearch's too: a short word has too many neighbours —
`cart` is one letter from `card`, `care`,
`cars`, `cat` and `art`. Words are folded
as the index folds them before they are compared, so
`İSTANBL` finds `istanbul`. Only the last word is a
prefix; the words before it are whole.

The words a word reaches count as one term of the query, a document's
counts of them summed, so BM25's saturation holds over them as over a word
written three times, and the term's rarity is theirs together — a rare
misspelling in the corpus ranks no higher than the word it misspells. A
word one edit away, or one a prefix begins, counts as the word would; two
edits away, a quarter as much. Where the index holds the word as typed,
its variants together are kept below the least it adds to a document, so a
one-word query ranks every document holding it above every document
holding only what is near it. Measured on BEIR with one typo put into each
query, a word one edit away was the word meant and two edits away as often
another word, which the weights follow.

The words are found in a sorted copy of the index's words, made the
first time a `match` asks for `fuzzy` or
`prefix` and kept up by every write after it, as the index is:
derived, never in the file. A prefix is a range of it; the words within a
few edits are found by walking it as the tree of prefixes it is, giving a
branch up once no word below it can be near enough. Over 100 000 documents
of 60 000 words, sorting them took 6 ms and 1.3 MB; finding a word's
neighbours over FiQA's 135 500 words, 14 µs at one edit and 120 at two, its
completions 4 µs.

**Measured on BEIR**, nDCG@10 (`make beir`), one
  realistic typo put into a word of each query — a letter dropped, a key
  beside it typed as well or instead, or two letters swapped, the first
  letter 7% of the time:
  SciFact's queries as written score 0.662 and 0.665 with
  `fuzzy`; with the typo 0.636, and 0.663 with
  `fuzzy`. FiQA's 0.232 and 0.233; with the typo 0.213, and
  0.229. With each query's last word half typed, `prefix` takes
  SciFact from 0.616 to 0.655 and FiQA from 0.188 to 0.208, and leaves the
  queries written whole where they were. See the whole table.

## Hybrid search

`fuse` combines a text search and a vector search into one
ranking. Use it when some answers share your words and others only your
meaning.

```fenecql
get articles match body $1 near embed $2 fuse limit 10
get articles where year >= 2024 match body $1 near embed $2 fuse limit 10
get articles match body $1 near embed $2 fuse k 60 candidates 50 limit 10
```

- **It adds ranks, not scores**, so there is no weight to tune.

- **It needs an `@hnsw` index** for the vector side.

Details

`fuse` lets `match` and `near` rank the
same query together. Each side takes its own first `candidates` —
20 unless given, never fewer than `limit + offset` — with the
`where` filter applied to both, and a document scores
`1 / (k + rank)` from each list it is on, `k` being 60
unless given. The sum is the `_score`; ties go to the lower id.

Ranks rather than scores, because the two have no scale in common: a BM25
score grows with the length of the query and the rarity of its words, a
cosine distance does neither, and a weighted sum would need its weight tuned
again for every corpus. Where `rerank` can only reorder what the
words found, `fuse` also takes what they missed — at the price of
the `@hnsw` graph `near` walks.

**Measured on BEIR**, nDCG@10, all-MiniLM-L6-v2 vectors (the whole table). On SciFact,
  where a claim shares its words with the evidence, `fuse` scores
  0.700 against 0.662 for `match` and 0.645 for `near`
  alone. On FiQA, where a question shares few words with its answer, it scores
  0.366 against 0.232 and 0.366: the weak side does not drag the strong one
  down. Up to 61 candidates a side at `k = 60`, a document both
  searches found outranks every document only one found; deeper, agreement in
  the middle of both lists starts to outvote the top of one, and at 100 a side
  the two scores fall to 0.688 and 0.359.

## Highlights and snippets

`highlight()` and `snippet()` in the select list
show where the words of a `match` are in a row's text, so a
results page can mark them.

```fenecql
get articles select title, highlight(body) match body $1 limit 10
get articles select title, highlight(body, "<mark>", "</mark>") match body $1 limit 10
get articles select title, snippet(body, 20, "…") match body $1 limit 10
get articles select title, snippet(body, 20, $2, $3, $4) match body $1 limit 10
```

- **Without tags you get positions**: `highlight(body)` is a
    list of `[start, end]` pairs; `snippet(body, 20)` is
    `{"text": …, "marks": [[start, end], …]}`.

- **With tags you get the marked text**, each mark between the two
    values you pass — literals or parameters. The engine adds no markup of
    its own and escapes nothing.

- **Offsets count UTF-16 code units**, what a JavaScript, Java,
    Kotlin, C#, Dart or Swift `NSString` index counts:
    `text.slice(start, end)` is the marked word.

- **They need `match`**, and go with `fuse` and
    `rerank`; a query without `match` has no words to
    mark, and is refused.

Details

A row's text is read again at query time by the tokenizer of the
`@text` index the `match` searched — the same splitting,
the same folding (`İstanbul`, `ISTANBUL` and
`istanbul` are one word), the same prefixes and runs of
characters — and every span a term of the query was read from is marked. So
a mark is exactly what the index matched: a word; the whole word a prefix
of `@text(prefix=N)` was cut from (`kitapların` for
`kitap`); the characters of a pair or a triple of Han, kana,
Hangul or Thai, overlapping pairs made one mark (`東京都` for
`東京都`). Nothing is kept in the index for it, and the file does
not change. Marks come in order and never overlap; a mark never ends inside
a character, and takes the marks that belong to the characters at its ends
— a combining accent, a Thai tone mark, the variation selector and keycap
after a digit — so tagged text never splits a letter from its accent.

Any `text` field can be marked, read by the tokenizer of the
field `match` searched: `select highlight(title)`
beside `match body $1` marks the title's words the query holds.
Under `fuse` a row only `near` found holds no word of
the query and has no marks; a null field is `null`. Each item
answers under its label, `highlight(body)` or
`snippet(body)`, after the fields `select` names, where
the list puts it — and after every field with `select *,
highlight(body)`. The same item twice is refused: the two columns would
answer to one name.

A snippet is the window of `N` words holding the most marks,
centred on the first and the last of them — the first `N` words
when none is marked — each character of a run of Han, kana or Hangul counted
a word. The ellipsis, when given, stands for what it leaves out at either
end; a window holding the start or the end of the text has none there. Its
marks count from the start of its `text`, the ellipsis
included.

**The text is not escaped.** A page that puts a marked text into
  HTML lets the text's own `<` through. Build the page from the
  offsets instead — the text as text nodes, each mark a
  `<mark>` element — or escape the text first and the tags
  after.

Over 100 000 documents of 30 words, a mark costs a row about
1 to 3 µs (2 to 4 for a snippet) on top of the `match`; a
query that asks none runs as it did, `match` 0.06 ms either way
(`make search-bench`).

## The lookup clause

`lookup` attaches matching documents from another collection to
each row — reviews to their products, say. A `limit` after it counts
children per parent.

```fenecql
get products limit 20 lookup reviews on product_id order created desc limit 3
```

- **Clauses after `lookup` apply to the child
    collection**; clauses before it apply to the parent.

- **The child's key needs `id`, a `@hash`
    index, or an index over several fields it leads.** A key of several
    fields is written `on (org, order) = (org, id)`.

- **`required` keeps only parents that have a match.**

- **A level's list may aggregate** — `lookup orders on shop
    select count(*) as n, sum(total) as revenue` — each parent's
    children folded, one row a parent; and a `get` whose list
    aggregates takes a `lookup` off its groups. See A level that folds.

Details

```fenecql
get products lookup reviews on product_id
get products limit 20 lookup reviews on product_id order created desc limit 3
get products where price >= 10000 select name
  lookup reviews on product_id select body where stars >= 4 limit 3
get products lookup tags on sku = sku
```

`lookup` attaches the matching documents of another collection
to the row they belong to. `on` names the child's field; the
parent's key is `id` unless `= <parent>` says
otherwise, which is the foreign-key-to-primary-key case spelled out.

It is **terminal**: everything before it binds to the driving
collection, everything after it to the looked-up one. That is the whole
scoping rule, and it is why there are no qualified names in the language —
`where` on either side means what it has always meant. The price
is that a parent clause cannot follow a child one, which reads the way the
query runs anyway.

```fenecql
get shops limit 20
  lookup orders on shop_id limit 3
  lookup lines on order_id limit 5
```

A second `lookup` **chains** onto the first: its clauses bind
to its collection, and `on <child> = <parent>`
names a field of the level immediately above. Position keeps scoping the query
one level deeper, so exactly one collection is in scope at any point and it is
the last one named — which is what keeps qualified names out of the language at
any depth. Up to **8** levels; deeper is an error, not a silent truncation.

**`limit` after `lookup` counts children per
  parent.** This is the clause's reason to exist. A join's limit counts
  pairs, so a single product with 56 374 reviews consumes the whole page and
  the other nineteen never appear; asking a join for three reviews each needs
  a window function. Here twenty products with three reviews apiece is one
  query with a bounded answer.

The child key must be `id` or carry `@hash`. Refusing
an unindexed one is the same rule `near` and `match`
follow: the clause is one bucket probe per parent, and quietly becoming a
full scan of the child collection instead — 12.98 ms against 0.092 ms on the
same 200 000 rows — is not a thing a query should do without saying so.

### A key over several fields

```fenecql
create collection lines (org int, order int, line int, qty int, @sorted(org, order, line))
get orders where org = $1 limit 20
  lookup lines on (org, order) = (org, id) order line limit 5
```

Where one field does not say which children are a row's — a tenant's
rows by `(org, user)`, an order's lines in a file every tenant's
orders share, a child keyed by two fields of its parent — `on`
pairs a field of each side for each part, in order: `(org, order) =
(org, id)` is `lines.org = orders.org and lines.order =
orders.id`. The positional rule is the one above: the names on the
left are the looked-up collection's, those on the right the level
immediately above's, `id` among them. A null on either side
matches nothing, as for a single key.

The child side is answered by an [index over several
fields](#composite) whose first parts are the child's fields, in any order:
`@sorted(org, order, line)` or `@unique(order, org)`
answer `on (org, order) = …`. A parent's values, each made its
child field's type as a single key's is, laid end to end as the index lays
a row's, are the start of its children's keys, so its children are one run
of the index, found by two searches. No such index is refused, naming the
one to declare — `needs an index whose first fields are those
(declare it with @sorted(org, order))` — for the reason an unindexed
single key is. A child's `id` cannot be one of several parts (it
names one row alone), and the two sides name as many fields.

Where the level's `order` is the index's next parts — in its
directions or every one reversed, a part the key fixes ordering nothing —
each parent's run is **walked** in that order and stops at the page:
"the five latest lines of each order" reads five lines an order, where a
bucket is read whole and sorted. Any other order is read and sorted per
parent, as for a single key, its ties in ascending id either way. A single
field leading an index over several is answered the same way: with no
`@hash` of its own, or where the level's order walks the index
past it — `lookup reviews on product_id order created desc limit
3` over `@sorted(product_id, created desc)`.

`required` takes either side here too: where a part is paired
with the parent's `id` and the child filter holds a
`@hash` equality, the children it names map back to their
parents by that part, each held to the rest of the key by its own parts
against its parent's — the same bytes the probe from the parent would
find.

### A level that folds

```fenecql
get shops limit 20 lookup orders on shop select count(*) as n, sum(total) as revenue
get couriers limit 20 lookup deliveries on courier order at desc limit 50
  select count(*) as n, avg(km) as km
get shops lookup lines on shop select product, sum(qty) as sold group product
  order sold desc limit 3
get orders select courier, count(*) as n where at >= $1 group courier
  order n desc limit 10 lookup couriers on user = courier select name
```

A level's list may aggregate, as a `get`'s does, by the same
rules: each parent's children — probed once, as any level's are, held to
its `where` — folded into one row, or with `group`
into one row a group, ordered and paged a parent at a time. A level is the
`get` its children would be with the parent's key in its
`where`, and its clauses mean what they mean there: ungrouped,
`order`, `limit` and `offset` pick the
children folded — a courier's latest 50, walked off `@sorted(courier,
at desc)` as far as the page and no further — and grouped they order
and page the groups, a shop's three best sellers, a window function's
question in SQL. A parent with no child gets the one row every ungrouped
list answers, `count` 0 and a sum null, and no group.

A `get` whose list aggregates takes a `lookup` too:
it hangs off the rows the list answers, its groups, and `on child =
parent` names a column of the list — the busiest ten couriers with
their names, above, where the list named the courier. The same holds below
a level that folds. The positional rule does not move: the names on the left
are the looked-up collection's, those on the right the level immediately
above's, and that level's rows are its groups.

What stays out is a list of two collections: grouping parents by a field
of theirs and folding their children's — revenue by the shops' city — would
put a parent's name and a child's beside each other in one list, which is
the qualified name the scoping keeps out. Fold the level a parent at a time
and group the parents in the client, or keep the field on the child.

`required` on a level that folds asks of its children, as on
any level: an ungrouped list answers a row for every parent, so asked of the
rows it would keep them all, and "couriers who delivered, with their counts"
is the question. A `required` level below one that folds drops
its groups. Over 2 000 couriers and 200 000 deliveries, a page of the 20
busiest with the count and mean of each one's latest 50 takes 53 µs in
process, against 81 for the 50 attached and folded by the client; every
delivery of each folded, 3.5 to 4.1 ms against 6.4 to 6.5; the ten busiest
with their names off a week's groups 8.3 ms, as the groups and a query for
the names apart do, less the round trip between them (`make
analytics-bench`).

```fenecql
get products limit 20 lookup reviews on product_id required where stars = 5
get products count lookup reviews on product_id required where stars = 5
```

`required` drops a parent that no child matches — "products that
have a five-star review" rather than "products, with their five-star reviews".
It is tested before `offset` and `limit`, so a page of
twenty comes back with twenty parents that have children rather than whatever
survived; a child `offset` that walks past every match does not undo
it, because the parent has matches all the same. It is the one shape where
`count` combines with `lookup`: nothing is being attached,
so counting the parents that have a match is a question with an answer.

**`required` is a pass, and it is not cheap.** It is
  answered from whichever side is smaller: walking the candidate parents and
  probing each, or — when the child filter carries a `@hash`
  equality and the parent's key is `id` — reading the children that
  equality names and taking their parents. Both counts that decide are exact,
  so the choice is not a guess; what neither plan escapes, once the page is
  put in order or a facet counts every parent, is that one full pass has to
  happen. With no `order` the page is the first parents kept, and
  walking the parents stops once it holds them. Over 20 000 products and
  200 000 reviews, for a page of twenty:

  | `required`, child filter on a `@hash` field | 12.07 ms |

  | the same, walking the parents instead | 15.63 ms |

  | `required`, no child filter, no `order` | 0.036 ms |

  | `count … required` | 4.50 ms |

  | a `bool @hash` field on the parent | 0.02 ms |

Ask it occasionally and `required` is the right tool. Ask it on
  every page load and the answer belongs in a field on the parent, maintained
  on write — which is the same trade the rest of this database makes.

In a chain, `required` is a statement about its own level: it
drops rows of the level immediately above it and stops there. Requiring lines
drops the orders that have none; the shops keep their rows, with an
empty group, unless the orders level says `required` as well. Said
at every level it composes into "shops with an order that has a line", and the
existence check walks the whole way down — a row is only kept if it has a
descendant that is itself kept. Nothing acts at a distance, and the flag means
the same thing wherever it is written.

`lookup` combines with neither `near`,
`match` nor `rerank`: each ranks one collection's
documents, and a score spanning a parent and its children has no defensible
meaning. `count` is refused as well unless the clause is
`required`, since otherwise it collapses the very rows the children
would hang from.

`order` inside a `lookup` is bounded in advance: the
clause carries its own `limit`, so only `offset + limit`
children can ever be emitted and the rest are never put in order. On a parent
with 56 374 children, `order created desc limit 3` costs 3.71 ms
rather than 7.52 ms. What remains is reading one key per child, which no
ordering can avoid — you cannot know the first three without having seen all
of them. The query's own `order` works the same way.

Over JSON the children are nested under the collection's name, and a chain
nests all the way down. Flattened into rows, for a caller that wants a table,
they arrive the way a join presents them — the same data; a chain widens once
per level, one row per
root-to-leaf path, and a level that ran out fills its own columns and every
column below it with nulls. A parent with no matches keeps its row either way:
the page is the parents. What the flat shape cannot show is the per-level
`limit`, which is exactly why the nesting is what the other
transports carry.

**What it buys, measured** (5 000 products, 100 007 reviews, twenty
  parents with three children each): the clause is **22.7 µs** in process
  and **0.188 ms** over HTTP, against **0.332 ms** for the same work as
  a `/batch` of twenty-one queries merged on the client.

The larger gain is that there is one way to write it. The same question
  in SQLite, over the same data: a correlated subquery takes 0.053 ms, a
  window function with the restriction pushed inside the partition 0.331 ms,
  and the same window function with the restriction left outside
  **81.34 ms** — a factor of 1 500 between three spellings of one
  question, with nothing in the query text to tell them apart.

A second level is where batching stops being an answer. One level of N+1
  fits in a single round trip, because every query in it can be written from
  the page's ids; the level below cannot, because its keys are in rows that
  have not come back yet — so the client pays two round trips whatever it
  does. Over 2 000 shops, 20 000 orders and 200 000 lines, for a page of
  20 × 3 × 5: **49.1 µs** in process against **139.8 µs** for the same
  page as 81 separate queries, and **0.204 ms** for the one HTTP request
  against **0.415 ms** for two `/batch` trips — measured on
  loopback, where the extra trip costs almost nothing and still doubles it.

## Subqueries

`in (get …)` filters by rows of another collection: the reverse
of `lookup`. The inner `get` runs once, before the
query, and the one column it selects is the list.

```fenecql
get orders where customer in (get customers select id where country = "TR")
```

- **It answers as its list written out.** One bucket per value over a
    `@hash` field or `id`, a scan otherwise — the
    `in [..]` you would have written.

- **The inner `get` selects exactly one column.** A field,
    or one aggregate; no `lookup` and no `count`.

- **It goes wherever a filter goes**: `get`,
    `count`, aggregates, a `near`'s or a
    `match`'s filter, a `lookup` level's, `set …
    where` and `del … where`.

- **At most 100 000 distinct values** and 4 levels of nesting; past
    either the query is an error, never a set cut short. A value held by many
    rows counts once.

Details

```fenecql
get orders where status = "open" and customer in (get customers select id where tier >= $1)
get orders where not customer in (get customers select id where country = "TR") count
del sessions where user in (get users select name where banned = true)
get customers limit 20 lookup orders on customer
  where id in (get returns select order_id) limit 3
get orders where customer in (get customers select id
  where tier in (get tiers select n where active = true))
```

It is uncorrelated: the inner `get` reads nothing of the row
being tested, so it runs once, with the same parameters as the query around
it — `$1` inside is the query's `$1`. Its column is a
field, or one aggregate (`get orders select max(total)`); a
`null` in it is left out of the list, since it would otherwise find
the rows whose field is null, and `not … in` would lose every row.
`not` is FenecQL's own `not`, written before:
`not customer in (get …)`; `customer not in` is not
FenecQL.

The values become the list an `in [..]` is answered with, and
the same rule holds: a list reaches the `@hash` index only when
every value is one the field's type can express, or the whole list goes to
the scan. A long list is looked up rather than walked — a scan of 200 000
rows by an unindexed field against the 2 000 codes an inner `get`
handed it took 2 302 ms compared a value at a time and takes 16. The bound
is on the values the list holds, not on the rows that hold them: a
funnel's 125 000 `buy` events from 937 users is a list of 937.
Past 100 000 rows the values are told apart, and the inner `get`
read until they number more than 100 000; with a `limit`,
`group` or ranking it is read one row past the bound, and a
`limit` inside is the query's own cut, and answers. `explain` names what each inner `get`
found.

Over HTTP a scoped token's rules hold the inner `get` as they
hold any `get`: the list holds only rows the token may read, so
nothing of the others reaches it through what the outer query finds, and a
column its rules show on some rows alone is listed from those rows
([Fields a read shows](https://fenecdb.com/docs//docs/http#field-grants)). A token's rule
may hold one too, held to 1 000 values and answered before each statement
([Scoped access](https://fenecdb.com/docs//docs/http#scoped-access)). The JavaScript
builder takes a query
as an `in`'s value:
`{ customer: { in: db.from('customers').select('id').where({ country: 'TR' }) } }`.
A subscription's shape takes a list only — its stream follows its own
collection's writes, and an inner set would not follow the other's.

**What it costs, measured** (`make subquery-bench`, 20 000
  customers and 200 000 orders, the median of 21). The orders of one
  country's 2 000 customers by the `@hash` on
  `customer`: 0.83 ms, against 0.72 ms with the 2 000 ids written
  out; by an unindexed code, 16.1 ms against 15.9. The customers with a rare
  order — the inner `get` reading a bucket — 0.20 ms against
  0.065 written out and 0.44 for `lookup orders on customer required
  where status = "rare"`, the same question asked from the other side;
  with a filter on the customers as well, 0.56 against `lookup`'s
  0.95. Where the inner `get` scans 200 000 orders it is that scan,
  10.4 ms, and `lookup … required` probes each customer's orders
  instead, 32.5. The difference to the list written out is the inner
  `get`: a subquery is one round trip where the list is two.

## Indexes

An index makes one kind of question fast. Declare it on a field when you
create the collection, or add it later with `create index`.

```fenecql
create collection articles (
  title      text,
  body       text @text(k1=0.9, b=0.4, prefix=0),
  year       int @hash,
  slug       text @unique,
  published  timestamp @sorted,
  embed      vector<768> @hnsw(cosine, m=16, ef_construction=200)
)
```

- **One index per field:** `@hash` for equality,
    `@sorted` for ranges and order, `@text` for ranked
    text search, `@hnsw` for nearest neighbours, `@geo`
    for points near a place. A path into a
    `json` field takes `@hash`, `@unique`
    or `@sorted` of its own ([Objects and
    paths](#json)), and an index over several fields is the collection's, beside
    them ([below](#composite)).

- **`@unique` is a `@hash` that refuses a second
    document** holding a value one holds already. `null` is no
    value: any number of documents may hold it.

- **Only filters joined by `and` use an index.** Anything
    under `or`, and `!=`, `~` or
    `has`, reads every row.

- **For Turkish, Chinese or Japanese text**, see `prefix`
    and `chars` below.

Details

```fenecql
create index on articles (embed) @hnsw(cosine)
create index on articles (body) @text
create index on articles (body) @text(prefix=6)   -- for an inflected language
create index on products (title) @text(chars)     -- short Chinese or Japanese text
create index on articles (title) @sorted
create index on users (email) @unique             -- refused if a value is held twice
```

A field takes one index. `@hash` answers equality,
`@sorted` ranges and order, `@text` ranked text
retrieval, `@hnsw` nearest neighbours. `@sorted` is
for `int`, `float`, `timestamp` and
`text` — the types with an order.

`@unique` is a `@hash` index that refuses a
second document holding a value one holds already: a `put`, an
`insert` or a `set` that would write one is refused,
and nothing of the statement is written — `409` over HTTP, as a
taken id is. It answers equality and a `lookup` as
`@hash` does. A block — a `/batch`, a run of several —
is checked against its own earlier writes, so it may free a value and take
it again, and a document keeping its own value is no second one.
`null` is no value, as in SQL: any number of documents hold it.
Values are one where `=` finds them one, so `-0.0`
and `0.0` collide. `create index ... @unique` over
documents that hold a value twice is refused and names the value and two
of its documents. A replica takes what its primary took and does not ask
again; `describe` shows the index as `@unique`. The
refusal names the document holding the value and the value, but to a
[scoped token](https://fenecdb.com/docs//docs/http#scoped-access) it names the field
alone (``users.email` is unique, and the value is taken`): the
row is another user's, which the token may not read.

The check asks the bucket the value would go in, so the index has to
hold every document: the first write after an open builds it, where a
plain `@hash` waits for the first read that wants it. An open
costs what it did — 19 ms for an image of a million rows either way — and
that first write takes 15 ms over 100 000 rows and 259 ms over a million;
after it a `put` takes 940 ns against 850 for the same field
under `@hash`, the price of the lookup. A collection with no
unique field writes as fast as before (848 and 852 ns).

An `@hnsw` graph holds a vector once. Documents written with
the same vector — the embedding of an empty chunk, a line of boilerplate,
a zero vector — share its node, and `near` hands back each of
them at the one distance. A node each, a vector written hundreds of times
filled its neighbours' lists with copies of itself: over 20 000 rows of 128
dimensions, a third of them copies of 20 vectors, 1 812 copies were found by
no search and other queries' recall fell from 0.82 to 0.67; one node each,
none is missed and it is 0.87.

**`prefix=N` indexes each word's prefixes alongside it**
  (3..N by default; `prefix_min` moves the lower bound). It is a
  stemmer that needs no dictionary and no language setting, and it is off
  unless asked for because it is a trade the corpus decides. Word-boundary
  matching suits English; it does not suit a language that inflects by gluing
  suffixes on, where `kitap`, `kitabı` and
  `kitapların` are three terms sharing a stem the index never
  sees.

Measured on Turkish WebFAQ (144 846 documents, 10 000 queries):
  nDCG@10 0.4841 → 0.5547, +14.6%, closing about two fifths of the gap to
  a 278M-parameter multilingual transformer (0.650). The cost is the 3.4×
  postings it indexes — 58 MB → 182 MB, and 82 → 485 µs per query.
  On English SciFact the same option is worth +0.0137.

**A script written without spaces is indexed in runs of
  characters.** Chinese, Japanese and Thai run their words together, and
  Korean hangs its particles on them, so a word is not a term there: a run of
  Han, kana or Hangul is indexed as its overlapping pairs of characters —
  `東京都に` is `東京`, `京都`,
  `都に` — and one of Thai, Lao, Khmer or Myanmar, whose
  characters are letters, as its triples. A query is split the same way, so
  `match body "東京"` finds the run it is written in. There is no
  dictionary and no language setting; a script that spaces its words keeps
  its words.

`chars` indexes each character of Han, kana and Hangul as
  well, so a query of one character finds every run holding it, which pairs
  alone do only for a run of one. It pays on short text and costs on long:
  product titles gain a fifth, news and questions lose a little, and the
  index holds 1.6 to 1.9× the postings for a query 3 to 8×
  slower. Off unless asked for.

| nDCG@10, BM25 | documents | words whole | runs of characters | with `chars` |

| C-MTEB EcomRetrieval, Chinese product titles | 100 902 | 0.006 | 0.439 | **0.528** |

| C-MTEB CovidRetrieval, Chinese news | 100 001 | 0.162 | 0.868 | **0.872** |

| WebFAQ, Chinese | 132 491 | 0.146 | **0.686** | 0.666 |

| JaGovFaqs-22k, Japanese | 22 135 | 0.119 | **0.582** | 0.576 |

| WebFAQ, Korean | 102 000 | 0.537 | **0.682** | 0.651 |

| WebFAQ, Thai (triples; pairs 0.495) | 47 429 | 0.448 | **0.547** | — |

The same pipeline as BEIR (`make beir` over each set in its
layout); English SciFact scores what it did, to the fourth decimal. A query
over those sets takes 0.09 to 0.66 ms at the median.

**A filter reaches an index only inside an `and` chain.**
  Equality — `=` and `in [..]` — reaches a
  `@hash` field; `in` is a set of equalities written
  short, so it is answered as the union of one bucket per element. Either
  reaches `id`, which the store's own id index answers without a
  `@hash` being declared or possible. Comparisons —
  `<`, `<=`, `>`, `>=`
  and `=` — reach a `@sorted` field, which narrows them
  to a range of keys.

Everything else — `!=`, `~`, `has`,
  comparisons on a field without `@sorted`, and anything under an
  `or` — is a full scan: each matching row is decoded and
  evaluated. `~` in particular is substring matching without
  ranking and reaches no index; ranked retrieval is `match` over a
  `@text` field.

A full scan reads rows in id order, and starts at the lowest id an
  `id > x` or `id >= x` in the `and`
  chain lets through, so a page read after the last id of the one before —
  `where id > $1 limit 1000` — reads only its own rows: a
  million rows read that way a page at a time took 8.6 s and take 110 ms.
  The browser module leaves this out.

## Changing a collection

A collection takes a field, lets one go, or renames one, and none of it
rewrites a document: it is as quick over a million documents as over
none.

```fenecql
alter collection orders add field note text
alter collection orders add field code text @unique
alter collection orders drop field note
alter collection orders rename field total to amount
```

- **A field added reads `null`** in every document
    written before it, and cannot be `required`.

- **A field dropped is gone** from every read and write, and its
    index with it. `compact` gives its space back.

- **A type does not change in place.** Add a field of the new type,
    `set` it from the old one, and drop the old one.

Details

A document is its values one after another in field order, so what a
change costs is what it does to that order. A field added goes last: a
document written before it ends before the field's place, and reads
`null` there. A rename is the schema alone. A field dropped
leaves its place in the documents written before the drop — skipped on
read, written as `null` after it, refused by name everywhere —
and `compact` writes every document without it, copying each
other value as its bytes stand. A field added under a dropped one's name is
a new field: the old values stay in the dropped place, and the new field
reads `null` until it is written. A change of type is the one
thing that would rewrite every document under the write lock, and is not
done.

Each is a write as a `create index` is: one of a block, put
back by its `ROLLBACK` with the writes after it, seen by a
subscriber as a schema change, applied by a replica, carried by the change
stream (`op: "alter"`, the documents after it read by the
schema it made). In the file it is record kind 12, and a dropped place a
nameless field of type tag 12, both of which a binary from before them
refuses rather than read the fields after a dropped one a place early. The
index of a field added is built over the documents there, every one of them
`null`; a dropped field's goes, and a renamed one's goes with
the name, a vector field's graph included.

Measured over a million rows in memory: an add, a rename and a drop take
0.025, 0.021 and 0.004 ms. Reading a field past where a document ends costs
nothing a scan can measure: five scans of the million took 21.2 to 39.1 ms
before the feature and 21.6 to 39.3 after, from 1.7% slower to 1.7% faster
by the scan — as the same build with the reads written as before was,
which makes it where the code lands, not what it does — and the browser
module's scan of 20 000 rows 1.70 ms against 1.71. A `compact` that takes
a dropped field out took 95 ms against 56 for one with none to take: it
writes each document again rather than copy it. On a server it holds the
write lock, where a compact otherwise runs beside the database.

## Objects and paths

A `json` field holds any value JSON does — an object, a list,
a number, text, `true`, `false` or
`null` — with no type declared for what is inside. A path
reads into it wherever a field is read.

```fenecql
create collection docs (title text, meta json)
put docs {title: "a", meta: {lang: "tr", source: {site: "x", rank: 3}}}

get docs where meta.lang = "tr" and meta.source.rank >= 2
get docs select title, meta.source.site order meta.source.rank desc
set docs {meta.lang: "en"} where title = "a"

create index on docs (meta.lang) @hash
create index on docs (meta.source.rank) @sorted
```

- **A path that leads nowhere is `null`**: a key a
    document does not have, or a value on the way that is not an
    object.

- **A path takes `@hash`, `@unique` and
    `@sorted`**, and they answer what the scan would, row for
    row.

- **`set` with a path changes one key** and keeps the
    rest of the object; it makes the object where there is none.

- **Typed fields are unchanged.** An `int` or a
    `text` field still refuses an object.

Details

An object's keys are kept sorted and unique: a key given twice — in
FenecQL, in a JSON body, in a parameter — refuses the write rather than
keep one of them, since which one a reader keeps differs from reader to
reader. Its numbers are kept as written, an integer an integer and
`19.99` the decimal it is, everywhere one comes in: FenecQL, a
parameter over HTTP and in the browser, a JSON body, a row
`fenec import` reads. A list of numbers alone is read the quick way, into the
`f32`s of the vector it nearly always is; where it is given a
`json` field, or compared with a path into one, the statement's
text or the parameter's JSON is read again as written, so
`[0.1, 12345678901]` is kept as those two numbers. A vector that
was never text — a `Float32Array` handed to the browser module,
a vector built through the Rust API — is refused there rather than kept as
numbers it was not given. A timestamp becomes its ISO text, as JSON writes
one; bytes, a sparse vector and a number that is not finite have no JSON
and are refused. A value nests at most 64 levels, and a path
names at most 64 keys past its field: past either a write or a query is
refused, never cut.

A path is a field's name, a dot, and keys, each a name as a field's is
(`meta.source.rank`); a key with a space or a dash in it cannot
be named, and is reached by reading the whole field. It works in every
comparison, `in`, `has`, `~` and
`is null`, in `select` — answering under the path's
text as its column's name — in `order`, and in
`set` and `put`, which set the one key inside the
field. A filter binds a path once a query, as it binds a field, and walks
its keys over each row's bytes, passing the members before the key without
decoding them. An aggregate and `group` read a field, not a
path; a `lookup`'s keys are fields too. Comparisons are the
ones every value already has: `3` equals `3.0`,
every number orders below every text, text below a list and a list below
an object, and `null` is equal only to `null`.

An index on a path is a field of its own beside the fields, named by the
path, and its kind is the field's: `@hash` keys a value as a
typed field's does, a whole float filed as the int it equals so that
`3` and `3.0` share a bucket; an equality or an
`in` reaches it with null, a boolean, text and a number an
`f64` holds exactly, and anything else — an int past
253, which equals floats near it that no bucket holds — is
answered by the scan. `@unique` refuses a value held twice as a
field's does, `3` and `3.0` the same value.
`@sorted` orders the numbers as `f64`s and the text by
its bytes, every number before every text; a value it has no place for —
`true` or `false`, a list, an object, an int past
253 — is held apart, and while the index holds one it answers
nothing and the scan does: a wrong order is never what an index costs.
`tests/json.rs` holds both to a twin collection without them,
every filter, order and page, through writes and a reopen. Text, vector
and sparse indexes stay on the fields that declare their type. On a server
an index on a path is built under the write lock, as it reads one value a
row: 11 ms for `@hash` and 21 for `@sorted` over
100 000 documents.

Renaming a `json` field renames the paths into it, indexes
and all, and dropping it drops them. In the file a `json` field
is type tag 14 and an object value tag 13 — not 11 or 12, which a schema
writes for a collation and a dropped place — and each index on a path is
written inside its field's entry, its keys and its kind: a binary from
before them refuses the file (unknown type tag 14) rather than read
it. The indexes are derived, as every hash and ordered index is: rebuilt
from the documents by the first read after an open.

Measured over 100 000 documents of a `title`, a
`year`, a `price` and a `meta` object of
three keys: `where meta.lang = "tr"` scans in 5.7 ms where the
same comparison of a text field takes 4.2; `meta.source.rank >=
990` in 7.2; both together in 10.4; the first 20 by
`meta.source.rank` in 7.3. With `@hash` and
`@sorted` on the two paths: 0.025, 0.009, 0.12 and 0.010 ms. A
scan of a collection with no `json` field takes what it took
before them (3.47 against 3.38 ms by two numbers, 4.17 against 4.21 by a
text, 3.06 against 3.08 by `~`, 2.72 against 2.60 for an
ordered page), and the browser module's filter 1.67 against 1.66.

## Points

A `geo` field holds a point on the earth, `[lon,
lat]` in degrees. `distance` measures in metres,
`within` tests a box, and `near` over the field puts
rows nearest first. The recipes, and what they cost against PostGIS and
Redis, are in [Points and distances](https://fenecdb.com/docs//docs/geo).

```fenecql
create collection places (name text, loc geo @geo)
put places {name: "Catania", loc: [15.087269, 37.502669]}
get places select name, distance(loc, $1) as m where distance(loc, $1) <= 2000
get places where within(loc, [12, 37, 16, 39]) count
get places select name near loc $1 limit 10
```

- **`distance(a, b)`** is the haversine on a sphere of
    Redis's radius, 6 372 797.56 m: Redis's `GEODIST`. PostGIS's
    sphere, the earth's mean radius, gives 0.03% less. Either side is a
    point field or a point; a null on either side is null.

- **`within(p, [west, south, east, north])`** holds a
    point on the box's edge; a west east of its east crosses the
    antimeridian.

- **`near loc p`** orders by distance, a tie by id, with
    the metres as `_score`; `exact` measures every
    row, and `ef`, a graph's, is refused.

- **`@geo`** narrows `distance(loc, p) <= r`
    (or `<`, either way round) and `within(loc, box)`
    in an `and` chain, and walks `near` nearest first.
    It is derived, as `@sorted` is: built by the first read after
    an open, nothing in the file but the field's declaration.

- **A point is refused** past 180 degrees of longitude or 90 of
    latitude, and a list of other than two numbers.

## Quantized vector indexes

`quant=` makes a vector index smaller by storing a compact code
for each vector. Scores stay exact, because the final order uses the full
vectors.

```fenecql
create collection articles (embed vector<768> @hnsw(cosine, quant=int8))
create index on articles (embed) @hnsw(cosine, quant=bit)
```

- **`int8`** is a quarter of the size; **`bit`**
    about a thirtieth, and cosine only.

Details

`quant=` sets what the index holds of each vector:
`int8` a byte a component, a quarter of `f32`, and
`bit` a bit a component, about a thirtieth: the signs of how far
the vector lies from the nearest of 128 centres the index learns from its
first 2 048 vectors, which it holds whole until then — cosine only, since
the vectors are taken at unit length. The documents keep their vectors whole. The codes
find the candidates, the beam's `ef` of them, and the documents'
own vectors put those in order, so `_score` is the exact distance.
An int8 code knows how far off it can be, so a candidate is read only while
it can still make the page; a bit code does not, and the whole beam is read.
A filtered set small enough to search without the walk is ranked by its codes
the same way, and `exact` reads every vector. Bit codes take a
wider beam: `ef_search` is 200 there unless it is named. What they save and what they cost is in
[How it works](https://fenecdb.com/docs/concepts#quantized-indexes).

## Sparse vectors

`sparse<N>` stores the output of a learned sparse model
such as SPLADE. `near` over it finds the documents with the
largest dot product, exactly.

```fenecql
create collection passages (body text @text, splade sparse<30522> @inverted)
put passages {body: "...", splade: "{1037:0.84,2003:1.27,7592:2.1}/30522"}
get passages near splade $1 limit 10
get passages match body $1 near splade $2 fuse limit 10
```

- **It is written as pgvector's `sparsevec`**:
    `{index:value,...}/N`, indices from 1.

- **Index it with `@inverted`.**

Details

`sparse<N>` holds a vector of `N` dimensions by
its non-zero entries alone: what a learned sparse model such as SPLADE gives a
text, a weight for a few dozen or hundred of its vocabulary's 30 522 words. It
is written and read in pgvector's `sparsevec` text form,
`{index:value,...}/N` with the indices counted from 1, as a string
in JSON and in FenecQL; entries come back in index order and
zeros are dropped. `@inverted` indexes it as a word index is
indexed, dimension by dimension, and `near` over it returns the
documents with the largest dot product — the exact ones, walked with the
pruning `match` uses — and ranks only documents sharing at least one
dimension with the query. A filter is tested as the lists are merged;
`exact` scores every document instead, and `fuse` takes a
sparse `near` as it takes a dense one.

## Ordered indexes

`@sorted` makes ranges, counts over a range and
`order … limit` fast. It always gives the same rows as reading
every row would.

```fenecql
get articles order published desc limit 20
get articles where published >= "2026-09-01" order published desc limit 20
```

- **It costs memory and write time**: about 20 bytes a row for a
    number, plus the text for `text`.

- **It is rebuilt after an open** by the first query that needs it; it
    is never written to the file.

Details

```fenecql
get articles where title >= "k" and title < "l" count
get articles where published >= "2026-01-01" near embed $1 limit 10
```

`order` over a `@sorted` field with a
`limit` walks the index in order and stops at the page, testing
the rest of the filter on each row it passes; without one, every match's key
is read and only `offset + limit` of them are put in order. A
filter that matches too little to fill the page within an eighth of the
collection sends the walk back to the scan: over a million rows one matching
nothing cost 152 ms against the scan's 121, while one matching 1% fills the
page in 0.29 ms against 125. A range on a `@sorted` field is read
from the index as a set of ids — for a filter, a `count`, or the
allowed set of a filtered `near`.

| 1 000 000 rows unless noted | scan | `@sorted` |

| `order created desc limit 20` | 26.8 ms | 0.001 ms |

| the newest 1%, `order created desc limit 20` | 63.1 ms | 0.001 ms |

| `where price >= 99900 limit 20`, 0.1% of the rows | 1.44 ms | 0.008 ms |

| the same range, `count` | 65.8 ms | 0.008 ms |

| `where pct < 1 near embed $1 limit 10`, 1% of 200 000 × 128 | 17.9 ms | 0.16 ms |

A comparison with `now()` — `run_at <= now()`,
`at >= now() - 3600000` — is a range of the index as one with a
literal is: the time is worked out once, before the query is planned, and
is the statement's one time. Left a call, a queue's claim walked past its
ready jobs into the leased and delayed ones after them, testing each.

The planner prefers a smaller set when it has one: an equality over a
`@hash` field, or a narrow range on another `@sorted`
field, names fewer rows than the walk would pass, so those are sorted instead.
A range wider than half the collection is left to the scan, which is cheaper
than gathering and sorting that many ids.

**A comparison reaches the index only when its literal says exactly
  what the scan would compare.** `price < 12.5` on an
  `int` field, or an integer past 253 against a
  `timestamp`, is evaluated row by row — the index narrows with the
  comparisons it can express and the rest are tested — because an answer that
  differs from the scan's is a wrong answer, not a fast one. Ties come out in
  ascending `id` either way, as the scan leaves them.

The cost is paid on write and in memory: about 20 bytes a row for a
  number, plus the text for `text`, and an insert into a sorted
  run. Over a million rows two ordered indexes took a bulk load from 0.52 s
  to 0.88 s, and the index is built from the documents by the first query
  that reads it after an open, like the hash and text indexes — it is never
  written to the file.

## Indexes over several fields

`@sorted(owner, created desc)` keeps the rows in the order of
several fields, the first first. The page an app reads most — one user's
latest twenty, a tenant's inbox, a feed — is a run of it, read to the page
and no further. `@unique(org, slug)` refuses a second row holding
the same values in every part.

```fenecql
create collection posts (
  owner    int,
  created  timestamp,
  title    text,
  @sorted(owner, created desc)
)
get posts where owner = $1 order created desc limit 20     -- a run of the index, 20 rows read
get posts where owner in [$1, $2, $3] order created desc limit 20   -- three runs, merged
get posts where owner = $1 and created >= $2 count          -- counted off the index
create index on pages (org, slug) @unique                   -- or @unique(org, slug) in the collection
```

- **Equalities on its first fields, then a range or an order on the
    next.** `where owner = $1 order created desc` reads its
    run; `where created > $1` alone does not use
    `(owner, created)`, since `owner` comes first.

- **An order matches it** in its fields' directions, or every one the
    other way round: `(owner, created desc)` answers
    `order created desc` and `order created` for one
    owner.

- **It gives the scan's answer, row for row**, ties in ascending
    `id` either way, as `@sorted` does.

- **A `lookup` keyed by its first fields** reads each
    parent's children as a run of it, walked in the level's order where
    that is the index's next parts:
    [`lookup lines on (org, order)
    = (org, id) order line limit 5`](#a-key-over-several-fields).

- **Up to 8 fields** of `bool`, `int`,
    `float`, `timestamp`, `text` (in its
    collation) or `json`, a path into one among them
    (`meta.lang`). A field keeps its own index beside it.

Details

```fenecql
create index on posts (owner, created desc) @sorted        -- built beside the database on a server
drop index on posts (owner, created)                        -- by its fields
explain get posts where owner = 7 order created desc limit 20
-- order: walked the index on (owner, created desc), owner fixed, 20 of 100 rows read, 20 kept
```

A key is its fields' values laid end to end, each written so that the
bytes order as the values do — an int's sign turned over, a float's bits
made to order, a text escaped and ended, a descending field's bytes
inverted — so an equality on its first fields is a run of the keys and a
range on the next one is a run inside it. A text in a collation is kept as
written and compared in it. `null` stands below every value of a
field, as `order` puts it; a row holding a float's
`NaN`, which compares equal to everything, or a json value no
order has a place for (a boolean, a list, an object, an int past
253) is held apart: while one is, the index is walked for no
`order`, and a filter's run takes those rows with it, each
tested.

The planner takes the `and` chain's terms in the index's order
— an equality, or an `in` whose every element it can say exactly,
on each field as far as one is fixed, then the comparisons on the next — and
tests every other term row by row; a term it cannot say exactly (a
`null`, `12.5` against an `int`) is left to
that test, so the answer never changes. An `in` makes a run for
each element, 256 at most together; under an `order` on the
fields after it the runs are walked together, merged by those fields. Of the
plans, the narrower set wins by exact counts: the run is two searches, a
`@hash` bucket its length, so a filter the index says whole reads
the page and nothing more, and one it does not is walked only while its run
is no longer than any set another index names — and gives up past an eighth
of the collection, as a `@sorted` walk does. A
`count` the index says whole is counted off it without a row
read. `explain` names the index, the fields fixed, the runs and
the range.

`@unique` over several fields is asked before every
`put`, `insert` and `set`: a second row
holding the same values in every part is refused,
`409` over HTTP, naming the fields and the other row — to a
scoped token the fields alone. A `null` in any part is no value.
`put ... if absent` passes over the row holding them, and
`if absent else set` sets it. `create index ...
@unique` over rows already holding a key twice is refused naming two of
them.

It is derived data, built from the documents by the first read after an
open, kept on every write, put back with a block that does not land, and
never written to the file. A field renamed keeps it under the new name; a
field dropped takes it off. A sync replica builds what the schema declares,
a unique one as an ordered one, as its `@unique` fields are
`@hash`.

| 1 000 000 posts over 10 000 owners | p50 | p99 | built in | memory |

| `where owner = $1 order created desc limit 20`, `@sorted(owner, created desc)` | 0.006 ms | 0.011 ms | 147 ms | 40 MB |

| the same over `owner @hash`, its bucket sorted | 0.012 ms | 0.017 ms | 75 ms | 11 MB |

| the same over `created @sorted`, walked and given up | 24.4 ms | 40.5 ms | 34 ms | 20 MB |

| the same by the scan | 21.3 ms | 37.4 ms | — | — |

| a feed, `owner in` ten owners, `@sorted(owner, created desc)` | 0.022 ms | 0.029 ms |  |  |

| the feed over `owner @hash` | 0.100 ms | 0.136 ms |  |  |

Writes pay for it: a lone `put` into those million rows took
715 ns with no index, 851 beside `created @sorted` and
`owner @hash`, and 1 561 beside `@sorted(owner, created
desc)`, whose first field puts each write anywhere in it. A unique one
asks its run before each write: 1 745 ns a put against 1 285 for
`@unique` on the two joined into one field, and a read by both
fields 2.5 µs against 1.3 by the joined field — a hash index
over several fields would answer that read sooner, by a microsecond, and was
not worth a third index's code beside this one (`make
composite-bench`).

## Rows that expire

`@ttl` on a `timestamp` field makes a row expire that
long after its value — for sessions, caches and anything else kept for a
while.

```fenecql
create collection sessions (user text @hash, seen timestamp @ttl(30m))
```

- **Reads are exact at once.** A row past its time is never returned,
    even before it is deleted: every read of the collection leaves it out.

- **The server deletes it.** A primary sweeps once a minute, as
    ordinary deletes that replicas, `/_changes`, archives and
    subscriptions see as any.

- **`expired()` reads what the others leave out,** for
    a reaper that gives back what a row held before the sweep deletes it
    ([below](#reaper)).

- **Durations** are whole `ms`, `s`,
    `m`, `h` or `d`: `@ttl(45s)`,
    `@ttl(12h)`, `@ttl(7d)`.

Details

```fenecql
put sessions {user: "ada", seen: now()}
set sessions {seen: now()} where user = "ada"        -- seen again: lives on
get sessions where user = "ada"                       -- nothing once 30 minutes pass
create index on events (at) @ttl(7d)                  -- an existing timestamp field
alter collection sessions alter field seen @ttl(1h)   -- another expiry
alter collection sessions alter field seen @sorted    -- none: the rows live
```

A read of the collection — `get`, `count`, an
aggregate, `near`, `match`, a `lookup` level,
an inner `get`, a `set` or `del` by filter,
and a subscription's shape — has `not (seen <= now - ttl)`
added to its filter, the way a token's rule is added: a row is gone at the
moment its time plus the ttl is reached. A row whose field is
`null` has no time to expire from, and lives. In the browser the
module has no clock: `now` is the page's, `Date.now()`
unless `db.now` is set, as every time is handed to it; the page
deletes nothing, and a row past its time stays in its file, out of every
read. A live query in the page is woken by writes, not by time passing.

`@ttl` is an ordered index whose rows expire, so the field
also does everything `@sorted` does, and a collection's rows
expire by one such field. The sweeper deletes a range
of it at a time rather than scan for the rows past their time: found under
the read lock, deleted under the write lock 1 000 at a time, each batch a
block of ordinary deletes, its fsync run once the lock is let go. A replica
never sweeps — its writes are its primary's, the sweep's among them — and
neither does a tenant that follows another node. The server's
`Idempotency-Key`s are kept this way.

The ttl is a kind of index in the file (kind 9, the milliseconds behind
it), so a binary from before it refuses the file rather than hand out the
rows past their time. A write takes the place of a row past its time:
an `insert` naming its id writes over it, and a
`@unique` value it holds is let go of with it, the row deleted
in the same statement — so a lock taken by `insert` and left to
expire can be taken again at once ([Instead of
Redis](https://fenecdb.com/docs//docs/redis#lock)). BM25's statistics count the row until the sweep.

### Reading what expired: the reaper

A row past its time is out of every read, so a hold that lapsed cannot be
found to give its money back, nor a reservation its stock. `expired()`
in a filter reads those rows — and only those, where the filter says so —
wherever a filter goes: `get`, `count`, a
`lookup` level, an inner `get`, `set` and
`del`. It is `until <= now - ttl`, a range of the
field's ordered index; `not expired()` is what a read finds without
it.

```fenecql
create collection holds (account text @hash, amount int, until timestamp @ttl(1ms))

get holds select id, account, amount where expired() limit 100   -- what lapsed

-- each given back once, in one block (a /batch): the delete is the guard
del holds where expired() and id = $1 require 1
set accounts {held: held - $2} where ext = $3 and held >= $2 require 1
```

A capture needs no guard of its own against the lapse:
`get holds where id = $1 limit 1 require 1` finds no hold past
its time. Two reapers, or a reaper and a capture, racing for one hold: the
`del ... require 1` lets one through, and the other's block is put
back whole. The sweeper leaves a row a minute past its time before it deletes
it, so a reaper that comes by once a minute sees every one; a row it misses
for longer is deleted unseen, and what it held is the reconciliation's to
find. A scoped token reads them only by the policy's `expired`
grant ([Scoped access](https://fenecdb.com/docs//docs/http#scoped-access)). Without
`@ttl` on the collection, `expired()` is an error.

**What it costs, measured** (`make ttl-bench`, a million
  rows, half past their time). A read tests each row's time as part of its
  filter, as the same test written by hand does: `count` becomes a
  scan, 68.8 ms against 1.1 with no expiry, and a filter that scanned
  already reads one field more, `where s = "w7" count` 68.5 ms
  against 51.9 — 70.1 with the test written out. A collection with no
  `@ttl` reads as it did. The sweep of 100 000 rows past their
  time out of a file of 200 000 took 0.56 s in batches of 1 000, each
  holding the write lock 1.06 ms at the median and 2.3 at most.

## Collation

Text sorts by its bytes unless you ask otherwise, which is wrong in every
language. `collate und` sorts it the way people expect in any
language, and `collate tr` in Turkish order.

```fenecql
get people order name collate und limit 20
get people order city collate tr desc, name collate tr
get people select city, count(*) group city order city collate und

create collection people (name text collate und @sorted, city text)
get people where name > $1 order name limit 20   -- the page after $1
```

| `order name` | `order name collate und` | `order name collate tr` |

| Irmak, Zeynep, cem, Çağla, İlker, ılgaz | Çağla, cem, İlker, Irmak, ılgaz, Zeynep | cem, Çağla, ılgaz, Irmak, İlker, Zeynep |

| Zoë, anna, Ζωή, 山田, Борис, Émile | anna, Émile, Zoë, Ζωή, Борис, 山田 | anna, Émile, Zoë, Ζωή, Борис, 山田 |

- **Name it on one key** in `order`, or **declare it on a
    field** so every comparison of that field uses it.

- **Equality still compares bytes**, unless the field is
    `nocase` ([Case-insensitive text](#nocase)).

- **In the browser**, data for scripts other than Latin is fetched when
    first needed.

Details

Text is ordered by its bytes unless a key says otherwise, and the bytes are
wrong in every language: every capitalised word comes before every lowercase
one, an accented letter lands after `z`, and every script past
ASCII falls where its code points do. `collate und` orders the key
as Unicode's root collation does — CLDR's, which PostgreSQL calls
`und-x-icu` — the order every language's starts from: each script
in a run of its own, its letters in their order, accents and case deciding
only between words whose letters are the same. A language whose letters sort
otherwise is a tailoring over it, and `collate tr` is Turkish,
PostgreSQL's `tr-x-icu`: `ç` after `c`,
`ğ` after `g`, `ı` before `i`
with `I` as its capital and `İ` as the capital of
`i`, `ö`, `ş` and `ü` after
`o`, `s` and `u`.

It compares a level at a time over the whole string, as ICU does: the
letters, then their accents (`kar` < `kâr` <
`kara`), then their case (`ince` < `İnce`),
so an accent or a capital only decides between strings whose letters are the
same. What all three leave equal goes by its bytes, as under a deterministic
PostgreSQL collation — the order is total, and a page never changes with the
plan. It applies wherever `order` does: the query's rows, a
`lookup`'s children, an aggregate's groups by their key or a
`min` or `max` of text. A key that holds no text —
a number, `id`, `count(*)` — refuses it rather than
ignore it.

A field can be declared in a collation, `name text collate
und`, and its text is in it wherever it compares: `order`
without naming one, `<`, `<=`, `>`
and `>=` in `where`, `min` and
`max`, and a `@sorted` index over it, which holds that
order and is walked for it. So a listing pages by the last name it
showed — `where name > $last order name limit 20` — and the
index answers as the scan does, row for row. Equality stays the bytes', since
the collation ties no two different strings. The collation is the field's:
it is written with the schema into the file, and `describe`
shows it. A
`@sorted` field without one keeps its bytes' order, so a key
collated in the query is sorted rather than walked, and a comparison in
`where` compares bytes.

Over a million Turkish names (160 797 of them distinct) a comparison costs
33.5 ns in `tr` and 35.2 in `und` where the bytes cost
6.7, because it starts where two strings part — `Ahmet Yılmaz`
against `Ahmet Kaya` is `Y` against `K` —
which took it from 68 ns, and a Latin letter's weights are made once per
collation and read, where looking each up through the tailoring and the
root's tables took it to 41:

| 1 000 000 names | bytes | `collate tr` | `collate und` | PostgreSQL, `"C"` | PostgreSQL, `"tr-x-icu"` |

| a page, `limit 20` | 94.7 ms | 124 ms | 130 ms | 116 ms | 136 ms |

| all of them in order | 374 ms | 890 ms | 925 ms | 506 ms | 1 208 ms |

fenecdb in process, PostgreSQL 17 by its own execution time with the sort
in memory; both on one core (`make collate-bench`).

**The weights are ICU's own, for every assigned code point.**
  `tools/collate/gen.py` reads them out of ICU 76.1 into
  `crates/fenec-core/src/collate/`: the root's cut into chunks by
  script, and each tailoring as what it changes. Han ideographs sort in the
  radical-and-stroke order ICU's root gives them, those of the other planes
  among them; a Hangul syllable sorts as its jamo; and two or three
  characters ICU sorts as one sort as one — a letter and a mark it composes
  with (`c` and U+0327 is `ç`), a vowel sign written in
  two parts, a Thai or Lao vowel written before the consonant it follows in
  speech. An unassigned or private code point goes where ICU puts one. Checked
  against ICU itself over 1 250 000 random pairs in every script, of which
  five sort apart, and against `Intl.Collator("und")` over two thousand
  words in the browser module on every test run. What it does not do is ICU's
  normalisation: a mark that composes with its letter across another mark
  between them, or with marks out of their canonical order, sorts as a mark
  of its own — those five.

ICU numbers every ideograph and most alphabets' letters one by one —
  133 535 primary weights, which no packed entry has the bits for — so
  consecutive code points that alone give consecutive weights share a rank
  and are told apart by their code points, and a run of them is held as one
  range; a table's words are written as differences in LEB128 besides:
  154 KB for every script, 324 KB plain, which a native build carries whole
  — 161 KB of the command line's 1040.

The browser module carries the Latin chunk and fetches the others as a
page first needs them, from `collate/` beside the module (JavaScript). A statement that would
compare text of a script whose data it has not got is refused, the chunk
named, rather than answered in another order; a write is refused before it
changes anything, and an image whose collated text needs a chunk loads once
it is there — a `@sorted` index built without it would be out of
order.

| chunk | bytes | gzip | brotli | holds |

| `latin` | 9.0 KB | 4.9 KB | 4.2 KB | ASCII, Latin-1, Extended-A to -C, Vietnamese, punctuation, currency; in the module |

| `greek`, `cyrillic` | 3.6, 2.1 KB | 1.6, 1.0 KB | 1.4, 0.9 KB | Greek with polytonic, Cyrillic |

| `middle-east` | 7.3 KB | 3.6 KB | 3.3 KB | Armenian, Hebrew, Arabic, Syriac, Thaana, N'Ko |

| `indic` | 3.0 KB | 1.6 KB | 1.5 KB | Devanagari to Sinhala |

| `southeast-asia` | 8.5 KB | 2.2 KB | 2.2 KB | Thai, Lao, Tibetan, Myanmar, Khmer, Javanese … |

| `east-asia` | 8.2 KB | 4.6 KB | 4.2 KB | Hangul, kana, Bopomofo, Yi, full-width forms |

| `han`, `han-ext` | 31.1, 59.0 KB | 14.3, 27.3 KB | 12.7, 24.9 KB | ideographs of this plane, of planes 2 and 3 |

| `symbols` | 6.9 KB | 2.8 KB | 2.6 KB | arrows, mathematics, emoji, mathematical letters |

| `other`, `smp` | 5.7, 9.8 KB | 3.0, 5.0 KB | 2.8, 4.8 KB | every other script, of this plane and the next |

The root and its chunks made the browser module 17.5 KB larger, 8.2 KB
under gzip and 6.8 KB under brotli: the Latin chunk, the Turkish tailoring
over it, and the code that reads them and has the others fetched. Every
other chunk together is 145 KB, 67 KB gzip, 61 KB brotli. Their words are
written as differences, which gzip and brotli take a third off: plain they
were 102 and 90 KB. The Latin chunk alone comes out 0.3 KB larger under
gzip that way, 0.9 KB under brotli, and 2.9 KB smaller in the module.

## Case-insensitive text: `nocase`

A text field declared `nocase` compares by its case fold
wherever it is compared, and keeps the text as it was written: an email
under `@unique` takes `Alice@x.com` once, and finds it
as `alice@X.COM`. It is PostgreSQL's `citext`, with
Unicode's case folding where `citext` has `lower()`.

```fenecql
create collection users (email text nocase @unique, name text collate tr nocase @sorted)
put users {email: "Alice@Example.com", name: "İsmail"}

get users where email = "alice@example.COM"    -- the row, its email as written
put users {email: "ALICE@example.com"}          -- refused: users.email is unique
put users {email: "alice@example.com"} if absent   -- passed over, and not counted
get users where name = "ismail"                 -- the row: İ is the capital of i in Turkish
get users select name order name limit 20       -- in Turkish order, case aside
```

- **Everything that compares it folds it:** `=`,
    `!=`, `in`, `<` and
    `>`, `~`; `@hash`,
    `@unique`, `@sorted` and an index over several
    fields; `order`, `group`,
    `count(distinct …)`, `facet`, `min`
    and `max`; a `lookup` on it; a scoped token's
    rules over it.

- **It is kept as written.** A row reads back as it was put. A group
    or a facet value is one fold, shown as the first row holding it — in id
    order — wrote it.

- **Unicode's full case folding:** `ß`, `ẞ`,
    `SS` and `ss` are one, a final `ς` is
    `σ`, `fi` is `fi`. In `collate
    tr`, Turkish's: `I` is the capital of `ı`,
    `İ` of `i`.

- **Case alone.** Accents, width and how a text is composed are not
    case: `résumé` is not `resume`, `Ａ` is
    `ａ` and not `A`, and `é` as one code
    point is not `e` and a combining accent.

- **A text field's.** Not a list's or a `json` field's.
    Made `nocase` or no longer, a field is built again — the
    schema check's `rebuild` — which a `@unique` one
    refuses when two of its values would become one.

Details

The fold is Unicode's full case folding, the `C` and
`F` mappings of CaseFolding.txt, which Python's
`str.casefold()` is too, and it is held to that at every code
point (`crates/fenec-core/tests/nocase.rs` reads Python's own
fold of each). Full rather than simple folding, which maps one code point to
one: simple folding keeps `straße` apart from
`STRASSE`, which ICU's case-insensitive comparison and every
German reader find equal. A field in `collate tr` folds by
Turkish's `T` mappings over it, as a Turkish reader expects:

|  | `nocase` | `collate tr nocase` |

| `Alice`, `ALICE`, `alice` | one | one |

| `straße`, `STRASSE`, `STRAẞE` | one | one |

| `ΟΔΟΣ`, `οδος`, `οδοσ` | one | one |

| `ᏣᎳᎩ`, `ꮳꮃꭹ` (Cherokee) | one | one |

| `ISTANBUL`, `istanbul` | one | apart |

| `ISTANBUL`, `ıstanbul` | apart | one |

| `İstanbul`, `istanbul` | apart: `İ` folds to `i` and a combining dot, as its decomposed spelling does | one |

| `ＡＢＣ`, `ａｂｃ` / `abc` | one / apart | one / apart |

| `résumé`, `resume`; `é`, `e`+U+0301 | apart | apart |

A `nocase` field in a collation orders by the collation over
the folds — `collate tr nocase` puts `ılık` before
`İnce` and finds `ince` equal to the second — so its
order and its equality agree: a collation tells every two different texts
apart, and so it ties exactly the texts that fold to one, which come out
in id order. Without a collation it orders by the folds' bytes. A query's
`collate` orders it in another collation, still by its folds.
`~`, which ignores case on any text by lowercasing both sides,
asks of a `nocase` field whether its fold holds the needle's:
`straße ~ "SS"`.

Its indexes hold the folds: a `@hash` or `@unique`
bucket every spelling of one text, a `@sorted` index and an
index over several fields their order, so an equality, an
`in`, a range and an `order … limit` walk are answered
as the scan answers them, row for row — a test holds every filter, order,
group, facet and unique index over mixed-case data in ASCII, Turkish,
German, Greek, Cherokee and full width to a twin collection holding Python's
folds in plain fields. A `lookup` finds a child by its key as the
child's field compares: a `nocase` one by the parent's value's
fold. The schema writes the word into the file (a binary from before refuses
it), `describe`, `GET /_schema?as=fenecql` and the
declared schema carry it, a replica — a server's or a synced page's —
compares as the primary does, and `fenec import` takes
PostgreSQL's `citext` and SQLite's `COLLATE NOCASE`
as it.

What folding costs, over 100 000 rows (`make nocase-bench`:
a plain field holding one spelling and asked it, a `nocase` one
holding a mix and asked another; names a third past ASCII):

| 100 000 rows | plain | `nocase` |

| a put, asking `@unique` | 1 209 ns | 1 200 ns |

| an email by `@unique` | 1.08 µs | 1.16 µs |

| a city's rows counted by `@hash` | 3.15 µs | 3.27 µs |

| a name by the scan | 5.47 ms | 6.79 ms |

| a name's part by `~` | 12.7 ms | 10.4 ms |

| an order by name, a page, by the scan | 8.48 ms | 16.2 ms |

| rows by city, `group` | 7.22 ms | 7.23 ms |

| a facet of names by the scan | 6.89 ms | 8.32 ms |

| `in` a list of 10 names, by the scan | 6.90 ms | 11.9 ms |

An index lookup folds the one value it is asked. A scan folds a row only
as far as it differs from the value — mostly its first character — with
nothing allocated; `~` folds it whole, and still comes out
quicker than lowercasing both sides. An order folds each row's key once,
then sorts the folds; a `@sorted` index holds them folded and is
walked instead. The fold is the standard library's lowercase mapping, and
for the 297 code points whose fold is not their lowercase — 41 runs — its
lowercase, uppercase and lowercase again: no table of its own beyond those
runs. The browser module grew 11.5 KB, 5.0 KB gzip, 3.7 KB brotli — the
engine's indexes, filters, groups, facets and orders 6.4 KB of it, the fold
and its comparisons 2.1 — and a collection without a `nocase`
field reads and writes as it did.

## Columns worked out

An item of the select list can be an expression over the row: its
fields and paths, parameters and functions, with `+`
`-` `*` `/`. It answers under the name
`as` gives it; a field can be named again the same way.

```fenecql
get ticks select sym, px * qty as notional where qty > 100
get ticks select px as price, bucket(at, 1m) as minute, greatest(px, $1) as floor
get orders select id, case when total >= 100 then "large" else "small" end as size
```

Details

The types are a `set`'s ([Writes that read
the row](#writes)): two `int`s make an `int`, and one past
64 bits is refused, never wrapped; a null is null. An expression is bound
to the collection once a query — each field it reads by its place, every
field the list reads decoded in one pass a row — and its error comes at
the first row it fails on. An item that is neither a field nor an
aggregate of one has no name of its own: `select px * qty` is
refused, and asks for `as`.

## Aggregates

Put `count(*)`, `count(…)`, `count(distinct …)`,
`sum`, `avg`, `min`, `max`,
`first` or `last` in the select list to get totals.
Add `group` for one row per distinct set of its keys' values;
a list of the keys alone, `select city group city`, answers each
set once, as SQL's `select distinct city` does.

```fenecql
get orders select count(*), sum(total), avg(total), min(at), max(at)
get orders select status, count(*), avg(total) where year = 2026 group status
get orders select customer, sum(total) group customer order sum(total) desc limit 10
get events select bucket(at, 1h) as hour, name, count(distinct user) as users group hour, name
get ticks select bucket(at, 1m) as bar, first(px by at) as open, max(px) as high, min(px) as low,
  last(px by at) as close, sum(qty) as volume where sym = $1 group bar
get ticks select sym, sum(px * qty) / sum(qty) as vwap group sym
get events select props.plan, sum(props.v) group props.plan
```

- **Nulls are skipped**, as in SQL, by everything but
    `count(*)`.

- **An aggregate folds any expression** — `sum(px * qty)`
    — and an item can be an expression over aggregates and keys —
    `sum(px * qty) / sum(qty)`.

- **Groups can be ordered and paged** by any column of the list.
    Without `group`, `order`, `limit` and
    `offset` pick the rows the one row folds: `get deliveries
    where courier = $1 order at desc limit 50 select count(*), avg(km)`
    is the latest 50.

- **A `lookup` hangs off the groups**, keyed by a column
    of the list, and a [lookup level's list may
    aggregate](#lookup-folds), each parent's children folded.

- **Aggregates do not combine** with `near` or
    `match`.

Details

A select list with an aggregate in it folds the rows the query finds —
through the same indexes any `where` reaches — into one row, or,
with `group`, one row per distinct set of the keys' values. Each
column answers under its name, in the order the list is written: the name
`as` gives it, or for a field or an aggregate of one the text it
was written as — `sum(total)`, `count(distinct user)`,
`first(px)` (its `by` left out). Anything else needs
`as`.

- **Keys.** `group` takes one key or several, each a field,
    a path, or an expression — `group sym, bucket(at, 1m)`. A
    name the list gives an item with `as` stands for that item,
    before a field of the same name: `select bucket(at, 1h) as hour,
    count(*) group hour`. A plain item of the list is a key itself, or
    is refused as neither aggregated nor grouped by; an expression written
    out again in `group` is not matched to the list's — name it
    once, with `as`.

- **`count(*)`** counts rows. **`count(distinct
    e)`** counts the values `e` takes, a null none, two
    values the same when their encodings are — as `group` tells
    its keys apart. It is exact: every value is held once a group, up to a
    million over every group and item of the query — about 70 MB — and a
    query past that is refused, never counted short.

- **`sum` and `avg`** take numbers: an
    `int` sum is an `int`, and one past 64 bits is an
    error rather than a wrapped number; a float among the values makes the
    sum a `float`, and an average is one. **`min` and
    `max`** take any value with an order — numbers,
    `timestamp`, `text`, `bool` — a text
    field's in its collation. A field whose type cannot be folded is refused
    before any row is read, an expression's value at the first row that
    cannot.

- **`approx_count_distinct(e)`** counts distinct values in a
    HyperLogLog sketch of 214 registers, 16 KB a group at most,
    with a standard error of 0.81% at any count and no million to stop at.
    `hll_accumulate(e)` answers the sketch as bytes,
    `hll_combine(s)` merges sketches and the function
    `hll_estimate(s)` reads a count off one, so days kept as
    sketches count a month. A query's sketches are held to 64 MB.

- **`having <expr>`** after `group` keeps
    the groups it holds for, before `order` and
    `limit`: it reads the keys, the list's columns by name and any
    aggregate, `group user having count(*) >= 2`. `count`
    after it answers how many groups passed.

- **`first(e by k)` and `last(e by k)`** are
    the value of `e` on the group's row least, or greatest, by
    `k` — a bar's open and close are `first(px by
    at)` and `last(px by at)`, whatever order the ticks
    were written in. A tie on `k` goes to the row written first
    for `first` and last for `last`; without
    `by`, the order is the one the rows were written in. A row
    whose value or key is null is passed over, as every aggregate passes a
    null over. The query's `order` orders the groups, not the
    rows inside them, so it says nothing to these.

- **Nulls are skipped**, as SQL skips them, by everything but
    `count(*)`. Over no rows — or only nulls — a sum, an average,
    a minimum and a maximum are `null`, not 0.

- **Groups** come out by their keys, in the order `group`
    names them, the null key first, unless `order` names a column
    of the list — `order sum(total) desc`, `order
    vwap` — and `limit` and `offset` page them.

- **Ungrouped**, a list answers one row, which an order or a page
    would have nothing to do with; so there `order`,
    `limit` and `offset` pick the rows it folds, as
    they pick a `get`'s page — an `@sorted` field's
    index walked to it — and the one row folds those: `order at desc
    limit 50 select avg(km)` is the mean of the latest 50, what SQL
    writes as a subquery. Its keys are the rows' values; a column of the
    list among them is refused.

- **A `lookup` after the list** hangs off its rows — the
    groups, or the one row — and names a column of the list as the
    parent's key: `get orders select courier, count(*) as n group
    courier order n desc limit 10 lookup couriers on user = courier select
    name` is the ten busiest couriers with their names. The column
    holds a field's values: a key, its `bucket`, its
    `min`, `max`, `first` or
    `last`; a sum's or a count's is refused, having no type to
    hold the child's key to. `required` there drops the groups
    no child matches, before `order` and `limit`, and
    `count` after `group` counts those kept.

- **What does not combine** is what ranks rows an aggregate
    collapses: `near` and `match`.

**How it runs.** Every item is bound to the collection once a query:
each field and path the list reads is decoded in one pass a row, a field
folded as it is is read with nothing copied, and a `bucket` over
a constant interval works the interval out once. Rows written in time order
come a bucket at a time, so the group the row before went to is asked
first. A `count(*)` by `bucket` of a
`timestamp` or `int` field with an ordered index
(`@sorted`, `@ttl`), whose filter is a range of that
field and nothing more, reads the index's keys and no row; `explain`
says so.

A million rows in 100 groups, `select cat, count(*), avg(price) group
cat`, took 80 ms in process where PostgreSQL 17 — one worker, as
`make pgvector-up` runs it — took 189 ms of execution time on the
same rows; the whole set's count, sum, minimum and
maximum, 37 ms; the same groups over the tenth a `@sorted` range
found, 9.3 ms. The rows are read, not an index: a filter narrows them, nothing
precomputes the totals. The questions analytics and market data ask —
bars, VWAP, distinct users, counts by the minute — are on Analytics and market data, each with what it
costs over a million rows.

## Functions over a row

Functions work wherever an expression goes: `where`, a
`set`'s values, a select list's items and a group's keys.

```fenecql
get events select bucket(at, 15m) as q, count(*) group q
get events where bucket(at, 1d) = "2026-05-03" count
set bars {high: greatest(high, $1), low: least(low, $1)} where key = $2
get orders select id, case when total is null then 0 when total < 0 then -1 else 1 end as sign
```

- **`bucket(ts, interval)`** is the start of the interval
    a `timestamp` falls in, in UTC: `ms`,
    `s`, `m`, `h` and `d` count
    from the epoch, `w` from a Monday as ISO's weeks begin, and
    `mo` and `y` by the calendar —
    `bucket(at, 3mo)` is a quarter. The interval is written as a
    duration, `15m`, or passed as a parameter holding one as
    text or a number of milliseconds. An `int` of epoch
    milliseconds gives an `int`.

- **`bucket(ts, interval, zone)`** is the same by a zone's
    clock: an IANA name, `bucket(at, 1d, 'Europe/Istanbul')`, in
    any case, or an offset written out — `'+03:00'`,
    `'-0530'`, `'+03'`, `'Z'`,
    `'UTC'` — or a parameter holding either, which is how a
    page passes its own. A day, a week, a month or a year starts at local
    midnight — the first instant whose clock reads that date, where a clock
    skips midnight the instant it jumps past it — so where daylight time
    begins or ends a day is 23 or 25 hours. A shorter interval starts each
    time the clock reads its start or jumps past it: the hour a clock shows
    twice is two buckets, and one it skips none. The rules are IANA's from
    2000 on, of the tzdata the build was made with (2026c), each zone's
    transitions and the rule after them; a time before 2000 in a named zone
    is refused rather than answered by rules the build does not hold, and
    an offset answers any time. A named zone's table is 31 KB that native
    builds carry and the browser module is handed beside its collation data
    the first time a statement names one.

- **`greatest(a, b, …)` and `least(a, b, …)`**
    are the largest and smallest of their values, as PostgreSQL's are: a
    null is passed over, and the answer is null only when every value
    is.

- **`case when c then v … else e end`** is the value of
    the first condition that holds, or the `else`, or null.
    Only the branch taken is worked out: `case when qty = 0 then null
    else px / qty end` divides only where it may.

## Facets

`facet` counts the values of a field over every row the query
matches — not only the page — for a filter sidebar: how many results each
brand, color or tag has.

```fenecql
get products where price < 500 facet brand, color limit 20
get products match title $1 where price < 500 facet brand top 10, tags limit 20
get products where category = "phones" count facet brand
get products facet meta.origin top 5 limit 0
```

```json
{"rows": [ ... ],
 "facets": {"brand": [{"value": "acme", "count": 12}, {"value": "nova", "count": 7}],
            "color": [{"value": "red", "count": 9}, {"value": null, "count": 2}]}}
```

- **Most first**, ties by value; `top N` keeps the
    commonest `N`.

- **A list counts each value it holds**: a row tagged
    `["sale", "new"]` counts once under each.

- **`null` is a value**, so a field's counts add up to the
    rows.

- **They come back beside the rows**, never in one: the answer is
    `{"rows": [...], "facets": {...}}` when a query asks for
    facets, and the bare list of rows when it does not.

Details

The rows counted are the rows the query selects before
`offset` and `limit`: those the `where`
keeps, and with `match`, those holding a word of the query
(`rerank` reorders them and counts the same). A
`lookup ... required` decides who is counted, as it decides who
is on the page; with `count` the facets go beside the number.
`near` ranks every row the filter passes rather than selecting
some, so counts beside it would be the filter's alone whatever the vector:
a facet with `near` or `fuse` is refused. Ask them
without `near`, in the same `/batch`.

A field is a field or a path into a `json` one. The counts are
`group ... count`'s over the same filter, value for value
(`tests/facets.rs` holds them to it), but for a list, counted a
value at a time, and a path, which `group` does not read. A
`@hash` field of text, int, bool or timestamp is counted from its
buckets — over every row a bucket's length, under a filter how many of a
bucket's rows the set holds — and anything else reads the field of every
matched row. Without `top` a facet answers every value it found,
and one holding more than 10 000 is refused rather than cut: a list cut where
it happened to stop is a wrong answer believed right.

Over 100 000 rows (`make search-bench`): a `@hash`
field over every row 0.01 ms, read from every row 6.1 ms
(a list of tags 17); under a filter keeping half, 5.8
and 9.3 ms against the filter's own 5.9 ms
for a `count`. Beside a `match` reaching about
16 667 rows, two facets took a query from 0.06 to 1.6 ms.

### Counted without its own filter

With a brand chosen, a sidebar's "Brand" should still list the other
brands to add, each with the count it would have. `disjunctive`
counts a facet over the rows the query selects with the filter's own
conditions on that field left out; the page, and every other facet, stay
the query's.

```fenecql
get products where category = "tents" and brand in ["acme", "nova"] and color = "red"
  facet brand disjunctive, color disjunctive, material
  order price limit 24
```

Here `brand` counts over the red tents of every brand,
`color` over the acme and nova tents of every colour, and
`material` over the page's rows. A condition left out is a term
of the `where`'s `and` chain that reads the field
alone — `brand = ...`, `brand in [...]`, `(brand =
"a" or brand = "b")`, a range — and one reading the field beside
another (`brand = "a" or sale`) is refused rather than guessed
at. A token's rules and a collection's expiry are never left out: they are
ANDed into each facet's own filter too.

### Ranges

`ranges [b1, b2, ...]` counts the rows whose number falls in
each range — from a bound, included, to the next, excluded — rather than by
value: every range, in order, an empty one with 0, its value the
`[from, to]` pair. A null, a `NaN` and a number
outside every range count in none; a list counts once in each range a value
of it falls in.

```fenecql
get products where category = "tents" limit 24
  facet price ranges [0, 2500, 5000, 10000, 25000] disjunctive
```

```json
"price": [{"value": [0, 2500], "count": 31}, {"value": [2500, 5000], "count": 112},
          {"value": [5000, 10000], "count": 204}, {"value": [10000, 25000], "count": 96}]
```

The bounds are numbers written out, rising, two to 10 001 of them, and no
`top` goes with them. A `@sorted` field counts them from
its index when every bound is a key it holds exactly — over every row two
searches a range and the lengths of the chunks between, under a filter its
entries walked against the filter's rows — and anything else reads the field
of every row (`12.5` against an `int` field is read).

Measured

Over 100 000 rows (`make search-bench`, an M1 Air): six ranges
of a `@sorted` price over every row 0.004 ms against 4.7 ms read
from every row; under a filter keeping 3%, 0.15 against 0.19 ms; keeping
half, 5.1 against 8.1. A category page with a brand and a colour chosen,
its page and two disjunctive facets beside a range facet in one statement,
took 0.75 ms in process against 0.75 ms for the three statements it was
before; over HTTP 0.95 ms against 1.19 for the three in turn and 0.61 for
the three at once on three connections. What one statement buys is one
request, one connection and counts read under one lock — the same rows
every facet counted — not less work.

## Explain

Put `explain` in front of a `get` to see how the
query ran: which index it used and how many rows each step read.

```fenecql
explain get events where cat = "c7" order created desc limit 20

plan
order: the ordered index on created not walked, an equality names fewer rows
filter: the hash index on cat, 9753 rows, which is the answer
order: created desc, every key read, 9753 matches, the first 20 put in order
rows: 20
```

- **It runs the query**, because some choices depend on the data.

- **The wording is for reading**, not parsing; it may change between
    versions.

Details

`explain` in front of a `get` runs the query and
answers with the path it took instead of its rows: one row a step, in the
order the steps ran, in a single `plan` column. The last row is
the size of the answer.

```fenecql
explain get events where cat ~ "c77" order created desc limit 20

plan
order: walked the ordered index on created desc, 2153 rows tested, 20 kept
rows: 20
```

Both are over a million events, `cat` carrying
`@hash` and `created` `@sorted`. The first
filter names a hundredth of the rows through an index, so they are sorted;
the second is one no index narrows, so the walk tests rows newest first
until twenty pass.

It runs the query because several choices depend on the data rather than
the text: whether a filter's rows outnumber the ANN's budget, whether a walk
along an ordered index fills the page before it gives up, whether the ANN
came up short and fell back. The steps name each of them.

| A step that starts | says |

| `filter:` | which index narrowed the filter — the id index, a hash index, an `in` over one, an ordered index's range — or that it was a full scan, how many rows were tested, and whether the scan stopped at the page |

| `order:` | a walk along an ordered index and how many rows it tested, why one was not walked, or the sort: how many matches, how many put in order |

| `near:` | the ANN and its `ef`, the filter's set searched exactly, the probe that stopped past the budget, the fallback when the ANN came up short, and the wider beam or exact search when tombstones cut it short |

| `match:`, `rerank:` | the text index and how many rows it ranked, the vectors read for the rerank |

| `fuse:` | the rank offset, and how many candidates each side brought — after its own `match:` and `near:` steps |

| `lookup:`, `required:` | the probe per parent and how many children it attached — over several fields the index and whether its runs were walked in the level's order or probed and sorted; which side `required` was answered from |

The same statement goes through every transport: the shell prints
the rows, `POST /query` returns
them as JSON, and the JavaScript builder has `.explain()`, which
resolves to the lines. The steps are for reading, not parsing — their wording
may change between versions.

## Writes that read the row

A value in a `set` can be an expression over the row it
writes: its fields, parameters and functions, with `+`
`-` `*` `/`. Every write answers how many
rows it changed, so a write that names what it expects to find says
whether it found it.

```fenecql
set hits {n: n + 1, seen: now()} where key = "ip1"      -- an increment
set hits {n: coalesce(n, 0) + $1} where key = $2         -- from nothing, by a parameter
set docs {body: $1, version: version + 1}
  where id = $2 and version = $3                          -- compare-and-set: 1 or 0
put locks {name: "nightly", owner: $1, at: now()} if absent   -- 1 if taken, 0 if held
insert journal {entry: $1 + ":dr", amount: 0 - $2}           -- two texts joined
```

- **An increment is atomic.** The row is read and written under the
    write lock, and there is one writer, so increments from any number of
    clients all land.

- **A field read is the row as it was.** In `{n: n + 1, m:
    n}` both read the old `n`; a name on the value side is
    a field, a quoted string a string.

- **`null` stays `null`.** `n + 1`
    over a row with no `n` is `null`, as in SQL; write
    `coalesce(n, 0) + 1` to count from nothing.

- **Every write answers its count** — `{"affected": n}`
    over HTTP, the `count` of the browser module's answer, and
    the write result of every SDK.

Details

**Types.** The result goes through the field's type check as a
literal would. Two `int`s make an `int`, and a result
past 64 bits is refused, never wrapped; `/` between
`int`s divides whole, toward zero, as PostgreSQL's does
(`7 / 2` is `3`), and a division by zero is refused.
An `int` and a `float` make a `float`,
which an `int` field takes when it is whole and refuses when it
is not — `n * 2.0` fits, `n + 0.5` does not, as the
literal `2.5` does not. A float that is no longer finite is
refused. A `timestamp` plus or minus an `int` is a
timestamp that many milliseconds on (`now() + 30000`), and two
timestamps apart are the milliseconds between them. `+` joins two
texts (`$1 + ":dr"`), and only two texts. Anything else — text
and a number, either way round, or `-` between texts — is a type
error: a number turned into text without asking would hide the mistake a
typed field is there to catch. An error refuses the statement whole, a
block of one, so nothing of it is written; over no rows there is no error,
as there is no row to work it out over.

**Precedence.** `*` and `/` bind tighter than
`+` and `-`, and all four tighter than a comparison:
`where n + 1 > 2 * m`. `-1` where a value starts is
a number; after a value, `n-1` and `n - 1` are a
subtraction.

**Time.** `now()` is worked out once a statement, so every
row a `set` writes holds the same time. In the browser it is the
time the page hands the module with every statement, which has no clock of
its own.

**What travels.** The record holds the document as written, the value
worked out, never the expression: a replica, `/_changes`, an
archive and a subscriber see the result, and a restore replays it as it
was. A scoped token's rules test the document a `set` writes, as
they test a literal's ([Scoped
access](https://fenecdb.com/docs//docs/http#scoped-access)). A replica of [Sync](https://fenecdb.com/docs//docs/sync) works the
expression out over its own row at once and sends the statement, which the
server works out again over its own: two replicas' offline increments both
land, and both replicas come to the server's sum.

**How it runs.** The pairs are bound to the collection once a
statement, as a filter is: each field written and read by its position, and
a value that reads no field — a literal, a parameter, `now()` —
worked out once rather than a row. Each row is read once: a
`set` read it twice before, once for the values and once for the
indexes it takes out.

**`if absent`** passes over a document whose
`id`, or whose `@unique` value, a live row holds,
where `insert` refuses the statement: the others are written,
and the answer counts those alone. That is Redis's `SET NX` in
one statement, and it neither aborts a `/batch` nor makes an
error a client has to catch. A row past its `@ttl` holds
nothing — it is out of every read — so a write takes its place.

**What it costs, measured** (`make counters-bench`, in
  process over 100 000 rows of a text, an int and a note). A
  `set` of a constant over every row took 65.0 ms against 85.5
  before expressions — the row read once instead of twice — and
  `{n: n + 1}` 66.6 ms; one row by id 1.07 us against 1.26, and
  1.18 for `n + 1`. Sixteen threads incrementing one key 10 000
  times each ended at 160 000, 320 000 increments a second, and sixteen HTTP
  clients the same against `fenec-server`, 102 000 a second.

## Writes that must find their rows

A write that matches no row is no error: it answers
`{"affected": 0}`, and a `/batch` goes on to the next
statement. `require N` after a `set`, a
`del`, a `put` or an `insert` makes the
count a condition: unless the statement wrote exactly `N` rows
it is refused, and the block it is in — a `/batch`, a run of
several in the page, a keyed write — is put back whole, as on any other
error.

```fenecql
set accounts {balance: balance - $1} where name = $2 and balance >= $1 require 1
del holds where order_ref = $1 and state = "held" require 1
put locks {name: "nightly", owner: $1} if absent require 1      -- refused while it is held
```

- **Refused over HTTP with 412**, the error naming the statement:
    ``set accounts` wrote 0 rows, and requires 1`. A
    `/batch` also says which statement stopped it
    (`"at"`, from 0). Every SDK has its own kind for it
    (`unmet`), and every builder an option:
    `.update(patch, { require: 1 })`.

- **The count is the write's own**, taken under the write lock that
    made it, so nothing can come between the check and the write. A scoped
    token's count is of the rows its rules let it write.

- **`require 0`** asserts that nothing matches.

### A read that must find its rows

`require N` after a `get` guards a read the same
way: unless the `get` answers exactly `N` rows it is
refused (412, `"at"` naming it) and its batch put back. In a
`/batch` it reads under the batch's write lock, after the
statements before it and before those after, so what it saw is what the
writes around it see: "the price is still 12", "the coupon is still
good".

```fenecql
get products where sku = $1 and price = $2 and stock > 0 require 1
get coupons where code = $1 and uses < max_uses limit 1 require 1     -- one is enough
get holds where order_ref = $1 and state = "held" require 0           -- none may be held
```

- **It counts the rows the `get` answers**, after
    `offset` and `limit`: `limit 1 require 1`
    asks that one exists, and reads no further. A `lookup`'s
    children are not counted, and `require` goes before the
    `lookup`, whose clauses are the children's.

- **Not with `count`** or an aggregate without
    `group`, which answer one row whatever matched; over groups
    it counts the groups. An inner `get` of `in (get
    …)` takes none.

- **On its own** it is a read like any other, refused with 412 when
    the count is not met. A scoped token's count is of the rows it may
    read.

- **Every builder** has it as a step, `.require(1)`; a
    write takes it as its option instead. A replica's sync refuses one in a
    batch of synced writes: it would count the replica's rows, and the
    server, which lands the batch, would never see it — send a guarded
    batch to the server itself.

### Transfers

Moving money between two accounts is a debit where the money is, a
credit where the account is, and a journal of both, as one
`/batch`. Without `require` an overdraft's debit
matched nothing, answered 0, and the credit went ahead: the sum of the
balances grew with every refused transfer. With it, a debit that finds too
little money or a credit that finds no account puts the whole transfer
back, journal and all:

```fenecql
create collection accounts (name text @unique, balance int)
create collection journal (tx text @hash, account text @hash, amount int)

set accounts {balance: balance - $3} where name = $1 and balance >= $3 require 1
set accounts {balance: balance + $3} where name = $2 require 1
insert journal [{tx: $4, account: $1, amount: 0 - $3}, {tx: $4, account: $2, amount: $3}]
```

```bash
printf '%s\n' \
  '{"query":"set accounts {balance: balance - $3} where name = $1 and balance >= $3 require 1","params":["ada","bob",30,"t1"]}' \
  '{"query":"set accounts {balance: balance + $3} where name = $2 require 1","params":["ada","bob",30,"t1"]}' \
  '{"query":"insert journal [{tx: $4, account: $1, amount: 0 - $3}, {tx: $4, account: $2, amount: $3}]","params":["ada","bob",30,"t1"]}' |
curl -s -X POST localhost:8080/batch -H "Idempotency-Key: t1" --data-binary @-
# an overdraft: 412 {"error":"unmet: `set accounts` wrote 0 rows, and requires 1","completed":0,"at":0}
```

- **Amounts are whole minor units** in an `int`, so a
    sum is exact.

- **An `Idempotency-Key`** makes a retry after a timeout
    answer what the first try did, rather than move the money twice
    ([Retrying a write](https://fenecdb.com/docs//docs/http#idempotency)).

- **Who may move what is the app's.** The credit is another user's
    row, which a scoped token may not write: transfers go through an app
    server holding the server's token.

- **A read can guard it too.** A client that showed a balance and
    moves money only while it is still that puts `get accounts where
    name = $1 and balance = $5 require 1` first; changed meanwhile, the
    transfer stops there, 412 with `"at":0`.

- **Held to it:** eight threads making 4 800 random transfers among
    six accounts in process, and eight clients making 2 400 as
    `/batch`es over HTTP, an eighth of them to an account that
    is not there, leave the sum where it started, no balance below zero,
    and a journal that accounts for every balance
    (`crates/fenec-core/tests/require.rs`,
    `crates/fenec-http/tests/require.rs`).

- **A whole ledger** built this way — refunds, holds, an
    append-only journal, reconciliation, a change-stream sink, scoped
    tokens — is
    [examples/ledger](https://github.com/fenecdb/fenec/tree/main/examples/ledger).

### Checkout

An order is placed at the price the shopper saw, against stock that is
there, with a coupon that is still good — or not at all. Each condition is
a `require`: the price and the coupon read, the stock taken, the
coupon's use counted and the order written, as one `/batch`.

```fenecql
create collection products (sku text @unique, price int, stock int)
create collection coupons (code text @unique, off int, uses int, max_uses int)
create collection orders (ref text @unique, sku text @hash, paid int)

get products where sku = $1 and price = $2 require 1
get coupons where code = $3 and uses < max_uses require 1
set products {stock: stock - 1} where sku = $1 and stock > 0 require 1
set coupons {uses: uses + 1} where code = $3 require 1
insert orders {ref: $4, sku: $1, paid: $2 - $5}
```

- **A price that moved** stops the batch at its first line, 412 with
    `"at":0`: the shopper is shown the new price, and nothing was
    written.

- **A coupon used up** stops it at the second, **no stock** at the
    third; the order's `@unique` `ref` refuses the
    same checkout twice (409), and an `Idempotency-Key` answers a
    retry what the first try did.

- **Under the write lock**, every line sees the ones before it: two
    shoppers taking the last one cannot both pass the stock line.

## Writes that pick their rows

`order` and `limit` after a `set` or a
`del` pick the rows it writes: the page a `get` with
the same `where`, `order` and `limit` would
answer, found and written under the one writer's lock.
`returning` answers them rather than their count — a
`set`'s rows as it wrote them, a `del`'s as they were,
in the order they were picked. A job queue's claim is one statement:

```fenecql
create collection jobs (payload json, run_at timestamp @sorted, owner text, attempts int)
put jobs [{payload: {"to": "ada"}, run_at: now(), attempts: 0}, {payload: {"to": "bo"}, run_at: now(), attempts: 0}]
set jobs {owner: "w1", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 10 returning *
del jobs where id = 1 and owner = "w1" require 1
del jobs where run_at <= now() order run_at limit 10 returning id, payload.to
```

- **The page is a `get`'s.** `order` over an
    `@sorted` field walks its index and stops at the page;
    `limit` without `order` takes the first matches by
    `id`; `order` without `limit` writes
    every match, in that order. `offset` is a read's
    alone.

- **Two writers never pick the same row.** The rows are found and
    written under the write lock, where PostgreSQL's `UPDATE … WHERE id
    IN (SELECT … FOR UPDATE SKIP LOCKED) RETURNING *` locks rows and
    skips those another holds. A client that reads the page and then writes
    it loses the race whenever another read the same page.

- **`returning *`** answers the id and every field, as
    `select *` does; a list answers those named, paths among
    them. Over HTTP the answer is the rows, as a `get`'s.

- **`require N`** counts the rows answered: `limit 1
    require 1` is one row or a refusal.

- **A scoped token** picks among the rows its rules let it update or
    delete, and what it answers is held to its read rules as well — a
    `set`'s rows as it writes them, so a worker that reads only
    the jobs it holds is answered the job it claims, and a
    `set` that would leave a row it may not read is refused
    whole (403). Every row it writes is held to its rules after the write,
    its fields to an `update(…)` grant's and its change to a
    transition's ([Scoped
    access](https://fenecdb.com/docs//docs/http#scoped-access)). A row answered shows the fields of the read rules
    admitting it, and a value, a `where` or an
    `order` reads a field a rule withholds only as a read may
    ([Fields a read shows](https://fenecdb.com/docs//docs/http#field-grants)).

- **A sync replica refuses one** on a synced collection: it would
    pick and answer the replica's rows, and the server, sent the text, its
    own.

- **Every builder** writes it: `order` and
    `limit` before an `update` or a
    `delete`, and `{ returning: true }` — or the
    fields — as its option. See [Job queues](https://fenecdb.com/docs//docs/queues) for
    the whole queue: ack, extend, backoff, dead letters, priorities.

- **An `order` key may be an expression**, so the
    nearest ready job is one claim too:
    `set jobs {owner: $1} where run_at <= now() order
    distance(pickup, $2) limit 1 returning *`
    ([Ordering by an expression](#order-by-expression)). A write
    takes no `near`: the distance is a key of
    `order` there, or an inner `get`'s
    `near` picks the id — `where id in (get jobs select id
    where run_at <= now() near pickup $2 limit 1)`, run first under
    the same lock.

### The rows an insert made

`returning` after a `put` or an
`insert` answers the rows it wrote rather than their count,
ids and all, in the order of its documents — what a client needs to know
the id of the row it made, without reading it back by a key of its
own:

```fenecql
create collection orders (item text, customer text, n int)
insert orders {item: "tea", customer: "ada"} returning id
insert orders [{item: "pie", customer: "ada"}, {item: "jam", customer: "bo", n: 2}] returning *
put orders {item: "tea", customer: "ada"} if absent else set {n: coalesce(n, 0) + 1} returning id, n
```

- **Each row as the statement left it**: a row `if
    absent` passed over is not answered, the row `else set`
    set is, and a document written twice in one statement is answered
    twice, as it ended. `require N` counts the rows
    answered.

- **Over HTTP** the answer is the rows, with the change it left the
    database at in `Fenec-Seq`; `POST
    /orders?returning=id` answers a REST insert's rows (201) where it
    otherwise answers `{"inserted": n}`.

- **A scoped token** is answered only rows its read rules let it
    read, as a `set`'s are: each row written is held to them as
    written — a row made as an insert, a row set as an update — and one they
    leave out refuses the write, put back whole (403). An append to a stream
    the token may write and not read is written without
    `returning`.

- **Every builder** takes it: `insert(docs, { returning: ['id']
    })` and `upsert(docs, patch, { returning: true })`,
    Python's `returning=`, Go's `Returning(...)` among
    the documents, .NET's `returning:`, Swift's, Kotlin's and
    Dart's `insertReturning` and `upsertReturning`. A
    sync replica refuses it on a synced collection, as a claim: its ids are
    the replica's until the server answers.

### Ordering by an expression

A key of `order` is a field, or any expression over the row:
a distance, a product, a `case`. The rows a `get`
answers, and those a `set` or a `del` picks, are put
in its order as by a field's, ties in id order:

```fenecql
get orders order n * 2 desc, item limit 10
get orders order case when customer = "bo" then 0 else 1 end, id
```

- **Every match's key is worked out** and the page put in order: the
    cost follows the matches, as an order over a field without
    `@sorted` does. A field beside an expression keeps its own
    collation and fold; an expression's text orders by its bytes, or in the
    collation `collate` names.

- **`distance(<field>, <point>)` ascending**
    over a field with `@geo`, with a `limit`, walks
    the point index nearest first and stops at the page, as
    `near` does — where no row the filter passes lacks a point,
    whose null distance comes first: the index holds every row, or the
    filter puts a radius or a box on the field
    ([Nearest first](https://fenecdb.com/docs//docs/geo#nearest)).

- **Not in every place an order goes:** over groups a key names a
    column of the list (give the expression a name with `as`);
    a `lookup` level orders its children by fields; a key holds
    no inner `get`, aggregate or `expired()`.

## Time

A `timestamp` field takes an ISO-8601 date or epoch
milliseconds, and always comes back as ISO-8601 in UTC.

```fenecql
put events {name: "login", t: "2026-09-19T12:34:56Z"}
put events {name: "logout", t: 1758285296000}        -- epoch ms
get events where t >= "2026-01-01" and t < now()
```

Details

`timestamp` holds UTC epoch milliseconds as an
`i64`. Writes accept both text and numbers; a text literal in a
comparison is parsed.

The representation is always ISO-8601
(`2026-09-19T12:34:56.789Z`), in JSON and in FenecQL alike. The calendar conversion is
integer arithmetic including leap-year and century rules — no table, no
dependency.

It exists as a separate type rather than an alias over `int`
because `ResultSet` does not carry the schema, so an alias would
show every client a raw number.

## Bulk loading

For a bulk load, build the index afterwards. The write path becomes a pure
append and the graph is built in one parallel pass.

```fenecql
create collection docs (title text, embed vector<768>)
put docs [ ... 100 000 documents ... ]
create index on docs (embed) @hnsw(cosine)
```

## Worked examples

Common queries, ready to adapt.

### A hybrid query

```fenecql
get articles select title, year
  where year >= 2024 and tags has "rust" and not (title ~ "draft")
  near embed $1 ef 128
  limit 10
```

### Counting

```fenecql
get articles where year >= 2024 count
-- one row, one column
```

### Totals per group

```fenecql
get orders select status, count(*), sum(total) where year = 2026 group status
  order sum(total) desc limit 3
```

### Bars by the minute

```fenecql
get ticks select bucket(at, 1m) as bar, first(px by at) as open, max(px) as high,
  min(px) as low, last(px by at) as close, sum(qty) as volume
  where sym = $1 and at >= $2 group bar
```

### Update and delete

```fenecql
set articles {year: 2025} where id = 42
set articles {views: coalesce(views, 0) + 1} where id = 42   -- an increment
del articles where year < 2000
```

### Inspecting the database

```fenecql
collections
describe articles
compact articles
```

`compact` drops deleted data and rewrites the file.

Details

`compact` drops the dead records and rewrites the file. The
indexes stand — the documents are the same ones, and every index is keyed by
id — except a vector graph holding tombstones, which is rebuilt: nothing else
takes a deleted or rewritten document's node out of it. See
[Limits](https://fenecdb.com/docs/limits#index-build-open-and-maintenance).


---

Source: https://fenecdb.com/docs/sql

# From SQL

The SQL you write by habit, read as the FenecQL it means
— and refused, saying what to write, where it has no exact meaning
here.

FenecQL is not SQL, but the everyday spellings of SQL are FenecQL too.
The parser reads `CREATE TABLE`, `INSERT INTO …
VALUES`, `UPDATE … SET a = …`, `JOIN`,
`IN (…)`, `EXISTS`, `DISTINCT`,
`LIKE` and the rest of the tables below into the very
statements FenecQL writes, so a text in either reaches the engine the same
way — over HTTP, through [MCP](https://fenecdb.com/docs//docs/mcp), in the
`fenec` shell, in the page and in an app. There is no SQL mode
and no second parser: the two spellings mix in one statement. What would
answer otherwise than SQL does — a `RIGHT JOIN`, an aggregate
over two tables' rows, a `DEFAULT`, a transaction left open — is
refused with the FenecQL that does it, so an agent writing SQL by habit
learns the language from its first error.

Every SQL text on this page is run by a test
(`crates/fenec-core/tests/sql_docs.rs`): the session below
statement by statement, each `-- → x` the answer it checks, and
each line of the tables held to the FenecQL beside it — the two parse to
the same statement — or refused.

## A session written as SQL

```sql
CREATE TABLE users (id serial PRIMARY KEY, name text, email citext NOT NULL UNIQUE, age integer, city text, visits integer)
CREATE INDEX users_age ON users (age)
INSERT INTO users (name, email, age, city) VALUES ('Ada', 'ada@example.com', 36, 'London'), ('Bob', 'bob@example.com', 25, 'Leeds'), ('Cem', 'cem@example.com', 52, 'İzmir') RETURNING name  -- → "Ada"
SELECT name FROM users WHERE age BETWEEN 25 AND 36 ORDER BY age DESC LIMIT 10  -- → "Ada"
SELECT name FROM users WHERE city IN ('Leeds', 'İzmir') AND name LIKE 'C%'  -- → "Cem"
SELECT count(*) AS n FROM users WHERE email ILIKE '%@EXAMPLE.COM'  -- → 3
UPDATE users SET visits = coalesce(visits, 0) + 1 WHERE name = 'Ada' RETURNING visits  -- → 1
INSERT INTO users (email, visits) VALUES ('ADA@example.com', 1) ON CONFLICT DO UPDATE SET visits = users.visits + excluded.visits RETURNING visits  -- → 2
INSERT INTO users (email) VALUES ('bob@example.com') ON CONFLICT DO NOTHING  -- → 0
INSERT INTO users (email, visits) VALUES ('bob@example.com', 1) ON CONFLICT (email) DO UPDATE SET visits = excluded.visits RETURNING visits  -- → 1
SELECT name || ' <' || email || '>' AS who FROM users WHERE id IN (SELECT id FROM users WHERE age > 30) ORDER BY id  -- → "Ada <ada@example.com>"
SELECT name FROM users WHERE visits IS DISTINCT FROM 2  -- → "Bob"
SELECT city, count(*) AS n FROM users GROUP BY 1 ORDER BY 2 DESC, 1  -- → "Leeds"
CREATE TABLE orders (id serial PRIMARY KEY, user_id integer NOT NULL, total integer, status text)
CREATE INDEX ON orders USING hash (user_id)
INSERT INTO orders (user_id, total, status) VALUES (1, 1200, 'paid'), (1, 800, 'open'), (2, 450, 'paid')  -- → 3
SELECT u.name, o.total FROM orders o JOIN users u ON u.id = o.user_id WHERE o.status = 'paid' ORDER BY o.total DESC  -- → "Ada"
SELECT u.name, o.total FROM users u LEFT JOIN orders o ON o.user_id = u.id ORDER BY u.id DESC  -- → "Cem"
SELECT count(*) AS n FROM orders o JOIN users u ON u.id = o.user_id WHERE u.city = 'London'  -- → 2
SELECT DISTINCT u.city FROM users u JOIN orders o ON o.user_id = u.id ORDER BY u.city  -- → "Leeds"
SELECT name FROM users u WHERE NOT EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id)  -- → "Cem"
SELECT name FROM users WHERE age = (SELECT max(age) FROM users)  -- → "Cem"
DELETE FROM users WHERE age NOT BETWEEN 20 AND 40  -- → 1
CREATE TABLE docs (id serial PRIMARY KEY, body text, emb vector(3))
CREATE INDEX ON docs USING hnsw (emb vector_cosine_ops)
INSERT INTO docs (body, emb) VALUES ('north', [0, 1, 0]), ('east', [1, 0, 0]), ('up', [0, 0, 1])  -- → 3
SELECT body FROM docs ORDER BY emb <=> '[0.9, 0.1, 0]' LIMIT 1  -- → "east"
```

`email` is `citext`, a `nocase` field,
so its `=`, its `@unique` and its `LIKE`
compare by the case fold, and `'ADA@example.com'` is Ada's row.
Each join reads as the `lookup` it is, or as one table's rows
filtered by the other's ([below](#joins)); the last statement
is `near`, by the cosine distance the index measures.

## Read as written

Each line is the SQL and the FenecQL it is — the same statement, to the
last expression.

| SQL | FenecQL | Notes |

| `INSERT INTO t (a, b) VALUES (1, 'x'), (2, 'y')` | `insert t [{a: 1, b: "x"}, {a: 2, b: "y"}]` | a document a row; the fields are named ([below](#insert)) |

| `INSERT INTO t (a) VALUES ($1) RETURNING id, a` | `insert t {a: $1} returning id, a` | the rows written, the ids made among them |

| `INSERT INTO t (k, n) VALUES ('a', 1) ON CONFLICT DO NOTHING` | `insert t {k: "a", n: 1} if absent` | passed over where its id or a `@unique` value is held |

| `INSERT INTO t (k, n) VALUES ('a', 1) ON CONFLICT DO UPDATE SET n = t.n + excluded.n` | `insert t {k: "a", n: 1} if absent else set {n: n + new.n}` | `excluded.f` is `new.f`, `t.f` the row's `f` ([below](#upsert)) |

| `UPDATE t SET a = 5, n = n + 1 WHERE k = 'a' RETURNING *` | `set t {a: 5, n: n + 1} where k = "a" returning *` |  |

| `UPDATE jobs SET owner = $1 WHERE run_at <= now() ORDER BY run_at LIMIT 1 RETURNING *` | `set jobs {owner: $1} where run_at <= now() order run_at limit 1 returning *` | MySQL's `ORDER BY` and `LIMIT` on a write: a [queue's claim](https://fenecdb.com/docs//docs/queues) |

| `DELETE FROM t WHERE k = 'a' RETURNING *` | `del t where k = "a" returning *` |  |

| `SELECT a, b FROM t WHERE n > 1 ORDER BY a DESC LIMIT 10 OFFSET 20` | `get t select a, b where n > 1 order a desc limit 10 offset 20` |  |

| `SELECT k, count(*) AS n FROM t GROUP BY k HAVING count(*) > 1` | `get t select k, count(*) as n group k having count(*) > 1` |  |

| `SELECT * FROM t WHERE x IN (1, 2, 3)` | `get t where x in [1, 2, 3]` | through `@hash` as `in [..]` is |

| `SELECT * FROM t WHERE x NOT IN ('a', 'b')` | `get t where not (x in ["a", "b"])` | a null `x` is taken in ([below](#nulls)) |

| `SELECT * FROM t WHERE x IN (SELECT y FROM u WHERE z = 1)` | `get t where x in (get u select y where z = 1)` | a [subquery](https://fenecdb.com/docs//docs/fenecql#subqueries), run once |

| `SELECT * FROM t WHERE x IN (SELECT DISTINCT y FROM u)` | `get t where x in (get u select y)` | `DISTINCT` changes nothing there |

| `SELECT * FROM t WHERE x BETWEEN 1 AND 3 AND y = 2` | `get t where x >= 1 and x <= 3 and y = 2` | both bounds in; the first `AND` is `BETWEEN`'s |

| `SELECT * FROM t WHERE x NOT BETWEEN 1 AND 3` | `get t where not (x >= 1 and x <= 3)` |  |

| `SELECT * FROM t WHERE name LIKE 'A%'` | `get t where name like "A%"` | `%` any run of characters, `_` one ([below](#like)) |

| `SELECT * FROM t WHERE name NOT ILIKE '%!_%' ESCAPE '!'` | `get t where not (name ilike "%!_%" escape "!")` | `ILIKE` by the case fold |

| `SELECT * FROM t WHERE x <> 1` | `get t where x != 1` |  |

| `SELECT * FROM t WHERE x IS DISTINCT FROM $1` | `get t where x != $1` | FenecQL's `=` and `!=` take a null as a value |

| `SELECT * FROM t WHERE x IS NOT DISTINCT FROM $1` | `get t where x = $1` |  |

| `SELECT * FROM t WHERE done IS NOT TRUE` | `get t where not (done = true)` |  |

| `SELECT count(*), count(age) FROM users` | `get users select count(*), count(age)` | `count(age)` the rows where `age` is not null |

| `INSERT INTO t VALUES {a: 1}, {a: 2}` | `insert t [{a: 1}, {a: 2}]` | documents name their fields; `put t {…}, {…}` too |

| `SELECT * FROM t WHERE x IS NOT NULL ORDER BY x DESC NULLS LAST` | `get t where x is not null order x desc` | `NULLS` where it says what the order does |

| `SELECT given || ' ' || family AS name FROM t` | `get t select given + " " + family as name` | `||` is `+` |

| `SELECT * FROM t WHERE name = 'O''Brien'` | `get t where name = "O'Brien"` | `''` is a quote in single quotes |

| `UPDATE "t" SET "n" = 1 WHERE id IN (1, 2) RETURNING "n"` | `set t {n: 1} where id in [1, 2] returning n` | a quoted name where only a name can stand ([below](#quotes)) |

| `SELECT count(DISTINCT u), max(at) FROM t WHERE at >= $1` | `get t select count(distinct u), max(at) where at >= $1` |  |

| `SELECT a, CASE WHEN n > 1 THEN 'big' ELSE 'small' END AS size FROM t` | `get t select a, case when n > 1 then "big" else "small" end as size` |  |

| `SELECT u.name FROM users AS u WHERE u.age > 30` | `get users select name where age > 30` | an alias, and the names it qualifies, written bare |

| `SELECT DISTINCT city FROM users ORDER BY city` | `get users select city group city order city` | a group of the list's items, each set of values once ([below](#distinct)) |

| `SELECT city FROM users GROUP BY city` | `get users select city group city` | a group with no aggregate answers its keys |

| `SELECT city, count(*) AS n FROM users GROUP BY 1 ORDER BY 2 DESC` | `get users select city, count(*) as n group city order n desc` | a key by its place in the list |

| `SELECT city c, count(*) n FROM users GROUP BY city` | `get users select city as c, count(*) as n group city` | a name without `AS`, where the list goes on or ends at `FROM` |

| `SELECT count(*) FILTER (WHERE age IS NULL) AS unknown, sum(total) FILTER (WHERE paid) AS paid FROM t` | `get t select count(case when age is null then 1 end) as unknown, sum(case when paid then total end) as paid` | what `case` leaves of each row: every aggregate passes a null over |

| `SELECT * FROM users u WHERE EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id AND o.total > 100)` | `get users where id in (get orders select user_id where total > 100)` | the one equality with the row around it is the list, the rest the query's |

| `SELECT * FROM users u WHERE NOT EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id)` | `get users where not (id in (get orders select user_id))` | a row whose key is null has no match: kept, as SQL keeps it |

| `SELECT * FROM t WHERE at = (SELECT max(at) FROM t)` | `get t where at in (get t select max(at))` | a query's one value: an aggregate, or `LIMIT 1` |

| `UPDATE jobs SET owner = $1 WHERE id = (SELECT id FROM jobs ORDER BY run_at LIMIT 1)` | `set jobs {owner: $1} where id in (get jobs select id order run_at limit 1)` |  |

| `CREATE TABLE users (id serial PRIMARY KEY, name text NOT NULL, email varchar(200) UNIQUE, age integer, tags text[], meta jsonb)` | `create collection users (name text required, email text @unique, age int, tags [text], meta json)` | the row's `id` is the database's; types as [`fenec import`](https://fenecdb.com/docs//docs/import) maps them ([below](#create-table)) |

| `CREATE TABLE lines (order_id int, n int, sku text, PRIMARY KEY (order_id, n))` | `create collection lines (order_id int required, n int required, sku text, @unique(order_id, n))` | a primary key is its rules: unique, and present |

| `CREATE INDEX users_age ON users (age)` | `create index on users (age) @sorted` | a btree is `@sorted`; the index's name is not kept |

| `CREATE UNIQUE INDEX ON users (email)` | `create index on users (email) @unique` |  |

| `CREATE INDEX ON orders USING hash (user_id)` | `create index on orders (user_id) @hash` |  |

| `CREATE INDEX ON posts (owner, created DESC)` | `create index on posts (owner, created desc) @sorted` | over several columns, an index over several fields |

| `CREATE INDEX ON docs USING hnsw (emb vector_cosine_ops) WITH (m = 16)` | `create index on docs (emb) @hnsw(cosine, m=16)` | pgvector's operator class names the distance |

| `CREATE INDEX ON docs USING gin (to_tsvector('english', body))` | `create index on docs (body) @text` | full text, for [`match`](https://fenecdb.com/docs//docs/fenecql#the-text-clause) |

## Joins

A `JOIN` on a key is a [`lookup`](https://fenecdb.com/docs//docs/fenecql#the-lookup-clause)
from each row of the first table to its matches in the next, answered as
SQL answers it: flat rows, a row a pair — and, for a `LEFT
JOIN`, a row of `null`s where nothing matched — the
columns the list names under SQL's names. Where only one table's columns
are answered and each of its rows is answered once — the other table found
by its `id`, or `DISTINCT` — it is that table's
`get`, the other's rows a filter. Each line below is the
FenecQL it parses to; a `lookup`'s line answers flat, which no
FenecQL text asks for.

| SQL | FenecQL | Notes |

| `SELECT o.id, u.name FROM orders o JOIN users u ON u.id = o.user_id WHERE o.total > 100 ORDER BY o.placed DESC LIMIT 20` | `get orders select id where total > 100 order placed desc limit 20 lookup users on id = user_id required select name` | each order's user; `required` keeps an order only where a user matched |

| `SELECT u.name, o.total FROM users u LEFT JOIN orders o ON o.user_id = u.id AND o.status = 'paid' ORDER BY u.name` | `get users select name order name lookup orders on user_id select total where status = "paid"` | a user with no paid order is a row, `total` null |

| `SELECT l.qty, o.status, u.name FROM lines l JOIN orders o ON o.id = l.order_id JOIN users u ON u.id = o.user_id` | `get lines select qty lookup orders on id = order_id required select status lookup users on id = user_id required select name` | a chain: each join hangs off the table before it |

| `SELECT o.id FROM orders o JOIN users u ON u.id = o.user_id WHERE u.email = $1` | `get orders select id where user_id in (get users select id where email = $1)` | the users only a filter: each order found once |

| `SELECT DISTINCT u.name FROM users u JOIN orders o ON o.user_id = u.id WHERE o.status = 'open'` | `get users select name where id in (get orders select user_id where status = "open") group name` |  |

| `SELECT u.id, count(*) AS n FROM users u JOIN orders o ON o.user_id = u.id WHERE u.city = 'Leeds' GROUP BY u.id` | `get orders select user_id as id, count(*) as n where user_id in (get users select id where city = "Leeds") group user_id` | the orders' rows, the users a filter: `u.id` is each order's `user_id` |

| `SELECT sum(o.total) AS t FROM orders o, users u WHERE u.id = o.user_id AND u.city = 'Leeds'` | `get orders select sum(total) as t where user_id in (get users select id where city = "Leeds")` | a comma is a join, its key in the `WHERE` |

| `SELECT u.email FROM users u LEFT JOIN orders o ON o.user_id = u.id WHERE o.id IS NULL` | `get users select email where not (id in (get orders select user_id))` | the rows a `LEFT JOIN` finds none for |

| `SELECT id, body FROM docs WHERE kind = 'note' ORDER BY emb <=> $1 LIMIT 10` | `get docs select id, body where kind = "note" near emb $1 limit 10` | pgvector's distance is `near` where the index measures it ([below](#pgvector)) |

What a join is read as, and what it cannot be:

- **The key.** `ON` pairs a column of the joined table
    with one of the table joined just before it — several, joined by
    `AND`, are a key over several fields. Where the answer is
    the joined rows beside each row — read as a `lookup` — the
    joined table is read by its column, which must be its `id`,
    a field with `@hash` or the first fields of an index over
    several, as a `lookup`'s child key must be; anything else
    is refused naming the index to declare. Where only one table's columns
    are answered and the other is a filter, `key in (get …)`,
    no index is needed, and a small joined table is answered through the
    first table's index — that list is held to 100 000 values, as any
    `in (get …)` is. A join that finds the joined table by its
    `id` looks each row's match up by that id past them, at any
    size, whichever table comes first and with the joined table filtered,
    where the other key is an `id`, an int, a float or a
    timestamp;
    an aggregate, `DISTINCT`, `EXISTS` or
    `LEFT JOIN … IS NULL` over a larger joined table is refused,
    never cut. Its other conditions read one table's columns. A comma between tables is a join
    whose key is in the `WHERE`, and `USING (k)` pairs
    `k` of each.

- **The names.** A column is named by its table's name or alias,
    `o.total`; a name with no table is the first table's — the
    parser reads no schema, and SQL would have refused one two tables
    hold — and one the first table has no field of is refused, saying to
    name it with its table. A column no table has is refused, as SQL refuses
    it, wherever the statement names it — an `ON` term too — and
    so is a table no database holds. A
    column answers under its name, `total`, or its
    `AS`; two that would answer under one name — every table's
    `id` under `SELECT *` — are refused, since a row
    holds a name once: `SELECT o.*, u.name AS who`.

- **The order and the page** are the first table's, whose rows the
    others are found for: `ORDER BY` names its columns, and
    `LIMIT` counts the flat rows, the first table read as far
    as the page could need. Ordering by a joined table's column would put
    each row's matches in order rather than the rows: refused, saying to
    name that table first.

- **Aggregates and `DISTINCT`** read one table's columns,
    the other's rows a filter, where each row of it is counted once:
    folded over pairs, a row matching three times would be counted three
    times. Grouping one table's rows with another's column beside each
    group is a `lookup` after the groups: `get orders
    select user_id, count(*) as n group user_id lookup users on id =
    user_id select name`.

- **A `LEFT JOIN`'s table in the `WHERE`**
    reads the rows it kept with nulls: `o.id IS NULL` is the rows
    with none, and anything else is refused — a condition on its rows goes
    in `ON`, and one that drops the rows kept with nulls makes it
    an inner join, written `JOIN`. A condition in a
    `LEFT JOIN`'s `ON` on a table before it keeps that
    table's rows and drops only their matches, which no `lookup`
    says: refused.
    An inner join after a `LEFT JOIN`, a `RIGHT`,
    `FULL` or `CROSS JOIN` and a condition between two
    tables' columns other than the key are refused.

## Refused, and what to write

Each is refused with the FenecQL in its error.

| SQL | FenecQL | Why |

| `ALTER TABLE t ADD COLUMN a int` | `alter collection t add field a int` |  |

| `DROP TABLE t` | `drop collection t` |  |

| `TRUNCATE t` | `del t` |  |

| `BEGIN` | a `/batch` | no transaction is held open across requests; a [`/batch`](https://fenecdb.com/docs//docs/http#batches) lands whole or not at all, and [`require`](https://fenecdb.com/docs//docs/fenecql#require) makes a count its condition |

| `CREATE TABLE t (n int DEFAULT 0, u int REFERENCES users(id))` | `create collection t (n int, u int @hash)` | no default, foreign key or `CHECK`: dropped, the collection would take writes SQL refuses ([below](#create-table)) |

| `CREATE TABLE t (price numeric(10, 2))` | `create collection t (price_cents int)` | no decimal: an `int` of the smallest unit keeps it exact |

| `CREATE TABLE t (id uuid PRIMARY KEY, n int)` | `create collection t (uuid text required @unique, n int)` | a row's `id` is an int the database makes |

| `CREATE INDEX ON docs USING ivfflat (emb vector_l2_ops)` | `create index on docs (emb) @hnsw(l2)` | the vector index is a graph |

| `SELECT o.id FROM orders o JOIN users u ON u.id = o.user_id ORDER BY u.name` | `get users order name lookup orders on user_id required select id` | a join's rows are put in order by the first table's columns: name the table it orders by first |

| `SELECT u.name, count(*) FROM users u JOIN orders o ON o.user_id = u.id GROUP BY u.name` | `get orders select user_id, count(*) as n group user_id lookup users on id = user_id select name` | an aggregate over a join reads one table's columns; the groups, then each one's user |

| `SELECT * FROM users u RIGHT JOIN orders o ON o.user_id = u.id` | `select * from orders o left join users u on u.id = o.user_id` | a `RIGHT JOIN` is a `LEFT JOIN` the other way round; `FULL` and `CROSS` are none |

| `SELECT u.name FROM users u LEFT JOIN orders o ON o.user_id = u.id WHERE o.total IS NULL` | `select u.name from users u left join orders o on o.user_id = u.id and o.total is null` | the `WHERE` would read the rows kept with nulls; in `ON` it filters the orders |

| `SELECT * FROM t WHERE x = (SELECT y FROM u)` | `get t where x in (get u select y)` | the query may answer several rows, where SQL's `=` refuses a second: `IN`, or `LIMIT 1` |

| `SELECT (SELECT name FROM products p WHERE p.id = l.product_id) AS name FROM lines l` | `select p.name from lines l join products p on p.id = l.product_id` | another table's row beside each row is a join |

| `SELECT * FROM t WHERE meta->>'source' = 'web'` | `get t where meta.source = 'web'` | a json field's member is a path; `json_extract(meta, '$.source')` is answered so too |

| `SELECT id, emb <-> $1 AS d FROM docs ORDER BY emb <-> $1 LIMIT 5` | `get docs select id near emb $1 limit 5` | each row's `_score` holds what the distance is turned into: the L2 distance, the cosine similarity, the inner product ([below](#pgvector)) |

| `SELECT * FROM docs WHERE to_tsvector(body) @@ to_tsquery('rust')` | `get docs match body "rust"` | full text is [`match`](https://fenecdb.com/docs//docs/fenecql#the-text-clause) over an `@text` field; MySQL's `MATCH … AGAINST` too |

| `SELECT * FROM users WHERE email ~ '@example\.org$'` | `get users where email like '%@example.org'` | `~` is contains, not a regular expression ([below](#tilde)) |

| `SELECT * FROM t ORDER BY x NULLS LAST` | `get t order x is null, x` | a null comes first ascending |

| `INSERT INTO t VALUES (1, 'x')` | `insert into t (a, b) values (1, 'x')` | the fields are named ([below](#insert)) |

| `INSERT INTO t (a) VALUES (1), (2, 3)` | `insert into t (a) values (1), (2)` | a row of as many values as fields |

| `INSERT INTO t (a) SELECT a FROM u` | `get u select a`, then `insert t $1` | an insert writes documents, not a query's rows |

| `INSERT INTO t (k) VALUES ('a') ON CONFLICT ON CONSTRAINT t_k DO NOTHING` | `insert into t (k) values ('a') on conflict (k) do nothing` | the target is the unique field ([below](#upsert)) |

| `INSERT INTO t (k, n) VALUES ('a', 1) ON CONFLICT DO UPDATE SET n = 1 WHERE t.n < 3` | `insert t {k: "a", n: 1} if absent else set {n: case when n < 3 then 1 else n end}` | the upsert sets the row it finds, with no `WHERE` |

| `SELECT * FROM t WHERE "name" = 'Ada'` | `get t where name = 'Ada'` | `"name"` is text here ([below](#quotes)) |

Not read, with no error of its own: `WITH`, window
functions, `CAST` and `::`, `LIMIT x,
y`, `UNION`, `INSERT OR IGNORE` and
`REPLACE`, MySQL's `ON DUPLICATE KEY`, `?`
placeholders (`$1` is a parameter).

## Where the meaning is not SQL's

### Nulls: two values, not three

SQL's logic has three values: a comparison with a null is unknown, and
`WHERE` keeps only what is true. FenecQL's has two. A null is
equal to a null and to nothing else, so `=` and `!=`
take it as a value — they are SQL's `IS NOT DISTINCT FROM` and
`IS DISTINCT FROM` — while `<`,
`<=`, `>`, `>=`,
`LIKE` and `BETWEEN` are false with a null on either
side, and `NOT` makes a false true. Where SQL's unknown leaves
a row out, FenecQL may take it in:

| Where `x` is null | SQL | FenecQL |

| `x = NULL` | never true | true: it is `x IS NULL` |

| `x <> 1`, `NOT (x > 1)` | left out | taken in |

| `x NOT IN (1, 2)`, `x NOT LIKE 'a%'`, `x NOT BETWEEN 1 AND 3` | left out | taken in |

| `x IN (1, NULL)` | left out | taken in: the list holds a null |

| `2 NOT IN (1, NULL)` | never true | true |

| `x NOT IN (SELECT y …)`, where some `y` is null | never true | the subquery's nulls are left out of its list |

Add `AND x IS NOT NULL` to leave out what SQL leaves out.
Arithmetic and `||` with a null are null, as in SQL;
aggregates skip nulls, as in SQL — `count(x)` counts the rows
where `x` is not null — and `sum` over none is null.
A null sorts first ascending and last descending, as in SQLite and MySQL —
PostgreSQL's default is the other way round — so `NULLS FIRST`
is read after `ASC` and `NULLS LAST` after
`DESC`, and either other way is refused. `x LIKE
'a%'` in a select list is null for a null.

**`<>` is `!=`, null and all.** One could
read `<>` as SQL's and keep `!=` FenecQL's, but
not exactly: SQL's unknown runs through every `NOT` above it,
which no rewrite of the one comparison carries, so the two would part
somewhere else instead. Two operators that differ on null alone would be
the subtler trap, and FenecQL read `<>` as
`!=` before it read any other SQL, so a text that used it would
change its answer. One meaning, then, written here.

### Double quotes are text

In SQL `"name"` is a column; in FenecQL it is text, as
`'name'` is. Where only a name can stand — a collection, the
fields of an `INSERT`, a field `SET` assigns,
`RETURNING`, an index's fields, a facet — `"name"` is
the name, as in SQL. Where a value can stand it is text, as it always was:
`where title = "Rust"`. Where SQL would read a column and text
would make the answer the same for every row, it is refused rather than
answer all the rows or none:

- alone on the left of a test against values: `"name" = 'Ada'`,
    `"age" > $1`, `"x" IS NULL`, `"x" IN (1,
    2)`, `"x" LIKE 'a%'`, `"x" BETWEEN 1 AND 2`;

- on the right of a test against a literal: `'Ada' = "name"`;

- an item of a select list, a key of `ORDER BY` or
    `GROUP BY`, an aggregate's argument: `SELECT
    "name"`, `max("at")`.

A parameter tested against double-quoted text stays a test: a
policy's `$jwt.role = "admin"`. Text on the right of a field
stays text: `UPDATE t SET a = "b"` sets `a` to the
text `b`, where SQL would copy the column `b`.

### `~` is contains

In PostgreSQL `~` matches a regular expression; in FenecQL
it finds the text as written, case-insensitively: `email ~
"@example.org"`. Written as a pattern, it would run and match
nothing — and an agent that ran `email ~ '@example\.org$'`
told its user there were no such rows. So a text that reads as a regular
expression — anchored, `^a` or `a$`, or holding
`.*` or `.+` — is refused, pointing at
`LIKE`: `like 'abc%'` for a start, `like
'%abc'` for an end, `like 'a%b'` for a gap. A parameter is
taken as the text it is.

### `LIKE` and `ILIKE`

`%` stands for any run of characters and `_` for
one character — of however many bytes. There is no escape character
unless `ESCAPE` names one, as in standard SQL and SQLite;
PostgreSQL's default is `\`, and a FenecQL string reads
`\` as an escape of its own, so `'50\%'` is the text
`50%`: write `LIKE '50!%' ESCAPE '!'`. A pattern
that ends in its escape is refused.

A `nocase` field compares by its case fold wherever it is
compared, `LIKE` too, as PostgreSQL's `citext` does.
`ILIKE` compares the case folds — Unicode's full fold, the one
[`nocase`](https://fenecdb.com/docs//docs/fenecql#nocase) uses — so
`'Straße' ILIKE 'STRASSE'`, and `_` stands for one
character of the fold. Its escape takes no character with a case. A
collated field's `LIKE` compares its bytes, as its
`=` does. `~`, FenecQL's own, is a
case-insensitive contains.

`LIKE` reads every row the rest of the filter leaves: no
index answers it. A prefix over a `@sorted` field walks its
index written as a range, `name >= 'abc' and name <
'abd'`.

### An insert names its fields

`INSERT INTO t VALUES (…)` without its fields is refused.
The parser reads no schema, and a collection's fields have no order a
statement could lean on: `id` is the database's and no
field's, and a field added goes last. Each row of `VALUES` has
as many values as there are fields named. An `id` among them
names the row's id; one that is taken is refused, as a key is, unless
`ON CONFLICT` says otherwise.

### `ON CONFLICT`

FenecQL's upsert is held by any unique key — the `id`, or a
`@unique` value, a field's or several fields' — and
`ON CONFLICT` is read with no target, as SQLite reads it:
`DO NOTHING` passes a row over where any of them is held, and
`DO UPDATE SET` sets the row holding its `id`, or
else the first `@unique` value it holds, in the order the
fields are declared. A target, `ON CONFLICT (email)`, is
taken where it names the one unique key the collection's rows are found by
— its `@unique` field, the fields of a unique index over several,
or `id` where there is no other — and refused, naming the keys,
where the collection has another or a row names an `id` beside
it: the target and the upsert, held by any, would part there. In
`DO UPDATE SET`,
`excluded.f` is the row offered (`new.f`) and
`t.f`, or `f`, the row held; a `WHERE` is
refused.

### Text, `||`, numbers

- **`||` is `+`.** It joins two texts, and a
    text and a number are refused rather than made text: `'n' ||
    1` is an error where PostgreSQL gives `n1`. MySQL's
    `||`, an `OR`, is not read.

- **Text compares by its bytes**, as `COLLATE "C"` does.
    A field declared `collate tr` or `und` orders by
    that collation, and its `=` and `LIKE` compare
    bytes — see [Collation](https://fenecdb.com/docs//docs/fenecql#collation).

- **Division between integers is whole**, toward zero:
    `7 / 2` is `3`, as in PostgreSQL and SQLite
    (MySQL's is `3.5`); an integer and a float make a
    float.

- **Every row has an `id`**, an int the database makes
    and no field declares; `/_schema?as=fenecql` and the MCP
    `schema` tool say so in a comment. `rowid` and
    `_id` are refused naming it.

- **Errors name what to write.** `contains(…)`,
    `starts_with(…)` and `ends_with(…)` are answered
    with `LIKE`; `users.email` — a column named with
    its table — with the field written bare.

### `DISTINCT`, `EXISTS` and a query's one value

`SELECT DISTINCT a, b` is a group of its items, which
answers each set of their values once, as does a `GROUP BY`
with no aggregate; `ORDER BY` then names the list's columns, as
SQL requires of `DISTINCT`. Over an aggregating list whose keys
it lists it changes nothing, and `DISTINCT ON` is refused:
`first(name by signup)` is the row of each key it asks for.
`ORDER BY 2` and `GROUP BY 1` name the list's items
by their places — written with `BY`, as SQL writes them: in
FenecQL's `order 1` and `group 1` the number is a
key, as it always was. Over `*`, whose places the parser
reading no schema cannot count, a number is refused, saying to name the
columns; one naming an aggregate as a group's key is refused as SQL
refuses it.

`EXISTS (SELECT 1 FROM orders o WHERE o.user_id = u.id AND …)`
takes the one equality between a column of its query's table and one of
the row around it as the list an `in (get …)` holds, and the
rest of its `WHERE` as that query's: `u.id in (get orders
select user_id where …)`. A row whose key is null has no match — so
`NOT EXISTS` keeps it, as in SQL — and a name with no table is
the query's own, as SQL reads it first. Correlated any other way, it is
refused. `= (SELECT …)` is the same list where its query answers
one row — an aggregate with no group, or `LIMIT 1` — and refused
where it may answer several, which SQL would refuse at the second; where
it answers none, SQL's comparison is unknown and FenecQL's `in`
false, the same under `NOT` as [`NOT
IN`](#nulls).

### `CREATE TABLE`

Each column's type is the one [`fenec
import`](https://fenecdb.com/docs//docs/import) gives a column of it — one table for both, PostgreSQL's
names and then SQLite's affinity rules for any other word:

| SQL | FenecQL |

| `smallint` `integer` `bigint` `serial`, SQLite's `INT…` | `int` |

| `real` `double precision` `float` | `float` |

| `text` `varchar(n)` `char(n)` `uuid` | `text` |

| `citext` | `text nocase` |

| `boolean` | `bool` |

| `timestamp` `timestamptz` `date` `datetime` | `timestamp` |

| `bytea` `blob` | `bytes` |

| `json` `jsonb` | `json` |

| `text[]` `integer[]` | `[text]` `[int]` |

| `vector(n)` `halfvec(n)` `sparsevec(n)` | `vector<n>` `vector<n, f16>` `sparse<n>` |

| `point` | `geo` |

| `numeric` `decimal` `money`, `interval` `time` `inet` | refused: no counterpart |

- **The row's `id`** is an int the database makes: a
    column `id` of an integer type that is the primary key —
    `serial`, `generated … as identity`,
    `AUTOINCREMENT` — is it, and is no field. An `id`
    of another type, or that is no primary key, is refused.

- **A primary key on other columns** is its two rules,
    `@unique` and `required`, over several columns an
    index over them — beside the row's `id`, which every row
    still has. `ON CONFLICT (k)` finds a row by it.

- **`NOT NULL` is `required`**: a write must
    give the field a value, and a `null` is none.
    `UNIQUE` is `@unique`, `COLLATE NOCASE`
    `nocase`, `COLLATE "C"` the bytes' order.

- **A type's own rules are not held**, as SQLite holds none:
    `varchar(80)` takes a longer text, and `char(n)`
    is not padded with spaces; a `uuid` is text as written, its
    case kept, where PostgreSQL writes it lowercase; a `date` is
    a timestamp, which keeps a time written with it; `smallint`
    and `integer` take any 64-bit int, past their 16 and 32 bits;
    `real` is a 64-bit float, more precise than its 32 bits.

- **What fenecdb has no rule of is refused** rather than dropped,
    which would take writes SQL refuses: `DEFAULT`,
    `REFERENCES` and `FOREIGN KEY`, `CHECK`,
    a computed column, a sequence on another column. A key a join reads is
    a field with `@hash` — `CREATE INDEX … USING hash`.

- **`CREATE INDEX`** keeps no name. A btree is
    `@sorted` (`UNIQUE`, `@unique`), over
    several columns an index over them; `USING hash`
    `@hash`; `USING hnsw` `@hnsw` by its
    operator class — `vector_l2_ops`, pgvector's default,
    `vector_cosine_ops`, `vector_ip_ops` — with
    `WITH (m, ef_construction)`, held to pgvector's bounds; `USING gin
    (to_tsvector(…, body))` `@text`. A partial index, an
    expression's and `ivfflat`, `gist` or
    `brin` have no counterpart.

### pgvector's distances

`ORDER BY emb <-> $1 LIMIT 10` is `near emb $1
limit 10`, nearest first, where the field's index measures the
operator's distance: `<->` L2 over `@hnsw(l2)`,
`<=>` cosine over `@hnsw(cosine)`,
`<#>` the negative inner product over `@hnsw(dot)`
or a sparse field's `@inverted`. Another index's order would be
another, and is refused naming the operator to write. The vector is a
parameter, a list, or pgvector's text `'[1, 2, 3]'`. The search
walks the graph, as pgvector's HNSW index scan does: the nearest are found
approximately, and `near … exact` measures every row.

Each row's distance is in its `_score`, turned to a
similarity: for `<->` it is the L2 distance itself, for
`<=>` 1 minus the cosine distance — the cosine similarity
— and for `<#>` the inner product, minus pgvector's
value. The operator in the list or the `WHERE`, `DESC`
and a second key are refused: `near` ranks the rows, and the
distance is the score.


---

Source: https://fenecdb.com/docs/javascript

# JavaScript and TypeScript

Dependency-free ES modules run fenecdb in the page:
open a database, query it, and keep it between visits — or query a server
through the client alone. Their types come with them.

## Which package

Pick by where the queries run: on a server, through the client entry and
no module, or in the page, through the full module. A synced replica loads
the same module, so its `near` walks a graph as a database in
the page does.

| For | What it imports and loads | Brotli | What it can do | What it cannot |

| Queries on a server | `@fenecdb/web/client`: `connect`, no module | 9 KB | every query and write over HTTP through the builder, live queries over the server's subscriptions, the code's schema checked against the server's | answer without the network, or keep data in the page |

| A database in the page, or a synced replica | `Fenec.open('./fenec.wasm')`; `sync()` loads it too | 273 KB | everything: the graph, every index, the schema check, persistence and files | — |

Each way has a full example in the repository, the same Notes app:
queries on a server from Node through the client entry
([node-server](https://github.com/fenecdb/fenec/tree/main/examples/node-server)),
a database in the page kept in IndexedDB
([web-local](https://github.com/fenecdb/fenec/tree/main/examples/web-local)),
and a React app over either that or a synced replica
([react](https://github.com/fenecdb/fenec/tree/main/examples/react)).
A larger one is a shop in Next.js
([shop](https://github.com/fenecdb/fenec/tree/main/examples/shop)):
10 000 products whose pages the server renders through the client entry,
search with highlights and facets, carts under each shopper's token, and a
checkout that cannot oversell, with its Lighthouse scores, timings and
load test measured. Another is a payments ledger on Node
([ledger](https://github.com/fenecdb/fenec/tree/main/examples/ledger)):
transfers, refunds and holds as guarded `/batch` blocks, an
append-only journal, reconciliation and a change-stream sink, held to its
invariants by 16 clients, a crash under `kill -9` and its
security tests. A third is courier dispatch in Next.js
([dispatch](https://github.com/fenecdb/fenec/tree/main/examples/dispatch)):
a company a tenant behind `fenec-shard` with its plan's quota,
orders claimed by the nearest courier under a lease or held at the server
until one comes, couriers on a live map, and couriers' and customers'
tokens held to their own rows, with the gaps it found listed.

Beside a module the page loads `fenec.js` and the two modules it
imports, 51 KB brotli as they are served with no build step; a bundler keeps
what the app uses of them. An app whose queries run on a server can import
the client entry instead:

```ts
import { connect } from '@fenecdb/web/client';   // or './client.js', beside builder.js and http.js

const db = connect('https://db.example.com', { token: 'a token' });
const rows = await db.from('docs').where('year', '>=', 2024).limit(10).rows();
const stop = db.live(db.from('docs').where('year', 2024), (rows) => console.log(rows.length));   // again after every write it may read
```

- **What the entry guarantees** is that it cannot pull in the engine,
    the sync layer or storage. The bytes it saves are few: a page that only
    calls `connect` never downloaded the module through
    `@fenecdb/web` either, since only `Fenec.open`
    fetches it.

- **The same builder:** a query moves between the client, a replica
    and a database in the page unchanged, and so do its types.

- **A live query over the server** runs the query there again after
    a write to a collection it reads; a FenecQL text names those with
    `{ collections }`.

Details

`@fenecdb/web/client` is `client.js`, which
re-exports the builder (`builder.js`) and the HTTP client
(`http.js`), the two modules `fenec.js` imports beside
its glue, persistence and the sync layer. It reaches nothing else — no
WebAssembly, no IndexedDB, no files — and a test fails if it does. An app
that calls `connect` and reads a builder query's rows bundles to
9 KB brotli through it and to 12 KB through `@fenecdb/web`,
whose `Fenec` class and its glue a bundler cannot drop (esbuild,
minified): 2.5 KB of JavaScript, and no `.wasm` either way.

Its live queries open a subscription to each collection the query reads
(`GET /<collection>/changes`), of a shape that holds no row:
the seed is empty, and every write to the collection is a change that runs
the query again on the server. The first rows wait for every stream's seed,
a burst of writes runs it once after the run under way, and a stream that
ends is opened again, its seed running the query, since writes may have
come between. `useLiveQuery` takes the endpoint as it takes a
replica. Under a scoped token the server tells such a stream of the writes
to rows the token may read, and of no other ([HTTP](https://fenecdb.com/docs//docs/http#subscriptions)).
A token refused, or lapsed under an open stream -- the server ends a stream
at its token's `exp` -- is a `FenecError` of status
401 to `onError`, and the live query stops there, for the app to
open it again with a fresh token.

### A shape's rows, as they change

`db.subscribe` keeps a shape -- the rows of a collection a
filter selects -- rather than a query's answer: a `seed` with the
whole shape, then each `change`, the rows written into it and the
ids that left it. The shape is the REST surface's: a field's filter,
`where` and `select`.

```ts
import { connect } from '@fenecdb/web/client';

const db = connect('https://db.example.com/t/acme', { token: 'a token' });
const tasks = new Map<number, unknown>();
const stop = db.subscribe('tasks', { team: 'eq.design', select: ['title', 'status'] }, (ev) => {
  if (ev.type === 'seed') {
    tasks.clear();
    for (const r of ev.rows) tasks.set(r.id, r);
  } else {
    for (const r of ev.puts) tasks.set(r.id, r);
    for (const id of ev.dels) tasks.delete(id);
  }
}, {
  onError: (e) => console.warn(e),          // status 401: a fresh token, then subscribe again
  onState: (s) => console.log(s),           // 'open' at each seed, 'retry' while it waits
});
```

A stream that ends -- a tenant moved, the network -- is opened again
(250 ms, doubling to 15 s) and seeds again; one refused or ended for its
token is not. `sseEvents(res)`, the reader under it, takes any
`fetch` answer of server-sent events: `EventSource`
cannot send `Authorization`.

### Batches, retries and polling

`db.batch` sends statements as one `/batch`: every
write lands or none does. Each is a builder query, a FenecQL text, or
`[text, params]` as `toInsert`, `toUpdate`
and `toDelete` make them; with an `idempotencyKey` a
retry after a timeout is answered as the first was.

```ts
import { connect, inc, FenecError } from '@fenecdb/web/client';

const db = connect('https://db.example.com', { token: 'a token' });
const stock = db.from('inventory');
try {
  const { results, replayed } = await db.batch(
    [
      stock.where('sku', 'TENT-1').where('available', '>=', 2).toUpdate({ available: inc(-2) }, { require: 1 }),
      db.from('orders').toInsert({ number: 'SG-1', sku: 'TENT-1', qty: 2 }),
    ],
    { idempotencyKey: 'checkout-7f3a' },
  );
  console.log(results, replayed);
} catch (e) {
  // 412: the statement at `at` wrote no row where it required one, and nothing landed.
  if (e instanceof FenecError && e.status === 412) console.log('sold out', e.at);
}
await db.run('put orders {number: $1}', ['SG-2'], { idempotencyKey: 'order-SG-2' });
await db.withIdempotencyKey('order-SG-3').from('orders').insert({ number: 'SG-3' });
// No stream held: the server answers 304, not running the query, while nothing it reads was written.
const stop = db.live(stock.select('available').where('sku', 'TENT-1'), (rows) => console.log(rows[0]), { poll: 5000 });
```

- **A refusal names its statement:** `FenecError`'s
    `at` (from 0), its `status` (412 a
    `require` not met, 409 a value taken, 422 a key sent with
    another request) and `completed`. A database in the page
    names it the same way: `db.run` of several statements, one
    block, throws with the `at` of the one that stopped it, and
    `db.batch` there takes the same items and answers as a
    server's, each statement with its own parameters.

- **A write answers where it left the database,**
    `seq`, and the client keeps the last as `db.seq`.

- **`{ poll }` is for a page many view:** a stream is a
    server thread and a wake-up at every write, a poll holds nothing between
    rounds ([Polling a read](https://fenecdb.com/docs//docs/http#polling)).

- **`{ wait: ms }` holds a claim at the server** until a
    job comes, on `update`, `delete`,
    `run` and `batch`, and `{ signal }`
    gives it up ([Waiting for a job](https://fenecdb.com/docs//docs/queues#wait)). A
    database in the page refuses it: there is no other worker to wait
    for.

- **`connect(url, { reads: 'nearest' })` reads near:**
    the builder's reads, and a `run` or `batch` given
    `{ read: true }`, are answered by the tenant's read replica in
    the router's region, each after the furthest position an answer from
    home named to this client, its change and its history — it reads nothing
    older than it wrote, nor a write a failover replaced; writes and
    subscriptions go home. A `live` query's streams stay home and
    each of its runs is a read near, sent after the change its stream told
    of and the rows it last showed too, so it shows every write home takes
    and never goes back; its polls go home (Read replicas). A write sent
    with `{ read: true }` is refused, 403.

## Opening a database

Copy `fenec.js`, the `builder.js` and
`http.js` it imports, and `fenec.wasm` next to your
page, open the module, and run FenecQL or the query builder.
`persist` keeps the database in IndexedDB and
`restore` reads it back.

```ts
import { Fenec, persist, restore } from './fenec.js';

const db = await Fenec.open('./fenec.wasm');

db.run('create collection docs (title text, embed vector<384> @hnsw(cosine))');
db.run('put docs {title: $1, embed: $2}', ['hello', embedding]);

const { rows } = db.run('get docs near embed $1 limit 5', [queryVector]);
const titles = await db.from('docs').select('title').limit(10).rows();   // the builder

await persist(db);     // into IndexedDB: the image once, then only the writes since
await restore(db);     // read it back
```

- **The examples are TypeScript.** The same code runs as plain
    JavaScript, and the module needs no build step: `fenec.d.ts`
    beside it holds the types ([TypeScript](#typescript)).

- **`db.run()` is synchronous;** the builder's endpoints
    return promises.

- **Text in a collation** also needs the `collate/`
    directory beside the module ([below](#collation)).

Details

`web/fenec.js` is a dependency-free ES module: the
WebAssembly glue, persistence and the sync layer, with the query builder
and the HTTP client in the two modules it imports and re-exports. Copy four
files — and the `collate/` directory beside the module, for text
in a collation — and you are done.

`db.run()` is synchronous because raw FenecQL is a single wasm
call. The builder's endpoints return promises, because a builder has to pick
its transport first.

## Keeping a database

In a dedicated worker, `openFile` keeps the database in a file
of the origin private file system — the same file `fenec-server`
keeps — and each write is flushed before `run` returns.
`persist` and `restore` keep the same bytes in
IndexedDB, and work anywhere a page runs.

```ts
// in a dedicated worker
import { Fenec, openFile } from './fenec.js';

const db = await Fenec.open('./fenec.wasm');
const file = await openFile(db, 'notes.fenec');   // loads what it holds, or takes the database's image
db.run('put notes {text: $1}', ['in the file before run returns']);
const bytes = file.bytes();                        // what fenec and fenec-server open
```

- **The file is portable:** download it and `fenec
    notes.fenec` opens it, and a server's file loads in a page.

- **One owner at a time:** a second worker or tab that opens the file
    is refused.

- **A crash leaves a whole file:** the next `openFile` holds
    every write `run` answered.

Details

`openFile` keeps the database in a file of the origin private
file system, byte for byte the file `fenec-server` keeps on disk:
download it and `fenec notes.fenec` opens it, and a server's file
handed to a page loads there. `run` appends each statement's
writes to it and flushes the file before it answers. It needs a dedicated
worker, the one place a page gets synchronous access to a file, and it takes
the file for itself: a second worker or tab that opens the file is refused,
where a second process over a server's file would corrupt it.

A `compact`, and writes appended past half the image (and past
64 KB), put a new image in the file. It is written into a copy beside the
file (`notes.fenec~`) and flushed before the file is touched, so a
crash at any point leaves one of the two whole, and the next
`openFile` takes whichever holds every write `run`
answered. A write a crash cut short is cut off. A new image holds the live
records alone, so what updates and deletes left behind goes with it, from
the file and from `persist`'s images alike; the page's memory
keeps those versions until it opens the file again or runs
`compact`, since a page has no thread to compact beside its
queries the way a server and an app do. Once the file refuses a write, every later write is
refused too; the database has to be opened from the file again.

`persist` and `restore` keep the same bytes in
IndexedDB, wherever a page runs: the image once, then only the writes since
the last call, as chunks. `persist` stores what was written when
it is called, and IndexedDB decides when that reaches the disk; the file is
flushed before every answer. Over 32.4 MB (61 000 rows with a
`vector<128>`), in a worker (`make file-bench`):

|  | IndexedDB, Chrome 153 | File, Chrome 153 | IndexedDB, Safari 26.1 | File, Safari 26.1 |

| the first image | 92 ms | 59 ms | 418 ms | 156 ms |

| a new row stored, p50 | 0.18 ms | 0.56 ms | 0.98 ms | 0.36 ms |

| opened again | 47 ms | 20 ms | 190 ms | 118 ms |

The file's row includes its statement, 0.03 ms in Chrome and 0.10 ms in
Safari, its vector handed over as `f32`s, and its flush. Firefox
was not measured.

## Sealed in IndexedDB

Pass a WebCrypto AES-GCM key and `persist` encrypts what it
stores; `restore` then needs the same key. Where the key comes
from is up to the page — never keep it in IndexedDB beside the data.

```ts
import { persist, restore } from './fenec.js';

// A key from a passphrase, never stored: the page asks for it again.
const salt = new TextEncoder().encode('myapp:' + userId);
const base = await crypto.subtle.importKey('raw', new TextEncoder().encode(passphrase), 'PBKDF2', false, ['deriveKey']);
const cryptoKey = await crypto.subtle.deriveKey(
  { name: 'PBKDF2', salt, iterations: 600000, hash: 'SHA-256' },
  base, { name: 'AES-GCM', length: 256 }, false, ['encrypt', 'decrypt']);

await persist(db, 'notes', { cryptoKey });   // the image and each chunk sealed
await restore(db, 'notes', { cryptoKey });   // another key, or none, is refused
```

Details

With a `cryptoKey` (AES-GCM, from WebCrypto) the image and
every chunk are sealed before they reach IndexedDB, each with a random nonce
and a tag over the key it is stored under and its place. A chunk moved,
dropped from the middle, or left from an image before is refused, as is one
byte changed. A restore without the key refuses a sealed record, and a
restore with one refuses a record stored in the clear. Where the key comes
from is the page's decision: derived from a passphrase, as above, or handed
over by the server after a login, so that what IndexedDB holds on the disk
reads as nothing without it. A key kept in IndexedDB beside the data protects
nothing. The sync layer takes it as `sync({ persist, cryptoKey })`.
Its cursors and unsent writes are collections of the replica, sealed with it. The same 32.4 MB in
Chrome 154: the first image 129 ms against 76, a new row 0.16 ms p50 either
way, a restore 37 ms against 29. Only the image pays: it is 32 MB to seal.
The file of `openFile` stays plain, since it is the file
`fenec-server` opens. The disk's own encryption covers it.

## State in the page

A database in the page can hold an app's whole state, with no server.
Declare its collections in code, and the open makes them.
`db.live(query, fn)` hands `fn` the query's rows now,
and again after every write to a collection the query reads. Event handlers
write, and `persist` keeps the state between visits.

```ts
import { Fenec, persist, restore } from './fenec.js';
import { fenecTable, text, boolean, timestamp, index } from './schema.js';

export const todos = fenecTable('todos', {
  key: text(),
  title: text().notNull(),
  done: boolean(),
  at: timestamp(),
}, (t) => [
  index('todos_key').using('hash', t.key),
  index('todos_done').using('hash', t.done),
  index('todos_at').on(t.at),
]);

const db = await Fenec.open('./fenec.wasm', { schema: { todos } });
await restore(db, 'todos');     // the stored database in its place, checked against todos

// the rows now, and again after every write to todos
const stop = db.live(db.from(todos).where('done', false).order('at'), draw);

async function add(title: string) {
  await db.from(todos).insert({ title, done: false, at: new Date() });
  await persist(db, 'todos');            // only this write, not the whole image
}
async function finish(id: number) {
  await db.from(todos).where('id', id).update({ done: true });
  await persist(db, 'todos');
}
```

In React, `useLiveQuery` from `@fenecdb/react` takes
the same database, and the rows it hands back are typed by the table:

```ts
import { FenecProvider, useFenec, useLiveQuery } from '@fenecdb/react';
import type { Fenec, SchemaOf } from './fenec.js';
import { todos } from './tables.js';   // the tables above, in a module of the app's

type Db = Fenec<SchemaOf<{ todos: typeof todos }>>;

function Todos() {
  const db = useFenec<Db>();
  const open = useLiveQuery(db.from(todos).where('done', false).order('at'));
  if (open === undefined) return null;
  return open.map((t) => (
    <li key={t.id} onClick={() => db.from(todos).where('id', t.id).update({ done: true })}>{t.title}</li>
  ));
}

root.render(<FenecProvider db={db}><Todos /></FenecProvider>);
```

To share the todos through a server later, open a
[synced replica](https://fenecdb.com/docs/sync) instead. The components stay as
they are: a replica has the same `from` and `live`,
and its writes reach the server. The server owns the schema there: the
tables are checked against it, and nothing is applied.

```ts
import { sync } from './fenec.js';
import { FenecProvider } from '@fenecdb/react';
import { todos } from './tables.js';

const db = await sync({
  url: 'https://todos.example.com',
  schema: { todos },
  shapes: [{ collection: todos, key: 'key' }],
});
root.render(<FenecProvider db={db}><Todos /></FenecProvider>);
```

The whole of it is the repository's
[react](https://github.com/fenecdb/fenec/tree/main/examples/react)
example: its database opens in the page or as a replica by one line of
`src/db.ts`, and the components are the same.

- **One run per task:** the writes made in one task, such as a
    `run` of several statements or a loop, run each live query
    once.

- **Only what it reads:** a query built with `db.from`
    knows its collections, including each `lookup`'s. A text
    names them: `db.live('get todos count', fn, { collections: ['todos'] })`.

- **A restore runs them all:** `restore`,
    `openFile` and `load` hand every live query the
    loaded rows.

- **`live` returns its stop function.** An error goes to
    `{ onError }`, or `db.onError`.

Details

The engine notes every write in its change ring, a collection id and a
document id each, and `fenec_changes(since)` names the collections
written since a cursor. A block's writes are noted as it lands, so a text of
several statements that fails names nothing. A `create`, an
`alter` and a `create index` name their collection. A
`drop` answers `null`, which means every collection:
a dropped collection has no name left to give. Its live query runs again and
gets the error that the collection is gone. Queries over other collections
also run again for nothing, which is harmless.

The notice costs a write nothing. A statement's answer does not carry
it. A page with a live query asks once, in a microtask after the task that
wrote: a write to a collection no live query reads costs that one call, about
2 µs in Node. Running a query again is the query itself: a page of 50 open
todos out of 1 000 to 3 000, ordered by an `@sorted` field, took
0.19 ms. No incremental maintenance is needed ([Sync](https://fenecdb.com/docs/sync)
says why). The module did not grow, and `make wasm-speed`'s
`put` did not move.

A synced replica runs its live queries at the next frame instead, so that
a seed landing in chunks is not shown half done. Both share one
implementation. A `load` runs every live query whatever the ring
says, since the image's counter may stand where the cursor did.

With `openFile`, the database and its live queries live in the
dedicated worker that holds the file. The worker runs them, and posts the
rows to the page that draws them. The page holds no database to subscribe
to.

The `key` field in the example is what a replica reconciles an
optimistic insert by. It fills the field in for you. Without a key, the row
shows up once the server has it, not at once.

## Schema in code

Declare the collections in TypeScript, the way Drizzle declares tables,
and open the database with them. The open compares the database with the
code: what only adds is made, and anything that could lose data is refused,
listed with how to resolve it. The tables also type every query.

```ts
import { Fenec } from './fenec.js';
import { fenecTable, text, integer, json, vector, index, uniqueIndex, defineRelations } from './schema.js';

export const products = fenecTable('products', {
  name: text().notNull().collate('tr'),
  sku: text().unique(),
  meta: json(),
  embed: vector({ dimensions: 384 }),
}, (t) => [
  index('products_name').using('bm25', t.name).with({ prefix: 6 }),
  index('products_embed').using('hnsw', t.embed.op('vector_cosine_ops')).with({ m: 16 }),
  uniqueIndex('products_lang').on(t.meta.path('lang')),
]);

export const reviews = fenecTable('reviews', {
  productId: integer().notNull(),
  stars: integer(),
}, (t) => [index('reviews_product').using('hash', t.productId)]);

export const relations = defineRelations({ products, reviews }, (r) => ({
  products: { reviews: r.many.reviews({ from: r.products.id, to: r.reviews.productId }) },
}));

const db = await Fenec.open('./fenec.wasm', { schema: { products, reviews }, relations });

type Product = typeof products.$inferSelect;    // { id: number; name: string; sku: string | null; ... }
const top = await db.from(products).select('name').lookup('reviews', { where: { stars: 5 } }).rows();
//    top: { name: string; reviews: { id: number; productId: number; stars: number | null }[] }[]
```

- **Made or checked at every open.** An empty database is made as
    declared. One that exists gets what only adds — a collection, a field,
    an index where there is none — in one block. A
    `restore` or `openFile` checks the database it
    brings in.

- **Refused, nothing is written.** A field the code lacks (dropped,
    or renamed?), a type changed, an index changed or taken off, a collation
    changed, a field made required: the open throws a
    `FenecError` that names each difference and how to resolve
    it.

- **Migrations resolve them.** `migrations: [rename('todos',
    'title', 'name')]` runs once, in order, recorded in the database's
    `_migrations`.

- **A schema is FenecQL too.** `schema` also takes the
    `create collection` statements of a
    `schema.fenecql` file, which every fenecdb SDK can hand
    over.

Details: declaring

A column builder takes no name: the key is the field's name. Columns are
nullable unless `.notNull()`, FenecQL's `required`. A
field takes one index, declared on the column (`.unique()`) or in
the third argument; two on one field are refused as the table is declared,
as is anything FenecQL cannot hold. An index over several fields is
`.on()` with several columns, each `.desc()` or
`.asc()` as Drizzle writes it, beside a field's own
([Indexes over several fields](https://fenecdb.com/docs//docs/fenecql#composite)), and
`dropIndex(table, ...fields)` is the migration that takes one
off.

| Declared | FenecQL |

| `text()` `integer()` `doublePrecision()` `boolean()` | `text` `int` `float` `bool` |

| `timestamp()` `bytea()` `json()` | `timestamp` `bytes` `json` |

| `vector({ dimensions: 384 })` / `halfvec(...)` | `vector<384>` / `vector<384, f16>` |

| `sparsevec({ dimensions: 30522 })` | `sparse<30522>` |

| `text().array()` | `[text]` |

| `.notNull()` `.collate('tr')` `.nocase()` `.unique()` | `required` `collate tr` `nocase` `@unique` |

| `index().on(t.at)` | `@sorted` |

| `index().on(t.at).ttl('30d')` | `@ttl(30d)` |

| `index().using('hash', t.n)` / `uniqueIndex().on(t.n)` | `@hash` / `@unique` |

| `index().on(t.owner, t.at.desc())` / `uniqueIndex().on(t.org, t.slug)` | `@sorted(owner, at desc)` / `@unique(org, slug)` |

| `index().using('hnsw', t.e.op('vector_l2_ops')).with({ m, ef_construction, ef_search, quant })` | `@hnsw(l2, m=.., ...)` |

| `index().using('bm25', t.body).with({ k1, b, prefix, prefix_min, chars })` | `@text(...)` |

| `index().using('inverted', t.splade)` | `@inverted` |

| `index().using('hash', t.meta.path('lang'))` | `create index on t (meta.lang) @hash` |

`typeof products.$inferSelect` is a row as read, `id`
and all, a nullable column `T | null`: the same type `fenec
types` generates for the collection, which the tests hold it to field
for field. `$inferInsert` is what `insert` takes: the
`.notNull()` columns given, the rest optional. `db.from(table)`,
`lookup(table, ...)`, a shape's `collection` and
`useLiveQuery` are typed by the table, and a misspelt field or a
wrong value's type is a compile error.

A relation of `defineRelations` is a `lookup`:
`on <to> = <from>`, its rows attached under the table
it reaches. `to` must be `id` or a column with a hash or
unique index, since `lookup` finds children by an index and never
by a scan; a relation it cannot answer is refused as it is declared.
`r.one` attaches the first child, still in a list.

Details: the check at an open

The engine compares, in the module or on the server, so every SDK applies
and refuses the same things with the same words. What it does:

| Difference | At an open | Resolved by |

| A collection, a field, an index where there is none, a path index | made | — |

| A collection the code does not declare | left as it is | `dropTable('t')` to drop it |

| A field the code does not declare | refused: dropped, or renamed? | `rename(...)` or `drop(...)` |

| A type changed | refused | a migration: add a field, `set` it from the old, drop the old |

| An index changed or taken off, a collation changed | refused | `rebuild('t', 'field')`; an expiry: `alter field` |

| A required field made optional | refused | `rebuild`: made again, not required |

| A field made required, or a new required field | refused | declare it as the database has it |

```text
FenecError: the database's schema differs from the code's, and nothing was applied:
  - `todos.title` is in the database and not in the code
    if it was renamed to `name`, the migration `alter collection todos rename field title to name`; if it goes, `alter collection todos drop field title`
```

Everything an open applies is one block: the migrations not yet
recorded, their records, and the additions land together, or — on a
refusal or an error — none of them. A database made from the code records
its migrations without running them, since they lead from schemas it never
had. A migration is known by its place in the list and held to its text:
one changed after it ran, or a database holding more migrations than the
code lists — an older app opening what a newer one migrated — is refused,
and nothing runs. `rebuild` adds the field again as declared,
copies its values with `set`, drops the old field and renames
the new one; a field it rebuilds is not required.

`db.describe()` is the database's schema as the engine
describes a server's (`GET /_schema`), and
`db.describe('fenecql')` the statements that make it.
`db.checkSchema(description, 'plan')` says what an open would
do and writes nothing, the pending migrations run in a block put back. The
check costs an open of 10 collections 0.09 ms in Node (0.11 ms without, 0.20
with), and the module 7.5 KB brotli (23 KB raw): the comparison and its
messages, and FenecQL read into a schema. `make wasm SCHEMA=0`
leaves it out.

Details: where it runs, and the description

A database in the page (`Fenec.open`, `restore`,
`openFile`) is the code's, and the check applies. A synced
replica's schema is its server's: `sync({ schema })` compares
the code with the server's (`POST /_schema/plan?mode=follow`),
refuses what the code declares and the server lacks, and applies nothing;
the replica takes its collections as the server declares them, as it
always did. `connect(url, { schema })` compares the same way;
with `migrate: true` and the server's token it runs the
migrations and the additions on the server (`POST /_schema/apply`),
as a deploy step runs drizzle-kit's `migrate`.

Every SDK's declarations compile to one description, versioned JSON:
`{"format": 1, "collections": [...], "migrations": [...]}`, a
field `{"name", "type", "required", "collate", "nocase", "index", "paths"}`
and an index `{"kind": "hnsw", "metric", "m", ...}` holding what
the declaration says, the engine's defaults for the rest — or the same
collections as FenecQL text, `{"format": 1, "fenecql": "create
collection ..."}`, which the module reads.
`integrations/schema-golden.json` holds declarations and the
description each compiles to, FenecQL texts, and plans with the statements
and refusals the engine makes of them, which the Rust tests run and every
SDK will be held to.

Details: coming from Drizzle, and which way to choose

One to one: a table function with columns and an array of indexes,
column builders without a name, `.notNull()`,
`.unique()`, `.array()`, `$type`,
`index` and `uniqueIndex` with `.on`,
`.using` and `.with`, pgvector's
`vector`, `halfvec` and `sparsevec` with
their operator classes, `$inferSelect`,
`$inferInsert`, `getColumns` and
`defineRelations` with `r.many` and
`r.one`.

fenecdb's own: `fenecTable` (also `collection`),
`bm25` and `inverted` methods,
`.ttl()`, `.collate()`, `t.meta.path(...)`,
and `quant`. Not there: `.default()` (FenecQL has no
defaults), `.primaryKey()` (every document has its
`id`), `.references()` (no foreign keys; a relation is
`defineRelations`), an index on two columns, and index names,
which are checked to be distinct and kept nowhere. Drizzle generates SQL
migrations from a diff; here the engine applies what only adds by itself,
and only what it refuses needs a migration, written beside the
schema.

Declare the schema in code for an app that makes its own database: a
page's state, an app's file, a service that owns its server. Generate the
types from the database ([TypeScript](#typescript)) for an app
over a database another owns, a server's whose schema changes elsewhere.
To move from the second to the first, `fenec types --schema
data.fenec > schema.ts` writes the tables the file holds, every
option written, and `fenec types --fenecql` the same schema as
FenecQL; an open with either finds no difference.

## Collation data

Text ordered with `collate und` or `collate tr`
needs each script's sorting data. The module carries Latin and fetches the
rest from `collate/` beside it the first time a page needs
them.

```ts
import { Fenec } from './fenec.js';

const db = await Fenec.open('./fenec.wasm');            // collate/ beside it
db.run('create collection people (name text collate und @sorted)');

await db.from('people').insert([{ name: 'Ζωή' }, { name: '山田' }]);   // fetches greek, han
const rows = await db.query('get people order name');                  // the same, raw

await db.collation('cyrillic', 'arabic');   // ahead of the first statement that needs them
await db.collation('all');                  // every script's: 145 KB, 67 KB gzip, 61 KB brotli
```

- **`db.query()` and the builder** fetch what is missing
    and run the statement again.

- **`db.run()`** cannot wait, so it throws a
    `FenecError` naming the chunks to load.

Details

`collate und` and `collate tr` need the weights of
the scripts they compare ([Collation](https://fenecdb.com/docs/fenecql#collation)).
The module carries the Latin ones and is handed the others as a page first
needs them, a chunk a group of scripts, from `collate/` beside the
module — or from where `Fenec.open(src, { collation })` says: a
URL, or a function handed the chunk's name that returns its bytes.
`make wasm` writes the chunks next to `fenec.wasm`.

A statement that would compare text of a script whose chunk the module
has not got is refused rather than answered in another order:
`db.run()`, which is synchronous, throws a
`FenecError` whose `collation` names the chunks, and
`db.query()` and the builder fetch them and run the statement
again. A write is refused before it changes anything, so running it again is
safe; of several statements in one text only the first is run again by
itself. `restore` and `db.loadAsync()` fetch what an
image's collated text needs — its `@sorted` indexes would be
out of order without it — and the sync layer fetches what its seed, the
changes it is sent and its optimistic writes need
(`sync({ collation })` says where from). A chunk of another
version's tables is refused: its ranks would mean something else.

## A smaller module

A page that does not use every feature can load a module built without
the rest. The release ships `fenec.wasm` alone, with every
feature; another set is a build option.

```bash
make wasm                              # every feature: 273 KB brotli
make wasm FEATURES="text sorted"       # those two alone, and the schema check
make wasm FEATURES=indexes             # the indexes, none of the rest
make wasm FEATURES=language SCHEMA=0   # no index, nor the schema check: 202 KB
make wasm FEATURES=none SCHEMA=0       # no feature at all: 179 KB
```

| Feature | What needs it |

| `vector` | nothing but speed: without the graph `near` over a vector field, and `fuse`, measure every vector, as `exact` does, with the same rows and scores |

| `text` | `match`, and the `rerank` after it, which reads the vectors themselves: BM25 needs the index's statistics, so without it `match` is refused |

| `sparse` | nothing but speed: without it `near` over a `sparse<N>` field scores every document, as `exact` does |

| `sorted` | nothing but speed: without it the scan answers a `@sorted` field's comparisons and orders, with the same rows |

| `composite` | nothing but speed: without it the scan answers an index over several fields, `@sorted(owner, created desc)`, and a `@unique(org, slug)` is asked by reading the rows; it needs `sorted` |

| `geo` | `distance()`, `within()`, `near` over a `geo` field, and its `@geo` index (with `sorted`; without it the scan answers): without it a point is read, written and checked, and the rest is refused |

| `nocase` | a text compared by its case fold: without it every statement over a collection declaring one is refused, since its text compared as written would be a wrong answer |

| `collate` | `collate und` and `collate tr`, ICU's order: without it a statement over a collection declaring one is refused, as is `order ... collate` |

| `facets` | `facet`, its value counts, ranges and disjunctive facets |

| `highlight` | `highlight()` and `snippet()`, which mark what `match` found and so need `text` too |

| `analytics` | `approx_count_distinct` and the `hll_*` sketches, and `bucket()`; every other aggregate — `count`, `sum`, `avg`, `min`, `max`, `count(distinct ...)`, `first` and `last` — and `having` are in every module |

What a `near` costs in a module built without
`vector`, `limit 10`, the median of 50 queries in Node
(`make wasm-exact-speed`):

| Rows × dimensions | `fenec.wasm`, through the graph | Built without `vector` |

| 1 000 × 128 | 0.20 ms | 0.38 ms |

| 10 000 × 128 | 0.22 ms | 2.4 ms |

| 50 000 × 128 | 0.34 ms | 11.7 ms |

| 1 000 × 384 | 0.47 ms | 0.90 ms |

| 10 000 × 384 | 0.51 ms | 6.2 ms |

| 50 000 × 384 | 0.75 ms | 29.4 ms |

Details

The module carries five indexes and six more parts of the language, and
a page that uses fewer can load one built without the rest:
`FEATURES` names what it keeps, `indexes` and
`language` each group. The release and `@fenecdb/web`
carry `fenec.wasm` alone; another set is a `make wasm`
away. What each feature weighs is in Benchmarks. The builds without
the indexes and without anything are also tests: CI builds them
(`make wasm-lite`, `make wasm-min`) and opens the
full module's files in them.

Everything else is in every build: documents, filters, `order`,
`count`, `sum`, `avg`, `min` and
`max` by `group`, `lookup`, `in (get
...)`, json fields and their paths, `@ttl`, the change
feed, persistence and the sync layer.

The file does not change with the build. A collection that declares an
index the module lacks is made and loaded all the same, and its documents
are read and written. A `near` over it measures every vector, a
sparse one scores every document; a `match`, or a `create
index` of the kind, throws a `FenecError` naming the
feature. A file declaring a `nocase` or collated field loads too;
a statement over that collection throws, the feature named, and the file's
other collections answer. A module that has the feature builds what it
needs when it loads what the other wrote, so a page can hand a store to the
other module and take it back.

## The query builder

The builder writes FenecQL for you, with every value passed as a
parameter. Each call returns a new query, so a base query can be shared and
extended with optional filters.

```ts
const rows = await db.from('articles')
  .select('title', 'year')
  .where('year', '>=', 2024)
  .where('tags', 'has', 'rust')
  .near('embed', queryVector, { ef: 128 })
  .limit(10)
  .rows();
```

- **Each `where` is `and`ed** with the ones
    before it.

- **Queries are immutable,** so `if (year) q = q.where(...)`
    branches safely.

- **No npm, no build step,** and no second schema definition.

Details

FenecQL reads well, but on the application side the job is still building a
string — and every interface with conditional filters ends up here:

```ts
let q = 'get articles where 1=1';
if (year) q += ` and year >= ${year}`;    // quote escaping and $n by hand
```

What usually drives people to an ORM is not the ORM but two things: dynamic
queries and type safety. The builder gives both, without npm, a build step, or
a second schema definition.

Every `where` is `and`ed with the previous one, and the
query object is **immutable** — so a body can be shared and branched:

```ts
let q = db.from('articles').select('title');
if (year) q = q.where('year', '>=', year);
if (tag)  q = q.where('tags', 'has', tag);
const rows = await q.limit(20).rows();
```

## Filters

A filter can be written three ways, which all build the same query. The
table lists everything else a query can carry.

```ts
const articles = db.from('articles');

articles.where('year', '>=', 2024);                          // field, operator, value
articles.where('category', 'book');                          // two arguments means equality
articles.where({ year: { gte: 2024 }, tags: { has: 'rust' } });
articles.where('meta.lang', '=', 'tr');                      // a path into a json field
```

A path into a [`json` field](https://fenecdb.com/docs/fenecql#json)
goes wherever a field does — `where`, `select`,
`order`, and the keys of `update`, which sets one key
inside the field — and a plain object is a `json` field's value:
`insert({ meta: { lang: 'tr' } })`. `fenec types`
writes a `json` field as `Json`, and the builder's
types take `'meta.lang'` as a path of it.

| **Operators** | `=` `!=` `<` `<=` `>` `>=` `~` `has` `in` |

| **Word forms** | `eq` `ne` `lt` `lte` `gt` `gte` `like` `has` `in` |

| **Null** | `{ summary: null }` becomes `is null`; `{ summary: { not: null } }` becomes `is not null` |

| **Combining** | `or(...)` `not(...)` `and(...)` — `where` already `and`s |

| **Escape hatch** | `raw('cosine(embed, ?) > ?', vector, 0.5)` |

| **Ordering** | `order('year','desc').order('title')` — successive calls add keys; `order('name', 'asc', { collate: 'und' })` puts text in Unicode's order for every language, `{ collate: 'tr' }` in Turkish order ([Collation](https://fenecdb.com/docs/fenecql#collation)) |

| **Ranking** | `near('embed', v, { ef, exact })`, `match('body', text)`, `rerank('embed', v, { candidates })` after a `match`, and `match(...).near(...).fuse({ k, candidates })` to rank by both ([Hybrid search](https://fenecdb.com/docs/fenecql#hybrid)) |

| **Relations** | `lookup('reviews', { on: 'product_id', limit: 3 })` — children per row, chainable |

| **Endpoints** | `rows()` `first()` `count()` `run()`, and `explain()`: the path the query took, one line a step ([Explain](https://fenecdb.com/docs/fenecql#explain)) |

| **Writes** | `insert(doc|docs)` `update(object)` `delete()`, each resolving to the rows it changed; `insert(doc, { ifAbsent: true })` passes over a held id or `@unique` value (`put … if absent`); `order(...)` and `limit(n)` before an `update` or a `delete` pick its rows, and `{ returning: true }` resolves to them — a job queue's claim ([Job queues](https://fenecdb.com/docs/queues)) |

| **Values worked out** | `update({ n: inc(1) })` renders `n: coalesce(n, 0) + $1`, and `expr('price * ?', 1.2)` any expression over the row — see [Writes that read the row](https://fenecdb.com/docs/fenecql#writes) and [Instead of Redis](https://fenecdb.com/docs/redis) |

Details: lookup, or, not and raw

Use whichever spelling reads better at the call site.

`lookup` hangs another collection's matching documents off each
row. Everything in its options object binds to that collection, so
`limit` there counts children per parent — twenty products with
three reviews each, in one query:

```ts
const rows = await db.from('products')
  .where('price', '>=', 10000)
  .limit(20)
  .lookup('reviews', {
    on: 'product_id',
    where: { stars: { gte: 4 } },
    order: [['created', 'desc']],
    limit: 3,
  })
  .rows();

// [{ id: 1, name: 'Kahve', price: 12000, reviews: [ {...}, {...} ] }, ...]
```

Calling it again chains rather than replaces: the second call binds to the
collection the first one named, the way position scopes it in FenecQL.

```ts
const rows = await db.from('shops')
  .limit(20)
  .lookup('orders', { on: 'shop_id', limit: 3 })
  .lookup('lines',  { on: 'order_id', limit: 5 })
  .rows();

// [{ name: 'Merkez', orders: [{ code: 'A', lines: [ {...} ] }, ...] }, ...]
```

A level's `select` may aggregate, as the query's own does:
each parent's children folded — one row a parent, or with
`group` one a group, ordered and paged a parent at a time.
Ungrouped, `order` and `limit` pick the children
folded. A query whose list aggregates hangs a lookup off its groups,
`parentKey` naming a column; and `bucket(field, interval,
zone)` counts by a zone's clock, the zone a parameter.

```ts
import { bucket } from './fenec.js';

// Each product's latest 50 reviews, counted and averaged.
const page = await db.from('products').limit(20)
  .lookup('reviews', { on: 'product_id', order: [['created', 'desc']], limit: 50,
                       select: ['count(*)', 'avg(stars)'] })
  .rows();
// The ten most reviewed, with their names.
const most = await db.from('reviews')
  .select('product_id', 'count(*)').group('product_id').order('count(*)', 'desc').limit(10)
  .lookup('products', { on: 'id', parentKey: 'product_id', select: ['name'] })
  .rows();
// Reviews a day, by the page's own clock.
const days = await db.from('reviews')
  .select(bucket('created', '1d', Intl.DateTimeFormat().resolvedOptions().timeZone).as('day'), 'count(*)')
  .group('day').rows();
```

The row type follows the nesting, so
`rows[0].orders[0].lines[0]` is typed without a cast; a level
that folds is typed as the children it folds, its columns not narrowed to.
`required` keeps meaning what it means at its own level: on
`lines` it drops the orders that have none, and the shops
stay unless `orders` asks for it too.

The child's field names cannot be reached from a builder that carries one
collection's types, so in TypeScript they default to a loose row; naming them
tightens it:
`.lookup<'reviews', FenecSchema['reviews']>('reviews', {...})`.

`or` and `not` are separate functions because the
`where` chain is an `and`. A nested group is written
explicitly and the parenthesising is left to the builder:

```ts
import { or } from './fenec.js';

db.from('articles')
  .where('year', '>=', 2024)
  .where(or({ tags: { has: 'rust' } }, { title: { like: 'rust' } }))
// get articles where year >= $1 and (tags has $2 or title ~ $3)
```

`raw` covers what the builder cannot express, which in practice
today means function calls. Its `?` placeholders bind to parameters
in order, so the escape hatch does not become string concatenation either.

## A search page

A results list with the words marked, and a sidebar counting each brand
and color over every match, from one query:
`highlight()` and `facet()`.

```ts
import type { Mark } from '@fenecdb/web';

/** The text with each mark in a <mark>, as text nodes: nothing in it is HTML. */
function marked(text: string, marks: Mark[]): DocumentFragment {
  const out = document.createDocumentFragment();
  let at = 0;
  for (const [start, end] of marks) {
    const m = document.createElement('mark');
    m.textContent = text.slice(start, end);
    out.append(text.slice(at, start), m);
    at = end;
  }
  out.append(text.slice(at));
  return out;
}

async function search(words: string, brand: string | null = null) {
  let q = db.from('products')
    .select('name', 'price')
    .highlight('name')
    .match('name', words, { fuzzy: true, prefix: true })
    .facet('brand', { top: 10 })
    .facet('color')
    .limit(20);
  if (brand) q = q.where('brand', brand);
  const rows = await q.rows();

  results.replaceChildren(...rows.map((r) => {
    const li = document.createElement('li');
    li.append(marked(r.name ?? '', r['highlight(name)'] ?? []), ` ${r.price}`);
    return li;
  }));
  sidebar.replaceChildren(...rows.facets.brand.map((f) => {
    const b = document.createElement('button');
    b.textContent = `${f.value ?? 'no brand'} (${f.count})`;
    b.onclick = () => search(words, f.value);
    return b;
  }));
}
```

- **It finds what is being typed**: `fuzzy` reads each
    word within a typo or two, `prefix` the last word as the
    start of one — `databse` and `datab` both find
    `database`, and the marks fall on it (Typos and words begun).

- **The offsets are a JavaScript string's own** (UTF-16), so
    `text.slice(start, end)` is the marked word, past an emoji
    or an accent.

- **`rows.facets` counts every match**, not only the
    twenty on the page; typed by the fields: `f.value` is a
    `string | null` here.

- **The same query live**: `useLiveQuery(q)` hands rows
    with their `facets`, as `db.live` does.

Details

`.highlight(field, { pre, post })` answers the marked text
instead of the offsets, and `.snippet(field, words, { ellipsis, pre,
post })` a window of that many words around the marks — see Highlights and snippets. Text with
tags is not escaped, which is why the page above builds its nodes from the
offsets. `.facet(field, { top })` takes a field or a path into a
`json` one, and `run()` hands the counts as its
`facets` too — see [Facets](https://fenecdb.com/docs//docs/fenecql#facets).
A facet beside `near` is refused: `near` ranks every
row rather than selecting some, so the counts would be the filter's alone.
`{ disjunctive: true }` counts a facet without the filter's own
conditions on its field, and `{ ranges: [0, 2500, 5000] }` counts
a number by range ([
Counted without its own filter](https://fenecdb.com/docs//docs/fenecql#disjunctive-facets), [Ranges](https://fenecdb.com/docs//docs/fenecql#range-facets)).

A query built in steps keeps the type it was declared with, so a facet
asked after the declaration is typed as any facet: `rows.facets`
is there, each field's counts `FacetCount[]`, and maybe absent.
To keep each field's own type, ask the facets in the declaration and add
the steps that keep the row's type after it:

```ts
import { connect } from '@fenecdb/web/client';

const db = connect('https://db.example.com');
const chosen: string | undefined = 'acme';
let q = db.from('products').select('name').facet('brand', { disjunctive: true });
if (chosen) q = q.where('brand', chosen);
const rows = await q.limit(24).rows();
rows.facets.brand.map((f) => f.count);   // typed
```

## Seeing what it produces

To see the text and parameters a chain builds, for logging or a test,
call `toFenecQL()`. It returns exactly what will run, and runs
nothing; a query needs no call to it before it runs.

```ts
db.from('articles').where('year', '>=', 2024).limit(10).toFenecQL()
// ['get articles where year >= $1 limit $2', [2024, 10]]
```

- **Values are always parameters,** never pasted into the text.

- **Collection and field names** are checked against FenecQL's
    identifier rule; anything else throws.

Details

Nothing is hidden. Every leaf value is bound to a parameter and text is
never embedded into the query, which leaves a single injection boundary:
**names**. Collection and field names cannot be parameterised, so they are
validated against FenecQL's identifier rule and anything that does not fit
errors out.

```ts
db.from('articles').where('title', '"; del articles; --')
// get articles where title = $1     ← a value, not text
```

A parameter that is a vector — a `Float32Array`, or any array
of finite numbers, which the module reads as a vector either way — goes into
the module as its `f32`s beside the JSON rather than as text:
200 768-dim vectors are put in 4.4 ms, against 21.9 written out as digits
and read back. The vector stored is the same, `-0` as
`0` as JSON writes it; over HTTP a parameter is JSON. Where such
a parameter is given a `json` field, or compared with a path
into one, the module names it before running anything and `run`
sends it again as JSON, so a plain array keeps its numbers as written and a
`Float32Array` goes as the numbers it holds.

## Independent of the transport

The same query code runs against the module in the page, a
`fenec-server` server or an HTTP endpoint: build it with
`from()`, then `bind()` it to where it should run.

```ts
import { from, connect } from './fenec.js';

const q = from('articles').where('year', '>=', 2024);

await q.bind(db).rows();                                // wasm, local
await q.bind(connect('http://localhost:8080')).rows();  // remote server
q.toFenecQL();                                          // or just the text
```

Details

`connect()` returns the same builder; the FenecQL it produces
goes to `POST /query` as is. See [HTTP
endpoint](https://fenecdb.com/docs/http).

## TypeScript

`fenec.d.ts` beside the module types the client, and
`@fenecdb/web` ships it. A schema declared in code types it
([Schema in code](#schema)); for a database another owns,
generate the schema type from the database file, and queries are checked
against its real schema: unknown fields, wrong value types and unselected
fields are compile errors.

```bash
fenec types data.fenec > web/fenec-schema.d.ts     # or: make types FILE=data.fenec
fenec types schema.fenecql > web/fenec-schema.d.ts # from a schema file, with no database
fenec types --lang python data.fenec              # a row in Python, Go, C#, Swift, Kotlin or Dart
```

```ts
import { Fenec } from './fenec.js';
import type { FenecSchema } from './fenec-schema.js';

const db = await Fenec.open<FenecSchema>('./fenec.wasm');

const rows = await db.from('articles').select('title', 'year').rows();
//    rows: { title: string | null; year: number | null }[]

await db.from('articles').insert({ title: 'Dunes', year: 2021, published: new Date() });

const products = await db.from('products')
  .lookup<'reviews', FenecSchema['reviews']>('reviews', { on: 'product_id', limit: 3 })
  .rows();
//    products[0].reviews[0].stars: number | null
```

Details

The schema is not written a second time — it is generated from the live
file, so the two cannot drift apart. A schema declared in code is written a
second time on purpose, and the check at every open is what keeps the two
one.

| Schema | TypeScript |

| `bool` `int` `float` `text` | `boolean` `number` `number` `string` |

| `timestamp` | `Timestamp` — ISO text on read; a `Date` or number works on write |

| `vector<N>` / `bytes` | `Vector` / `Bytes` — `number[]` on read, each component the shortest decimal that reads back as its `f32` (`0.1` as written, not `0.10000000149011612`); `Float32Array` also accepted on write |

| `[type]` | `type[]` |

| a field that is not required | `| null` — an unwritten field reads as `null` |

What the types catch: a collection or field that does not exist, a wrong
value type, access to an unselected field, `near` applied to a
non-vector field, and a type mismatch inside `insert`. Objects
inside `or`, `and` and `not` are checked
against the query's collection rather than the first argument. A query using
`near` or `match` gains `_score: number`.

A standalone `from()` cannot know the collection name, since the
schema lives in the call rather than in `db`. In TypeScript it binds
through `TypedFrom`; at runtime it is the same function and the only
difference is name completion.

```ts
import { from as fenecFrom, type TypedFrom } from './fenec.js';
import type { FenecSchema } from './fenec-schema.js';

const from: TypedFrom<FenecSchema> = fenecFrom;
from('articles').where('year', '>=', 2024);   // completes like db.from
```

`fenec.d.ts` is hand-written — it is the client's own surface —
and `fenec-schema.d.ts` is generated. There is no link between them:
the `Timestamp`, `Vector` and `Bytes` brands
are structural, so the generated file stands alone and imports nothing.

## What the builder will not do

The builder adds nothing FenecQL lacks, and refuses a few things that are
easy to get wrong.

- **No aggregate FenecQL lacks.** Aggregates go in
    `select` spelled as FenecQL spells them —
    `.select('status', 'count(*)', 'sum(total)').group('status')`
    — and each row answers under those names; a median or a distinct count is
    not there to compile to.

- **An `update` or `delete` without a filter is
    refused.** Covering a whole collection by accident is far too easy and
    impossible to undo; when it is deliberate you write
    `delete({ all: true })`. PostgREST decides the same.

- **`near`, `order`, `limit` and
    `lookup` error out on write statements.** FenecQL does not
    support them on `set` or `del`, and ignoring them
    silently would breed the illusion that `limit(1)` deletes a
    single row, or that a required `lookup` spares the parents
    with no children.

- **No JOIN, no `returning`.** The builder adds
    no capability on top of FenecQL — it only builds it safely.
    `lookup` is the exception FenecQL itself makes, and
    `.lookup(name, opts)` builds it like any other clause; a
    query given as an `in`'s value is FenecQL's
    `in (get …)`, its parameters bound with the outer one's.

Details: the tests

The Python, Go and .NET clients have the same builder, and
`integrations/builder-golden.json` holds all four to one text: the
chains there and what this builder makes of each, which every language's
tests run ([Your language](https://fenecdb.com/docs/languages#builder)).

The tests run with `node --test web/fenec.test.js` and are part
of `make test`. When `web/fenec.wasm` is present, an
end-to-end test that actually parses the generated text runs too.


---

Source: https://fenecdb.com/docs/mobile

# Mobile and native apps

Keep the database in a file on the device: no server, no
network, the same engine and file format as the server and the browser.

An iOS, macOS, Android or Flutter app opens a fenecdb file, writes to it
and asks it questions in FenecQL or through the query builder, with
`near`, `match` and `lookup` as anywhere
else. A live query hands the UI its rows again after every write to what it
reads.

|   | Swift | Kotlin and Java | Dart and Flutter |

| install | SwiftPM: `github.com/fenecdb/fenec`, product `FenecDB` | `com.fenecdb:fenecdb-android` (the AAR), `com.fenecdb:fenecdb` (the JVM) | `fenecdb_flutter`, or `fenecdb` in Dart alone |

| platforms | iOS 15, macOS 12 | Android 5 (API 21): arm64-v8a, armeabi-v7a, x86_64; the JVM | iOS, Android, macOS |

| calls | `async`, off the caller's thread | `suspend` on `Dispatchers.IO`; `...Blocking` for Java | `Future`s, on an isolate of the database's |

| live queries | `LiveQuery` (`ObservableObject`), `LiveRows` (`@Observable`), an `AsyncThrowingStream` | a `Flow`: Compose's `collectAsState` | a `Stream`: `StreamBuilder` |

| rows | `Row`, or any `Decodable` | `Row`, a `Map` with typed getters | `Map<String, Object?>` |

Details

Each binding calls the same native library, `crates/fenec-ffi`:
the engine built for the device, behind a C ABI of a few functions --
open, query, the change notice, sync, flush, checkpoint, close. A statement
goes in as text with its parameters as JSON, and its answer comes back as
the JSON the browser module answers with, from the same code
(`fenec-abi`), so a page and an app handed the same text get the
same bytes. A vector goes in as its `f32` bytes beside the
JSON, never as digits. The same file can be a replica a server keeps in
step ([Sync with a server](#sync)), and an app can ask a server
with no file at all ([Without a file](#connect)).

## A todo list

The file in the app's own directory, a collection made once, a list on
screen that is a live query, and writes from the buttons. The list redraws
itself: nothing calls it after a write.

import FenecDB
import SwiftUI

struct Todo: Codable, Sendable, FenecValue {
    var id: Int?
    var title: String
    var done: Bool
}

@main struct TodosApp: App {
    @State var db: Fenec?

    var body: some Scene {
        WindowGroup {
            if let db { TodoList(db: db) } else { ProgressView().task { db = try? await open() } }
        }
    }

    func open() async throws -> Fenec {
        let dir = try FileManager.default.url(for: .applicationSupportDirectory, in: .userDomainMask,
                                              appropriateFor: nil, create: true)
        let db = try await Fenec.open(dir.appendingPathComponent("todos.fenec"))
        try await db.execute("create collection if not exists todos (title text, done bool @hash)")
        return db
    }
}

struct TodoList: View {
    let db: Fenec
    @StateObject var open: LiveQuery<Todo>

    init(db: Fenec) {
        self.db = db
        _open = StateObject(wrappedValue: LiveQuery(db, try! db.from("todos").where("done", false), as: Todo.self))
    }

    var body: some View {
        List(open.rows, id: \.id) { todo in
            Button(todo.title) {
                Task { try await db.from("todos").where("id", todo.id!).update(["done": true] as Value) }
            }
        }
        .toolbar {
            Button("Add") { Task { try await db.from("todos").insert(Todo(title: "milk", done: false)) } }
        }
    }
}

On iOS 17 and macOS 14, `LiveRows` is the same as an
`@Observable`, held in `@State`.

import com.fenecdb.*

class TodosModel(app: Application) : AndroidViewModel(app) {
    val db = Fenec.open(File(app.filesDir, "todos.fenec").path)   // in a background start-up, or openAsync

    init {
        db.executeBlocking("create collection if not exists todos (title text, done bool @hash)")
    }

    val open: Flow<List<Row>> = db.live(db.from("todos").where("done", false))

    fun add(title: String) = viewModelScope.launch { db.from("todos").insert(mapOf("title" to title, "done" to false)) }

    fun finish(id: Long) = viewModelScope.launch { db.from("todos").where("id", id).update(mapOf("done" to true)) }

    override fun onCleared() = db.close()
}

@Composable
fun Todos(model: TodosModel) {
    val open by model.open.collectAsState(emptyList())
    LazyColumn {
        items(open) { todo -> Text(todo.string("title") ?: "", Modifier.clickable { model.finish(todo.long("id")!!) }) }
    }
    Button(onClick = { model.add("milk") }) { Text("Add") }
}

A live query is a conflated `Flow`: a collector that falls
behind sees the latest rows.

import 'package:fenecdb_flutter/fenecdb_flutter.dart';
import 'package:flutter/material.dart';
import 'package:path_provider/path_provider.dart';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();
  final dir = await getApplicationSupportDirectory();
  final db = await Fenec.open('${dir.path}/todos.fenec');
  await db.execute('create collection if not exists todos (title text, done bool @hash)');
  runApp(MaterialApp(home: Todos(db)));
}

class Todos extends StatelessWidget {
  final Fenec db;
  const Todos(this.db, {super.key});

  @override
  Widget build(BuildContext context) => Scaffold(
        body: StreamBuilder(
          stream: db.live(db.from('todos').where('done', false)),
          builder: (context, snap) => ListView(children: [
            for (final todo in snap.data ?? const <Map<String, Object?>>[])
              ListTile(
                title: Text(todo['title'] as String),
                onTap: () => db.from('todos').where('id', todo['id']).update({'done': true}),
              ),
          ]),
        ),
        floatingActionButton: FloatingActionButton(
          onPressed: () => db.from('todos').insert({'title': 'milk', 'done': false}),
          child: const Icon(Icons.add),
        ),
      );
}

The plugin's `example/` is this app, built for Android and
the iOS simulator on every change.

A fuller app, Notes -- search with `match` and
`fuse`, filters, a schema file, an add form -- is in the
repository for each: [SwiftUI](https://github.com/fenecdb/fenec/tree/main/examples/swift)
(and a CLI), [Jetpack
Compose](https://github.com/fenecdb/fenec/tree/main/examples/kotlin-android) (and a JVM CLI) and [Flutter](https://github.com/fenecdb/fenec/tree/main/examples/flutter),
each built on every change and its headless part run.

## The builder

The same query builder as JavaScript, Python, Go and .NET:
`from`, `select`, `where` with every
operator, `near`, `match`, `fuse`,
`rerank`, `highlight`, `snippet`,
`facet`, `lookup`, `order`,
`limit`, and the writes, each value a parameter and each name
checked. All seven make the same text of the same chain, to the byte, held
to one golden file; each language's spelling is on
[Your language](https://fenecdb.com/docs/languages#builder). A builder query also
knows the collections it reads, which is what a live query runs again for.
To see the text and parameters a chain builds, for logging or a test,
`toFenecQL()` returns them and runs nothing.

```swift
let near = try await db.from("notes")
    .select("title")
    .where("lang", "en")
    .near("embed", embedding, ef: 64)   // a [Float]
    .limit(5)
    .rows(as: Note.self)
```

## The schema in code

An app that makes its database declares it once, as
`create collection` and `create index` statements --
a `schema.fenecql` in the app's resources -- and hands it to
`schema` after each open. The engine compares the file with it:
what only adds (a collection, a field, an index) is made, the migrations
not yet recorded run first, in order, and are kept in
`_migrations`, all one block; a field the text lacks, a type or
an index changed, is refused with what resolves it, and nothing is written.
It is the check the browser's `Fenec.open({ schema })` and the
server's `/_schema` run ([Schema in
code](https://fenecdb.com/docs/javascript#schema)).

let plan = try await db.schema(schemaText, migrations: [
    "alter collection notes rename field title to name",
])

val plan = db.schema(schemaText, migrations = listOf(
    "alter collection notes rename field title to name",
))

final plan = await db.schema(schemaText, migrations: [
  'alter collection notes rename field title to name',
]);

`fenec types --lang swift|kotlin|dart schema.fenecql` writes
a row of each collection as the language reads it: a `Codable`
struct, a data class with `from(row)`, a class with
`fromJson`.

## Live queries

A live query hands over its rows at once, and again after a write to a
collection it reads. The writes of a burst -- several at once, a loop, a
text of several statements -- run it once, a frame after the last of them
lands. A write to another collection runs nothing, and a text that fails
is put back whole and runs nothing either.

Details

The engine keeps a ring of the writes it landed, and
`fenec_changes(since)` names the collections written since a
cursor -- or answers "everything" when the cursor fell behind the ring or a
collection written since was dropped. A write asks for a look; the looks
asked for within a frame (16 ms) are one, taken once no write through the
database is under way, and each query that reads a collection named runs
again from scratch -- well under a millisecond over a local file, where
keeping track of which rows changed would cost more than the query. Knowing
costs a write nothing: the ring is asked once after a burst, and only by a
database holding a live query. It is how the browser's
`Fenec.live` works, with a frame in place of a microtask: an
app's writes run on other threads and come back one at a time.

A builder query reads its own collection and each `lookup`'s; a
FenecQL text names its collections, or runs after every write.

## Sync with a server

The todo list kept in step with a server, every device seeing every
change: open the file as a replica of the rows you want. That is the one
line that changes -- the list is still a live query of the file, and a tap
still writes to it.

let db = try await Fenec.sync(url: "https://api.example.com", token: jwt,
                              shapes: [Shape("todos", key: "key")],
                              path: dir.appendingPathComponent("todos.fenec").path)

val db = Fenec.sync(url = "https://api.example.com", token = jwt,
                    shapes = listOf(Shape("todos", key = "key")),
                    path = File(app.filesDir, "todos.fenec").path)

final db = await Fenec.openSynced(url: 'https://api.example.com', token: jwt,
                                  shapes: [Shape('todos', key: 'key')],
                                  path: '${dir.path}/todos.fenec');

The collection is made on the server, with a key the app's inserts are
matched by -- `create collection todos (key text @unique, title text,
done bool @hash)` -- and comes to the replica from there, so the
`create collection` line goes. Dart's is
`openSynced`: `sync()` is the fsync, and Dart has no
static and instance member of one name.

- **Reads are the file's.** A query, a live query and the builder
    read the replica: no network, and the app works offline.

- **A shape is a set.** The rows of one collection a filter on the
    server picks, `Shape("todos", where: ["owner": me])`, in the
    JS sync layer's form; one shape a collection. Order and limits are a
    query's, not a shape's.

- **A write shows at once.** It is applied to the replica as it is
    made, and sent. An insert gets a temporary id and a key, and the
    server's copy takes its place when it arrives.

- **A refused write is put back.** When the server refuses it -- a
    `409` for a unique value held, a `403` from a
    token's rules, a `507` from a tenant's quota of bytes or rows
    (`quota` names it) -- the replica is put back as it was and
    the app is told: `db.replica.refusals`.

- **Offline, writes wait.** A write made while the server cannot be
    reached waits in the file, and goes once it can, in order, once each.
    An app closed meanwhile sends it when it opens again.

- **It catches up from where it stopped**, after a dropped
    connection, a server restart or a week in a drawer -- or is sent the
    whole shape again when the server's ring of changes no longer reaches
    back that far.

|   | Swift | Kotlin | Dart |

| open | `Fenec.sync(url:token:shapes:path:)` | `Fenec.sync(url, token, shapes, path)` | `Fenec.openSynced(url:, token:, shapes:, path:)` |

| status | `replica.status`, `statuses()` (an `AsyncStream`) | `replica.status` (a `StateFlow`) | `replica.status`, `statuses` (a `Stream`) |

| first seed, writes answered | `ready()`, `pushed()` |

| refused writes | `refusals()` | `refusals` (a `SharedFlow`) | `refusals` |

| a new token | `setToken(jwt)`, or a `tokenProvider` asked on a `401` |

| the network | `setOnline(bool)`, `resume()` |

| the server itself | `replica.remote` ([below](#connect)) |

The status says `online`, `offline` or
`catchingUp` (connecting, seeding, or sending the writes left
from before), how many writes the server has not answered, and the last
error.

Details

What to ask, what an answer means and what is written are one state
machine in Rust (`fenec_abi::sync`), with no network of its own:
a binding feeds it what happened -- a request's answer, a stream's bytes, a
timer, the network, a new token -- and does what it hands back: make this
request, open this stream, wait this long. So the logic exists once, with
one suite of tests driving it through scripted server event sequences, and
each binding's loop is a page over its platform's HTTP client. A write to a
shape's collection through `run`, the builder or
`execute` goes to it on its own: applied to the replica, what
puts it back kept, and the write queued with an idempotency key -- all one
block in the file. Writes go one at a time, in order, each under its key, so
a write whose answer was lost is sent again and answered as the first time,
not made again. A server's answer of `0` (no answer),
`408`, `429` or `5xx` is tried again with
backoff, 250 ms doubling to 15 s, or after the wait a tenant's quota (to
the millisecond, `retry_after_ms`), a move or a failover
(`retry_after`) names where that is longer; a
`401` asks for a token; a `507` naming a tenant's
quota, and any other `4xx`, is a refusal, and the writes behind
it go on
([why](https://fenecdb.com/docs/sync#refusals-retries-and-offline)). The status's
error names the quota too: a stream refused by the quota of streams leaves
the replica `offline`, as the network does. A change applies in one block with the
shape's cursor, so a crash leaves the two agreeing. A change saying the
server's schema changed, or rows with a field the replica has not got, has
the collection made again from `GET /collections` -- its rows kept
in the fields that are left -- and the whole shape sent again.

The file holds three collections of the sync's own beside the
replica's: `_sync_shapes` (each shape's cursor),
`_sync_queue` (the writes not yet answered) and
`_sync_temps` (rows an insert made under a temporary id, until
the server's copy). A collection with no shape is the app's own, written
and read as in a file of its own; a synced collection's schema is the
server's, and `create index` or `drop` over it is
refused. A change of one row costs the replica 18.4 µs against 7.1 for the
same put straight into the engine (the cursor is written in the same
block), rows in changes of 100 cost 3.8 µs either way, and a seed of 10 000
rows 72 ms (`make sync-bench`).

The browser's `sync` ([Sync](https://fenecdb.com/docs/sync)) is
written apart -- this core in the browser module was 21 KB brotli more --
and the two are held to one script: `integrations/sync-scenarios.json`,
71 scenarios of shapes, writes, answers and server events, each step's
replica, queue and requests stated once and run against both. What is left
between them is the platforms':

- A write returns once it is applied and kept: `await` on a
    button would hang offline. A refusal comes on `refusals`, and
    `pushed()` waits for the answers. In the browser the write's
    promise settles with the server's answer, which a page need not await;
    `onRefused` and `pushed()` are there too.

- A text of several writes is one write, sent as one
    `/batch`; the browser's builder writes one statement at a
    time, and `batch()` makes several one write the same way.

- There is one replica a file, held by one process, so no leader is
    elected among tabs; the file is the persistence, where a page stores its
    replica in IndexedDB (`persist`) or an `openFile`.

- Every collation is in the library; the browser module fetches a
    script's chunk the first time a row needs it.

### TLS on devices

iOS (App Transport Security) and Android refuse plain `http://`
by default, and so should you: put TLS in front of
`fenec-server`. Caddy does it in three lines, certificate
included:

api.example.com {
    reverse_proxy 127.0.0.1:8080
}

Each binding makes its requests with the platform's client --
`URLSession`, `HttpURLConnection`,
`dart:io`'s `HttpClient` -- so the system's trust
store, proxies and pinning apply. An Android app declares the
`INTERNET` permission and a sandboxed macOS app the
`com.apple.security.network.client` entitlement, as for any
request. During development, an emulator reaches
a server on the machine at plain `http://` once allowed: iOS's
`NSAllowsLocalNetworking`, Android's
`network_security_config` with `cleartextTrafficPermitted`
for `10.0.2.2`.

Details

The change stream is Server-Sent Events: a response that does not end.
Caddy passes `text/event-stream` through as it comes; behind
nginx, the server's `X-Accel-Buffering: no` does the same. A
proxy that closes an idle connection sooner than the server's keepalive
(a comment every 20 s) cuts the stream, which then opens again from its
cursor -- harmless, a round trip each time. fenecdb's own HTTP client, the
server's, speaks no TLS and is not used on a device.

### Battery and the background

A replica holds one connection a shape open while the app runs, quiet
but for the keepalive. iOS suspends an app soon after it leaves the
screen, and Android stops one's network in Doze: the streams are cut, and
nothing is lost -- a write waits in the file, a change waits on the server.
As the app comes back, call `resume()` (on
`scenePhase == .active`, `onStart`,
`AppLifecycleState.resumed`): the streams open at once from their
cursors, the waiting writes go first, and the live queries run again as the
changes land.

Details

`setOnline(false)` as the app leaves the screen closes the
streams at once rather than when the system cuts them, and spares the radio
the keepalive. Backoff after a failure doubles from 250 ms to 15 s, with
jitter, so a server back up is not met by every device at once;
`resume()` and `setOnline(true)` forget it. Nothing
here runs in the background: a write made just before suspension is sent on
the next resume, or by an app's own background task (iOS's
`BGAppRefreshTask`, Android's WorkManager) that opens the replica
and waits for `pushed()`. A replica away longer than the server's
ring of changes reaches back (`--changes`, 4 096 writes by
default) is sent its shapes whole on return.

## Without a file: the server alone

An app that keeps nothing on the device asks the server directly:
`Fenec.connect` makes every query a request, through the same
builder, as `connect` does in JavaScript.

```swift
let db = Fenec.connect(url: "https://api.example.com", token: jwt)
let open = try await db.from("todos").where("done", false).rows()
```

Kotlin's is `Fenec.connect(url, token = jwt)`, Dart's
`Fenec.connect(url, token: jwt)`; a replica's
`replica.remote` is the same, for what the replica does not
hold. Live queries are the file's alone.

Several writes that must land together are one `batch`
(`POST /batch`): the builder's `toInsert`,
`toUpdate` and `toDelete` under one write lock, all
of them or none. One that stops it is an error naming its place
(`at`, from 0) and the HTTP `status` -- 412, code
`unmet`, for a write whose `require` was not met. A
retry after a timeout cannot tell whether the write ran, so a write that
must not run twice goes under an idempotency key: the second answer is the
first one kept (`replayed`), and the key sent with another
request is refused (422).

```swift
let accounts = try db.from("accounts")
let done = try await db.batch([
    try accounts.where("id", from).where("balance", ">=", 10).toUpdate(["balance": .inc(-10)] as Value, require: 1),
    try accounts.where("id", to).toUpdate(["balance": .inc(10)] as Value, require: 1),
], idempotencyKey: transferID)
// done.results, done.seq, done.replayed
try await db.withIdempotencyKey(orderID).from("orders").insert(order)
```

Kotlin's is `db.batch(listOf(...), idempotencyKey = id)` and
`runList(text, params, idempotencyKey = id)`, Dart's
`db.batch([...], idempotencyKey: id)`; all three have
`withIdempotencyKey(key)`, a copy whose every write -- the
builder's too -- carries the key, and `seq`, the furthest change
a write left the server at, through the connection or any copy of it.
`withWait` -- Swift's in seconds,
Kotlin's in milliseconds, Dart's a `Duration` -- is a copy whose
`set`s and `del`s the server holds while they write
nothing, a worker's claim waiting there for a job
([Waiting for a job](https://fenecdb.com/docs//docs/queues#wait)); cancelling the task or
the coroutine gives it up.

An app far from its tenant's home reads near itself where the tenant has
a [read replica](https://fenecdb.com/docs//docs/sharding#read-replicas) in the app's
region: connected through that region's router with reads nearest, the
builder's reads, and an `asRead()` copy's requests, are answered
by the replica there, each after the furthest position an answer from home
named to this connection -- its change and its history, shared by every
copy -- so the app never reads older than it wrote; writes go home.

```swift
let db = Fenec.connect(url: "https://ap.example.com/t/acme", token: jwt, reads: .nearest)
try await db.from("orders").insert(["number": "SG-1"] as Value)   // home, in eu
let mine = try await db.from("orders").rows()                      // ap's replica, after that write
let n = try await db.asRead().query("get orders count")           // a text that only reads
```

Kotlin's is `Fenec.connect(url, token = jwt, reads =
FenecRemote.Reads.NEAREST)`, Dart's `Fenec.connect(url, token:
jwt, reads: Reads.nearest)`, and all three have
`asRead()`. A write sent through `asRead()` is
refused (status 403), never landed.

## Where the file lives

Anywhere the app may write. On iOS the app's Application Support
directory, which is backed up and never shown in the Files app; on Android
`context.filesDir`, or `noBackupFilesDir` to keep it
out of a backup; in Flutter `path_provider`'s
`getApplicationSupportDirectory()`. Beside the file the library
keeps `<name>.lock` while it is open.

- **Open a file once.** A second open of the same file, in the app or
    in another process -- an app extension sharing the app group's container
    -- is refused (`locked`): two databases over one file corrupt
    it.

- **iOS file protection.** A file under the `complete`
    protection class cannot be read while the device is locked, and a page
    of a mapped file read then is the app's end, not an error. Keep the
    default class (complete until first user authentication), or open the
    file with `.inMemory`.

- **The file compacts itself.** An update writes the row again at the
    file's end; a thread of the library's looks every 5 s and compacts the
    file beside your calls once half of it is versions updates and deletes
    left behind, and at least 64 MB, so a file you keep updating stays under
    about twice what it holds -- sooner once it reaches three quarters of
    the memory the device has for its pages, so its reads stay in
    memory. Open with `.noAutoCompact`
    (`NO_AUTO_COMPACT`, `noAutoCompact`) to compact
    only when you run `compact`.

Details

The file is mapped, as a server maps its own: the documents stay in it
and the app holds only what it derived from them, the indexes and the
graph. The mapping takes address space past the file's end -- a gigabyte at
least -- for the writes to grow into; where the system refuses that much, as
an iOS app's address space may, the file alone is mapped and mapped again
as it grows. A 32-bit Android device (armeabi-v7a) reads the file into
memory. The lock is a file of its own, held with a record lock
(`fcntl`), which belongs to the process: a checkpoint renames a
new file over the database, which a lock on the database itself would not
survive, and an `flock` belongs to the open file, which a child
process started meanwhile shares until it runs its program -- a close and
an open again at that moment were refused.

## Durability

Every write is on disk before the call returns: written and fsynced,
outside the database's lock, so reads go on meanwhile and writes arriving
together share one fsync. On a Mac that fsync is about 4 ms. For a burst of
writes, open the file without it and sync when they are done:

|   | what it does | what it survives |

| a write, by default | written and fsynced before it returns | the device losing power |

| `noSync`, then `sync()` | writes wait in a buffer, then one fsync | the device losing power, once synced |

| `flush()` | the buffer handed to the system, no fsync: microseconds | the app being killed |

| `checkpoint()` | the file written anew, graphs and all | -- (the next open links nothing) |

| `close()` | saves the graphs, syncs, lets the file go | everything |

A phone ends an app in the background without a word, so an app opened
with `noSync` calls `flush()` or `sync()`
as it goes there -- `scenePhase` turning to
`.background`, `onStop`, `AppLifecycleState.paused`.

Details

With `noSync` the buffer goes into the file once it holds
64 KB, written by the call that filled it after it has let go of the
database, as a server's buffered writes are: a read beside texts of 1 000
rows of 128 dimensions waits 1.1 to 1.4 ms at p99, where written under the
lock, the disk's waits and all, it waited 2.2 to 6.6 ms and up to 61.

A write the disk refuses stops the database's writes, as on a server:
every later write and sync is refused with `io` until the file
is opened again, and reads go on. The vectors a database holds are linked
into its graph as they are written; the open links those written since the
graph was last saved. `close()` saves a graph once the file has
grown three times the graph's record since its last save, so an app that
opens and closes often does not grow its file by a graph each time;
`checkpoint()` saves everything, and an open after it links
nothing.

The hash, text, ordered and sparse indexes are never in the file: an
open leaves each for the first statement that reads it, which builds it
first -- a `@text` index of 100 000 products took that search
141 ms. `warm()` builds them off the main thread as the app
starts, each under the read lock on its own so reads go on beside it, and
answers how many it built; `warm(["docs", "orders.status"])`
builds those collections' and fields' alone (Kotlin
`warm(listOf(...))`, Dart `warm([...])`).

## Threads

One database is safe to use from every thread, task or coroutine at
once: a read runs beside other reads, a write takes the database to itself,
a long read — an aggregate, a scan, a text of several reads — pins what it
reads and lets writes land beside it, as a server's does, and a lone
`create index` or `compact` is built beside the
database while reads and writes go on. Every binding makes its calls
off the main thread, since any of them may wait -- for the lock, an fsync, a
whole open.

## Sizes and speed

| the native library, stripped | size |

| macOS, Apple silicon | 1.68 MB |

| macOS, Intel | 1.77 MB |

| Linux arm64, with the JNI functions | 1.71 MB |

| Linux x86-64, with the JNI functions | 1.88 MB |

The sync with a server is 165 to 198 KB of each. iOS's and Android's
are measured by CI for every change, in the run's summary. An app ships the code it links: the XCFramework's static library
is the objects before the app's linker takes what it uses.

| 20 000 × 128, Apple M1 | through the library | fenec-server's handler, in process |

| open, checkpointed | 2.9 ms | 2.6 ms |

| a put with its vector, buffered | 4.6 µs | 23.2 µs |

| a put with its vector, fsynced | 4.0 ms | 4.0 ms |

| `near`, `limit 10` | 97.8 µs | 151.4 µs |

Details

`make ffi-bench` makes the file through the library and
opens it both ways: the library as an app opens it, the server's handler as
`fenec-server` serves it, the request built in process with no
socket. The medians of 1 000 calls each. The library is handed a vector as
its `f32` bytes, as the bindings hand it; as JSON a buffered put
takes 23.3 µs, the server's time, most of it the vector's digits.

## The C ABI

Any language with a C FFI can call the library: the header is
`crates/fenec-ffi/include/fenec.h`, and `make ffi`
builds the library for this machine, or `TARGET=...` for another.
Every call returns a code -- 0, an error's kind, or the boundary's own:
`FENEC_MISUSE`, `FENEC_LOCKED`,
`FENEC_PANIC` -- and writes its answer or its error as
NUL-terminated JSON, freed with `fenec_free_string`.

Details

A handle is a number, not a pointer: a handle used after its close finds
nothing and is refused, where a pointer would reach freed memory, and a
call holds its database for as long as it runs whatever closes meanwhile.
A panic never crosses the boundary: each call runs under
`catch_unwind` and answers `FENEC_PANIC` with its
message, which is why the library is built to unwind, not to abort. The
Kotlin binding reaches the same calls through JNI functions the library
carries itself (its `jni` feature): each call answers one
`byte[]`, its code then its JSON, and text goes in as UTF-8
bytes, since JNI's own strings write an emoji as two halves.


---

Source: https://fenecdb.com/docs/http

# HTTP endpoint

Talk to fenecdb with `fetch`, `curl` or
any standard library — no driver needed.

## Start it

Add `--http` to `fenec-server`. Every collection gets REST
routes from its schema.

```bash
fenec-server --file data.fenec --http 127.0.0.1:8080

curl 'http://127.0.0.1:8080/articles?select=title,year&year=gte.2024&tags=has.rust&order=year.desc&limit=10'
# [{"title":"the rust book","year":2024}]

curl 'http://127.0.0.1:8080/articles?year=gte.2024&count'
# {"count":1}

curl -X POST http://127.0.0.1:8080/articles -d '{"title":"new","year":2025}'
# {"inserted":1}

curl -X POST 'http://127.0.0.1:8080/articles?returning=id' -d '{"title":"newer","year":2026}'
# [{"id":3}]
```

Details

For clients that want no driver at all: `fetch`,
`curl`, or any language's standard library. The schema is already in
the database, so the surface is derived from it.

**Not a separate binary.** The HTTP endpoint is a second listener in
  the same process as `fenec-server`. fenecdb has a single writer, and
  two processes writing one file corrupt it. Being in the same process also
  keeps the sync policy, the checkpoint, the memory ceiling and the shutdown
  signal in one place — `--sync always` covers HTTP writes too.

## Routes

```http
GET    /                        version and collection list
GET    /collections             schemas
GET    /<name>?<filter>         rows
GET    /<name>?<filter>&count   number of matching rows
GET    /<name>?select=status,count(*),sum(total)&group=status&order=sum(total).desc
                                aggregates, whole or per group
GET    /<name>?<filter>&facet=brand%20top%205,color
                                rows, and each value's count beside them
POST   /<name>                  body: {...} or [{...}] -- an id that is taken: 409
POST   /<name>?returning=id,title   the rows made, ids and all (* every field), not their count
PATCH  /<name>?<filter>         body: {...}
DELETE /<name>?<filter>
PATCH  /<name>/all              no filter — deliberately explicit
DELETE /<name>/all
POST   /<name>/near             body: {"vector":[...], "limit":10}
GET    /<name>/changes?since=N  subscription (SSE): shape plus incremental diff
POST   /query                   body: {"query":"<FenecQL>","params":[...]}
POST   /batch                   one query body per line (NDJSON)
GET    /_metrics                counters and gauges, Prometheus's text format
GET    /_changes?since=N        every write on disk after N, a JSON line each (--cdc)
GET    /_changes?consumer=C     from where consumer C committed
POST   /_changes/consumers/C    body: {"since": N} -- make C, or move it
GET    /_changes/consumers      each consumer, where it is and how far behind
DELETE /_changes/consumers/C
GET    /_whoami                 who the server takes the token for, and the rules that apply to it
GET    /_schema                 the schema as a description (Schema in code); ?as=fenecql: as its statements
POST   /_schema/plan            body: a description -- what an apply would do; ?mode=follow: what the server lacks
POST   /_schema/apply           body: a description -- migrations, then what only adds; the server's token
POST   /mcp                     MCP for agents (--mcp): one JSON-RPC message a request
```

Details

With `--replication-token`, `/_replication` is taken
as well: the stream a replica is fed, its status, and a replica's promotion —
see [Replication](https://fenecdb.com/docs//docs/replication). It asks for that token
rather than `--http-token`, and a collection named
`_replication` is then out of HTTP's reach.

`/_schema` compares a schema declared in code with the
server's, in the engine every SDK's check goes through — see Schema in code. Its body is the
description, `{"format": 1, "fenecql": "create collection ...",
"migrations": [...]}` or its collections as JSON. A plan answers
`{"applied", "ran", "migrations", "statements", "refusals"}`, and
writes nothing; with `?mode=follow` it says only what the code
declares and the server lacks, which a scoped token may ask of the
collections it reads. An apply takes the server's token, runs the
migrations not yet recorded and the additions under the write lock as one
block, and answers 409 with the refusals when anything is refused, having
written nothing.

With `--mcp`, `/_mcp` is the Model Context
Protocol's endpoint: an agent's `schema`, `search`,
`query` and `get` tools, and `write` and a
memory where asked for, each a statement held to the request's token as
`/query` is — see [Agents and MCP](https://fenecdb.com/docs//docs/mcp).
Without the flag it answers 404.

`/_metrics` is the server's statements, latency and data in
Prometheus's text format, readable with `--http-token` or the
admin token — see [Monitoring](https://fenecdb.com/docs//docs/monitoring). The underscore
keeps it out of the way of a collection called `metrics`; a
collection called `_metrics` cannot be read over
`GET`.

## Filters

Filter in the query string as `field=operator.value`, for example
`?year=gte.2024`. For anything the pattern cannot say, use
`?where=` with FenecQL.

| **Operators** | `eq` `neq` `lt` `lte` `gt` `gte` `like` `has` `in` `is` |

| **Negation** | the `not.` prefix — `?summary=not.is.null`, `?year=not.eq.1999` |

| **Shorthand** | an unrecognised prefix makes the whole value an equality: `?year=2024` |

| **List** | `?year=in.(1999,2023)` |

| **Paths** | a path into a `json` field where a field goes: `?meta.lang=tr`, `?meta.source.rank=gte.2`, `?select=title,meta.source.site`, `?order=meta.source.rank.desc`; the value is read as JSON reads it — a number, `true`, `null`, a quoted string — and as the text it is otherwise. A body's object is a `json` field's value, every number in it as written, and `PATCH` with `{"meta.lang": "en"}` sets one key inside the field ([Objects and paths](https://fenecdb.com/docs/fenecql#json)) |

| **Clauses** | `select` `order` `limit` `offset` `count` `where` `lookup` |

| **Ordering** | `?order=year.desc,title.asc`; a collation among the modifiers puts text in its order — `?order=name.und.desc` is `order name collate und desc`, `name.tr` Turkish ([Collation](https://fenecdb.com/docs/fenecql#collation)) |

| **Free expression** | `?where=year >= 2024 and tags has "rust"` — full FenecQL |

Details

Values are parsed by the field's type: `?year=gte.2024` is an
int, `?published=gte.2024-01-01` is a timestamp. An unknown field
returns 404. `where=` exists for what the pattern cannot hold —
`or` groups and function calls.

## Vector search

Send the vector in a `POST` body to
`/<name>/near`, with an optional filter. Add
`"match"` to mix in keyword search.

```bash
curl -X POST http://127.0.0.1:8080/articles/near -d '{
  "vector": [0.1, 0.2, 0.3],
  "ef": 128, "limit": 5,
  "select": ["title"],
  "where": "year >= 2024"
}'
# [{"title":"the rust book","_score":0.97}]
```

- **`field`** can be left out when the collection has one
    vector field.

- **Query-string filters** are combined with the body's
    `where`.

Details

`near` is a `POST` rather than a query parameter
because a 768-dimension embedding does not fit in a query string. Squeezing it
into the URL — base64, truncation — is unreadable and runs into proxy and
server URL ceilings. PostgREST's model does not cover fenecdb's headline
feature, so `near` gets its own endpoint, and it takes a filter in
the body.

If the collection has a single vector field, `field` can be left
out. Filters in the query string are `and`ed with the
`where` in the body.

A vector in an answer, and a `_score`, is its `f32`s
written as the shortest decimals that read back as them — `0.1`
as it was written, the text pgvector gives — rather than the
`0.10000000149011612` of the `f64` each widens to. A
reader that takes a number as an `f64` and rounds it to
`f32`, as a `Float32Array` does, gets every one back:
the one `f32` an `f64` would round away,
`7.038531e-26`, is written as its `f64`.

A `"match"` makes it hybrid: BM25 over the collection's
`@text` field ranks the same request, and the two rankings are
fused by reciprocal rank ([fuse](https://fenecdb.com/docs/fenecql#hybrid)).
`match_field` names the text field when there are several;
`match_fuzzy` (`true`, or 0 to 2 edits) and
`match_prefix` read its words as `match ... fuzzy` and
`prefix` do; `fuse_k` and `candidates`
are the rank offset and how many candidates each side ranks.

```bash
curl -X POST http://127.0.0.1:8080/articles/near -d '{
  "vector": [0.1, 0.2, 0.3],
  "match": "borrow checker lifetimes",
  "limit": 10
}'
```

## Raw FenecQL

When the REST routes do not fit, send FenecQL to `/query`.
Parameters stay parameters.

```bash
curl -X POST http://127.0.0.1:8080/query \
  -H 'content-type: application/json' \
  -d '{"query":"get articles where year >= $1 limit $2","params":[2024, 10]}'
```

- **A text is parsed once.** `/query` keeps what it
    parsed by the text, so the values go as parameters — a page's
    `limit` and `offset` too (Pages by parameters): written into the
    text, every page was a statement of its own.

- **The answer is the rows**, a JSON array — or, when the query asks
    for [facets](https://fenecdb.com/docs//docs/fenecql#facets),
    `{"rows": [...], "facets": {"brand": [{"value": "acme", "count":
    12}, ...]}}`. A write answers `{"affected": n}`, and
    one with [`returning`](https://fenecdb.com/docs//docs/fenecql#pick) the
    rows it wrote, as a read does, with its `Fenec-Seq`: a job
    queue's claim ([Job queues](https://fenecdb.com/docs//docs/queues)).

- **From Python:** `pip install fenecdb` gives a client
    with no dependencies.

Details

The counts answer for the whole set the query matched, not for a row, so
they go beside the rows rather than into one, and a query that asks for none
keeps the bare array every client reads. The same holds for a line of a
`/batch` (`{"rows": [...], "facets": {...}}`), for
`GET /<name>?facet=...`, whose list is FenecQL's
(`facet=brand top 5,color`), and beside a count
(`{"count": 3, "facets": {...}}`). A subscription keeps a shape's rows and carries no
facets; a live query over the server (`live` on
`connect()`, `useLiveQuery`) runs its query again
after each write, facets and all. A `highlight()` is a column
like any other, under its label.

From Python, `pip install fenecdb` — the standard library
alone — makes the same calls, blocking or in an event loop:

```python
from fenecdb import AsyncClient, Client

db = Client("http://127.0.0.1:8080", token="...")
db.query("get articles where year >= $1 limit 10", [2024])

async with AsyncClient("http://127.0.0.1:8080", token="...") as db:
    await db.query("get articles where year >= $1 limit 10", [2024])
```

`AsyncClient` keeps one connection alive and opens it again
when the server has closed it; calls made at once over one client go one
after another over it, so requests side by side take a client each.
`Client.follow` reads the [change stream](#changes)
as a consumer, and `python -m fenecdb.relay` hands it on to
another program or a webhook: see [Services
and events](https://fenecdb.com/docs//docs/microservices).

## Embedding a collection

Use `lookup` to nest matching rows of another collection under
each row, such as each product's latest reviews.

```bash
curl 'http://127.0.0.1:8080/products?limit=20&select=name\
&lookup=reviews&reviews.on=product_id&reviews.stars=gte.4\
&reviews.order=created.desc&reviews.limit=3'
# [{"name":"Kahve","reviews":[{...},{...},{...}]}, ...]
```

- **Prefix the child's options** with its name:
    `reviews.on`, `reviews.limit`.

- **Chain levels** with a comma: `lookup=orders,lines`.

- **A key over several fields** lists a field of each side:
    `lines.on=org,order&lines.parent=org,id`, FenecQL's
    `on (org, order) = (org, id)`.

- **A level that folds** aggregates in its `select`, as
    the query's own does: `orders.select=count(*),sum(total)`,
    each parent's children folded, and `orders.group=status` a
    row a group, its `order` naming columns
    (`orders.order=count(*).desc`). Off a query's groups,
    `<name>.parent` names a column of its list:
    `/orders?select=courier,count(*)&group=courier&lookup=couriers&couriers.on=user&couriers.parent=courier`.

Details

`lookup=<name>` names the collection once; every key prefixed
with it configures the clause — `on` (required, the child's field
holding the key), `parent`, `select`,
`order`, `limit`, `offset`,
`required`, `where`, and any field name as a condition,
in the same `op.value` form the parent's filters use. Prefixing is
what keeps the two sides apart in a query string, the way the clause's
position does in FenecQL.

```bash
curl 'http://127.0.0.1:8080/shops?limit=20&select=name\
&lookup=orders,lines&orders.on=shop_id&orders.limit=3\
&lines.on=order_id&lines.limit=5'
# [{"name":"Merkez","orders":[{"code":"A","lines":[{...}]}, ...]}, ...]
```

A comma-separated list chains, read left to right, each name binding to the
one before it and each level keeping its own prefix. A query string has no
position to scope by, so the order is written out; the meaning is the same one
FenecQL gets from position, up to the same 8 levels.

`lookup` is a reserved key here, so a field named
`lookup` cannot be filtered over the REST path — the FenecQL and
`fenec-server` paths are unaffected, as with the other reserved
names.

A query with a `lookup` answers with the children nested under
the collection's name — one object per parent, with an empty array where
nothing matched, and a chain nests all the way down.

```bash
curl -X POST http://127.0.0.1:8080/query \
  -H 'content-type: application/json' \
  -d '{"query":"get products limit 20 lookup reviews on product_id limit 3"}'
# [{"id":1,"name":"Kahve","reviews":[{"id":1,"stars":5,"body":"guzel"}]}, ...]
```

## Batches

`POST /batch` runs several statements, one per line, as a single
all-or-nothing write. If one fails, the ones before it are undone and the
answer says so.

```json
{"error":"collection `tasks` has no field `nofield`","completed":0,"at":1}
```

A statement that matches no row is no error, and the batch goes on. To
make a count a condition, end the write with `require N`: unless
it wrote exactly `N` rows the batch stops with
**412** and is put back, a transfer's debit and its credit together
([Transfers](https://fenecdb.com/docs//docs/fenecql#transfers)).

```json
{"error":"unmet: `set accounts` wrote 0 rows, and requires 1","completed":0,"at":1}
```

Every client sends one: `db.batch([...], { idempotencyKey })` in
JavaScript, `batch(...)` in Python, `Batch` in Go,
`BatchAsync` in .NET, and the remote clients of Swift, Kotlin and
Dart; a batch that stops throws the client's error with the statement's
`at` and the status.

Details

`POST /batch` takes one query body per line as NDJSON. The
statements run in order under a single write lock, as one block: their writes
land in the file as one record, or none of them do. The first error puts back
what the statements before it did, and the answer says so: `at`
is the statement that stopped it, counted from 0, and `completed`
how many stay applied — none of a block.

A batch whose every statement only reads — a `get`, with or
without `require` — is a snapshot instead: every read at the one
change `Fenec-Seq` names, no write landing between the first and
the last. When it is long — an aggregate, a scan, a filtered count over
10 000 rows or more — the server pins what it reads under the read lock and
reads it with no lock held, so writes land beside it; otherwise, and where
an index answers a statement, it runs under one read lock, and writes wait
for it as they wait for any read. A reconciliation over a million journal
entries took 140 to 200 ms, and a transfer arriving meanwhile waited up to
187–512 ms; pinned, 5.4–29 ms (`make recon-bench`). A long
`POST /query` read is pinned the same way.

No other writer slips in between, a read in the batch sees the writes
before it, and a subscriber is woken once, for all of them. A
`create`, a `drop` or a `create index` in it
is put back with the rest. A `compact` rewrites the file, which
cannot be put back, so a batch holding one runs each statement on its own:
it stops at the first error, and
`completed` says how many were applied. Either way a failed batch
says what it left behind, because a silent error would
permanently separate a client's optimistic local state from the
server's.

The body is NDJSON rather than a JSON array because fenecdb's JSON parser
deliberately does not accept nested objects, so
`{"statements":[{...}]}` could not be parsed anyway. Instead of
writing a second parser, the boundary was put at the end of a line — by the
escaping rules no JSON encoder can write a bare newline into the body.

## Retrying a write

Send an `Idempotency-Key` header with a write, and retrying it
after a timeout cannot write it twice: a repeat gets the first answer back.

```bash
curl -X POST http://127.0.0.1:8080/query \
  -H 'Idempotency-Key: 5f0c2d6e-order-1142' \
  -d '{"query":"put orders {customer: $1, total: $2}","params":[7, 129.5]}'
```

- **One key per write.** The same key with a different request is
    refused (422).

- **A failed write keeps no key**, so a retry runs it again.

Details

A `put` with no id makes a row each time it runs, so a client
that sends a write again after a timeout or a dropped connection — not
knowing whether the first landed — may write it twice. Send an
`Idempotency-Key` with it, one per write, and a request sent again
with the key is answered as the first was, with
`Idempotent-Replayed: true`, and not made again. Its
`Fenec-Seq` is where the database stands as it replays, which
holds the write, so a read sent to a replica with `Fenec-After`
waits for it as after the first answer.

- The key and the answer are kept in the same block as the write, under
  the write lock: they land together, so a crash cannot leave a write whose
  retry makes it again, and a second request with the key waits for the
  first and is handed its answer.

- A write that failed keeps no key: sent again, it runs again.

- A key is for one request: the same key with another method, path or
  body is refused (422).

- Under a scoped token the keys are its subject's (`sub`), so
  no user is handed another's answer; a token with no subject is refused a
  key (400).

- REST writes, `/query` and `/batch` take one, a
  `compact` — which cannot be put back — does not; a read ignores
  it. An answer over 64 KB is kept as its status and a note.

- Keys are kept for `--idempotency-ttl` seconds, a day unless
  given, in a `_idempotency` collection the change stream leaves
  out. Its `at` is [`@ttl`](https://fenecdb.com/docs//docs/fenecql#ttl):
  a key past its time is gone at once, and the sweeper deletes it.

A write without a key costs what it did; with one, 16 200 single puts a
second over HTTP against 18 300 without.

## Holding a write until it can write

Send `Fenec-Wait: <ms>` with a `set` or a
`del` on `/query` or `/batch`, and while it
would write nothing the server holds it, answering the moment it writes —
or at the end of the wait, 30 seconds at most, as it would have answered
at once. A job queue's claim waits for a job that way (Waiting for a job).

```bash
curl -X POST http://127.0.0.1:8080/query -H 'Fenec-Wait: 30000' \
  -d '{"query":"del tasks where run_at <= now() order run_at limit 1 returning *"}'
```

- **What wakes it:** a write to a collection it reads, and the time
    reaching a row's moment — a field compared with `now()`, a
    `@ttl`. A filter that reads the time any other way is
    refused (400), as is a request holding a `put`.

- **Wrote nothing** means no row written, or a
    `require` not met; any other refusal is answered at
    once.

- **With `Fenec-After`** the header is that wait's, as
    below, and holds no write.

## Marking a read

Send `Fenec-Read: replica` with a request that only reads, and
a router standing in a region where the tenant has a read replica answers it
there rather than from the tenant's home (Read replicas); with
`Fenec-After` and `Fenec-History`, as an answer from home
named them, it is answered from nothing older than that change of that
history ([Reading your own
writes](https://fenecdb.com/docs/replication#read-your-writes)), a subscription and a `/_changes` page as much as
a read: each waits for it before its head. Every server runs a marked
request read-only on every route —
home, a replica, a single file; a statement, REST, `/batch`, an
MCP tool, a change-stream consumer's commit — so a write sent with the mark
is refused with `403` and words that say why, never landed; a
value but `replica` is a `400`. A copy that has not
caught up with its primary yet answers it `503`.

```bash
curl -X POST https://ap.example.com/t/acme/query -H 'Fenec-Read: replica' \
  -H 'Fenec-After: 1043' -H 'Fenec-History: 2-6c1f0a9e44d7b2c8' \
  -d '{"query":"get orders where customer = $1","params":["c-7"]}'
```

## The data ceiling

With `--max-memory`, writes that would grow the data past the
limit are refused with `507`. Reads, deletes and
`compact` still work.

Details

Inside `fenec-server` started with `--max-memory`, the
listener holds its writes to that ceiling: above
it, a statement that grows the data — a put, a patch, a `create
index` — is refused with `507`, a batch stops there as at an
error, and reads, deletes and `compact` go on, since they are the
way out. The rule and why it is there are under
[Server](https://fenecdb.com/docs/server#limits).

## Refusals and their statuses

A refusal is a status and `{"error": "..."}`. The status says
what to do about it; a `/batch` that stopped adds where
(`at`) and what stayed applied (`completed`), a
tenant's quota names itself (`quota`, and the
`Fenec-Quota` header), and one to come back from says when
(`retry_after`, the seconds of its `Retry-After`
header: a page on another origin, or an app's replica, reads the body; a
quota's 429 says it to the millisecond as well, `retry_after_ms`,
which is the one to wait). A write whose outcome the server cannot know
says so (`"outcome": "unknown"`, a 500), with where it left the
database in the body (`seq`, `history`) rather than in
`Fenec-Seq`, which a client takes as its own position. Every
SDK's error carries the rest; the outcome and its position are the Go,
.NET, JavaScript and Python clients' (Go's and .NET's code
`outcome_unknown`, JavaScript's and Python's
`outcome`).

| Status | Means | What to do |

| 400 | a statement that does not parse or run, a bad parameter | fix the request |

| 401 | no token, or a wrong one | send the right token |

| 403 | outside the token's rules — a row, a field, a change no transition names, a `returning` row it may not read, the write put back whole — a write to a replica or a read-only server, or one marked `Fenec-Read` | write where it is taken |

| 404 | a collection, field or tenant that is not there |  |

| 409 | an id or a `@unique` value taken, a collection that exists | a lost race for a value |

| 410 | a `/_changes` cursor the server no longer reaches | start again from the `since` it names |

| 412 | a `require` not met, its batch put back | read again, decide again |

| 413 | a body past the ceiling, 64 MB | send less at once |

| 422 | an `Idempotency-Key` sent with another request | a new key for a new request |

| 429 | a tenant's quota of `streams`, `held`, `requests`, `writes` or `write_bytes` | come back after `retry_after_ms` |

| 500 | outcome unknown (`"outcome": "unknown"`): a write a tenant's node landed and answered after its lease from the router lapsed — its tenant may have been failed over without it, or still be there with it; a `/batch` that ran a statement at a time says how far it got, its `seq` the last landed statement's | send it again with the same `Idempotency-Key`; without one, wait a lease and a tenth (`--auto-failover` × 1.1) after the answer — until the tenant settles the node answers reads with the write in them, one sent with `Fenec-After` too — then read it back, or send a read with `Fenec-After` and `Fenec-History` set to its `seq` and `history`, which a primary without it answers 504 (for a batch, a 504 does not say which earlier statements survived: read those back) |

| 500 | an fsync that failed under `--sync always`: the write may or may not be on the disk, and the server takes no write until its file is opened again | restart the server, then read the write back, or send it again with the same `Idempotency-Key` |

| 503 | too many connections or subscriptions; a tenant being moved or failed over; a node going down | come back after `Retry-After` seconds |

| 504 | a `Fenec-After` change, sent with a read, a subscription or a `/_changes` page, that did not reach a replica in the wait, or one a primary's history does not hold — lost in a failover; `seq` and `history` say where it stands | ask again, or ask the primary; take the primary's position as yours |

| 507 | past `--max-memory`, or a tenant's quota of `bytes` or `rows`; the write put back whole | delete, then write again: waiting does not help |

```bash
HTTP/1.1 429 Too Many Requests
Fenec-Quota: writes
Retry-After: 1

{"error":"the tenant writes past its quota of 100 rows a second","quota":"writes","retry_after":1,"retry_after_ms":10}

HTTP/1.1 503 Service Unavailable
Retry-After: 1

{"error":"the tenant is being moved; retry shortly","retry_after":1}

HTTP/1.1 500 Internal Server Error

{"error":"the write landed on this node, but by the time it was answered ...","outcome":"unknown","seq":4182,"history":"2-6d1f0c2a9b3e4471"}
```

A quota is a tenant node's (`fenec-server --dir`): what each
bounds, how it is set and why the statuses are these is under
[Tenants and sharding](https://fenecdb.com/docs/sharding#quotas). 507 is the storage
the request needs and the tenant has not got — the request is fine, and a
smaller one may fit, so not 413, which is about the body; 429 is a rate or
a share the tenant comes back within.

## Subscriptions

`GET /<name>/changes` is a server-sent events stream: first
the rows matching your filter, then each change to them as it happens.

```bash
curl -N 'http://127.0.0.1:8080/articles/changes?year=gte.2024'
# event: seed
# data: {"seq":12,"rows":[...]}
```

- **Limited separately** from ordinary requests:
    `--http-max-streams`, 64 by default, a database's -- on a
    tenant node each tenant's own -- and each one of
    `--max-connections` too, which a node's tenants share
    ([a tenant's share](https://fenecdb.com/docs/sharding#quotas)).

- **A row reaching its `@ttl`** is told as a deletion at
    its time, as a write would be, not when the sweep deletes it a minute
    and its grace later.

- **Sent after a write** (`Fenec-After`, and
    `Fenec-History` as the write's answer named it), a stream
    waits for it before its head, as a read does: its seed holds the write,
    or it is answered `504` with where the server stands (Reading your own
    writes). One resumed with `since` waits the same way:
    `since` is where the stream left off, not a write it must
    see.

- **Its head says where the server stood** as it opened:
    `Fenec-Seq`, the change its seed is at or after, and
    `Fenec-History`, the history that change and every event's
    `seq` are of, as a write's answer names it (none on a
    database that never had a replica). A read sent after an event's
    `seq` to another copy goes with both (Reading your own
    writes): a number alone names no write across a failover.
    `FenecHttp.live` on a client reading near sends its runs
    so.

- **In the browser,** [Sync](https://fenecdb.com/docs/sync) builds on this
    stream.

Details

`GET /<name>/changes` is a server-sent events stream: a seed
carrying the current shape, then incremental diffs. No driver needed.

A subscription does not poll. `fenec-core` has no waiting
primitive at all — there are no threads in wasm — it only reports that the
change counter has reached a point, and `fenec-http` ties that to a
`Condvar`. A 50 ms polling loop would take 20 read locks per second
per subscriber and still add 50 ms of latency.

Every subscription is a connection and a thread, so they are counted
separately from ordinary requests as well (`--http-max-streams`,
default 64, a database's: on a tenant node each tenant's). Sharing one
ceiling would let 100 subscribers shut the server to plain requests. A
stream is also one of the node's `--max-connections`, the bound
every tenant of a node shares, of which each tenant's streams take a quarter
at most unless its quota says otherwise. The client-side layer built on this
stream is [Sync](https://fenecdb.com/docs/sync).

A row past its `@ttl` leaves every read at once and the file at
the sweep, a minute and its grace later; a stream of its collection is told
of it at its time. It works out when the next row of the collection reaches
its time -- a step of the field's ordered index, again after each write to
the collection -- sleeps until then if no write comes first, and reads the
rows that reached it since it last looked, the shape ANDed in, telling each
as a deletion as the sweep's delete would be told: a scoped stream of the
rows it was sent alone, a shape that holds nothing (what
`FenecHttp.live` opens) that something changed. Heard at the
sweep, a courier whose phone went quiet stayed on a dispatcher's map 73 s
after it had gone from every read; at its time, it is told about 10 ms after
it. The sweep's delete comes after, a deletion of what is gone already.

What a stream costs, measured on an M1 with streams of a shape that holds
nothing (what `FenecHttp.live` opens): 45 to 57 KB of resident
memory and a thread each, and every write wakes every stream, which reads
what it changed: a write's p50 went from 0.26 ms with none to 0.94 with 60
and 3.1 with 500. A page thousands view should not hold one a viewer; it
polls (below).

Under a scoped token, a stream of a shape that holds nothing
(`where=false`) is told of the writes to rows the token may
read, and of nothing else: the server follows the token's own filter, keeps
the ids it selects and sends none, and a write to one of them, or one of
them leaving the token's rows, is a `change` holding no rows.
ANDed with the token's filter, the shape matched nothing and
`FenecHttp.live` never ran again; told of every write, the
stream would hand the token the moments of other users' writes.

## Polling a read

Send `If-None-Match` with a `/query` read and the
answer carries an `ETag`: the change the database stood at. Sent
back, the tag is answered `304`, with no body and the query not
run, while no write has touched a collection the read reads.

```bash
curl -si http://127.0.0.1:8080/query -H 'If-None-Match: "0"' \
  -d '{"query":"get inventory select available where sku = $1","params":["TENT-1"]}'
# HTTP/1.1 200 OK
# ETag: "48211"
curl -si http://127.0.0.1:8080/query -H 'If-None-Match: "48211"' -d '...the same...'
# HTTP/1.1 304 Not Modified
```

- **Nothing is held between polls:** no stream, no thread, no wake-up
    at a write. `FenecHttp.live(query, cb, { poll: 5000 })` polls
    so.

- **Untagged**, and run each time: a read of a collection whose rows
    expire (`@ttl`), one calling `now()`, one holding an
    inner `get` — answers that move with no write.

- **A scoped token's tag is its answer's** (`"c…"`, a
    digest of the rows): its query runs every time and is answered 304 when
    the rows are the ones it had. Tagged by the change counter, 304 against
    200 -- and a 304's speed -- told it when anyone wrote to what it reads,
    other users' rows among them, and the tag how many writes the database
    had taken.

Details

Whether a collection was written since a tag is the change ring's answer
(`Database::changed_collections_since`): a tag older than the ring
reaches, or a collection dropped since, counts as a change and the query runs.
A read without the header is answered as before, no tag worked out. Over one
keep-alive connection a 304 took 54 us at the median, against 55 for a read
by key and 95 for a page of 24 run; the round trip is most of either. A
browser keeps its connection between polls, so a public page polls through
the site's own server or a proxy, which keeps few connections to
fenec-server however many viewers it has.

A tag is a change of the copy that answered. A read replica holding that
change of its history (`Fenec-After`) judges it as home does, but
only up to where it stands: behind home it answers 304 to a write home holds,
so `FenecHttp.live` polls home on a client reading near (Read replicas).

## Change data capture

To see every write exactly as it happened — to feed a search index, Kafka
or a read model — start the server with `--cdc` and read
`GET /_changes`: one JSON line per write.

```bash
fenec-server --file data.fenec --http 127.0.0.1:8080 --sync 250 --cdc

curl -i 'http://127.0.0.1:8080/_changes?since=0&limit=1000&wait=10000'
# Content-Type: application/x-ndjson
# Fenec-Next: 5
# Fenec-Seq: 5
#
# {"seq":1,"at":1790784252138,"collection":"a","op":"create"}
# {"seq":2,"at":1790784252158,"collection":"a","op":"put","id":1,"doc":{"id":1,"n":1,"t":"x"}}
# {"seq":3,"at":1790784252158,"collection":"a","op":"put","id":2,"doc":{"id":2,"n":2,"t":"y"}}
# {"seq":4,"at":1790784252173,"collection":"a","op":"put","id":1,"doc":{"id":1,"n":1,"t":"z"}}
# {"seq":5,"at":1790784252187,"collection":"a","op":"del","id":2}
```

- **Send `Fenec-Next` back as the next `since`**
    and nothing is missed or seen twice.

- **Only writes already on disk** are handed out.

- **`Fenec-Seq` is the last write the database holds:**
    when `Fenec-Next` has reached it, you have every write.

- **Sent with `Fenec-After`**, a page waits until the
    server holds that change, as a read does, and its
    `Fenec-Seq` is never behind it; a write not yet on the
    server's disk comes on a later page.

Details

A subscription keeps a query's rows up to date, and a client that falls
behind its ring of ids is sent the rows again. A service that must see every
write — to index it elsewhere, send it on to Kafka, or keep a read model —
reads `GET /_changes` instead: the writes on disk, documents and
all, one JSON object a line, from any write the server still keeps.

- `seq` is the change counter's number for the write, and
  `Fenec-Next` the last one in the answer: sent back as the next
  `since`, nothing is missed or had twice. Keep it where your
  consumer keeps its own state, and commit the two together.

- `Fenec-Seq` on every answer is the last write the database
  holds, on disk or not yet, the writes the stream leaves out (a consumer's
  cursor, an idempotency key) among them. An answer whose
  `Fenec-Next` is that has every write there is: a worker that
  catches up before it serves knows it at once, where an empty page alone
  could not tell it from one whose writes are not on disk yet under
  `--sync 250`, and waited out pages to be sure. A
  `Fenec-Next` behind it has writes to come: read again with
  `wait`. A client that notes `Fenec-Seq` for
  `Fenec-After` reads a replica from there on.

- A batch's writes are one record on disk and come together,
  numbered one by one; a cursor inside one goes on after the write it
  names.

- Only what an fsync has covered is handed over, as a replica is fed:
  a write the server lost in a crash never reached a consumer.

- `since` left out is the last write on disk: what is written
  from now on. `limit` is 1 000 unless given, 10 000 at most;
  `wait` is how long an answer with nothing in it waits for a
  write first, in milliseconds, 30 000 at most.

- The server keeps as many writes as `--replication-buffer`
  holds (64 MiB unless given). A cursor further back is answered 410 with
  the first `since` it does serve, never with writes missing
  from the middle; one past the last write — another database's cursor, or
  this one restored — 409. The feed starts at the change counter when the
  server starts.

- `op` is `put`, `del`,
  `create`, `alter` or `drop`; a put
  carries the document as it was written, as JSON writes a row.

- It needs `--http-token` or no token at all: a scoped
  token's filter could not hold back the deletion of a row it never saw,
  and is refused (403).

`--cdc` keeps the feed a primary keeps for its replicas and
feeds none; with `--replication-token` it is kept already, and a
tenant node's tenants have theirs at `/t/<tenant>/_changes`.
It needs `--file` and a `--sync` other than
`off`. Keeping it cost nothing measurable: 16 400 single puts a
second over HTTP either way, and 9 000 rows of 128 dimensions read back at
283 000 a second.

## Consumers

A consumer with no storage of its own can let the server remember its
place, like a Kafka consumer group: create it, read from it, then commit how
far it got.

```bash
curl -X POST http://127.0.0.1:8080/_changes/consumers/search-index
curl -i 'http://127.0.0.1:8080/_changes?consumer=search-index&wait=10000'
# Fenec-Next: 42 -- index the lines, then:
curl -X POST http://127.0.0.1:8080/_changes/consumers/search-index -d '{"since": 42}'
```

- **At least once:** a crash between reading and committing means
    those writes come again.

Details

A consumer that keeps no state of its own can have the server keep where
it is, as a Kafka consumer group keeps its committed offset. Made with a
`POST` — at the last write on disk unless the body names a
`since` — it reads with `?consumer=` from where it
stands, and moves once it has done with what it read.

So a consumer has each write at least once, and a write again after a
crash between reading it and committing. The cursors are rows of a
`_consumers` collection, written as any write is — on disk
before the answer under `--sync always`, and on the replicas —
and their own writes are left out of the stream, the cursor going past
them. A read with `wait` waits on past them too, so a consumer's
own commit does not wake its next read, and a commit past nothing but
cursors writes nothing: a consumer that commits after every answer, an
empty one too, sits idle while nothing else is written. An unknown consumer is 404 rather than read from "now" each time,
which would miss what is written between two reads; a commit past the last
write on disk is 409. `GET /_changes/consumers` lists each
consumer's `since` and how many writes it is
`behind`.

## Security

Protect the endpoint with a bearer token, and put a TLS terminator in front
of it on an open network.

```bash
fenec-server --file data.fenec --http 127.0.0.1:8080 \
         --http-cors 'http://localhost:8787' \
         --http-token "$TOKEN" \
         --http-read-only
```

| `--http-token <value>` | every request needs `Authorization: Bearer <value>`, compared in constant time |

| `--http-cors <origin>` | sets `Access-Control-Allow-Origin`; without it no CORS header is sent at all. A browser’s preflight (`OPTIONS`) from that origin, or any with `*`, is answered 204 before a token is asked for -- the browser sends none with it -- on a single database and under `/t/<tenant>/`. Every answer exposes `Fenec-Seq`, `X-Request-Id` and `Retry-After` and lets `Idempotency-Key` in. Through `fenec-shard`, give the router the flag: it answers the preflight itself and puts its headers on its own answers and in place of the node's ([Tenants and sharding](https://fenecdb.com/docs//docs/sharding#cors)) |

| `--http-read-only` | write endpoints return 403; `fenec-server`'s own path is unaffected |

| a replica | every write returns 403, whatever the flags: its writes come from its primary |

| non-loopback address | refuses to bind without a token, unless `--insecure` is given |

Details

There is no TLS, the same rule as `fenec-server`: on an open network
it needs a TLS terminator in front. A `PATCH` or `DELETE`
without a filter is refused, and covering a whole collection needs a separate
**path** rather than a flag — a key like `?all=true` would
collide with a field named `all`.

Limits are shared with `fenec-server` (`--max-connections`,
`--idle-timeout`: a connection silent that long between requests is
closed with nothing written, and one that went silent part way through a
request is answered 408); the body ceiling is 64 MiB and the header block
64 KiB. `Transfer-Encoding: chunked` is unsupported and refused with
411, which beats reading half a body. A field whose name matches a clause name
(`select`, `order`, `limit`,
`offset`, `count`, `where`,
`group`) cannot be filtered
over HTTP; the FenecQL and `fenec-server` paths are unaffected.

**A shape alone is not a security boundary.** Under the server's own
  token, which ids changed is visible to a subscriber even when the rows are
  not. Rows are hidden by a scoped token, below.

## Scoped access

Give each user a JSON Web Token, and a policy file decides which
collections and which rows that token can read or write. A browser can then
talk to the database directly.

```text
# collection   access         rows                                role
notes          read,write     where owner = $jwt.sub
posts          read           where published = true or author = $jwt.sub
posts          write          where author = $jwt.sub
messages       read,insert    where room in $jwt.rooms
journal        read,insert    where owner = $jwt.sub
journal        append-only
accounts       update(balance, held)                              for app
orders         update(status, run_at)  where customer = $jwt.sub and status = "ready" and new.status = "cancelled"
orders         read(pickup, run_at)    where run_at <= now()                    for courier
positions      read           where courier in (get orders select courier where customer = $jwt.sub)
*              read                                               for dashboard
```

- **Reads** only see the rows the rule allows.

- **Fields.** `read(pickup, run_at)` shows the rows it
    admits with those fields alone, and `id`: a row shows the
    fields of every read rule admitting it, the rest `null`. A
    courier sees a ready order's pickup, and the customer's door once the
    order is theirs.

- **Writes** must stay inside the rule, or get 403. `write`
    grants `insert`, `update` and `delete`,
    and each can be granted alone; `update(balance, held)` grants
    an update of those fields and no other.

- **Transitions.** `new.status` reads a row as the write
    leaves it, and a bare name as it stands: the `orders` rule
    above lets a customer cancel a ready order, and not take a cancelled
    one back.

- **`returning`** answers rows the token may read as the
    write leaves them: a courier who reads the orders they carry claims a
    ready one and is answered it, claimed.

- **`append-only`** collections take inserts from scoped
    tokens, and refuse every update and delete with 403.

- **A list claim**, such as `"rooms": ["r1", "r2"]`,
    goes after `in`.

- **Another collection.** `in (get …)` reads a rule's
    list from rows: the `positions` rule above lets a customer
    read where the courier carrying their order is, and nothing of the
    others.

- **The server's own `--http-token`** still has full
    access.

A whole product built on these rules is
[examples/saas](https://github.com/fenecdb/fenec/tree/main/examples/saas):
sign-up and sessions owned by the app, a tenant per organisation behind
`fenec-shard`, roles and teams as grants and list claims, live
boards and search under each person's token, with its threat model and the
attacks its tests try.

Details

`--http-token` is everything or nothing. With
`--jwt-secret` the endpoint also takes HS256 JSON Web Tokens, each
held to the rules of a `--policy` file, collection by collection and
down to the rows, so a browser can talk to the database directly with a token
per user:

```bash
fenec-server --file data.fenec --http 0.0.0.0:8080 --http-token "$ROOT" \
         --jwt-secret-file jwt.secret --policy policy.txt

fenec-server --jwt-secret-file jwt.secret --mint-token '{"sub":"alice","exp":1893456000}'
```

- **Reads** get the rule's filter ANDed in at every level: the
    collection's `where`, each `lookup` below it, a
    subscription's shape, a `count`. A collection no rule grants
    a token anything on does not exist for it: 404, and absent from the
    list.

- **Writes** get it twice, as PostgreSQL's `USING` and
    `WITH CHECK`: a `set` or `del` touches only
    the rows the filter matches, and every document written — a put, or a row
    a `set` rewrote — has to match it afterwards, or the write is
    refused with 403. A put that leaves out a field the filter pins to a claim
    gets the claim's value, so `{"title":"a"}` is alice's. A scoped
    put names no `id`, which could overwrite a row the token cannot
    see, and a scoped token changes no schema and runs no maintenance.

- **`returning`** answers what the write wrote, so the
    rows it answers are held to the read rules: a `set`'s as it
    wrote them, a `del`'s as they were. A row the read rules
    leave out by a field the write does not change is passed over, as a
    `get` passes it over; a `set` that would leave a
    row the token may not read is refused with 403 and writes nothing.
    PostgreSQL asks its `SELECT` policies of the row before an
    `UPDATE … RETURNING` as well as after, which is why a courier
    who reads the orders they carry, claiming a ready order, would take
    nothing there and be told nothing; here the terms over the fields the
    `set` writes are asked of the row it leaves, and the claim
    answers the order, claimed. Without `returning` a write's
    rows are those its own rules reach, read rules or not.

- **Grants** are `read`, `insert`,
    `update` and `delete`, joined by commas, and
    `write` is the three writes. An insert is a put of new
    documents, held to the filters of the rules granting
    `insert`; an update is a `set` and a delete a
    `del`, of rows the token may read and the filters of the
    rules granting them, a `set`'s rows checked again after.
    `update(balance, held, status)` names the fields an update
    may change: a write over a row — a `set`, a REST
    `PATCH` or `PUT`, a line of a `/batch` —
    is judged by the fields it changes, and each of them must be one rule's
    whose filter admits the row before and after, or it is refused with 403
    naming a field (a batch holding it is put back whole). A field written
    with the value it holds is not changed. Rules widen each other, each
    with its fields and its rows: `notes update where owner =
    $jwt.sub` beside `notes update(title)` changes every
    field of the token's notes and the title of anyone's. A token may insert into a collection it cannot read, such as an event
    stream: its reads of it are refused with 403 and it is not listed.
    `expired` grants the rows past their `@ttl`, which
    `expired()` reads ([the
    reaper](https://fenecdb.com/docs//docs/fenecql#reaper)): a statement asking it is held to the filters of the rules
    granting `expired` beside the ones its operation needs, and
    neither `write` nor `*` grants it — a lapsed hold
    is the reaper's, not a user's to see again.

- **Fields a read shows.** `read(pickup,
    run_at)` grants the rows its filter admits with those fields and
    `id`; `read` grants every field. A row shows the
    fields of every read rule admitting it, and the rest answer
    `null` — in `/query`, `/batch`, REST,
    each `lookup` level by its own collection's rules, a write's
    `returning`, a subscription's seed and changes, and the MCP
    tools. Beside `orders read where courier = $jwt.sub for
    courier`, the rule above shows a courier every ready order's
    pickup and the whole of the orders they carry. A field no read rule
    grants is not there: `select *` leaves it out,
    `describe`, `/collections` and
    `/_schema` do not list it, and naming it is refused (403)
    as a name the collection lacks is.

```text
get orders where dropoff = $1                 -- no ready order matches, whatever its door
get orders where not (dropoff = $1)           -- every ready order does: no door told from another
get orders order dropoff                      -- 403: the order would tell the doors
get orders where courier = $1 order dropoff   -- $1 their sub: every order reached shows it
```

    A field withheld from a row must not show through what a statement does
    with it either. A `where` term reading it holds only where
    the row shows it — `dropoff = $1` is asked as
    `(courier = $jwt.sub and dropoff = $1)`, the `or`
    of the filters of the rules granting `dropoff` — in a
    `get`, a `set` or `del`, a
    `lookup` level, a facet's and an inner `get`'s;
    `and` stops at the first false, so the term is never worked
    out over a value it may not see. `order`,
    `group`, `having`, a `facet`, an
    aggregate, `match`, `near`, a highlight, a
    `lookup` key and a `set`'s values read a field
    only where every row they reach shows it, and are refused with 403
    naming it otherwise: when every read rule of the collection grants it,
    or when one rule granting it has no filter or one whose
    `and` terms are each a term of the statement's own
    `where`, parameters bound — as the last line above. A
    computed column over such a field is `null` where the row
    does not show it and never worked out there, and an inner
    `get`'s column of one is listed from the rows showing it. A
    write reaches the rows it reached before; its `returning` is
    masked as a read's rows are. PostgreSQL's column privileges are a
    role's and the same on every row, and refuse a statement naming a
    column the role lacks, `SELECT *` included; a column shown
    on some rows is a view of `CASE WHEN … THEN col END`
    there, whose `WHERE` reads `NULL` on the rest
    where this reads the term as false. A token whose rules name no fields
    reads as before; one whose do loses the stored-bytes path of a plain
    `get` where a column is masked, and pays each rule that could
    show a row more, per row: over HTTP on loopback a row by id took 28.0 µs
    against 26.0 under the same two rules naming no fields, a page of
    twenty 39.5 against 34.4, about 0.13 µs a row of it in process
    (`make policy-bench`).

- **Transitions.** A rule's filter reads the row as it stands, and
    `new.<field>` reads it as the write leaves it, as
    `new.` names the document in an upsert's `else
    set`. One filter asked of the row before and after cannot say
    which way a row may go: `status = "ready" or status =
    "cancelled"` lets a customer cancel an order and take a cancelled
    one back, and `run_at` granted lets them move it to the head
    of the queue. A rule naming `new.` is a transition, asked of
    the two rows together:

```text
# a customer cancels a ready order, out of the queue for good
orders  update(status, run_at)  where customer = $jwt.sub and courier is null and status = "ready" and new.status = "cancelled" and new.run_at is null  for customer
# a courier claims an order whose lease is out, for themselves, counting the attempt
orders  update(status, run_at, courier, claim, attempts)  where run_at <= now() and new.status = "assigned" and new.courier = $jwt.sub and new.attempts = attempts + 1  for courier
# and delivers only what they carry
orders  update(status, delivered_at, run_at, claim)  where courier = $jwt.sub and (status = "assigned" or status = "picked") and new.status = "delivered" and new.run_at is null  for courier
```

    A write over a row is the token's when one rule granting
    `update` names every field it changes (or names none) and
    admits the change: a transition by being true of the row before and
    after together, any other rule by admitting each. Rules naming no
    fields and no `new.` widen each other as before, the row
    before one rule's and the row after another's, as PostgreSQL's
    permissive policies do; a transition lends neither half. A
    `set` picks the rows a transition reaches by what the row
    before can answer of it — `status = "ready"` of the first
    rule above — and each row it writes is then asked the whole rule, a
    refusal 403 and the statement (or the batch) put back. The row after
    is held to what the rule says of `new.` and nothing more:
    the field list keeps the rest where it was, so a transition with none
    should pin what must not move (`new.customer = customer`).
    A rule naming `new.` grants `update` alone — a
    read, an insert or a delete has no row after, or none before — and
    one granting more stops the server at startup. A scoped
    `set` of one row over HTTP on loopback took 5.0 µs more
    than the server token's 45 µs under a plain rule, 5.7–5.9 under a
    field list and 6.1–6.8 under a transition
    ([`make
    policy-bench`](https://github.com/fenecdb/fenec/tree/main/crates/fenec-http/examples/policy.rs)).

- **`append-only`** is a line of its own,
    `journal append-only`, with no filter and no role. No scoped
    token updates or deletes there, by any route — `/query`,
    REST, `/batch` — whatever its rules grant: `write`
    and `*` mean `insert` there, and a rule naming
    `update` or `delete` for the collection stops the
    server at startup. It binds JSON Web Tokens and not the server's own
    token: corrections, erasure and `@ttl`'s sweep stay with
    whoever holds that, and an app server that should only append is given
    a scoped token of its own, a role for instance.

- **List claims.** A claim that is a JSON array goes after
    `in`: `room in $jwt.rooms` matches the rooms it
    lists, `room in [$jwt.rooms, "lobby"]` those and one more,
    and a text claim there is a list of one. It holds reads, writes
    (`WITH CHECK`: a post into a room the list does not hold is
    403) and subscriptions alike. A token whose list claim holds more than
    1 000 values is refused (401). A `role` claim may be a list
    too, and a rule `for` a role applies when the list holds
    it. Membership kept in another collection is a rule's `in (get
    …)` (below).

- **Rules that read another collection.** A rule's filter takes
    `in (get …)`, its claims bound inside as anywhere else:

```text
# a customer follows the courier carrying their order
positions  read  where courier in (get orders select courier where customer = $jwt.sub and (status = "assigned" or status = "picked"))  for customer
# a member reads and posts in the rooms they belong to
messages   read,insert  where room in (get members select room where user = $jwt.sub)
```

    The inner `get` reads the rows of its collection the
    token's own `get` would, held to the token's rules there;
    one no rule lets the token read gives none of its rows, as
    PostgreSQL's row security gives a policy's subquery none of a table no
    policy grants. The fields it reads are the policy's own: its list is
    never shown, only what it admits. It is answered before each
    statement, under the lock that runs it, as the `in` list it
    is: reads, a `lookup` level, a facet, a `match`
    and an agent's MCP tools see the list as it stands, and a write's
    `WITH CHECK` — an insert's document, an update's row before
    and after, a transition's pair, an upsert's row written over,
    `returning`'s rows — is tested by it where no query could
    run. A rule naming the fields it shows may read another collection
    too, `positions read(courier, at, loc) where courier in (get
    orders …)`: each row is shown the fields of the rules admitting
    it as their lists stand, in a read, a `lookup` level and a
    stream ([Fields a read shows](#field-grants)). A
    `/batch` answers it again for each
    statement: a courier's claim, then a read of the customer's door the
    claim lets them read, is one round trip. The list is held to a list
    claim's bound, 1 000 values, and a statement whose list would hold
    more is refused (400) rather than answered with a list cut short; a
    value held by many rows counts once. Rules that read each other's
    collections — `positions` reading `orders` whose
    rules read `positions`, at any depth, whatever their roles
    — stop the server at startup, since none of them could be answered
    first; so do a rule on `*` that reads a collection, a claim
    anywhere in the inner `get` but its `where`, and
    what an inner `get` may not say anyway. A scoped
    subscription whose rules read another collection is bound again when
    that collection is written: the courier enters a customer's stream of
    `positions` as the order is claimed and leaves it, a
    deletion of the row it was sent, as it is delivered, with no write to
    `positions` between. It costs one inner `get` a
    statement — a `@hash` bucket's read, 1.8 to 2.1 µs in
    process and 4.3 to 4.7 µs over HTTP on loopback above a rule naming a
    claim — and a
    stream one at each write to the collection its rules read: beside 100
    scoped streams a write took 1.89 ms of the server's CPU against 1.65
    under a rule naming the courier as a claim, every scoped stream
    looking at every write either way. A token with no such rule runs as
    it did (`make
    policy-bench`, `ARGS=streams`).

- **`match` scores over the rows the token may read.**
    BM25's statistics — how many documents, their mean length, how many
    hold each word — are taken over the rows the query's filter selects,
    the token's rules among them, not over the collection: there a user's
    own memo scored 9.87, then 4.79 once another user wrote 200 private
    memos holding the same word, which told the first how many of the
    second's held it. A score is what a collection holding the user's rows
    alone would give, and so is the order of `match`,
    `rerank`'s candidates and `fuse`'s text side. It
    costs the token's query its rows' lengths and each word's count among
    them: over 100 000 documents, a filter keeping 2.5% took 0.16 ms and
    takes 0.24, one keeping half 5.4 and 6.2. The server's own token
    scores over the collection, as before.

- **Subscriptions** are told only of rows they were sent: a scoped
    subscriber hears of a row leaving its shape only if the row was in it,
    so the ids other users write never reach it — rows leaving as a rule
    that reads another collection moves among them. Where a rule names the
    fields it shows, a row written again that shows what it showed is not
    sent: a write to a field the subscriber does not see would tell it
    when.

- **Tokens** are HS256, or RS256 with `--jwt-keys` (below).
    The algorithm is the key's, not the token's, so `alg: none` is
    refused. Every token is checked against `exp` and `nbf`,
    and one with no `exp` is refused (401): it would be good for
    ever, and a token that leaked could only be outlived by rotating every
    key. `--jwt-require-exp off` takes such tokens again, and
    `--jwt-max-age <s>` also refuses one whose
    `exp` lies further ahead than that. `--mint-token`
    gives claims naming no `exp` one an hour away. A rule with `for` applies
    to tokens whose `role` claim names that role, or lists it; rules on one
    collection widen each other; a rule whose filter names a claim the token
    lacks does not apply. A subscription is checked when it opens, and
    ended at its token's `exp` with
    `event: error` and
    `{"error":"the token has expired","status":401}`: the
    client takes it as a refused request, fetches a fresh token and opens
    the stream again (`FenecHttp.live`, `subscribe`,
    the sync layer, Go's `*Error` and .NET's
    `Event.Status`). Without it a stream outlived its token, and
    a member taken off a team heard the team until the client went away.
    No other request is held open past its token, nor before it: a read,
    a subscription or a `/_changes` page sent with
    `Fenec-After` waits only once its token is taken
    — one the server refuses is answered 401 at once, never after the
    wait — and is checked again once its wait ends, a write
    held by `Fenec-Wait` asks its token again each time it runs,
    and `/_changes` takes no scoped token.

- **Refusals** tell a scoped token only what it may read: a
    `@unique` clash (409) names the field, not the document
    holding the value nor the value, since that row is another user's.

- **Tenants.** On a `--dir` node a token reaches
    `/t/<t>/` only when its `tenant` claim names
    `<t>` (a text, or a list holding it), and is refused
    with 403 under every other tenant's prefix; one naming no tenant is
    refused there unless the node runs with
    `--jwt-unbound-tenants` ([Tokens](https://fenecdb.com/docs//docs/sharding#tokens)).

The server's own `--http-token` keeps full access alongside.

## Keys, rotation and identity providers

To accept tokens from Auth0, Clerk, Firebase or another identity provider,
give `--jwt-keys` its published key file. Edit the file to rotate
keys; no restart is needed.

```bash
# the provider's public keys, as it publishes them
curl -s https://example.auth0.com/.well-known/jwks.json > jwks.json
fenec-server --file data.fenec --http 0.0.0.0:8080 --jwt-keys jwks.json --policy policy.txt
```

Details

`--jwt-keys` takes a JWKS file, `{"keys": [...]}`, in
place of a secret. An `oct` key verifies HS256 tokens and needs at
least 32 bytes. An `RSA` key of at least 2048 bits verifies RS256
tokens, which is what Auth0, Clerk, Firebase and most identity providers
sign with. A token issued by one of them is taken with its public keys
alone. Keys of any other kind (EC, OKP) are skipped. A key is only used for
its own algorithm: an RSA modulus is public, and taken as an HS256 secret it
would let anyone sign. A token that names a `kid` is checked
against that key only. A token without one is checked against each key of
its algorithm.

The server looks at the file at most once a second and reads it again
when it has changed, so keys rotate without a restart. To rotate:

- Add the new key to the file.

- Start signing with it.

- Once the tokens signed with the old key have expired, remove the old
    key.

If the file can no longer be read, the server keeps the keys it read
last and logs why. `--mint-token` signs with the first
`oct` key and names its `kid`.

A token that has already been verified is kept by its text, so that
later requests with it skip the check. The server keeps up to 4 096 tokens,
and the kept tokens are dropped whenever the keys change. Checking a token
from that cache takes 0.3 µs for HS256 and 0.5 µs for RS256. Checking a new one takes 2.9 µs for HS256 and
38 µs for RS256 over 2048 bits (Apple M1).


---

Source: https://fenecdb.com/docs/sync

# Sync

Keep a copy of your data in the browser or an app that reads without
the network, writes instantly and stays up to date with the server.

## Start a replica

Call `sync` with the server and the rows you want. Then query it
with the same builder you use anywhere else.

```ts
import { sync } from './fenec.js';
import type { FenecSchema } from './fenec-schema.js';   // fenec types, over the server's file

const db = await sync<FenecSchema>({
  url: 'http://127.0.0.1:8080',
  shapes: [{ collection: 'tasks', where: { status: 'open' }, key: 'key' }],
});
await db.ready();

// read: from the local copy, no network
const rows = await db.from('tasks').where('priority', '>=', 3).rows();

// live query: re-runs on every change
const stop = db.live(db.from('tasks').order('priority', 'desc'), draw);

// write: to the local copy first, instantly, then to the server
await db.from('tasks').insert({ title: 'new', status: 'open' });
```

**The replica loads `fenec.wasm`**, the full module,
unless given another, so its `near` walks a graph.

Details

A copy on the client, changes streaming from the server, optimistic
writes. In fenecdb that is not a library — it is an endpoint and a transport,
because the same engine already runs on both sides.

With no module given, `sync` opens
`./fenec.wasm` beside the page, its collation data from
`collate/` beside it; `wasm` takes another URL, the
bytes or a compiled module, and `local` a database already
open.

The query code does not change. `db.from(...)` is the same
builder producing the same FenecQL; the only difference is where it is
bound.

### The server owns the schema

A replica takes its collections as the server declares them. Code that
declares its tables ([Schema in code](https://fenecdb.com/docs/javascript#schema))
hands them over too, and they are checked against the server's, never
applied:

```ts
import { sync } from './fenec.js';
import { todos } from './tables.js';

const db = await sync({
  url: 'https://todos.example.com',
  schema: { todos },
  shapes: [{ collection: todos, key: 'key' }],
});
```

- **What the code declares must be there.** A collection, a field
    or an index the server lacks, or one of another type, refuses the open:
    a `FenecError` naming each difference.

- **What the server holds beside it is its own.** A field or a
    collection the code does not declare is fine: an older app keeps
    working while the server moves ahead.

- **Nothing is applied, not even an addition.** The schema changes
    where it lives, on the server: its owner's migrations, a deploy step's
    `connect(url, { schema, migrations, migrate: true })`.

Details

The comparison is the server's engine (`POST
/_schema/plan?mode=follow`), the same that checks a page's own
database at an open, so the two say the same things in the same words. A
replica reconciles rows by its server's field order and types, which is why
it never declares a collection of its own: a field added in the page and
not on the server could not be decoded either way.

## Native apps sync too

An iOS, Android or Flutter app keeps its replica in a file on the
device, with the same shapes, the same optimistic writes and the same
catching up -- and its writes wait in the file while the server cannot be
reached. Opening the file as a replica is the one line that changes:
`Fenec.sync` in Swift and Kotlin, `Fenec.openSynced`
in Dart ([Sync with a server](https://fenecdb.com/docs/mobile#sync)).

## The stream carries state, not transactions

The server sends the current state of each row that changed, not a log of
every write. That is what a replica needs; it is not an audit log.

Details

The classic route is a transaction log: every write is an event and the
subscriber replays them. fenecdb sends the current state of what changed
instead — the stream is a clock, not a record. That is right for a
replica and wrong for anyone who wanted an audit log.

## A shape is a set, not a window

A shape picks the rows of one collection the client keeps, by a filter the
server applies. It cannot be "the latest 100": `order`,
`limit`, `offset` and `count` are
refused.

Details

The whole database is in memory and the WASM32 address space is 4 GB, so a
client cannot pull an entire collection. A subscription is a subset and the
filter is applied on the server.

**`order`, `limit`, `offset` and
  `count` are refused in a subscription** with a 400. A shape like
  "the latest 100 rows" looks correct but is not: when a new row arrives the
  oldest has to drop out, and an incremental diff cannot express that. An
  explicit error beats a silently wrong stream.

The shape condition uses the `?field=op.value` form, so there is
no `or` group and no function call. The reason is escaping: a free
`where=` would turn into string concatenation, exactly what the
builder avoids. Escaping goes to the transport via
`URLSearchParams` and types go to the server, leaving no injection
surface.

**One collection per shape.** The seed has to mean "this is the whole of
this collection", or the cleanup step becomes ambiguous.

## Optimistic writes

A write shows up locally at once, then goes to the server. It is kept
until the server answers: if the server refuses it, the local change is
undone; if the server cannot be reached, it waits and goes again.

```ts
const p = db.from('tasks').insert({ title: 'new', status: 'open' });
await db.from('tasks').where('title', 'new').first();  // already here
await p;                                               // the server has it
```

- **Inserts need a `key`,** a `text @hash` field,
    to match the local row with the server's. Without one, an insert waits
    for the server.

- **Updates and deletes need no key:** they work by the filter, by
    id or anything else -- a row an insert made before the server answered
    included.

- **Writes go one at a time, in order,** each under an
    `Idempotency-Key` it keeps until it is answered.

- **An increment lands from every replica.**
    `update({ n: inc(1) })` is worked out over the replica's row
    at once and sent as the statement, which the server works out again
    over its own: two replicas that each increment offline leave the
    server 2 up, and both replicas come to its sum.

Details

A write is three steps and the order matters: first gather what undoes the
local change, then apply it locally, then queue it for the server. If the
server refuses, the local side rolls back fully — no network needed, the undo
is already in hand. The queue and the undo are kept in the replica itself, in
collections of the sync's own (`_sync_queue`,
`_sync_temps`, `_sync_shapes` with the cursors), as an
app's file keeps them ([Sync with a server](https://fenecdb.com/docs/mobile#sync)).

Everything up to the first `await` is synchronous. An
`async` function body runs synchronously until its first
`await`, so the optimistic row is local before the caller awaits the
promise. A single microtask in between would have broken the "visible
instantly" promise.

**Reconciliation by key.** The id space belongs to the server and the
client cannot know it in advance, so an optimistic row gets a temporary id
starting at 252 — far from the server's range and below
`Number.MAX_SAFE_INTEGER`. What matches the two is a **business
key**: when a row with the same `key` arrives from the
subscription, the temporary one is dropped. An update or a delete that
reaches a row the server has not sent yet is sent a second way as well, by
that key -- `set tasks {..} where key in [..]` beside the
statement -- since the temporary id is the page's alone. An insert the shape
does not hold leaves its temporary row once the stream is past the change
the server answered it at (`Fenec-Seq`).

**An insert into a keyless shape is not applied optimistically**,
  unless it names its id. With nothing to reconcile, the row would stay a
  local duplicate; one round trip of latency beats a silent copy.
  `key` must be `text @hash`.

Writes go one at a time because a write that landed after one written
after it would leave the server holding the earlier. The promise a write
returns settles with the server's answer, so it waits through an outage;
the row is in the replica from the start, and nothing needs to await it.

## Refusals, retries and offline

Only a refusal puts a write back. Anything else that keeps the server from
answering keeps the write, and it goes again until the server answers it --
once, whatever the network did meanwhile.

| The server answers | The write |

| 2xx | landed: it leaves the queue, its promise resolves |

| nothing (the network failed), `408`, `429`, `5xx` | kept and sent again under its key, after 250 ms doubling to 15 s -- or after the `Retry-After` the server names, where that is longer: a tenant's quota, a tenant being moved or failed over |

| `401` | kept; nothing is sent until a new token (`tokenProvider`, `setToken`) |

| `507` naming a tenant's quota of `bytes` or `rows` | refused: put back, its promise rejects, `onRefused` is told with `e.quota`, and the writes behind it go on |

| any other `4xx` | refused: put back, its promise rejects, `onRefused` is told |

```ts
import { sync } from './fenec.js';

const db = await sync({
  url: 'https://api.example.com',
  shapes: [{ collection: 'tasks', key: 'key' }],
  persist: 'tasks',                                // writes survive a reload
  tokenProvider: () => auth.refresh(),             // asked on a 401
  onRefused: (e) => toast(e.quota ? 'This workspace is full' : `${e.status}: ${e.message}`),
});
await db.pushed();                                 // every write answered
```

Details

**Once, by the key.** A write whose answer was lost -- the request
reached the server, the response never came back -- is sent again under the
`Idempotency-Key` it was first sent with, and the server answers it
as it answered the first time ([Idempotency](https://fenecdb.com/docs/http#idempotency))
rather than make the row twice.

**Offline.** A write made while the server cannot be reached waits in
the queue; the page's `online` event, or
`setOnline(true)`, sends it at once, the backoff forgotten -- unless
the server said when to come back (below) -- and
`offline` or `setOnline(false)` closes the streams.
Writes left from before go before the streams open again, so what they
bring holds them.

**A reload.** With `persist` -- or a `local` kept in
a file (`openFile`) -- the queue is stored with the replica the
moment a write is made, and a page opened again sends what it had not,
under the same keys. Without either, a closed page takes its unsent writes
with it.

**Backoff.** 250 ms, doubling to 15 s, with 30% jitter on top so a
server back up is not met by every client at once; a stream that cannot
reconnect waits the same way, and starts the count again once it opens.

**A tenant's quota.** A tenant node holds each tenant to its quotas
([Quotas](https://fenecdb.com/docs/sharding#quotas)), and names the one that refused
a request in its body, which the replica reads. A `507` of
`bytes` or `rows` is a refusal: waiting makes no room,
deleting does, and a write that only deletes always lands -- held, the
write would hold every write behind it for good, the delete that would make
its room among them. A `507` naming no quota is the server's own
ceiling (`--max-memory`) or a proxy's, and is waited out as a
`5xx` is. A `429` of `streams`,
`held`, `requests`, `writes` or
`write_bytes` is waited out for its `retry_after_ms` --
the wait to the millisecond, a bucket refilling a row in 2 ms where
`Retry-After` rounds it up to a second -- or its
`retry_after`, the seconds `Retry-After` says, where
no milliseconds are given, where that is longer than the backoff, an hour at
most, with the jitter on top, since every client the quota refused
in one moment is told the same wait; a stream refused so waits the same
way, its shape not `connected` meanwhile, and
`status()` names the quota (`quota`) beside the
error. `setOnline(true)` forgets the backoff but not the
server's word: the network says nothing of a tenant's quota.

**A move or a failover.** A node answers a write to a tenant it is
moving, or no longer holds, `503` with
`"retry_after": 1` beside its `Retry-After`
([Moving a tenant](https://fenecdb.com/docs/sharding#moving-a-tenant)); a standby
router whose directory has not arrived answers a stream so too. The write is
kept and goes again through the router after that second where it is longer
than the backoff, as after a quota's `429`: the node's word for
how long a move holds its writes, never the backoff's guess under it.

**Put back in order.** A refused write's rows are written back as they
stood before it. A later write to the same rows, still waiting, is not
replayed on top; the server's answer to it, and the change it sends, has the
last word.

## Live queries

`db.live(query, fn)` runs the query again whenever a collection
it reads changes, and hands `fn` the new rows. Running it again on
the local copy takes under a millisecond.

A database in the page with no server has the same `live`: see
[State in the page](https://fenecdb.com/docs/javascript#state). Moving from one to
the other leaves the code that reads unchanged.

Details

There is no incremental maintenance and none is needed. The local side is an
indexed database, so re-running the query from scratch is already under a
millisecond — the differential dataflow that a `Map`-backed
collection requires buys nothing here.

It would not even be correct for `near`. A single insert
can change the entire top-k ordering, and there is no incremental HNSW
maintenance that is both cheap and correct — the machinery would have to be
disabled for fenecdb's headline feature.

Invalidation is at collection granularity: the change stream says which
collections changed, the live queries bound to them re-run, and all of it
coalesces into one frame. Anything finer, like intersecting id sets, would cost
more than the local query itself. A query built with `db.from`
knows the collections it reads, each `lookup`'s included; a FenecQL
text names them with `{ collections }`, or runs again after every
write.

A database in the page alone runs its live queries in a microtask after
the task that wrote, not at the next frame: its writes are the page's own,
and nothing lands in chunks. Both use the same code.

## Batches

`db.batch` makes several writes one: one request, one key. They
land on the server together or not at all, and on a refusal the local copy
is put back too.

```ts
await db.batch(async (t) => {
  await t.from('tasks').insert({ title: 'a', status: 'open' });
  await t.from('tasks').where('key', 'x').update({ status: 'closed' });
});
```

Details

The batch is one block on the server: its statements run in order under a
single write lock, and their writes land together, as one record, or not at
all. The gain is one round trip instead of N, no other writer slipping in
between, and nothing half done.

**Both sides roll back.** A batch that is refused lands nothing on the
server, and the local side puts back exactly what it applied optimistically,
the last write first, so the two stay one state. A batch whose answer did not
come is kept and sent again, whole, as any write is.

## Multiple tabs share one stream

Open tabs elect one leader that holds the connection to the server, passes
changes to the others and sends every tab's writes, one queue for all.

- **Turn it off** with `leader: false`; without
    `navigator.locks`, as in Node, it is off already.

Details

Every tab has its own wasm instance. If each opened its own subscription
there would be N copies and N connections, so a single **leader** is elected
with `navigator.locks`; the others receive the same batches over
`BroadcastChannel` and apply them to their own local copies. The
apply path is identical — only the transport differs.

Writes go through the leader, so one queue keeps them in order. A tab
applies its own write to its own copy at once, as the leader does, and hands
it to the leader, which keeps it in its queue -- stored with its replica when
it persists -- sends it under its key and tells every tab the answer; the tab
that made the write lands it or puts it back in its copy. When the leader
goes, the lock passes to the next tab, which takes over the queue its
predecessor stored and is handed every other tab's unanswered writes again;
the key makes a write sent twice land once.

## Dropped connections and the horizon

A client that reconnects gets only what changed since it left. If it was
away too long, the server sends it a fresh copy instead.

```bash
--changes <n>    entries in the ring (default 4096, 24 bytes per entry)
```

Details

A client returns with `?since=<cursor>` and asks for no
seed. If the cursor has fallen behind the ring, the server reseeds by itself
rather than silently sending incomplete events.

That number is directly **how far behind a subscriber may fall**. At 100
writes per second, 4096 entries is roughly a 40 second window. With
`persist`, the image and the cursor are written to IndexedDB,
so a reopened tab is not reseeded from scratch — unless its cursor has fallen
past the horizon.

A seed writes the shape over the replica and deletes what it does not
hold, keeping the rows of writes the server has not answered -- and of those
it answered after the seed was taken -- whose copies come later. Written over
rather than cleared first, a row that stayed keeps its vector's node.

**The server's schema changed.** A change says so
(`"schema": true`), or rows come with a field the replica has not
got: the replica asks `GET /collections` again, makes the
collection again as the server now declares it, its rows kept in the fields
that are left, and is sent the whole shape. A schema declared in code is
checked against the server's again, and a difference goes to
`onError`.

When the server restarts, the horizon is set to the counter: a database
loaded from a file has no history, only its present state. A cursor standing
exactly there, after a quiet restart, gets an empty response; anything behind
it is reseeded.

## Cost

The server's change ring is about 96 KB at the default size, and
subscriptions do not poll.

Details

At the default `--changes 4096` the ring is about 96 KB and costs
one push per entry on the write path. Subscriptions do not poll — see
[the HTTP endpoint](https://fenecdb.com/docs/http#subscriptions) for why — and are
counted separately from ordinary requests under
`--http-max-streams`.

## Limits

A few things sync does not do, or does differently from what you might
expect.

- **A shape is not a security boundary** on its own; use a per-user
    token with scoped access.

- **`bytes` fields cannot be synced.**

- **The server has the last word** over the local copy.

- **A write that picks its rows** — `order` and
    `limit` before an `update` or a
    `delete`, or `{ returning }`, a
    [job queue's claim](https://fenecdb.com/docs//docs/queues) — is refused on a synced
    collection: the replica would pick and answer its own rows, and the
    server its own. Send it to the server itself (`db.remote`).

Details

- **A shape alone is not a security boundary.** Under the server's own
    token, a changed id that does not match the shape is sent as a deletion:
    harmless locally, but which ids changed leaks. A per-user token
    with [scoped access](https://fenecdb.com/docs//docs/http#scoped-access) hides both:
    the rows outside its rules are filtered out, and it hears only of rows it
    was sent.

- **`bytes` fields cannot be synced.** In JSON they become
    number arrays with no way back. That is a limit of the HTTP surface, not
    something sync introduces.

- **In a projected shape the other fields read as `null`.**
    With `select`, the local row carries only those fields — which
    is all the client knows anyway.

- **`batch()` does not nest** and the calls inside it return
    no result: writes pile up and go in a single request.

- **Rolling back needs no network, but the server has the last word.**
    A rolled-back local state is overwritten by the server's in the next
    batch.

- **Rows a `@ttl` lets go of** leave a replica's reads at
    once -- the replica's collection has the server's `@ttl` -- and
    arrive as deletes when the server's sweep takes them.

## Persisting the replica

Add `persist` to keep the copy, its place in the stream and the
writes the server has not answered in IndexedDB, so a reopened tab starts from
there instead of downloading everything again, and sends what it had not.

```ts
import { sync } from './fenec.js';
import type { FenecSchema } from './fenec-schema.js';

const db = await sync<FenecSchema>({
  url: 'https://api.example.com',
  shapes: [{ collection: 'tasks', where: { status: 'open' }, key: 'key' }],
  persist: 'tasks',           // the replica, its cursors and its unsent writes
});
await db.ready();

db.live(db.from('tasks').order('priority', 'desc'), draw);
await db.from('tasks').insert({ title: 'new', status: 'open' });
```


---

Source: https://fenecdb.com/docs/integrations

# Integrations

Use fenecdb from LangChain, LlamaIndex, LangChain.js, the
Vercel AI SDK and React, each tested the way its framework tests its own,
from Go and .NET through their SDKs, and embedded in Swift, Kotlin and Dart
apps.

## LangChain

A LangChain vector store over fenec-server's HTTP endpoint. Install it, point it
at the server, and use it like any other store.

```bash
pip install "fenecdb[langchain]"
```

```python
from fenecdb.langchain import FenecVectorStore

store = FenecVectorStore(
    embeddings, "handbook",
    url="http://127.0.0.1:8080", token="...",
    metadata_fields={"source": "text", "year": "int"},
)
store.add_documents(documents)
store.similarity_search("how do I compact", k=4, filter={"source": "ops"})
retriever = store.as_retriever(search_kwargs={"k": 8})
```

- **Filter only on declared fields:** list them in
    `metadata_fields`.

- **Keyword and hybrid search:** pass `full_text=True`, then
    `mode="text"` or `"hybrid"`.

Details

A document is a row: its id, its text, its metadata as JSON and its vector
under `@hnsw`, in a collection created on the first write, once the
embedding's size is known. Adding an id that is already there replaces it —
the old row out and the new one in, sent as one `/batch` and run
under one write lock, so no reader sees the id twice or not at all.

A metadata field named in `metadata_fields` gets a column of its
own under `@hash`, and `filter` takes equality and
`$in` over those: `{"source": "ops"}` becomes
`where source = $2`, answered by the index before
`near` looks for neighbours among the rows that passed. A filter on
a field not declared is refused rather than applied to the JSON after the
search, which would return fewer than `k` rows and not say
so.

The score is fenecdb's own, the one the HTTP endpoint and the browser
module return: the cosine similarity under `cosine` (the default),
the distance under `l2`, the product under `dot`.
`similarity_search_with_relevance_scores` maps them the way
LangChain's own functions do.

```python
store = FenecVectorStore(embeddings, "handbook", url=..., token=...,
                         full_text=True, quant="int8")
store.similarity_search("compact dead records", k=4, mode="text")     # BM25 alone
store.similarity_search("how do I compact", k=4, mode="hybrid")       # both, fused
retriever = store.as_retriever(search_kwargs={"k": 8, "mode": "hybrid"})
```

With `full_text=True` the text is indexed for BM25 as well
(`@text`, added to a collection made before without it), and a
search takes `mode`: `"text"` ranks by the words alone
with `match` and embeds nothing, and `"hybrid"` runs
`match` and `near` each to their own depth and fuses
the two rankings ([`fuse`](https://fenecdb.com/docs/fenecql#hybrid)). Their
scores are BM25's and the fusion's, not similarities. `quant`,
`"int8"` or `"bit"` (`cosine` only), keeps the
graph over codes, and `near` still orders by the vectors
themselves.

## LangChain.js

The same store from JavaScript. It runs over a database in the page, with no
server, or over fenec-server's HTTP endpoint.

```js
import { Fenec, connect } from '@fenecdb/web';
import { FenecVectorStore } from '@fenecdb/langchain';

// The database in the page -- or behind fenec-server: connect('http://127.0.0.1:8080')
const db = await Fenec.open('./fenec.wasm');
const store = new FenecVectorStore(embeddings, {
  client: db,
  collection: 'handbook',
  metadataFields: { source: 'text' },
  fullText: true,
});
await store.addDocuments(documents);
await store.similaritySearch('how do I compact', 4, { source: 'handbook' });
await store.search('compact', 4, { mode: 'hybrid' });
const retriever = store.asRetriever({ k: 4 });
```

Details

The Python store's collection, from JavaScript: a document is a row of
its id, text, metadata as JSON and vector under `@hnsw`, made on
the first write; `metadataFields` get columns of their own under
`@hash`, and `filter` takes equality and
`$in` over them; with `fullText`,
`search(query, k, {mode})` ranks by the words
(`'text'`) or both (`'hybrid'`). `client`
is anything with `run(sql, params)`: a `Fenec` keeps
the whole RAG index in the page, no server, and a `FenecHttp` the
server's. Writing an id already there replaces it — two statements, which
over HTTP are two requests.

## LlamaIndex

A LlamaIndex vector store over fenec-server's HTTP endpoint, with metadata
filters, deletion by document, and text and hybrid search.

```bash
pip install "fenecdb[llama-index]"
```

```python
from fenecdb.llama_index import FenecVectorStore
from llama_index.core import StorageContext, VectorStoreIndex

store = FenecVectorStore("handbook", url="http://127.0.0.1:8080", token="...",
                         metadata_fields={"author": "text", "year": "int"})
index = VectorStoreIndex.from_documents(
    documents, storage_context=StorageContext.from_defaults(vector_store=store)
)
index.as_retriever(similarity_top_k=4).retrieve("how do I compact")
```

- **Hybrid search:** pass `full_text=True` and query with
    `vector_store_query_mode="hybrid"`.

Details

A node is a row with its document's id beside it, so
`index.delete_ref_doc(...)` is one `del` over a hashed
field. Filters take `==`, `!=`, `<`,
`<=`, `>`, `>=`, `in` and
`nin` over the declared fields, joined by `and` or
`or` and nested; `node_ids` and `doc_ids`
narrow the same way. `get_nodes`, `delete_nodes` and
`clear` are there; `delete_nodes()` with neither ids nor
filters deletes nothing — everything is `clear`, asked for by name.
With `full_text=True` the text is indexed for BM25 as well, and
a query takes the modes that need it: `TEXT_SEARCH` ranks by the
words alone (`match`), and `HYBRID` runs
`match` and `near` each to their own depth —
`hybrid_top_k`, or `sparse_top_k`, deep — and fuses the
two rankings ([`fuse`](https://fenecdb.com/docs/fenecql#hybrid)). The fusion
adds ranks rather than scores, so `alpha` has nothing to weigh and
is not read. `quant` is taken as the LangChain store takes it.

```python
store = FenecVectorStore("handbook", url=..., token=..., full_text=True)
retriever = index.as_retriever(similarity_top_k=4, vector_store_query_mode="hybrid")
```

## Vercel AI SDK

Three functions to copy from `integrations/ai-sdk/rag.js`:
`index` your documents, `retrieve` for a question, and
`searchTool` for a model to call.

```js
import { generateText, stepCountIs } from 'ai';
import { openai } from '@ai-sdk/openai';
import { Fenec } from '@fenecdb/web';
import { index, retrieve, searchTool } from './rag.js';   // integrations/ai-sdk/rag.js

const db = await Fenec.open('./fenec.wasm');
const model = openai.textEmbeddingModel('text-embedding-3-small');
await index(db, { collection: 'docs', model, documents });   // [{text, source?}]

const { text } = await generateText({
  model: openai('gpt-4.1'),
  tools: { search: searchTool(db, { collection: 'docs', model }) },
  stopWhen: stepCountIs(4),
  prompt: 'How do I compact the database?',
});
```

Details

The AI SDK has no vector store interface to implement — it has embedding
models and `tool` — so the integration is a file of three
functions to copy, `integrations/ai-sdk/rag.js`:
`index` embeds with `embedMany` and writes the
documents, their text under `@text` and their vectors under
`@hnsw`; `retrieve` embeds the question with
`embed` and answers with `near`, or with
`hybrid` the words and the vector fused, kept to a
`source` through its `@hash` index;
`searchTool` is a `tool` a model calls with a query.
It takes a `Fenec`, so the index lives in the page or a Worker,
or a `FenecHttp` for the server's.

## React

`useLiveQuery` renders a query over the database in the page
— a `Fenec` of its own or a synced replica — and runs it again
after every write it may read. No render waits on the network.

```ts
import { sync, type FenecSync, type Row } from '@fenecdb/web';
import { FenecProvider, useFenec, useLiveQuery } from '@fenecdb/react';
import type { FenecSchema } from './fenec-schema.js';   // fenec types, over the server's file

type TaskRow = Row<FenecSchema['tasks']>;

const db = await sync<FenecSchema>({ url: 'http://127.0.0.1:8080', shapes: [{ collection: 'tasks' }] });

function Open() {
  const db = useFenec<FenecSync<FenecSchema>>();
  const rows = useLiveQuery<TaskRow>(db.from('tasks').where('done', false).order('title'));
  if (rows === undefined) return <Spinner />;
  return rows.map((t) => <Task key={t.id} task={t} />);
}

<FenecProvider db={db}><Open /></FenecProvider>
```

- **No server needed:** `<FenecProvider db={await Fenec.open()}>`
    makes the page's own database the app's state
    ([State in the page](https://fenecdb.com/docs/javascript#state)).

- **No module needed:** `<FenecProvider db={connect(url)}>`,
    from `@fenecdb/web/client`, runs each query on the server and
    again after every write to a collection it reads.

- **It is `undefined`** until the first answer.

- **A failing query throws** to the nearest error boundary.

| Provider | Where a query runs | A render after a write | Loads |

| `connect(url)`, `@fenecdb/web/client` | on the server, each time | waits for the server: the subscription's change, then the query's round trip | no module, 9 KB brotli of client |

| `sync({ url, shapes })` | in the page, over the replica | waits for nothing: this tab's write is local, another client's arrives on the stream | `fenec.wasm`, 273 KB brotli |

| `Fenec.open()` | in the page, over its own database | waits for nothing | the same `fenec.wasm` |

Details

Over a replica or a database in the page the rows come from the page, so
no render waits on the network: `live` runs the query again after
every write it may read — a write of this tab's, or one the server streamed
into a replica from another client — and the component renders what it
answers. Over `connect` the query runs on the server, again after
each write the server's subscriptions report to a collection it reads; a
FenecQL text names those with `{ collections }` there, since
nothing in the page can know what a text reads. A FenecQL text
works too, on the provider's database:
`useLiveQuery('get todos count', { collections: ['todos'] })`. `useLiveQuery` is
`undefined` until the first answer, and subscribes by the query's
text and parameters, so the query built again on every render, which is the
natural way to write one, does not subscribe again unless it asks something
else; the rows of the query it asked before are not handed out as this one's.
A query that fails throws while rendering, to the nearest error boundary. A
query that asks for [facets](https://fenecdb.com/docs/fenecql#facets) has them on
its rows, `rows.facets.brand`, counted over every match: a
results list and its filter sidebar from one hook, live alike. It
lives in `integrations/react`, with React as its one peer
dependency.

## Go and .NET

An SDK each over fenec-server's HTTP endpoint, each its standard library
alone, beside Python's `fenecdb` and JavaScript's
`@fenecdb/web`.

```bash
go get github.com/fenecdb/fenec/integrations/go
dotnet add package FenecDb
```

- **The same surface in both:** a statement and its rows, typed
    rows, writes with the change they left the database at, atomic batches,
    an idempotency key, a read that waits for a write on a replica, reads
    sent to a tenant's read replica near the client, after its own writes
    (`WithReadsNearest()`, `ReadsNearest`; Read replicas), a subscription,
    the change stream, a tenant, the health check, and a typed error
    carrying the status.

- **The query builder,** as Python's and JavaScript's have it:
    `db.From("docs").Where("year", ">=", 2024).Near("embed",
    v).Limit(5)`, immutable, ending in `Rows`,
    `First`, `Count` or a write. All four build the same
    text from the same chain, held to one golden file
    ([The builder](https://fenecdb.com/docs/languages#builder)).

- **Vectors round-trip to the bit:** a `[]float32`,
    `float[]` or `ReadOnlyMemory<float>` goes out
    as the decimals that read back as each float.

Each language's example is on [Your
language](https://fenecdb.com/docs/languages), and the API in `integrations/go` and
`integrations/dotnet`.

Details

The server reads a number as a double and rounds it to a float, and one
float, `7.038531e-26`, has a shortest text whose double lands
exactly between it and the float above: Go's `encoding/json`
wrote it so, and the server stored the float above. Both SDKs write that one
as its double's text, as the server itself writes it, and their tests try
one float in 4 099 over every exponent and read a vector holding it back
from a server.

A subscription takes the REST surface's filter
(`year=gte.2024`, `where=`, `select=`),
not a statement with parameters: the stream is
`GET /<collection>/changes`. A request is bounded by the
client's timeout through its context or token, never the HTTP client's own,
which would cut a subscription short.

## Swift, Kotlin and Dart

Not over HTTP: the database in the app, a file on the device, through
the native library (`crates/fenec-ffi`). A live query drives the
UI -- a SwiftUI observable, a Kotlin `Flow` for Compose, a Dart
`Stream` for a `StreamBuilder`.

```text
.package(url: "https://github.com/fenecdb/fenec", from: "0.1.7")   // Swift: product FenecDB
implementation("com.fenecdb:fenecdb-android:0.1.7")           // Kotlin, Android
flutter pub add fenecdb_flutter                                     // Flutter
```

- **The query builder,** the same seven ways, held to the one golden
    file ([The builder](https://fenecdb.com/docs/languages#builder)).

- **Vectors as their bytes:** a `[Float]`,
    `FloatArray` or `Float32List` goes to the library
    as its `f32`s, never as digits.

- **Live queries,** run again once a burst of writes to a collection
    they read.

The examples, where the file goes and what a write survives are on
[Mobile and native apps](https://fenecdb.com/docs/mobile).

## Their tests

Each integration runs its framework's own tests, or the ones its framework
holds its integrations to, against a real fenecdb.

| Integration | Held to | Run with |

| LangChain | LangChain's standard vector store suite (`langchain-tests`), sync and async, and filters, relevance, `from_texts`, the text and hybrid modes and int8 codes besides | `make python-test`: fenec-server built and started here, the tests from a `python:3.13` container |

| LlamaIndex | the tests LlamaIndex's own vector store integrations run — the class, a write and a nearest match, deletion by document, nodes by id and by filter, every filter operator, an index on top — and the text search and hybrid modes |

| LangChain.js | what LangChain.js's own vector store integrations are held to — documents in and found again with their ids and scores, replaced under an id, deleted, filtered, `fromTexts`, a retriever — and the text and hybrid modes, each over a database in the page and over fenec-server's HTTP endpoint; LangChain.js publishes no standard suite | `make langchain-test` |

| Vercel AI SDK | the SDK's own mock models: documents embedded with `embedMany` and found again with `near`, kept to a source and fused with the words, and a model that calls the search tool in `generateText` and answers from what it returned | `make ai-sdk-test` |

| React | a stand-in database for the contract, and a real fenec-server, a replica and `connect` for a row another client writes | `make react-test` |

| Go | a primary with a token, a JWT policy and the change stream, a replica and a node of tenants, started by the tests: `near` with exact scores, parameters of every type, the errors, a batch put back whole, an idempotency key, `After` on the replica, a subscription, `/_changes`, a tenant and a scoped token; the query builder against every case of `integrations/builder-golden.json`, and its queries' answers against the same statements sent as text | `make go-test` |

| .NET | the same, with xunit | `make dotnet-test` |

| Python client | the query builder against every case of `integrations/builder-golden.json`, sync and async, and its queries' answers against the same statements sent as text | `make python-test` |

| Swift | the engine through the library -- `near` with exact scores, rows decoded, a json field's list as written, every kind of error, a file opened again and refused a second open, many tasks at once -- the query builder against every case of the golden file and its answers against the statements as text, live queries as a stream and as observables, a burst of writes running each once; on macOS, and in CI on an iOS simulator | `make swift-test` |

| Kotlin | the same with JUnit on the JVM, the library built for Linux with its JNI functions; the AAR built with the NDK in CI | `make kotlin-test` (Docker) |

| Dart and Flutter | the same with `dart test`, and the Flutter plugin's test and its example built for Android and the iOS simulator in CI | `make dart-test` |

Details

LangChain's suite writes the same documents twice under the same ids, and
that is how it found a bug in the graph: a node whose every candidate
neighbour had been deleted was linked to nothing, so a collection deleted and
written again answered `near` with nothing. It is fixed in the
engine, with the test that shows it.


---

Source: https://fenecdb.com/docs/mcp

# Agents and MCP

An agent — Claude Code, Cursor, Claude Desktop, one you build
— searches and remembers through tools it calls. fenecdb serves those tools
over MCP, the Model Context Protocol: on its HTTP endpoint, or as a process
the agent starts.

Each tool runs a statement as `/query` runs one, under the
token the agent was given: a scoped token's agent reads its own rows and no
others, the tools that write are off unless the server is told, and an answer
too large for a model's context is refused with what to narrow — never cut,
since a model told nothing of the cut takes half an answer for the whole.

## Turn it on

On a server, `--mcp` serves the endpoint at
`POST /_mcp` (and `/t/<tenant>/_mcp` on a tenant
node, which a router forwards like any tenant path) — an underscore, as
`/_metrics` has one, so that no collection's path is taken:

```bash
fenec-server --file notes.fenec --http-token "$FENEC_TOKEN" --mcp
fenec-server --file notes.fenec --http-token "$FENEC_TOKEN" --mcp-write --mcp-memory memories
```

Over a file, `fenec mcp` is the server as a process the agent
starts, speaking stdio — a message a line on stdin, an answer a line on
stdout, its log on stderr:

```bash
fenec mcp --file notes.fenec --memory memories
fenec mcp --url http://127.0.0.1:8080/_mcp        # in front of a server, the token from FENEC_TOKEN
```

`--file` holds the database itself and takes
`notes.fenec.lock` while the session lasts — a server or an app
over the same file is refused, as it is refused while they hold it: two
processes writing one file corrupt it. Each write is on the disk before it is
answered. `--read-only` takes no lock and writes nothing, and reads
the file as it stood when the session began, beside a server writing it; to
read what a server serves now, use `--url`. Over
`--url` the tools are the server's, under its options and the
token's rules, and the bridge decides nothing.

## Claude Code

```bash
claude mcp add fenecdb -- fenec mcp --file notes.fenec --memory memories
claude mcp add --transport http fenecdb http://127.0.0.1:8080/_mcp --header "Authorization: Bearer $FENEC_TOKEN"
claude mcp list                                  # fenecdb: ... - ✓ Connected
```

With `--scope project` each goes into the project's
`.mcp.json`, which Claude Code loads for whoever opens it — the
token left in the environment:

{
  "mcpServers": {
    "fenecdb": {
      "type": "stdio",
      "command": "fenec",
      "args": ["mcp", "--file", "notes.fenec", "--memory", "memories"]
    }
  }
}

{
  "mcpServers": {
    "fenecdb": {
      "type": "http",
      "url": "http://127.0.0.1:8080/_mcp",
      "headers": { "Authorization": "Bearer ${FENEC_TOKEN}" }
    }
  }
}

## Cursor

{
  "mcpServers": {
    "fenecdb": {
      "command": "fenec",
      "args": ["mcp", "--file", "notes.fenec", "--memory", "memories"]
    }
  }
}

{
  "mcpServers": {
    "fenecdb": {
      "url": "http://127.0.0.1:8080/_mcp",
      "headers": { "Authorization": "Bearer ${env:FENEC_TOKEN}" }
    }
  }
}

## Claude Desktop

Claude Desktop starts its tools as processes, so a server is reached
through the bridge. An app started from the dock does not see a shell's
`PATH`: name `fenec` by its full path.

{
  "mcpServers": {
    "fenecdb": {
      "command": "fenec",
      "args": ["mcp", "--url", "http://127.0.0.1:8080/_mcp"],
      "env": { "FENEC_TOKEN": "the token" }
    }
  }
}

Every configuration on this page is loaded by a test
(`integrations/mcp/test.js`, `make mcp-test`) and each
of its servers called with the official TypeScript MCP client.

## The tools

| tool | what it does | takes |

| `schema` | the collections the token may read, each as the FenecQL that makes it, and — to the server's token — how many rows each holds | `collection` |

| `search` | rows ranked best first: by words over an `@text` field (BM25), by an embedding over an `@hnsw` field, or both fused; each row with `_score` and, by words, a `snippet` with the words found in `**bold**`; `fuzzy` matches words a typo away ([typos and prefixes](https://fenecdb.com/docs/fenecql#typos)), `prefix` the last word as the start of one | `collection`, `text`, `fuzzy`, `prefix`, `vector`, `filter`, `params`, `limit`, `fields`, `text_field`, `vector_field` |

| `query` | one FenecQL read: `get` with any clause, aggregates, `count`, `lookup`, `explain` | `query`, `params`, `include_vectors` |

| `get` | rows by id; the ids with no row come back in `missing` | `collection`, `ids`, `fields` |

| `write` | one `put`, `insert`, `set` or `del`, with `require n` if it must touch n rows (`--mcp-write`) | `statement`, `params` |

| `remember` | keeps a memory (`--mcp-memory`) | `text`, `vector`, `tags`, `expires_in` |

| `recall` | the memories that best match, or the latest | `text`, `vector`, `tags`, `limit` |

| `forget` | deletes a memory by its id | `id` |

A tool answers JSON — `{"rows": [...], "count": n}` for a read
— as its structured content and, written out, as the text a model reads. Vector
fields are left out of the rows: a 768-dim embedding is some 7 KB of JSON a
row, which a model reads as noise and pays for. `fields` names the
ones to keep, and `query`'s `include_vectors` keeps them.
A search's call and its answer:

{"name": "search", "arguments": {"collection": "docs", "text": "fox", "filter": "year >= $1", "params": [2024]}}

{"rows": [{"id": 3, "title": "cat", "body": "the cat watches the fox", "year": 2025,
           "snippet": "the cat watches the **fox**", "_score": 0.67}], "count": 1}

**Embeddings are the client's to compute.** fenecdb stores and searches
vectors and runs no model: an agent that has one passes `vector`,
made with the model the collection was filled with — a vector of another
length is refused, naming the field's. Without one, search by words.

**Values go in `params`, never into the text.** A filter is
a FenecQL condition, `year >= $1 and tags has $2`, parsed on its
own and refused unless it is a condition and nothing more — no clause after
it, no second statement, no parenthesis closing on the statement around it —
and it goes into the statement the tool builds as the condition parsed, never
as text. Names are checked against FenecQL's rule for one.

**What a tool refuses comes back for the model to correct**, as its
result with `isError`: an argument it does not take, a vector of
the wrong length, a statement the token's rules refuse, an answer past a
bound — each saying what to change, and with the status `/query`
would have answered it with. A call that is not one — an unknown tool, a
malformed message — is a JSON-RPC error.

## Bounds

| flag | default | past it |

| `--mcp-max-rows` | 100 | refused, 413: a `query` with no `limit` is asked for one row more than the bound, so a larger answer is known without being fetched |

| `--mcp-max-bytes` | 64K | refused, 413, measured on the JSON the model would read |

| `--mcp-calls-per-sec` | 20 | refused, 429, with when to call again: a scoped token's subject, or an address for the server's token |

A hundred rows and 64 KB are about 16 000 tokens of JSON, inside what
Claude Code takes from one tool before it warns. A tenant node's quotas hold
an agent's calls as they hold a client's requests.

## Security

- **The token is the request's.** The endpoint takes the bearer token
    `/query` takes: the server's token reads and writes everything,
    and a JSON Web Token is held to its [policy](https://fenecdb.com/docs//docs/http#scoped-access),
    ANDed into every statement a tool runs — so an agent given a scoped token
    sees exactly what its `/query` sees. `schema` lists
    only the collections it may read and counts none of their rows, and on a
    tenant node the token reaches only the tenant its claim names.

- **Writes are off unless asked for.** `--mcp-write` adds
    `write`, and `--mcp-memory` the memory tools alone.
    `write` runs `put`, `insert`,
    `set` and `del`, each held to the token's grants and
    to its policy as the row it writes; a schema change or a
    `compact` is never made over MCP, by any token. A server with
    `--http-read-only` takes no write over MCP either.

- **Where a request comes from is asked before its token.** An
    `Origin` other than the server's own, or the one
    `--http-cors` names, is refused (403). A server that asks for no
    token answers only a `Host` naming this machine —
    `localhost` or an address — so a page whose name was rebound to
    127.0.0.1 reaches nothing.

- **No TLS.** fenecdb speaks none: on an open network put a TLS
    terminator in front of the server, and give the agent's client the
    `https` URL. The stdio bridge speaks plain HTTP only.

- **An agent's writes are audited.** With `--audit` each
    call of `write`, `remember` or `forget` is
    a line (`"event":"mcp"`, the tool, whether it failed), beside
    the schema changes and the refused tokens. `/_metrics` counts
    calls in `fenec_mcp_tool_calls_total` by tool and outcome — no
    collection is a label.

## Memory

`--mcp-memory memories` gives an agent a memory that outlives
its conversations: `remember` keeps a sentence,
`recall` finds the ones that match what it is asked, and
`forget` lets a wrong one go. The collection is made at the first
memory, with a vector field if that memory came with a vector — and given one
at the first that does:

```fenecql
create collection if not exists memories (text text required @text, tags [text], owner text @hash, at timestamp @sorted, expires timestamp @ttl(1ms), vector vector<3> @hnsw(cosine))
```

The text is searched by its words (`@text`), so an agent with
no embedding model still recalls: BM25 finds the deploy runs on fridays
for when do we deploy. With vectors — `vector<N>` of
the embedding's length; 3 here so the examples fit — recall fuses the words
with the meaning. `expires` is when a memory goes:
`@ttl(1ms)` takes a row out of every read the moment its time
passes, and the sweeper deletes it; a memory with none is kept for good.

`remember` runs `insert into memories {text: $1, tags: $2,
at: now(), expires: now() + $3, vector: $4}`, its expiry the
milliseconds `expires_in` names (`30m`,
`12h`, `7d`). With the values written in:

```fenecql
insert into memories {text: "the deploy runs on fridays", tags: ["ops"], at: now(), expires: null, vector: [0.9, 0.1, 0]}   -- → 1
insert into memories {text: "staging is in eu-west-1", tags: ["ops"], at: now(), expires: null, vector: [0, 0.2, 1]}   -- → 1
insert into memories {text: "the user is debugging the login page", tags: ["session"], at: now(), expires: now() + 10000}   -- → 1
```

`recall` with words alone, with words and a vector, by a tag,
and with nothing — the latest first:

```fenecql
get memories select text, tags, at match text "when do we deploy" limit 5   -- → "the deploy runs on fridays"
get memories select text, tags, at match text "which region" near vector [0, 0.1, 1] fuse limit 5   -- → "staging is in eu-west-1"
get memories select text where tags has "session" limit 5   -- → "the user is debugging the login page"
get memories select text, tags, at order at desc, id desc limit 5   -- → "the user is debugging the login page"
```

Ten seconds on, the session's memory is out of every read:

```fenecql
get memories where tags has "session" count   -- → 1
-- 20 s later
get memories where tags has "session" count   -- → 0
```

Every FenecQL block on this page is run, statement by statement, by a test
(`crates/fenec-core/tests/mcp_docs.rs`), and the collection above
is held to the one `remember` makes.

**Each user's memories apart.** A policy line keeps each token's
memories its own: a memory written without an owner gets the token's
subject, and a recall reads that subject's alone. The collection is made by
the server's token, once — a scoped one may not make collections, and its
`remember` says what to run.

```text
memories  read,write  where owner = $jwt.sub
```

**An example agent.** `examples/mcp-agent` is Claude with
these tools over `fenec mcp --file`, through the Anthropic SDK's
tool runner: told something in one run, it recalls it in the next.

```bash
cd examples/mcp-agent && npm install
node agent.mjs "I indent with tabs, and the deploy runs on Fridays"
node agent.mjs "when does the deploy run?"
```

## The protocol

The revision is **2026-07-28**, the stateless one: each request names
its revision and the client's capabilities in `_meta`, there is no
handshake and no session, and `server/discover` says what is here.
Beside it are the handshake revisions clients still open with —
**2025-11-25**, **2025-06-18** and **2025-03-26**, an
`initialize` first — served the same way and without a session. A
request carrying `_meta`'s version is served by 2026-07-28's rules,
an `initialize` by the revision it agrees on.

- Over HTTP a 2026-07-28 request's headers — `MCP-Protocol-Version`,
    `Mcp-Method`, `Mcp-Name` — must say what its body
    says (400, `-32020`); a revision not served is refused naming
    those that are (400, `-32022`); an unknown method is a 404 with
    a JSON-RPC body. An answer is one JSON object, never a stream: no tool
    here has anything to say before its answer. `GET` and
    `DELETE` are 405, and a batch is refused.

- Resources and prompts are not offered. A collection as a resource is
    what `schema` answers already, and the clients here put tools
    before the model and resources behind a person's click; a prompt is a
    recipe this page gives.

## What it costs

A tool call runs the statement `/query` would, and adds
reading the JSON-RPC message, building the statement and writing the answer
twice — as structured content and as its text. Over 10 000 rows
(`make mcp-bench`, an M1):

|  | `/query` | MCP |

| a row by id, in process, `query` tool | 1.2 us | 3.8 us |

| a row by id, in process, `get` tool | 1.2 us | 4.4 us |

| two words, top ten with snippets, in process | 261 us | 263 us |

| a row by id over HTTP, p50 | 17.7 us | 21.9 us |

| two words over HTTP, p50 | 316 us | 323 us |

A few microseconds a call, against the seconds a model's turn takes. A
server without `--mcp` asks one comparison more of a request's path
and nothing else: `make requests-bench` in eight runs taken in turns
against the server before it, a row by id answered one client in 0.027 ms
against 0.028 at the median and eight in 122 000 requests a second against
111 000, inside the runs' own spread.


---

Source: https://fenecdb.com/docs/redis

# Instead of Redis

The jobs an app adds Redis for — counters, rate limits,
locks, sessions, queues — done by the database it already has, each a
statement or two.

Four things make these work. A `set` can read the row it
writes (`{n: n + 1}`), and the read and the write happen under
the one writer's lock, so an increment is atomic. Every write answers how
many rows it changed, so a write that names what it expects says whether it
found it, and `require` makes that count a condition of its
`/batch`. `@ttl` lets a row expire, and a write take
its place the moment it does. And every write can be watched: a
subscription is sent the rows of a shape as they change, and
`/_changes` every write on disk. See Writes that read the row for the
rules.

Every FenecQL block on this page is run, statement by statement, by a
test (`crates/fenec-core/tests/redis_docs.rs`), and each
`-- → x` is the answer it checks: a write's count, or the first
value of the first row a read answered. A line `-- 20 s later`
moves the test's clock, and `$1` is the rows the read before it
answered, as a client sends them back. The costs are `make
counters-bench` on an M1 Air, in process unless said otherwise; over
HTTP a request adds its round trip. Where another bench measured a recipe,
it is named.

## Redis's patterns, and where each one is

| Pattern | Redis | Here |

| Counters | `INCR`, `INCRBY`, `HINCRBY` | [`set c {n: n + 1}`](#counter) |

| Rate limit, fixed window | `INCR` and `EXPIRE` | [a row a window, `@ttl`](#fixed-window) |

| Rate limit, sliding window | `ZADD`, `ZREMRANGEBYSCORE`, `ZCARD` in `MULTI` | [a row a request, `@ttl`, a read that must find none](#sliding-window) |

| Rate limit, token bucket | a Lua script | [one `set` over the bucket's row](#token-bucket) |

| Lock with a lease | `SET NX PX`, a check-and-delete script | [`put … if absent`, `@ttl`, `del … where owner`](#lock) |

| Fencing token | none of its own: `INCR` beside the lock | [counted by the write that takes the lock](#fencing) |

| Optimistic transaction | `WATCH`, `MULTI`, `EXEC` | [a version, and `require 1` in a `/batch`](#optimistic) |

| Sessions | `SET EX`, `GET`, `GETEX`, `EXPIRE` | [`@ttl`, `set … returning`](#sessions) |

| Cache with a TTL | `SET EX`, `GET`, `DEL` | [`@ttl` and an upsert — no eviction](#cache) |

| Leaderboard | `ZADD`, `ZINCRBY`, `ZREVRANGE`, `ZRANGEBYSCORE`, `ZREVRANK` | [`@sorted`, `order … limit`, `count`](#leaderboard) |

| Sets, deduplication | `SADD`, `SISMEMBER`, `SREM`, `SCARD`, `SINTER` | [`@unique` and `if absent`](#sets) |

| Unique visitors | `PFADD`, `PFCOUNT`, `PFMERGE` | [`approx_count_distinct`, sketches kept as bytes](#unique-visitors) |

| Job queue | `LPUSH`, `BRPOP`, `BLMOVE` | [a claim: `set … order … limit … returning`, `Fenec-Wait`](#queue) |

| Delayed jobs | `ZADD` by time, `ZRANGEBYSCORE` polled | [a `run_at` ahead, the claim woken at it](#delayed) |

| Streams, consumer groups | `XADD`, `XRANGE`, `XREADGROUP`, `XACK` | [a collection, `/_changes` and its consumers](#streams) |

| Pub/sub, keyspace notifications | `PUBLISH`, `SUBSCRIBE`, `notify-keyspace-events` | [a row with a short `@ttl`, a subscription](#notification) |

| Idempotency keys | `SET NX EX` on a request's id | [`Idempotency-Key`, or `@unique` and `if absent`](#idempotency) |

| Places near a point | `GEOADD`, `GEOSEARCH`, `GEODIST` | [a `geo` field, `@geo`](#geo) ([Points and distances](https://fenecdb.com/docs//docs/geo)) |

| Counts by the minute | `INCR` on a key a minute | [`bucket(at, 1m)` over the events](https://fenecdb.com/docs//docs/analytics#rollups) ([Analytics](https://fenecdb.com/docs//docs/analytics)) |

| A cache that evicts under memory pressure | `maxmemory`, `allkeys-lru`, `allkeys-lfu` | [not covered](#not-covered): a row lives until its `@ttl` |

| Scripts | `EVAL`, `FUNCTION` | [not covered](#not-covered): a `/batch` with `require` |

| Bitmaps, Bloom filters, Top-K | `SETBIT`, `BITCOUNT`, `BF.ADD`, `TOPK.ADD` | [not covered](#not-covered) |

What every recipe can count on:

- **Atomic.** There is one writer: a statement finds its rows and
    writes them under the write lock, and a `/batch` is one
    block — every statement in it lands, or none does, with no other write
    between them.

- **Durable by `--sync`.** Under `--sync
    always` a write is answered once it is on disk; under the
    default, `250`, a crash loses at most the last 250 ms of
    writes ([Durability](https://fenecdb.com/docs//docs/server#durability)).

- **On a replica.** A replica is sent each write as the primary
    wrote it — a counter's new value, never `n + 1` — once it
    is on the primary's disk, in the primary's order. It refuses writes
    (403), so increments, locks and claims go to the primary; a read sent
    there with `Fenec-After` waits for the write it names
    ([Replication](https://fenecdb.com/docs//docs/replication)). A row past its
    `@ttl` is out of a replica's reads as it is out of the
    primary's, and deleted when the primary's sweep is sent.

## Counter

An increment is a `set` that reads the field it writes.

```fenecql
create collection counters (name text @unique, n int)
put counters {name: "visits", n: 0}
set counters {n: n + 1} where name = "visits"     -- → 1
get counters select n where name = "visits"       -- → 1
set counters {n: n + 5} where name = "visits" returning n   -- → 6: INCRBY's answer
put counters {name: "signups", n: 1} if absent else set {n: n + 1}   -- → 1: made at 1
put counters {name: "signups", n: 1} if absent else set {n: n + 1}   -- → 1: added to
get counters select n where name = "signups"      -- → 2
```

- **None is lost.** Sixteen HTTP clients each incrementing one key
    10 000 times ended at 160 000, 102 000 increments a second against
    `fenec-server`; sixteen threads in process, 320 000 a
    second.

- **An increment costs** 1.36 us in process.

- **From nothing:** `coalesce(n, 0) + 1` counts a row
    with no `n` yet; `n + 1` over a `null`
    is `null`. A key that may not be there yet is made or added
    to by one statement, an upsert: `if absent else set`.

- **`returning`** answers the row as written, so the
    new value comes back with no second read.

In JavaScript, `inc()` writes the `coalesce` for
you:

```ts
import { connect, inc } from '@fenecdb/web/client';

const db = connect('http://127.0.0.1:8080');
await db.from('counters').where('name', 'visits').update({ n: inc(1) });
```

Python's `inc(1)`, Go's `fenecdb.Inc(1)`, .NET's
`Computed.Inc(1)`, and Swift's, Kotlin's and Dart's builders
render the same text.

## Rate limits

Three ways, each decided by what a write finds: a fixed window, the
cheapest; a sliding window, exact to the moment; a token bucket, which lets
a burst through up to its size and then a steady rate.

### Fixed window

A row a client and window, made if absent and expiring after the window;
the increment names the limit, so its answer is the decision.

```fenecql
create collection hits (key text @unique, n int, at timestamp @ttl(2m))
put hits {key: "ip1:29630000", n: 0, at: now()} if absent    -- → 1: the window's row
put hits {key: "ip1:29630000", n: 0, at: now()} if absent    -- → 0: there already
set hits {n: n + 1} where key = "ip1:29630000" and n < 2     -- → 1: allowed
set hits {n: n + 1} where key = "ip1:29630000" and n < 2     -- → 1: allowed
set hits {n: n + 1} where key = "ip1:29630000" and n < 2     -- → 0: over the limit
```

- **The window is in the key** — here the minute,
    `floor(now / 60000)`, which the client works out — so a new
    window is a new row, and `@ttl` lets the old ones go.

- **A request costs** its two statements, 3.5 us; send them as
    one [`/batch`](https://fenecdb.com/docs//docs/http#batches) for one round
    trip.

- **A fixed window lets a burst through at its edge:** the limit
    at the end of one minute and again at the start of the next. The two
    below do not.

### Sliding window

A row a request, expiring after the window, and a read that must find
fewer than the limit: `offset 2 limit 1 require 0` asks that no
third row is there, so it allows three a minute. Sent as one
`/batch` with the request's row, the read refuses the batch (412)
once three are, and the row is not written.

```fenecql
create collection calls (key text @hash, at timestamp @ttl(1m))
get calls where key = "ip1" offset 2 limit 1 require 0    -- → no row: allowed
put calls {key: "ip1", at: now()}                          -- → 1
get calls where key = "ip1" offset 2 limit 1 require 0    -- → no row: allowed
put calls {key: "ip1", at: now()}                          -- → 1
-- 30 s later
get calls where key = "ip1" offset 2 limit 1 require 0    -- → no row: allowed
put calls {key: "ip1", at: now()}                          -- → 1
get calls where key = "ip1" offset 2 limit 1 require 0    -- → refused: three in the last minute
-- 30 s later
get calls where key = "ip1" offset 2 limit 1 require 0    -- → no row: the first two are a minute old
```

- **Exact:** the rows of the last minute, counted the moment the
    request comes, since a row past its `@ttl` is out of the
    read at once. The guard and the write are one block under the write
    lock, so two requests cannot both take the last place.

- **A row a request** is what it keeps, as Redis's sorted set keeps
    a member a request; a limit of thousands a window is a fixed window or
    a bucket instead.

### Token bucket

A bucket holds up to two tokens and gains one a second; a request takes
one. The row keeps the tokens and when they were counted, and one
`set` works out what the time since has added — the two
timestamps apart are milliseconds — and takes a token only where there is
one:

```fenecql
create collection buckets (key text @unique, tokens float, at timestamp @ttl(1h))
put buckets {key: "ip1", tokens: 2.0, at: now()} if absent   -- → 1: a full bucket
set buckets {tokens: least(2.0, tokens + (now() - at) / 1000.0) - 1, at: now()} where key = "ip1" and tokens + (now() - at) / 1000.0 >= 1   -- → 1: allowed
set buckets {tokens: least(2.0, tokens + (now() - at) / 1000.0) - 1, at: now()} where key = "ip1" and tokens + (now() - at) / 1000.0 >= 1   -- → 1: allowed, empty now
set buckets {tokens: least(2.0, tokens + (now() - at) / 1000.0) - 1, at: now()} where key = "ip1" and tokens + (now() - at) / 1000.0 >= 1   -- → 0: refused
-- 1 s later
set buckets {tokens: least(2.0, tokens + (now() - at) / 1000.0) - 1, at: now()} where key = "ip1" and tokens + (now() - at) / 1000.0 >= 1   -- → 1: one token back
```

- **The script Redis needs** for this is the one statement: the
    row is read and written under the write lock, so no two requests take
    the same token.

- **A refused request writes nothing,** so the time it waited
    still counts toward the next token.

- **The `@ttl` is longer than a refill:** a bucket idle
    for an hour would be full again, so its row may go, and the next
    request makes it full.

From a client, the two statements go as one `/batch`, with
`require 1` on the take so that a refusal is a 412:

```ts
import { connect, FenecError } from '@fenecdb/web/client';

const db = connect('http://127.0.0.1:8080');

// true: serve the request; false: answer it 429.
async function allow(key: string): Promise<boolean> {
  try {
    await db.batch([
      ['put buckets {key: $1, tokens: 2.0, at: now()} if absent', [key]],
      ['set buckets {tokens: least(2.0, tokens + (now() - at) / 1000.0) - 1, at: now()} ' +
        'where key = $1 and tokens + (now() - at) / 1000.0 >= 1 require 1', [key]],
    ]);
    return true;
  } catch (e) {
    if (e instanceof FenecError && e.status === 412) return false;
    throw e;
  }
}
```

## Lock with expiry

A lock is a row: taken by the write that makes it, held while it lives,
and free again once its time passes, so a holder that dies cannot keep
it.

```fenecql
create collection locks (name text @unique, owner text, at timestamp @ttl(30s))
put locks {name: "nightly", owner: "w1", at: now()} if absent      -- → 1: w1 holds it
put locks {name: "nightly", owner: "w2", at: now()} if absent      -- → 0: w2 does not
set locks {at: now()} where name = "nightly" and owner = "w1"      -- → 1: renewed
set locks {at: now()} where name = "nightly" and owner = "w2"      -- → 0: not w2's
del locks where name = "nightly" and owner = "w1"                  -- → 1: released
```

- **Exactly one wins.** The write lock decides: of any number of
    clients racing, one is answered 1 and the rest 0 — sixteen threads are
    raced for it in `crates/fenec-core/tests/writes.rs`, and again
    once its time has passed.

- **Thirty seconds after its last renewal** the row is out of every
    read, the renewal's answer is 0 — the holder has lost it — and the next
    `put … if absent` takes it, the old row deleted in the same
    statement. The sweeper need not have come by.

- **The check-and-delete is the `del`:** it names the
    owner, so a holder whose lock lapsed and was taken by another deletes
    nothing — the script Redis's documentation gives for the release.

- **Costs:** refused while held 0.91 us, renewed 2.34, released and
    taken again 2.67.

Details

`if absent` passes over a document whose `@unique`
value or `id` a live row holds, and counts what it wrote: Redis's
`SET NX`. `insert` refuses the same document with a
`409` instead, which serves too, but makes losing the race an
error to catch and aborts a `/batch` it is part of. A lock named
by a number can be its `id`:
`put locks {id: 1, owner: $1, at: now()} if absent`.

Time is the server's clock, and a lock lasts its ttl from its last
renewal: renew well within it. As with any lock that expires, a holder
paused past its ttl may still believe it holds the lock; give the work it
guards a check of its own — a fencing token, below — where that
matters.

### Fencing tokens

A holder paused past its lease — a long garbage collection, a stalled
disk —
wakes believing it holds the lock, while another does. A fencing token
tells the two apart: a number the lock hands out, one more at every
acquisition, which the work it guards carries and the resource checks. Here
the lock's row keeps it, so the lock is a row that stays, and the lease is
its `until`:

```fenecql
create collection leases (name text @unique, owner text, until timestamp, fence int)
put leases {name: "report", fence: 0} if absent        -- → 1: the lock's row, made once
set leases {owner: "w1", until: now() + 30000, fence: fence + 1} where name = "report" and (owner is null or until <= now()) returning fence   -- → 1: w1 holds it, token 1
set leases {owner: "w2", until: now() + 30000, fence: fence + 1} where name = "report" and (owner is null or until <= now()) returning fence   -- → no row: held
-- 40 s later
set leases {owner: "w2", until: now() + 30000, fence: fence + 1} where name = "report" and (owner is null or until <= now()) returning fence   -- → 2: w1's lease lapsed
set leases {until: now() + 30000} where name = "report" and owner = "w2" and fence = 2 and until > now() require 1   -- → 1: renewed
```

The resource keeps the highest token it has been written with, and
takes a write only from that token or a newer one:

```fenecql
create collection reports (day text @unique, body text, fence int)
put reports {day: "2026-05-03", fence: 0} if absent    -- → 1
set reports {body: "by w2", fence: 2} where day = "2026-05-03" and fence <= 2 require 1   -- → 1: token 2 is the newest
set reports {body: "by w1", fence: 1} where day = "2026-05-03" and fence <= 1 require 1   -- → refused: w1 woke late, with token 1
set leases {owner: null, until: null} where name = "report" and owner = "w2" and fence = 2 require 1   -- → 1: released
```

- **The token is counted by the write that takes the lock,** under
    the write lock: no two acquisitions get one number, and none is
    skipped. Redis's `SET NX` hands out none: a counter beside
    the lock is a second key, kept in step with it by a script.

- **The row stays** so the count does: under `@ttl` it
    would be deleted when it lapsed, its count with it. A lapsed lease is
    `until <= now()`, which the next acquisition reads.

- **Waiting for it:** sent over HTTP with
    `Fenec-Wait`, an acquisition that takes nothing is held at
    the server until the holder releases it or its lease lapses, whichever
    comes first — both are a write or a time the server knows of (Holding a write until it can write).

- **A resource in fenecdb** checks the token in the write itself,
    as above. One elsewhere — a file store, another service — needs a
    check of its own; the token is a number it can keep.

## Optimistic concurrency

Read a row with its version; write it back only where the version is
still the one read, and move the version on in the same statement — what
Redis's `WATCH` and `MULTI` are for.

```fenecql
create collection docs (body text, version int)
put docs {id: 7, body: "draft", version: 1}
get docs select body, version where id = 7                                   -- → "draft"
set docs {body: "final", version: version + 1} where id = 7 and version = 1   -- → 1: written
set docs {body: "other", version: version + 1} where id = 7 and version = 1   -- → 0: read again
set docs {body: "other", version: version + 1} where id = 7 and version = 1 require 1   -- → refused: 412, and its batch put back
```

- **0 means someone wrote first:** read the row again and retry, or
    tell the user.

- **In a `/batch`, add `require 1`:** a 0
    would let the statements after it go on, where a lost race should stop
    them. With it the write that found another version is refused (412) and
    the batch is put back whole ([Writes that
    must find their rows](https://fenecdb.com/docs//docs/fenecql#require)). A `get … require 1` guards a
    batch on a value it does not write (A read that must find its
    rows).

- **Costs:** a write that wins 1.13 us, one that loses
    0.66.

## Session store

A session is a row that lives while it is used: `@ttl` on the
time it was last seen, a `@unique` token to find it by.

```fenecql
create collection sessions (token text @unique, user text, data json, seen timestamp @ttl(30m))
put sessions {token: "s-81f2", user: "ada", data: {"cart": 2}, seen: now()}
get sessions select user where token = "s-81f2"            -- → "ada"
set sessions {seen: now()} where token = "s-81f2"           -- → 1: seen again
set sessions {seen: now()} where token = "s-81f2" returning user, data.cart   -- → "ada": read and seen again, GETEX
del sessions where token = "s-81f2"                         -- → 1: signed out
```

- **An idle session is gone at once** for every read, thirty
    minutes after it was last seen, and the server's sweeper deletes it
    one to two minutes later ([Rows that
    expire](https://fenecdb.com/docs//docs/fenecql#ttl)).

- **Read and renewed in one** by `set … returning`: the
    row as written comes back, and a session already gone answers no
    row.

- **Costs,** over 10 000 sessions: a read 1.32 us, seen
    again 2.26.

## Cache with expiry

A value kept for a while in front of something slow to work out: read it
by its key, and on a miss work it out and keep it. The upsert writes a key
that is there and one that is not alike, and starts its time again.

```fenecql
create collection cache (key text @unique, value json, at timestamp @ttl(5m))
get cache select value.name where key = "user:7"      -- → no row: a miss
put cache {key: "user:7", value: {"name": "Ada"}, at: now()} if absent else set {value: new.value, at: now()}   -- → 1: kept for five minutes
get cache select value.name where key = "user:7"      -- → "Ada": a hit
put cache {key: "user:7", value: {"name": "Ada L."}, at: now()} if absent else set {value: new.value, at: now()}   -- → 1: written over
del cache where key = "user:7"                         -- → 1: invalidated
put cache {key: "user:7", value: {"name": "Ada"}, at: now()} if absent else set {value: new.value, at: now()}   -- → 1
-- 300 s later
get cache select value.name where key = "user:7"      -- → no row: five minutes old
```

What it is, and what it is not:

- **No eviction.** There is no LRU or LFU and no
    `maxmemory` policy: a row lives until its
    `@ttl` or a `del`, however rarely it is read.
    What the cache holds is what was written within its ttl. With
    `--max-memory`, a write past the ceiling is refused (507)
    rather than make room by dropping another row (Limits). A cache that must stay inside
    a memory budget by dropping what is used least wants Redis.

- **It survives a restart.** The rows are in the file, durable by
    `--sync`, so a server started again answers from a warm
    cache, and a replica holds it too.

- **The values are in the file, not the heap.** The file is mapped:
    the process holds the indexes derived from the rows, and the rows
    themselves are pages the system keeps or lets go — a 1 GB file of 2.3
    million rows with two indexes held 188 MB (The file is mapped).

- **Expired rows go in the background.** Out of every read at once,
    deleted by the server's sweeper one to two minutes later, 1 000 a
    block: 100 000 of 200 000 rows went in 0.56 s, each block holding the
    write lock 1.06 ms at the median and 2.3 at most (`make
    ttl-bench`). The space an overwrite leaves is taken back by the
    server's compaction once half the file is dead (Compaction).

- **Many clients missing one key** at once each work it out; to
    have one do it, the first takes the key's [lock](#lock)
    and the rest read again after.

## Leaderboard

A score field with `@sorted`: the top of the board is a walk
of the index from its end, and a player's rank is a count.

```fenecql
create collection scores (player text @unique, score int @sorted)
put scores [{player: "ada", score: 0}, {player: "bo", score: 90}, {player: "cy", score: 40}]
set scores {score: score + 120} where player = "ada"            -- → 1: ZINCRBY
get scores select player, score order score desc limit 10       -- → "ada": ZREVRANGE 0 9
get scores select player, score order score desc limit 10 offset 1   -- → "bo": the next page
get scores select score where player = "ada"                    -- → 120: ZSCORE
get scores where score > 120 count                              -- → 0: ada is first, ZREVRANK
get scores select player where score >= 40 and score <= 100 order score desc   -- → "bo": ZRANGEBYSCORE
```

- **A rank is a count** of the scores above the player's: read
    the score, then count, two statements — or one, where the score is
    known.

- **Ties** come out in the order the players were first written,
    their ids', whichever way the board is read; Redis orders them by the
    members' names.

- **Costs,** over 100 000 players: a score added 1.79 us,
    the top ten 1.52. A player's rank is a count of the scores
    above it, the index's range gathered: 1.5 ms at the median of
    the board.

## Sets and deduplication

A member is a row, and `@unique` over the set and the member
keeps it there once: `if absent` adds it and answers whether it
was new.

```fenecql
create collection members (list text @hash, member text, @unique(list, member))
put members {list: "tags:7", member: "rust"} if absent      -- → 1: SADD, new
put members {list: "tags:7", member: "rust"} if absent      -- → 0: there already
put members [{list: "tags:7", member: "db"}, {list: "tags:8", member: "rust"}] if absent   -- → 2
get members where list = "tags:7" and member = "rust" count   -- → 1: SISMEMBER
get members where list = "tags:7" count                       -- → 2: SCARD
get members select member where list = "tags:7" and member in (get members select member where list = "tags:8")   -- → "rust": SINTER
del members where list = "tags:7" and member = "db"           -- → 1: SREM
```

- **A set's members** are its `@hash` bucket, read
    without the others: `SMEMBERS` is `get members select
    member where list = $1`.

- **Seen-it sets** — message ids, a feed's items already shown —
    are a collection of one `@unique` key and a
    `@ttl`, so an id is remembered for as long as a repeat
    can come.

- **Exact,** at a row a member: a Bloom filter's fixed memory and
    its false positives are [not here](#not-covered).

## Unique visitors

`approx_count_distinct` is HyperLogLog with 214
registers — Redis's register count, so its standard error, 0.81% —
counted over the visits as they were written. `count(distinct)`
is the exact count, up to a million values a query.

```fenecql
create collection visits (user text, page text @hash, at timestamp @sorted)
put visits [{user: "ada", page: "/", at: "2026-05-03T09:00:00Z"}, {user: "bo", page: "/", at: "2026-05-03T09:05:00Z"}, {user: "ada", page: "/docs", at: "2026-05-03T10:00:00Z"}, {user: "cy", page: "/", at: "2026-05-04T08:00:00Z"}]   -- → 4: PFADD, a row a visit
get visits select approx_count_distinct(user) as users                   -- → 3: PFCOUNT
get visits select approx_count_distinct(user) as users, bucket(at, 1d) as day group day   -- → 2: on the 3rd
get visits select count(distinct user) as users where page = "/"         -- → 3: exact
```

A day's sketch kept in a row is Redis's key a day: read out as bytes
with `hll_accumulate`, put back as a client sends rows, and
merged — `PFMERGE` — by `hll_combine`, each visitor
of two days counted once:

```fenecql
create collection daily (day timestamp @unique, users bytes)
get visits select bucket(at, 1d) as day, hll_accumulate(user) as users group day   -- → "2026-05-03T00:00:00Z"
put daily $1                                                                       -- → 2: the rows above
get daily select hll_estimate(hll_combine(users)) as users                        -- → 3: both days, ada once
get daily select hll_estimate(users) as users where day = "2026-05-03"            -- → 2
```

- **Costs:** per hour over a million events, 121 ms against
    `count(distinct)`'s 128, the furthest of the 169 hours
    1.3% off (`make analytics-bench`).

- **Sketches are the same bytes** wherever they are made, the
    browser's or a server's, and a group of few values keeps few words
    rather than 16 KB ([Distinct
    users](https://fenecdb.com/docs//docs/analytics#distinct)).

## Job queue

A job is a row. A worker claims a batch with one statement, which finds
the oldest ready jobs and makes each its own for a lease, and acks a job by
deleting it.

```fenecql
create collection jobs (payload json, run_at timestamp @sorted, owner text, attempts int)
put jobs [{payload: {"to": "ada"}, run_at: now(), attempts: 0}, {payload: {"to": "bo"}, run_at: now(), attempts: 0}]
set jobs {owner: "w1", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 10 returning id   -- → 1: both, w1's for 30 s
del jobs where id = 1 and owner = "w1" require 1                       -- → 1: done
```

- **None is taken twice:** the claim finds its jobs and writes them
    under the write lock. A job whose worker died is ready again once its
    lease lapses, and the next claim takes it — at least once, where
    `BRPOP` loses a job its worker dies holding.

- **Costs,** over a million jobs: a claim of ten 18.7 us; sixteen
    threads claimed and acked them all in 5.4 s, 185 800 jobs a second,
    and sixteen HTTP clients at 138 500 (`make
    queue-bench`).

- **`BLPOP`:** sent over HTTP with
    `Fenec-Wait`, a claim that finds no job is held at the
    server until one is enqueued, a delayed one comes due or a lease
    lapses, and handed to the worker that has waited longest: a job
    enqueued reached a waiting worker in 0.38 ms at the median (Waiting for a job).

- **Retries with a backoff, dead letters, delays and priorities**
    are on their own page, [Job queues](https://fenecdb.com/docs//docs/queues).

### Delayed jobs

A job delayed is one whose `run_at` is ahead: no claim takes
it before then, and the claim that would take it is the same one.

```fenecql
create collection reminders (payload json, run_at timestamp @sorted, owner text, attempts int)
put reminders {payload: {"to": "ada"}, run_at: now() + 60000, attempts: 0}   -- → 1: due in a minute
set reminders {owner: "w1", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 10 returning id   -- → no row: not yet
-- 60 s later
set reminders {owner: "w1", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 10 returning id   -- → 1: due
```

- **No poll:** a claim held with `Fenec-Wait` is woken
    at the moment the next job comes due, which the server works out from
    the `@sorted` index: 0.04 ms after its time at the median
    and 1.9 at p99, against 1.2 and 8.4 polling every 10 ms (Waiting for a job).

- **No ceiling** on the delay, and a delayed job is read like any
    row: `get reminders where run_at > now() count`.

## Streams

A stream is a collection written to and never written over: each entry a
row, its `id` counted up as it is added. A range is a walk of
the ids from the last one read, and old entries go by `@ttl`.

```fenecql
create collection activity (kind text @hash, user text, at timestamp @ttl(7d))
put activity {kind: "signup", user: "ada", at: now()}                                         -- → 1: XADD
put activity [{kind: "order", user: "ada", at: now()}, {kind: "order", user: "bo", at: now()}]   -- → 2
get activity select id, kind where id > 1 limit 100      -- → 2: XRANGE, after entry 1
get activity where kind = "order" count                  -- → 2
```

A reader that must see every write once, and pick up where it left off,
reads the server's change stream: every write on disk, a JSON line each.
Started with `--cdc`, a named consumer keeps its place on the
server, as a consumer group's last delivered id does:

```bash
fenec-server --file data.fenec --http 127.0.0.1:8080 --cdc

curl -X POST http://127.0.0.1:8080/_changes/consumers/mailer                  # XGROUP CREATE … $
curl -i 'http://127.0.0.1:8080/_changes?consumer=mailer&wait=10000'          # XREADGROUP … BLOCK 10000
# Fenec-Next: 42 -- handle the lines, then XACK up to there:
curl -X POST http://127.0.0.1:8080/_changes/consumers/mailer -d '{"since": 42}'
```

- **At least once** where the server keeps the place: a consumer
    that crashes between reading and committing is handed those writes
    again ([Consumers](https://fenecdb.com/docs//docs/http#consumers)). One that keeps
    `Fenec-Next` beside its own state, committed with it, has
    each write once ([Change data
    capture](https://fenecdb.com/docs//docs/http#changes)).

- **Only what is on disk:** a write is handed out once an fsync
    has covered it, so none a crash lost reaches a consumer.

- **Every collection's writes** are in it, each line naming its
    collection, puts, deletes and schema changes alike; a consumer passes
    over the rest.

- **One consumer, every write.** A consumer group whose consumers
    share the entries — each to one of them, the pending ones claimed
    again after a while — is a [job queue](#queue): a claim and
    its lease.

- **How far back:** as many writes as
    `--replication-buffer` holds, 64 MiB unless given; a cursor
    further back is answered 410 with the first `since` it
    does serve, never with writes missing from the middle.

## Pub/sub and change notification

A message is a row, kept for as long as a late subscriber should still
see it, and a subscription is sent the rows of a shape — first those there
are, then each change as it lands.

```fenecql
create collection messages (room text @hash, body text, at timestamp @ttl(1m))
put messages {room: "lobby", body: "hello", at: now()}     -- → 1: PUBLISH
```

```bash
# SUBSCRIBE: the rows of a shape, then each change to them as it lands
curl -N 'http://127.0.0.1:8080/messages/changes?room=eq.lobby'

# every write on disk, from where a named consumer left off
curl 'http://127.0.0.1:8080/_changes?consumer=mailer&wait=10000'
```

```ts
import { connect } from '@fenecdb/web/client';

const db = connect('http://127.0.0.1:8080');
const stop = db.subscribe('messages', { room: 'eq.lobby' }, (ev) => {
  if (ev.type === 'change') for (const m of ev.puts) console.log(m.body);
});
await db.run('put messages {room: $1, body: $2, at: now()}', ['lobby', 'hi']);
stop();
```

- **A subscription** ([Subscriptions](https://fenecdb.com/docs//docs/http#subscriptions))
    is woken by the write, not by polling: from a write sent to its event read
    by a subscriber took 0.037 ms at the median and
    0.103 at p99 over HTTP on one machine.

- **A late subscriber** is sent the messages still inside their
    `@ttl` first, where Redis's pub/sub hands it only what is
    published after it subscribed.

- **Fan-out costs the writer.** Every write wakes every stream,
    which reads what it changed: a write's p50 went from 0.26 ms with no
    stream to 0.94 with 60 and 3.1 with 500, each stream a connection and
    a thread, 64 at most unless `--http-max-streams` says
    otherwise. A page thousands view polls instead (Polling a read).

- **A live query** keeps a query's answer rather than a shape's
    rows: `db.live(query, cb)` runs it again on the server
    after every write to what it reads (JavaScript).

- **The change stream** ([Change data
    capture](https://fenecdb.com/docs//docs/http#changes)) hands out every write on disk exactly once to a consumer
    that sends back where it is, which a subscription — a shape's rows kept
    up to date — does not promise: `notify-keyspace-events`
    with a cursor.

## Idempotency keys

A client that sends a write again after a timeout, not knowing whether
the first landed, sends it with the same `Idempotency-Key`: the
server keeps the key and the answer in the same block as the write, and a
repeat is handed the first answer.

```bash
curl -X POST http://127.0.0.1:8080/query \
  -H 'Idempotency-Key: 5f0c2d6e-order-1142' \
  -d '{"query":"put orders {customer: $1, total: $2}","params":[7, 129.5]}'
# sent again with the key: the first answer, Idempotent-Replayed: true, nothing written
```

- **The key and the write land together,** so a crash cannot leave
    one without the other; a failed write keeps no key, and the key with
    another request is refused (422). Keys are kept a day unless
    `--idempotency-ttl` says otherwise (Retrying a write).

- **Costs:** 16 200 single puts a second over HTTP with a key,
    against 18 300 without.

A message that arrives more than once — a webhook redelivered, an event
replayed — is told apart by its own id: a row of it, made if absent, and
the work done only where it was.

```fenecql
create collection deliveries (event text @unique, at timestamp @ttl(1d))
put deliveries {event: "evt_1J8", at: now()} if absent             -- → 1: first time, handle it
put deliveries {event: "evt_1J8", at: now()} if absent             -- → 0: a repeat, already handled
put deliveries {event: "evt_1J8", at: now()} if absent require 1   -- → refused: in a /batch, its work put back with it
```

- **With `require 1` in a `/batch`** beside
    the writes it guards, the mark and the work land together or not at
    all: a repeat is refused (412) and writes nothing, and a crash cannot
    leave work done with no mark.

## Places near a point

`GEOADD` and `GEOSEARCH` are a `geo`
field with `@geo`: the metres between two points, the rows
within a radius or a box, the nearest first — see
[Points and distances](https://fenecdb.com/docs//docs/geo).

```fenecql
create collection sicily (name text, loc geo @geo)
put sicily [{name: "Palermo", loc: [13.361389, 38.115556]}, {name: "Catania", loc: [15.087269, 37.502669]}]   -- → 2
get sicily select name where distance(loc, [15, 37]) <= 200000 near loc [15, 37] limit 10   -- → "Catania"
```

- **Costs,** over a million points: every point within 1 km 12.6
    us, against 34.3 us for Redis's `GEOSEARCH`, and the ten
    nearest 33 us, against 198 ms — the server's own time for Redis, the
    network left out (`make geo-bench`, What it costs).

## What fenecdb does not do

- **Evict.** No LRU or LFU, no `maxmemory` policy: a row
    lives until its `@ttl` or a `del`, and
    `--max-memory` refuses a write past it (507) rather than
    drop another row ([Cache with expiry](#cache)). A cache that
    must stay inside a memory budget by forgetting what is used least
    wants Redis.

- **Keep its data in memory alone.** A server always keeps a file,
    and a write is as durable as `--sync` says: the default,
    `250`, answers before the fsync and loses at most the last
    250 ms in a crash; `always` answers after it, a durable
    update 4.0 ms at the median on a Mac (`F_FULLFSYNC`) and
    0.325 ms on Linux in Docker (One request's round trip);
    `off` syncs at shutdown alone (Durability). A request is HTTP: a
    read by id took 0.022 ms at the median on one machine (A request over HTTP), and a
    network's round trip adds to it. Memory alone, a function call a
    statement, is the engine embedded in the app — in Rust, where an
    increment costs 1.36 us, or in the page (Embedded Rust, JavaScript).

- **Run scripts.** No Lua, no `EVAL` or
    `FUNCTION`. What a script checks before it writes is
    `require` on the statements of a `/batch`, which
    lands whole or not at all, and a value worked out over the row is a
    `set`'s expression or `case`. A batch cannot read
    a value and branch on it: a decision between two reads is the
    client's, a round trip each.

- **Write from more than one writer.** A file has one: writes run
    one after another under its lock — 320 000 increments a second in
    process and 102 000 over HTTP from sixteen clients (`make
    counters-bench`), 185 800 jobs claimed and acked a second
    (`make queue-bench`) — and reads go beside them: four
    threads read 701 000 rows a second by id beside four writers (Writers and readers at
    once). Past one machine's writer the data is split by tenant, a
    file each ([Tenants and sharding](https://fenecdb.com/docs//docs/sharding)); there
    are no hash slots.

- **Fan out across tenants.** A subscription and
    `/_changes` are one file's: on a tenant node each tenant is
    a file of its own, and the router forwards a stream to the node
    holding it. There is no channel every tenant's subscribers hear; a
    message for all of them is a write to each.

- **Pub/sub without storage.** Every message is a write that lands
    in the file and moves the change counter; there is no fire-and-forget
    channel. A row with a short `@ttl` and a subscription is
    the nearest ([above](#notification)).

- **Speak RESP.** A Redis client cannot connect: the server speaks
    HTTP, and every language reaches it through it or an SDK (Your language).

- **Bitmaps, Bloom filters and Top-K.** No `SETBIT` or
    `BITCOUNT`, no Bloom or Cuckoo filter, Count-Min sketch or
    Top-K. Membership is exact, a row a member (Sets); HyperLogLog is there (Unique visitors).

- **Deliver exactly once.** A queue's lease that lapses under a
    slow worker hands its job to another, and a consumer that crashes
    before it commits reads again: the work's own check — a
    `@unique` key, an `Idempotency-Key` — makes the
    second run harmless.


---

Source: https://fenecdb.com/docs/queues

# Job queues

Many workers taking jobs from one collection: each job
claimed by one of them under a lease, then acked, extended or failed —
every step a statement.

One statement claims. A `set` with `order` and
`limit` writes the rows a `get` with the same clauses
would answer, and `returning` hands them back as written:

```text
set jobs {owner: $1, run_at: now() + 30000, attempts: attempts + 1}
  where run_at <= now() order run_at limit 10 returning *
```

The ten oldest ready jobs are found and written under the one writer's
lock, so two workers never take the same one — what PostgreSQL's
`FOR UPDATE SKIP LOCKED` is for, with no locks to skip. A job's
`run_at` is the time it may next run: when it was enqueued, or
later for a delayed one, and while a worker holds it, the end of its lease.
So a worker that dies holds nothing for long: its lease lapses, the job is
ready again, and the next claim takes it. See
[Writes that pick their rows](https://fenecdb.com/docs//docs/fenecql#pick) for the
rules.

This is Redis's `LPUSH` and `BRPOP`, with a lease
so that a job outlives the worker that took it. The other jobs an app adds
Redis for — locks, rate limits, caches, streams, pub/sub — are recipes of
[Instead of Redis](https://fenecdb.com/docs//docs/redis).

Every FenecQL block on this page is run, statement by statement, by a
test (`crates/fenec-core/tests/queues_docs.rs`), and each
`-- → x` is the answer it checks: a write's count, or the first
value of the first row it answered. A line `-- 20 s later` moves
the test's clock. The costs are `make queue-bench` on an M1 Air,
over a million jobs.

## The queue

A collection a queue. `run_at` takes `@sorted`, so
a claim walks the ready jobs from the oldest and stops at its page.

```fenecql
create collection jobs (payload json, run_at timestamp @sorted, owner text, attempts int, error text)
put jobs {payload: {"to": "ada@example.com"}, run_at: now(), attempts: 0}
put jobs [{payload: {"to": "bo@example.com"}, run_at: now(), attempts: 0}, {payload: {"to": "cy@example.com"}, run_at: now(), attempts: 0}]
put jobs {payload: {"to": "di@example.com"}, run_at: now() + 3600000, attempts: 0}     -- → 1: delayed an hour
```

- **A delayed job** is one whose `run_at` is ahead: no
    claim takes it before then.

- **Jobs enqueued at the same time** come out in the order they
    were written, their ids'.

- **Several queues** are several collections. Kept in one, a
    `queue` field goes without `@hash`: the walk tests
    it on each job it passes, where through a hash index a claim would read
    the whole queue to put it in order.

## Claim

Each worker claims a batch, the oldest ready jobs, each its own for 30
seconds:

```fenecql
set jobs {owner: "w1", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 2 returning id, payload.to   -- → 1: jobs 1 and 2
set jobs {owner: "w2", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 2 returning id, payload.to   -- → 3: the one left ready
set jobs {owner: "w3", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 2 returning id   -- → no row: none ready
```

- **None is taken twice.** The claim finds and writes its jobs
    under the write lock. Sixteen threads claiming batches of ten from
    6 000 jobs, and eight HTTP clients from 2 000, never handed a job to
    two of them, and every job was acked
    (`crates/fenec-core/tests/claims.rs`,
    `crates/fenec-http/tests/claims.rs`).

- **`returning *`** answers every field; a list answers
    those, paths into a `json` field among them. Without
    `returning` the answer is the count.

- **`limit 1 require 1`** is one job or a refusal (412):
    `require` counts the rows the claim answers.

- **An empty queue answers no rows** at once — or, sent over HTTP
    with `Fenec-Wait`, holds the claim at the server until a job
    comes ([Waiting for a job](#wait)).

## Waiting for a job

A worker whose claim finds nothing need not sleep and claim again. Sent
with `Fenec-Wait`, milliseconds, the claim is held at the server
while it would take nothing, and answered the moment it takes a job — one
enqueued, a delayed one come due, a lease lapsed — or at the end of the
wait with no row: Redis's `BLPOP`, SQS's long poll.

```bash
curl -X POST http://127.0.0.1:8080/query -H 'Fenec-Wait: 30000' \
  -d '{"query": "set jobs {owner: $1, run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 10 returning *", "params": ["w1:1842"]}'
```

- **A job enqueued** reaches a waiting worker in 0.38 ms at the
    median and 0.76 at p99, where a worker polling every 10 ms took 0.83
    and 10.1 and every 100 ms 15.5 and 85.8 — and polling every 10 ms sent
    ten times the claims (costs).

- **Time wakes it too.** No write announces a delayed job's
    `run_at` or a lease's end, so the server works the moment
    out from the claim's filter: `run_at <= now()` changes
    its answer only as the time passes a job's `run_at`, and the
    next one is a step of the `@sorted` index past now. The
    claim is held until that moment and runs then, never before.

- **One job, many workers:** it goes to the worker that has waited
    longest, and the others stay held. Claims that would take the same
    jobs — the same filter, scope and parameters in it, whatever they
    write — wait in line, and the line looks once whether a job is there;
    the first runs, and having taken a job hands the turn to the next,
    since more may be ready. A write to the queue costs a line one look
    under the read lock, not a claim a worker.

- **Claims of their own, one job:** it goes to the claim whose own
    order puts it first, and on a tie to the one that has waited longest.
    A courier's claim is a line of its own -- its point is in its filter,
    its token's subject in its rules -- so every courier's claim looks
    when an order is placed, all of them by one held request under one
    read lock, each among the orders placed since it last looked; each
    that finds a row bids for it, and once every claim that could beat the
    best bid has looked, the row is the best's, its claim alone is woken,
    and the rest wait behind its run ([Who gets the
    job](#fairness)).

- **At most 30 seconds**, and the answer is then what the claim
    answers on an empty queue: `[]`, or a
    `require`'s 412. A worker sends the next.

- **A `set` or a `del`**, alone or in a
    `/batch` beside reads: a `put` writes every time,
    and a request holding one is refused (400), as is a filter that reads
    the time other than as a field compared with `now()` give or
    take a number — `bucket(now(), 60000)` — since nothing says
    when its answer changes.

### Who gets the job

A job every waiting claim would take goes to one claim, by the claim's
own order:

- **A claim ordering by a field** -- `order run_at`, a
    queue's -- ranks a job as every other claim does, so the tie goes to
    the claim that has waited longest: first come, first served across
    claims as within one line, as Redis serves `BLPOP`'s
    clients. A burst of jobs goes to the waiting claims in the order they
    came, the oldest job to the claim held longest, each claim looking for
    the next once the run before it is done.

- **A claim ordering by its own point** -- a courier's
    `order distance(pickup, $1) limit 1`
    ([The nearest job](#nearest)) -- ranks a job by how far it
    is from the courier, so of the couriers waiting the nearest takes it,
    the others looking again once its run is done, each for the job nearest
    it then. That needs every waiting claim's look, which costs what was
    written rather than what each claim would read: one order at a time
    to couriers waiting in a city 10 km across, the nearest took every
    one, run for once, 0.18 ms from the put at 10 couriers, 0.31-0.32 ms
    at 100 and 1.7 ms at 1 000, the server's CPU 0.2, 0.4-0.5 and 3.1-3.2
    ms an order, against a raced 0.2-0.4 ms to a courier a median of
    3.3-5 km further, two runs an order (`make queue-bench`'s
    `held nearest`, in turns).

- **No header turns it on.** The claim's order says what "first"
    means, and a tie keeps the order claims came in.

Details

Raced, which it was before, an order placed woke every courier's claim
and whichever ran first took it: of ten couriers waiting within reach the
nearest was handed it one time in ten, of a hundred one in a hundred, and
every claim that found the order ran under the write lock for nothing. One
job every 2 ms at least to claims of their own that would all take it,
raced in brackets (`make queue-bench`'s `held
contested`, in turns):

| Claims waiting | Held longest took it | Runs a job | Put to taken, p50 | Server CPU a job |

| 10 | 100% (4-6%) | 1 (5.2-5.3) | 0.30-0.39 ms (0.37-0.43) | 0.7-0.9 ms (1.8-2.1) |

| 100 | 100% (2%) | 1 (5.2-5.5) | 0.75-0.80 ms (0.57-0.68) | 1.8-2.3 ms (11-14) |

| 1 000 | 100% (3%) | 1 (4.7-5.2) | 2.4 ms (2.6-5.2) | 5.9 ms (138-144) |

Each claim looking for itself, a job placed beside a thousand claims
of their own woke a thousand threads, each running its whole pick under
the read lock: 0.38-0.49, 0.86-0.91 and 10-12.5 ms from the put at 10,
100 and 1 000 claims, the server's CPU 1.9-2.3, 12-13 and 110-126 ms a
job; over 2 000 leased orders a courier's pick took 80-86 µs, the point
index walked past every one, and an order took 20 ms to reach the nearest
of 1 000 couriers and 175 ms of the server's CPU. Now one held request looks
for every claim under one read lock and wakes only the claim whose bid
won, and a claim whose last look found nothing knows no job it could take
is there but those written since, until the time reaches its next moment:
it tests those alone, with its own filter and in its own order, a
microsecond a claim, the job its whole pick would have found -- held so
over 3 000 generated rounds of claims, tokens' rules and writes. A claim
where that cannot be shown -- an inner `get`, rows that
expire, a schema changed, more than 32 rows written since -- runs its
whole pick, under the same lock. A thousand claims held also take less
memory, 46 KB each against 63, their threads no longer each running a
pick.

A tie settles as soon as the claim held longest has looked: no claim
held after it could take the job before it. That look, made with every
other claim's, is what the job waits for, where raced it went to the
fastest of a thousand claims woken at once; the one run a job, against
five under the write lock, is every other writer's. A row a claim ranking
by its own point may bid for waits for every claim's look. A single
line of workers -- the same claim -- is served as before: a job enqueued
reached one of eight in 0.35 ms either way.

Details

A held claim is the same request run at a later moment: the first
moment it would write something, or the end of the wait.

- **When it runs again.** A write to a collection it reads has its
    line look under the read lock whether a job is there to take — the
    claim's own filter as a `get` of one row — and the line's
    first claim is run only if one is. A write of another collection
    wakes none: woken by every write, 150 claims held on
    `orders`, each a line of its own, took a put of
    `positions` beside them from 22 µs at the median to 50-95,
    and at p99 from 30-90 to 200-480 µs; now it takes 22-25 and 30-97,
    as beside none. A look that found nothing is
    not repeated for a millisecond, however many writes land meanwhile:
    beside sixteen claims held on a queue that stayed empty, sixteen busy
    workers took 148 000 jobs a second against 150 600 with none held, the
    held ones looking 730 times a second; woken at every write they looked
    3 400 times and the workers took 142 000. A job enqueued in that
    millisecond is taken at most that much later. The time wakes
    it at the next row's moment: each comparison of a field with
    `now()` plus or minus a number, wherever it is in the
    filter, and a `@ttl` field's expiry; every row's value
    counts, so one queue among several in a collection may be woken by
    another's delayed jobs, to find nothing and sleep to the next.

- **A delayed job** was held by a waiting worker 0.04 ms after its
    time at the median and 1.9 at p99, against 1.2 and 8.4 polling every
    10 ms; with nothing due and nothing written, a held claim does
    nothing at all: a thousand held for ten seconds took no measurable
    CPU, where a thousand polling every 100 ms took 86% of a core, and a
    hundred 11%.

- **Each held claim is a connection** and its thread, as a
    subscription is: 46 KB of the server's memory each at a thousand,
    against a subscription's 45. They count against
    `--max-connections`, which a server with many waiting
    workers raises. On a tenant node a tenant's held claims are its
    `held` quota, apart from its `streams` -- a
    quarter of the node's connections unless set
    ([quotas](https://fenecdb.com/docs/sharding#quotas)) -- and a claim costs its
    tenant's rate of rows written only when a run writes: one held while
    the tenant is past its rate stays held, and runs once a row's worth is
    back.

- **Under an `Idempotency-Key`** a claim that waited and
    then took jobs keeps them under the key: sent again, it is answered
    those rows and takes none. While it takes nothing it keeps no key, so
    a retry after a timeout waits again.

- **A worker's token** waits as it claims: what a look reads is the
    claim as the token runs it, its rules ANDed in, so it wakes for jobs
    it could take and no others, and each run asks the token again — one
    that expired meanwhile is refused (401).

- **A client gone** before a job came hands the job to the next
    waiting worker, rather than leasing it to no one — through the router
    too, which passes its client's end on to the node.

- **Tenants and the router.** Under `/t/<t>/` a
    held claim holds nothing of its tenant between its runs, so a move
    goes on; a tenant moved away or deleted answers its held claims 503
    with `Retry-After`, to come back through the router, which
    waits for a held request its wait longer than
    `--upstream-timeout`. A claim held on a node the router
    fails over is answered then: under an `Idempotency-Key` it
    is held again on the tenant's new primary for what is left of its
    wait; without one it is a `502`, since the node may have run
    it as it died, and sent again it would take a second job
    ([Requests out to a node failed
    over](https://fenecdb.com/docs/sharding#in-flight)).

- **Shutdown.** At SIGTERM every held claim is answered at once,
    with the nothing it found, before the server syncs and exits.

- **A replica** refuses a claim as it refuses every write (403),
    held or not.

- **With `Fenec-After`** the header is that read's wait,
    as before, and holds no write.

- **Not in a page or an app's library.** A database in the page,
    or one an app opens through the native library, has no other worker to
    wait for, and the builders refuse `wait` there: an app that
    queues work for itself knows when it enqueues, and a live query of the
    queue runs again at every write.

## Ack, extend

A worker acks a job it holds by deleting it, and keeps one longer by
moving its lease on. Each names the owner it was claimed by and requires
its row, so a worker whose lease lapsed, and whose job another worker
claimed meanwhile, finds out:

```fenecql
del jobs where id = 1 and owner = "w1" require 1                       -- → 1: done
-- 20 s later
set jobs {run_at: now() + 30000} where id = 2 and owner = "w1" require 1   -- → 1: 30 s more
-- 15 s later
set jobs {owner: "w3", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 10 returning id, attempts   -- → 3: w2's lease lapsed
del jobs where id = 3 and owner = "w2" require 1                       -- → refused: 412, w3 holds it now
del jobs where id = 3 and owner = "w3" require 1                       -- → 1
```

- **At least once.** A job whose worker died is taken again once
    its lease lapses, so a job can run twice — once by a worker that was
    only slow. Work that must not happen twice keeps a check of its own:
    a `@unique` key on what it writes, or an
    [`Idempotency-Key`](https://fenecdb.com/docs//docs/http#idempotency) on
    the request it sends.

- **A token a claim** rather than the worker's name as the owner —
    `"w1:1842"` — also tells apart two claims of one job by the
    same worker.

- **To keep a job done** rather than delete it, give it no ready
    time: `set jobs {run_at: null, owner: null} where id = $1 and
    owner = $2 require 1`. No claim takes a job without one.

## Fail, back off, dead-letter

A job that failed goes back with a backoff that grows with its attempts,
and after the fifth it has no ready time — a dead letter, which no claim
takes and a read finds:

```fenecql
set jobs {owner: null, error: "smtp timeout", run_at: case when attempts >= 5 then null else now() + 1000 * attempts * attempts end} where id = 2 and owner = "w1" require 1   -- → 1: again in 1 s
-- 2 s later
set jobs {owner: "w1", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order run_at limit 1 returning id, attempts   -- → 2: its second attempt
set jobs {attempts: 5} where id = 2                         -- three failures on
set jobs {owner: null, error: "smtp timeout", run_at: case when attempts >= 5 then null else now() + 1000 * attempts * attempts end} where id = 2 and owner = "w1" require 1   -- → 1: the fifth: dead
get jobs select id, error where run_at is null              -- → 2: the dead letters
set jobs {run_at: now(), attempts: 0, error: null} where id = 2 and run_at is null   -- → 1: sent again by hand
```

- **The backoff here is the square** of the attempts in seconds —
    1, 4, 9, 16 — worked out over the row. One that doubles, or has jitter,
    is the worker's to send: it read `attempts` in its claim.

- **A worker that dies on a job** never fails it, and its lease
    lapses instead: `attempts` counts claims, so a job that comes
    back to a worker with `attempts` past five is failed by it,
    not run again — SQS's receive count.

## Priorities

With a priority, a claim takes the highest first, and among jobs of one
priority those written first:

```fenecql
create collection tasks (payload json, run_at timestamp @sorted, priority int @sorted, owner text, attempts int)
put tasks [{payload: {"n": 1}, run_at: now(), priority: 0, attempts: 0}, {payload: {"n": 2}, run_at: now(), priority: 9, attempts: 0}]
set tasks {owner: "w1", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order priority desc limit 1 returning payload.n   -- → 2: the higher priority
```

- **The walk is the priority's index**, testing each job's ready
    time as it passes: jobs of a high priority that are leased or delayed
    are passed over at every claim, so a queue holding many of them is a
    collection of its own. While the ready jobs number a few thousand at
    most, the claim reads them by `run_at` and puts them in
    order instead.

## The nearest job

Where a job has a place and a worker one of its own — a pickup and a
courier, a ride and a driver — the claim takes the nearest ready job, in
the same one statement: a key of `order` may be an expression,
and the distance from a point is walked to the page by a `@geo`
index ([The nearest job](https://fenecdb.com/docs//docs/geo#dispatch)):

```fenecql
create collection rides (pickup geo @geo, run_at timestamp @sorted, owner text, attempts int)
put rides [{pickup: [29.0100, 41.0400], run_at: now(), attempts: 0}, {pickup: [29.0300, 41.0500], run_at: now(), attempts: 0}]
set rides {owner: "w1", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order distance(pickup, [29.0290, 41.0490]) limit 1 returning id   -- → 2: the nearer
set rides {owner: "w2", run_at: now() + 30000, attempts: attempts + 1} where run_at <= now() order distance(pickup, [29.0290, 41.0490]) limit 1 returning id   -- → 1: the one left
```

- **Any expression orders the claim** — `order priority * 10 -
    attempts desc`, `order case when vip then 0 else 1 end,
    run_at` — every ready job's key worked out and the page put in
    order. Only the distance from a point over `@geo` and an
    `@sorted` field are walked to the page.

- **The nearest worker to each job, where they wait.** A claim
    takes the job nearest the worker that asks; claims held at the server
    for a job (`Fenec-Wait`) hand a job placed to the waiting
    worker nearest it, each claim's own distance its rank
    ([Who gets the job](#fairness)).

## A pop

Where a job may be lost but never run twice — Redis's
`LPOP` — a `del` takes and answers it at once:

```fenecql
del tasks where run_at <= now() order priority desc limit 10 returning id, payload   -- → 1: taken and gone
```

## A worker

The loop, in JavaScript over HTTP: claim a batch — held at the server
until there is one — run each job, and ack it or fail it.

```ts
import { connect, expr, inc, raw } from '@fenecdb/web/client';

const db = connect('http://127.0.0.1:8080', { token: 'a worker token' });
const jobs = db.from('jobs');
const me = `w-${crypto.randomUUID()}`;

async function send(to: unknown): Promise<void> {
  // the work
}

for (;;) {
  const owner = `${me}:${Date.now()}`;
  const claimed = await jobs
    .where(raw('run_at <= now()'))
    .order('run_at')
    .limit(10)
    .update(
      { owner, run_at: expr('now() + ?', 30000), attempts: inc(1) },
      { returning: true, wait: 30000 },
    );
  for (const job of claimed) {
    try {
      await send((job.payload as { to: string }).to);
      await jobs.where('id', job.id).where('owner', owner).delete({ require: 1 });
    } catch {
      const backoff = 1000 * 2 ** Number(job.attempts);
      await jobs.where('id', job.id).where('owner', owner).update(
        { owner: null, run_at: expr('now() + ?', backoff) },
        { require: 1 },
      ).catch(() => {});
    }
  }
}
```

- **Every builder writes the claim** — Python's
    `.update(patch, returning=True)`, Go's
    `Update(ctx, patch, fenecdb.Returning("*"))` and its
    `Result.Rows`, .NET's `UpdateAsync(patch, returning:
    ["*"])`, and Swift's, Kotlin's and Dart's
    `updateReturning` — held to the same text.

- **And waits for a job** the client's way: JavaScript's
    `{ wait: 30000 }` beside `{ signal }`, Python's
    `wait=30` (seconds), Go's `db.Wait(30 *
    time.Second)` under the request's context, .NET's
    `db.WithWait(TimeSpan.FromSeconds(30))` under its
    cancellation token, and Swift's, Kotlin's and Dart's
    `withWait`. Each client waits for the answer as much
    longer than its own timeout, and gives the request up when it is
    cancelled.

- **Over HTTP** the claim is a `POST /query`; sent
    under an [`Idempotency-Key`](https://fenecdb.com/docs//docs/http#idempotency),
    a retry after a timeout is answered the jobs the first try took, and
    takes none.

- **A worker's token** claims through its rules: a policy line
    `jobs update(owner, run_at, attempts) where queue = $jwt.queue for
    worker` lets it take its queue's jobs alone and change nothing
    else of them, and what it answers is held to its read rules as well,
    as it writes them: a worker that reads only `where owner =
    $jwt.sub` is answered the jobs it claims. A transition says what
    a claim may make of a job — `where run_at <= now() and
    new.owner = $jwt.sub and new.attempts = attempts + 1`, so no
    claim is made for another worker or wipes the count a dead letter
    keeps ([Scoped access](https://fenecdb.com/docs//docs/http#scoped-access)).

- **Not on a sync replica.** A replica would pick its own rows and
    answer them, and the server its own: a claim there is refused, and goes
    to the server itself.

## What it costs

Over a million jobs, `make queue-bench`:

| a million jobs | jobs a second | a claim of 10, p50 | p99 |

| one worker, `@sorted` on `run_at` | — | 18.7 us | 25.8 us |

| one worker, no index: a scan | — | 132.7 ms | 134.8 ms |

| 16 threads, the claim one statement, each job acked | **185 800** | 22.4 us | 1.52 ms |

| 16 threads, read, compare-and-set, read back | 93 800 | 0.40 ms | 1.89 ms |

| 16 HTTP clients, a claim and a `/batch` of its acks | **138 500** | 0.48 ms | 2.99 ms |

| 16 HTTP clients, three round trips to claim | 48 200 | 0.60 ms | 2.51 ms |

- **The three statements a client sends without
    `returning`** — read the ready page, set the jobs only
    where they are still ready, read back which it won — lose a race
    whenever two workers read the same page: 43% of the jobs read in
    process, and 77% over HTTP, had gone to another worker by the time it
    wrote, and were read again. Half the jobs a second in process, a third
    over HTTP.

- **Sixteen claims wait for each other,** as every write does: a
    claim's p99 among sixteen threads is the lock's queue, where one
    worker's is 25.8 us.

- **The claim reads its page and no more:** the ready jobs are a
    range of the index, the time worked out once before the plan. A queue
    holding a million leased or delayed jobs and three ready claims the
    three, where a walk testing each job's time would pass every leased
    one.

## Compared

- **PostgreSQL**: `UPDATE jobs SET ... WHERE id IN (SELECT id
    FROM jobs WHERE run_at <= now() ORDER BY run_at LIMIT 10 FOR UPDATE
    SKIP LOCKED) RETURNING *`. Its row locks let claims run side by
    side; here the one writer runs them one after another, each a few
    microseconds, and none has rows to skip.

- **SQS**: a claim is `ReceiveMessage`, the lease its
    visibility timeout, an extension `ChangeMessageVisibility`,
    `attempts` its receive count and a dead letter its
    redrive. A delayed job is `DelaySeconds`, with no
    fifteen-minute ceiling.

- **BullMQ**: delayed, prioritised, retried with a backoff and kept
    as failed, as here; its jobs live in Redis's memory, these in the file,
    durable by the server's `--sync` policy, replicated with it,
    and read with any query.

## What this does not cover

- **Exactly once.** A lease that lapses under a slow worker hands
    its job to another; the work's own check makes the second run
    harmless.

- **Claims side by side.** They are writes, so each waits for the
    one before: tens of thousands a second, not millions.


---

Source: https://fenecdb.com/docs/geo

# Points and distances

Places on the earth in a collection: the metres between
two, every row within a radius or a box, the nearest first — what an app
adds Redis's `GEOADD` and `GEOSEARCH` or PostGIS
for.

A `geo` field holds a point, `[lon, lat]` in
degrees. `distance(loc, p)` is the metres between two points,
`within(loc, [west, south, east, north])` whether one lies in a
box, and `near loc p` puts rows in order nearest first.
`@geo` indexes the field, so a radius, a box and the nearest
read the rows near the point rather than every row. Every answer is the one
a scan of every row gives, row for row.

The other jobs an app adds Redis for — counters, locks, caches, job queues, pub/sub — are recipes of
[Instead of Redis](https://fenecdb.com/docs//docs/redis).

Every FenecQL block on this page is run, statement by statement, by a
test (`crates/fenec-core/tests/geo_docs.rs`), and each
`-- → x` is the answer it checks: a write's count, or the first
value of the first row it answered. The costs are `make
geo-bench` on an M1 Air, over a million points.

## Points

A point is written as a list of two numbers, its longitude first —
GeoJSON's order, and the order Redis and PostGIS take them in.

```fenecql
create collection places (name text, kind text @hash, loc geo @geo)
put places [{name: "Palermo", kind: "city", loc: [13.361389, 38.115556]}, {name: "Catania", kind: "city", loc: [15.087269, 37.502669]}]   -- → 2
get places select loc where name = "Catania"   -- → [15.087269,37.502669]
```

- **Redis:** `GEOADD Sicily 13.361389 38.115556 "Palermo"
    15.087269 37.502669 "Catania"`.

- **Kept as written.** The two numbers are `f64`s and
    come back as they went in. Redis keeps a 52-bit geohash, which moves a
    point to the centre of a cell 0.6 m by 0.3 m at the equator: its
    `GEOPOS` answers
    `15.08726745843887329` for the `15.087269` it
    was given.

- **A longitude from -180 to 180 and a latitude from -90 to 90,**
    or the write is refused: a latitude of 91 is most often the two given
    the other way round. Redis takes latitudes to 85.05 only.

- **`null` is no point.** A row without one is in no
    radius and no box, and `near` answers only rows that hold
    one.

## Distance

`distance(a, b)` is the metres between two points, on a
sphere of Redis's radius (6 372 797.56 m) — so it is
`GEODIST`'s answer.

```fenecql
get places select distance(loc, [15.087269, 37.502669]) as m where name = "Palermo"   -- → 166274.2577914073
```

- **Redis:** `GEODIST Sicily Palermo Catania` answers
    `166274.1516`, between the points it keeps: 0.11 m off,
    since it moved each to its cell. Over 2 000 pairs of points, crowded
    and spread over the earth, the two agree to 0.05 mm from the points
    Redis keeps (`GEOPOS`), and to 0.54 m from the points as
    written.

- **PostGIS:** `ST_Distance(a, b, false)` over
    `geography` is on a sphere of the earth's mean radius,
    6 371 008.8 m, so its metres are 0.03% fewer: Palermo to Catania is
    166 227.6 m. Its default, the WGS84 ellipsoid, differs from a sphere by
    up to 0.5%: 166 506.2 m.

- **The same bits everywhere.** The trigonometry is fenecdb's own,
    not the platform's: a point on a radius's edge is in or out alike on a
    server, a phone and a page.

## Within a radius

```fenecql
get places select name where distance(loc, [15, 37]) <= 200000   -- → "Palermo": in id order
get places select name where distance(loc, [15, 37]) <= 100000   -- → "Catania"
get places where distance(loc, [15, 37]) <= 200000 count         -- → 2
```

- **Redis:** `GEOSEARCH Sicily FROMLONLAT 15 37 BYRADIUS 200
    km`.

- **A radius reaches the index** written `distance(loc, p)
    <= r` or `< r`, either way round, in an
    `and` chain, the point and the radius each a literal or a
    parameter. A ring — `distance(loc, p) > 1000` beside it —
    is tested on the rows the radius found.

- **Any radius:** 0 is the point itself, and one past half the
    earth's circumference is every row. A radius over a pole takes every
    longitude, and one across the antimeridian both sides of it.

## Nearest first

`near loc p` orders the rows by their distance from
`p`, a tie by id, and `_score` is the distance in
metres:

```fenecql
get places select name, distance(loc, [15, 37]) as m near loc [15, 37] limit 1   -- → "Catania"
get places select name where distance(loc, [15, 37]) <= 200000 near loc [15, 37] limit 10   -- → "Catania": then Palermo
get places select name where distance(loc, [15, 37]) <= 200000 order distance(loc, [15, 37]) limit 10   -- → "Catania": the same rows
```

- **Redis:** `GEOSEARCH Sicily FROMLONLAT 15 37 BYRADIUS 200
    km ASC COUNT 10 WITHDIST`. Redis needs a radius to search in;
    `near` needs none.

- **The index is walked nearest first** and stops at the page:
    the ten nearest of a million points took 33 to 38 us, whether they were a
    street or an ocean away.

- **`_score` is an `f32`,** as every score
    is: to the metre out to 16 777 km, and 2 m past it. Select
    `distance(loc, p)` for the `f64`.

- **`near loc p exact`** measures every row, as the
    collection without the index does: the same answer.

- **`order distance(loc, p)`** is the same order as a
    key of `order`, ties by id, beside other keys and in a write
    that picks its rows ([The nearest job](#dispatch)). With a
    `limit` it walks the index as `near` does where no
    row the filter passes lacks a point; a row with none has a null
    distance, which comes first, so where some may pass -- no radius or box
    on the field, and rows without a point -- every row the filter passes is
    measured. `near` leaves such rows out.

## In a box

`within(loc, [west, south, east, north])` holds a point
inside the box or on its edge. A box whose west is east of its east crosses
the antimeridian:

```fenecql
get places where within(loc, [12, 37, 16, 39]) count   -- → 2
put places [{name: "Suva", kind: "city", loc: [178.4419, -18.1248]}, {name: "Apia", kind: "city", loc: [-171.7513, -13.8333]}, {name: "Nuku'alofa", kind: "city", loc: [-175.2018, -21.1394]}]   -- → 3
get places where within(loc, [170, -25, -170, -10]) count   -- → 3
get places select name near loc [180, -15] limit 1   -- → "Suva"
```

- **Redis's `BYBOX`** is a width and a height in metres
    around a centre; `within` takes the corners, as a map's
    visible area gives them. The rows of a map's view are
    `within(loc, $bounds)` with `limit`.

- **PostGIS:** `loc && ST_MakeEnvelope(w, s, e, n,
    4326)`, whose box does not wrap: one across the antimeridian is
    two.

## With the rest of a query

A radius or a box is one condition of an `and` chain, so the
rest of the query goes with it: other conditions, an order, a count, a
facet, a `lookup`.

```fenecql
put places [{name: "Cafe A", kind: "cafe", loc: [15.0912, 37.5047]}, {name: "Cafe B", kind: "cafe", loc: [15.0855, 37.5076]}, {name: "Fish market", kind: "market", loc: [15.0868, 37.5005]}]   -- → 3
get places select name where kind = "cafe" and distance(loc, [15.087269, 37.502669]) <= 1000 near loc [15.087269, 37.502669] limit 5   -- → "Cafe A"
get places where kind = "cafe" and distance(loc, [15.087269, 37.502669]) <= 1000 count   -- → 2
get places select name where distance(loc, [15.087269, 37.502669]) <= 1000 facet kind
```

- **Whichever index names fewer rows is read,** and the other
    condition tested on each: a `@hash` bucket of a few dozen
    cafes, or the rows of a radius narrower than the bucket. A
    `near` over a set of a few thousand rows measures it whole
    rather than walk the points.

- **A scoped token** finds only the points its rules let it read:
    its filter is ANDed in as it is into every statement
    ([Security](https://fenecdb.com/docs//docs/server#security)).

- **`or` reads every row,** as it does for any index.
    Two radii are two statements, or one statement each in a
    `/batch`.

## Positions that expire

Couriers, drivers, a user's last known place: each one's position
written over as it moves, and gone when it stops being sent.
`@ttl` takes a row out of every read once its time is
past.

```fenecql
create collection couriers (name text @unique, loc geo @geo, seen timestamp @ttl(30s))
put couriers {name: "ada", loc: [15.0901, 37.5031], seen: now()} if absent else set {loc: new.loc, seen: new.seen}   -- → 1
put couriers {name: "bo", loc: [15.0702, 37.5120], seen: now()} if absent else set {loc: new.loc, seen: new.seen}    -- → 1
get couriers select name near loc [15.087269, 37.502669] limit 1   -- → "ada"
-- 20 s later
put couriers {name: "bo", loc: [15.0880, 37.5030], seen: now()} if absent else set {loc: new.loc, seen: new.seen}    -- → 1: bo moved
get couriers select name near loc [15.087269, 37.502669] limit 1   -- → "bo"
-- 20 s later
get couriers select name near loc [15.0901, 37.5031] limit 1       -- → "bo": ada's position is 40 s old
get couriers count   -- → 1
```

- **An upsert a position:** the row made the first time, its point
    and time set after, one statement whichever it is.

- **Gone at once:** a row past its `@ttl` is out of every
    read the moment it is, and the server's sweeper deletes it one to two
    minutes later ([Rows that expire](https://fenecdb.com/docs//docs/fenecql#ttl)).

## The nearest job

An order waiting for a courier, a ride for a driver: the job nearest a
worker that is ready, taken in one statement under the write lock, so two
workers never take the same one. `order` takes the distance as
its key in a `set` that picks its rows, and
`returning` answers what was taken
([Writes that pick their rows](https://fenecdb.com/docs//docs/fenecql#pick)):

```fenecql
create collection deliveries (pickup geo @geo, run_at timestamp @sorted, courier text)
put deliveries [{pickup: [15.0912, 37.5047], run_at: now()}, {pickup: [15.0855, 37.5076], run_at: now()}, {pickup: [15.0880, 37.5031], run_at: now() + 600000}]   -- → 3
set deliveries {courier: "ada", run_at: now() + 45000} where run_at <= now() and distance(pickup, [15.0901, 37.5031]) <= 3000 order distance(pickup, [15.0901, 37.5031]) limit 1 returning id   -- → 1: the nearest ready; 3 is nearer, and waits ten minutes
set deliveries {courier: "bo", run_at: now() + 45000} where run_at <= now() and distance(pickup, [15.0901, 37.5031]) <= 3000 order distance(pickup, [15.0901, 37.5031]) limit 1 returning id   -- → 2: the one left
set deliveries {courier: "cy", run_at: now() + 45000} where run_at <= now() and distance(pickup, [15.0901, 37.5031]) <= 3000 order distance(pickup, [15.0901, 37.5031]) limit 1 returning id   -- → no row: none ready within 3 km
-- 50 s later
set deliveries {courier: "cy", run_at: now() + 45000} where id in (get deliveries select id where run_at <= now() near pickup [15.0901, 37.5031] limit 1)   -- → 1: ada's lease lapsed
```

- **The lease is the ready time moved on**: a courier who goes quiet
    holds the job until it lapses, and the next claim takes it, with nothing
    written to say so. Its pickup, delivery and every beat are
    `set ... where id = $1 and courier = $2 require 1`, refused
    once another holds it ([Job queues](https://fenecdb.com/docs//docs/queues)).

- **The radius bounds the walk**: the point index is walked
    nearest first and stops at the page or 3 km out, whichever comes first.
    With a radius on the field the walk is taken even where some rows hold
    no point, since none of those passes it.

- **Or an inner `get`** with `near`, run
    first and its one id written, under the same lock. A
    `near` takes no `order`, and a write takes no
    `near` of its own: the distance is a key of
    `order` there.

## From a client

Over HTTP a point is a JSON array, and a filter is FenecQL's:

```bash
curl -X POST http://127.0.0.1:8080/places -d '{"name": "Catania", "loc": [15.087269, 37.502669]}'
curl 'http://127.0.0.1:8080/places?select=name&where=distance(loc,%5B15,37%5D)%3C%3D200000'
curl -X POST http://127.0.0.1:8080/places/near -d '{"field": "loc", "vector": [15, 37], "limit": 10}'
curl -X POST http://127.0.0.1:8080/query \
  -d '{"query": "get places select name near loc $1 limit 10", "params": [[15, 37]]}'
```

The builders write it as an object condition:

```ts
import { connect, distance, type Point } from '@fenecdb/web/client';
import type { FenecSchema } from './fenec-schema.js';   // fenec types app.fenec

const db = connect<FenecSchema>('http://127.0.0.1:8080');
const here: Point = [15.087269, 37.502669];
const cafes = await db.from('places')
  .select('name', distance('loc', here).as('m'))
  .where({ kind: 'cafe', loc: { distance: { from: here, lte: 1000 } } })
  .near('loc', here)
  .limit(10)
  .rows();
const inView = await db.from('places').where({ loc: { within: [12, 37, 16, 39] } }).rows();
```

Python's, Go's, .NET's, Swift's, Kotlin's and Dart's builders take the
same `within` and `distance` conditions and render the
same text. `fenec types` writes a `geo` field as each
language's pair of doubles, and a Drizzle table's
`geometry('loc', { type: 'point' })` with a GiST index is a
`geo` field with `@geo`.

- **The degrees as written.** Every reader without a schema — a
    query's parameters, a JSON body, the browser module — reads a list of
    numbers as a vector's `f32`s, which would move a point by up
    to 1.7 m at the antimeridian. A list going to a `geo` field
    or a `distance`, `within` or `near`
    over one is read again as written.

- **In the page.** The browser module answers the same statements
    over a database in the page or a synced replica, its index and all.

## From PostGIS

`fenec import` takes a `geometry(Point, 4326)` or
`geography(Point)` column, or PostgreSQL's own
`point`, as a `geo` field:

```bash
fenec import postgres://app@db:5432/app --table stores --into stores --index "loc@geo"
```

A column of other shapes — lines, polygons — or of another SRID is
refused naming it; `--follow` mirrors the table as it changes
([Import](https://fenecdb.com/docs//docs/import)).

## What it costs

A million points crowded round 48 cities, as a shop's or a courier's lie,
`make geo-bench`: fenecdb in process; PostGIS 3.5 (a
`geography(Point)` column, GiST, `ST_DWithin(…, false)`
and `<->`) and Redis 7 (`GEOSEARCH`) in Docker,
each server's own mean time of the query from
`pg_stat_statements`, planning included, and `INFO
commandstats` — the network's 0.3 ms left out. The median of a
thousand queries near a point of the data:

| a million points | rows | fenecdb `@geo` | no index: the scan | Redis | PostGIS |

| within 100 m | 0.3 | **7.2 us** | 21.7 ms | 12.6 us | 1.26 ms |

| within 1 km | 27 | **12.6 us** | 21.9 ms | 34.3 us | 1.31 ms |

| within 10 km | 2 543 | **0.56 ms** | 22.8 ms | 1.73 ms | 4.84 ms |

| within 100 km | 36 341 | **5.9 ms** | 28.2 ms | 7.6 ms | 32.2 ms |

| the ten nearest within 50 km | 10 | **38 us** | — | 6.0 ms | 1.77 ms |

| the ten nearest | 10 | **33 us** | 6.1 ms | 198 ms | 1.75 ms |

| the index's build, its memory |  | 34 ms, 20 MB | — | 86 MB | 5.4 s, 82 MB |

- **A put** with `@geo` costs 1.19 us, against 0.62
    without an index and 0.98 with `@sorted` on a float.

- **Spread evenly** over the earth, a million points are about 20 km
    apart: a radius of up to 10 km took 5 to 10 us, 100 km 24 us, the ten
    nearest 35 us.

- **Redis** sorts every member of the cells round the point by
    distance: the ten nearest anywhere is a search past half the earth,
    every member measured. **PostGIS** plans each statement again,
    half a millisecond of its time, and its GiST over
    `geography` read 132 pages for a radius of 100 m.

- **Over HTTP** a request adds its round trip, as it does to
    PostgreSQL's and Redis's: 0.3 ms through Docker's network here.

## What this does not cover

- **Shapes other than points:** no lines, polygons or a point in a
    polygon. A box is the one area a query takes; a polygon's bounding box
    narrows, and the app tests the rest.

- **The ellipsoid.** Distances are on the sphere, as Redis's are:
    within 0.5% of WGS84's, which PostGIS's `geography` takes by
    default.

- **A distance between two rows** in one statement: the other
    row's point is read first and passed as a parameter.

- **`GEOHASH` strings and `STORE`**: a
    search's rows are answered, not written to another key.


---

Source: https://fenecdb.com/docs/analytics

# Analytics and market data

Counts by the minute, distinct users, funnels, bars and
VWAP: one statement each, over the rows as they were written.

Four things in the select list make these work. An item is an
expression — `px * qty`, or an aggregate of one,
`sum(px * qty)`, or an expression over aggregates,
`sum(px * qty) / sum(qty)`. `group` takes several
keys, and `bucket(at, 1m)` makes a time one. `count(distinct
user)` counts values, exactly. And `first(px by at)` and
`last(px by at)` take the value of the earliest and latest row of
a group. See [Aggregates](https://fenecdb.com/docs//docs/fenecql#aggregates) for the
rules.

Every FenecQL block on this page is run, statement by statement, by a
test (`crates/fenec-core/tests/analytics_docs.rs`), and each
`-- → x` is the first value of the first row it checks. The
costs are `make analytics-bench` on an M1 Air, in process, over
a million events and a million ticks; over HTTP a request adds its round
trip.

Counts by the minute and distinct users are what an app often keeps in
Redis — `INCR` on a key a minute, `PFADD` and
`PFCOUNT`. Here they are statements over the events as they were
written; the other patterns, [unique
visitors](https://fenecdb.com/docs//docs/redis#unique-visitors) among them, are recipes of [Instead of
Redis](https://fenecdb.com/docs//docs/redis).

These recipes make a whole application in
[examples/analytics](https://github.com/fenecdb/fenec/tree/main/examples/analytics):
a 572-byte tracker, beacons written once however often they are sent,
rollups kept from the change stream exactly once, a dashboard that reads
the raw events for a day and the rollups past it, a site a tenant, and
the market data below with live prices — with its tests, Lighthouse
scores and measurements at ten million events.

## Analytics

Raw events, one row each. `at` takes `@sorted` —
or `@ttl(30d)`, which is an ordered index whose rows expire — so
a time range reads only its rows.

```fenecql
create collection events (user int @hash, name text @hash, at timestamp @sorted, country text, props json)
put events [{user: 1, name: "view", at: "2026-05-03T09:00:10Z", country: "TR", props: {plan: "pro", v: 3}}, {user: 1, name: "cart", at: "2026-05-03T09:02:00Z", country: "TR", props: {plan: "pro", v: 1}}, {user: 1, name: "buy", at: "2026-05-03T09:05:30Z", country: "TR", props: {plan: "pro", v: 9}}, {user: 2, name: "view", at: "2026-05-03T09:01:00Z", country: "US", props: {plan: "free", v: 2}}, {user: 2, name: "view", at: "2026-05-03T10:15:00Z", country: "US", props: {plan: "free", v: 2}}, {user: 3, name: "view", at: "2026-05-03T10:20:00Z", country: "DE", props: {plan: "free", v: 1}}, {user: 3, name: "buy", at: "2026-05-03T10:40:00Z", country: "DE", props: {plan: "free", v: 5}}, {user: 1, name: "view", at: "2026-05-04T08:00:00Z", country: "TR", props: {plan: "pro", v: 1}}]
```

### Rollups

Counts by the hour, or the minute: `bucket` truncates a
timestamp to the interval it falls in, in UTC — `ms`,
`s`, `m`, `h`, `d` from the
epoch, `w` from a Monday, `mo` and `y` by
the calendar. Name it with `as` and group by the name.

```fenecql
get events select bucket(at, 1h) as hour, count(*) as n where at >= "2026-05-03" and at < "2026-05-04" group hour   -- → "2026-05-03T09:00:00Z"
get events select count(*) as n, bucket(at, 1h) as hour where at >= "2026-05-03" and at < "2026-05-04" group hour   -- → 4: the first hour's
get events select count(*) as n, name, bucket(at, 1d) as day group name, day order n desc, name   -- → 4: views on the 3rd
get events select sum(props.v) as v, props.plan group props.plan order v desc   -- → 14: a json path summed
```

A count by `bucket` of the field a range is on reads its
ordered index and no row: per-minute counts over a day of a million events
— 1 441 groups of 143 000 rows — take 4.0 ms, where grouping by a minute
field written at ingest, which reads the rows, takes 19.5. A raw collection under `@ttl` keeps a few days; a job that
runs these statements writes the rollups into a collection without one
before the raw rows go.

### A day by a zone's clock

`bucket` counts in UTC; a third argument counts by a zone's
clock — an IANA name, or an offset written out. A day then starts at local
midnight: Istanbul's 3rd of May began at 21:00 UTC on the 2nd.

```fenecql
get events select bucket(at, 1d, "Europe/Istanbul") as day, count(*) as n group day order day   -- → "2026-05-02T21:00:00Z"
get events select count(*) as n, bucket(at, 1d, "Europe/Istanbul") as day group day order day   -- → 7: the 3rd in Istanbul
get events select count(*) as n, bucket(at, 1h, "+05:30") as hour where at < "2026-05-04" group hour order hour   -- → 4: 14:00 to 15:00 in India
```

Where daylight time begins or ends a local day is 23 or 25 hours, and
an offset written into the statement — `bucket(at + 10800000,
1d)` — cannot follow it. Berlin's clocks went back on the 25th of
October 2026, so that day holds the hour its clock showed twice.

```fenecql
create collection beats (at timestamp @sorted)
put beats [{at: "2026-10-24T22:30:00Z"}, {at: "2026-10-25T22:30:00Z"}, {at: "2026-10-26T00:30:00Z"}]
get beats select bucket(at, 1d, "Europe/Berlin") as day, count(*) as n group day order day   -- → "2026-10-24T22:00:00Z"
get beats select count(*) as n, bucket(at, 1d, "Europe/Berlin") as day group day order day   -- → 2: one day of 25 hours
get beats select count(*) as n, bucket(at, 1d) as day group day order day   -- → 1: in UTC, three days
```

A page passes its own zone as a parameter —
`Intl.DateTimeFormat().resolvedOptions().timeZone` in a browser.
The rules are IANA's from 2000 on; a time before it in a named zone is
refused, and an offset answers any time. Counted off the ordered index,
a week's days of a million events by Istanbul's clock take 29 to 34 ms
against 23 to 27 in UTC, where the offset written in reads every row, 133
to 154; its hours by Berlin's clock 31 against 25; with a filter that reads
the rows, 94 to 103 against 91 to 98 (`make analytics-bench`).

A rollup kept as the events come is a row a key, made at its first
count and added to after: `if absent else set` is that upsert,
one statement for a page of keys, the set reading the row it finds and
`new.` the document put. With the page as a parameter,
`put hourly $1 if absent else set {n: n + new.n}`, its text is
the same whatever the page holds.

```fenecql
create collection hourly (key text @unique, n int)
put hourly [{key: "2026-05-03T09|view", n: 2}, {key: "2026-05-03T09|buy", n: 1}] if absent else set {n: n + new.n}   -- → 2: made
put hourly [{key: "2026-05-03T09|view", n: 3}, {key: "2026-05-03T10|view", n: 2}] if absent else set {n: n + new.n}   -- → 2: one added to, one made
get hourly select n where key = "2026-05-03T09|view"   -- → 5
```

A page of 1 000 keys, half of them new, is one statement in 1.09 ms in
process, where reading which keys exist and writing the new rows and the
old ones over took 1.49 ms and a round trip more.

### Distinct users

`count(distinct f)` holds each value once a group, exactly,
up to a million values a query — about 70 MB — and refuses a query past
that rather than count short.

```fenecql
get events select count(distinct user) as users, bucket(at, 1d) as day group day   -- → 3: on the 3rd
get events select count(distinct user) where at >= "2026-05-04"   -- → 1
get events select user, count(distinct bucket(at, 1d)) as days group user order days desc, user   -- → 1: active two days
```

Distinct users per hour over seven days of a million events, 169
groups: 125 ms. Before, a query an hour listed the users and the client
counted them: 7.7 s.

Past a million values, or to add days up, count approximately:
`approx_count_distinct(f)` is HyperLogLog, 214
registers — 16 KB a group, sparse while a group has few values — with a
standard error of 0.81%, whatever the count. `hll_accumulate(f)`
answers the sketch itself as bytes, to keep a day's in a rollup row;
`hll_combine(s)` merges sketches and `hll_estimate(s)`
reads a count off one, so a month is its days' sketches merged — not their
counts added, which counts a user of two days twice. Sketches are the same
bytes wherever made, the browser's or a server's. A query's sketches are
held to 64 MB, 4 096 dense groups, and refused past that.

```fenecql
get events select approx_count_distinct(user) as users   -- → 3
get events select hll_estimate(hll_accumulate(user)) as users where at >= "2026-05-04"   -- → 1
get events select bucket(at, 1d) as day, hll_accumulate(user) as users group day   -- → "2026-05-03T00:00:00Z"
```

Per hour over the million events it takes 121 ms, the furthest of
the 169 hours 1.3% off; over thirty sketches of 200 000 values the error's
root mean square is 0.77%.

### Funnels

How many users reached each step, and how many of the viewers
bought:

```fenecql
get events select count(distinct user) as users, name where name in ["view", "cart", "buy"] group name order users desc   -- → 3: viewed
get events select count(distinct user) where name = "buy" and user in (get events select user where name = "view")   -- → 2: viewed and bought
```

The inner `get` is a list of up to 100 000 values; past that
ask from the users' side with `lookup … required` (Subqueries). An ordered funnel — a
view, then a buy within the hour — compares each user's first of each:
`having` keeps the groups a condition holds for, over their
keys, the list's columns by name and any aggregate, and
`count` after it counts them rather than send a row a user.

```fenecql
get events select user, min(case when name = "view" then at end) as viewed, min(case when name = "buy" then at end) as bought where name in ["view", "buy"] group user having bought >= viewed and bought <= viewed + 3600000 count   -- → 2
get events select user, count(*) as n group user having n >= 3 order n desc   -- → 1
```

### Retention

Each user's first day is their cohort, and the days they came back are
a group of two keys. The table of cohorts by days since is folded from
these two answers on the client.

```fenecql
get events select user, min(bucket(at, 1d)) as cohort group user   -- → 1
get events select user, bucket(at, 1d) as day, count(*) as events group user, day   -- → 1
```

## Market data

Ticks, one row each, written as they come — not always in time
order.

```fenecql
create collection ticks (sym text @hash, at timestamp @sorted, px float, qty int)
put ticks [{sym: "S07", at: "2026-05-03T10:00:05Z", px: 100.25, qty: 10}, {sym: "S07", at: "2026-05-03T10:00:40Z", px: 101.5, qty: 5}, {sym: "S07", at: "2026-05-03T10:00:20Z", px: 99.75, qty: 20}, {sym: "S07", at: "2026-05-03T10:01:10Z", px: 102.25, qty: 1}, {sym: "S07", at: "2026-05-03T10:01:50Z", px: 101.75, qty: 4}, {sym: "S08", at: "2026-05-03T10:00:30Z", px: 50.5, qty: 100}, {sym: "S08", at: "2026-05-03T10:01:30Z", px: 51.25, qty: 50}]
```

### Bars

Open, high, low, close and volume, a bar a minute. `first(px by
at)` is the price of the bar's earliest tick, whatever order they
were written in; a tie in `at` goes to the row written first,
and `last`'s to the row written last. Without `by`
they take the order the rows were written in.

```fenecql
get ticks select bucket(at, 1m) as bar, first(px by at) as open, max(px) as high, min(px) as low, last(px by at) as close, sum(qty) as volume where sym = "S07" group bar   -- → "2026-05-03T10:00:00Z"
get ticks select first(px by at) as open, bucket(at, 1m) as bar where sym = "S07" group bar   -- → 100.25
get ticks select last(px by at) as close, bucket(at, 1m) as bar where sym = "S07" group bar   -- → 101.5
get ticks select max(px) - min(px) as range where sym = "S07"   -- → 2.5
```

An hour of one-minute bars of one symbol out of a million ticks over
50: 1.7 ms in process, 2.3 over HTTP. Before, open and close were two
queries a bar, 120 for the hour: 195 ms in process, 278 over HTTP.

### VWAP

The volume-weighted average price is an expression over two
aggregates:

```fenecql
get ticks select sum(px * qty) / sum(qty) as vwap, sym group sym order sym   -- → 100.35625
get ticks select sym, sum(px * qty) as notional, sum(qty) as volume group sym order notional desc   -- → "S08"
```

Every symbol's over a million ticks: 118 ms. Before, the client read
every tick out and summed them: 173 ms in process, and a million rows
over the wire.

### The latest price

```fenecql
get ticks select last(px by at) as px, sym group sym order sym   -- → 101.75
```

Every symbol's, a scan of the million ticks: 108 ms, against 129 for
a query a symbol (`where sym = $1 order at desc limit 1`, each
sorting its symbol's 20 000 rows). A page that shows it all the time
keeps a row a symbol instead:

```fenecql
create collection quotes (sym text @unique, px float, high float, low float)
put quotes {sym: "S07", px: 101.75, high: 101.75, low: 101.75} if absent   -- → 1
set quotes {px: 99.5, high: greatest(high, 99.5), low: least(low, 99.5)} where sym = "S07"   -- → 1
get quotes select low where sym = "S07"   -- → 99.5
get quotes select case when px < high then "below" else "at the high" end as where_it_is where sym = "S07"   -- → "below"
```

`greatest` and `least` pass a null over, and
`case` works out only the branch it takes.


---

Source: https://fenecdb.com/docs/server

# Server

Run fenecdb as a server: one process over one file, reached
over HTTP from any language.

## Start and query

-

Start the server on a file.

```bash
fenec-server --file data.fenec --http 127.0.0.1:8080
```

-

Send it a query.

```bash
curl -s -X POST localhost:8080/query \
  -d '{"query": "get articles select title near embed $1 limit 5", "params": [[0.1, 0.2]]}'
```

Everything the server does goes over that one listener: queries and the
REST routes, subscriptions, replication, change data capture, metrics and the
admin routes of a tenant node. The requests are on [HTTP
endpoint](https://fenecdb.com/docs//docs/http), and each language's way of sending them on Your language.

Details

One process writes one file. Two processes over the same file corrupt it,
which is why everything a server does — HTTP, replication, the graph keeper,
the sweeper of rows past their time ([`@ttl`](https://fenecdb.com/docs//docs/fenecql#ttl)),
the `--follow` mirror — runs inside this one process rather than
as programs of its own.

One thread per connection. Read-only statements run at the same time under
a shared lock; writes take the exclusive one. HNSW search buffers are
thread-local, so the same index can be searched in parallel — eight
concurrent full scans on eight cores measured 4.3×.

A `/batch` is one block: its writes land in the file as one
record, or none of them do, and a replica applies it whole. There are no
interactive transactions held open across requests — the database has one
writer, and a transaction a client held open would hold every other write
with it. An `Idempotency-Key` makes a write, or a
`/batch`, safe to retry ([HTTP endpoint](https://fenecdb.com/docs//docs/http)).

With `--replica-of` the process follows another
`fenec-server` as a read replica: it serves every read and refuses
every write with 403. The flags, promotion and what a failover loses are on Replication. With `--dir <path>`
instead of `--file` it serves a directory of tenant files, one per
tenant under `/t/<tenant>/`; that mode, its
`--admin-token` and `--idle-close`, and the router in
front of it are on [Tenants and sharding](https://fenecdb.com/docs//docs/sharding).

## Flags

| Flag | What it does |

| `-f, --file <path>` | The database file. In memory without it. The server holds `<path>.lock` while it runs, so a second process over the file — another server, `fenec mcp --file`, an app through the native library — is refused rather than let corrupt it. |

| `--dir <path>` | One file per tenant in this directory ([Tenants and sharding](https://fenecdb.com/docs//docs/sharding)). |

| `--http <address>` | Where to listen. Default `127.0.0.1:8080`. |

| `--http-token <value>` | Requires `Authorization: Bearer <value>`. Also read from `FENEC_HTTP_TOKEN`, since `argv` shows up in `ps`. |

| `--jwt-secret`, `--jwt-secret-file`, `--jwt-keys`, `--jwt-require-exp`, `--jwt-max-age`, `--jwt-tenant-claim`, `--jwt-unbound-tenants`, `--policy` | Tokens held to rules: which collections, which rows ([Security](#security)). |

| `--sync off|always|<ms>` | When writes reach the disk. Default `250` ([Durability](#durability)). |

| `--no-checkpoint` | Writes no checkpoint on shutdown. |

| `--no-mmap` | Reads the file into memory instead of mapping it. |

| `--warm all|off|<list>` | Builds the hash, text, ordered and sparse indexes after the open, beside the first requests, rather than on the first read of each. Default `all` for a file, `off` for a node's tenants ([Warm-up](#warm)). |

| `--auto-compact <ratio>|off` | Compacts a file on its own once this share of it is dead, beside the queries. Default `0.5` ([Compaction](#compaction)). |

| `--max-connections`, `--idle-timeout`, `--max-memory` | [Limits](#limits). |

| `--http-cors`, `--http-read-only`, `--http-max-streams`, `--http-keepalive`, `--changes`, `--idempotency-ttl` | The HTTP endpoint's options ([HTTP endpoint](https://fenecdb.com/docs//docs/http)). |

| `--replication-token`, `--replica-of`, `--promote`, `--replication-buffer`, `--cdc` | [Replication](https://fenecdb.com/docs//docs/replication) and [change data capture](https://fenecdb.com/docs//docs/microservices). |

| `--admin-token`, `--idle-close`, `--lease` | A tenant node's ([Tenants and sharding](https://fenecdb.com/docs//docs/sharding)). The admin token is also read from `FENEC_ADMIN_TOKEN`. |

| `--follow <postgres://...>`, `--follow-*` | Mirrors a PostgreSQL table into the file and serves it ([Import](https://fenecdb.com/docs/import#serving)). |

| `--metrics`, `--slow-ms`, `--audit`, `--auth-delay` | [Monitoring](https://fenecdb.com/docs//docs/monitoring) and the audit log. |

| `--otlp-endpoint`, `--otlp-header`, `--trace-sample` | OpenTelemetry traces to a collector; off unless an endpoint is given ([Tracing](https://fenecdb.com/docs//docs/monitoring#tracing)). |

| `--studio`, `--studio-connect` | fenec studio, the admin pages, at `/_studio/`; off unless given ([Studio](https://fenecdb.com/docs//docs/studio)). |

| `--mcp`, `--mcp-write`, `--mcp-memory`, `--mcp-max-rows`, `--mcp-max-bytes`, `--mcp-calls-per-sec` | MCP for agents at `POST /_mcp`, each tool held to the request's token; off unless given ([Agents and MCP](https://fenecdb.com/docs//docs/mcp)). |

| `--insecure` | Allows a non-loopback address without a token. |

| `--ping` | Asks the server whether it is up and exits ([health checks](#limits)). |

`fenec-server --help` prints every flag with its default.

## Security

Give the server a token, or give each user a JSON Web Token that names what
they may read and write.

```bash
fenec-server --file data.fenec --http 0.0.0.0:8080 --http-token "$TOKEN"
```

**There is no TLS yet.** On a network, put a TLS terminator such as
  Caddy, nginx or a load balancer in front.

Details

`--http-token` is the server's own token: whoever sends it reads
and writes everything. The comparison is constant time. Listening on a
non-loopback address with no token is refused outright; it can be opened
deliberately with `--insecure`.

With `--jwt-secret` (HS256, at least 32 bytes) or
`--jwt-keys` (a JWKS file: `oct` keys for HS256,
`RSA` keys for an identity provider's RS256, a token's
`kid` picking the key, the file read again as it changes) the
server also takes JSON Web Tokens, each held to the rules of
`--policy`: which collections, to read, insert, update or delete,
which rows (`where owner = $jwt.sub`, `where room in
$jwt.rooms`, the rows another collection names: `where courier in
(get orders select courier where customer = $jwt.sub)`), which fields of
them a read shows (`read(pickup, run_at) where run_at <= now()`,
[Fields a read shows](https://fenecdb.com/docs//docs/http#field-grants)), and what an
update may make of a row (`where
status = "ready" and new.status = "cancelled"`). A token that may only read is the reporting login a
dashboard needs, one that may only insert a client appending events, and an
`append-only` collection takes no update or delete from any token
but the server's. The rules are ANDed into every statement
a token runs, and every document it writes is checked against them as well
([Scoped access](https://fenecdb.com/docs//docs/http#scoped-access)).
`--mint-token '{"sub": "ada"}'` prints a token signed with the
secret, for testing, which expires in an hour unless the claims name an
`exp`. A token with no `exp` is refused, since it
would be good for ever (`--jwt-require-exp off` takes it), and
`--jwt-max-age <s>` refuses one whose `exp` lies
further ahead than that. On a `--dir` node a token reaches only
the tenant its `tenant` claim names
([Tokens](https://fenecdb.com/docs//docs/sharding#tokens)). Pass the secret with `--jwt-secret-file` or
`FENEC_JWT_SECRET` — `argv` shows up in `ps`
output.

Every token and key is read from the environment too:
`FENEC_HTTP_TOKEN`, `FENEC_ADMIN_TOKEN`,
`FENEC_REPLICATION_TOKEN`, `FENEC_JWT_SECRET` and
`FENEC_JWT_KEYS`. One set and empty -- an environment file's line
left blank -- is no token. One given empty as a flag --
`--http-token "$TOKEN"` with `TOKEN` unset -- is
refused at the start, as is a secret's file holding nothing: a request that
carries no `Authorization` reads as the empty token, so an empty
admin token let anyone into `/_admin/`.

A request refused for its token waits before it is answered — 100 ms,
twice as long for each refusal from the same address within a minute, 5 s at
most (`--auth-delay`; 0 turns it off). A token the server
verifies clears its address's count; an answer that looked at no token,
whatever `Authorization` it carried, clears nothing. `--audit
<path>` appends a JSON line for each refused request, each change
of the schema and each admin request ([The
audit log](https://fenecdb.com/docs//docs/monitoring#audit)).

## Durability

Choose how often writes reach the disk with `--sync`.

| `--sync` | What it means |

| `250` (default) | Flushed every 250 ms. A crash loses at most that much. |

| `always` | Each write is answered once it is on disk. |

| `off` | Flushed only on shutdown. |

Details

Writes accumulate in a 1 MB buffer. Under `always` every write
waits for an `fsync`, about 3.7 ms on macOS for a write alone, and
writes arriving together share one; `250` costs next to nothing;
`off` nothing. A `/batch` lands with one
`fsync` however many writes it holds.

Under `always` the `fsync` runs after the write has
let go of the exclusive lock, and the bytes reach the file there too. Readers
therefore do not wait on the disk, and writes that arrive while one
`fsync` runs are made durable together by the next. One client
still gets one write per `fsync`. The sync of `250`
takes the same path, for a single database and for each tenant of a
`--dir` node: held through the `fsync`, a tenant's lock
had its reads and writes wait up to 27 ms once an interval.

`SIGINT`, `SIGTERM` and `SIGHUP` are caught
and a final sync runs before exit. The exclusive lock is **held** until
exit, so no write can be accepted between the sync and the exit and no write is
left that looked successful to the client but never reached the disk. Under
`kill -9` you get exactly what the chosen policy promised, and no
more.

When the disk refuses a write or a sync, the request fails with 500 —
under `always`, before the client is told it succeeded — and every
later write is refused until the process restarts; reads go on answering. The
sync is not retried: after a failed `fsync` the kernel may already
have dropped the pages, so a retry that succeeds would prove nothing.

**The shutdown checkpoint.** After the sync, if a vector index exists,
a checkpoint is written: the HNSW graph lands in the file and the next open
does not rebuild it — 9 ms instead of 3.4 s at 100 000 × 128. Vectors still
waiting to be linked after a crash are written as waiting, and linked after
the next open. The cost is that shutdown takes longer on a large database:
the image is written into the new file as it is produced, so it is a copy on
disk rather than one in memory. `--no-checkpoint` turns it off.
The order is deliberate. The sync comes first, so even if the process is
killed while the checkpoint is being written, the data is already on disk and
only the graph is lost. The rewrite goes to a side file and is renamed, so a
half-written checkpoint cannot corrupt anything.

## Warm-up

An open builds no hash, text, ordered or sparse index: the first query
that reads one builds it, and pays for it. `fenec-server` builds
them after the open instead, beside the first requests, one at a time: the
port opens as soon as before, and the first search finds its index built.
`--warm products,orders.status` names the collections and fields
to warm, `--warm off` none.

Details

Each index is built under the read lock on its own, so reads go on beside
it, a read that needs the index being built waits for that build rather than
starting its own, and a write waits out one index at most (a
`@text` index of 100 000 products: 141 ms). The thread starts once
the HTTP endpoint is up, whose start takes the write lock. Over 100 000
products with a text, two hash and an ordered index, on an M1: the first
answer 16 ms after the start either way, the indexes built 208 ms after it,
and the first `match` 1.4 ms warm against 166 ms cold; the process
held 99 MB against 88 for a cold one that had only searched, and 25 before
any read.

It is on for a file because an index is declared to be read, and a
first query that builds one is the slowest a server answers: a shop's
first search page took 2.68 s to its largest paint against 1.40 s warm. The
memory is what the reads would have taken anyway; only an index nothing
ever reads is held for nothing, and `--warm` names the rest. A
node's tenants open on their first request and close when idle, and
warming each would build every index of every tenant a request touches, so
there it is off unless asked. The native library has the same call,
`fenec_warm`, and the Rust API `Database::warm` and
`warm_index`.

## Compaction

An update writes the whole record again at the end of the file, and a
delete writes a tombstone; the version left behind is dead until a
`compact` writes the live records into a new file. The server
runs one on its own: every 5 s it looks at each file it serves, the
database, each open tenant and a replica alike, and compacts one once half
of it is dead and the dead part is at least 64 MB -- or sooner, once a
third of what the compact writes is dead, where the file has reached three
quarters of the memory the system has for its pages or the server is
waiting for the disk, so that its reads stay in memory. The compact runs
beside the queries, and takes the write lock only to put the new file in
place.

| Flag | Default | What it does |

| `--auto-compact <ratio>` | 0.5 | The share of the file that may be dead before a compact runs. |

| `--auto-compact off` |   | Compacts only on `compact`. |

Details

Left alone, a file of updated rows grows by a record a write, and its live
records end up a few to a page among dead ones: under YCSB's updates, a
million records of 1 KB took the file to 4.7 to 6.5 GB on an 8 GB machine,
and one read in five then waited for the disk. With the default ratio the
file stays under about twice what the last compact left it at, and each
compact writes the live records once for as many bytes of updates.

What it costs, measured with `make compact-bench`: a million
1 KB records, five million updates of a field at keys drawn at random from
one thread, buffered, while another reads whole records. With
auto-compaction the file went from 1.03 GB to between 2.1 and 3.0 GB and
back to 1.1 to 1.6 GB three or four times; each compact took 8 s at the
median with no lock held and held the write lock 6.1 to 12.6 ms to put its
file in place. The reads while a compact ran had a p99 of 1.3 to 5.3 ms.
The updates ran at about 30 000 a second through a compact, against 110 000
to 170 000 while the file was smaller: on this 8 GB machine, beside its
other programs, the page cache holds about 2 GB of the file, the size at
which a compact comes due, and past it an update reads its row from the disk
whether a compact runs or not -- 28 000 to 46 000 a second with
auto-compaction off. At 250 000 records, whose file fits, the updates ran at
153 000 a second through a compact of 0.3 s, and at no less than 83 000 in
its worst tenth of a second, against 180 000 with none. Once the updates
were done the reads ran at 435 000 a second, p99 2.2 µs, from the first
second. Without it the file ended at 6.2 GB, and the reads after the updates
started at 19 000 a second and took 25 s to reach 400 000, their p99 0.11 ms
over those 30 s. Durable, an fsync a write from 16 writers, 100 000 records
updated 400 000 times at about 900 a second: the file went from 103 to
211 MB and back three times, each compact held the write lock 13.7 to
14.8 ms -- two `F_FULLFSYNC`s of this Mac's among it -- and ended
at 190 MB against 515 without it; that file fits the page cache either way,
and the reads ran alike, p99 2.2 µs.

**Before the file outgrows the page cache.** fenecdb keeps no cache of
its own: a document is read where it lies in the mapped file, so what the
reads need in memory is every page of the file that holds a live record --
and under updates at keys drawn at random that is every page until a
compact (a million 1 KB records updated until the file stood at 2.1 GB
left one on every 16 KB page and on 94% of 4 KB ones). Once the file
outgrows what the system has for it, every read and update waits for the
disk in turn. So a file is also compacted once a third of what the
compact writes is dead, where its live records fit in that memory and the
file has reached three quarters of it -- which Linux reports exactly, its
available memory and a container's limit -- or the server takes 500 major
faults a second, which is how macOS shows it: under pressure it compresses
the other programs' memory and reports more room than it gives the file.
That costs up to four bytes written for each byte updated, against two at
half the file dead. A compact whose new file would not fit beside the old
one writes it past the page cache and reads it in after the swap, so the
old file, which the reads need until then, stays in memory. Measured with
`make compact-bench` in a container held to 2 GB
(`docker run --memory 2g`): at half the file dead the file stood
over 2 GB for 69 to 74% of the run, the five million updates took 358 to 511
s, 3 500 to 4 100 a second at the median, and the process took 811 000 to
900 000 major faults; with the memory rule, 80 to 100 s and 34 000 to
39 000, the file at most 1.9 GB, the reads during a compact p99 0.44 to
0.49 ms against 4.6 to 6.5, for 3.0 to 3.2 bytes written a byte updated
against 1.9 to 2.0. On this Mac beside 1 GB of memory held by another
program, 191 to 260 s against 82 to 535: macOS gives a growing file more
of the machine as it compresses the other programs' memory, which it did
in the fastest run at half the file dead and not in the slowest, and a
file kept smaller is not given it -- steadier, never the best case. A file
that fits in memory is compacted as before.

The compact copies what it reads under the read lock, writes the live
records into a side file with no lock held, and copies the writes made
meanwhile in rounds -- a few thousand documents looked up under the read
lock at a time and copied once it is let go -- until a round finds fewer
than a thousand; it reads every page of the new file in before the swap,
asking for them ahead in large reads, so no reader waits for one after it --
unless the two files would not fit in memory together, when it writes the
new one past the page cache and reads it in once the old one is gone.
The write lock is then held for the last of those writes, the side file's
fsync and the rename. The server looks for a compact due every 5 s, and a
look waits for a write under way rather than pass a compact over. A replica
compacts its own file the same way: a compact changes no document, and an
image its primary sends meanwhile fails the compact rather than be written
over. `/_metrics` has the file's size,
what a compact would give back of it and the compacts run
([Monitoring](https://fenecdb.com/docs//docs/monitoring)), and each is logged with the
file before and after and how long it held the lock.

## Limits and health checks

| Flag | Default | What it does |

| `--max-connections` | 100 | Refuses connections past it. |

| `--max-memory <MiB>` | off | Refuses writes that would grow the data past it; reads go on. |

| `--idle-timeout <s>` | 60 | Closes a quiet keep-alive connection (0 = never). |

| `--http-max-streams` | 64 | Ceiling on a database's subscriptions, counted apart; on a tenant node, each tenant's. |

```bash
fenec-server --ping --http 127.0.0.1:8080   # exits 0 when the server is up
```

Details

One thread per connection means unlimited connections would leave the stack
memory ceiling to the client. Past `--max-connections` a client
gets 503; `--idle-timeout` closes a quiet connection. A
subscription holds a thread for as long as it lasts, so subscriptions are
counted apart as well, under `--http-max-streams`, a database's
own: on a tenant node each tenant's, where `--max-connections` is
what all of them share, and each tenant's streams and held writes take a
quarter of it at most unless its quota says otherwise
([quotas](https://fenecdb.com/docs/sharding#quotas)).

`--max-memory` puts a ceiling on the data footprint. Above it,
requests that **grow** the data are refused with 507, while reads,
`del` and `compact` keep working: hitting a ceiling with
no way out would be no better than a cgroup OOM. The point is to bring the
OOM forward: nobody warns you about a process felled by `SIGKILL`,
and both the last sync and the checkpoint are lost with it. The measurement
happens before the statement, so the overshoot is at most one
statement.

`--ping` asks `GET /_health` on the
`--http` address and exits 0 when it answers. The route needs no
token and takes no lock: every query takes the database lock first, so during
a long `compact` a probe that ran one would wait too and a healthy
server would look dead. Because the listener binds after the file is
opened, "healthy" also means "the documents are read".

What the server is doing is at `/_metrics`, in Prometheus's
text format: `--metrics <address>` opens a listener for it
alone, and `--slow-ms <ms>` logs every statement at least
that slow with its text, a JSON line on stderr. The metrics, a Grafana
dashboard, the log and the Datadog Agent's configuration are in
[Monitoring](https://fenecdb.com/docs//docs/monitoring) and [Datadog and
other monitoring systems](https://fenecdb.com/docs//docs/datadog).

## Run it in a container

The image is under 3 MB: the binary and nothing else.

```bash
docker run -d --name fenecdb -v fenecdata:/data \
  -p 127.0.0.1:8080:8080 -e FENEC_HTTP_TOKEN=a-token \
  ghcr.io/fenecdb/fenec-server
```

The image's command is `--http 0.0.0.0:8080 --file
/data/data.fenec --sync 250`.

- **Set a token.** Without one it refuses to listen beyond
    localhost.

- **Run one instance per file.** Two writers corrupt it: on Kubernetes,
    a `StatefulSet` with one replica.

- **Write every flag out** when you override the command, or the data
    file is dropped.

Details

The `Dockerfile` is two-stage: a static musl build copied into
`scratch`. There are no dependencies and the binary runs its own
health check (`--ping`), so the runtime image holds nothing but the
binary — no shell, no package manager, no libc. The image is **2.99 MB** on
amd64 and 2.44 MB on arm64 — since no target is written in, both come out of
the same file. `make docker` builds it, and
`docker compose up -d` starts the compose file's.

**The memory limit comes from the data.** With no page cache there is
no knob to tune: the file is mapped and what the process holds is the
indexes, which is what `--max-memory` counts. Below that the
cgroup sends `SIGKILL`, the clean shutdown never runs and up to
one sync interval is lost. Set `--max-memory` to a third of the
container limit to bring that forward.

**The restart cost depends on how it stopped.**
`docker stop` sends `SIGTERM` and a checkpoint is
written, so the next open takes 9 ms. After `kill -9` or a
cgroup OOM the graph of the last checkpoint is restored, and the vectors
written since it go in unlinked: the server answers once the documents
are read, and links them on a thread beside its queries,
`near` measuring each of them until then. The only checkpoint
the server writes on its own is the one at shutdown, but it appends its
graphs to the file as they change — a graph that changed in 10 000 nodes,
once the file has grown by three times its record, logged as
`1 graph written into the file`, with writes held off for
the 5 ms that takes at 100 000 × 768 — so what waits is what came
after the last of them: at 100 000 × 768 at most 10 000 vectors, linked
in 2.6 s with `near` at 1.56 ms p50 meanwhile. With no graph
in the file every vector waits: at 100 000 × 768 never checkpointed the
first `near` was answered 1.23 s after the start, against
67.7 s when the open linked them first, and linking takes 19 s.
`fenec_vectors_unlinked` counts down to 0.

Overriding `CMD` drops `--file`, and the container
comes up with an in-memory database. There is no shell in the image, so
`docker exec ... sh` does not work; what is left is
`docker logs` and `docker run --rm
ghcr.io/fenecdb/fenec-server --help`.

On machines of your own, the repository's `deploy/` has a
systemd unit for a server of every role — a tenant node, a region's router,
the global router, an archiver — with its tokens in an environment file,
the alert rules and the operator's runbook (Deploying it).

Two containers loading the same file and writing it corrupt it. On
Kubernetes that means a `StatefulSet` rather than a
`Deployment`, `replicas: 1`, `strategy:
Recreate` and a `ReadWriteOnce` volume — a rolling update
keeps two pods up together for a moment. There is a connection ceiling but no
queue: an excess connection does not wait, it gets 503.

## More

- **Mirror a PostgreSQL table.** `fenec-server --follow
    postgres://...` keeps a copy as the table commits, and serves it.
    See [Import](https://fenecdb.com/docs/import#serving).

- **Embed it instead.** The same engine runs inside a Rust program
    with no server at all ([Embedded Rust](https://fenecdb.com/docs//docs/embedding)), and
    in a browser tab ([JavaScript](https://fenecdb.com/docs//docs/javascript)).


---

Source: https://fenecdb.com/docs/replication

# Replication

Keep read replicas of a server, promote one when the
primary is gone, and back up every write so you can restore to any
moment.

## Start a replica

Give the primary a replication token, then start a second
`fenec-server` on its own file pointed at the primary. The replica
serves reads; when the primary is gone, promote it to take writes.

```bash
# the primary: replicas are fed over its HTTP listener
fenec-server --file primary.fenec --http 0.0.0.0:8080 --http-token "$HTTP" \
         --replication-token "$REPL" --sync always

# a replica, over a file of its own
fenec-server --file replica.fenec --http 0.0.0.0:8081 --http-token "$HTTP" \
         --replication-token "$REPL" --replica-of http://primary:8080

# the primary is gone: the replica takes writes from here on
curl -X POST replica:8081/_replication/promote -H "Authorization: Bearer $REPL"
# {"promoted":true,"seq":41873,"history":"5f0e2c9a81b44d17"}
```

- **Every read works on a replica,** subscriptions included.

- **Writes are refused** until it is promoted, by hand.

- **A failover can lose** only what the primary's `--sync`
    had not yet put on disk.

Details

A primary sends its replicas the writes that have reached its disk, and
each applies them to a file of its own and serves reads from it.

A replica is a whole `fenec-server`: every read works on it,
subscriptions included, from its own copy in memory, and a write gets 403.
`GET /_replication` says which role a server has, so a client or a
load balancer can tell a replica from the primary.

## What is sent

A replica is sent the primary's writes in order and applies each one to
its own file. Only writes already on the primary's disk are sent.

- **A restarted replica** asks only for what it missed.

- **A replica far behind** (past 64 MiB of writes) is sent a whole
    copy of the database instead.

- **`--sync off` is refused** on a server with a
    replication token.

Details

A fenecdb file is already a log: an image, then every write appended as a
record. A replica is sent those records, whole and in order, and applies each
the way the write path made it — the document, and the upkeep of every index
on it — and appends it to its own file. Its file therefore ends as the
primary's does, and a replica stopped and started again reopens where it was
and asks only for what came after. The primary keeps the last 64 MiB of
writes for this (`--replication-buffer`); a replica further behind
than that is sent an image of the whole database instead, and follows write
by write from there.

Only writes on the primary's disk are sent. A write enters the feed when it
is appended and leaves it once an fsync has covered it, so a primary that
crashes comes back holding every write any replica was sent, and a replica
never has to be told to forget one. The price is latency, set by the
primary's `--sync`: under `always` a write goes out right
after its own fsync, under `250` up to 250 ms later. Under
`off` nothing is on disk before shutdown, so a server with a
replication token refuses to start with it.

## Positions and histories

A replica's position is a change number that means the same on the
primary and on every replica. After a promotion the history forks, and a
replica is only continued from a point the new primary also passed
through; otherwise it is sent a fresh copy.

| A replica that… | is sent |

| reconnects to a primary that restarted, even after a crash | the writes it missed |

| was stopped and started again over its file | the writes it missed |

| is pointed at a promoted replica, holding nothing past the fork | the writes it missed, and the new history |

| is pointed at a promoted replica, holding writes past the fork — the old primary, say | an image |

| is further behind than the primary keeps writes for | an image |

| starts empty | the writes from the first, or an image |

Details

Every write moves the change counter by one, on the primary and on each
replica alike, so a replica's position is its own counter and a subscriber's
cursor means the same change on either. A block — the writes of one statement,
a `/batch` — is one record numbered by its last
write, and is sent and applied whole: a replica never stands inside one.
A replica of a replica is fed exactly as its primary was.

Numbers alone stop being enough once a history forks. A replica promoted
after its primary died writes its own change 1 001; the old primary, or a
second replica that had got further along, may hold a different 1 001.
Streaming the new primary's 1 002 on top of that would go unnoticed. So each
fork gets a fresh id and remembers the change where it left its parent, as
PostgreSQL numbers its timelines, and a replica is continued only from a
position the primary's own history passed through, as the table above
shows.

The history lives in the file ([record kind
8](https://fenecdb.com/docs//docs/file-format)), and the first time a primary serves replicas it draws an id of its
own — the root history is every database's, so it names none.

## Reading your own writes

A replica can be a moment behind, so a read sent right after a write may
miss it. Send the number a write's answer carries in
`Fenec-Seq` as `Fenec-After`, and the replica waits
until it has that write.

```bash
curl -i -X POST http://primary:8080/query -d '{"query":"put carts {user: 7}"}'
# Fenec-Seq: 1042

curl -H 'Fenec-After: 1042' -X POST http://replica:8080/query \
  -d '{"query":"get carts where user = 7"}'
```

- **It waits 5 seconds** unless `Fenec-Wait` says otherwise
    (in milliseconds, 30 s at most), then answers 504.

- **A subscription and `/_changes` wait too,** before
    their head: a stream sent with the header is seeded with the write, and
    a change-stream page's `Fenec-Seq` is never behind it.

- **Across a failover, send `Fenec-History` too:** the
    answer names the history its change is of, which a promotion forks. A
    replica left on the old branch holds a change of that number of its
    own, and answers no read sent after the new one.

- **Its token comes first:** a request the server refuses (401, or
    403 for another tenant's token) is answered at once, not after the
    wait, and learns nothing of where the replica stands.

- **A read marked `Fenec-Read: replica` names where it
    read:** the answer carries the copy's `Fenec-Seq` and
    `Fenec-History`, so a read sent after it never reads older
    ([Read replicas](https://fenecdb.com/docs/sharding#read-replicas)). A read nobody
    marked names nothing.

- **The Python client** keeps the number for you:
    `client.seq`, sent with `query(..., after=...)`.

Details

A replica is behind its primary by the time a write takes to reach it —
0.04 ms at the median under `--sync always`, up to the sync
interval otherwise — so a client that writes on the primary and reads on a
replica can read from before its own write. Over HTTP a write's answer
names the change it left the database at, `Fenec-Seq`; a request
sent to a replica with `Fenec-After: <that number>` is
served once the replica holds it, and answered 504 with where the replica
stands if it does not within the wait — never from before the write.

A change number names a write within one history. A promotion forks the
history where the replica promoted stood: a write the old primary answered
and the replica never got is lost, the new primary's next write takes its
number, and a replica still following the old primary holds a change of that
number — the lost write. So an answer of a database that has had a replica
also names its history, `Fenec-History: <term>-<id>`,
the term one more at each promotion; a read sent with it beside
`Fenec-After` is served once the replica holds that change of that
history — a replica on the old branch waits until it follows the new primary
and takes its image. A primary asked for a position its history does not
hold — the write lost — answers 504 at once, with its own
`seq` and `history`, which a client takes as where it
stands from then — over the position it sent that read after alone, and
never an earlier term of the same lineage, which only a fenced old primary
names. A history's lineage is the top half of its id: every promotion and
restore keeps it, and a database made anew — a tenant deleted and made again
under its name, starting again at term 1 — has another, whose position a
client takes, its term lower or not.

The wait is 5 seconds unless `Fenec-Wait` gives it in
milliseconds, 30 seconds at most, and a replica waits on the same
`Condvar` its subscriptions do, so nothing polls. On a tenant node
a wait takes one of the tenant's streams, but only once it must wait: a read
after a write the node holds already takes none, as a tenant whose streams
its own subscriptions fill still reads its writes. Writing the
header costs the server nothing measurable; a client that parses every
header, as Python's `http.client` does, reads one more.

## Promotion

Promote a replica by hand: `POST /_replication/promote` on its
HTTP listener, or restart it over its file with `--promote` in
place of `--replica-of`. Keeping the old primary from taking more
writes is up to you.

Details

Either way forks the history where the replica stands and turns writes on.
A replica's file does not open any other way — started with neither flag,
`fenec-server` refuses and says which to give — because a replica
taking writes without a fork would continue its primary's history with
writes the primary never made.

Making sure the old primary takes no more writes is the operator's part, as
it is with PostgreSQL: nothing here fences it. Brought back, it rejoins under
the new primary with `--replica-of`, and is sent an image if it
wrote past the fork.

## What a failover loses

With `--sync always` on the primary, no acknowledged write was
lost in ten failovers. With the default `--sync 250`, a failover
loses up to the last 250 ms of writes.

| primary's `--sync` | acknowledged writes/s | lost per failover |

| `always` | 284 | 0 in all ten |

| `250` | 53 565 | 9 700–10 788: the last ≤ 201 ms |

Details

How it was measured: a primary killed with `SIGKILL` while one
client writes to it as fast as it answers, the replica promoted half a second
later, ten times per policy; counted are the writes the client was told
succeeded that the promoted replica does not hold (`make
replica-bench`, macOS, loopback).

Under `250` the loss is the policy's own: those writes were
answered before any fsync and were not on the primary's disk either. Under
`always` a write is answered after its fsync and once the stream
to each replica has written it into its socket, which the kernel delivers
after the process that wrote it is killed. Sent after the answer, as it was
before, a write answered in the fraction of a millisecond before it
was sent was lost: none of these ten failovers caught one, while a node of
the SaaS example killed under eight writers lost one in 3 of 60 runs, and
none in 120 since. A stream that has not sent a write within a second is
not waited for until it catches up again, so a stalled replica costs the
writes one wait. A machine that dies takes its sockets with it: this holds
for a process killed, not for a power loss. No failover left the replica
holding a write the client had not been answered for.

## What it costs

A write is readable on a replica 0.04 ms after the primary answered it
under `--sync always`, and within the sync interval otherwise. A
replica keeps up with a bulk load.

| primary's `--sync` | p50 | p95 | p99 | max |

| `always`, writes back to back | 0.04 ms | 0.06 ms | 0.09 ms | 2.77 ms |

| `250`, 250 writes a second | 148 ms | 262 ms | 273 ms | 282 ms |

Details

How it was measured: primary and replica in one process over loopback, each
over its own file; lag is from a write's answer on the primary to the write
being readable on the replica, over 1 000 writes.

A replica keeps up with a bulk load: 100 000 documents of 128 dimensions
with an HNSW index, which the primary wrote in 23.4 s in statements of 1 000,
were applied from the stream in 19.7 s — 5 088 writes a second, the graph
built as they arrive. A new replica of that database sent it as an image —
58.3 MB, the graph in it — answered from it 0.17 s after it connected.

The primary keeps the writes in memory up to its buffer, and copies the
image it sends; a replica holds the database in memory as a primary does. The
replica's HNSW graph is built from the same vectors in the same order, but not
in the same batches, so an approximate `near` can rank a close call
the other way on the two; everything else answers identically.

## Backups and restoring to a moment

An archive is the replication stream written to a directory: base images
and every write after them, timestamped. From it, `fenec restore`
rebuilds the database as it stood at any moment or change.

```bash
# the running database, taken whole: a file that opens as a database of its own
fenec backup http://primary:8080 nightly.fenec --token "$REPL"

# every write that reaches the primary's disk, until interrupted; an image of
# its own every hour, and what no restore to the last week needs let go of
fenec archive http://primary:8080 /backups/archive --token "$REPL" --image-every 1h --keep 7d

# what the archive can restore, read as a restore would; exits 1 if it cannot
fenec verify /backups/archive

# the database as it stood just before the mistake
fenec restore /backups/archive before.fenec --to 2026-09-22T10:14:59Z
fenec restore /backups/archive before.fenec --to-change 41873
```

- **`--to`** stops at a moment; **`--to-change`**
    after a change number.

- **`--keep`** (or `fenec prune`) deletes what no
    restore within that window needs.

- **`fenec verify`** says from when to when the archive can
    restore.

Details

The same stream written to a directory instead of applied is an archive:
base images, and every write after them with the time the primary appended
it. The `fenec` command line writes and reads one.

A restore takes the latest image at or before the point asked for and
appends the archived writes after it up to there — a fenecdb file is an image
and the writes after it, so that is the whole restore — then opens the result
to check it, forks its history and checkpoints it. `--to` keeps
every write the primary appended at or before that moment and stops at the
first one after it; `--to-change` stops after that change. A
statement writes each document as a change of its own, so a `del`
of ten documents is ten changes.

The archive is fed as a replica is: only writes that were on the primary's
disk, so none a crash took back, and a `compact` on the primary
loses it nothing — the writes were archived before a compact folded them into
an image. An archiver that falls further behind than the primary keeps writes
for takes a new image and goes on from there; the archive then holds no write
between the two images, and a restore to a point in that gap says so rather
than inventing one. Restoring the 100 000 × 128 load above from an image and
its 100 000 writes took 19.9 s, most of it building the graph for them, and the
file then opened in 63 ms; to the load's middle, 7.8 s. A base image a day
keeps a restore to a day of writes.

```text
image-<change>-<time>.fenec   a base image: the database at that change
writes-<first change>.log      per record, the time it was appended and the record
history                        the history the archive is on
```

An archive takes images of its own: every `--image-every` (an
hour unless given), once a write came after the last one, it opens its last
image and the writes after it and adds an image of the database there — the
primary is asked for nothing — named by the time of its last write, and the
segment being written ends so that the writes before the image are in
segments of their own. A restore then replays at most that much. With
`--keep`, what no restore within that window needs goes after
each image: the images before the newest one taken by the window's start,
which a restore to that moment starts from, and the segments whose writes
all came before the oldest image kept — never the one being written.
`fenec prune <dir> --keep 7d` does the same once.

`fenec verify <dir>` reads the archive as a restore
would: every image opens at the change its name says, the segments run on
from the oldest image with no change missing, only the last segment may end
in a record cut short, and a restore to the end opens. It prints from when
to when the archive can restore, names any gap — changes in no segment
between two images, where a restore to the end passes but a restore to a
moment inside cannot — and exits 1 when no restore can be made.

## Archiving every tenant of a node

A node of tenants ([`fenec-server --dir`](https://fenecdb.com/docs//docs/sharding))
is archived by one process over one connection: an archive a tenant under a
root, each the archive above, which `fenec restore`,
`verify` and `prune` read as they read any.

```bash
# every tenant of the node into /backups/n1/<tenant>/, until interrupted
fenec archive --tenants http://n1:8081 /backups/n1 --token "$REPL" --image-every 1h --keep 7d

# every tenant's archive read as a restore would; exits 1 if any cannot
fenec verify --tenants /backups/n1

# the node's directory as it stood at a moment: <tenant>.fenec for each tenant it held then
fenec restore --tenants /backups/n1 /data/n1 --to 2026-09-22T10:14:59Z
fenec-server --dir /data/n1 --replication-token "$REPL" ...
```

- **One connection and one thread on the node**, however many
    tenants it holds, and no tenant held open: an idle tenant stays
    closed. One tenant's image or a slow archiver holds up no other
    tenant's writes.

- **A tenant made on the node is archived from its first image**;
    one deleted or moved to another node is no longer followed, and its
    archive is kept, noted gone.

- **A tenant restored to an earlier moment, or made again under its
    name,** begins a new archive; the one before is kept beside it, and
    answers for the time before.

- **`--key-file`, `--image-every` and
    `--keep`** apply to each tenant's archive.

- **Under load the archive trails the node.** Under `--sync
    always` a write is answered once the stream has sent it, as a
    replica's stream would, but only while the stream keeps up: while it is
    busy — a tenant's image, a closed tenant opened, a tenant's lock waited
    for, an archiver reading slower than the node writes — writes are
    answered without it. Lose the node's disk then and the archive lacks
    them; where the node survives they reach the archive later, at worst
    inside an image. Under `--sync 250` it trails anyway, as a
    replica does.

Details

A tenant's own stream (`/t/<tenant>/_replication`) is a
connection and a thread on the node for as long as it runs, one of
`--max-connections`, and it holds its tenant open the while.
Archived that way, a node of a thousand tenants holds a thousand of each and
every tenant open for good — and at the default of 100 connections it
cannot be archived at all. `fenec archive --tenants` asks the node
for `POST /_replication/tenants` instead, with the replication
token: every tenant's writes over one stream, each message led by its
tenant's name. Its body says where each archive under the root stands; the
node says hello to every tenant it holds — where the archive goes on from,
or an image where its place does not lead to the tenant's writes — and that
every tenant the archive lists and the node does not hold is gone.

The stream sends a tenant's writes from its feed while it is open, and
nothing while it is closed, whose file does not change. A tenant closes only
once the stream has sent what its disk holds — closed with writes unsent,
its feed would go with it and they would come again as an image — and is
looked at again at the next pass, never waited for. Under `--sync
always` a write waits for the stream as for a replica while the stream
keeps up; whatever can keep it busy — reading or opening a closed tenant,
waiting for a tenant's lock, which its longest write holds (a
`/batch`, a compact under the lock), an image's snapshot and its
sending, an archiver reading slower than the stream writes — it does held
up, and no write waits for it then. One stream for every tenant held each
tenant's next write for a second while it opened another one; each tenant's
own stream holds up its own tenant alone. An image waits for its tenant's
disk without holding up the stream — and is taken again, from the tenant as
it is then, where the tenant closed and opened again meanwhile — and the
node's syncer and a closing tenant wait for no stream at all, since no
client's answer waits on them.

So the archive of a node is synchronous only while its stream keeps up,
and under load it is not: a write answered while the stream was held up, or
behind after an image, is on the node's disk and not yet in the archive. It
is sent once the stream comes back to its tenant, or, where the stream fell
behind the tenant's feed, as part of an image; if the node's disk is lost
before that, the archive does not have it. A replica's own stream
(`/t/<tenant>/_replication`) keeps a write's answer waiting
for it, up to a second.

Which tenants have writes is told, not polled: a tenant's feed tells the
stream as its writes reach the disk — with no stream connected a sync pays
two loads, the same 13 ns a durable advance either way — and a tenant made,
installed, deleted, promoted or made to follow tells it too; one released
from under the stream is looked at again until it is gone or back. Where a
closed tenant stands is known where it closed in the node's process, and
read from its file otherwise — the first stream after the node starts: 1 001
tenants in 0.29–0.33 s, where an open, which fsyncs the file first, was
6.3 ms a tenant. A closed tenant the stream is told of is read again rather than
taken where it closed, since a holder of its database — the
`@ttl` sweeper — may have written it since; one with a sync log
beside it, after the machine went down, is opened.

A tenant's archive is `<root>/<tenant>/`, written
as `fenec archive` writes one; its segment file is open only while
the tenant writes, so the archiver holds a descriptor a writing tenant, not a
tenant. When the node says a tenant is gone — deleted, or moved to another
node, whose own archiver takes it up from its image on the same history —
the archive is kept as it stands and
`<root>/<tenant>/presence` notes the time; a tenant
that comes back is followed again and noted back, the time it was away a
gap. The times are the node's clock, as its writes' are: gone is when the
node deleted the tenant, though the stream was busy or the archiver away
when it did — or, where the node does not know (it restarted since), when
the stream saw it go — and back is when the node said hello to it again. A
tenant deleted and made again while the stream was busy is noted gone at the
delete before the new copy's hello; one deleted and made again while the
archiver was away is heard of as the new copy alone, and the line it moves
aside (below) is noted gone by that hello at the latest, so a moment between
the delete and then restores the old copy. A tenant whose copy on the node no longer leads from its archive's
place, at or before it — restored there to an earlier moment, or another
tenant made under the name — would land its writes among changes of the same
numbers: the line so far is moved aside to
`<root>/<tenant>.<history>/` and a new one begins.
(`fenec archive` of one database refuses such a primary instead,
and leaves its archive as it was.) A promotion's new history reaches the
archive before the writes after it, so a promoted tenant's archive goes on
without an image. `<root>/archiving.lock` keeps a second
archiver off a root.

`fenec restore --tenants <root> <dir>` writes
`<dir>/<tenant>.fenec` for each tenant the node held at
`--to` — from the line of its history that began last by then, so
a line moved aside answers for the time before the next began — or at each
archive's end. A tenant the node said was gone then, deleted or living on
another node, is passed over, and so is one archived only after it; one
whose moment `--keep` let go of is refused, and the restore exits
1 — each line knows when it began, which the prune that lets its first
image go writes down first (`began`). `--to-change` names one database's change, and is
refused. `fenec verify --tenants` and `fenec prune
--tenants` run on every line, those moved aside too, and verify names a
gap an image alone ends.

Measured with `make node-archive-bench`: a node of 1 000
tenants of ten rows each and one of 64 MB, closed when idle, ten of them
written ten times a second from before the archiver connected and for 20 s
after, under `--sync always`, in turns, on this M1:

|   | no archiver | a stream each | the node's stream |

| node: threads | 15 | 1 016 | 16 |

| node: descriptors | 27 | 4 011 | 29 |

| node: tenants open | 10 | 1 001 | 10 |

| node: CPU, of a core | 4.7–5.1% | 7.0–8.0% | 3.9–6.0% |

| archiver: threads | — | +1 001 | +1 |

| archiver: descriptors | — | 2 021 | 21 |

| archiver: CPU, of a core, the writers' 1.2–1.3% in it | — | 3.4–4.2% | 2.1–2.9% |

| first images of the 1 001 | — | 11.2–15.5 s | 14.8–15.2 s |

| writes as they were taken, p50 / p99 / max | — | 28–33 / 176–600 / 436–1 113 ms | 14.5–14.7 / 32–35 / 43–116 ms |

| writes after, p50 / p99 | 11.8–12.1 / 22.7–26.4 ms | 13.0–14.2 / 28.7–55.4 ms | 12.4–13.8 / 25.6–33.8 ms |

| node restarted: memory, tenants open | — | 121–157 MB, 1 001 | 2.2–4.6 MB, 0 |

| `--sync 250`: the syncer's pass, p50 / max | 30.7–32.6 / 45–63 ms | 65–86 / 84–353 ms | 52.3–52.7 / 91–168 ms |

| `--sync 250`: a tenant's close, p50 / max | 3.8–4.0 / 12–25 ms | — (none closes) | 4.0 / 15–33 ms |

The archiver's process runs the bench's writers as well. A write's time
under `--sync always` is the Mac's `F_FULLFSYNC`; a
write waits for the node's stream as for a replica, and the stream sends
within the fsync's noise. While the first images were taken no write
waited for the node's stream; held up as a replica's stream is, it held
every written tenant's next write for a second whenever it opened another
tenant. A stream each held the writes up as images went out over a thousand
sockets at once, the build before alike (p99 86–351 ms, at most 110–994).
The node's stream
takes a new archive's first images one tenant at a time, each tenant opened
— an fsync — and its image written: about 15 s for the thousand and one,
once, the 64 MB one among them. After the node restarted, the archiver
connecting again, the stream said hello to all 1 001 tenants in 0.29–0.33 s,
each read from its file rather than opened; a thousand streams opened every
tenant again and held it.

Under `--sync 250` (`node_archive_passes`: eight
tenants written every 5 ms, eight more opened, written and closed in turn,
one of 32 MB closed) the syncer's pass took 52 ms at the median beside the
node's stream against 31–33 with none: the archiver writes the same disk,
and the passes are its fsyncs and the node's together. Before the syncer
and a close stopped waiting for streams, a stream busy with one tenant held
a pass for a second a tenant, 5.06 s over five, and a close under its
lock for 1.01 s. Against the build before, in turns: with no archiver the
syncer's pass took 31.0–31.4 ms at the median and a close 3.6–3.9 ms, and
with a stream each the pass 67–73 ms, as here; the writes with no archiver
or a stream each as in the table.

## In object storage

To keep an archive in S3, R2 or GCS, run a sync tool beside the archiver;
fenecdb has no object-storage client of its own.

```bash
rclone sync /backups/archive r2:backups/archive      # every few minutes
aws s3 sync /backups/archive s3://backups/archive --delete

# to restore: bring it back, check it, restore it
rclone sync r2:backups/archive ./archive && fenec verify ./archive
fenec restore ./archive before.fenec --to 2026-09-22T10:14:59Z
```

Details

fenecdb speaks no TLS, and S3, R2 and GCS speak only HTTPS, so an
archive goes there the way any directory does: a sync tool beside the
archiver.

Every file but the segment being written is written once: a segment is
closed and fsynced before the next begins, and an image is written beside
and renamed into place. A sync tool lists the directory and then copies
each file as it stands, so the copy it leaves is the archive up to a
moment, its last segment at most cut in a record's middle — which
`verify` reports and a restore passes over. A test copies an
archive a file at a time while writes and images go on, and every copy
verifies and restores to a prefix of what the primary wrote.
`--delete`, or `rclone sync`'s own, carries a prune
over to the copy; without it the copy keeps everything.

## Sealed archives and backups

Give a key file and every archive file and backup is encrypted and
tamper-checked. Keep the key apart from the archive: without it nothing can
be restored.

```bash
fenec key /secrets/archive.key          # 64 hex digits, readable by its owner alone
fenec archive http://primary:8080 /backups/archive --key-file /secrets/archive.key
fenec verify /backups/archive --key-file /secrets/archive.key
fenec restore /backups/archive before.fenec --to 2026-09-22T10:14:59Z \
  --key-file /secrets/archive.key

fenec backup http://primary:8080 today.sealed --key-file /secrets/archive.key
fenec restore today.sealed today.fenec --key-file /secrets/archive.key
```

Details

A copy in a bucket or on another disk is out of reach of the machine's
disk encryption. With a key every file of an archive — its images, its
segments, its history — and a backup file are sealed: encrypted and
authenticated with ChaCha20-Poly1305 (RFC 8439), a random nonce a frame,
each frame's place in the file and whether it is the last under its tag.
A byte changed, a frame moved or dropped, a file cut short where it was
sealed whole, or the wrong key, and the file is refused — never read as
something else. A segment is sealed a frame for each batch of writes the
primary sends, so a crash cuts off at most the frame being written, as it
cut a record short before.

An archive is sealed or it is not; a file of the other kind in
it is refused. `FENEC_KEY_FILE` names the key file too. Sealing
and opening run at about 400 MB/s on one core, where writing the archive
waits on the network and the disk. The database file itself stays plain:
it is mapped and read where it lies, which encryption would take away —
encrypt the disk it is on (LUKS, FileVault, an encrypted cloud volume).

## Endpoints

Three endpoints — a `--dir` node's tenants each have them
under `/t/<tenant>/`, and the node one more, the stream of
every tenant — all on the HTTP listener and all wanting the replication
token, not the `--http-token`.

```http
GET  /_replication?since=<change>&history=<id>   the stream a replica is fed
GET  /_replication/status                          role, position, replicas, lag
POST /_replication/promote                         a replica's promotion
POST /_replication/tenants                         on a --dir node: every tenant's writes
                                                   over one stream (fenec archive --tenants)
```

Details

All of them want `Authorization: Bearer <replication token>`,
not the `--http-token`: the token is for replicas and for whoever
promotes them. The status of a replica names its primary and says whether it
is connected, how many writes it is `behind` the primary's disk,
and how long ago it last heard from it; a primary's lists the replicas it is
feeding and how far each has been sent.

## Flags

| `--replication-token <value>` | turns replication on; needs `--file`, and `--http` to feed replicas. Also read from `FENEC_REPLICATION_TOKEN` |

| `--replica-of http://host:port` | follows that primary into `--file`; needs the token |

| `--promote` | opens a replica's file to take writes, forking its history |

| `--replication-buffer <MiB>` | writes kept for replicas that fall behind, and for `/_changes`; default 64 |

| `--cdc` | keeps the writes for [`GET /_changes`](https://fenecdb.com/docs//docs/http#changes) with no replica fed; on already with `--replication-token` |

## What it does not do

- **No synchronous replication.** A write is answered when it is on the
    primary's disk, never when a replica has it.

- **No automatic failover of a single file.** Promotion is by hand,
    and so is keeping the old primary from taking writes. A router's tenants
    can have one: nodes that take its lease stop writing when they lose it,
    and are failed over after ([Automatic
    failover](https://fenecdb.com/docs//docs/sharding#automatic-failover)).

- **No TLS.** The stream is plain HTTP, like every other listener
    here: across an open network, a tunnel carries it.

- **No object-storage client.** An archive is a directory; a sync
    tool carries it to S3, R2 or GCS ([In object
    storage](#object-storage)).


---

Source: https://fenecdb.com/docs/monitoring

# Monitoring

fenec-server counts every statement it answers and serves the
counts to Prometheus, with a Grafana dashboard ready to load.

## Turn it on

Give `--metrics` an address and a token, and point Prometheus at
`/_metrics` there. Counts cover every request, reads and
writes apart, with how long each took.

```bash
# /_metrics on a listener of its own, away from the data's
fenec-server --file data.fenec --http 127.0.0.1:8080 \
         --metrics 0.0.0.0:9464 --admin-token "$SCRAPE" --slow-ms 100

curl -H "Authorization: Bearer $SCRAPE" http://127.0.0.1:9464/_metrics
# fenec_statements_total{kind="read"} 1834
# fenec_statement_duration_seconds_bucket{kind="read",le="0.001"} 1790
# fenec_documents{collection="people"} 90000
# ...
```

- **With `--http`**, the HTTP listener serves
    `/_metrics` too.

- **A scrape needs a token:** the server's (`--http-token`)
    or the admin one (`--admin-token`).

- **Datadog, New Relic, Elastic, Grafana Cloud** read the same
    metrics and logs: [Datadog and other monitoring
    systems](https://fenecdb.com/docs//docs/datadog).

Details

With `--http` the HTTP listener serves `/_metrics`
as well, and a `--dir` node serves it there. The path carries an
underscore, as `/_admin` and `/_replication` do: a
collection may well be called `metrics`, and
`GET /metrics` is how its rows are read. Prometheus takes the path
as `metrics_path`.

A scrape needs the server's token (`--http-token`) or the admin
token (`--admin-token`); with neither and no JSON Web Tokens the
server is open, and so are its metrics. A JWT is not taken: a user held to a
policy must not learn the other collections' names from a scrape. A
`--metrics` address outside loopback wants one of the two tokens,
as the HTTP listener does, or `--insecure`.

## What is counted

Each statement is counted as a read or a write, with its
latency as the client saw it. Collections, memory, replication and tenants
are reported as gauges.

| Metric | Type | What it says |

| `fenec_statements_total{kind}` | counter | statements answered; `kind` is `read` or `write` |

| `fenec_statement_errors_total{kind}` | counter | statements answered with an error, the client's (a typo, a 404) or the server's |

| `fenec_statement_duration_seconds{kind}` | histogram | arrival to answer, in buckets from 100 µs to 10 s |

| `fenec_slow_statements_total` | counter | statements over `--slow-ms`, each also in the log |

| `fenec_refused_total` | counter | requests refused for their token (401), each also in the audit log |

| `fenec_connections` | gauge | connections open now; a scrape is not one |

| `fenec_documents{collection}` | gauge | documents in a collection |

| `fenec_data_bytes{collection}`, `fenec_dead_bytes{collection}` | gauge | live records, and the overwritten and deleted ones `compact` gives back |

| `fenec_file_bytes`, `fenec_reclaimable_bytes` | gauge | the file as written, and what a compact would give back of it now: the dead records, a record's head a write, the graph records a server appended in the tail; `--auto-compact` runs one once it passes its share ([Compaction](https://fenecdb.com/docs//docs/server#compaction)) |

| `fenec_auto_compactions_total` | counter | compacts the server ran on its own, every file it serves together |

| `fenec_memory_bytes` | gauge | what the process holds of the database and `--max-memory` counts: the indexes, and the documents too where the file is not mapped (`--no-mmap`) |

| `fenec_change_sequence` | gauge | the last write's sequence number; its rate is the writes per second |

| `fenec_storage_failed` | gauge | 1 once the disk refused a write: writes stop until the file is reopened, reads go on |

| `fenec_vectors_unlinked` | gauge | vectors the open left out of the graph after a crash: `near` measures each one until a thread beside the queries has linked them, and the gauge falls to 0; on a `--dir` node, the open tenants' together |

| `fenec_replicas` | gauge | on a primary, the replicas streaming from it |

| `fenec_replica_connected`, `fenec_replica_behind` | gauge | on a replica: whether it hears its primary, and the writes on the primary's disk it has not applied |

| `fenec_replica_last_contact_seconds` | gauge | since the primary was last heard from, a keep-alive included |

| `fenec_replica_images_total`, `fenec_replica_reconnects_total` | counter | whole images taken, and connections made again |

| `fenec_tenants`, `fenec_tenants_open`, `fenec_tenants_disk_bytes` | gauge | on a `--dir` node: tenants on disk, open, and their files together |

| `fenec_build_info{version}`, `fenec_start_time_seconds` | gauge | the version running, and when the process started |

Details

A statement is timed from its arrival to its answer: the wait for the lock
is in it, and under `--sync always` the wait for the disk, since
that is the latency a client sees. A statement is a request, and
`/batch` is one however
many statements it carries. A statement that took the write lock is a write,
so a `get` sent through `POST /query` is a read.

A tenant node counts its tenants and not what they hold: a tenant's
collection names are not the node's to publish, and a label per tenant would
be as many series as there are customers. The statement counters are one set
per process, and start again at zero with it, which `rate()` takes
in its stride.

## Grafana

The repository's `monitoring/` folder runs fenec-server with
Prometheus and Grafana and the dashboards already loaded: the server's
metrics, and the database's own rows through fenecdb's data source plugin
and through the Infinity plugin.

```bash
FENEC_HTTP_TOKEN=secret FENEC_METRICS_TOKEN=scrape \
FENEC_JWT_SECRET=$(openssl rand -hex 32) \
  docker compose -f monitoring/docker-compose.yml up -d --build
# metrics: http://127.0.0.1:3000/d/fenecdb     data: http://127.0.0.1:3000/d/fenecdb-data-plugin
# Prometheus: http://127.0.0.1:9090
```

- **The metrics** go fenec-server → Prometheus → Grafana, behind the
    admin token: the dashboard is `grafana/fenecdb.json`.

- **The data** goes Grafana → the `fenecdb-datasource`
    plugin → `POST /query`, each panel a FenecQL statement with
    the time range and variables bound as its parameters, under a token
    that reads two collections and writes nothing, with an alert rule over
    the same rows: [Grafana](https://fenecdb.com/docs//docs/grafana).

Details

`monitoring/` in the repository holds a Grafana dashboard
(`grafana/fenecdb.json`), a Prometheus configuration and a compose
file that runs fenec-server with both, the dashboard already loaded. The
same compose seeds a site's events and a week of trades and loads the data
dashboards (`grafana/fenecdb-data-plugin.json` and
`grafana/fenecdb-data.json`) beside it.

It opens on the figures that decide whether anything needs doing — the
version and uptime, whether the disk has refused a write, statements and
errors per second, the memory footprint — then statements by
kind with their p50 and p99, errors, and slow statements a minute; documents,
writes a second and each collection's dead share, the one that says when to
`compact`; replication; tenants; and a router's requests, its
latency beside its nodes', the nodes it could not reach, the tenants on each
node and its moves. The instance variable picks one server or all of them.
Imported on its own, the dashboard asks for a Prometheus data source.

```text
# the p99 of reads, over five minutes
histogram_quantile(0.99, sum by (le) (rate(fenec_statement_duration_seconds_bucket{kind="read"}[5m])))

# worth an alert
fenec_storage_failed == 1
fenec_replica_behind > 10000
fenec_dead_bytes / (fenec_data_bytes + fenec_dead_bytes) > 0.5
```

## Slow statements

`--slow-ms 100` logs every statement that took 100 ms or
longer, with what it ran, as a JSON line on stderr.

```json
{"at":"2026-10-05T09:30:12.310Z","event":"slow","level":"warn","proto":"http","peer":"10.0.0.7:53112","request_id":"9f2c41d07ab35e18","duration_ms":312.408,"kind":"read","failed":false,"statement":"POST /query {\"query\":\"get people select name order name collate tr\"}"}
```

Details

The log line uses the same timing the histogram has. It is the audit
log's shape — `at`, `event`, `level`, the
peer and the [request id](#request-id) — with the duration in
milliseconds, whether the statement wrote (`kind`), whether it
failed, the tenant on a tenant node, and the statement, so one parser reads
both logs ([Datadog and other monitoring
systems](https://fenecdb.com/docs//docs/datadog#logs)). `--slow-ms 0` logs every statement, as PostgreSQL's
`log_min_duration_statement = 0` does.

A parameter's value is not logged, only how many there were: a value bound
as `$1` stays out of the log, a literal written into the text does
not. Over HTTP the line is the method and the target, and for
`/query` and `/batch` the body that holds the
statements. A line is cut at 1 000 bytes, so a bulk `put` of a
megabyte shows what it was without being all of it.

## What each statement cost

Statements are also grouped by shape — the same text with other values is
one line — with call counts, times, rows and failures, at
`/_stats/statements`.

```bash
curl -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8080/_stats/statements
curl -X DELETE -H "Authorization: Bearer $TOKEN" http://127.0.0.1:8080/_stats/statements
```

Details

Every statement is also counted by its shape — its text with each literal
and parameter written `$1`, `$2`… and a list of literals
alone, a vector's components, as one — so the same statement run with other
values is one line: how often it ran, its total, mean, least and most time,
the rows it returned or changed, and its failures. Over HTTP a
`/query` or `/batch` is counted as its FenecQL; a call
to a collection's path as its method and path. `GET
/_stats/statements` answers JSON, most time first, and
`DELETE` forgets it all.

The server keeps at most 5 000 shapes, as PostgreSQL's
`pg_stat_statements` does by default,
and forgets the least called when it is full. The counts need what reads the
data without a JWT's scope, or the admin token: a shape names every
collection. A tenant node keeps each tenant's apart —
`/t/<tenant>/_stats/statements` — and lists every
tenant's, named, to its admin alone.

## The audit log

`--audit <path>` appends a JSON line for every refused
request, schema change and admin action. Ordinary reads and writes are
not logged.

```bash
fenec-server --file data.fenec --http 0.0.0.0:8080 \
         --http-token "$TOKEN" --audit /var/log/fenec/audit.log
```

- **Refused tokens are slowed down,** from 100 ms doubling to 5 s per
    address (`--auth-delay`).

- **No literal reaches the log:** a schema change is logged by its
    shape.

Details

```json
{"at":"2026-10-01T09:30:12.310Z","event":"schema","level":"info","proto":"http","peer":"10.0.0.7:53112","request_id":"5d0e7c2a91f4b366","statement":"create collection notes (title text)","failed":false}
{"at":"2026-10-01T09:32:05.117Z","event":"refused","level":"warn","proto":"http","peer":"198.51.100.4:40031","request_id":"b81f03c6d2e94a57","method":"GET","path":"/notes"}
{"at":"2026-10-01T09:40:00.550Z","event":"admin","level":"info","proto":"http","peer":"10.0.0.2:61200","request_id":"deploy-7","method":"POST","path":"/_shard/tenants/acme/move","http_status":200}
```

`--audit <path>` appends one JSON line for each of these
events:

- **An HTTP request refused for its token** (401).

- **A statement that changes the schema**:
    `create`, `drop`, `alter` or
    `compact`. The statement is
    logged by its shape, as [`/_stats/statements`](#statements)
    shows it, so no literal it held reaches the log.

- **A request to `/_admin/` or `/_shard/` that
    changes something**, with its status. `fenec-shard` takes
    `--audit` as well. A [query
    across tenants](https://fenecdb.com/docs/sharding#across-tenants) (`POST /_shard/query`) is one too: it reads
    every tenant's rows, which is what an audit log is kept for. The read it
    sends each node (`POST /_admin/tenants/<t>/query`) is
    not, or one query over a thousand tenants would be a thousand lines; it
    carries the router's request id, which the router's line names.

- **A promotion forced past its node's refusal**: a
    `failover_forced` line at the router naming the node, and a
    `promote_forced` line at each node naming the tenant and, in
    `overrode`, the refusal it went past (see
    [automatic failover](https://fenecdb.com/docs/sharding#automatic-failover)).

Each line has the time, a level (`warn` for a refusal,
`info` otherwise), the protocol, the peer's address and the
[request id](#request-id); an admin request's status is
`http_status`, since a log collector takes a
`status` for the line's level. Reads and
writes are not logged, so the
log costs nothing on their path. Lines are written as the events happen and
are not synced: after a crash the log can lose its last lines, but the data
loses nothing.

**A refused token waits** before it is answered, the way PostgreSQL's
`auth_delay` works. The first failure from an address within a
minute waits `--auth-delay` (100 ms by default). Each further
failure from that address doubles the wait, up to 5 s. This applies to every
401, logged or not. A guessing client gets about one try every five seconds
per address. A request whose token the server verifies — its
`--http-token`, a JWT whose signature checks, the admin,
replication or scrape token, a router's own `--token` — waits
for nothing and clears its address's count, whatever the answer then says
of the rest of the request. Nothing else clears it: an answer given without
looking at a token — an unknown path, `/_health`, a preflight,
the studio's files, a body over the ceiling, every answer of a server that
asks for no token — leaves the count where it was, whatever
`Authorization` the request carried. The count is the
address's, not a token's: a client that holds one good token clears its
count with it. Behind `fenec-shard` the router waits out a
refusal itself, keyed by its client's address, and the node waits none for
what the router forwards: counted at the node, every client had the
router's address, four forged tokens in a row made a fifth client's expired
one wait 1.6 s, and any good token from anyone cleared the attacker's
count. The node knows a forwarded request by a mark the router derives
from the node's `--admin-token`, sent beside the client's address
(`Fenec-Router: <address> <mark>`); the router drops
the header from what a client sends, and a node believes it from no one
else. Its audit log names the client as `peer` and the router as
`via`. The router holds no keys: the node tells it, in its
answer, that it verified a forwarded request's token, and the router clears
its client's count on that alone. Behind the global entry every client reaches a
router from Cloudflare's addresses; a router started with the entry's token
(`--entry-token`) takes the address the entry names in
`Fenec-Client` for the request's, its mark checked and its time
within a minute, and counts, clears, logs (`peer`, the entry as
`via`) and names that address onwards. An IPv4 client is one
count however it arrives: a listener on `[::]` sees it as
`::ffff:a.b.c.d`, counted as `a.b.c.d`. Behind another reverse
proxy every client has the proxy's address, so the proxy's own rate limit
is the one that counts there. `--auth-delay 0` turns the wait
off.

## The router

`fenec-shard` serves its own `/_metrics`, behind its
`--token`: requests, the time its nodes took, failed nodes and
tenant moves. No label names a tenant.

```bash
curl -H "Authorization: Bearer $SHARD_TOKEN" http://127.0.0.1:8090/_metrics
# fenec_router_requests_total{route="tenant",code="2xx"} 48120
# fenec_router_upstream_duration_seconds_bucket{le="0.001"} 47655
# fenec_router_upstream_errors_total{node="n2"} 3
# ...
```

| Metric | Type | What it says |

| `fenec_router_requests_total{route, code}` | counter | requests answered; `route` is `tenant` (`/t/…`, forwarded), `shard`, `replication`, `metrics` or `other`, `code` the status class, `2xx` to `5xx`, or `gone`: a request held at its node (`Fenec-Wait`) whose client reset its connection, answered nothing. A client gone with its end — most timeouts and aborts — reads as one that half-closed and still reads, so its request is answered as the node answers it and counted by that status |

| `fenec_router_request_duration_seconds{route}` | histogram | arrival to answer, in buckets from 100 µs to 10 s |

| `fenec_router_upstream_duration_seconds` | histogram | a forwarded request's time at its node: from its sending to the whole answer. Taken from the request's duration, it leaves the router's own |

| `fenec_router_upstream_errors_total{node}` | counter | forwarded requests a node did not answer or broke off, each answered 502: the one to alert on |

| `fenec_router_refused_total` | counter | requests refused for their token (401): the router's own, and its nodes' it passed on |

| `fenec_router_entry_clients_total{outcome}` | counter | with `--entry-token`, what the router made of the [global entry](https://fenecdb.com/docs/sharding#global-entry)'s name for the client (`Fenec-Client`) of each request no other router relayed: `named`, the address taken for the request's; `bad_mark`, not the entry's mark (another token, or a client sending the header itself); `stale`, the entry's mark made more than a minute from the router's clock; `absent`, a tenant's request carrying none — a client reaching the router itself, or an entry with no `ENTRY_TOKEN`, which names nobody. All but `named` count by the connection's address |

| `fenec_router_moves_total{outcome}` | counter | tenant moves that began, `done` or `failed` and thawed; one refused before it began is not one |

| `fenec_router_move_duration_seconds` | histogram | how long a move that was done took, the tenant frozen for all of it, in buckets from 10 ms to 1 000 s |

| `fenec_router_fanouts_total{outcome}` | counter | [queries across tenants](https://fenecdb.com/docs/sharding#across-tenants): `done`, every tenant answered; `partial`, some failed and the caller asked for the others' rows; `failed`, answered with the tenants that failed; `refused` as a whole, before or after the asking (a request that does not read, a bound passed) |

| `fenec_router_fanout_duration_seconds` | histogram | a query across tenants from its arrival to the merged answer |

| `fenec_router_fanout_tenants_total{outcome}` | counter | the tenants those queries asked, `answered` or `failed`; no label names a tenant |

| `fenec_router_connections` | gauge | client connections open now, the scrape's own among them |

| `fenec_router_nodes`, `fenec_router_tenants{node, region}` | gauge | the nodes in the directory, and the tenants placed on each; `region` where the node names one ([Regions](https://fenecdb.com/docs/sharding#regions)), as on every gauge by node below |

| `fenec_router_replicas{node, region}` | gauge | the replicas of their own each node holds, with `--replicas`; a node's count falls to 0 after a failover took the node's replicas away, until `POST /_shard/replicas` |

| `fenec_router_same_zone_copies{region}` | gauge | tenants whose replica or standby copy is in their primary's zone, so a datacenter lost takes both: placed there only where the region had no node free in another zone |

| `fenec_router_tenants_moving` | gauge | tenants a move began on and did not record done — served from where they were, and cleared by moving them again |

| `fenec_router_following` | gauge | 1 on a standby router; its directory's `fenec_replica_*` gauges follow, and a primary's `fenec_replicas` |

| `fenec_router_lease_age_seconds{node, region}` | gauge | with `--auto-failover`, the seconds since each node last took its lease: renewed every third of a lease, and failed over past the lease and a tenth |

| `fenec_router_node_failed_over{node, region}` | gauge | 1 for a node the router failed over on its own and that has not answered since; 0 again once it does and a repair has its copies follow |

| `fenec_router_auto_failovers_total{outcome}` | counter | nodes failed over on their own, `done` when every tenant was promoted, `failed` when one was not |

| `fenec_router_cut_requests_total{outcome}` | counter | requests out to a node as a failover — by hand or on a lease — moved their tenant off it, ended at once rather than left to wait for `--upstream-timeout` ([Requests out to a node failed over](https://fenecdb.com/docs/sharding#in-flight)): `sent_again` to the tenant's new primary, the node having run nothing or the request being a read or a keyed write; `may_have_run`, a write sent whole with no key, answered `502`; `stream_ended`, a subscription ended for its client to open again. Each is counted in `fenec_router_requests_total` as well, by the status it was answered with; none is an upstream error |

Details

`fenec-shard` serves `/_metrics` on its own listener,
behind its `--token` as `/_shard/` is: a scrape names the
nodes. It counts what it answered and what the nodes took over it; what a
tenant's statements did is its node's to count, and no label names a tenant.
A request is timed from its arrival to its answer, the node's time included —
a subscription to its head, since the stream lasts as long as its client
stays; a standby router's replication stream is counted and not timed. A
request is counted before its answer is written, as a node counts its
statements, so a client holding its answer finds it in the next scrape; its
time ends once the answer is written.

```text
# what the router adds, at the median, over five minutes
histogram_quantile(0.5, sum by (le) (rate(fenec_router_request_duration_seconds_bucket{route="tenant"}[5m])))
  - histogram_quantile(0.5, sum by (le) (rate(fenec_router_upstream_duration_seconds_bucket[5m])))

# worth an alert
rate(fenec_router_upstream_errors_total[5m]) > 0
fenec_router_tenants_moving > 0
increase(fenec_router_auto_failovers_total[5m]) > 0
fenec_router_node_failed_over > 0
fenec_router_same_zone_copies > 0
# behind the global entry: a token or a clock to fix, or an entry naming
# nobody (its ENTRY_TOKEN unset) -- on a router only the entry reaches
rate(fenec_router_entry_clients_total{outcome=~"bad_mark|stale"}[5m]) > 0
rate(fenec_router_entry_clients_total{outcome="absent"}[5m]) > 0

# tenants by region
sum by (region) (fenec_router_tenants)
```

## Request ids

Every answer carries an `X-Request-Id`, and every line its
request wrote — the audit log's, the slow statement's, an error on stderr —
carries the same id. A client's own id is kept; one is made where none was
sent.

```bash
curl -si -H 'X-Request-Id: checkout-7f3a' http://127.0.0.1:8080/_health | grep -i request-id
# X-Request-Id: checkout-7f3a
```

Details

An id a client sends is kept when it is printable ASCII of at most 128
bytes; any other is replaced, as a header or a log line could not carry it
whole. A made one is 16 hex characters. `fenec-shard` takes the
client's id or makes one, sends it to the node, which takes it as its own,
and answers with it: one request through the router has one id in the
router's log, the node's and the answer. A text line on stderr written while
a request was served — a sync error — ends with
`request_id=<id>`. A subscription's and a replication
stream's head carry the id too. Making one and writing it cost a request
nothing measurable (`make requests-bench`, in turns).

## Tracing

With `--otlp-endpoint`, fenec-server and fenec-shard send
OpenTelemetry traces: a span a request, with its waits under it — the
lock, the statement, the disk, the replicas. Any system that takes OTLP
shows where a request's time went: Datadog, Grafana Tempo, Honeycomb, New
Relic, Elastic.

```bash
# an OpenTelemetry Collector, or the Datadog Agent, on this host
fenec-server --file data.fenec --sync always \
         --otlp-endpoint http://127.0.0.1:4318 --trace-sample 0.1

OTEL_SERVICE_NAME=orders-db fenec-server --file data.fenec \
         --otlp-endpoint http://127.0.0.1:4318
```

- **A collector on loopback.** fenecdb speaks no TLS, so the endpoint
    is a collector on the same host or network — the OpenTelemetry
    Collector, the Datadog Agent's OTLP receiver, Grafana Alloy — which
    forwards over TLS: [their setup](https://fenecdb.com/docs//docs/datadog#tracing).

- **W3C `traceparent`.** A request carrying one continues
    the caller's trace; `fenec-shard` continues it and hands it to
    the node, so the router's spans and the node's are one tree.

- **Off unless an endpoint is given**, and free when off. A ratio
    samples the traces that start at fenecdb; a caller's sampled trace is
    always followed.

| Span | Kind | What it times |

| `POST /query`, `GET /{collection}`, `POST /t/{tenant}/batch`… | server | the request, from its head read to its answer written: its method and route, never a value |

| `lock.wait` | internal | the wait for the database's lock, `fenec.lock` `read` or `write`: a read waits out a write's block, a write every read |

| `execute` | internal | the statement run, under the lock; a `/batch`'s statements together, `fenec.statements` of them, and a `create index` or `compact` beside the database with `fenec.beside` |

| `durability` | internal | a write's wait for the disk after the lock (`--sync always`); on a primary it holds `fsync` and `replication.wait_sent`, the wait for its replicas' streams to send the write |

| `fenec.wait_for_write` | internal | a replica's wait for the write a `Fenec-After` names |

| `forward` | client | `fenec-shard`'s forward to the node, to its whole answer: the node's server span is its child |

Details

The server span carries OpenTelemetry's names where one fits:
`http.request.method`, `http.route` (a template:
`/t/{tenant}/{collection}/near`), `url.path`,
`http.response.status_code`, `server.address` and
`server.port`, `client.address` — the client behind
a router, or behind the global entry where a router believes it — and the
connection's own address as `network.peer.address`,
`db.system` (`fenecdb`),
`db.operation.name` (`get`, `put`,
`create`…) and, for FenecQL, `db.query.text` — the
statement's shape, each literal and parameter written `$n`, as
[the statements' counts](#statements) show it, so no value
reaches a span. A REST request's target holds values in its query string,
so it has its route alone. A `--dir` node's and the router's
spans carry `fenec.tenant`, the router's `fenec.node`,
and every server span `fenec.request_id`, the
[request id](#request-id) its log lines carry. A 5xx marks the
server span an error; a 4xx is the client's and does not.

A request's `traceparent` is read as the W3C spec writes it;
one that does not parse starts a trace of its own, and
`tracestate` goes on unchanged. A trace starting at fenecdb is
kept when its id's low 56 bits fall under the ratio, OpenTelemetry's
`TraceIdRatioBased`, so a router and its nodes sampling at one
ratio keep the same traces. `--trace-sample` is 1 with an
endpoint; a busy server wants less. Health checks, scrapes, preflights and
the studio's files are no spans, and a subscription or a replication
stream, which lasts as long as its client, has none.

The endpoint is taken from `--otlp-endpoint`, then
`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`, both used as given unless
they name no path, then `OTEL_EXPORTER_OTLP_ENDPOINT`, under
which `/v1/traces` goes. `--otlp-header k=v` (and
`OTEL_EXPORTER_OTLP_HEADERS`) adds a header to every post, an
API key a collector asks for; `OTEL_SERVICE_NAME` names the
service (`fenec-server`, `fenec-shard`) and
`OTEL_RESOURCE_ATTRIBUTES` adds to it. Spans go as OTLP over
HTTP with JSON, `POST /v1/traces` with
`application/json`, the collector's port 4318: an
`OTEL_EXPORTER_OTLP_PROTOCOL` of `grpc` is refused
at the start rather than sent to a port that cannot read it, and an
`https://` endpoint likewise.

A finished request's spans go into a queue of
`OTEL_BSP_MAX_QUEUE_SIZE` spans (2 048), and a thread of their
own posts it every `OTEL_BSP_SCHEDULE_DELAY` (1 s), or as soon
as `OTEL_BSP_MAX_EXPORT_BATCH_SIZE` (512) are in, a post at
most that many, each bounded by `OTEL_EXPORTER_OTLP_TIMEOUT`
(10 s) and kept alive between posts. A request never waits for the
collector: with the queue full its spans are dropped and counted, and a
post that fails drops its spans, logged once a minute at most. `fenec-server` posts the
last second's spans as it shuts down.

| Metric | Type | What it says |

| `fenec_trace_spans_exported_total` | counter | spans the endpoint took |

| `fenec_trace_spans_dropped_total{reason}` | counter | spans left out: `queue_full`, the collector not keeping up, or `export_failed`, a post refused or unanswered |

## What it costs

Counting is close to free: a few nanoseconds a statement, and counting
one by its shape takes a fraction of the time parsing it does.

Details

Counting a statement is a handful of relaxed atomic adds, into counters a
thread each: 6.7 ns a statement with eight threads counting at once. Into one
shared set of counters the same eight cost 720 ns each, the cache line moving
from core to core, which is why they are spread. The router counts a request
the same way, 13.3 ns with eight threads counting, beside the 17 µs it adds
to the request (`make shard-bench`). A scrape reads the gauges
under the database's shared lock, beside the readers and ahead of a writer
for as long as a `count` would be.

Tracing off is one relaxed atomic load at each hook: in
`make requests-bench`, six runs in turns against the server
without it, one client's round trip stayed 0.022, 0.033, 0.059 and 0.021
ms at the median for a row by id, a filter, a `near` and a put,
against 0.022, 0.034, 0.059 and 0.022, and eight clients' requests a second
inside the runs' own spread. On, every request sampled, to a collector on
the same machine: 0.022, 0.034, 0.060 and 0.024 ms, and eight clients 2 to 9%
fewer requests a second — the exporter's thread writing 250 000 spans a
second as JSON took a core of the eight the clients and the server shared.
A sampled request's spans cost it their timestamps and a few allocations;
the JSON and the post are the exporter's.

Counting it by its shape is a pass over its text into a buffer the thread
keeps, and an entry found in the thread's shard: 0.34 µs for a read of 87
bytes, 0.56 µs with eight threads counting at once — a sixth of the 2.2 µs
parsing it takes. It grows with the text: 5.6 µs for a `near` with
768 components written into it, whose parse takes 85
(`make statements-bench`). A shape is made only as far as the
1 000 bytes kept of it, so two statements alike that far are one line: shaped
whole, a `put` of 1 000 rows of 128-dim vectors spent 1.7 ms of
its 12.6 there.


---

Source: https://fenecdb.com/docs/datadog

# Datadog and other monitoring systems

Everything a monitoring system needs from fenecdb is
three things it already reads: `/_metrics` in Prometheus's
format, logs written as JSON lines, and traces sent over OTLP. Datadog's
check, pipeline and dashboard are ready in the repository, and the Agent's
OTLP receiver takes the traces ([Tracing](#tracing)).

## Datadog in three steps

The repository's `monitoring/datadog/` holds the Agent's
configuration, a logs pipeline and a dashboard.

```bash
# 1. the check and the logs: the Agent reads /_metrics and both logs
sudo cp monitoring/datadog/conf.yaml /etc/datadog-agent/conf.d/openmetrics.d/conf.yaml
sudo systemctl restart datadog-agent     # with logs_enabled: true in datadog.yaml

# 2. the pipeline that remaps the logs' attributes
curl -X POST https://api.datadoghq.com/api/v1/logs/config/pipelines \
  -H "DD-API-KEY: $DD_API_KEY" -H "DD-APPLICATION-KEY: $DD_APP_KEY" \
  -H 'Content-Type: application/json' -d @monitoring/datadog/pipeline.json

# 3. the dashboard
curl -X POST https://api.datadoghq.com/api/v1/dashboard \
  -H "DD-API-KEY: $DD_API_KEY" -H "DD-APPLICATION-KEY: $DD_APP_KEY" \
  -H 'Content-Type: application/json' -d @monitoring/datadog/dashboard.json
```

- **Every server**, replica, tenant node and router is an instance of
    the same check, each with its own address and token.

- **The names are tested:** a test scrapes a real server, a replica,
    a tenant node and a router, and fails if the check or the dashboard
    names a metric none of them sends.

## The metrics

The Agent's OpenMetrics check reads `/_metrics`. With
`namespace: fenecdb` and the `fenec_` prefix taken off,
the metrics arrive as `fenecdb.*`.

```yaml
instances:
  - openmetrics_endpoint: http://127.0.0.1:9464/_metrics
    headers:
      Authorization: "Bearer CHANGE-ME"   # --admin-token or --http-token
    namespace: fenecdb
    raw_metric_prefix: fenec_
    metrics:
      - statements
      - statement_errors
      - statement_duration_seconds
      - refused
      - change_sequence
      - ^replica_.*$
      - ^tenants.*$
      - ^router_.*$
      # ... the whole list is in monitoring/datadog/conf.yaml
    histogram_buckets_as_distributions: true
    tags:
      - service:fenecdb
      - fenec_role:server       # server, replica, tenant-node or router
```

| In `/_metrics` | In Datadog | Type |

| `fenec_statements_total{kind}` | `fenecdb.statements.count`, tag `kind` | count: `.as_rate()` is a second's |

| `fenec_documents{collection}` | `fenecdb.documents`, tag `collection` | gauge |

| `fenec_statement_duration_seconds` | `fenecdb.statement_duration_seconds.bucket`, `.sum`, `.count` | distribution: `p99:` once percentiles are on |

| `fenec_router_requests_total{route, code}` | `fenecdb.router_requests.count`, tags `route`, `code` | count |

Details

A counter loses its `_total` and arrives as a count,
`fenecdb.<name>.count`; a gauge arrives under its name. A
histogram's buckets arrive, with
`histogram_buckets_as_distributions`, as one distribution,
`fenecdb.<name>.bucket`, whose percentiles Datadog takes
over every server's buckets together — as Prometheus's
`histogram_quantile` does — once they are turned on in the
metric's summary; its `.sum` and `.count` give the
mean. Every label is a tag: `kind`, `collection`,
`route`, `code`, `node`,
`outcome`, `version`.

**No label names a tenant.** A tenant node publishes its tenants'
counts together (`fenec_tenants`, `fenec_tenants_open`,
`fenec_tenants_disk_bytes`): a tenant's collection names are not
the node's to publish, and a tag per tenant would be as many series as there
are customers, which Datadog bills by. So a node's role goes into the
instance's `tags` — `fenec_role:tenant-node`,
`fenec_node:n1` — and a tenant's own figures come from the logs,
whose lines carry a `tenant` attribute, and from
`/t/<tenant>/_stats/statements`.

The token goes in the `Authorization` header: kept out of the
file with the Agent's secrets management (`"Bearer
ENC[fenec_metrics_token]"`), or under Autodiscovery as
`"Bearer %%env_FENEC_METRICS_TOKEN%%"`. The metric names
fenecdb publishes are in [Monitoring](https://fenecdb.com/docs//docs/monitoring).

## The logs

The audit log (`--audit`) and the slow statements
(`--slow-ms`, on stderr) are JSON lines, one event each, with the
same attributes. The Agent tails both files; the pipeline makes their
attributes Datadog's own.

```json
{"at":"2026-10-05T09:30:12.310Z","event":"slow","level":"warn","proto":"http","peer":"10.0.0.7:53112","request_id":"9f2c41d07ab35e18","duration_ms":312.408,"kind":"read","failed":false,"tenant":"acme","statement":"POST /query {\"query\":\"get people order name\"}"}
{"at":"2026-10-05T09:32:05.117Z","event":"refused","level":"warn","proto":"http","peer":"198.51.100.4:40031","request_id":"5d0e7c2a91f4b366","method":"GET","path":"/notes"}
```

```yaml
logs:
  - type: file
    path: /var/log/fenec/audit.log       # --audit /var/log/fenec/audit.log
    service: fenecdb
    source: fenecdb
  - type: file
    path: /var/log/fenec/server.log      # fenec-server ... 2>> /var/log/fenec/server.log
    service: fenecdb
    source: fenecdb
```

| Attribute | Remapped to | By |

| `service` | the log's service | the Agent's `service: fenecdb` |

| `level` | the log's status: `info`, `warn` | status remapper |

| `at` | the log's date | date remapper |

| `duration_ms` | `duration`, in nanoseconds | arithmetic processor, `duration_ms * 1000000` |

| `request_id` | `http.request_id`, kept as it is too | attribute remapper |

| `peer` | `network.client.ip`, `network.client.port` | grok parser |

| `method`, `path`, `http_status` | `http.method`, `http.url_details.path`, `http.status_code` | attribute remappers |

Details

Every line has `at`, `event`
(`slow`, `refused`, `schema` or
`admin`), `level`, `proto`,
`peer` — with `via`, the router, where one forwarded
the request — and the request's `request_id`. A slow statement
adds `duration_ms`, `kind` (`read` or
`write`), `failed`, `tenant` on a tenant
node, and `statement`; a refusal `method` and
`path`; a schema change `statement`, by its shape,
`tenant` and `failed`; an admin request
`method`, `path` and `http_status` — not
`status`, which Datadog takes for the line's level before
`level`.

stderr also holds the server's own text lines — that it listens, what it
linked, an error. One written while a request was served ends with
`request_id=<id>`, which the pipeline's first processor, a
grok parser over the message, takes out. Under Docker, the container's log
is stderr: `type: docker` with the same `source` and
`service`, or the label
`com.datadoghq.ad.logs='[{"source":"fenecdb","service":"fenecdb"}]'`.

`--slow-ms 0` logs every statement, as PostgreSQL's
`log_min_duration_statement = 0` does: a pipeline is tried out on
what a server answers now. A test starts `fenec-server`, makes it
write each kind of line, and fails if the pipeline or the dashboard reads an
attribute none of them has. The [audit
log](https://fenecdb.com/docs//docs/monitoring#audit) and the [slow statements](https://fenecdb.com/docs//docs/monitoring#slow) are
described in Monitoring.

## The dashboard

`monitoring/datadog/dashboard.json` opens on whether the disk
has refused a write, statements and errors a second, refusals and the memory
footprint, then requests and their latency, writes and storage,
replication, tenants, the router, and the logs.

Details

Its groups, each on the metrics `/_metrics` sends:

- **Requests:** statements a second by kind, errors, p50, p95 and p99
    by kind, the mean, slow statements, refusals (401) and connections.

- **Writes and storage:** writes a second — the rate of
    `fenecdb.change_sequence` — documents and each collection's
    dead share, the file and what a compact would give back, compactions
    `--auto-compact` ran, and vectors waiting to be linked.

- **Replication:** a replica's lag in writes, whether it is
    connected, since it last heard from its primary, a primary's replicas,
    reconnects and whole images.

- **Tenants:** on disk and open, their files, and the tenants placed
    on each node.

- **Router:** requests by route and status class, its p99 against
    its nodes', nodes it could not reach, moves and refusals.

- **Logs:** the slow statements, the refusals, and the schema
    changes and admin requests, each with its request id.

The `host` template variable picks one server or all of them.
The percentiles need the distribution's percentiles turned on, once, in the
metric's summary.

## One id a request

Every answer carries `X-Request-Id`, and every line the
request wrote carries it as `request_id`: search the logs for the
id a client reports.

```bash
curl -i -H 'X-Request-Id: checkout-7f3a' -X POST http://127.0.0.1:8080/query \
  -d '{"query":"get orders where id = 7"}'
# HTTP/1.1 200 OK
# X-Request-Id: checkout-7f3a
```

- **The client's id is kept** when it is printable ASCII of at most
    128 bytes; any other, or none, is replaced by 16 hex characters.

- **Through `fenec-shard`** the router hands its id to the
    node, so the router's lines, the node's and the answer have one id.

Details

A load balancer or a gateway that sets `X-Request-Id` has its
id carried through to fenecdb's logs; a client that sends none can read the
one made for it off the answer — a subscription's and a replication
stream's too — and report it. A made id is 64 bits of a mix of a key the
process reads from the system once, the connection's thread and its count of
requests: never the same twice within a process, and 15 ns to make, 12 to
keep a client's. The id lives with the connection's thread, as the audit log's peer
does, so nothing between the request's reading and a line's writing carries
it. Making it, sending it and logging it cost a request nothing measurable:
in `make requests-bench`, eight runs in turns, one client's
round trip for a row by id stayed 0.016–0.022 ms either way.

## New Relic, Elastic, Grafana Cloud

Each reads the same two things: `/_metrics` as Prometheus
metrics, and the JSON lines as logs.

- **New Relic:** the infrastructure agent's Prometheus OpenMetrics
    integration (`nri-prometheus`) scrapes
    `/_metrics` with the token as a bearer header, and the
    metrics keep their names, `fenec_statements_total` and the
    rest. Its log forwarder tails both files and parses each JSON line into
    attributes; `request_id` is one.

- **Elastic:** Metricbeat's Prometheus module, its
    `collector` metricset, reads `/_metrics`
    (`metrics_path: /_metrics`, the token in
    `headers`). Filebeat's `filestream` input with the
    `ndjson` parser reads the logs, and an ingest pipeline's
    `date` processor takes `at` as
    `@timestamp` and `level` as
    `log.level`.

- **Grafana Cloud:** Grafana Alloy's
    `prometheus.scrape` reads `/_metrics` and
    `prometheus.remote_write` sends it on — or a Prometheus of
    your own with `remote_write` — and
    `monitoring/grafana/fenecdb.json` is the dashboard. Alloy's
    `loki.source.file` with a `stage.json` sends the
    logs to Loki, `request_id` one of the fields.

## Tracing

fenec-server and fenec-shard send OpenTelemetry traces to an OTLP
endpoint: a span a request, and under it the wait for the lock, the
statement, the disk and the replicas ([the
spans](https://fenecdb.com/docs//docs/monitoring#tracing)). fenecdb speaks no TLS, so it sends them to an agent or collector
on the same host, which forwards them over TLS. The Datadog Agent takes them
on its OTLP receiver:

```yaml
# datadog.yaml: the Agent's OTLP receiver, HTTP on loopback
otlp_config:
  receiver:
    protocols:
      http:
        endpoint: localhost:4318
# in a container: DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT=0.0.0.0:4318
```

```bash
sudo systemctl restart datadog-agent
OTEL_SERVICE_NAME=orders-db \
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=prod \
  fenec-server --file data.fenec --otlp-endpoint http://localhost:4318 --trace-sample 0.1
```

- **APM** shows the service `orders-db`, its resources by
    route (`POST /query`), and each trace's waits;
    `db.query.text` is the statement's shape, with no value in
    it.

- **Traces join logs** by `fenec.request_id`, the
    `request_id` of every line the request wrote.

- **A client's trace continues:** a `traceparent` from an
    instrumented app makes fenecdb's spans children of its call, and the
    router hands it on to the node.

- **Grafana Tempo:** Alloy's `otelcol.receiver.otlp` with
    `http { endpoint = "127.0.0.1:4318" }`, sent on by
    `otelcol.exporter.otlp` to Tempo or Grafana Cloud.

- **Honeycomb:** an OpenTelemetry Collector on loopback whose
    `otlphttp` exporter sends to `https://api.honeycomb.io`
    with the `x-honeycomb-team` header.

- **New Relic and Elastic** take OTLP the same way, through a
    collector's `otlphttp` exporter to `otlp.nr-data.net`
    with an `api-key` header, or to an Elastic APM Server's OTLP
    intake.

Details

Spans go as OTLP over HTTP with JSON, port 4318; the Agent's gRPC port,
4317, is not for them. A header a collector asks for goes with
`--otlp-header k=v` or `OTEL_EXPORTER_OTLP_HEADERS`.
With no endpoint, tracing is off and costs nothing; on, a request never
waits for the Agent, and what the queue cannot hold is counted in
`fenec_trace_spans_dropped_total`
([Monitoring](https://fenecdb.com/docs//docs/monitoring#tracing)).


---

Source: https://fenecdb.com/docs/grafana

# Grafana

Grafana shows fenecdb two ways: the server's health
through Prometheus, and the data itself — visitors, top pages, price
candles — through fenecdb's own data source, each panel one FenecQL
statement with the dashboard's range and variables bound as parameters,
alert rules included. The repository's `monitoring/` runs all of
it with one `docker compose up`.

## Metrics and data

The dashboards read different things, by different paths, with
different tokens.

|  | The server's metrics | The data |

| What | statements, latency, memory, replication, tenants: `/_metrics` | your collections' rows, grouped and counted by FenecQL |

| Path | fenec-server → Prometheus → Grafana | Grafana → `fenecdb-datasource` → `POST /query` |

| Dashboard | `monitoring/grafana/fenecdb.json` | `monitoring/grafana/fenecdb-data-plugin.json`, and `fenecdb-data.json` through Infinity |

| Token | the admin token, in Prometheus | a JSON Web Token that reads two collections, in the data source |

| Docs | [Monitoring](https://fenecdb.com/docs//docs/monitoring) | this page |

## Run it

```bash
FENEC_HTTP_TOKEN=secret FENEC_METRICS_TOKEN=scrape \
FENEC_JWT_SECRET=$(openssl rand -hex 32) \
  docker compose -f monitoring/docker-compose.yml up -d --build
# the data:     http://127.0.0.1:3000/d/fenecdb-data-plugin
# the metrics:  http://127.0.0.1:3000/d/fenecdb

# down, the data and the token with it
docker compose -f monitoring/docker-compose.yml down -v
```

- **Seeded data.** The first start writes thirty days of a product
    site's events, made by the traffic simulator of
    [examples/analytics](https://github.com/fenecdb/fenec/tree/main/examples/analytics),
    and a week of trades in twenty invented symbols, 60 000 a day: about
    108 000 events and 470 000 trades, written in two seconds.

- **Grafana 13.2.2 with the plugin built from the checkout**
    (`integrations/grafana/Dockerfile`: webpack for its
    frontend, mage for its backend), unsigned and let in by name; the
    Infinity plugin 4.1.1 beside it. The first build took 1 min 25 s on an
    M1 Air.

- **No code in fenecdb's server.** Both data sources speak the
    [HTTP endpoint](https://fenecdb.com/docs//docs/http) every client speaks; the plugin
    through the [Go SDK](https://fenecdb.com/docs//docs/languages).

Details

The compose file starts fenec-server with a `--policy`, a
one-shot `seed` container (Node, running
`monitoring/seed/seed.mjs` with no install: Node strips the
simulator's types as it imports it), Prometheus, and Grafana once the seed
has finished. The seed makes `events` and `ticks` only
where they are missing, so a restart keeps the data, and mints the token
Grafana reads with each time it runs. Provisioning loads the three
dashboards, both data sources and the alert rule
(`grafana/provisioning/`).

The JWT secret goes to the server as a file
(`--jwt-secret-file`, from a compose secret) and not as
`FENEC_JWT_SECRET`, which every process in the container would
see; the seed reads the same file to mint Grafana's token. `FENEC_PORT`,
`GRAFANA_PORT` and `PROMETHEUS_PORT` move the ports
published on loopback.

Into a Grafana of your own, the same three pieces:

- **The plugin:** build it (`integrations/grafana/README.md`:
    `npm run build`, then mage for your Grafana's platform), copy
    `dist/` to `<plugins>/fenecdb-datasource/`,
    and set `GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS=fenecdb-datasource`
    until it is signed (the README says how).

- **A token** for a role the policy lets read ([below](#token)):
    `fenec-server --jwt-secret-file jwt.secret --mint-token
    '{"sub":"grafana","role":"grafana","exp":1823000000}'`.

- **The data source:**
    `monitoring/grafana/provisioning/datasources/fenecdb-plugin.yml`
    with your server's URL and the token, then the dashboard's JSON
    imported.

```yaml
apiVersion: 1
datasources:
  - name: fenecdb
    type: fenecdb-datasource
    uid: fenecdb
    url: http://fenecdb:8080
    jsonData:
      tenant: acme          # optional: requests go to /t/acme/
    secureJsonData:
      token: $__file{/run/fenec/grafana.jwt}
```

## A panel's query

A panel's query is FenecQL. Grafana's range and interval, and the
dashboard's variables, are named in it; the plugin's backend makes each
name a `$n` parameter and sends the statement with the values
beside it, never in its text.

```fenecql
get ticks select bucket(at, $__interval_ms) as time, sym, sum(qty) as volume
  where sym in $syms and at >= $__from and at < $__to
  group time, sym order time
```

```json
{"query": "get ticks select bucket(at, $1) as time, sym, sum(qty) as volume where sym in [$2, $3, $4] and at >= $5 and at < $6 group time, sym order time",
 "params": [900000, "S19", "S04", "S03", 1791072000000, 1791683333856]}
```

- **`$__from` and `$__to`** are the range in
    epoch milliseconds, which a timestamp field compares with as it is;
    **`$__interval_ms`** is the interval Grafana picks for the
    panel's width, which `bucket` takes as milliseconds;
    `$__max_data_points` its width in points, which
    `limit $__max_data_points` holds a series to. The backend works
    them out from the request, so an alert rule has them as a panel
    does.

- **A dashboard variable** is `$sym` or
    `${sym}`, a text; `${n:int}` reads it as
    `number`, `int`, `float`,
    `bool`, `timestamp` or `json`. A name
    used twice is one parameter.

- **A multi-value variable** goes after `in`:
    `sym in $syms` is sent as `sym in [$2, $3, $4]`,
    a parameter a value, since FenecQL's `in` takes a list of
    values and a parameter binds one. Nothing picked is `in []`,
    no rows.

- **The columns come in the select list's order**, typed: numbers,
    times, text, JSON, a `geo` point as its longitude and
    latitude. No transformation between the answer and the panel.

- **A time series** when the statement groups by its first column, a
    time — or when the editor's format says so. Text columns beside it are
    labels, so a group by time and symbol is a series a symbol.

Details

The editor colours FenecQL by the docs' own rules (the same file as
fenec studio's editor), completes `$__from` and the dashboard's
variables, and under **Explain** shows the statement as the backend binds
it, its parameters, and the server's plan. Beside the text it holds a list
of parameters, `$1`, `$2`, ...: each a literal of its
type, or one variable whole; names in the text are numbered after it.

The answer carries no types — a float written whole is `12`,
a time is ISO text — so a column is typed by the collection's schema where
it is one of its fields (the `collections` statement, kept 30 s)
and by its values where it is computed: a text in the server's form of a
time is a time. An int past 253 stays exact; a vector or a sparse
vector is left out with a notice, as a panel cannot draw it and it can be
thousands of numbers a row. `facet`'s counts are frames of their
own; a `lookup` is flattened as every flat transport flattens
it, one row per path under `<collection>.<field>`.

Annotations come from a query's `time`, `title`,
`text` and `tags` columns — the dashboard marks each
day's high of the picked symbol on its candles, `last(at by px) as
time, max(px) as text` — and a query variable takes its first column,
or `__text` and `__value`. Several queries of one panel
go as one read-only `/batch`, which the server answers at one
change; one that fails it is answered with its refusal and the rest are
asked on their own.

The data source runs reads: a statement starting with anything but
`get`, `select` or `explain` is refused
before it leaves Grafana. A refusal reaches the panel with the server's
status and message: a parse error's position, a 412 of an unmet
`require`, a 401 for the token.

The dashboard's statements

Every panel aggregates on the server, so Grafana is sent the points it
draws and not the rows behind them; the recipes are Analytics and market data's.

```fenecql
-- Events, visitors and signups
get events select count(*) as Events, count(distinct user) as Visitors, sum(case when name = "signup_completed" then 1 else 0 end) as Signups where at >= $__from and at < $__to
-- Visitors over time
get events select bucket(at, $__interval_ms) as Time, count(*) as Events, count(distinct user) as Visitors where at >= $__from and at < $__to group Time order Time
-- Top pages
get events select path as Page, count(*) as Views, count(distinct user) as Visitors where name = "pageview" and at >= $__from and at < $__to group Page order Views desc limit 10
-- Distinct users per day
get events select bucket(at, 1d) as Day, count(distinct user) as Users where at >= $__from and at < $__to group Day order Day
-- The symbol variable, and the symbols, most traded first
get ticks select sym, count(*) as trades group sym order sym
get ticks select sym, sum(qty) as volume group sym order volume desc
-- Candles of $sym, and each day's high of it as annotations
get ticks select bucket(at, $__interval_ms) as Time, first(px by at) as open, max(px) as high, min(px) as low, last(px by at) as close, sum(qty) as volume where sym = $sym and at >= $__from and at < $__to group Time order Time
get ticks select last(at by px) as time, max(px) as text, bucket(at, 1d) as day where sym = $sym and at >= $__from and at < $__to group day order day
-- Latest price per symbol
get ticks select sym as Symbol, last(px by at) as Price, (last(px by at) - first(px by at)) / first(px by at) * 100 as Change, sum(qty) as Volume where at >= $__from and at < $__to group Symbol order Symbol
-- Volume by symbol: a series each
get ticks select bucket(at, $__interval_ms) as Time, sym, sum(qty) as volume where sym in $syms and at >= $__from and at < $__to group Time, sym order Time
```

## Alert rules

A rule runs its query in Grafana's backend, with no browser to fill in
a value, so it needs a data source that binds them itself. The compose
provisions one over the trades: the last ten minutes' volume of each
symbol, firing above 2 000.

```yaml
# monitoring/grafana/provisioning/alerting/fenecdb.yml (its query)
datasourceUid: fenecdb
relativeTimeRange: {from: 21600, to: 0}
model:
  format: time_series
  intervalMs: 600000
  queryText: >-
    get ticks select bucket(at, $__interval_ms) as time, sym, sum(qty) as volume
    where at >= $__from and at < $__to group time, sym order time
```

- **An instance a symbol**, labelled `sym=S01`, ...: the
    group by time and symbol is a series a symbol, which the rule's reduce
    and threshold take one at a time.

- **The rule's range and interval** are bound as a panel's are: the
    plugin reads them off the request Grafana's backend makes.

- **A dashboard variable has no value in a rule**, and a query naming
    one is refused saying so rather than run without it.

## Refreshes

The plugin keeps the answer to each statement and its parameters with
its `ETag`, and sends the same request again with
`If-None-Match`: a 304 is answered from the kept answer
([Polling a read](https://fenecdb.com/docs//docs/http#polling)).

Over the seeded week, the median of 21 runs of each, the ways in turns,
from the host into Docker on an M1 Air: fenec-server directly, Grafana
through Infinity, Grafana through the plugin with a new answer each time
(the range's end moved a millisecond a run, as a refresh's `now`
moves), and the same request again, answered 304 — with the read-only JWT
and with the server's token.

| Panel | Direct | Infinity | Plugin | Plugin, 304, JWT | Plugin, 304, server token |

| Events, visitors and signups | 10.1 ms | 14.7 ms | 14.5 ms | 14.0 ms | 4.9 ms |

| Visitors over time | 8.2 ms | 15.6 ms | 12.2 ms | 11.4 ms | 4.8 ms |

| Top pages | 11.7 ms | 14.6 ms | 13.6 ms | 13.4 ms | 2.9 ms |

| Distinct users per day | 7.6 ms | 10.5 ms | 9.6 ms | 9.3 ms | 2.8 ms |

| Candles, a symbol | 9.9 ms | 17.8 ms | 13.2 ms | 13.0 ms | 4.2 ms |

| Latest price per symbol | 79.8 ms | 83.1 ms | 82.4 ms | 82.3 ms | 3.0 ms |

| Volume by symbol, three | — | — | 10.9 ms | 11.0 ms | 3.9 ms |

- **The plugin adds 2 to 5 ms a panel** over the direct request,
    Infinity 3 to 8.

- **With the server's token a 304 runs nothing**: the server answers
    it from its change counter while nothing the statement reads was
    written, 3 to 5 ms whatever the statement.

- **With a scoped token it runs the statement** and compares a digest
    of the answer, which saves the transfer and the plugin's parsing only:
    a 304 by the counter would tell the token when anyone wrote to what it
    reads.

- **A range ending at `now` is a new request every
    refresh.** A variable's query, an absolute range, or one rounded
    (`now/h`) ask the same one again. Grafana's own query caching
    is Grafana Enterprise's and Cloud's, not the open-source build's.

`node monitoring/grafana/bench.mjs` measures it against a
running stack.

## The plugin or Infinity

The Infinity plugin reads fenecdb with nothing to build or install
beyond Grafana's catalog; `monitoring/` keeps its dashboard
(`/d/fenecdb-data`) beside the plugin's.

|  | fenecdb-datasource | Infinity 4.1.1 |

| Install | built from `integrations/grafana`, unsigned until you sign it | Grafana's catalog, signed |

| A query | FenecQL in an editor that colours it | a JSON body, FenecQL inside a string |

| Values | bound as parameters by the backend | written into the body's text by Grafana's frontend |

| Multi-value variables | `in $syms`, a parameter a value | a list in one parameter (`:json`), which `in` does not take |

| Columns | the select list's order, typed from the schema and the values | sorted by name; an Organize transformation and a type a column, by hand, in each panel |

| Time series | a series a label (long to wide) | each number column a series, no labels |

| Alert rules | the range, the interval | the range alone; `$__interval_ms` written into the body by hand |

| Annotations, query variables | yes | yes |

| A refresh unchanged | a 304, the kept answer | the statement runs again |

| Where a request goes | the configured URL and the plugin's paths; no redirect followed | `allowedHosts` holds it to `POST /query` |

| Writes | refused before they leave Grafana; the token's policy on the server | the token's policy |

Infinity's configuration and its limits

Each panel posts a JSON body to `/query`: type JSON, parser
Backend, source URL, method POST, URL `/query`, body raw
`application/json`, and a column for each item of the select
list — `timestamp` for a time, `number`,
`string`. Grafana's range and interval go in as
`${__from}`, `${__to}` and
`$__interval_ms`, and a text variable as
`${sym:doublequote}`, with no quotes around it.

```json
{
  "query": "get ticks select bucket(at, $4) as t, first(px by at) as open, max(px) as high, min(px) as low, last(px by at) as close, sum(qty) as volume where sym = $3 and at >= $1 and at < $2 group t order t",
  "params": [${__from}, ${__to}, ${sym:doublequote}, $__interval_ms]
}
```

```yaml
apiVersion: 1
datasources:
  - name: fenecdb data
    type: yesoreyeram-infinity-datasource
    uid: fenecdb-data
    url: http://fenecdb:8080
    jsonData:
      auth_method: bearerToken
      allowedHosts:
        - http://fenecdb:8080/query
    secureJsonData:
      bearerToken: $__file{/run/fenec/grafana.jwt}
```

- **Grafana writes variables into the body as text** before it sends
    it. `:doublequote` puts quotes around the value and a
    backslash before each `"` in it; `"${sym}"` leaves
    a `"` free to end the string, and `${sym:json}`
    writes a single value as it is. The format leaves a backslash as it is,
    so a value holding `\"` does end its string: keep a text
    variable the body's last string, and what follows can only make a body
    that is not JSON, which fenec-server refuses with 400.

- **Columns come back in the order of their names**, so each panel
    has Grafana's Organize fields transformation put them back.

- **Alerting needs the backend parser**, and a body that names its
    values: a rule's query runs where nothing writes
    `$__interval_ms` or a dashboard variable, and Infinity's
    frontend parsers cannot alert.

- **A refresh runs every statement again**: Infinity keeps no
    `ETag`, and runs its own filters and computed columns over
    every row the answer holds — aggregate in FenecQL.

- **`allowedHosts`** holds every query's URL to
    `POST /query`, host and path: through the data source,
    `/_whoami` is refused by the plugin.

## The read-only token

Both data sources read with a JSON Web Token whose role the server's
policy lets read two collections and write nothing.

```text
# monitoring/policy.txt
events          read                   for grafana
ticks           read                   for grafana
```

- **It decides what a Grafana user reads.** Any viewer can send a
    data source any query, through `POST /api/ds/query`,
    whatever its dashboards say. A `put`, `set` or
    `del` is 403, and so is a schema change or
    `/_changes`; `/_metrics` and the statement counts
    take no JSON Web Token at all, and a collection the policy does not name
    does not exist for it.

- **It is a secret of the data source**, encrypted in Grafana's
    database and handed to the plugin's backend alone: in no dashboard JSON,
    query, inspector or page's settings.

- **A variable is a value.** Opened with a symbol written to close a
    string and send a statement of its own, the plugin sends it as one
    parameter and the candles show no data.

Details

A rule's filter narrows the rows further — `events read where site
= $jwt.site for grafana` — and is ANDed into every statement the token
sends, so a Grafana per customer sees its own rows
([Scoped access](https://fenecdb.com/docs//docs/http#scoped-access)). The seed mints the
token for a year, and Grafana reads it from a file when it starts
(`$__file{}` in the provisioning). To rotate one, mint another,
put it into the data source and restart Grafana; `--jwt-max-age`
on the server bounds how far ahead any token's `exp` may lie.

## Checked end to end

`make grafana-check` starts `monitoring/`'s compose
on ports of its own and asks Grafana's `/api/ds/query` for every
panel's, variable's and annotation's query of both data dashboards, as a
viewer, holding each frame to the same FenecQL sent to fenec-server
directly — through the plugin, column by column in the select list's order,
the statement it sent included, and a series a label where it made one.
Then the alert rule must have evaluated, an instance a symbol, firing where
the server's rows say; and the refusals: a write, symbols written to close
their string, the token in no page's reach. CI runs it with the plugin's
own tests and its backend built for linux/amd64 and arm64.


---

Source: https://fenecdb.com/docs/studio

# Studio

fenec studio is the server's own admin page: its collections,
their rows and their schema in a browser, the edits you make there, a
query editor, the rows as they change and the server's own numbers. Start
the server with `--studio` and open `/_studio/`, or
try it on the [Playground](https://fenecdb.com/docs/../playground), where it runs
over a database in your own tab.

## Turn it on

Add `--studio` to `fenec-server`, open
`/_studio/` on the same address, and paste a token.

```bash
fenec-server --file data.fenec --http 127.0.0.1:8080 \
         --http-token "$TOKEN" --studio

open http://127.0.0.1:8080/_studio/
```

100 000 orders, counted by their `@hash` fields, one row unfolded.

- **Off by default.** Without `--studio`,
    `/_studio/` answers 404, as an unknown path does.

- **No new authority.** The page reads and writes with the token
    you paste, over the same HTTP surface every client uses.

- **A tool for operators:** keep it on a private network or behind
    your own sign-in ([Security](#security)).

Details

The studio is plain ES modules and CSS from the repository's
`studio/` folder, with the HTTP client and query builder of
`@fenecdb/web/client` beside them — no framework and no build
step. The server embeds the files, so the binary is all you deploy. Its
first load is 66 KB of JavaScript and CSS, gzipped, its modules preloaded
together from the page; the query editor, the schema, the live rows and the
admin view are fetched the first time each is opened, 7 to 19 KB more. The server sends the
files as they are, uncompressed; a proxy in front may compress them.

A router serves it too: `fenec-shard --studio` puts the
same pages on the router's address, where each tenant is reached through
the router ([Tenants](#tenants)).

## Connect

The server is the page's own address unless you name another, and the
token is the one the server takes: its `--http-token`, or a JSON
Web Token its policy holds to its rows. The top bar says who the server takes
it for.

- **Your token stays in the tab.** It is kept in session storage and
    forgotten when the tab closes: never in local storage, a cookie or a
    URL.

- **Who you are:** the token's subject, role and tenant, the rules
    that apply to it, and how long until its `exp`. The line
    under the top bar shortens as the token's life runs out, and the page
    warns you five minutes before it does.

Details

The identity comes from `GET /_whoami`, which any client may
ask: `open` on a server that asks for no token,
`full` for the server's token, and `scoped` for a JSON
Web Token, with each rule of the policy that applies to it — the
collection, the grants, the fields an update may change, the fields a read
shows (`read_fields`, `null` for every one) and the
rows, as the policy writes the filter. A token is shown its own rules and no
other's: a rule for a role it does not hold is not among them. It grants
nothing; a token the server refuses gets the same 401 here as anywhere.

```bash
curl -H "Authorization: Bearer $ALICE" http://127.0.0.1:8080/_whoami
# {"node":"single","tenant":null,"read_only":false,"kind":"scoped","sub":"alice",
#  "tenants":null,"unbound":false,"append_only":[],
#  "rules":[{"collection":"orders","grants":["read","insert","update","delete"],
#            "fields":null,"read_fields":null,"rows":"owner = $jwt.sub"}]}
```

The token's claims — its subject, role and `exp` — are read
in the page to show them, and decide nothing: the server checks the token on
every request.

## Collections and rows

The sidebar lists the collections the token can read, each with its rows
and its fields' types and indexes. The grid shows a collection's rows, as many
as it holds, read a page at a time as you scroll.

- **Sort** by `id`, or by a field an index orders:
    `@sorted` or `@ttl`.

- **Filter** with a `where` clause in FenecQL and its
    parameters, or a box under each column: `abc`,
    `=v`, `>=v`, `a..b`,
    `null`.

- **Counts by value** for each `@hash` field, over what
    the filter selects; pick one to filter by it.

- **A row unfolded** (Space): a `json` field as a tree, a
    vector as its dimension, its length and its first components.

- **File size** and what a compact would give back, where the token
    may read `/_metrics`.

Details

A page is 100 rows. In id order the page after one already read is asked
for by its last id — `get orders where id > $1 limit 100` —
which walks the id index from there; a page reached by a jump, or under an
order, by its offset. Only the rows in view are drawn, so 100 000 rows cost
what 40 do: a scroll down the whole collection and back in Chrome took no
task over 50 ms (`make studio-test`). Past 8 million pixels of
rows the scroll bar stands for the rows in proportion, since browsers cap an
element's height.

The count and the counts by value are one `/batch` of reads,
under one read lock, so they are of one moment. A vector is never written
out whole: 768 numbers in a cell are noise.

## Edit

Enter writes a cell, N opens a form for a new row, and Delete removes one
after you confirm the statement it will send.

```fenecql
set orders {note: $1} where id = $2 require 1
insert orders {customer: $1, total: $2}
del orders where id = $1 require 1
```

- **Every value is a parameter.** A text shaped as FenecQL is stored
    as it reads.

- **Every write is a `/batch`** with an
    `Idempotency-Key`, so a retry cannot write it twice.

- **A refusal says why:** 403 names the field the policy refused, and
    412 says the row changed — deleted, or no longer the token's — since it
    was read.

Details

`require 1` makes a write that found no row a refusal (412)
rather than a quiet `affected 0`, so an edit of a row someone
deleted meanwhile is told so. The form is built from the schema: a box a
field, typed as the field is, the statement written out below it as you
fill it. A name FenecQL cannot write is refused rather than spliced into the
text, and the `where` clause you type is held inside parentheses
it cannot close, so it stays one condition among the page's.

## Query

The Query tab is an editor for FenecQL. It runs exactly what you type,
with your token: it can do what the token may, and nothing more.

A query with its parameters, its rows, and the request id to find it in the logs.

- **Run** with `Ctrl+Enter` or `⌘Enter`; a
    selection runs alone.

- **Parameters** are a JSON list, bound to `$1`,
    `$2`…

- **Several statements**, separated by `;`, run as one
    `/batch`: they land together or not at all. Give their
    parameters as a list each, `[[1], ["a"]]`.

- **The answer** as rows or as JSON, with its status, its time, its
    `Fenec-Seq` for a write and its `X-Request-Id`.

- **The plan** of a `get`: the Plan tab asks the server
    for `explain` of the same text with the same parameters.

- **A refusal** shows the server's message and status, and for a
    batch which statement stopped it, which the editor selects.

- **History and saved queries** stay in this browser. The token is
    never kept there.

Details

The colours are the docs' own rules: `site/build.py` writes
the same patterns and keywords into the editor's highlighter, and
`site/test_highlight.py` holds the two to the same output. The
text is drawn over the textarea as text nodes, so the caret, selection,
input methods and undo are the browser's. Nothing you type is rewritten:
the editor only cuts the text at each `;` outside strings and
comments, and a statement goes to `POST /query`, several to
`POST /batch`, as any client sends them.

## Schema

The Schema tab shows a collection as the statements that make it, and
its fields. With the server's token you can add an index or a field, rename
or drop a field, and drop the collection.

An index on its way: the statement, and what the server's plan says of it.

- **Every change is shown before it runs:** the exact statement,
    and what `POST /_schema/plan` says of the schema as it would
    be after it.

- **A drop waits for its name.** Dropping a field or a collection
    runs only once you type its name.

- **A scoped token reads the schema** of the collections it reads,
    and changes nothing.

```fenecql
create index on orders (total) @sorted
alter collection orders add field due timestamp @ttl(30d)
alter collection orders rename field note to memo
alter collection orders drop field memo
drop collection scratch
```

Details

The statements come from the server: `GET /_schema?as=fenecql`
answers the description with its collections as FenecQL, written by the
engine. The plan is asked of the collection as it would be after the
change, and of no other, so it speaks of this change alone. It writes the
statements an index or a field added would need, and refuses to guess a
field that is gone: a drop or a rename? The statement says which. A change
runs as a `/batch` under an `Idempotency-Key`, as the
studio's other writes do. A name FenecQL cannot write is refused, and an
index's options and a ttl's duration are checked before they reach the
text.

## Live

The Live tab follows a collection's rows as they change: new and
rewritten rows are marked as they arrive, deleted ones struck through
before they go.

One customer's orders as they are written, each change listed.

- **A `where` clause** picks the rows to follow.

- **Pause** holds the changes back and counts them; resume applies
    them in order.

- **A scoped token sees its own rows:** the server holds the stream
    to them, deletions included.

Details

The view is a subscription, `GET /<name>/changes` with
your filter as its shape: first every row it selects, then each write. So
a shape of more than 10 000 rows is refused before it opens: narrow it
with the `where` clause. Under a scoped token the server ANDs the
token's filter into the shape and keeps the ids it sent, so a row the token
may not read never reaches the page, nor does the deletion of one.

## Admin

The Admin tab is shown to the server's own token. It reads the
statements by shape, the server's counts and, on a router, its
tenants.

The slowest shapes, and the server's numbers.

- **Statements:** the slowest and the most run shapes, from
    `/_stats/statements`. Pick one to open it in the editor.

- **Server:** reads, writes, errors and their latency, the file and
    what a compact would give back, memory and compactions, from
    `/_metrics`, read again every 5 seconds.

- **On a router:** each tenant with its node, state and size on
    disk, from `GET /_shard/tenants?bytes`.

Details

The tab appears when `/_whoami` says the token is the
server's, or the server asks for none; the server decides again on every
request. Through a router the tenants' list takes the router's
`--token`: start the router with the nodes' data token and one
token reads both.

## In your browser

The site's [Playground](https://fenecdb.com/docs/../playground) is this studio —
the same files the server embeds — over a database in your browser tab
rather than a server's: fenecdb compiled to WebAssembly, on a worker of the
tab, seeded with products, orders and a week of events. Nothing is sent
anywhere. Rows, the query editor, the schema and the live rows work there as
they do here; the query editor lists examples to run.

| On a server | In your browser |

| A token pasted in, its subject and expiry in the top bar | **Running in your browser.** in its place, and nothing to sign in to |

| The tenant picker on a node or a router | none: one database |

| The admin view, the file's sizes in the sidebar | none: those are a server's numbers |

| Writes through `/batch` with an `Idempotency-Key` | the same statements, one block each batch |

| Live rows through a subscription | a live query over the module's change ring |

| Kept in the server's file | kept nowhere unless asked: **Keep it in this browser** stores it in IndexedDB, and **Reset data** puts every collection back |

Details

Every view reads through one surface: `run`,
`batch` and `subscribe` as `FenecHttp`
has them, and `request` for an answer shown whole — the editor's
status and change, the schema and its plan. Over a server that surface is
HTTP (`studio/connect.js`); in the page it is the browser module
in a dedicated worker (`studio/local.js`,
`worker.js`), which answers each request as a server would:
the same rows, the status a server refuses a statement with, the change a
write leaves the database at, the schema as the engine describes it
(`Fenec.describe`) and a batch as one block
(`Fenec.batch`). `studio/test/transport.test.mjs`
sends the same statements to a real `fenec-server` and to the
module and holds every answer equal. A server's first load holds none of
it: the page's own modules load only where a page asks for them.

## Keys

| `/` | the `where` clause |

| `Alt+1`, `Alt+2` | the collections, the view |

| arrows, `Page Up`, `Page Down` | move between cells |

| `Enter`, `F2` | edit the cell |

| `Space` | the whole row |

| `C` | copy the row as JSON |

| `S` | sort by the column, where its index orders it |

| `N`, `Delete` | a new row; delete the row, once confirmed |

| `R`, `?` | read the rows again; every key |

| `1` to `5` | rows, query, schema, live, admin (`1` to `4` in your browser) |

| `Ctrl+Enter`, `⌘Enter` | run the query, in the editor |

## Tenants

On a tenant node or a router, the top bar picks the tenant: those the
token's tenant claim names, or the node's or router's list where the token
may read it. Otherwise type the tenant when you connect.

```bash
fenec-server --dir tenants --http 127.0.0.1:8081 --admin-token "$ADMIN" --studio
fenec-shard --listen 127.0.0.1:8090 --directory shard.fenec --studio
```

Details

A token is bound to the tenants it names on a node, so the picker offers
nothing it would be refused. Through a router the studio is on the router's
address and every tenant is that same origin; a page served by a node would
be another origin to the router. `--studio-connect <origin>`
lets the pages a server serves send requests to one more origin, which its
content security policy names.

On a phone, light and dark: the collections slide in from the side.

## Security

The studio is a new surface, not a new authority. Treat it as you would
the server's own port: keep it on a private network, or behind your own
sign-in and TLS.

| authority | none of its own: every statement carries the pasted token, so a scoped token sees its rows alone — in the grid, the counts and the facets |

| the token | session storage of one tab; sent only as `Authorization: Bearer` |

| headers | `Content-Security-Policy` with `default-src 'self'`, no inline script, `connect-src 'self'` and the `--studio-connect` origin; `X-Frame-Options: DENY`; `nosniff`; no referrer |

| values | shown as text, never as markup; sent as parameters, never spliced |

| the editor | runs what you type, as typed, with your token: no more authority than the token |

| schema changes | shown as their statement and the server's plan before they run; a drop runs once its name is typed |

| this browser | keeps the editor's history and saved queries, and the theme; never the token |

| caching | the page is never kept; its modules are asked about again each load (`ETag`, 304) |

| off | without `--studio`, 404 |


---

Source: https://fenecdb.com/docs/sharding

# Tenants and sharding

When one machine is not enough, give each tenant a file of its
own, spread the files over nodes, and put `fenec-shard` in front.

## Get started

Start two nodes and a router, tell the router about the nodes, create a
tenant, and use it through the router.

```bash
# two nodes, each a directory of tenant files
fenec-server --dir /data/n1 --http 127.0.0.1:8081 --admin-token a1
fenec-server --dir /data/n2 --http 127.0.0.1:8082 --admin-token a2

# the router, and its directory of which tenant lives where
fenec-shard --listen 127.0.0.1:8090 --directory shard.fenec --token root

curl -X PUT localhost:8090/_shard/nodes/n1 -H 'Authorization: Bearer root' \
     -d '{"addr":"127.0.0.1:8081","token":"a1"}'
curl -X PUT localhost:8090/_shard/nodes/n2 -H 'Authorization: Bearer root' \
     -d '{"addr":"127.0.0.1:8082","token":"a2"}'
curl -X PUT localhost:8090/_shard/tenants/acme -H 'Authorization: Bearer root'
# {"tenant":"acme","node":"n1"}

curl -X POST localhost:8090/t/acme/query -d '{"query":"create collection notes (title text)"}'
curl -X POST localhost:8090/t/acme/notes -d '{"title":"hello"}'
curl localhost:8090/t/acme/notes
```

- **Same API.** Under `/t/<tenant>` everything works as
    on a single database; a client only changes its base URL.

Details

A single fenecdb is bounded by one machine's memory and one writer. Past
that, the data is split by tenant: each tenant is a file of its own, a node
holds many of them, and `fenec-shard` knows which node holds
which.

Below `/t/<tenant>` the surface is exactly the
[HTTP endpoint](https://fenecdb.com/docs//docs/http) of a single database — routes,
`/query`, `/batch`, subscriptions. A client changes its
base URL and nothing else; the [sync layer](https://fenecdb.com/docs//docs/sync) included:

```js
const db = await sync({ url: 'http://127.0.0.1:8090/t/acme', shapes: [...] });
```

## A file per tenant

Each tenant is its own file, chosen by the request path and never by the
query. Tenants cannot see each other's rows, and ids, change numbers, search
ranking and `lookup` all stay per tenant.

Details

The tenant is picked by the request path, never by the query text, so
nothing rewrites a query to add `tenant = ...`. Two tenants sharing a
collection would then read each other's rows. A file each keeps them apart by
construction, and the same decision settles four things a shared file would
have had to solve:

- **Ids** are counted per file, so they never collide and a moved tenant
    keeps its own.

- **The change sequence** behind `/changes` is one number per
    file, so a client's cursor stays a plain number.

- **BM25 statistics** — document count, average length, document
    frequency — are the tenant's own, which is what ranking inside a tenant
    should mean.

- **`lookup`** never crosses a file, so it stays one bucket
    probe per parent.

## Why tenants and not hash partitioning

A tenant lives whole on one node, so every query runs where all its data is
and means what it means on a single database. The cost: one tenant has to fit
in one node's memory.

Details

With a tenant whole on one node, every query runs where all of its data is:
`near`, `match`, `order`, `count`
and `lookup` keep their single-database meaning, and the router never
parses a query. Spreading one collection across nodes would need a merge for
every read shape — and global statistics before `match` could rank
anything, because per-node IDF gives scores that do not compare and a merged
order that is quietly wrong. That is a different feature, and the case for it
is a single tenant that outgrows a node. Until then, a tenant has to fit one
node's memory.

## The node: `fenec-server --dir`

A node is one `fenec-server` serving a directory of tenant files
instead of one file. The path names the tenant: `/t/acme/` is
`acme.fenec`.

```bash
fenec-server --dir /data/n1 --http 127.0.0.1:8081 --admin-token a1
curl -X POST 127.0.0.1:8081/t/acme/query -d '{"query": "get notes limit 5"}'
```

- **Opened on first use, closed when idle**
    (`--idle-close`, 300 s by default).

- **Unknown tenants are 404.** A request never creates one.

- **`/_admin/`** is how the router drives the node; it needs
    `--admin-token`.

- **A user's token is bound to its tenant.** A JSON Web Token reaches
    `/t/acme/` only when its `tenant` claim names
    `acme` ([Tokens](#tokens)).

- **Archived as a whole** by `fenec archive --tenants`,
    one connection for every tenant ([Archiving a
    node](#archive)).

Details

`--dir` serves a directory of `<tenant>.fenec`
files instead of one file. It is still one process per directory — the
single-writer rule is per file, and one process owning all of them is what keeps
it. `--http` is required. The tenant is looked up for every
request — so one created a moment ago is there, one closed as idle reopens,
one deleted meanwhile answers 404, and one frozen for a move refuses writes
while its reads go on.

- **Opened on first use.** Open cost is proportional to the file, so a
    node does not spend its start-up on tenants nobody asks for. A tenant that
    has not been created is 404 — a request never brings one into being, or a
    typo would make an empty database.

- **Closed when idle.** `--idle-close <s>` (default 300)
    syncs, checkpoints when there is a vector index, and closes a tenant nothing
    has touched for that long. One with an open request or subscription is never
    closed.

- **`--max-memory` covers the open tenants together.**
    Opening one more over the ceiling first closes idle tenants, least recently
    used first. With everything busy the open goes ahead anyway: refusing a
    read because other tenants are busy would turn a memory warning into an
    outage. A write that grows a tenant is refused once that tenant alone is
    past the ceiling, over either listener.

- **Shutdown** syncs every open tenant and checkpoints the ones with a
    graph, holding their write locks until exit.

The router drives the node through `/_admin/`, which answers only
to `--admin-token` and is off (404) without it — a client allowed to
read and write a tenant must not be able to delete it or take another tenant's
image:

| Route | Does |

| `GET /_admin/stats` | tenant count, bytes on disk, memory of the open ones |

| `GET /_admin/open` | the open tenants: name, memory, frozen |

| `GET /_admin/tenants` | names on disk |

| `GET /_admin/sizes` | each tenant's bytes on disk, which the router's `GET /_shard/tenants?bytes` joins into its list |

| `PUT /_admin/tenants/<t>` | create an empty tenant (409 if it exists) |

| `DELETE /_admin/tenants/<t>` | end its streams, close it, remove the file |

| `POST /_admin/tenants/<t>/schema` | a schema description, applied as `/_schema/apply` applies it: what the router sends for a tenant created with a schema |

| `POST /_admin/tenants/<t>/query` | a read, `{query, params}` as `/query` takes it, answered as `/query` answers it; a statement that writes is refused (403). What the router sends each tenant for a [query across tenants](#across-tenants) |

| `POST /_admin/tenants/<t>/freeze` | refuse writes and wait for the ones in flight |

| `POST /_admin/tenants/<t>/thaw` | accept writes again |

| `GET /_admin/tenants/<t>/file` | the whole image, graph included |

| `PUT /_admin/tenants/<t>/file` | install an image as a new tenant |

| `POST /_admin/tenants/<t>/promote` | a replica's file takes writes: its history forks there. `{"lineage", "history"}`, as the router sends them, refuse a copy of another lineage, or one lacking that history that has not heard from the node it follows since it was last told to follow it (409); without them, a copy with no write and no word from its primary since it opened or was told to follow is refused; `{"force": true}` promotes it whatever it holds, an audit event |

| `POST /_admin/tenants/<t>/follow` | the file follows `{"from": "http://host:port"}`, kept in `<t>.follows` beside it, or without it the node's `--replica-of` |

An image is replayed in memory before it is written, so a truncated transfer
is refused rather than installed, and the file appears by rename, never
half-written. Tenant names are `[a-z0-9_-]`, 1 to 64 characters:
they become file names, and that set means the same thing on every file
system.

## The router: `fenec-shard`

The router looks up which node holds a tenant and passes the request there
unchanged. Its only state is a small directory of nodes and tenants, managed
under `/_shard/`.

- **New tenants go to the node with the least data on disk** — in the
    region you name, if you name one ([Regions](#regions)) —
    unless you name the node.

- **A page on another origin needs `--http-cors` on the
    router**, as it would on a node: `fenec-shard --http-cors
    https://app.example`.

Details

The router reads a request's head and body, looks the tenant up, and forwards
it byte for byte — the original target, not a re-encoding of it. An answer with
a length is read and passed back on a kept-alive connection; one without is a
subscription, copied as it arrives until either side closes, so the stream
format lives in one place, the node.

Connections to a node are kept and reused, and each is looked at before a
request goes out on it: one the node closed for its silence
(`--idle-timeout`) is let go of. A request that fails before it is
wholly sent goes again on a fresh connection. One sent whole whose connection
ends before an answer goes again only where running it twice is harmless — a
`GET`, `HEAD` or `OPTIONS`, or a write with an
`Idempotency-Key`, which the node makes once
([Retrying a write](https://fenecdb.com/docs//docs/http#idempotency)). Any other write is
answered `502`, since the node may have run it: a client that sends
it again should send it with a key.

**Requests out to a node failed over.** A node that dies
with no reset — a machine lost, a network cut — answers nothing and closes
nothing, so what the router has out to it would wait for
`--upstream-timeout` (60 s), even once its tenants answer on
their new primaries. When the router fails a node over, by hand or on a
lease, it ends what it has out to the node for each tenant it promoted, as
soon as the directory points the tenant at its new primary, and answers
each by the rule above: a request the node cannot have run, a read, or a
write with an `Idempotency-Key` goes to the new primary — a keyed
write the old node made and sent its copy is answered from the key there,
one it never sent runs there once — and any other write is answered
`502`, saying it may have run. A held request (`Fenec-Wait`) goes by the same rule, with
what is left of its wait: the router cannot tell one still held from one
the node had just run as it died, and a claim run twice takes two jobs. A
subscription ends, for its client to open it again where the tenant is now.
A request whose answer had begun is left to the node, as is everything out
to a node only suspected — a grant missed, its lease still running — whose
writes may well land, and whose reads have nowhere better to go, and what
is out to the node for a tenant the failover left there. A request still
connecting to the node gives up within a second of the failover, and one
refused as it comes is not a `502`: both go to the new
primary. Under leases a promoted copy takes its tenant's
writes once the router's grant names it there, which the failover sends
each node it moved tenants to as it ends; a write sent there before is
refused unrun, and the router then grants the copy its list at once and
asks again — twice, since a grant under way for another tenant may have
read the list before this one was promoted. What is cut and what became of it is counted
(`fenec_router_cut_requests_total`,
[Monitoring](https://fenecdb.com/docs/monitoring#router)). A standby router, and a
region's router forwarding by its copy of the global directory, fail
nothing over themselves, but end what they have out to the node by the same
rule as soon as their copy of the directory shows the failover: the
directory names the node a failover took each tenant off, so a move ends
nothing, its source still answering. They grant nothing: a write one sends
on before the leading router's grant has reached the new primary is asked
again there until it has -- for two thirds of a lease or
`--upstream-timeout`, whichever is sooner, and only while its
client is still there. In the deploy kit's drill (`make drill`, a
lease of 2 s, the routers' timeout 4 s, a node cut off the network and
killed) the dead node's writers wrote again as the router failed it over,
1.93 and 2.52 s after the kill, where they waited for the timeout before,
3.94 and 3.95 s; none of their acknowledged writes was lost.

Its only state is the **directory**: two collections in a fenecdb file of
its own, `nodes` and `tenants`. Every change is written and
synced before it takes effect, and a tenant's node and state change in one
statement, so there is no moment on disk where a tenant belongs to both nodes or
to neither. The node admin tokens stay in it; `GET /_shard/nodes`
does not return them.

| Route | Does |

| `GET /_shard/nodes` | name, address, region and zone of each node |

| `PUT /_shard/nodes/<n>` | `{addr, token, region, zone}`; checked against the node before it is recorded; a region or zone left out is the one recorded |

| `DELETE /_shard/nodes/<n>` | refused while the node holds a tenant |

| `GET /_shard/tenants` | name, node, state and region of each tenant; with `?bytes`, its bytes on disk, asked of each node; with `?region=<r>`, one region's |

| `GET /_shard/tenants/<t>` | one tenant: its node, state, region and zone, and the node holding its copy |

| `PUT /_shard/tenants/<t>` | place and create; `{region}` to place it in a region, `{node}` to choose the node, otherwise the node with the fewest bytes on disk; `{schema}`, FenecQL `create collection` text, applied there before the tenant is answered (below) |

| `DELETE /_shard/tenants/<t>` | delete on the node, then forget |

| `POST /_shard/tenants/<t>/move` | `{to}`, `{region}` or both; see below and [Regions](#regions) |

| `POST /_shard/query` | one read on every tenant, or those named, the answers merged as the request says ([A query across tenants](#across-tenants)) |

| `GET /_metrics` | what the router answered and forwarded, the nodes' time and the ones it could not reach, and its moves, in Prometheus's text format ([Monitoring](https://fenecdb.com/docs/monitoring#router)) |

Placement goes by disk rather than memory: memory counts only the open
tenants, and which ones are open depends on who asked last.

A page on another origin — an app, the [sync
layer](https://fenecdb.com/docs//docs/sync), `FenecHttp.live`, the studio served by a node — reaches
its tenants through the router once the router has
`--http-cors <origin>`, read as `fenec-server`
reads it ([Security](https://fenecdb.com/docs//docs/http#security)): `*` or one
origin. The router answers a browser's preflight itself, before it looks the
tenant up, and puts the CORS headers on every answer it gives. Its own — an
unknown tenant's `404`, a standby's `503`, an
unreachable node's `502`, the `503` past
`--max-connections`, the `401` of `/_shard/` without its token — reach the
page as the refusals they are; without the headers a browser hands the page a
network error for each, and the sync layer backs off from what was an answer.
On a node's answer the router's headers take the place of the node's, so the
page is told the rule the router's preflight told it, and never two
`Access-Control-Allow-Origin`s, which a browser refuses whole. The
`503`s carry their wait as `Retry-After` and as
`retry_after` in the body, as a quota's refusal does. Without the
flag the router sends none of its own and passes a node's through as the node
wrote them, and a preflight goes to the node.

## A tenant with its schema

A tenant is created empty. Applying its schema afterwards took the nodes'
data token, which reaches every tenant on every node, so the app that
creates tenants -- one an organisation signs up -- held it. Hand the schema
to the router instead, and the app needs the router's token alone:

```bash
curl -X PUT localhost:8090/_shard/tenants/acme -H 'Authorization: Bearer root' \
     -d '{"schema":"create collection notes (owner text @hash, title text)"}'
# {"tenant":"acme","node":"n1","schema":{"kind":"schema","applied":true,...}}
```

- **Applied on the node** with the admin token the router holds for
    it (`POST /_admin/tenants/<t>/schema`), as
    `/_schema/apply` applies a description: one block, synced as
    a write is.

- **With its schema or not at all:** a schema that does not read is
    refused (400) before anything is placed, and one the node refuses takes
    the tenant back off the node and out of the directory.

- **Under leases** it is applied once the node's lease names the
    tenant, since the node takes no write before; a replica follows it as it
    follows every write.

## Quotas

Tenants share a node: its disk, its memory, its threads. A quota keeps one
tenant from taking all of them. Set defaults on the node, and a tenant's own
through the router, which sends it wherever the tenant goes.

```bash
# every tenant on this node, unless its own quota says otherwise
fenec-server --dir /data/n1 --admin-token a1 \
    --tenant-max-bytes 1G --tenant-max-rows 1000000 \
    --tenant-max-streams 50 --tenant-max-held 200 --tenant-max-requests 16 \
    --tenant-writes-per-sec 500 --tenant-write-bytes-per-sec 8M

# a tenant on a plan of its own: set as it is created ...
curl -X PUT localhost:8090/_shard/tenants/acme -H 'Authorization: Bearer root' \
     -d '{"schema":"create collection notes (title text)","quota":{"bytes":104857600,"rows":50000}}'

# ... or changed later: the whole set, 0 for none, left out for the node's
curl -X PUT localhost:8090/_shard/tenants/acme/quota -H 'Authorization: Bearer root' \
     -d '{"bytes":1073741824,"rows":0,"writes":100}'

# what it is held to, and what it uses
curl localhost:8090/_shard/tenants/acme/quota -H 'Authorization: Bearer root'
# {"tenant":"acme","open":true,"set":{"bytes":1073741824,"rows":0,"writes":100},
#  "quota":{"bytes":1073741824,"rows":0,"streams":50,...},
#  "usage":{"bytes":48211,"rows":312,"streams":1,"requests":0,"held":0}}
```

| Quota | Bounds | Past it |

| `bytes` | the tenant's file, as written | **507**, the write put back whole |

| `rows` | the documents of its collections, together | **507**, the write put back whole |

| `streams` | subscriptions, `/_changes` waits and `Fenec-After` waits while they wait: a read after a write the node holds takes none | **429**, `Retry-After: 1` |

| `held` | writes held until they can write ([`Fenec-Wait`](https://fenecdb.com/docs/queues#wait)): workers waiting for jobs | **429**, `Retry-After: 1` |

| `requests` | requests in flight | **429**, `Retry-After: 1` |

| `writes` | rows written a second, charged as each write lands | **429**, `retry_after_ms` until a row's worth is back |

| `write_bytes` | bytes written a second, charged as each lands | **429**, `Retry-After` until the debt is paid |

A refusal names its quota in the body and in a header, and every SDK's
error carries it typed (`quota`, `retryAfter`). A
`429` says its wait twice: in whole seconds, rounded up, in
`Retry-After` and `retry_after`, as HTTP has it, and to
the millisecond in `retry_after_ms`, which every SDK and a
replica's sync read first -- a bucket of 600 rows a second refills a row in
1.7 ms, and fifty couriers told to come back in a second waited fifty
seconds between them:

```bash
HTTP/1.1 429 Too Many Requests
Fenec-Quota: writes
Retry-After: 1

{"error":"the tenant writes past its quota of 600 rows a second","quota":"writes","retry_after":1,"retry_after_ms":2}
```

```bash
HTTP/1.1 507 Insufficient Storage
Fenec-Quota: rows

{"error":"the write would take the tenant past its quota of 50000 rows: delete rows, or raise the quota","quota":"rows"}
```

- **A write that writes nothing costs nothing.** `writes`
    counts the rows of the tenant's collections a write landed -- a put of
    ten rows ten, a claim that took no job, a `require` unmet or
    a held claim looking none -- and lets a write through while the bucket
    holds a row's worth, so one of many rows goes through and the next
    waits out the debt. Charged as a write started, 45 claims held at once
    by a fleet on a small plan were refused for `writes` before
    any for being held. A held claim the rate refuses as it is to run stays
    held, and runs once a row's worth is back. The node's own landings --
    the sweep of rows past their `@ttl`, a compact -- are not
    the tenant's rate.

- **A tenant waits in a share of the node.** Every stream and every
    held write is a connection and its thread, and a node's
    `--max-connections` is what all its tenants share:
    `--http-max-streams` counts each tenant's own subscriptions,
    and nothing counted a tenant's held writes, so a hundred workers of one
    tenant waiting for jobs took every connection of a default node and its
    neighbours were answered 503. Where the node sets neither
    `--tenant-max-streams` nor `--tenant-max-held`,
    each is a quarter of `--max-connections` -- 25 of a default
    node's 100 -- so no one tenant's dashboards and workers hold more than
    half; a node serving a fleet raises `--max-connections`
    (each is about 50 KB of memory) and sets its plans' quotas.

- **Deletes always land.** A write that only takes away — a
    `del`, a `drop` — is never refused, and neither is
    a `compact`, which gives bytes back. A tenant at its quota
    deletes its way under it.

- **A replica's sync knows them.** A synced write a `507`
    refuses is put back on the replica and the app told, the quota named,
    and the writes behind it -- a delete that makes the room among them --
    go on; a `429` is waited out for its `Retry-After`,
    as a move's `503` is
    ([Sync](https://fenecdb.com/docs/sync#refusals-retries-and-offline)).

- **A full file is compacted by the node** once an eighth of its quota
    is dead, where a file of its own waits for 64 MB: deleted rows become room
    without the tenant running `compact` itself.

- **Never passed by a write that adds.** Bytes and rows are asked as
    the write lands, its record built, under the write lock: writers at once
    cannot take the file a byte past its quota, nor the rows one past
    theirs.

- **It goes with the tenant.** The router keeps each tenant's quota in
    its directory and sends it to the tenant's node, its standby and its
    replica; a move sends it to the target before the directory points
    there, and a failover finds it on the copy it promotes.

- **Seen by quota, not by tenant:** `/_metrics` counts
    refusals as `fenec_quota_refused_total{quota="rows"}`, the
    audit log writes each as a `quota` event naming the tenant,
    and `GET /_admin/quotas` on a node — or
    `/_shard/tenants?quota` on the router, which the studio's admin
    view shows — lists every tenant's usage against its quota.

Details

**What is bounded is what is known exactly and cheaply.** A file's
bytes are the engine's count of what it appended, read with no system call;
rows are each store's count, a sum over the collections (those whose names
start with `_` are the server's: idempotency keys, consumers'
places); streams and requests are counters a request takes and gives back;
the rates are a token bucket each. A tenant with no quota touches none of it.
Left out: the memory an open tenant holds (`memory_bytes` counts
what has been built, and a read builds an index on first use — a quota on it
would refuse a write because of a search, and the same data holds less after
a restart; `--max-memory` closes idle tenants for the node as a
whole), a statement's time (the engine cannot stop a running scan; requests
in flight bound how many cores one tenant's long reads take), and rows per
collection (a tenant's own design: `@ttl` on a log).

**The block is refused before it lands.** A quota asked before a write
runs would let that write past it by whatever it adds, and a
`set` over every row adds the whole collection again. The engine
asks the quota as a block lands, with its record's length to the byte and
its documents counted (`Database::set_limit`); one past it is put
back as a failed one is. A block larger than 16 MB spills into the file
before it lands, and each spill is asked first: let through and refused
after, its bytes would stay in the file, dead and past the quota. The file
passes its quota only by what a write that takes away adds — a delete's
tombstones, a drop's record — and by what no client writes: the graphs a
server keeps in its file's tail. The next compact gives both back. A rate is
not asked at the landing: a write lands while its bucket holds a row's worth
or any bytes, and its rows and bytes are charged as it lands -- the rows of
the tenant's collections it wrote, not the server's own, such as an
`Idempotency-Key`'s -- so a write larger than a second's worth
goes through and the next waits out the debt.

**A refusal tells the quota, not what the tenant holds:** a scoped
token's user is told the tenant is full, not how many rows the other users
make. Only the admin token reads usage.

**The router enforces none of it.** It holds no keys and parses no
query, and a tenant lives whole on one node, so the node's count is the
tenant's. A quota a node does not take — a node down when it was set — is
logged, and sent again as that copy is promoted.

**What it costs.** A tenant with no quota touches none of it: against
the code before quotas, in turns, a request to a tenant took 19.5–21.7 µs
straight to the node and 43.7–46.0 µs through the router, against
19.9–21.8 and 43.6–45.7. A tenant with every quota answered as one with none
over HTTP — reads 20.9 µs either way, writes 23.7 against 23.6 — and in
process a `put` held to bytes and rows took 593 ns against 566,
618 with a bytes-a-second bucket besides (`make quota-bench`).

## Moving a tenant

Move a tenant to another node with one call. Reads keep working throughout;
writes are asked to retry for the moment the copy takes.

```bash
curl -X POST localhost:8090/_shard/tenants/acme/move -H 'Authorization: Bearer root' \
     -d '{"to":"n2"}'
# {"tenant":"acme","from":"n1","to":"n2","bytes":58685634,"ms":267,"source_removed":true}
```

- **Writes during a move** get 503 with `Retry-After: 1`,
    and `"retry_after": 1` in the body, which a replica's sync
    waits out ([Sync](https://fenecdb.com/docs/sync#refusals-retries-and-offline)).

- **Subscribers reconnect** and, if they were caught up, carry on
    without a reseed.

- **A failed move** puts everything back on the source.

Details

- The directory marks the tenant `moving`; it is still routed to
    the source.

- The source freezes it: writes in flight finish, new ones get
    **503 with `Retry-After: 1`**. Reads go on being served.

- The image is copied from the source and installed on the target.

- The directory points the tenant at the target, in one statement.

- The source copy is deleted — once every [standby
    router](#standby-router) keeping up has been sent the new placement, a second at most,
    so one that forwards from it sends the tenant's requests to the target
    rather than to a node that no longer holds it. Its open subscriptions end with an
    `error` event, and the client reconnects through the router to
    the target. A client that had seen everything **resumes from its cursor
    without a reseed**: the change sequence travels in the image and the
    source was frozen, so nothing happened in between. One that was behind is
    reseeded, as after any restart — the target has the sequence but not the
    history before it.

The router refuses nothing itself during a move, so there is no window in
which the tenant is nowhere — only one in which writes are asked to retry. If a
step before the flip fails, the source is thawed and the directory put back. A
router that stops mid-move comes back with the tenant still served from the
source and marked `moving`; moving it again replaces whatever copy the
interrupted attempt left on the target, which is stale by definition, since the
directory never pointed at it.

## Archiving a node

Every tenant of a node is archived by one process over one connection to
the node — not a connection, a thread and an open tenant each — an archive
a tenant under a root, which restores a tenant or the whole node to a
moment.

```bash
fenec archive --tenants http://n1:8081 /backups/n1 --token "$REPL" --image-every 1h --keep 7d
fenec verify  --tenants /backups/n1
fenec restore --tenants /backups/n1 /data/n1 --to 2026-09-22T10:14:59Z
```

- **Moved or deleted:** a tenant that leaves the node is no longer
    followed and its archive is kept, noted gone, and noted back if it
    returns; the target node's archiver takes it up from its image. A
    restore of the node to a moment it was away leaves it out, as it does a
    tenant deleted before.

- **Restored in place:** a tenant restored on its primary's node
    keeps its lineage, and fails over as before once its copies have taken
    the restored file ([automatic
    failover](#automatic-failover)).

- **Replicas and standbys** can be archived the same way: their
    tenants' writes are the primary's, numbered alike, and a promotion goes
    on in the same archive.

The node needs `--replication-token`. What it costs and how
the archive follows tenants made, closed, moved, restored and promoted:
[Archiving every tenant of a
node](https://fenecdb.com/docs//docs/replication#node-archive).

## A query across tenants

An operator's question about every tenant — how many orders each took
since yesterday, the twenty newest sign-ups of all of them — is one request
to the router, which runs the read on each tenant and merges the answers as
the request says.

```bash
# each tenant's count, a row each with its tenant
curl -X POST localhost:8090/_shard/query -H 'Authorization: Bearer root' \
     -d '{"query":"get orders select count() as n where at >= $1","params":["2026-10-11T00:00:00Z"]}'
# {"rows":[{"tenant":"acme","n":31},{"tenant":"globex","n":7}],"tenants":2,"answered":2,"failed":[],"ms":2}

# the twenty newest of all of them: each tenant's twenty, merged
curl -X POST localhost:8090/_shard/query -H 'Authorization: Bearer root' \
     -d '{"query":"get users select email, created order created desc limit 20",
          "merge":{"order":["created desc"],"limit":20}}'

# counts added up by day, over the tenants whose names start with eu-
curl -X POST localhost:8090/_shard/query -H 'Authorization: Bearer root' \
     -d '{"query":"get orders select bucket(at, 1d) as day, count() as n group day",
          "merge":{"sum":["n"],"group":["day"]},"prefix":"eu-"}'

# the first twenty names of all of them in Turkish's order, a field
# declared `name text collate tr nocase`
curl -X POST localhost:8090/_shard/query -H 'Authorization: Bearer root' \
     -d '{"query":"get people select name order name limit 20",
          "merge":{"order":[{"column":"name","collate":"tr","nocase":true}],"limit":20}}'
```

- **The router's token alone.** A user's token, bound to its tenant,
    and the nodes' data token are refused (401).

- **Reads only.** A statement that writes is refused on every node
    (403) and writes nothing.

- **Never cut.** A tenant that fails or times out fails the query,
    named; `"partial": true` answers the others' rows with the
    failures listed. The rows and bytes gathered are bounded, and a query
    past either is refused (413).

- **Each answer is its tenant's own.** Tenants are read at different
    moments: there is no one moment across them.

Details

Tenants are the isolation boundary, and this crosses it, so it is a
`/_shard/` endpoint behind the router's `--token`. A
JSON Web Token is refused there as any token other than the router's is:
the router holds no keys, and a token bound to one tenant must not reach
the others through it. The router asks each tenant through its node's
`POST /_admin/tenants/<t>/query` with the node's admin
token, which it holds already: a token that can take a tenant's whole
image (`GET /_admin/tenants/<t>/file`) learns nothing more
by reading it, where the nodes' data token, which nothing else here needs,
would reach every tenant and write. The node refuses a statement that
writes before it runs, and runs the rest in read-only mode besides.

The router still parses no FenecQL. The statement and its parameters go
to every node as they came, and what is done with the rows is said in the
request, never read out of the query:

| Member | Does |

| `query`, `params` | the statement each tenant runs, as `/query` takes it |

| `tenants` or `prefix` | the tenants named, or those whose names start so; every tenant in the directory without either. A name the directory does not hold is refused (404), nothing asked |

| `merge` | none: the rows one after another, tenants in name order, each with its tenant's name first; `{"order": [..], "limit": n}`; `{"sum": [..], "min": [..], "max": [..], "group": [..]}` (below) |

| `column` | the name of the tenant's column, `tenant` by default |

| `partial` | answer the rows of the tenants that answered, the failures listed beside them |

| `timeout_ms` | the query's one timeout, under the router's `--fanout-timeout` (30 s) |

Three merges, each exact from the tenants' answers alone:

- **One after another**, the default: every row, its tenant
    named.

- **An order**: a k-way merge by the named columns, each
    `asc` or `desc`, of rows each tenant gives in that
    order. With each tenant's own `order ... limit 20`, the first
    twenty of the merge are the first twenty of all of them. The router
    checks each tenant's rows come in the order, and a tenant whose do not
    — its statement orders otherwise — fails, never merged wrong. Values
    compare as the engine compares them, text by its bytes; a timestamp
    travels as ISO text, which leaves its milliseconds out where they are
    0, so a text that is one compares as its time. Ties go in tenant name
    order. A key whose text is not ordered by its bytes says how it is
    (below).

- **A fold**: `sum`, `min` and
    `max` of the named columns, by the `group`
    columns, groups in order. These are the aggregates that combine across
    tenants: a count is a sum of counts. An average of averages is not an
    average, so there is none: sum its sum and its count. Every column of a
    row must be grouped by or folded, or the tenant fails: a column the
    merge would drop is an answer cut. A sum of whole numbers past 64 bits
    is refused, as the engine refuses one. A `min`,
    `max` or `group` column may be an object naming
    its text order as an order's key is: a tenant's `min` over a
    collated field is the collation's least, so the least of those is too,
    and a `nocase` group is one for every text folding to it,
    under the first tenant's text.

A column whose text the tenants compare otherwise than by its bytes is
an object in place of its name: `{"column": "name", "collate": "tr",
"nocase": true, "desc": true}` where `"name desc"` would
be. The router reads no schema, so what the field declares —
`collate tr` or `und`, `nocase` — or what
the statement's `order name collate tr` names is said again
here, and the column compares by the engine's own rule: the collation over
the case folds, the texts that fold to one tied, and a text read as text
even where it looks like a timestamp. `"nocase": true` folds as
a field in the column's collation does, Turkish's under `tr`
(`I` the capital of `ı`) and Unicode's otherwise;
where the statement's `collate` names another than the field's
own, `"nocase"` names the field's, as `"nocase":
"und"` for a plain `nocase` field ordered `collate
tr`. A collation other than `tr` or `und` is
refused (400), named, and `desc` is an order's alone. Not told,
the router compares bytes: the tenants whose rows come in another order
fail, and a `min` or `max` is the bytes' least or
greatest of the tenants' own.

A tenant that fails — its node unreachable, its call past the timeout,
an answer that does not fit the merge, or one the node refused — is named
with its node, the status and why. Without `"partial": true`
the query fails with every failure named; the first stops the asking, and
the answer says how many tenants were not asked. Its status is the
tenants' own where they all failed alike — a statement every node refused
is 400 or 403 — and 502 otherwise. A tenant moved while the query runs is
asked again on the node the directory names now; one deleted meanwhile is
a failure. The tenants come from the directory as the query begins: one
created after that is not in it.

The rows gathered from the tenants together are bounded by
`--fanout-max-rows` (100 000) and the bytes of their answers by
`--fanout-max-bytes` (64 MiB), counted as the answers arrive: a
query past either is refused (413), its partial rows or not, and the
asking stops. The calls go out over the router's connections to the nodes,
`--fanout-concurrency` (16) at once, the tenants taken node by
node in turn, and each tenant's answer is read for the merge on the thread
that fetched it, beside the calls still out.

Each tenant's answer is one statement's, as `/query`'s is.
Tenants are separate databases read at different moments, so two answers
are of no one moment, and a write landing in one while the query runs may
or may not be in its answer. A read that opens a tenant opens it on its
node, which closes it again once idle.

A query across tenants is a span of the router's trace, a span a node
under it, and each tenant's read on its node under that node's span, all
with the request's id; the router counts them in
`fenec_router_fanouts_total` and two more (Monitoring), no label naming a tenant.
Each is a line of the router's audit log; the nodes' reads for it are not,
as a read is not.

Over 100 tenants of 200 rows on three nodes, on one machine
(`make shard-bench ARGS=fanout`):

|   | a count, summed | the 20 newest, merged | the 20 first names, `collate tr nocase` |

| `POST /_shard/query` | 1.15–1.23 ms | 1.35–1.39 ms | 1.33–1.48 ms |

| each tenant through the router, in turn | 4.28–5.16 ms | 5.11–5.45 ms | 4.34–5.62 ms |

| each tenant through the router, 16 at once | 1.92–1.98 ms | 1.86–2.17 ms | 1.90–2.21 ms |

p50 over 300 rounds in three runs, the names' in six beside other
work on the machine; the client asking 16 at once merges nothing. A merge
in a collation over case folds costs what one by a timestamp does. A
request forwarded to one tenant costs what it did.

## What it costs

The router adds about 21 µs a request on one machine: one extra round trip.
Moving a 58.7 MB tenant took under 0.3 s.

Details

Node and router in one process on loopback, one keep-alive client, 5 000
requests after 500 of warm-up (`make shard-bench`):

|   | p50 | p95 | p99 |

| straight to the node | 23.1 µs | 31.5 µs | 38.1 µs |

| through the router | 44.7 µs | 53.2 µs | 64.0 µs |

About 21 µs per request, which is one more loopback round trip; across a
network the extra hop is whatever that network charges for one. Moving a tenant
of 100 000 × 128 with an HNSW index — a 58.7 MB image — took 188–267 ms end to
end over three runs, and the first query on the target about 70 ms, most of it
opening the file: the graph travels in the image, so nothing is rebuilt.

## What it does not do

No join across tenants -- a query across them is an operator's read,
merged as it says -- no tenant bigger than a node, and only one router
makes changes at a time. Failover is never guessed.

Details

- **Failover is decided on a lease, not a guess.** A router cannot
    tell a node that is gone from one it cannot reach, and guessing would
    make two primaries for one tenant. Without `--auto-failover`
    someone who knows the node is gone calls the failover; with it, the
    router promotes a node's tenants only once the node's own lease has
    certainly lapsed, and the node stops its writes as it does (Automatic failover).

- **No join across tenants.** A data request names one tenant. The
    router's [query across tenants](#across-tenants) runs one
    read on each and merges the answers by a rule the request states --
    one after another, the first rows of an order, sums and extremes by
    group -- and never parses the query: nothing joins one tenant's rows to
    another's, and the answers are each tenant's own, of no one moment.

- **No tenant larger than a node.** A tenant has to fit one node's
    memory, with the same 2× open and 3× checkpoint peaks as any file.

- **One router takes changes at a time.** A standby follows its
    directory and forwards from the same placements, but only the primary
    places, moves and deletes; promoting the standby is a decision someone
    makes, not a vote the routers hold.

## Replicas of tenants

Pair a node with a standby node, and every tenant on it is copied there as
it is written. When the node is lost, one call promotes its tenants on the
standby.

```bash
fenec-server --dir /data/n1 --http 127.0.0.1:8081 --admin-token a1 \
         --replication-token r1 --sync always
fenec-server --dir /data/n2 --http 127.0.0.1:8082 --admin-token a2 \
         --replication-token r1 --replica-of http://127.0.0.1:8081

curl -X PUT localhost:8090/_shard/nodes/n1 \
  -d '{"addr":"127.0.0.1:8081","token":"a1","standby":"n2"}'
```

- **Failover:** `POST /_shard/nodes/<n>/failover`.

- **Use `--sync always` on the primary** so a failover
    loses no write.

Details

Every tenant file has a feed of its own, so a node's tenants are followed
by another node's — the standby — tenant by tenant, through
`/t/<tenant>/_replication`.

The router creates a tenant on both, so the replica has a file to follow
into, and deletes it from both; a move takes the replica with it. Recording a
pair gives the standby a following copy of every tenant already on the node.
A replica node opens every tenant it holds at start and keeps them open — a
follower that is not running is a replica falling behind — so
`--idle-close` does not apply to them. No tenant is placed on a
standby or moved to one: it would open there as a replica of a tenant the
other node does not have.

A tenant's role is its file's, not the node's. On a node started with
`--replica-of`, a replica's file follows and a new one follows,
but a primary's file stays a primary — a tenant a failover promoted there
included, whatever the node restarts with. Were every file on such a node
made to follow, a restart with the same flags would undo the failover, and
the old primary's copy would wipe the writes taken since.

When a node is gone, `POST /_shard/nodes/<n>/failover`
promotes its tenants on the standby, one at a time, and points the directory
at it. They are separate databases: there is nothing to make atomic between
them, so the answer lists what was promoted and what was not. A copy that
is not its tenant as the directory knows it — of another lineage, or an old
primary's copy that never heard from the new one — is refused by its node
and listed as not promoted, the tenant left where its rows are; with
`-d '{"force": true}'` it is promoted whatever it holds, which
the audit log records. Under `--auto-failover` the router grants
each node the tenants went to its lease over them before it answers, so
they take writes as soon as the answer comes: left to the next round, a
promoted copy refused them for up to a third of a lease.

```bash
curl -X POST localhost:8090/_shard/nodes/n1/failover
{"node":"n1","to":"n2","promoted":["acme","beta"],"failed":[]}
```

Measured with `make shard-bench`, node, standby and router in
one process on loopback: a write through the router is answered in 2.93 ms
(`--sync always` on the primary, which is an fsync a write) and is
on the standby 0.089 ms later, 0.448 ms at p99 — timed by reading the
standby's own copy. A failover of 20 tenants took 63.5 to 79 ms, the
medians of three rounds of four to twelve runs, 3 to 4 ms each, and the
first write on the promoted node 0.18 ms.

A promoted tenant's history forks where the replica stood, so what the
primary had not sent is gone — the same trade a database's failover makes,
and `--sync always` on the primary is what makes it none. A
failover that moved every tenant ends the pair; one that left a tenant behind
keeps it, so a second call can take it. The old node rejoins as the standby of
the new one: point its `--replica-of` there and record the pair the
other way round. The router then has each of its tenants follow
(`POST /_admin/tenants/<t>/follow`) — its own files are a
primary's, and follow only when told to — and the writes it took that were
never sent are gone.

```bash
fenec-server --dir /data/n1 --http 127.0.0.1:8081 --admin-token a1 \
         --replication-token r1 --replica-of http://127.0.0.1:8082
curl -X PUT localhost:8090/_shard/nodes/n2 \
  -d '{"addr":"127.0.0.1:8082","token":"a2","standby":"n1"}'
{"node":"n2","unreplicated":[]}
```

A tenant file that still follows on a node that follows nobody — a failover
that left it behind, and the node restarted without `--replica-of`
— refuses writes, says so in the log, and is promoted where it stands with
`POST /_admin/tenants/<t>/promote` (with
`-d '{"force": true}'` where it holds no write and has heard
nothing from its primary since it opened).

## Replicas of their own

Instead of an idle standby, `fenec-shard --replicas` gives each
tenant its own replica on another node, so every node holds both primaries
and replicas and a lost node's tenants spread over the rest.

```bash
fenec-server --dir /data/n1 --http 127.0.0.1:8081 --admin-token a1 --replication-token r1 --sync always
fenec-server --dir /data/n2 --http 127.0.0.1:8082 --admin-token a2 --replication-token r1 --sync always
fenec-server --dir /data/n3 --http 127.0.0.1:8083 --admin-token a3 --replication-token r1 --sync always
fenec-shard --replicas

curl -X PUT localhost:8090/_shard/tenants/acme -d '{"node":"n1"}'
{"tenant":"acme","node":"n1","replica_on":"n2"}
```

- **After a failover**, `POST /_shard/replicas` gives every
    tenant without a replica a new one.

Details

A standby waits idle, and takes every tenant of its node at once when the
node goes. With `--replicas` the router gives each tenant a
replica of its own instead, on another node in no pair: the one holding the
fewest replicas of that tenant's node, then the fewest tenants and replicas
altogether. Each node's tenants then follow on all the others, and every
node holds primaries and replicas both.

The replica's node is told which node to follow
(`POST /_admin/tenants/acme/follow` with
`{"from": "http://127.0.0.1:8081"}`) and keeps it in
`acme.follows` beside the file, so the tenant follows again when
the node starts over. The replication token is the nodes', one for the
cluster. A move takes the replica along: the copy goes on following the
tenant on its new node, since the image carried its history, and when the
tenant moves onto the node holding its replica, that copy goes and another
is placed. A delete removes both.

A failover of a node in no pair promotes each of its tenants on its own
replica, one at a time, and routes it there. The tenants whose replicas it
held are named as unreplicated. `POST /_shard/replicas` gives every
tenant without a replica one, on a node holding a copy of it first: a node
back from a failover has its old primaries follow the nodes that took them
over, and nothing of theirs is left lying there.

```bash
curl -X POST localhost:8090/_shard/nodes/n1/failover
{"node":"n1","promoted":[{"tenant":"acme","to":"n2"},{"tenant":"beta","to":"n3"}],"failed":[],"unreplicated":["gamma"]}
curl -X POST localhost:8090/_shard/replicas
{"replicated":[{"tenant":"acme","replica":"n1"},{"tenant":"beta","replica":"n1"},{"tenant":"gamma","replica":"n1"}],"failed":[]}
```

Measured with `make shard-bench`, three nodes and the router in
one process, 30 tenants: each is created with its replica in 33 ms, the
replicas 10 a node, and a write is on its replica 0.179 ms after it was
answered, 0.421 ms at p99. A failover of one node's 10 tenants took 57 ms, 5
onto each of the other two, and the repair after it gave 20 tenants a
replica in 176 ms, all on the returning node's copies. Placed by the fewest
altogether alone, every tenant of one node followed on the same other: a
tie went by name each time, and its failover put them all there.

## Automatic failover

With `--auto-failover` the router fails a node over by itself.
It works on leases: a node stops writing when its lease runs out, and only
after that does the router promote its tenants elsewhere, so a tenant never
has two primaries.

```bash
fenec-server --dir /data/n1 --http 127.0.0.1:8081 --admin-token a1 \
         --replication-token r1 --sync always --lease
fenec-shard --listen 127.0.0.1:8090 --directory shard.fenec --replicas --auto-failover 5
```

- **Nodes need `--lease`**; a node without it is never
    failed over automatically.

- **With a 5-second lease**, a lost node's tenants cannot be written
    for 5.5 to 7.2 s. A node cut off from the router while its clients still
    reach it stops writing two thirds of a lease after its last renewal at
    the most: its lease runs from the grant before that one.

- **What was out to the lost node is answered at its failover**, not
    at `--upstream-timeout`: a read or a keyed write goes to the
    new primary, any other write is a `502` that it may have
    run ([Requests out to a node failed over](#in-flight)).

Details

With `fenec-shard --auto-failover <s>` the router fails a
node over on its own. It leases each node the tenants it places there for
that long, renews the lease as it starts and every third of it after while
the node answers, and fails over a node that has gone a tenth past its lease
unrenewed. A node
started with `--lease` writes a tenant only while its lease names
the tenant and has not lapsed, and starts with none.

The router still cannot tell a node that is gone from one cut off from it
while its own clients reach it, and it does not have to. A node that hears
from the router no more stops taking writes when its lease lapses, by its
own clock, and the router promotes its tenants elsewhere only a tenth of a
lease later, measured from the answer that came back rather than from the
grant it sent — so no tenant has two primaries, whichever it was. What such
a node took from clients of its own before its lease lapsed, and had not
sent its replicas, goes with the failover as a crash takes it; a client
going through the router loses nothing that way, since the router could not
reach the node either. A write is refused as a replica's is: `503`
with `Retry-After`.

That holds of a grant however late the node reads it. Each grant is
numbered and names the newest grant whose answer came back to the router in
time, and the node runs its lease from the moment it answered that one,
never from when it reads a grant. A grant read late — held on its way, or
left unread by a node stopped or slow — acknowledges only answers the router
had already counted, whose lease ran out before the failover; a grant whose
answer never came back is acknowledged by no grant after it. Neither gives
the node a lease, so a node that reads grants waiting in its socket after
its tenants were promoted elsewhere takes no write for them: with the lease
run from when it read them, as it was, it took a whole lease's writes beside
the new primary. A grant newer than the one the lease came from — by the
answer it acknowledges, then by its own number — is taken whole, its list
with its lease, which may cut the lease held short: a router restarted or
promoted with a shorter lease shortens the node's with its first
acknowledged grant. One that is not newer is taken not at all while a lease
runs, so a late one cannot hand back a tenant failed over elsewhere, and a
node says so in its answer (`took`), which has the router send
the grant acknowledging it at once, with the list. The price is about a
round: a node's lease runs from the grant before the last it read, so a
node cut off stops writing a third of a lease sooner than it did, while the
failover comes when it did; and a node holding no lease — new, restarted,
back from a failover — needs a second grant, acknowledging the first, which
the router sends at once. A router from before numbered grants sends none,
and a node takes its grants from when it reads them, as before: a late grant
can still outlast a failover until every router is upgraded. A node from
before them is leased as before too, and the router says so once in its
log.

A node measures its lease on its machine's boot clock —
`CLOCK_BOOTTIME` on Linux, `mach_continuous_time` on
macOS — which counts the time the machine was suspended. A node woken from
a suspend that outlasted its lease holds none, and the grant it reads on
waking acknowledges an answer from before the suspend, which runs nothing
past that answer's lease. Measured on the monotonic clock, as it was, a
suspend did not count: the node woke with the lease it had left and wrote
beside its tenants' new primaries until it ran out. The router counts on
its own monotonic clock, which its machine's suspend stops, so a router
suspended fails over later, never sooner.

A write is checked against the lease as it lands, and again as it is
answered. One that landed while the lease ran and is answered after it
lapsed — its process stopped or starved between the two, its machine
suspended, or its fsync or its wait for the replicas' streams outlasting
the lease — is answered `500` with `"outcome":
"unknown"` ([statuses](https://fenecdb.com/docs/http#statuses)): it is in the
node's file, and its tenant may have been failed over without it or may
still be on the node with it, which the node cannot know. Sent with an
`Idempotency-Key`, send it again with the same key: the node
refuses it while its lease has lapsed, before it looks the key up, and once
the tenant settles a node still holding the write answers it as made, and a
new primary makes it once. Sent with none, read it back — but not at once:
the node still answers reads, the write among them, and routers send them
there until they fail the tenant over, a lease and a tenth
(`--auto-failover` × 1.1) after the answer, while
the copy promoted may lack it. After that, and not before, read it back
through the router, or send a read with `Fenec-After` and
`Fenec-History` set to the `seq` and
`history` the answer's body names, which a primary holding the
write answers and one without it refuses 504 — sent at once, that read is
answered 200 by the lapsed node, whose history holds the write. The body
names them, not `Fenec-Seq`, which a client would take as its
own position. Answered as a success, as it was, a failover lost it. A
`/batch` holding a `compact`, whose statements run on
their own, is answered so too where one of them landed before a later one
was refused, with `completed` and `at` (it was told
to come back, 503, as if none had); its `seq` is the last landed
statement's, so a 200 after it says every landed statement survived, and a
504 does not say which earlier ones did, which are read back.

What leases do not cover. A virtual machine stopped by its host is no
suspend to its kernel. Under QEMU/KVM's defaults, x86 and arm64, a guest
stopped and continued (`virsh suspend` and `resume`)
finds every clock where it stopped, the boot clock too, and so does a guest
saved and restored or live-migrated under any of its settings; so do
Hyper-V's saved state and Azure's freezes for maintenance, VMware's suspend
and Firecracker's snapshot restore. A node paused so wakes with lease left
and writes beside its tenants' new primaries until the lease runs out or it
reads the router's first grant after the failover, which names none of them
— with no late grant needed — and nothing inside the guest tells it soundly
that it was paused: every signal there is comes once it runs again. Stop
the node before its machine is paused, saved or migrated (`systemctl
stop fenec-node`), on a cloud's notice where it gives one (Azure's
Scheduled Events, GCE's `maintenance-event`), and where a host
may pause it unannounced, keep the lease short: two primaries last no
longer than the lease the node had left. The
[runbook's
known gaps](https://github.com/fenecdb/fenec/blob/main/deploy/RUNBOOK.md#known-gaps) say which pauses are counted and which are not. An answer that passed the second
check just before its process stopped still goes out whenever the process
runs again. A stall of sixteen rounds or more can leave a node without the
answer its next grant acknowledges, holding no lease until the grant after
it, which the router sends at once. Two directories leasing one node — a
misconfiguration — can take it in turns as each other's lease lapses,
neither holding one for long.

A router may start with a shorter lease than the one before it —
`--auto-failover` lowered, then a restart or a standby promoted.
A node it never reaches may still hold the longer lease, so the directory
records the longest lease a router leasing from it may have granted, raised
before a router's first grant with a longer one and sent to its standbys
first. A router counts a node by that record until the node has run a lease
from one of its grants, which the node says in its answer
(`ran`); from then on, by its own lease, since that grant cut the
longer one short. A grant taken for its list alone, by a node whose lease
had lapsed, runs none, and a late grant of the router before could still run
the longer lease again. Once the router has led for the recorded lease and a
tenth, every lease granted before it has run out, and it lowers the record
to its own. Where the record cannot be written, a grant runs no longer than
the record, renewed every third of that, or the router's own lease where
there is none.

What the record does not cover: it is a directory's own, so a node
leased by one directory and then by another — the global one's, then a
region's — is counted by the second's record, which knows nothing of the
first's lease. And a router from before the record writes none, so the first
router with it after an upgrade starts with no record, and counts by its own
lease. Lower `--auto-failover` at a restart after that one; never
at the upgrade itself.

The lease names tenants, not the node. A node failed over comes back with
the files of tenants served elsewhere now, primaries' files still; a lease
over the whole node would have them take writes again the moment it was
renewed. Its new lease names none of them, and the router runs a repair as
the node answers, which has those copies follow the tenants' new primaries.
The list rides with a grant only when it changed, and a create or a move
grants the node its new one at once, so a tenant takes writes as soon as it
is placed. A router that starts leasing — a restart, or a standby promoted —
counts every node as granted then, at its first round as the primary: its
predecessor may have granted a lease just before it stopped. A grant names the directory's term, which a
promotion moves on, and a node refuses grants from a lower term once it has
taken a higher one, so a router promoted over one still running fences it at
every node it reaches ([Fencing](#region-directories)). A node started without `--lease` answers
that it takes none, and is never failed over on its own.

No node waits for another. Each node's grant goes out on a thread of its
own and the router judges the leases without waiting for any, so a node
slow to answer — or to be found at all: one recorded by a name its resolver
is slow to refuse once the machine is gone — holds up its own renewal and
its own failover, nobody else's. A name is looked up within the call's own
bound, a third of the lease for a grant; the repair a node's return
triggers runs beside the leasing, as do failovers; and a replica being
promoted stops reconnecting to the node it followed at once rather than
wait out the connect. A node's lapse is judged only once the router has
stopped waiting for any grant it sent the node, so an answer that comes
back is counted (a grant it gave up on gives the node nothing). A tenant that
another operation was holding — a quota being set, a move — when its node
was failed over is failed over once that operation ends, onto the copy it
had then and no other, and its node's grants name it no more meanwhile, so
the node does not take it back as it answers again. A
copy that is not its tenant as its primary has it is never promoted: the
router tells the node the tenant's lineage and the history its primary
forked at its last failover, and the node refuses a copy of another
lineage — one that never took its primary's image, or a file put in its
place — and an old primary's copy that has not heard from the new primary
since, which would take back every write after that failover; the tenant
waits for its node, where its rows are. A copy that has heard from the node
it follows since it was last told to follow it is promoted whatever
history it holds, and its node remembers the hello across a restart, so a
tenant restored in place on its primary's node — `fenec restore`
installed with `DELETE` and `PUT .../file`, or the
whole node with `fenec restore --tenants` — fails over as before
once its copies have taken the restored file. A tenant not yet written
fails over as any other. A file of another lineage put in a primary's place
by hand — another tenant's, or one made again under the name — is not the
tenant the router knows: every failover of it is refused until a read
replica's look teaches the router the new lineage, or an operator forces
one. An operator who knows better sends
`{"force": true}` to `POST /_shard/nodes/<n>/failover`
or to the node's `POST /_admin/tenants/<t>/promote`, which
promotes the copy whatever it holds and is written to the audit log. Nor is
a tenant given a new replica while its primary does not answer; the repair
looks again a few times over the next minutes, and `POST
/_shard/replicas` asks at once. Before, one such node held
up the leasing of its whole region: in a drill on GitHub's runner, whose
Docker resolver is slow to refuse a removed container's name, the region's
healthy nodes went 18 s without a write and one was failed over, and the
dead node's tenants wrote again only after 30 s. In the same drill run
locally, every unknown name sent to a resolver that never answers, the
router now fails the dead node over 2.1 to 3.3 s after its kill at a lease
of 2 s, and the region's other tenants never pause more than 19 ms.

Measured with `make shard-bench`, three nodes and the router in
one process and a lease of a second: a node cut off stopped taking writes
386 to 406 ms after — its lease ran from the grant before its last renewal,
about two thirds of a lease before the cut — and its 10 tenants took writes
on their replicas again 1.27 to 1.36 s after, the lease and a tenth from the
last answer, the next round and the promotions. Before grants were numbered,
the node ran its lease from its last renewal and stopped 714 to 777 ms after
the cut, and the failover came as it does now, 1.26 to 1.42 s after (four
runs each, in turns). With a lease of five seconds, a node lost costs its
tenants' writes 5.5 to 7.2 s.

## A standby router

A second router can follow the first one's directory and forward requests
too. Only the primary router makes changes; promote the standby when the
primary is gone.

```bash
fenec-shard --listen 127.0.0.1:8090 --directory shard.fenec --replication-token r1
fenec-shard --listen 127.0.0.1:8091 --directory standby.fenec --replication-token r1 \
            --replica-of http://127.0.0.1:8090
```

- **Promote:** `POST /_replication/promote` on the
    standby.

Details

The directory is a fenecdb file, so it replicates the way any other does.
A second router follows it and forwards from the same placements; the
primary is the one that changes it.

Both forward `/t/<tenant>/...`; the standby's
`/_shard/` answers reads and refuses changes with 409, since its
directory takes no write of its own. When the primary is gone,
`POST /_replication/promote` on the standby forks its directory's
history and it starts taking them — the same promotion a replica of a
database gets, and the same rule after it: the old primary rejoins with
`--replica-of` pointed at the new one, or its file is refused.

A standby is behind by whatever the last directory change has not reached
it yet, which is one write and a sync: placements change when someone places,
moves or deletes a tenant, not when clients read and write. Nothing of the
tenants' own data goes through the router's directory, so a standby that is a
second behind still forwards every request correctly. One whose directory has
not arrived at all — just started, its primary not yet reached — answers
`503` with `Retry-After` rather than `404`:
it knows of no tenant, which is not knowing there is none.

Each change the primary makes reaches the standby's maps through the
change ring, as a subscription's rows do: the placements it put or deleted
since the last one, 4 µs at 100 000 tenants, where reading every map again
took 38 ms under the router's write lock. An image adopted, or more changes
than the ring holds, and the maps are read again whole.

## Regions

Give each node a region and a zone — a datacenter in the region. A tenant
lives in one region, its node's, and its replica is kept in the same region in
another zone, so a failover keeps both its latency and where its data is.

```bash
curl -X PUT localhost:8090/_shard/nodes/eu1 \
     -d '{"addr":"10.1.0.11:8081","token":"a1","region":"eu","zone":"fra"}'
curl -X PUT localhost:8090/_shard/nodes/eu2 \
     -d '{"addr":"10.1.1.11:8081","token":"a2","region":"eu","zone":"ams"}'

curl -X PUT localhost:8090/_shard/tenants/acme -d '{"region":"eu"}'
{"tenant":"acme","node":"eu1","region":"eu","replica_on":"eu2"}
```

- **A tenant is placed in the region it names**, on the node with the
    fewest bytes there. A region with no node is refused with
    `409`, and nothing is created.

- **It leaves its region only by a move that names the region** it
    goes to: `{"to":"us1","region":"us"}`.

- **Run one router a region**: standby routers follow the primary's
    directory and forward every tenant to its node, wherever it lives.

Details

**Nodes.** `PUT /_shard/nodes/<n>` takes
`region` and `zone` beside `addr` and
`token`, names of `[a-z0-9_-]` as a tenant's are. One
left out is the one recorded, so a node given a new token stays where it is.
A node's region is fixed while it holds a tenant, a tenant's replica, or a
place in a pair — its tenants' residency would change under them — so it
changes once they are moved off; its zone may change at any time. A node
recorded without a region is in the unnamed region, which is one region like
any other: a directory that names none places, replicates and fails over as
it did before regions, and `""` names it in a request.

**Tenants.** A tenant's region is its node's — kept on the node alone,
so a move or a failover changes it with the node and there is no second
record to drift. `PUT /_shard/tenants/<t>` takes
`region`, `node` or both, which must agree
(`409` otherwise). With neither it goes to the region
`fenec-shard --default-region` names, and without that flag to the
node with the fewest bytes whatever its region, as before. `GET
/_shard/tenants/<t>` answers its node, state, region and zone, its
copy's node — a replica of its own or its node's standby — and
`"same_zone": true` where that copy is in its zone; the list
carries each tenant's region, and `?region=eu` gives one region's
tenants.

```bash
curl localhost:8090/_shard/tenants/acme
{"name":"acme","node":"eu1","state":"active","region":"eu","zone":"fra","replica":"eu2"}
```

**Copies.** With `--replicas` a tenant's replica goes to a
node of its region only — the replica is what a failover promotes, and one in
another region would carry the tenant out of its own. Among those, a node of
another zone comes first, before one holding a copy already and before the
fewest replicas: a datacenter lost would take a replica in its primary's zone
with it. A zone not recorded is ranked between, neither known to be another
nor the same. Where every free node of the region is in the primary's zone,
the replica goes there all the same: the router logs it, the tenant answers
`"same_zone": true`, and
`fenec_router_same_zone_copies{region}` counts it for as long as it
stays. A region with no other node gives none (`"unreplicated": "there is
no other node in region `eu` to keep it"`) until a repair finds one. A
pair's standby must be in its node's region (`409` otherwise), and
one in its zone is logged. So a failover — by hand or on a lease — promotes
a tenant within its region: its copy was never placed anywhere else.

**Moves.** `{"to": "eu2"}` moves a tenant to a node of its
region; to a node of another, it is refused naming the region the request
must name. `{"region": "us"}` picks the node with the fewest bytes
there, `{"to": "us1", "region": "us"}` names both; either way the
answer carries the region the tenant is in now. Its replica goes with it
where it still fits — in the new region, in no worse a zone than a node free
there — and is otherwise taken off and placed again in the new region: kept,
a failover would take the tenant back out of the region it moved to. A move
across regions copies the tenant's image over the network between them, the
tenant frozen for writes meanwhile.

**One router a region.** The directory is one file with one primary
router, which takes every change. A standby router in each other region
follows it ([A standby router](#standby-router)) and forwards every
tenant, from the same placements, to its node wherever it is; its
`/_shard/` answers reads, `GET /_shard/tenants/<t>`
and `POST /_shard/query` included, and refuses changes with
`409` naming the primary — asked again it would refuse again, so
it is no `503`. Clients reach the router nearest them, by GeoDNS or
an anycast address in front; a request for a tenant in another region crosses
between regions once, router to node, as it would going there itself. A
standby router added 23.7 µs p50 to a request over the node's own, as the
primary did — 23.0 to 23.8 and 23.1 to 23.9 over five runs, the two asked in
turns: following costs a request nothing (`make shard-bench
ARGS=standby`, node and routers in one process on loopback).

```bash
# eu: the primary router, its directory replicated to the others
fenec-shard --listen 0.0.0.0:8090 --token $ROOT --directory shard.fenec \
            --replication-token $DIR --replicas --default-region eu

# us and tr: a standby router each, following it
fenec-shard --listen 0.0.0.0:8090 --token $ROOT --directory standby.fenec \
            --replication-token $DIR --replica-of http://router.eu.internal:8090

# every region's nodes, two zones each, recorded through the primary
for n in "eu1 10.1.0.11 eu fra" "eu2 10.1.1.11 eu ams" \
         "us1 10.2.0.11 us iad" "us2 10.2.1.11 us pdx" \
         "tr1 10.3.0.11 tr ist" "tr2 10.3.1.11 tr ank"; do
  set -- $n
  curl -X PUT router.eu.internal:8090/_shard/nodes/$1 -H "Authorization: Bearer $ROOT" \
       -d "{\"addr\":\"$2:8081\",\"token\":\"$ADMIN\",\"region\":\"$3\",\"zone\":\"$4\"}"
done

curl -X PUT router.eu.internal:8090/_shard/tenants/acme -H "Authorization: Bearer $ROOT" \
     -d '{"region":"tr"}'
{"tenant":"acme","node":"tr1","region":"tr","replica_on":"tr2"}
```

A standby router is behind the primary by a directory change on its way,
one network trip between regions; a move deletes the source copy only once
every standby keeping up has been sent the new placement, so a standby in
another region routes the tenant's next request to the target and not to a
node that no longer holds it. A promotion is the same as with one standby:
`POST /_replication/promote` on the router to take over, the others
pointed at it with `--replica-of`.

**A region's leases.** Under `--auto-failover` the router
that keeps a node's directory leases it. With one directory that is the
primary router, so a region cut off from the primary's would stop taking
writes as its leases lapse, and could not be failed over until the router
reached its nodes again. A region that keeps its own directory is leased and failed over
by its own router, and goes on writing through a partition. A client far
from a tenant's home reads near itself where the tenant has a read replica in its region; no SDK picks a router
by region: a client is given the router nearest it.

**The directory's format.** `nodes` took
`region` and `zone` fields, which a directory written
before them is given as it opens (an `alter`); its nodes read the
unnamed region. A router from before them reads the directory as it always
did — it ignores the fields, forwards every tenant where it is, and, promoted,
places without regard to regions.

## A region that keeps its own directory

Give a region a router of its own that keeps the region's directory: its
nodes and tenants. That router leases the region's nodes and fails them
over, so the region goes on taking writes — and fails a node over inside
itself — while it is cut off from every other region.

```bash
# us: the region's router, its directory in us.fenec, the global one followed from eu
fenec-shard --listen 0.0.0.0:8090 --token $ROOT --directory us.fenec \
            --replication-token $DIR --replicas --auto-failover 5 \
            --region us --global http://router.eu.internal:8090

# on the global router: us keeps its own directory, kept by that router
curl -X PUT router.eu.internal:8090/_shard/regions/us -H "Authorization: Bearer $ROOT" \
     -d '{"routers":["http://router.us.internal:8090"]}'

# us's nodes and tenants, through either router
curl -X PUT router.eu.internal:8090/_shard/nodes/us1 -H "Authorization: Bearer $ROOT" \
     -d '{"addr":"10.2.0.11:8081","token":"a3","region":"us","zone":"iad"}'
curl -X PUT router.eu.internal:8090/_shard/tenants/acme -H "Authorization: Bearer $ROOT" \
     -d '{"region":"us"}'
{"tenant":"acme","node":"us1","region":"us","replica_on":"us2"}
```

- **Inside the region, nothing waits for another:** leases, failovers,
    repairs, moves between its nodes, quotas and its nodes are its router's.

- **What needs every region waits for them:** a create (the name is
    global), a delete and a move across regions go through the global router
    and are refused with `503` and `Retry-After` while a
    region they touch cannot be reached — nothing is done by half.

- **Every router reaches every tenant:** one asked for a tenant of
    another region relays the request to that region's router.

Details

**Two levels.** The global directory — the router started without
`--region`, and its standbys — holds every tenant's name, the
regions keeping their own directory and their routers (`PUT
/_shard/regions/<r> {"routers": [...]}`, the one keeping it
first, its standbys after). For each tenant of such a region it holds a home,
the region and no node. The region's router holds the region's nodes and
placements in a directory of its own (`--region us`), the same
collections, and follows the global one from `--global`. A region
that keeps no directory of its own — the unnamed one, and every region of a
deployment that names none — is the global directory's, so a deployment
without `--region` places, leases and fails over exactly as
before, and one may move its regions out one at a time.

**Who changes what.** A change touching one region's nodes or
tenants is that region's router's: a node added, a tenant's quota, a move
between two of its nodes, a failover by hand, a repair. A change the global
directory takes is the global router's: a create takes the tenant's name
there first and then asks the region's router to make it (the name given back
if that router refuses or cannot be reached; kept where it says the tenant
exists, which takes in a create whose answer was lost), a delete asks the
region's router first and then gives the name back. Either router takes
either kind: what is not its own it relays, once — a region's router to the
global one, the global router to the region's, marked so it goes no
further.

**A move across regions** is the global router's, step by step: the
tenant frozen in the region it leaves and its image taken, installed and
recorded in the region it goes to — leased and given its replica there —
let go of in the region it left, whose directory then places it nowhere, so
no lease there names it; and only then its home moved in the global
directory, one statement, which routes it to its new region. Each step is
undone where the next fails: a region not reached leaves the tenant where it
was, taking writes again, and the move is answered `503`. A move
between a region keeping its own directory and one the global directory
keeps works the same way, the global router doing that region's part
itself.

**Forwarding.** A router asked for a tenant its own directory does not
place looks it up in the global directory — its own, or its copy — and
sends the request to its node, or, for a tenant of a region keeping its own,
to that region's routers, the one that answered last first. They forward it
to the node as the region's last failover left it: nobody outside the
region needs to have heard of the failover, and after a partition every
router reaches the new placement with its first request. The relay costs a
request a hop inside the tenant's region, which a client asking the router
of its own region never pays: 32 to 42 µs p50 on loopback, a router's 24 and
the proxy that stood for the link between the regions (`make
shard-bench ARGS=regions`). A router relaying names the client beside it,
marked with an HMAC of the routers' shared `--token`, so a refused
token waits by the client's address, at the first router alone.

**Fencing.** A lease names its holder: the directory it comes from and
that directory's term, the number of times its history forked. Promoting a
router forks it, so a region's router promoted over the one before — a
standby following its directory, in the region or in another — grants at a
higher term, and every node that hears from it refuses the old one from then
on: its grants (`409` naming who holds the node), its promotions
and its re-pointing of replicas. A node that only the old router still
reaches cannot be told; that is why a router is promoted once the one before
it is gone, as with one directory. A node from before terms takes a grant
without checking its term, so a region's router leases it but never fails it
over on its own. List a region's standby routers after its router in
`routers`: a relay moves to the next one where the first cannot be
reached, and a standby forwards from its copy of the directory and refuses
changes until it is promoted.

**A partition, step by step.** The global router in eu, us keeping its
own directory; the link between the regions goes down.

- Inside us, nothing changes: us's router renews its nodes' leases, and
    its clients write as before. A request from eu for a us tenant is
    answered `502` — a region out of reach, as a node out of reach
    is — and a client asks again; inside eu, eu's tenants are written as
    before.

- A node of us lost or cut off stops taking writes as its lease lapses,
    and us's router promotes its tenants on their replicas in us, a tenth of
    a lease later. With a lease of a second, a node cut off stopped writing
    0.38 to 0.63 s after the cut and its 10 tenants wrote again on their
    replicas 1.41 to 1.70 s after, ten runs; with five seconds, 3.9 to 4.1 s
    and 5.5 to 6.7 s — as the global router fails over its own, 1.27 to 1.36
    s at a second (`make shard-bench ARGS=regions`, the region cut
    off from the global router throughout, three nodes and both routers in
    one process).

- A create in us, a delete of a us tenant, a move into or out of us are
    refused with `503` and `Retry-After`, through any
    router, and change nothing.

- The link comes back: the global router's next relay reaches us's
    router, which places the failed-over tenants where it put them; the
    refused changes go through when asked again.

**Formats.** The global directory took a `regions`
collection, and a tenant at home in such a region is a row of
`tenants` whose node is `@<region>`, which no node
can be named; a region's directory names its region in `scope`,
and a router opened over the wrong one refuses it. A router from before
regional directories reading the global one forwards no request of such a
tenant (no node `@us`: `404`), refuses every change of
it, takes no name it holds, and leases none of the region's nodes, which it
does not know. A lease grant carries `authority`,
`term` and `history`, and its own number and the one
it acknowledges (`grant`, `acked`); a node from before
reads the grant as it did. Not done: a query across tenants (`POST
/_shard/query`) asks the tenants the router's own directory places, and
names each tenant of another region's directory as failed rather than leave
it out; ask that region's router for its own.

## One URL for every region

A Cloudflare Worker in front of the regions' routers gives every tenant
one URL: `https://db.example.com/t/<tenant>/...` goes to the
routers of the region the tenant lives in, and their answer comes back as it
comes. It is a plain Worker, not a Workers for Platforms dispatch, which
would put a dispatch in every request's path.

```bash
# entry/wrangler.jsonc: REGIONS, each region's routers, the one keeping its directory first
cd integrations/cloudflare
npx wrangler secret put ENTRY_TOKEN --config entry/wrangler.jsonc
# a Cloudflare Access service token, where Access guards the tunnels' hostnames
npx wrangler secret put ACCESS_CLIENT_ID --config entry/wrangler.jsonc
npx wrangler secret put ACCESS_CLIENT_SECRET --config entry/wrangler.jsonc
npx wrangler deploy --config entry/wrangler.jsonc

# every router: the same token, so it believes the client the entry names
FENEC_SHARD_ENTRY_TOKEN="$ENTRY_TOKEN" fenec-shard --token "$ROOT" ...

curl https://db.example.com/t/acme/query -H "Authorization: Bearer $TOKEN" \
     -d '{"query":"get notes count"}'
```

- **It learns where a tenant lives from the routers:** every request
    asks for the tenant's home, and the router that forwards it to the
    tenant's node names the node's region in its answer, to the entry alone.
    A tenant the entry has not seen goes to the region nearest the client,
    whose router serves it or relays it home once; its next request goes
    home.

- **A tenant moved to another region** is relayed by the router of
    the region it left, as any client's request is, and the answer names
    its new home: a stale home costs one hop, never an error.

- **A read marked for a replica** (`Fenec-Read: replica`,
    [read replicas](#read-replicas)) goes to the region of the
    client's continent first, whatever the tenant's home: its router answers
    it from the tenant's read replica there, or relays it home — a hop, where
    the region keeps none — and the home is asked where that region fails. A
    continent no region names sends it home alone: the default region is
    where an unknown tenant is guessed, no nearer such a client than the
    home. A marked read runs read-only wherever it lands, so it may go to
    two; an unmarked request never leaves a known home. Sent to the home the
    entry had learned, every marked read crossed to it, and the replica near
    the client was never asked.

- **Streams pass through as they come** — a subscription,
    `/_changes?wait`, a held write, a long `/batch`.
    A stream is let go of when its client goes; a held write is let go of at
    the router, which does not yet let go of it at the node.

- **Only `/t/<tenant>/...` is served:**
    `/_shard/` goes to the global router only where the entry is
    told to (`ADMIN`), and `/_admin/`,
    `/_replication` and the rest are a `404`, however
    the path is spelled.

- **A router that cannot be reached** is passed over for the region's
    standby router. An unmarked request of a tenant whose home the entry
    knows is never sent to another region: its region's routers all failing
    is a `503` with `Retry-After`. One it has not heard
    goes on to the next region, whose router relays it home, and a marked
    read goes home past its continent's region.

- **Set `CORS` wherever the routers have
    `--http-cors`:** without it the entry's own answers — a
    region down, a path refused — carry no CORS headers, and a page reads them
    as the network failing. The entry says so in its log once it sees the
    routers send them.

Details

**Where a tenant lives.** Nothing remote is asked on the way: a
request carries `Fenec-Home: 1`, and the router that forwards it
to the tenant's node answers with `Fenec-Home: <region>`, the
node's — a router relaying it to another region names nothing of its own,
and the answer it passes back carries the home router's. The entry takes the
header off before the answer goes to the client: a router names the home on
a refusal too, so to anyone holding a tenant's name. The entry keeps 50 000
tenants' homes an isolate (`HOMES`), forgotten first-heard first,
and shares each one it learns with the colo's other isolates through the
Cache API, read only by an isolate that has not heard the tenant
(`HOMES_CACHE`; it does nothing on `workers.dev`, so
the entry wants a custom domain). A tenant heard nowhere goes to the region
`REGIONS` names for the client's continent (Cloudflare's
`request.cf.continent`), else `DEFAULT_REGION`, then to
each other region in turn where one's routers cannot be reached — any
region's router relays the tenant home, and only the home's router places it
on a node, so no write lands anywhere else. A region whose routers failed is
asked last: by a read or a keyed write for ten seconds, then first again; by
any other write until one of its routers answers, since a write that may have
reached a router cannot be sent to another. A guessed region's router that
answers 5xx naming no home — cut off from the home's region, say — sends a
read on to the next region too; a write it answered is its answer. Asking the global router at each miss would have needed its
`--token`, which moves and deletes every tenant, in the Worker, a
round trip to the global router's region, and would have turned every unknown
name a client tries into a question for that one router; a table in Workers
KV is read eventually consistently and billed by the read, and still needs
something to write it. A router answers the header only to a request asking
for it, so no other client is told one; a router
from before it answers none, and the entry keeps sending such a tenant to the
nearest region, relayed each time.

**Streams and Cloudflare's limits.** The answer is handed back as its
body arrives, so a server-sent event reaches the client as the router sends
it. Cloudflare's [Workers
limits](https://developers.cloudflare.com/workers/platform/limits/): waiting on a `fetch` is not CPU time, and an
HTTP-triggered Worker has no limit on its duration while the client stays
connected, so a subscription or a held write is not cut; an invocation may
have six connections waiting for their headers, and the entry has one. A
router behind a hostname Cloudflare proxies must start its answer within
the proxy read timeout (100 seconds, fixed below Enterprise): a
`Fenec-Wait` holds a write 30 s at most, and a silent stream
writes a keep-alive every 20 s (`--http-keepalive`). The
Worker's `request.signal` (the `enable_request_signal`
flag) is handed to the fetch, so a client gone cancels the request to the
router. Under `wrangler dev` a stream is let go of at a
stand-in router 2 s after its client goes, workerd noticing as the next
keep-alive is written, and at a node through a router in 4 to 8 s, each
hop noticing at its next. A client gone before its answer's head is not
told to the Worker there at all — workerd notices a client only as it
writes to it, past wrangler's own proxy too — so that case is held in Node,
the handler over Node's `fetch`. Past the entry it goes on to the
node: a router waiting on a node's held answer watches its client and
passes the client's end on, as the node would read it from a client of its
own — a region's router relaying the request to the home router, which
passes it to the node — so a claim whose client went hands its job to the
next claim held, as it does asked of the node itself, rather than lease it
to nobody; a client that only half-closed is answered as the node answers
it.

**Headers.** Everything a client sends goes on —
`Authorization`, `Idempotency-Key`,
`Fenec-*`, `traceparent`, `X-Request-Id`
(Cloudflare's ray id where the client sent none, so the routers' and the
nodes' lines carry it) — but the hop-by-hop headers, the routers' own marks
(`Fenec-Router`, `Fenec-Global`,
`Fenec-Verified`), which a router believes from another router
alone, a `Fenec-Client` of the client's own, and every way a
client names an address
(`X-Forwarded-For`, `X-Real-IP`,
`Forwarded`, `True-Client-IP`,
`CF-*`). `Fenec-Verified` and
`Fenec-Home` are dropped from answers. The routers' CORS headers
come back as they sent them; with `CORS` set to their
`--http-cors`, the entry answers a preflight itself with the same
headers, a round trip saved and a region that is down no reason for a page to
see a network error, and puts them on its own answers alone — two
`Access-Control-Allow-Origin`s are refused whole by a browser. A
path is judged as the router and the node will read it, decoded as
fenec-http decodes one: `/t/acme/%5Freplication` and
`/t/acme%2F_replication` are a tenant's replication stream to
them, which a URL's own parsing leaves encoded.

**The client's address.** Behind the entry every request reaches a
router from Cloudflare's addresses, so the wait after a refused token would
count every client's refusals together, and any good token would start the
count again. The entry names its client in `Fenec-Client:
<address> <seconds> <mark>`, the address Cloudflare's
`CF-Connecting-IP`, the seconds the Unix time the entry made it
at, the mark 16 bytes of HMAC-SHA256 of `fenec-client <address>
<seconds>` under `ENTRY_TOKEN`, base64url. Bound to
the address and the time, a mark seen names that one address for a minute,
where a mark of the token alone would let whoever saw one name any address
for good. Not `CF-Connecting-IP` or `X-Forwarded-For`
believed from Cloudflare's addresses: a Worker sets
`CF-Connecting-IP` and `X-Real-IP` as it likes on a
request to an origin outside Cloudflare (Cloudflare's HTTP
headers page), so any account's Worker could claim any address from
those ranges.

A router started with the entry's token — `fenec-shard
--entry-token`, or `FENEC_SHARD_ENTRY_TOKEN` — checks the mark against its own HMAC, in constant time, and
the time within 60 seconds of its clock either way, and takes the address
for that request's in the connection's place. A refused token waits by it,
and a good one starts its count again and no other's; the audit log names
it as `peer`, the entry's address beside it as `via`,
and a trace as `client.address`, the connection's as
`network.peer.address`; and it is the client the router names to
the tenant's node, or to the home region's router it relays the request to,
which names it to the node in turn. A header seen on the wire can be
replayed for that minute, and names that one address: what the minute
costs, where a mark of the token alone would name any address for good. A
header that does not check, or a router without the flag, counts the
connection's address, and `fenec_router_entry_clients_total`
says which (`named`, `bad_mark`, `stale`,
or `absent` for a tenant's request carrying none, as every one
from an entry whose `ENTRY_TOKEN` is unset does), the router's
log a minute apart at most of each kind not believed — not the audit log,
which a header costing its sender nothing would fill at the router's
request rate. On a router only the entry reaches, alert on all but
`named` ([Monitoring](https://fenecdb.com/docs/monitoring#router)).
Every router drops the header from what it forwards, flag or not, so a
node never reads a client's claim. A mark costs a request one HMAC-SHA256,
0.75 µs on this machine, which a round trip does not show: through a router
with the flag and one without, in turns, a request the entry named its
client in took 41.7 µs p50 against 42.1, and one with no header 42.0
against 42.05 — a look among its headers and a count — the runs'
differences a few tenths of a microsecond either way (`make
shard-bench ARGS=latency`, eight runs). Without the flag the router's
added p50 stayed 20.4 µs, as before.

**The entry's token is its own.** The Worker holds it, so a router
refuses it where it holds it for anything else: equal to its
`--token`, which moves and deletes tenants, to its
`--replication-token`, whose stream carries every node's admin
token, or to the admin token of a node in its directory, at the start and
as a node is added. The nodes' data and replication tokens the router never
sees: keep the entry's apart from those too. A router takes one entry token
at a time. To change it, start each router with the new one, then put it
in the Worker (`npx wrangler secret put ENTRY_TOKEN`): in
between every request is served, and its client counted by Cloudflare's
address, as `bad_mark` shows — a few minutes of the counting
there was before, against a second token held by every router.

**When a router cannot be reached.** A region's routers are tried in
turn from the one that answered last. Every answer a router writes carries
its mark, `Fenec-Shard: 1`, and an answer at a router's URL
without it is not the router's — not `X-Request-Id`, which a
router's answers carry too and a load balancer may write on its own 502.
Cloudflare's own answers for an origin it never reached (521, 523,
525, 526, and 530, a tunnel with no connector) go on to the next router
whatever the request. One that may have reached the router — a
`fetch` that fails, which workerd says alike of a connection
refused and of one reset after the request was written; Cloudflare's 520,
[522](https://developers.cloudflare.com/support/troubleshooting/http-status-codes/cloudflare-5xx-errors/error-522/)
(a request whose bytes went unacknowledged for 90 seconds is among its
time-outs) and 524; and any other 5xx without the router's request id, such
as [a
tunnel's connector answering 502](https://developers.cloudflare.com/cloudflare-one/networks/connectors/cloudflare-tunnel/troubleshoot-tunnels/common-errors/) for a router it cannot reach — goes on only
for a `GET`, `HEAD`, `OPTIONS` or a request
carrying an `Idempotency-Key`, as the router's own rule towards a
node; any other is a `502` saying it may have run, and the
region's next request goes to the next router. A router's own
`502` — a node it could not reach — names the home and is its
answer, passed on, no other region tried. A
body up to a mebibyte is kept for the next router; a longer one is streamed
to one, its length kept (a router takes no chunked body), and one sent with
no length past a mebibyte is refused with `411`: read whole, it
was as large as a client cared to send, in isolates whose 128 MB are
shared.

**What it costs.** Under `wrangler dev` on this machine, a
read by id of a tenant in us took 81 µs p50 asked of us's router directly,
and through the entry with its home heard 1.18 to 1.22 ms, 0.66 to 0.68 past
wrangler's own proxy Worker — workerd taking the request, the entry and its
fetch, all on loopback; with the client named besides
(`ENTRY_TOKEN`, an HMAC a request), 1.23 to 1.24 ms (0.68 to
0.70). Never heard, it went to eu first, whose router relayed it: 1.29 to
1.31 ms (0.71 to 0.75), eu's router asked directly relaying it in 114 to
115 µs. Heard only through the colo's cache, 1.60 to 1.61 ms (1.00),
miniflare's cache a service of its own (`make entry-bench`: two
runs of six rounds of 2 000 in turns, at loads of 3.4 to 5). That is not
what Cloudflare adds: there a client reaches its nearest colo and the router
is a region away, the leg it would pay to reach the router itself; a home
not heard costs a relay between regions, where the cache is read inside the
colo. A router asked for the home answered in 82 to 83 µs p50 against 81 not
asked; not asked, its mark on every answer, its added p50 was 23.75 µs
against main's 23.9 (23.4 to 24.1 and 23.7 to 24.0, six runs a side in ABBA
turns, `make shard-bench ARGS=latency`).

## Read replicas

A tenant lives in one region, and a client on another continent pays the
trip between them on every request — Frankfurt to Singapore is 158 ms. Give
the tenant a read replica in the client's region: a follower of its primary
there, which the router of that region answers the client's reads from,
while its writes go home.

```bash
# a read replica of acme in ap, on the node of ap holding the least
curl -X PUT router.eu.internal:8090/_shard/tenants/acme/read-replicas/ap \
     -H "Authorization: Bearer $ROOT"
{"region":"ap","node":"ap1","follows":"http://10.1.0.11:8081"}

# ap's router stands in ap: a marked read of a tenant with a replica there goes to it
fenec-shard --listen 0.0.0.0:8090 --token $ROOT --directory standby.fenec \
            --replication-token $DIR --replica-of http://router.eu.internal:8090 --near ap
```

```ts
import { connect } from '@fenecdb/web/client';

// Through ap's router: reads near, writes home.
const db = connect('https://ap.example.com/t/acme', { token: 'a token', reads: 'nearest' });
await db.from('orders').insert({ number: 'SG-1' });        // home, in eu
const mine = await db.from('orders').rows();               // ap's replica, after that write
```

- **A read never comes from before the client's own write.** A client
    reading near sends each read after the furthest position home named to
    it — the change and the history it is a change of
    (`Fenec-After`, `Fenec-History`) — and the replica
    answers once it holds that change of that history; one it cannot in the
    wait goes home. A failover that loses a write does not fool it: the copy
    left on the old branch holds a change of that number, of another
    history.

- **A replica answers nothing before it has caught up.** One placed a
    moment ago, its image still to come — or reopened, or pointed at a new
    primary, until that primary has said hello — sends the read home.

- **Only a read the client marks goes near**
    (`Fenec-Read: replica`): the router parses no statement. The
    SDKs mark the builder's reads, and a text the caller says reads; every
    other request goes home, as it always did.

- **A read replica is never promoted.** A failover of the tenant's home
    promotes its standby or its replica, in its region; the read replica
    follows the new primary.

- **Cut off from home, it serves what it last applied** — stale, to a
    read that names no write — and catches up once the link is back; reopened
    or pointed at a new primary meanwhile, it serves nothing until that
    primary has said hello.

**Behind the global entry** ([one URL for every
region](#global-entry)) a marked read goes to the region of the client's continent
first, whatever the tenant's home, and its router answers it from the read
replica there; every other request goes to the home the entry has
learned.

Details

**Placing one.** `PUT
/_shard/tenants/<t>/read-replicas/<region>` makes the tenant
on the node of that region holding the fewest tenants and copies — never a
standby — has it follow the tenant's primary node, as a replica of its own
does, and records it once it follows; `201` with the node,
`200` where the region has one already, `502` where the
node would not follow, the file it made there deleted again. One a region,
and none in the tenant's own region (`409`: its standby and its
replica are there), none in a region with no node (`409`). It is
held to the tenant's quota, as the tenant's standby and replica are: sent it
as it is placed, and each quota set after — handed at once to a sender of
its node's by the directory setting it, the quota request answered once the
directory records it, and within a look by a router keeping a read replica in
another region's directory, which reads the quota with the tenant's
placement. One sender a node, on a thread of its own: a node that does not
answer holds up its own replicas' quotas alone, never a look or a quota
request, and each replica is sent its quotas in the order they were set, an
older one never after a newer. `DELETE` takes it out of the directory and, once
the standby routers have been sent that, off its node; a node that does not
take it off keeps it recorded (`502`) — a copy following its
primary unrecorded would never be stopped — and
`DELETE ...?forget` lets go of the record of a node gone for good.
A node holding a read replica is not removed (`409`) until it is
taken off. `GET /_shard/tenants/<t>/read-replicas` lists
them, `{tenant, read_replicas: [{region, node, follows}], failed,
changing}`, `changing` whether a change of one is under way.
Every node shares `--replication-token`: the replica's node asks
the primary's feed with its own. The record is kept by the directory that holds the replica's
node — the global one, or the region's where it keeps its own — so that region's router places
it, leases its node, routes to it, and goes on doing so cut off from the
tenant's home. Asked of another router, the request is relayed there; the
list is gathered from every directory by the global router, a region not
reached named in `failed`.

**Routing.** A router stands in a region: `--near ap`, or the
region whose directory it keeps (`--region`). Asked for a request
marked `Fenec-Read: replica` of a tenant with a read replica in
that region — and not living in it — it forwards the request to the replica;
every other request goes where it went. A marked read the replica does not
answer — its node not reached, or the request not written or answered by
the wait its `Fenec-After` asks, a `/_changes` page's
own `wait` for a write, and `--replica-timeout` past
them (2 s); not caught up with its primary yet, or going down (`503`);
still behind the position it was sent after at the end of the wait
(`504`); its file gone from the node (a `404` with
`Fenec-Tenant: absent`) — is sent home, so the client is answered
either way and from nothing older than its write; and so is one whose answer
breaks off before its end, the router reading a replica's answer whole
before any of it goes to the client (a stream's goes on as it comes). A
replica that did not answer is let be for 5 s, its reads going home at once
meanwhile: the node's, every tenant's, where it was not reached or took no
whole request. A tenant's copy that answered `504` could not
reach the position in the wait it was given, and asked again at every read,
a replica cut off from its primary held each one that long — every run of a
live query reading near came the wait late for as long as the partition
lasted: for 5 s the tenant's reads sent after a position, with no longer a
wait, go home at once through that router, the trip home a region away in
place of a wait. A read naming no position, which the copy answers as it
stands, and one willing to wait longer still go to the copy, home out of
reach or not. So a client holding a token of the tenant can send its
positioned reads through that router home: a `Fenec-After` past
the tenant's last change, with a wait of a second or more, once every 5 s —
as a failover that lost writes does on its own, its clients' positions on the
lost branch. What that costs is the trip home, never a wrong answer. Where it took one and did not answer in time, the tenant's
copy is let be at once and the read goes home, while the router asks the
node's `/_health` — answered before any token, lock or routing —
on a connection of its own, within 500 ms (`--replica-timeout`
where shorter), one question a node at a time. Answered, the copy alone is
let be, since its answer comes once the query has run and a slow read is
that read's, not the node's: no tenant's slow reads, nor several tenants',
let the node be. Not answered, the node is too — it takes connections and
answers nothing, stopped or deadlocked — and no other tenant waits it
out. A node that answers its health while every query on it stalls — its
disk stalled under the queries — costs each tenant with a copy there one
timeout every 5 s for as long as that lasts.
`fenec_router_replica_reads_total` counts marked reads by where
they were answered: `outcome="replica"`,
`"fallback"`, `"home"`. Every server runs a marked
request read-only on every route — a statement, REST, `/batch`, an
MCP tool, a change-stream consumer's commit — home as much as a replica, so a
write marked by mistake is refused with `403` the day it is
written, never landed; a value the header does not take is `400`.
A tenant frozen for a move, or on a node whose lease has lapsed, answers a
marked read, which can only read.

**Read your writes.** A write's answer carries `Fenec-Seq`,
the change it left the tenant at, and `Fenec-History`,
`<term>-<id>`: the history that change is of, which a
failover forks where the copy promoted stood, one term more. A replica numbers
the primary's changes as the primary did, and a read sent with
`Fenec-After: n` and that `Fenec-History` is answered
once the replica holds change `n` of that history, 5 s at most
unless `Fenec-Wait` says. The history is what makes it hold across a
failover: a write the old primary acknowledged and the promoted copy never
got is lost, the new primary's next write takes its number, and a replica
still on the old branch holds a change of that number — the lost one. It
answers no read after the new write until it is pointed at the new primary
and takes its image. A client reading near keeps the furthest position an
answer from home named — a write's, a read-only `/batch`'s; of a
later term, or further in one, so an answer arriving late is not the last —
and sends it with every marked read, whose own answers move nothing: it reads
its own writes and every write before them, and another client's writes after
its own may or may not be there yet. A marked `/query` read's
answer names where the copy that answered it stood as it read
(`Fenec-Seq`, `Fenec-History`), replica or home; the
SDKs take nothing of it as the client's, since a replica may be behind home
or on a branch a failover left, and a JavaScript live query sends its next run
after it (below). A subscription and a
`/_changes` page sent so wait for it as a read does, before their
head, and go home on the replica's `504`: a stream is seeded with
the client's write, never from the copy as it stood behind it, and a page's
`Fenec-Seq` is never behind it. Go's `Read()` and
.NET's `AsRead()` copies send the position on a subscription and
`/_changes` too, and take a stream's `504` as a
read's. A marked stream open on a copy that refuses its primary — the
tenant deleted and made again under its name — ends as the copy deletes
itself, as the reads after it are sent home. A client that has written
nothing reads the replica as it stands. Where the failover lost the client's last
acknowledged write, no copy holds its position: home answers
`504` at once, naming where it stands and of what history, the SDK
takes that as the client's position — only over the position that read was
sent after, so a write answered meanwhile is never set back, and never an
earlier term of the same lineage, which only a fenced old primary names — and
asks home once more with no mark. A history's lineage is the top half of its
id: every fork of the tenant keeps it, and a tenant deleted and made again
under its name, which starts again at term 1, has another — whose position
the client takes, from a 504 or any answer from home, its term lower or
not. A node from before lineages keeps none: a promotion or a restore it
makes starts a fresh one, so while a deployment is upgraded node by node a
router lagging a failover can again set a client back onto the old
branch.

**Failovers and moves.** Every router looks once a second at where the
primary of each tenant it keeps a read replica of is — its own directory, its
copy of the global one, or, for a tenant of another region keeping its own
directory, that region's router, asked once for all of its tenants — and
re-points the ones that follow another address, on a worker a node the
look waits for none of, each call to a node bounded by 10 s and no lock held
across one, so a node that does not answer holds up its own replicas alone.
A failover —
by hand or on a lease — or a move it makes has that look come at once, on a
thread of its own: the failover waits on no replica's node, which may be a
region away or cut off. So a region keeping its own directory follows a
failover in the tenant's home within about a second; a marked read sent after
a position meanwhile waits on the replica, and goes home past the wait. The
replica goes on from where it stood where the new primary's history passed
through it, and is sent an image otherwise; either way it answers again once
the new primary has said hello. Every 10 s, on a thread apart from the looks,
each router asks each replica's node it keeps which tenants it holds, and
makes again a copy gone — a node wiped, a file deleted by hand — which a
marked read meanwhile finds absent and sends home; a node out of reach holds
up that check alone, never a re-pointing. A tenant deleted takes its read
replicas with it, at once in its own directory — a change of one under way
waited out, 15 s at most and with no claim on the tenant held, so none is
left when the delete answers, and one whose node does not take it off keeps
the tenant (`502`), as `DELETE .../read-replicas` keeps
a record — and at the next look in another; the name is not made again
(`409`) while any directory keeps one of it, or is changing one:
made again, the look would point the old copy at another owner's tenant, in
a region nobody chose. A region keeping its own directory tells the global
router as it places one, before any copy exists, and that it keeps none
once the last is taken off; a create asks the regions so named for the
name alone, all at once, each within 10 s, and one not reached — or answered
by a router following the region's, which cannot see what the one it
follows has under way — refuses that name (`503`) and no
other. A name no region named asks none, so a region out of reach holds up
no create anywhere else. For a region gone for good, `DELETE
/_shard/tenants/<t>/traces/<region>` on the global router
lets go of its trace, and the name is made again. That cannot copy the new
tenant into the region: each read replica is bound to its tenant's
lineage — the top half of its history's id, which a failover and a
move keep and a tenant made again does not, recorded by the directory as
the tenant's node makes it or takes its image in — and its copy is told
it with every `follow` and keeps it across restarts. A copy
whose primary turns out to be of another lineage takes nothing from it and
deletes itself, file and `.follows`, so the deleted tenant's
rows do not outlive it on the node either
(`fenec_copies_refused_total` counts them); a look finding a
replica's tenant of another lineage takes it off rather than point it
anywhere. So does every other way a record outlives its tenant —
`?forget` of a node that comes back, a router lost between a
`follow` and the record. A primary of another lineage than the
directory's, on the node the directory places its tenant on, is the
tenant: another file of it promoted by force, an older file restored, a
file put in place by hand. (A copy of another lineage, such as a standby
that never took its first image, is promoted only by force: its node
refuses it, and the tenant stays where its rows are.) A failover records
the lineage the promotion answers, and a read replica's copy lost has the
primary's node asked which lineage it is of; the directory takes its
word, keeping the lineage before as one the tenant was, and the read
replicas bound to that one are bound to the new one rather than taken
off. A copy lost three times in a row anyway is made again no more —
`"given_up": true` in its list, counted in
`fenec_router_read_replicas_given_up` — until its primary or
lineage changes or it is placed again; its region's marked reads pass it
by meanwhile, rather than each meet its node's `404` and go home.
The router keeps that in memory alone: started again, it makes the copy
again up to three times more, and a standby router knows nothing of it. A tenant made before lineages were
kept has its lineage asked of its node at the next check, and each of its
read replicas is bound to it where its copy's node says the copy holds
its primary's history and is of it — taken off where the copy is another
tenant's, asked again later where it has not caught up. Mixed versions
protect less and break nothing: a node from before lineages ignores
`"lineage"` in a `follow` and has no
`/_admin/tenants/<t>/lineage`, and a router from before
them ignores `lineages` — no copy refuses on their word, until
every node and router runs a build that keeps them.
A marked read goes to no read replica of a tenant no directory places. One
moved into a region holding its read replica takes that replica
off — the tenant's own copies are nearer still — a change of it under way
waited out, and the replica held until the tenant is placed there, so no
look's `follow` lands on the node it moved to.

**A partition.** The replica's region cut off from the tenant's home:
writes from that region are answered `502`, as a node out of reach
is; a marked read naming no position is answered by the replica from the state
it last applied — as old as the cut, and, cut off across a failover of home,
on the branch the failover left — and one naming a position the replica does
not hold waits, goes home, and is answered `502`. Nothing is
answered from before a write the client was told landed. Once healed, the
replica's feed reconnects and it catches up. Reads that tolerate a bounded staleness —
a replica answering while it is at most so far behind, without a write of
the client's to wait for — are not offered.

**SDKs.** JavaScript `connect(url, { reads: 'nearest' })`
and `{ read: true }` on `run` and `batch`
(a batch of builder queries alone is marked on its own); Python
`Client(url, reads="nearest")` and `read=True`; Go
`fenecdb.WithReadsNearest()` and `db.Read()`; .NET
`ReadsNearest = true` and `db.AsRead()`; Swift
`Fenec.connect(url:reads: .nearest)`, Kotlin
`Fenec.connect(url, reads = FenecRemote.Reads.NEAREST)` and Dart
`Fenec.connect(url, reads: Reads.nearest)`, each with
`db.asRead()` ([Without a
file](https://fenecdb.com/docs/mobile#connect)). Each marks the builder's reads and the copy's requests alone,
and keeps one position for every copy of the client it makes. A
subscription opened through a Go `Read()` or a .NET
`AsRead()` copy, and its `/_changes` reads, are marked
and sent after that position too, and a server holds a stream for
`Fenec-After` as it holds a read: the read replica seeds it once
it has the client's own writes, and where it cannot in the wait its
`504` sends the stream home, so no stream is seeded from behind
them. JavaScript's `live` keeps its streams home, unmarked, and
sends each of its runs near: after the furthest of the client's position and
the one its streams named — a stream's head says where home stood as it
opened, `Fenec-Seq` and `Fenec-History`, and each
event's `seq` is a change of that history; a stream opened again
after a failover names the later term, and never sets the position back. A
run waits a second on the replica (`Fenec-Wait: 1000`), not five:
the replica has a write once home's fsync covers it — 283 ms behind at most
under `--sync 250` — and a link later, so one later than that is
behind, and home answers in a region's round trip; the router then lets the
copy be, so a replica cut off from home costs one second every 5 s, not one
every change. The second assumes that: a primary syncing its file every
second or more slowly (`--sync 1000` and up), or a link to the
replica slower than about half a second, has a replica take a write past a
run's second, and every run of a live query near it misses it — answered
from home, `fenec_router_replica_reads_total{outcome="fallback"}`
rising where `"replica"` does not, the rows a round trip home
later, a second later once every 5 s: right, never stale, but slower than a
client reading home (`reads: 'home'`), which such a deployment's
live queries are better made with. The read replica answers a run
once it holds the change that
ran it and the router sends one it cannot hold home; each answer names where
it read, and the next run goes after that too, so rows a run answered from
home showed — a change whose event is still on its way — never go back. A
`504` from home — the change lost in a failover — has the live query
take home's position and the run asked near again after it, home unmarked if
that fails too; answered home unmarked, the run names no position, so a run
after it may again show rows from before it — only after a failover that
lost writes, whose lost write the rows lose anyway. A stream whose head names
no history — a server from before heads named one — has the runs go home
until every stream's head names one again: its positions would pass none of
the client's. The rows never come from before a change the streams told of,
and the streams tell of every change home takes. Its polls
(`{ poll }`) go home: sent after its tag's change, a replica would
judge the tag as home does up to where it stands, but nothing in a poll names
a change home took since, so a replica behind — or cut off from home — would
answer `304` while home holds a write the query reads. Its
`subscribe` streams go home. A synced replica's streams, in the
browser and in the native libraries, go home; Python's client and the Swift,
Kotlin and Dart remotes open none.

**What it costs.** A marked read through the router of the replica's
region costs what one sent home does: 67.0 to 67.8 µs p50 against 66.2 to
66.9, three runs, with each node behind a loopback proxy standing for its
link, and 67.6 to 68.3 with a `Fenec-After` the replica holds
already (`make shard-bench ARGS=reads`, nodes and router in one
process). A client's read of the write it was just answered took 368 to 379
µs p50 on the replica — its wait for the write included — against 341 to 342
at home, both right after a durable write: under `--sync always`
the primary answers a write once its replicas' streams are sent it, and the
replica's part is applying it. A request no client marks pays a header looked
for: the router added 20.55 µs p50 to a request over the node's own against
20.7 for the code before read replicas, sixteen runs a side in ABBA turns,
both writing the node's answer in one `writev`, which took the
router from 23.6 µs (`make shard-bench ARGS=latency`). Between
regions the read near saves the round trip to the tenant's home, every
read.

## Tokens

There are three tokens: one for data, one per node for the router, and one
for managing the router. A user's JSON Web Token is bound to the tenant its
`tenant` claim names:

```bash
fenec-server --dir /data/n1 --http 127.0.0.1:8081 --admin-token a1 \
         --jwt-secret-file jwt.secret --policy policy.txt
fenec-server --jwt-secret-file jwt.secret --mint-token '{"sub":"alice","tenant":"acme"}'
```

Details

Three, each for a different job. Data requests carry whatever the nodes'
`--http-token` expects, and the router passes
`Authorization` through untouched. Each node has its own
`--admin-token`, which only the router holds. The router's
`--token` guards `/_shard/`; like the nodes, it refuses to
listen on a non-loopback address without one unless given
`--insecure`.

With `--jwt-secret` or `--jwt-keys` a node also takes
JSON Web Tokens held to its `--policy`, and the policy is the
node's, not a tenant's: `notes read,write where owner = $jwt.sub`
matches alice's rows in every tenant's file alike. So on a
`--dir` node a token reaches `/t/<t>/` only when its
tenant claim names `<t>`, a text or a list holding it, and
is refused with 403 elsewhere: on every route under the prefix, the
statements, REST, `/batch`, a subscription, `/_changes`,
`/_schema` and the statement counts. The check comes before the
tenant is looked up, so a refused token learns nothing of which tenants
exist. A token naming no tenant is refused on a `--dir` node
unless it is started with `--jwt-unbound-tenants`, which takes
such a token for every tenant it serves; `--jwt-tenant-claim
<name>` reads another claim than `tenant`. The node's
own `--http-token` and `--admin-token` are not bound.
The router forwards `Authorization` as it came and holds no keys,
so the node is what enforces it, behind the router or reached directly.
What a scoped token writes to a tenant is checked against its rules as on a
single file ([Scoped access](https://fenecdb.com/docs//docs/http#scoped-access)).

Every token is also read from the environment, which keeps it off a
command line every user of the machine reads with `ps`: a node's
from `FENEC_HTTP_TOKEN`, `FENEC_ADMIN_TOKEN` and
`FENEC_REPLICATION_TOKEN`, a router's from
`FENEC_SHARD_TOKEN`, `FENEC_SHARD_REPLICATION_TOKEN`
and `FENEC_SHARD_ENTRY_TOKEN`, an archiver's from
`FENEC_REPLICATION_TOKEN`. A variable set and empty, as an
environment file leaves a line it does not use, is no token; a token given
empty as a flag is refused at the start. A request carrying no
`Authorization` reads as the empty token, so an empty
`FENEC_SHARD_TOKEN`, taken for one, let anyone into
`/_shard/` -- and past the check that keeps a router with no
token off a public address. The kit's systemd units hand every one over
from an environment file ([Deploying it](#deploying)).

## Deploying it

The repository's [`deploy/`](https://github.com/fenecdb/fenec/tree/main/deploy)
is a kit for running all of the above on machines of your own: systemd
units, their environment files, alert rules, and the operator's
[runbook](https://github.com/fenecdb/fenec/blob/main/deploy/RUNBOOK.md).
`make drill` breaks a deployment of two regions on purpose, in
Docker, and holds it to losing no acknowledged write.

```bash
make deploy-check   # the units under a real systemd, the alert rules under promtool
make drill          # two regions: a node killed and back, a router killed, the global router cut off, a restore
```

- **A unit for each part**: a tenant node, a region's router and
    its standby, the global router and its standby, a node's archiver and
    its copy to a bucket (rclone), and the Cloudflare Tunnel the
    [global entry](#global-entry) reaches a router through. Each
    runs sandboxed as its role's user -- a node, the routers, the archiver
    and the bucket copy each their own, so none reads another's tokens --
    its tokens in an environment file only root reads. The tunnel lets a
    tenant's paths alone through, and Cloudflare Access with a service
    token the entry holds says who; the entry passes no cookie Access sets
    on to its clients, and the routers no Access header on to a node. The
    bucket copy refuses a new disk's first copy over the bucket's
    archive.

- **Alerts** for Prometheus and for Grafana: nodes and routers
    down, a region's primary router told from a standby, failovers and late
    leases, tenants without a replica, copies in their primary's zone, read
    replicas given up, connections near their ceiling, the bucket copy gone
    stale or unreadable by node_exporter, quota refusals, the entry's marks
    not believed, refusals and errors.

- **The drill's figures**, lease 2 s, three runs on one laptop --
    what they saw, not bounds: a node cut off the network and killed, as a
    machine lost, was failed over 1.9 to 2.6 s after the kill, the other
    tenants of its region never more than 18 ms between two answered
    writes, none of 41 577 acknowledged writes was lost, and a tenant
    restored from its node's archive held exactly the rows acknowledged by
    the moment it was restored to.

Details

`make deploy-check` boots systemd in two privileged
containers, installs every unit and environment file as the runbook
installs them, and runs them: `systemd-analyze verify` on each,
a region, two nodes and a tenant made through the global router, rows
written through the router and its standby, no token on any process's
command line or in an environment another role's user can read, the limits
and the sandbox in force (`systemd-analyze security` puts each
unit at 1.2), a node's stop syncing within its timeout, a sealed archive
verified with the key the unit hands its archiver and copied to a bucket
by real rclone while its tenant is written, a real node_exporter scraped
for the copy's success, a new disk's archive refused over the bucket's and
copied once the bucket is copied back, and a standby router promoted and
its unit switched as the runbook does it. cloudflared's own matcher is
asked where the tunnel sends 28 paths, every spelling of a tenant's
replication stream among them. promtool checks the alert rules and runs
their tests, and a test of fenec-shard holds every metric and label value
they read to a real scrape of every kind of process.

`make drill` runs each process in a container of its own on
one Docker network -- two regions, each a router keeping its own directory
and that router's standby, two nodes in two zones under
`--lease` and `--replicas`, an archiver for each node,
and the global router and its standby -- every token in the processes'
environments and no port fixed. A writer a tenant puts rows by id, the next
only once the one before was answered, while the drill cuts the global
routers off the network (each region writes on), cuts the node holding the
most primaries off the network and kills it, as a machine lost, and starts
it again, kills a
region's router and promotes its standby as an operator would, and marks a
moment it later restores a tenant to from its node's archive. Then every
acknowledged write is read back from each tenant's primary and its replica,
and the times are printed. The router fails a killed node over a lease
and a tenth after its last renewal and the promotions after: 1.92, 2.56 and 1.90 s
after the kill in three runs at a lease of 2 s. Its tenants' writers write
again once the request each had in flight to the dead node is let go of,
at the router's upstream timeout -- 4 s in the drill, twice its lease, as
the kit's routers run at twice theirs: 3.90, 3.97 and 3.93 s. The tenants whose
replica it held went 17, 18 and 17 ms between two answered writes at
most, the rest 16, 18 and 17; a reviewer's runs on another machine saw
28-30 ms, so these are what three runs saw, not bounds. The
runbook says what each alert means, the order to bring a deployment up in,
how to rotate each token and upgrade with no downtime, and what is not
measured yet.


---

Source: https://fenecdb.com/docs/serverless

# Serverless and Cloudflare

Run fenecdb inside a Cloudflare Worker, with one small
database for each Durable Object.

## Two ways to use it

Either the engine runs inside the function, one database per Durable
Object, or the function sends its queries to a `fenec-server` server.
Inside is faster; a server holds more data and more writers.

|   | the engine in the function | the function talks to a server |

| where the data is | in the object's memory, kept in its storage | in `fenec-server`'s file |

| a query | no network in the object: 2.5 ms for a request's `near` over 10 000 × 128 | a round trip to the server |

| writers | one an object; a tenant an object | one a file; a node holds many |

| size | about 60 000 rows of 128 dimensions, 12 000 of 768 ([Limits](#limits)) | the machine's disk, the file mapped |

| fits | a database a user, a team or a document; RAG over a tenant's own corpus | one large database, many writers, replicas |

Details

The browser module runs wherever V8 runs a WebAssembly module, a
Cloudflare Worker among them. A Durable Object gives it what a file gives a
server: one writer, and storage that outlives the process. So one object
holds one database, as a tenant is one file on a `fenec-server`
node.

The second needs nothing new: `Fenec.connect(url)`, the
Python `Client` or `fetch`, and a function's connections
are HTTP requests, so there is no pool to exhaust. Servers in several
regions take one URL through a Worker in front of them, which sends each
tenant's requests to its region: the [global
entry](https://fenecdb.com/docs/sharding#global-entry). The rest of this page is the first.

## A database a Durable Object

The object opens the engine on its first request, restores its database
from storage, and saves after every write, before it answers. Its alarm
keeps it: rows past their time deleted, a new image when one is worth
it.

```ts
import { DurableObject } from 'cloudflare:workers';
import { Fenec } from '@fenecdb/web';
import { maintain, persist, release, restore, schedule } from '@fenecdb/cloudflare';
import wasm from '@fenecdb/web/fenec.wasm';

interface Env {
  TENANT: DurableObjectNamespace<Tenant>;
}

export class Tenant extends DurableObject<Env> {
  #db: Promise<Fenec> | null = null;

  #open(): Promise<Fenec> {
    // Once, with every other event held back while it restores.
    return (this.#db ??= this.ctx.blockConcurrencyWhile(async () => {
      const db = await Fenec.open(wasm);         // the module as the Worker imports it
      await restore(db, this.ctx.storage);
      return db;
    }));
  }

  async query(sql: string, params: unknown[]): Promise<unknown> {
    const opened = this.#open();
    const db = await opened;
    try {
      return db.run(sql, params);
    } finally {
      let wrote = 0;
      try {
        wrote = await persist(db, this.ctx.storage); // durable before the answer goes
      } catch (e) {
        if (this.#db === opened) this.#db = null;   // the next request restores
        throw e;
      }
      // The alarm, once something is due: a row that will expire, an image
      // owed. Nothing else sets it -- an alarm is a row written too. The
      // write is stored either way, and the next one sets an alarm not set.
      if (wrote > 0) await schedule(db, this.ctx.storage).catch((e) => console.error(e));
    }
  }

  async alarm() {
    const opened = this.#open();
    const db = await opened;
    // Expired rows deleted, an image if one is worth it, the alarm set
    // again for what comes due next, or none.
    const done = await maintain(db, this.ctx.storage).catch(async (e) => {
      await release(db);                             // nothing more of it is stored
      if (this.#db === opened) this.#db = null;     // the retry restores
      throw e;
    });
    if (done.error) throw done.error;               // stored, the alarm not set: tried again
  }
}

export default {
  async fetch(req, env) {
    const tenant = env.TENANT.get(env.TENANT.idFromName('acme'));
    const { sql, params } = await req.json<{ sql: string; params: unknown[] }>();
    return Response.json(await tenant.query(sql, params));
  },
} satisfies ExportedHandler<Env>;
```

Details

A Worker imports a `.wasm` file compiled, and cannot compile
bytes at run time; `Fenec.open` takes the module so. The open
and the restore run inside `blockConcurrencyWhile`, once: opened
by whichever request found the object cold, two requests could each
restore an engine of their own and append to the storage as if alone.
`persist` runs in a `finally`, since a statement that
failed may follow ones in the same text that wrote. A `persist`
that fails leaves the database in memory holding writes its storage never
took, so the object lets it go and the next request restores what storage
holds, rather than answer from writes nobody kept. Its refusal, and a
failed `maintain`'s once `release` has let the
database go, come only once nothing of that database is on its way to
storage, and nothing more of it will be: the database restored next reads
all the first stored and is the only one writing. An error setting the
alarm fails neither a write, which is stored, nor `maintain`,
which answers it as `error` for the alarm to be tried again.
`integrations/cloudflare/example` is this Worker with a tenant
from the path, `POST /t/<tenant>/query`, and its test runs
it under `wrangler dev` — workerd, as Cloudflare runs it — then
stops and starts it over the same storage and reads the rows, a
`near`'s answer and each tenant's own database back, and sends
a cold object 32 writes at once.

## What the storage holds

`persist` saves a full image once, then only the new writes:
one row each in a SQLite-backed object's storage, which bills by the row
written. If storage stops part way through, the last whole database is
still there.

|   | rows written before | now |

| a single-row write, 1 writer | 3.4 | **1** |

| a single-row write, 8 writers at once | 3.4 | **0.52** |

| a single-row write, 32 writers at once | 3.4 | **0.15** |

| a checkpoint, 10 000 × 128 (a 5.7 MB image) | 101 | **5** |

| a checkpoint, 50 000 × 128 (a 27.6 MB image) | 479 | **16** |

Details

`persist` keeps what a file would hold: an image the first
time, then only the writes since the last call — the engine's journal, as
`persist` in the browser takes it for IndexedDB — until they
outgrow half the image and 64 KiB, and a new image replaces them.

A SQLite-backed object — the only kind a new account makes — has
`ctx.storage.sql`, and the database is a table a generation,
`fenec:1`, `fenec:2` and on (the key, then the
generation): the image's rows, cut under the 2 MB a row takes, then a row
each time writes are stored. Nothing names where the database stands
beside the rows: the table there is says it, so a write is the one row.
A new image is written, and the table before it dropped, in one
`transactionSync`: workerd counts a table made as two rows
written and a table dropped as none, where a `DELETE` of the
rows before counts each, and storage that stops part way holds the
generation before, whole. The rows are an `INTEGER PRIMARY KEY`
table with no other index, since an index is a row written more.

A write goes out at once, unless the one before it is still on its way
to being durable (`storage.sync()`): then it waits for that, and
every request whose writes land meanwhile — a Durable Object takes the
next request in while one waits — goes out with it, in one row. Each
`persist` resolves only once its own writes are durable, and a
read that may have seen writes on their way waits for them, so no answer
leaves before what it shows is kept, and a lone write answers as soon as
before: 2.0 to 2.3 ms p50 against 2.4 to 3.1. Waiting a turn of the event
loop instead, for requests to land, cost a lone write 1.7 to 2.4 ms more.
The figures above are `make cloudflare-bench`'s, under
`wrangler dev` with every storage call counted: a write before
was a value, a record naming it and an alarm set again, and a checkpoint a
value each 128 KiB of the image and a delete each of the values
before.

A key-value backed object, made before SQLite with
`new_classes`, has no `sql`; there the image and the
writes are cut into values under its 128 KiB, every image goes under a
generation of its own, and the record naming the whole one is written
last. A test stops either kind of storage after every number of
operations, an image's, an append's and a compact's, and restores the
last whole database each time; and a write that fails fails every write
that went out with it.

A database kept by an earlier version on a SQLite-backed object is in
values, which `restore` no longer reads there. To move one,
restore it through the key-value methods alone — an object holding
`get`, `put`, `delete` and
`list` of `ctx.storage` and no `sql` —
and `persist` through `ctx.storage`.

## A cold start

After an eviction or a deploy, the first request has to load the database
again: tens of milliseconds for 10 000 rows, under 200 for 50 000. The
alarm's `maintain` folds the writes into a new image once they
reach half of the last one, for fewer rows to read and no vectors to
link.

| rows | first `near`, as written | after a checkpoint | every `near` after |

| 1 000 | 20 ms | 21 ms | 2.1 ms |

| 10 000 | 39 ms | 47 ms | 2.1 ms |

| 50 000 | 175 ms | 157 ms | 2.1 ms |

Details

An object evicted, or a deploy, starts the next request with no
database in memory: the module instantiated, the image and the writes
after it read out of storage and loaded. The graphs the image holds are
restored as they were; the writes after it are applied as the write path
would, their vectors linked into the graph one after another — the browser
module has one thread. The first `near` after a restart, against
the ones after it, under `wrangler dev` on an M1 (`make
cloudflare-bench`, median of five restarts; rows of a text, an int and
a 128-dim vector, 1 000 a request, each kept before it was answered, no
alarm) are in the table above; a second run was within 2 ms of each.

Linking the writes after the image is no longer what a start waits
for: when this page was first written it took the first `near`
over 10 000 rows to 792 ms and over 50 000 to 1 455, and the same
measurement on today's module gives 39 and 175, so a checkpoint takes
50 000 rows from 175 ms to 157, 10 000 none quicker, and is mostly worth
the rows it folds. The
rows are read into the module a row at a time rather than joined first:
50 000 rows as written started in 175 ms against 186 to 196, after a
checkpoint 157 against 168 to 171. A request's round trip through
`wrangler dev` is in every figure. The rows went in at 5 200
to 15 000 a second, a request of 1 000 at a time with its vectors linked
and kept before the answer.

The figures under “after a checkpoint” are a new image
taken by hand (`checkpoint`); an object that keeps its
database from its alarm takes one by the rule below.

## Maintenance

A server deletes rows past their `@ttl`, compacts its file
and keeps its graphs on threads of its own. An object has one alarm, which
is a row written each time it is set and wakes the object each time it
rings: `schedule` sets it after a write only once something is
due, and `maintain`, its work, sets it again only for what comes
next. An idle object sets none.

| rows written, 10 000 × 128 | an alarm after every write, a checkpoint past 64 KiB | maintenance |

| 30 bursts of 20 single-row writes, a second's quiet after each | 651 to 652 | **600** |

| 12 bursts of 500 rows | 100 | **19** |

| 20 000 rows past their time, deleted | never | **2** and 2 alarms |

Details

`maintain` deletes the rows a minute (`grace`)
past their time, as a server's sweep does, so a reader of
`expired()` as frequent sees each before it goes; 10 000 a call
(`sweep`) in statements of 1 000 ordinary deletes, stored as any
write is, the alarm set again at once past the bound. The object answers
nothing while it runs: 1 000 rows of 100 000 took 0.46 ms to delete in the
module, so 10 000 hold it about 5 ms, far inside an alarm's 30 s of CPU and
15 minutes. Then it sets the alarm for the next row's time and the grace,
no sooner than a minute (`every`) after this sweep, or sets
none; each sweep is two rows at most, its deletes and the alarm, so rows
expiring a few a second cost two rows a minute rather than two a row. No
export of the module was added for it: a `del` naming
`expired()` and walking the field's index in order says what
the server's sweep does natively.

It writes a new image when the writes since the last reach half of it
and 64 KiB (`min`), where `persist` folds them in a
request, or when half of what storage holds is dead and a mebibyte —
what a sweep or deletes left, which the writes' size alone does not show.
Woken to sweep and nothing more, it stores the deletes as one row and
writes no image. Once `schedule` is called, a request folds the
writes only past the whole image: the alarm takes the image at half, and
no request waits for it. A threshold of 64 KiB, as the alarm used, wrote a
27.6 MB image as 16 rows for writes that were one or two; at a quarter of
the image, rewrites of 1 000 rows of 128 dimensions a request, a row each,
brought an 11 MB image, 8 rows, every 6.

The module never gives memory back, and holds every version of a record
written since it was opened. Once the dead versions reach half of what it
holds — at least 4 MiB, at most 16 — `maintain` makes sure
storage holds an image without them and answers `reload`: the
example lets the database go and restores a new module, as it does on any
failure — calls arriving wait while the ones holding it finish, a write on
its way to storage among them, then it is released and closed and another
restored. Let go of at once, a database an alarm dropped while a request's
write waited behind another's stored it after the next request had
restored a second database and written, under the same id: an answered
write lost. In Node, 50 000 rows of 128 dimensions with three
in five past their time held 42.7 MB of module through two sweeps of
10 000; the third left half of storage dead, its image took the module to
51.6 MB, and the module restored from it held 19.2. A
`compact` in the module would have rebuilt every graph holding
tombstones and built its image whole, and given nothing back.

## Limits

An object holds about 60 000 rows of 128 dimensions, or 12 000 of 768. For
more, use a `fenec-server` server or spread a tenant over more
objects.

- **One writer an object.** Writes to one tenant go one after another,
  as on a server; tenants write side by side.

- **Bundle collation data.** Collation data for scripts the module does
  not carry is fetched from a URL by default; in a Worker, pass
  `{collation: name => ...}` to `Fenec.open` and hand
  the chunks from your bundle.

- **Install it from npm.** `npm install @fenecdb/cloudflare
  @fenecdb/web`; both are published with each release.

Details

A Worker's isolate has 128 MB, the WebAssembly memory in it, and
`wrangler dev` does not hold a Worker to it: the figures above
reach past it. What the module holds, measured in Node on the same module
(rows of a text, an int and a vector under HNSW):

| rows | restored | written, 1 000 a request | then a checkpoint | the image |

| 10 000 × 128 | 16 MB | 24 MB | 28 MB | 6 MB |

| 20 000 × 128 | 29 MB | 37 MB | 42 MB | 12 MB |

| 30 000 × 128 | 41 MB | 46 MB | 53 MB | 18 MB |

| 50 000 × 128 | 67 MB | 73 MB | 83 MB | 30 MB |

| 60 000 × 128 | 80 MB | 85 MB | 97 MB | 36 MB |

| 3 000 × 768 | 21 MB | 40 MB | 40 MB | 9 MB |

| 5 000 × 768 | 35 MB | 58 MB | 58 MB | 16 MB |

| 10 000 × 768 | 66 MB | 84 MB | 84 MB | 32 MB |

| 12 000 × 768 | 80 MB | 102 MB | 102 MB | 38 MB |

A WebAssembly memory grows and never gives back, so an object holds
the most it ever held. The module keeps an index's vectors in chunks of
about a mebibyte and lets a segment of documents grow no further than the
record that seals it: grown by doubling, old and new side by side, 5 000
rows of 768 written 1 000 a request held 100 MB, 58 now, and 10 000 of
them 150, 84 now. A restore keeps the image it reads and the documents in
it, rather than copy each out: 10 000 rows of 768 restored held 97 MB, 66
now; and it copies the rows into that image as it reads them, one at a
time (`loadParts`): read whole, joined, then copied in, a
27.6 MB image — 50 000 rows of 128 — held 60 MB of JavaScript beside the
module's 42 at its peak, and holds 4, the process 105 MB more resident
against 49 to 55 (`make cloudflare-bench`, in Node). A checkpoint takes the image a mebibyte at a time
(`snapshotChunks`), each chunk made as it is taken out of the
documents the stores hold, not a copy of them, and let go once it is in
storage: made all at once, the image stood beside the rows, and 50 000
rows of 128 went from 73 to 109 MB at it, 83 now. So an object holds
about 60 000 rows of 128 dimensions and 12 000 of 768, however many a
request brings; past that, a
`fenec-server` node and `Fenec.connect`, or a tenant's
data spread over more objects.


---

Source: https://fenecdb.com/docs/microservices

# Services and events

Give each service a database of its own, and let other
services follow its writes as events.

## A database per service

Run one tenant node and give each service its own tenant: a separate file
that no other service can read. A token then decides which collections and
rows each caller may touch.

Details

Each service keeps its own data and tells the others what changed.
fenecdb gives a service a database of its own on a shared node, a token that
reaches only that, and the writes on its disk as a stream of events — which
is what an outbox table and a relay are usually built for.

A tenant node (`fenec-server --dir`) keeps a file per tenant, and
a tenant is named by the path, `/t/<tenant>/` — never by
the query, so one service cannot read another's rows. Give each service its tenant,
and a router (`fenec-shard`) in front of the nodes when there are
several: see [Tenants and sharding](https://fenecdb.com/docs//docs/sharding).

Within a database, a JSON Web Token held to a policy gives each caller
the collections, and the rows, its role may read or write
([Scoped access](https://fenecdb.com/docs//docs/http#scoped-access)). A service that only
reads can be pointed at a replica, which refuses every write
([Replication](https://fenecdb.com/docs//docs/replication)), or hold a token whose policy
grants `read` alone.

## Every write as an event

Turn on `--cdc` and every write that reaches disk becomes an
event a consumer can read. You do not need an outbox table: a batch's
rows are announced together.

```bash
fenec-server --file orders.fenec --http 127.0.0.1:8080 --cdc

# a consumer the server keeps the cursor of, made once
curl -X POST http://127.0.0.1:8080/_changes/consumers/billing
```

Details

An outbox table exists because a service must change its data and
announce the change together, or not at all. A fenecdb
`/batch` is one record on disk, so the rows it
writes are announced together already: `GET /_changes` hands
over every write once it is on disk, the documents with it
([Change data capture](https://fenecdb.com/docs//docs/http#changes)). The collection is
its own outbox. Where the event should say something the rows do not —
a command, an intent — write it into a collection of its own in the same
`/batch`, and it comes in the same record.

A consumer in Python:

```python
from fenecdb import Client

db = Client("http://127.0.0.1:8080", token="...")
for batch in db.follow("billing"):        # committed when the loop comes back
    for change in batch:
        if change["collection"] == "orders" and change["op"] == "put":
            bill(change["doc"])           # an exception: the batch is read again
```

## On to Kafka, or anywhere

The relay passes a consumer's events on, as JSON lines or to a webhook.
Pipe the lines into Kafka with any tool that writes to a topic.

```bash
python -m fenecdb.relay http://127.0.0.1:8080 --consumer kafka \
  | kcat -P -b broker:9092 -t orders
```

Details

`python -m fenecdb.relay` reads a consumer's writes and hands
them on — to standard output, one JSON object a line, or to a webhook —
and moves the cursor only once they are out. Kafka takes them through any
tool that writes lines to a topic, as above; a webhook:

```bash
python -m fenecdb.relay http://127.0.0.1:8080 --consumer search \
  --to https://indexer.internal/hook
```

## Delivery and replay

Each event arrives at least once, in order within one database. Make your
handler safe to run twice; a consumer that falls too far behind starts over
from a fresh read.

Details

- **At least once.** A consumer that crashes between reading and
  committing reads those writes again. Key what you do by
  `seq`, or by the collection and `id`: a
  `put` carries the whole document, so applying one twice
  leaves what applying it once did.

- **In order, one database at a time.** `seq` is the
  database's change counter; a batch's writes come together. There is
  no order across tenants.

- **Kept for a while, not for ever.** The server keeps as many writes
  as `--replication-buffer` holds, 64 MiB unless given. A
  consumer further behind is answered 410, never handed writes with a gap
  in the middle. To start over: take the last write on disk
  (`Fenec-Next` of `GET /_changes` with no
  `since`), read the collections whole, then go on from that
  number — a write read in both is applied twice, which a `put`
  by id makes harmless.

- **What is on disk.** A write the server lost in a crash was never
  handed to anyone, as it was never sent to a replica.

- **Reading one's own writes.** A service that writes on the primary
  and reads on a replica sends the write's `Fenec-Seq` back as
  `Fenec-After`, and the replica answers once it holds the write
  ([Reading your own
  writes](https://fenecdb.com/docs//docs/replication#read-your-writes)).

- **Retries.** A service that sends a write again after a timeout
  sends it with an `Idempotency-Key`, and a write that landed is
  answered as before rather than made twice
  ([Retrying a write](https://fenecdb.com/docs//docs/http#idempotency)).

## What it is not

It is not a message broker. To keep events for a long time or share them
with many teams, relay them into one.

Details

A consumer reads one database's writes; there are no topics,
partitions or retention of their own, and a server keeps its writes in
memory up to the buffer. For fan-out to many teams, a long history or
events that outlive the database, relay them into a broker and let it
keep them.


---

Source: https://fenecdb.com/docs/import

# Import

One command turns a SQLite file or a PostgreSQL table into a
fenecdb collection, its fields typed and its indexes built, and can keep it
following the table's changes.

## Import a table

Name the source, the table and the collection to make. The schema is worked
out from the table's columns, and each index you name is built once the rows
are in.

```bash
fenec import data.sqlite --table docs --into articles --index "category@hash"

fenec import postgres://user@host:5432/database --table docs \
    --into articles --index "category@hash"
```

```text
$ fenec import data.sqlite --table docs --into articles --index "category@hash"
source   data.sqlite -> docs
target   articles.fenec -> articles
3 records, articles.fenec (12.82ms)
index    category @hash
```

Details

The import derives the schema, the table's integer key becoming each
document's id, writes the rows in batches and builds the indexes after the
last, rather than keep them up a row at a time. The collection goes into
`<into>.fenec` unless `--file` names another. A
column of embeddings comes across too ([Vector
columns](#vector-columns)). Every option is in `fenec import --help`.

## Look before you write

`--dry-run` prints the schema it would make, and any warnings,
and writes nothing.

```text
$ fenec import data.sqlite --table docs --into articles --index "category@hash" --dry-run
source   data.sqlite -> docs
target   articles.fenec -> articles

create collection articles (
  title      text           <- text
  category   text           <- text
  score      int            <- int
  published  timestamp      <- datetime
)
  id  <- id (INTEGER PRIMARY KEY)
create index on articles (category) @hash

nothing was written (--dry-run); use --count for the row count
```

- **`--count`** also prints how many rows will be read.

Details

`--count` is a separate flag because counting means a full scan:
SQLite keeps the row count nowhere and `count(*)` in PostgreSQL
reads the table from the start. Silently reading everything twice would be a
bad default.

## Sources

A SQLite file is read directly, and a PostgreSQL server is read over the
network. Nothing else needs to be installed for either.

- **SQLite:** no `sqlite3` library or binary needed.

- **PostgreSQL:** no `psql` or `pg_dump` needed;
    pgvector's columns are recognised.

Details

**SQLite.** The file format is read directly — neither the
`sqlite3` library nor the binary is needed. `INTEGER PRIMARY
KEY` becomes the document id, `--vector field:N` turns
little-endian `f32` BLOBs into `vector<N>`, and
untyped columns are resolved from the first 1000 rows.

**PostgreSQL.** It connects to the live server and streams
`COPY ... TO STDOUT` — no `psql`, no
`pg_dump`. Authentication is SCRAM-SHA-256, hand-written in
`fenec-wire`'s client over `fenec-http`'s own crypto. pgvector's
`vector`, `halfvec` and `sparsevec` columns
are recognised along with their dimensions.

## Type mapping

Each source type becomes the nearest fenecdb type. Types with no exact
match are refused until you say what to do with `--cast`.

| Source | fenecdb |

| `integer` `bigint` `int2/4/8` | `int` |

| `real` `double` `float4/8` | `float` |

| `text` `varchar` `char` `clob` | `text` |

| `blob` `bytea` | `bytes` |

| `boolean` | `bool` |

| `date` `datetime` `timestamp(tz)` | `timestamp` |

| `int[]` `float[]` `text[]` | `[int]` `[float]` `[text]` |

| `vector(N)` `halfvec(N)` | `vector<N>` `vector<N, f16>` |

| `sparsevec(N)` | `sparse<N>`, read in the same text form |

| `geometry(Point, 4326)` `geography(Point)` `point` | `geo`, `[lon, lat]`; another SRID or shape refused ([Points](https://fenecdb.com/docs/geo#postgis)) |

| `uuid` | `text` |

| `citext`, a SQLite column `COLLATE NOCASE` | `text nocase`, compared by its case fold ([Case-insensitive text](https://fenecdb.com/docs/fenecql#nocase)) — every script's, where SQLite's `NOCASE` folds ASCII alone |

| **`numeric` `decimal`** | no equivalent — `--cast` required |

| **`json` `jsonb`** | `json`, its numbers as written; a path reads into it ([Objects and paths](https://fenecdb.com/docs/fenecql#json)) |

Details

Types without an equivalent are not silently rounded: say what you want with
`--cast <field>=<type>`. On PostgreSQL,
`--cast price=text` preserves a decimal exactly. In SQLite a
`decimal` is already stored as an `int` or a
`float` at the source, so the exact value is gone before the import
starts.

## Filtering

`--where` keeps only the rows that match a FenecQL expression,
written with the new collection's field names. It works the same on both
sources.

```bash
fenec import data.sqlite --table docs --into articles \
    --where 'category = "book" and score >= 10 and tags has "rust"'
```

- **`--source-where`** (PostgreSQL only) filters on the
    server in SQL, so skipped rows never travel.

- **`--limit`** counts rows that pass the filter.

Details

`--where` is a **FenecQL expression** and behaves identically
on both sources: it is applied after the row becomes a document and before it
is written. The field names are the target schema's — the same expression you
would later write with `get`.

On PostgreSQL there is also `--source-where`, which goes into the
`SELECT` inside the `COPY` so non-matching rows never
travel over the wire. That one is **SQL, not FenecQL**. They are separate
flags on purpose: one is a portable expression, the other a win pushed down to
the source. Both can be used together.

```bash
fenec import postgres://... --table docs --into articles \
    --source-where "year >= 2024" --where 'tags has "rust"'
```

With a local `--where` in place, `--limit` is not
  pushed to the source: cutting on the server first and filtering locally
  afterwards would yield fewer rows than asked for. `--limit` always
  counts rows that pass the filter.

## Following changes

With `--follow` the import keeps running after the copy,
applying the PostgreSQL table's changes as they commit. Run the same command
again and it picks up where it stopped.

```text
$ fenec import postgres://app@db:5432/app --table docs --into articles \
    --index "category@hash" --follow
source   app@db:5432/app -> docs
target   articles.fenec -> articles
follow   slot fenec_articles (created), publication fenec_articles (created)
note     the slot keeps the server's WAL until this follower confirms it; to stop following for good: select pg_drop_replication_slot('fenec_articles')
copying  there is none yet
100000 records, articles.fenec (339.37ms)
following docs: ctrl-c stops, and the same command resumes
42 transactions, 57 changes, at 0/2BEA458
```

- **The server needs `wal_level = logical`** and the table
    an integer key; nothing is installed on it.

- **Nothing is lost or applied twice** when the connection drops or the
    follower is killed.

- **Drop the slot when you stop for good,** or PostgreSQL keeps its log
    for it.

Details

With `--follow` the import does not end at the copy: it keeps
applying the table's inserts, updates, deletes and truncates as they commit,
until it is interrupted. Run the same command again and it picks up where it
stopped, without a second copy.

It reads a logical replication slot through PostgreSQL's own
`pgoutput`, so nothing is installed on the server. The
publication and the slot are made when missing (`--publication`,
`--slot`; both default to `fenec_<into>`). The
server needs `wal_level = logical`, and the table needs an integer
key that becomes the document id, since a changed row has to be found again.
It must also be in the replica identity: the primary key, or
`replica identity full`.

- **A row arrives as the copy would have written it.** The stream goes
    through the same type mapping, `--cast`,
    `--vector` and `--where` as the copy. A row that
    stops passing `--where` is deleted from the collection.

- **Nothing is confirmed before it is on disk.** The slot only moves
    past a transaction once an fsync has covered it. A broken connection or a
    killed follower starts again from there, and the writes it replays are
    puts and deletes by id, so applying them twice changes nothing.

- **An untouched vector is not resent.** A vector of 768 floats is
    3 KB, past PostgreSQL's TOAST threshold, so an update of any other column
    arrives without it; the follower takes it from the collection.

- **A copy is resumed only when it is whole.** A small
    `_follow` collection in the file records which collection
    follows which slot and whether its copy finished. A copy cut short, or
    one whose slot was dropped in the meantime, is made again rather than
    streamed on top of, and a collection the follower did not copy is never
    replaced.

Measured against the Makefile's server, 10 000 rows with a
`vector(384)`, the mirror a file (`make
follow-bench`):

|   | vector(384) | vector(768), TOASTed |

| commit to visible, one row a transaction | p50 0.16 ms, p99 1.56 ms | p50 0.16 ms, p99 1.57 ms |

| one transaction updating every row | 40 400 rows/s | 52 600 rows/s |

| 2 000 one-row transactions from one client | visible 0.3 ms after the last | 0.2 ms |

| a connection the server cut | 553 ms | 536 ms |

**The slot keeps the server's log** until the follower confirms it,
  for as long as it exists. To stop following for good, drop it:
  `select pg_drop_replication_slot('fenec_articles')`. PostgreSQL
  13 and later can cap what a forgotten slot holds with
  `max_slot_wal_keep_size`.

The follower is the file's only writer, as every fenecdb file has exactly
one: stop it to open the mirror elsewhere, or serve the mirror from the
server that follows ([below](#serving)). A column added to the
table after the copy is reported and left out. A column the collection needs
that is dropped from the table stops the follower.
`--source-where` and `--limit` do not combine with it:
the first narrows the copy but not the stream, and a mirror keeps every
row.

## Serving the mirror as it follows

`fenec-server --follow` runs the follower inside the server, so the
table is served over HTTP and subscriptions while it follows.

```text
$ fenec-server --file articles.fenec --http 0.0.0.0:8080 --http-token "$TOKEN" \
    --follow postgres://app@db:5432/app --follow-table docs --follow-into articles \
    --follow-index "category@hash"
opened: articles.fenec
following: docs
fenec-server 0.1.10: articles.fenec  [sync=250 ms]
fenec-http 0.1.10 listening on: http://0.0.0.0:8080  [token]
follow: docs: slot fenec_articles (created), publication fenec_articles (created)
follow: the slot keeps the server's WAL until this server confirms it; to stop following for good: select pg_drop_replication_slot('fenec_articles')
follow: copying docs: there is none yet
follow: 2000 rows of docs copied
follow: docs copied, 100000 rows in 160.10ms
follow: docs: streaming its changes
```

- **Import options take a `--follow-` prefix:**
    `--follow-index`, `--follow-cast` and so on.

- **The mirrored collection is read-only** to clients
    (403); the rest of the file takes writes.

Details

A second process over the follower's file corrupts it, so serving the
mirror took a copy of it and a process of its own. `fenec-server
--follow` runs the follower inside the server instead, over the file
the server serves: the table is served over HTTP and subscriptions as it
follows, with no second copy.

Every option of `fenec import` that says how a row becomes a
document — `--index`, `--vector`, `--cast`,
`--id`, `--where`, `--batch` — is taken as
`--follow-index` and so on, as are `--follow-slot` and
`--follow-publication`. The collection takes no write but the
follower's: a client's is refused with 403, since the next
change from PostgreSQL would write over it. The rest of the file takes
writes as ever. A subscriber of the collection
(`GET /articles/changes`) hears each change as the follower
applies it, and so does a replica of the server.

The follower stops with the server, its last changes on disk and
confirmed, before the shutdown's checkpoint. A server killed outright starts
the stream again from what it last confirmed; the changes it applies again
are puts and deletes by id, so nothing is lost and nothing comes twice. An
error the follower cannot wait out ends the server rather than leave it
answering from a mirror that no longer moves.

Against the Makefile's server, 10 000 rows with a `vector(384)`
under an HNSW index, a subscriber reading `/changes` over HTTP
(`make mirror-bench`):

| commit to the subscriber | p50 | p99 |

| an update of a title | 0.09 ms | 4.00 ms |

| a row inserted with its vector, put into the graph first | 0.19 ms | 3.34 ms |

Commit to visible in the file, with no server between
(`make follow-bench`), was 0.16 ms at the median on the same
machine. Killed while the table took a row a transaction, the server was up
again in 75 ms and held every one of the table's 14 063 rows 370 ms after it
started. The follower adds 178 KB to `fenec-server`.

## Vector columns

A column of embeddings becomes a `vector` field, and an
`@hnsw` index named with `--index` is built once every
row is in. pgvector's columns are recognised with their dimensions; a
SQLite BLOB of little-endian `f32`s is named with
`--vector`.

```bash
fenec import data.sqlite --table passages --into passages \
    --vector embed:384 --index "embed@hnsw(cosine)"

fenec import postgres://user@host:5432/database --table passages \
    --into passages --index "embed@hnsw(cosine)"
```

```text
$ fenec import data.sqlite --table passages --into passages --vector embed:384 --dry-run
source   data.sqlite -> passages
target   passages.fenec -> passages

create collection passages (
  text    text           <- text
  source  text           <- text
  embed   vector<384>    <- blob
)
  id  <- id (INTEGER PRIMARY KEY)

nothing was written (--dry-run); use --count for the row count
```

Details

The graph is built after the copy, over every vector at once and on
every core, which is the order a bulk load wants (Bulk load first, index
after). pgvector's `vector`, `halfvec` and
`sparsevec` columns become `vector<N>`,
`vector<N, f16>` and `sparse<N>` with
nothing named; `--follow` keeps them in step as any column, an
untouched vector taken from the collection rather than resent (Following changes).

## Speed and testing

100 000 rows × 3 dimensions, index build included, take about 3.1 s from
either source.

```bash
make import-test        # the PostgreSQL arm and --follow — needs Docker
cargo test -p fenec-import --test all -- --ignored pg:: follow::
```

Details

Import is a default feature of the `fenec` binary and accounts
for 275 KB of it. `--no-default-features` removes it, and
`fenec backup`, `archive`, `restore` and
`fenec mcp` with it (502 KB); with the `cli` profile as
well, the binary drops from 3013 KB to 1880 KB (`make small`).


---

Source: https://fenecdb.com/docs/embedding

# Embedded Rust

Use fenecdb as a Rust library inside your own
program.

## Open a file and query it

Open a file with `fenec_core`, parse a query with
`fenec_ql`, and run it.

```rust
use fenec_core::prelude::*;
use fenec_ql::parse;

let mut db = fenec_core::fs::open("data.fenec")?;
for stmt in parse("get docs near embed $1 limit 5")? {
    let r = db.execute_with(&stmt, &[Value::Vector(embedding)])?;
}
```

Details

`fenec-core` is a plain Rust library with no dependencies.
Parsing is `fenec-ql`; everything else is the engine.

## Reads and writes take different locks

Reads run side by side; writes run one at a time. Several statements run as
one block land together, or none of them does.

```rust
let stmts: Vec<Statement> = parse("put orders {total: 12}; set stock {n: 4} where id = 7")?;
let block: Vec<(&Statement, &[Value])> = stmts.iter().map(|s| (s, &[][..])).collect();
match db.execute_block(&block) {
    Ok(answers) => { /* both landed, as one record */ }
    Err((at, e)) => { /* statement `at` failed: neither landed */ }
}
```

**One process per file.** Two processes opening the same file and
  writing it corrupt it. There is no lock file and no advisory locking — the
  single-writer rule is yours to keep.

Details

`Database::query` takes a shared lock and
`execute_with` the exclusive one. Every write is a block: what a
statement writes lands in the file as one record, or — a hook's refusal, a
document the schema refuses half way through a `put` of many —
none of it does, and the database is as it was. Several statements are one
block with `execute_block`, or between `begin` and
`commit`, with `rollback` to put them back; whoever
opens one holds the exclusive lock until it ends. A `create`, a
`drop` or a `create index` in it is put back with the
rest: the collection made goes, the one dropped comes back, the index goes.
A `compact` rewrites the file and runs on its own, outside a
block. FenecQL has no `BEGIN` or `COMMIT`: over HTTP a
`/batch` is one block ([HTTP endpoint](https://fenecdb.com/docs/http)).

## Compaction is the caller's too

An update leaves the version it replaced dead in the file until a
`compact`. The server and the native library compact on their own;
a program embedding the crate starts the same thread over its database, or
asks from one of its own:

```rust
use fenec_core::engine::{compact_when_due, CompactPolicy, Compactor};

let db = Arc::new(RwLock::new(fenec_core::fs::open("app.fenec")?));
// Looks every 5 s; compacts beside the queries once half the file is dead
// and at least 64 MB, or sooner as the file nears the memory the system
// has for its pages. Stops when dropped.
let _compactor = Compactor::start(&db, CompactPolicy::default())?;
// Or, from a thread you already have:
compact_when_due(&db, &CompactPolicy::default());
```

`Database::garbage` says what the file holds against what is
live, without reading a document.

## Durability is the caller's job

Call `sync` yourself: until you do, recent writes can be lost in
a crash. The library never calls it for you.

Details

File writes are buffered. If the process crashes before `sync` is
called, the last writes are lost. In embedded use nothing calls it for you —
`fenec-server` has the `--sync` policy and a shutdown hook
precisely because the library does not.

A storage error stops writes. Once the sink refuses an append, a
`sync` or a rewrite, every later write — and every later
`sync` — returns `Error::Io` until the file is reopened,
and `Database::failure()` says what failed. A failed
`fsync` is not retried: the kernel may already have dropped the
pages, so a retry that succeeds would prove nothing. Reads go on, from
memory.

The same applies to the HNSW graph. It is derived data and is written only
during `snapshot`, `compact` and `checkpoint`,
never on the write path. Without a graph in the file it is rebuilt on open:
3.4 s at 100 000 × 128, against 9 ms with it. A crash does not cost the
graph: the next open restores the one the last checkpoint wrote and applies
the writes since — 0.07 s for 1 000 of them at 100 000 × 128, where the
rebuild took 3.4 s. A process that has to answer before those are linked
opens with `fs::open_serving`: the vectors written since go in
unlinked, `near` measures each of them until
`Database::link_pending(n)` has linked them, and the process
calls it under its write lock a few at a time, as `fenec-server`
does. `Database::unlinked()` says how many are left.

## Measuring the footprint

`Database::memory_bytes()` tells you, for free, how much memory
the engine's data takes. The process as a whole uses more.

Details

`Database::memory_bytes()` sums the segment bytes, offset
indexes, vector arenas, graph links and every other index from counters
they keep as they change, so it costs nothing to ask. It is not RSS —
allocator slack, HNSW build buffers, upper-level neighbour allocations and the
2–3× transient peaks all sit outside it. Measured ratios are in
[Limits](https://fenecdb.com/docs/limits#memory-and-scale).

## Features

Two parts of the library are left out of a wasm build.

| Feature | What it brings |

| `std-fs` | Buffered file I/O — `fenec_core::fs`. Off in wasm builds. |

| threads | The parallel HNSW build path, compiled out of wasm32 entirely. |

Details

`now()` errors on wasm32-unknown-unknown, which has no clock;
time is passed in as a parameter there.

## Errors

Every call returns the same error type, `Error`.

Details

`Error` in `fenec-core/src/error.rs` is the single
error type across the whole engine: allocation-free variants, no
`Box`. `fenec-server` maps it onto HTTP statuses: 404
for a missing collection, 409 for one that exists or a taken id, 400 for a
query or type error, 403 for a write refused, 500 for a storage error.

## Plugins

Add your own functions and write hooks to the engine by installing a
plugin.

```rust
use fenec_core::plugin::{Plugin, Registry, ScalarFn};

struct Shout;
impl ScalarFn for Shout {
    fn call(&self, args: &[Value]) -> Result<Value> {
        Ok(Value::Text(format!("{}!", args[0].as_text().unwrap_or(""))))
    }
}

struct MyPlugin;
impl Plugin for MyPlugin {
    fn name(&self) -> &str { "mine" }
    fn init(&self, reg: &mut Registry) -> Result<()> {
        reg.register_fn("shout", Arc::new(Shout))
    }
}

db.install_plugin(&MyPlugin)?;   // get t select shout(name)
```

Details

The registry takes after PostgreSQL's extension model. Scalar functions
and write hooks in the shape of triggers can be registered; a plugin is
installed once, and a second install of the same name is refused.
`fenec-server --follow` guards its mirrored collection with such a
hook.

## Module map

Where each part of the engine lives in `fenec-core`.

Details

| `store` | Segments and the offset index |

| `engine` | `Database`, `Collection`, replay, snapshot, compact, checkpoint |

| `vector` | HNSW and the distance kernels |

| `query` | `Statement` and plan execution |

| `schema` `value` `codec` `json` | Types, the binary codec, the JSON parser |

| `time` | Calendar arithmetic, integer only |

| `changes` | The change ring behind subscriptions |

| `plugin` | The registry |

| `fs` | Buffered file I/O, behind `std-fs` |


---

Source: https://fenecdb.com/docs/compare

# Compare

fenecdb beside the databases it is most often weighed
against: when to choose each, feature by feature.

Each page starts with when to choose the other database, then a table
in which every claim about the other links its source, the numbers we
measured with their method, and how to move across.

- [PostgreSQL](https://fenecdb.com/docs/vs-postgres)

A relational server, measured on YCSB, and with pgvector at a million vectors.

- [SQLite](https://fenecdb.com/docs/vs-sqlite)

With sqlite-vec, FTS5 and the official WebAssembly build.

- [PGlite](https://fenecdb.com/docs/vs-pglite)

PostgreSQL in the browser with pgvector, and what each downloads.

Details

A claim about another product says where it comes from, and the date
on each page is when its facts were checked. A number is on these pages
only where it was measured, on the machine and by the method its page
gives; where nothing was measured, the page says so. A correction is
welcome as an issue in the
[repository](https://github.com/fenecdb/fenec/issues).


---

Source: https://fenecdb.com/docs/vs-postgres

# fenecdb and PostgreSQL

PostgreSQL is a relational database server, and pgvector
adds vector types and indexes to it. fenecdb is a document database that
runs in a page, in an app, or as a server over HTTP, with full-text and
vector search built in.

The facts about PostgreSQL and pgvector on this page were checked on
2026-10-02 against the pages each one links to. fenecdb is at version 0.1.8,
and is alpha software.

## Which to choose

**Choose PostgreSQL with pgvector when**

- **You want SQL:** joins, subqueries, window functions, and every
    ORM, driver and tool written for it.

- **Several clients write at once,** in transactions they hold open
    across statements ([MVCC](https://www.postgresql.org/docs/current/mvcc.html)).

- **You need what a mature server has:**
    [row-level security](https://www.postgresql.org/docs/current/ddl-rowsecurity.html),
    [logical replication](https://www.postgresql.org/docs/current/logical-replication.html),
    extensions such as [PostGIS](https://postgis.net/), and
    managed services from many providers.

- **Your vectors already live beside relational data** you query
    together.

**Choose fenecdb when**

- **The same database runs in the browser, in a mobile app and on the
    server,** with one query language.

- **The client keeps a replica** that reads offline and redraws
    live ([Sync](https://fenecdb.com/docs/sync)).

- **Reads and writes over HTTP with nothing in front:** on YCSB
    fenec-server leads PostgreSQL in every workload at 4 and 16 clients,
    both in Docker ([measured below](#measured)).

- **A file per tenant** suits your service
    ([Tenants and sharding](https://fenecdb.com/docs/sharding)).

- **You search by words, with typos, highlights and facets, and by
    meaning too:** BM25 and rank fusion are built in
    ([Hybrid search](https://fenecdb.com/docs/fenecql#hybrid)).

- **Search latency at a million vectors matters**
    ([measured below](#measured)).

## Feature by feature

|   | fenecdb | PostgreSQL + pgvector |

| Where it runs | Browser (WebAssembly), Rust, iOS, Android, Flutter, a server over HTTP ([Where it runs](https://fenecdb.com/docs/./#where-it-runs)) | A server; in the browser and Node through [PGlite](https://pglite.dev/docs/about) ([fenecdb and PGlite](https://fenecdb.com/docs/vs-pglite)) |

| Data model | Typed documents in collections, `json` fields ([Objects and paths](https://fenecdb.com/docs/fenecql#json)) | Relational tables, plus [`jsonb`](https://www.postgresql.org/docs/current/datatype-json.html) |

| Query language | FenecQL; builders in JS, Python, Go, .NET, Swift, Kotlin, Dart ([FenecQL](https://fenecdb.com/docs/fenecql)) | [SQL](https://www.postgresql.org/docs/current/sql.html) |

| Clients | JS/TS, Python, Go, .NET, Swift, Kotlin, Dart; any language over HTTP ([Your language](https://fenecdb.com/docs/languages)) | Drivers in every major language; pgvector lists libraries for 40 or more ([pgvector](https://github.com/pgvector/pgvector)) |

| Transactions | One writer a file; a batch lands whole or not at all; no transaction held open between requests ([Single writer](https://fenecdb.com/docs/concepts#single-writer)) | ACID, MVCC, many writers, interactive transactions ([MVCC](https://www.postgresql.org/docs/current/mvcc.html)) |

| Replication | Read replicas over HTTP, archives restored to a moment ([Replication](https://fenecdb.com/docs/replication)) | [Streaming](https://www.postgresql.org/docs/current/warm-standby.html) and [logical](https://www.postgresql.org/docs/current/logical-replication.html) replication |

| Sync, live queries, offline | A replica in the page or app: offline reads, queued writes, live queries ([Sync](https://fenecdb.com/docs/sync)) | Not in core; [`LISTEN`/`NOTIFY`](https://www.postgresql.org/docs/current/sql-notify.html) and [logical decoding](https://www.postgresql.org/docs/current/logicaldecoding.html) |

| Access control | A server token; JWT rules added to every statement ([Scoped access](https://fenecdb.com/docs/http#scoped-access)). No TLS: a terminator goes in front | Roles, `GRANT`, [row-level security](https://www.postgresql.org/docs/current/ddl-rowsecurity.html) |

| Schema and migrations | Typed schema, `alter collection`, a schema declared in code and checked at every open ([Schema in code](https://fenecdb.com/docs/javascript#schema)) | Transactional DDL ([`ALTER TABLE`](https://www.postgresql.org/docs/current/sql-altertable.html)); migrations by outside tools |

| Full text | BM25 ([The text clause](https://fenecdb.com/docs/fenecql#the-text-clause)) | `tsvector` ranked by [`ts_rank`](https://www.postgresql.org/docs/current/textsearch-controls.html); BM25 through an extension such as [pg_textsearch](https://github.com/timescale/pg_textsearch) |

| Highlights and facets | `highlight()`, `snippet()` and `facet` ([Highlights](https://fenecdb.com/docs/fenecql#highlights), [Facets](https://fenecdb.com/docs/fenecql#facets)) | [`ts_headline`](https://www.postgresql.org/docs/current/textsearch-controls.html#TEXTSEARCH-HEADLINE) over a `tsquery`; a facet is a `GROUP BY` of its own |

| Vector index | HNSW ([The vector clause](https://fenecdb.com/docs/fenecql#the-vector-clause)) | HNSW and IVFFlat ([pgvector](https://github.com/pgvector/pgvector)) |

| Quantization | `f16` storage; `int8` and `bit` codes, re-ordered by the full vectors ([Quantized indexes](https://fenecdb.com/docs/fenecql#quantized)) | `halfvec`, `bit`, binary quantization by expression index ([pgvector](https://github.com/pgvector/pgvector)) |

| Filtered vector search | Planned with the filter, an exact fallback when the filter leaves too few ([A filter and a vector together](https://fenecdb.com/docs/concepts#a-filter-and-a-vector-together)) | Iterative index scans since 0.8.0 ([release](https://www.postgresql.org/about/news/pgvector-080-released-2952)) |

| Sparse vectors | `sparse<N>`, pgvector's text form ([Sparse vectors](https://fenecdb.com/docs/fenecql#sparse)) | `sparsevec` ([pgvector](https://github.com/pgvector/pgvector)) |

| Hybrid search | `fuse`, reciprocal rank fusion ([Hybrid search](https://fenecdb.com/docs/fenecql#hybrid)) | No operator; fusion written in SQL ([an example](https://www.tigerdata.com/blog/introducing-pg_textsearch-true-bm25-ranking-hybrid-retrieval-postgres)) |

| Size | A 2.99 MB server image ([Binary size](https://fenecdb.com/docs/benchmarks#binary-size)) | Not measured here |

| Licence and hosting | Apache-2.0; self-hosted only | [PostgreSQL Licence](https://www.postgresql.org/about/licence/) ([pgvector's](https://github.com/pgvector/pgvector/blob/master/LICENSE)); managed by many providers |

| Maturity | Version 0.1.8, first released in 2026 | Descends from POSTGRES, 1986 ([history](https://www.postgresql.org/docs/current/history.html)); pgvector 0.8.7 ([README](https://github.com/pgvector/pgvector)) |

Details

What fenecdb gives up for its size is listed in Limits: no SQL dialect ([its everyday spellings](https://fenecdb.com/docs/sql) aside) and a join only on a key (a query reads one
collection and attaches related rows with `lookup`), one writer
a file, no transaction held open across requests, and no TLS on the
server. PostgreSQL's server has none of those limits.

pgvector's HNSW indexes take `vector` up to 2 000
dimensions, `halfvec` up to 4 000, `bit` up to
64 000 and `sparsevec` up to 1 000 non-zero elements
([README](https://github.com/pgvector/pgvector)). pgvectorscale
adds a disk-based index, StreamingDiskANN, for data larger than memory
([pgvectorscale](https://github.com/timescale/pgvectorscale)).
fenecdb keeps its indexes in memory and its documents in a mapped file
([Memory and scale](https://fenecdb.com/docs/limits#memory-and-scale)).

## Measured

YCSB's core workloads over a million records, fenec-server over HTTP
against PostgreSQL 17, both in Docker on the Mac, durable, operations a
second, the median of three runs ([every
cell, and MongoDB](https://fenecdb.com/docs/benchmarks#ycsb)). We measured these on one laptop; the method is
below.

| YCSB, 1 000 000 records | fenec-server, HTTP | PostgreSQL |

| B, 95% reads, 16 clients | **18.2 k** | 13.0 k |

| A, 50% updates, 16 clients | **13.3 k** | 10.5 k |

| A, 50% updates, one client | **3.34 k** | 2.52 k |

| B, 95% reads, one client | **4.50 k** | 3.09 k |

| E, scans of 1 to 100 rows, one client | **1.69 k** | 1.28 k |

| C, reads alone, one client, buffered | 3.07 k | **3.14 k** |

- **Both in Docker,** because the VM's fsync takes a tenth of a
    millisecond where the Mac's `F_FULLFSYNC` takes four, and
    its network a quarter of a millisecond a request: fenec-server on the
    Mac against PostgreSQL in Docker would compare those, not the two.

- **C, reads alone, is where PostgreSQL leads,** by 2% at one
    client, and F by 1%: each server's own part of a read is under 0.02 ms
    of a 0.29 ms round trip, and the rest is Docker's network, which moves
    more than that from one day to the next (PostgreSQL's C at one client
    did 3.66 k on 2026-10-05, its median now 3.14 k). At 4 and 16 clients
    fenec-server leads every workload; at 16 fenec-server leads C,
    18.4 k against 14.8 k.

Then search by meaning: a million 128-dim vectors, fenec-server asked
over HTTP and PostgreSQL with pgvector over its own protocol, by the same
client program with the same index settings.

| 1 000 000 × 128 | fenec-server, HTTP | PostgreSQL + pgvector |

| load and index | **49.6 s** | 108.5 s |

| memory the server holds | **738 MB** | 1 131 MB |

| disk | **612 MB** | 1 432 MB |

| ef 100: recall@10, p50 | **99.1%, 0.141 ms** | 98.6%, 2.076 ms |

| a filter keeping 1%, ef 100 | **100%, 0.667 ms** | 99.0%, 16.1 ms |

| 8 clients, ef 100 | **18 852 queries/s** | 2 289 queries/s |

| load 100 000 rows, HNSW kept (`POST /docs` against `COPY`) | **19 151 rows/s** | 410 rows/s |

- **One machine, one run of each side.** An Apple M1 MacBook Air;
    PostgreSQL ran in Docker.

- **Every figure, and the 768-dim run,** is in
    [Benchmarks](https://fenecdb.com/docs/benchmarks#scale).

Method and details

YCSB is `make ycsb`'s, every cell of it in Everyday reads and writes with its method:
each cell 30 s after a 5 s warm-up, the systems in turns, PostgreSQL with
`shared_buffers` 1 GB and prepared statements. The vectors are
`make scale-bench`'s and `make load-bench`'s, as At scale and Loading rows report them.

```bash
make ycsb
make pgvector-up      # PostgreSQL 17 with pgvector, the pgvector/pgvector:pg17 image
make scale-bench
make load-bench
```

- **Hardware.** An Apple M1 MacBook Air. It has no fan and slows
    under minutes of load, so for the vectors each side was run on its own
    after four idle minutes, once.

- **Where each ran, for the vectors.** `fenec-server` on the host,
    asked over kept-alive HTTP connections. PostgreSQL in Docker's virtual
    machine (4 GB, 8 CPUs, `shared_buffers` 1 GB), asked through
    the `postgres` crate. An empty query's round trip was 0.13 to
    0.22 ms to PostgreSQL and 0.06 to 0.11 ms to fenec-server: part of each
    answer, not the rest of it.

- **Same index settings.** HNSW with m 16 and ef_construction 64,
    pgvector's defaults, on both.

- **Each side's best way to load.** fenec-server builds its graph
    as JSON arrays of rows arrive. pgvector builds after a binary
    `COPY`, on every core, then reads its index into its buffers
    (`pg_prewarm`) and searches a filter with
    `hnsw.iterative_scan = relaxed_order`.

- **Data.** Vectors spread along 32 directions around 64 centres,
    1 000 held-out queries, recall against the exact ten found by brute
    force.

**Not measured yet:** a Linux machine, PostgreSQL outside Docker,
data larger than memory and pgvectorscale. Several clients writing at once
are [YCSB's](https://fenecdb.com/docs/benchmarks#ycsb).

## Moving from PostgreSQL

`fenec import` copies a table into a collection over the
network, vectors and index included. With `--follow` it keeps
applying the table's changes as they commit.

```bash
fenec import postgres://app@db:5432/app --table docs --into articles \
    --index "embed@hnsw(cosine)" --follow
```

- **pgvector's types come across:** `vector`,
    `halfvec` and `sparsevec` keep their dimensions.
    A `numeric` needs `--cast`
    ([Type mapping](https://fenecdb.com/docs/import#type-mapping)).

- **Following needs** `wal_level = logical` and an integer
    key. `fenec-server --follow` serves the mirror as it
    follows ([Serving the mirror](https://fenecdb.com/docs/import#serving)).

- **Queries are rewritten.** fenecdb does not speak SQL or
    PostgreSQL's protocol, so a pgvector client library cannot connect to
    it.

The same query in each, and then a nearest-neighbour search:

```sql
SELECT title FROM articles
WHERE year >= 2024
ORDER BY published DESC
LIMIT 10;
```

```fenecql
get articles select title
  where year >= 2024
  order published desc
  limit 10
```

```sql
SELECT title FROM articles
WHERE year >= 2024
ORDER BY embed <=> $1
LIMIT 10;
```

```fenecql
get articles select title
  where year >= 2024
  near embed $1
  limit 10
```

Details

A join becomes `lookup`, which attaches each row's related
rows, or `in (get …)`, which filters by another collection's
rows ([The lookup clause](https://fenecdb.com/docs/fenecql#the-lookup-clause),
[Subqueries](https://fenecdb.com/docs/fenecql#subqueries)). A transaction of several
statements becomes a `/batch`, which lands whole or not at all
but cannot read between its writes ([Batches](https://fenecdb.com/docs/http#batches)).
Row-level security becomes a JWT rule
([Scoped access](https://fenecdb.com/docs/http#scoped-access)).

The import reads the table with `COPY ... TO STDOUT` and
needs nothing installed on the server, and the follower reads a logical
replication slot through PostgreSQL's own `pgoutput`. Drop the
slot when you stop following, or PostgreSQL keeps its log for it
([Following changes](https://fenecdb.com/docs/import#following)).


---

Source: https://fenecdb.com/docs/vs-sqlite

# fenecdb and SQLite

SQLite is an embedded SQL database in one file; FTS5 adds
full-text search to it and sqlite-vec vector search. fenecdb is an embedded
document database, which also runs as a server over HTTP, with full-text
and vector search built in.

The facts about SQLite, sqlite-vec and FTS5 on this page were checked on
2026-10-02 against the pages each one links to. fenecdb is at version 0.1.8,
and is alpha software.

## Which to choose

**Choose SQLite when**

- **You want the database that is already everywhere:** SQLite is
    "likely used more than all other database engines combined"
    ([sqlite.org](https://sqlite.org/mostdeployed.html)), with
    bindings, ORMs and tools in every language.

- **You want SQL** with joins, and transactions you hold open
    across statements, with savepoints
    ([transactions](https://sqlite.org/lang_transaction.html)).

- **Readers must never wait for the writer:** in WAL mode they do
    not ([WAL](https://sqlite.org/wal.html)).

- **You need a long track record:** 100% branch coverage in its
    tests ([testing](https://sqlite.org/testing.html)), and an
    on-disk format kept backwards compatible, supported through 2050
    ([long-term support](https://sqlite.org/lts.html)).

- **Exact vector search over a modest table** is enough, through
    sqlite-vec.

**Choose fenecdb when**

- **The page or app should stay in step with a server:** a replica
    with live queries and offline writes ([Sync](https://fenecdb.com/docs/sync)).

- **The same file should also be served** to other languages over
    HTTP, with replicas ([Server](https://fenecdb.com/docs/server)).

- **Many threads write at once:** durable writes share their disk
    syncs, 516 a second from 16 writers against SQLite's 258, and readers
    keep reading beside them ([measured below](#measured)).

- **A search box needs typos and facets** in the query that ranks
    the words ([Typos and words
    begun](https://fenecdb.com/docs/fenecql#typos), [Facets](https://fenecdb.com/docs/fenecql#facets)).

- **You want an approximate vector index:** HNSW with
    quantization and filtered search
    ([The vector clause](https://fenecdb.com/docs/fenecql#the-vector-clause)).
    sqlite-vec's approximate indexes are in its alpha releases.

- **You search by meaning and by words in one query,** fused by
    rank ([Hybrid search](https://fenecdb.com/docs/fenecql#hybrid)).

## Feature by feature

|   | fenecdb | SQLite + sqlite-vec + FTS5 |

| Where it runs | Browser (WebAssembly), Rust, iOS, Android, Flutter, a server over HTTP ([Where it runs](https://fenecdb.com/docs/./#where-it-runs)) | Everywhere a C library runs; the browser through the official WebAssembly build since 3.40.0 ([release](https://sqlite.org/releaselog/3_40_0.html)). No server of its own |

| Browser storage | IndexedDB, or a file of the origin private file system in a worker ([Keeping a database](https://fenecdb.com/docs/javascript#keeping)) | `kvvfs` (localStorage), `opfs` (a worker and COOP/COEP headers), `opfs-sahpool` ([persistence](https://sqlite.org/wasm/doc/trunk/persistence.md)) |

| Data model | Typed documents in collections, `json` fields ([Objects and paths](https://fenecdb.com/docs/fenecql#json)) | Relational tables |

| Query language | FenecQL; builders in JS, Python, Go, .NET, Swift, Kotlin, Dart ([FenecQL](https://fenecdb.com/docs/fenecql)) | SQL |

| Clients | JS/TS, Python, Go, .NET, Swift, Kotlin, Dart; any language over HTTP ([Your language](https://fenecdb.com/docs/languages)) | Bindings in nearly every language; sqlite-vec packages for Python, Node.js, Ruby, Go and Rust ([README](https://github.com/asg017/sqlite-vec)) |

| Transactions | One writer a file; a batch lands whole or not at all; no transaction held open across statements ([Single writer](https://fenecdb.com/docs/concepts#single-writer)) | `BEGIN` … `COMMIT`, savepoints ([transactions](https://sqlite.org/lang_transaction.html)); one writer at a time, readers beside it in WAL mode ([WAL](https://sqlite.org/wal.html)) |

| Sync, live queries, offline | A replica in the page or app: offline reads, queued writes, live queries ([Sync](https://fenecdb.com/docs/sync)) | No sync; the session extension records changesets to apply to another database ([sessions](https://sqlite.org/sessionintro.html)) |

| Access control | On the server: a token, JWT rules added to every statement ([Scoped access](https://fenecdb.com/docs/http#scoped-access)) | Embedded: none of its own |

| Schema and migrations | Typed schema, `alter collection`, a schema declared in code and checked at every open ([Schema in code](https://fenecdb.com/docs/javascript#schema)) | SQL DDL; migrations by your own code or tools |

| Full text | BM25 ([The text clause](https://fenecdb.com/docs/fenecql#the-text-clause)) | FTS5 with `bm25()`; unicode61, ascii, porter and trigram tokenizers ([FTS5](https://sqlite.org/fts5.html)) |

| Highlights and snippets | `highlight()` and `snippet()`, as UTF-16 offsets or between tags, by the index's own tokenizer ([Highlights and snippets](https://fenecdb.com/docs/fenecql#highlights)) | FTS5's `highlight()` and `snippet()`, the marked text between tags ([FTS5](https://sqlite.org/fts5.html)) |

| Facets | `facet`, each value's count over every match, beside the page ([Facets](https://fenecdb.com/docs/fenecql#facets)) | None built in; a `GROUP BY` a facet, a query each |

| Vector index | HNSW ([The vector clause](https://fenecdb.com/docs/fenecql#the-vector-clause)) | sqlite-vec's `vec0`; ANN indexes (rescore, DiskANN) in the v0.1.10 alpha releases, "pre-v1, so expect breaking changes" ([releases](https://github.com/asg017/sqlite-vec/releases), [README](https://github.com/asg017/sqlite-vec)) |

| Quantization | `f16` storage; `int8` and `bit` codes, re-ordered by the full vectors ([Quantized indexes](https://fenecdb.com/docs/fenecql#quantized)) | float, int8 and binary vectors ([README](https://github.com/asg017/sqlite-vec)) |

| Filtered vector search | Planned with the filter, an exact fallback when the filter leaves too few ([A filter and a vector together](https://fenecdb.com/docs/concepts#a-filter-and-a-vector-together)) | Up to 16 metadata columns and 4 partition keys, filtered inside the KNN query ([vec0](https://alexgarcia.xyz/sqlite-vec/features/vec0.html)) |

| Hybrid search | `fuse`, reciprocal rank fusion ([Hybrid search](https://fenecdb.com/docs/fenecql#hybrid)) | None built in; FTS5's page has no vector search ([FTS5](https://sqlite.org/fts5.html)) |

| Browser module, brotli | 273 KB ([below](#size)) | 341 KB, without sqlite-vec ([below](#size)) |

| Licence | Apache-2.0 | Public domain ([copyright](https://sqlite.org/copyright.html)); sqlite-vec Apache-2.0 or MIT ([README](https://github.com/asg017/sqlite-vec)) |

| Maturity | Version 0.1.8, first released in 2026 | Supported through 2050 ([long-term support](https://sqlite.org/lts.html)); sqlite-vec pre-v1 |

Details

Of the official build's browser storage, `opfs-sahpool`
needs no COOP/COEP headers but "does not support multiple simultaneous
connections to the same database"
([persistence](https://sqlite.org/wasm/doc/trunk/persistence.md)).
fenecdb's `openFile` also holds its file for one worker; other
tabs reach it through the one that holds it, or share a synced replica
([Multiple tabs](https://fenecdb.com/docs/sync#multiple-tabs-share-one-stream)).

FTS5's `bm25()` has its constants fixed at k1 = 1.2 and
b = 0.75 ([FTS5](https://sqlite.org/fts5.html)). fenecdb's
`@text` also splits Chinese, Japanese, Korean and Thai text into
character runs and can index word prefixes
([The text clause](https://fenecdb.com/docs/fenecql#the-text-clause)).

sqlite-vec's first stable release, v0.1.0, searched by brute force
([announcement](https://alexgarcia.xyz/blog/2024/sqlite-vec-stable-release/index.html)).
Its ANN indexes arrived in v0.1.10-alpha.1 on 2026-03-31: rescore and
DiskANN, with IVF "experimental, not enabled"; the latest is
v0.1.10-alpha.4, from 2026-05-18
([releases](https://github.com/asg017/sqlite-vec/releases)).

## Measured

YCSB's core workloads over a million records, fenecdb and SQLite in
one process, operations a second, the median of three runs (every cell). We measured these on one
laptop; the method is below.

| YCSB, 1 000 000 records | fenecdb | SQLite |

| B, 95% reads, 16 threads, buffered | **545 k** | 47.9 k |

| A, 50% updates, 16 threads, durable | **3.22 k** | 0.45 k |

| C, reads, one thread, after the updates | **438 k** | 279 k |

Writers and readers at once, a row a write:

|   | fenecdb | SQLite |

| durable writes/s, 1 / 4 / 16 writers | 252 / 487 / **516** | 248 / 248 / 258 |

| reads/s from 4 threads, beside 4 writers | **701 k**, p99 75 µs | 344 k, p99 79 µs |

| … beside blocks of 1 000 writes | **2 010 k**, p99 2.5 µs, the longest 0.5–1.2 ms | 630 k, p99 59 µs, the longest 0.4–1.9 ms |

And 100 000 rows of 128 dimensions, SQLite with no vector extension:
its rows are read back and scored by fenecdb's own code.

| 100 000 × 128 | fenecdb | SQLite |

| data write, no index | **936 k rows/s** | 430 k/s |

| disk size | **59.3 MB** | 59.8 MB |

| scalar filter, indexed | **1.25 ms** | 14.1 ms |

| vector top-10, exact | **2.9 ms** | 30.2 ms |

| vector top-10, approximate | **0.091 ms** | not measured |

| open, then a read by id | 6.0 ms | **0.46 ms** |

| open, then a vector top-10 | **6.1 ms** | 30.5 ms, exact |

- **fenecdb writes a record again at its file's end on every
    update**, where SQLite updates its page in place; the file compacts
    itself once half of it is dead, beside the reads. Before it did, C at
    one thread read 30.1 k after the updates, where SQLite read 279 k.

- **SQLite opens faster**, and its readers never wait for a block of
    writes. fenecdb's wait only while one lands: its rows are built and
    encoded beside them first, and the block holds the lock about 0.15 ms
    for 1 000 rows.

- **Not measured yet:** sqlite-vec, by brute force or with its
    alpha indexes; FTS5; and either in the browser.

Method and details

The numbers are `make ycsb`'s, `make
concurrency-bench`'s and `make compare`'s, as Everyday reads and writes, Writers and readers at once and Against SQLite and
pgvector report them, on an Apple M-series laptop.

```bash
make ycsb
make concurrency-bench
make compare
```

- **YCSB**: SQLite 3.53 through `rusqlite`, a connection
    a thread, WAL, the file mapped; durable is
    `synchronous=FULL` with `fullfsync`, buffered its
    default `synchronous=NORMAL`. Each cell 30 s after a 5 s
    warm-up.

- **The 100 000 rows**: SQLite 3.53, in WAL mode with
    `synchronous=NORMAL`, the rows written in one transaction and
    the index built afterwards; the median of three runs.

- **SQLite had no vector index.** Its exact search reads every row
    and scores it with the same kernel fenecdb uses, so the difference is
    in reading the rows, not in the arithmetic.

- **Writers and readers** ran in one process, after idle minutes,
    once: fenecdb's threads share one database behind a lock, SQLite's hold
    a connection each with a 30 s busy timeout. Durable is an fsync a
    commit, `F_FULLFSYNC` for both.

## What the browser downloads

The WebAssembly engine alone, as each project publishes it, compressed
the way a static file is served.

|   | brotli | gzip |

| fenecdb's module, `fenec.wasm` 0.1.8 | **273 KB** | **330 KB** |

| SQLite's official build, `sqlite3.wasm` 3.53.4-build2 | 341 KB | 393 KB |

Method and details

Measured on 2026-10-02: fenecdb's module as `make wasm` builds
it, and `dist/sqlite3.wasm` from the npm package
`@sqlite.org/sqlite-wasm` 3.53.4-build2, each compressed with
`brotli -q 11` and `gzip -9`. A KB is 1 024 bytes.
The JavaScript beside each is left out of both. sqlite-vec's WebAssembly
size is not published, and is not in the figure.

## Moving from SQLite

`fenec import` reads a SQLite file directly, with no SQLite
library installed, and makes a collection of a table.

```bash
fenec import data.sqlite --table docs --into articles \
    --vector embed:384 --index "embed@hnsw(cosine)" --dry-run
```

- **`--dry-run`** prints the schema it would make and
    writes nothing.

- **Vectors stored as BLOBs** of little-endian `f32`s
    become `vector<N>` with `--vector`.

- **`INTEGER PRIMARY KEY`** becomes the document id.

A query in each, and then a search by meaning and its filter:

```sql
SELECT title FROM articles
WHERE year >= 2024
ORDER BY published DESC
LIMIT 10;
```

```fenecql
get articles select title
  where year >= 2024
  order published desc
  limit 10
```

```sql
-- sqlite-vec, year a metadata column of the vec0 table
SELECT rowid, distance FROM vec_articles
WHERE embed MATCH ?1 AND k = 10 AND year >= 2024;
```

```fenecql
get articles select title
  where year >= 2024
  near embed $1
  limit 10
```

Details

The import reads the file format itself, and resolves untyped columns
from the first 1 000 rows. A table with no equivalent type is refused until
`--cast` says what to do ([Type
mapping](https://fenecdb.com/docs/import#type-mapping)). A join
becomes `lookup` or `in (get …)`
([The lookup clause](https://fenecdb.com/docs/fenecql#the-lookup-clause)), and a
transaction becomes a batch that lands whole
([Batches](https://fenecdb.com/docs/http#batches)).


---

Source: https://fenecdb.com/docs/vs-pglite

# fenecdb and PGlite

PGlite is PostgreSQL compiled to WebAssembly, run in the
browser, Node.js and Bun, with extensions such as pgvector. fenecdb is a
document database built small for the page, which also runs in apps and as
a server, with full-text and vector search built in.

The facts about PGlite on this page were checked on 2026-10-02 against
the pages each one links to, and its sizes measured from version 0.5.8.
fenecdb is at version 0.1.8, and is alpha software.

## Which to choose

**Choose PGlite when**

- **You want PostgreSQL itself in the page:** its SQL, joins,
    types and interactive transactions
    ([API](https://pglite.dev/docs/api)).

- **You want its extensions:** pgvector, pg_textsearch, pg_trgm,
    PostGIS and others ([extensions](https://pglite.dev/extensions/)).

- **The page and the server should share schema and queries**
    with a PostgreSQL server, or an ORM such as Drizzle
    ([Drizzle](https://orm.drizzle.team/docs/connect-pglite)).

- **You want live queries that send only what changed**
    ([`live.incrementalQuery`, `live.changes`](https://pglite.dev/docs/live-queries)).

**Choose fenecdb when**

- **Download size matters:** its module is about a twentieth of
    PGlite's files ([below](#size)).

- **The page writes while offline and syncs back** to a server,
    which PGlite's sync does not do yet ([Sync](https://fenecdb.com/docs/sync)).

- **You keep the database in a file in Safari too:** fenecdb's file
    of the origin private file system works there
    ([Keeping a database](https://fenecdb.com/docs/javascript#keeping)).

- **The same database runs in an iOS, Android or Flutter app**
    ([Mobile and native apps](https://fenecdb.com/docs/mobile)).

- **You search with nothing to install:** BM25 with typos,
    highlights and facets, and by meaning too, the two fused by rank
    ([Typos and words begun](https://fenecdb.com/docs/fenecql#typos),
    [Hybrid search](https://fenecdb.com/docs/fenecql#hybrid)).

## Feature by feature

|   | fenecdb | PGlite + pgvector |

| Where it runs | Browser, Node, Cloudflare Workers, Rust, iOS, Android, Flutter, a server over HTTP ([Where it runs](https://fenecdb.com/docs/./#where-it-runs)) | Browser, Node.js and Bun ([about](https://pglite.dev/docs/about)) |

| Browser storage | IndexedDB, or a file of the origin private file system in a worker, Safari included ([Keeping a database](https://fenecdb.com/docs/javascript#keeping)) | IndexedDB, recommended; an OPFS file system in a worker that does not work in Safari ([file systems](https://pglite.dev/docs/filesystems)) |

| Data model | Typed documents in collections, `json` fields ([Objects and paths](https://fenecdb.com/docs/fenecql#json)) | PostgreSQL's tables and types |

| Query language | FenecQL and a typed query builder ([The query builder](https://fenecdb.com/docs/javascript#the-query-builder)) | PostgreSQL's SQL, `$1` parameters ([API](https://pglite.dev/docs/api)) |

| Clients | JS/TS, React; Swift, Kotlin, Dart in apps ([Your language](https://fenecdb.com/docs/languages)) | TypeScript and JavaScript, with React and Vue bindings ([about](https://pglite.dev/docs/about)) |

| Transactions | One writer; a `run` of several statements lands whole or not at all ([Single writer](https://fenecdb.com/docs/concepts#single-writer)) | Interactive transactions, `.transaction()` ([API](https://pglite.dev/docs/api)); one connection, tabs share it through a leader ([multi-tab worker](https://pglite.dev/docs/multi-tab-worker)) |

| Live queries | `db.live`: the rows again after each write to a collection the query reads ([State in the page](https://fenecdb.com/docs/javascript#state)) | `live.query`, `live.incrementalQuery`, `live.changes` ([live queries](https://pglite.dev/docs/live-queries)) |

| Sync and offline | A replica of a fenecdb server: offline reads, writes queued and sent back ([Sync](https://fenecdb.com/docs/sync)) | Electric sync, alpha, server to page only: "We don't yet support local writes being synced out" ([sync](https://pglite.dev/docs/sync)) |

| Schema and migrations | Tables declared in code, Drizzle's way, checked at every open ([Schema in code](https://fenecdb.com/docs/javascript#schema)) | PostgreSQL's DDL; Drizzle and other ORMs ([Drizzle](https://orm.drizzle.team/docs/connect-pglite)) |

| Full text | BM25, built in ([The text clause](https://fenecdb.com/docs/fenecql#the-text-clause)) | BM25 through the pg_textsearch extension ([extensions](https://pglite.dev/extensions/)) |

| Highlights and facets | `highlight()`, `snippet()` and `facet` ([Highlights](https://fenecdb.com/docs/fenecql#highlights), [Facets](https://fenecdb.com/docs/fenecql#facets)) | PostgreSQL's `ts_headline` over a `tsquery`; a facet is a `GROUP BY` of its own |

| Vector index | HNSW, built in ([The vector clause](https://fenecdb.com/docs/fenecql#the-vector-clause)) | pgvector's HNSW and IVFFlat, an extension ([extensions](https://pglite.dev/extensions/), [pgvector](https://github.com/pgvector/pgvector)) |

| Quantization | `f16` storage; `int8` and `bit` codes, re-ordered by the full vectors ([Quantized indexes](https://fenecdb.com/docs/fenecql#quantized)) | `halfvec`, `bit`, binary quantization by expression index ([pgvector](https://github.com/pgvector/pgvector)) |

| Filtered vector search | Planned with the filter, an exact fallback when the filter leaves too few ([A filter and a vector together](https://fenecdb.com/docs/concepts#a-filter-and-a-vector-together)) | pgvector's iterative index scans ([pgvector](https://github.com/pgvector/pgvector)) |

| Hybrid search | `fuse`, reciprocal rank fusion ([Hybrid search](https://fenecdb.com/docs/fenecql#hybrid)) | No operator; written in SQL |

| Download, brotli | 273 KB ([below](#size)) | 3 740 KB, before pgvector ([below](#size)) |

| Licence | Apache-2.0 | Apache-2.0 and the PostgreSQL Licence ([about](https://pglite.dev/docs/about)) |

| Maturity | Version 0.1.8, first released in 2026 | Version 0.5.8 ([releases](https://github.com/electric-sql/pglite/releases)), on PostgreSQL |

Details

PGlite's IndexedDB file system loads every file of the database into
memory at the start and flushes them after each query that changed them;
its OPFS file system needs a worker, and Safari's limit of 252 open sync
access handles stops it working there, "due to a standard Postgres install
consisting of over 300 files"
([file systems](https://pglite.dev/docs/filesystems)). fenecdb's
`persist` writes the image once and then only the writes since,
and `openFile` keeps one file
([Keeping a database](https://fenecdb.com/docs/javascript#keeping)).

PGlite is "single connection only"; tabs elect a leader that runs the
database and the others reach it through the leader
([multi-tab worker](https://pglite.dev/docs/multi-tab-worker)).
A synced fenecdb replica elects a leader among tabs as well
([Multiple tabs](https://fenecdb.com/docs/sync#multiple-tabs-share-one-stream)).

What fenecdb leaves out to stay small is in Limits: no SQL dialect ([its everyday spellings](https://fenecdb.com/docs/sql) aside) and a join only on a key, one writer, and no
transaction held open while the page reads between its writes.

## What the browser downloads

The WebAssembly and data files each needs before its first query, as
each project publishes them, compressed the way a static file is served.
PGlite's files are about 21 times fenecdb's module.

|   | brotli | gzip |

| fenecdb's module, `fenec.wasm` 0.1.8 | **273 KB** | **330 KB** |

| PGlite 0.5.8: `pglite.wasm` | 2 585 KB | 3 310 KB |

| … `pglite.data` | 1 038 KB | 1 816 KB |

| … `initdb.wasm` | 117 KB | 141 KB |

| PGlite 0.5.8, the three | 3 740 KB | 5 267 KB |

- **PGlite's own figure** is "under 3MB gzipped"
    ([about](https://pglite.dev/docs/about)); the files of
    0.5.8 measure more.

- **pgvector adds** a 45 KB archive, gzipped as published.

Method and details

Measured on 2026-10-02. fenecdb's module as `make wasm`
builds it; PGlite's files from the npm package
`@electric-sql/pglite` 0.5.8, whose client starts downloading
all three when a database opens. Each file compressed with `brotli
-q 11` and `gzip -9`; a KB is 1 024 bytes. The JavaScript
beside each is left out of both. pgvector is
`dist/vector.tar.gz` of
`@electric-sql/pglite-pgvector` 0.0.9, 46 549 bytes; PGlite's
extensions page gives 42.9 KB
([extensions](https://pglite.dev/extensions/)).

**Not measured yet:** the time to start, load and search in either,
and the memory each holds in the page.

## Moving from PGlite

A table becomes a collection, a query becomes FenecQL or a builder
chain, and parameters stay `$1`, `$2`. Opening a
database kept in IndexedDB, with a vector and a text index:

```js
// PGlite
import { PGlite } from '@electric-sql/pglite';
import { vector } from '@electric-sql/pglite-pgvector';

const pg = new PGlite('idb://notes', { extensions: { vector } });
await pg.exec(`
  CREATE EXTENSION IF NOT EXISTS vector;
  CREATE TABLE IF NOT EXISTS notes (id serial PRIMARY KEY, title text, body text, embed vector(384));
  CREATE INDEX IF NOT EXISTS notes_embed ON notes USING hnsw (embed vector_cosine_ops);
`);
```

```ts
// fenecdb
import { Fenec, restore } from './fenec.js';
import { fenecTable, text, vector, index } from './schema.js';

const notes = fenecTable('notes', {
  title: text(),
  body: text(),
  embed: vector({ dimensions: 384 }),
}, (t) => [
  index('notes_body').using('bm25', t.body),
  index('notes_embed').using('hnsw', t.embed.op('vector_cosine_ops')),
]);

const db = await Fenec.open('./fenec.wasm', { schema: { notes } });
await restore(db, 'notes');   // what persist(db, 'notes') kept, if anything
```

A live query:

```js
// PGlite, with the live extension
const ret = pg.live.query('SELECT title FROM notes ORDER BY id DESC LIMIT 20', [], (res) => draw(res.rows));
await ret.unsubscribe();
```

```ts
// fenecdb
const stop = db.live(db.from(notes).select('title').order('id', 'desc').limit(20), draw);
stop();
```

A write and a nearest-neighbour search:

```js
// PGlite
await pg.query('INSERT INTO notes (title, body, embed) VALUES ($1, $2, $3)',
  [title, body, JSON.stringify(embedding)]);
const { rows } = await pg.query(
  'SELECT title FROM notes ORDER BY embed <=> $1 LIMIT 5', [JSON.stringify(queryVector)]);
```

```ts
// fenecdb, the builder
await db.from(notes).insert({ title, body, embed: embedding });
const rows = await db.from(notes).select('title').near('embed', queryVector).limit(5).rows();

// or FenecQL
const same = db.rows('get notes select title near embed $1 limit 5', [queryVector]);
```

Words and meaning in one ranking, which in PGlite is a query to write
over pg_textsearch and pgvector:

```ts
const hits = await db.from(notes).select('title')
  .match('body', 'offline sync')
  .near('embed', queryVector)
  .fuse()
  .limit(10)
  .rows();
```

Details

There is no importer from PGlite. Its data can be read out with a
query and written with `insert`, an array at a time, or, from
a PostgreSQL server holding the same tables,
[`fenec import`](https://fenecdb.com/docs/import) reads them over the
network.

What does not carry over: a join becomes `lookup` or
`in (get …)` ([The
lookup clause](https://fenecdb.com/docs/fenecql#the-lookup-clause)), a `.transaction()` that reads between its
writes becomes a `run` of several statements, which lands whole
but cannot branch on what it read, and an extension has no counterpart
unless fenecdb has the feature built in. A `numeric` has no
counterpart either: money is an `int` of cents.


---

Source: https://fenecdb.com/docs/benchmarks

# Benchmarks

How fenecdb compares with SQLite, PostgreSQL, MongoDB and
Redis as a database first, then at search, measured by make targets you can
run yourself.

## At a glance

The headline numbers from the sections below. Each section says how it
was measured.

**Everyday reads and writes, in one process against SQLite:** YCSB's
core workloads over a million records, operations a second, the median of
three runs ([Everyday reads and writes](#ycsb)).

| YCSB, 1 000 000 records | fenecdb | SQLite |

| B, 95% reads, 16 threads, buffered | **545 k** | 47.9 k |

| C, reads, one thread, after the updates | **438 k** | 279 k |

| A, 50% updates, 16 threads, durable | **3.22 k** | 0.45 k |

**The same over the wire:** fenec-server over HTTP against PostgreSQL
and MongoDB, all three in Docker, durable, an fsync a write.

| YCSB, 1 000 000 records | fenec-server | PostgreSQL | MongoDB |

| B, 95% reads, 16 clients | **18.2 k** | 13.0 k | 11.1 k |

| A, 50% updates, 16 clients | **13.3 k** | 10.5 k | 6.56 k |

| A, 50% updates, one client | **3.34 k** | 2.52 k | 1.97 k |

**Many writers, and readers beside them:** one process against
SQLite, a row a write, a row by id a read ([Writers
and readers at once](#concurrency)); then one request to fenec-server over HTTP on
loopback ([A request over HTTP](#requests)).

|   | fenecdb | SQLite |

| durable writes/s, 1 / 16 writers | 252 / **516** | 248 / 258 |

| reads/s from 4 threads, beside 4 writers | **701 k** | 344 k |

| … beside blocks of 1 000 writes | **2 010 k** | 630 k |

| fenec-server over HTTP | p50, one client | 8 clients |

| a row by id | **0.022 ms** | 149 268/s |

| a filter, limit 10 | **0.032 ms** | 107 322/s |

| put a row | **0.022 ms** | 129 476/s |

**Replicas, tenants and sync:** how far behind a copy is, what a
failover and a move take, and what a synced change costs the device
([Replication](https://fenecdb.com/docs/replication#what-it-costs),
[Replicas of tenants](https://fenecdb.com/docs/sharding#replicas-of-tenants),
[a tenant moved](https://fenecdb.com/docs/sharding#what-it-costs),
[sync on a phone](https://fenecdb.com/docs/mobile#sync)).

| a write readable on a replica after the primary answered it, p50, `--sync always` | **0.04 ms** |

| a write on a tenant's standby after the router answered it, p50 (p99 0.448 ms) | **0.089 ms** |

| 20 tenants failed over onto the standby, the medians of three rounds | **63.5–79 ms** |

| a 58.7 MB tenant moved to another node | **under 0.3 s** |

| a synced change of one row applied by a replica's sync core, in process (the same put alone 7.1 µs) | **18.4 µs** |

**Search:** a `match` over 100 000 documents of 30 words
takes 0.06 ms, a facet over every row through `@hash` 0.01 ms,
and a typo's words among FiQA's 135 500 are found in 14 µs
([Text search](#text-search), [Retrieval
quality](#retrieval-quality)). Vectors have sections of their own after those, against
SQLite and PostgreSQL with pgvector: at a million 128-dim vectors
fenec-server answers the nearest ten at 99.1% recall in 0.141 ms, pgvector
at 98.6% in 2.076 ms ([At scale](#scale)).

**What the browser downloads:** the WebAssembly module and the
JavaScript client, or the client alone for an app whose queries run on a
server, which never fetches the module either way ([Binary size](#binary-size), Which package).

| What the page loads | brotli | gzip |

| The WebAssembly module, `fenec.wasm` | **273 KB** | 330 KB |

| The JavaScript client, `fenec.js` and its modules | 51 KB | 60 KB |

| The client alone in an app's bundle, `@fenecdb/web/client` | 9 KB | — |

Where the numbers come from

Every number here comes from `crates/fenec-bench`
and can be reproduced with a make target. The method is written out beside
each table, because a number without one is a claim, not a measurement.

## Everyday reads and writes (YCSB)

fenecdb as a general database: the six core workloads of YCSB, the
Yahoo! Cloud
Serving Benchmark, over a million records of ten 100-character fields —
reads of a whole record, updates of one field, inserts, short scans — in
process against SQLite, and `fenec-server` over HTTP against
PostgreSQL and MongoDB, and beside Redis. Each figure is the median of three runs:
operations a second, and under it the p99 of the workload's slowest
operation. fenecdb compacts its file on its own, in process and in the
server, as it ships. Every cell is below, the ones fenecdb
loses too.

**In process: fenecdb against SQLite.** Durable, an fsync a write:

| durable | fenecdb, 1 thread | SQLite, 1 thread | fenecdb, 16 threads | SQLite, 16 threads |

| A: 50% read, 50% update | **0.50 k**
p99 4.9 ms | 0.47 k
p99 8.1 ms | **3.22 k**
p99 25.0 ms | 0.45 k
p99 2 533 ms |

| B: 95% read, 5% update | **4.90 k**
p99 4.8 ms | 3.68 k
p99 8.1 ms | **39.4 k**
p99 14.7 ms | 3.80 k
p99 1 669 ms |

| D: 95% read the latest, 5% insert | **4.97 k**
p99 4.6 ms | 4.04 k
p99 7.9 ms | **44.8 k**
p99 13.8 ms | 4.07 k
p99 1 653 ms |

| E: 95% scan, 5% insert | 2.94 k
p99 4.7 ms | 2.81 k
p99 8.0 ms | **37.7 k**
p99 11.5 ms | 3.60 k
p99 1 602 ms |

| F: 50% read, 50% read-modify-write | **0.50 k**
p99 4.9 ms | 0.47 k
p99 7.2 ms | **4.05 k**
p99 16.1 ms | 0.43 k
p99 2 122 ms |

Buffered, each engine's default:

| buffered | fenecdb, 1 thread | SQLite, 1 thread | fenecdb, 16 threads | SQLite, 16 threads |

| A: 50% read, 50% update | **212 k**
p99 0.01 ms | 32.1 k
p99 0.16 ms | **151 k**
p99 0.71 ms | 29.4 k
p99 9.9 ms |

| B: 95% read, 5% update | **410 k**
p99 0.01 ms | 78.8 k
p99 0.18 ms | **545 k**
p99 0.61 ms | 47.9 k
p99 38.0 ms |

| C: 100% read | **438 k**
p99 0.00 ms | 279 k
p99 0.01 ms | **1841 k**
p99 0.01 ms | 606 k
p99 0.20 ms |

| D: 95% read the latest, 5% insert | **432 k**
p99 0.01 ms | 183 k
p99 0.14 ms | **485 k**
p99 0.84 ms | 71.4 k
p99 24.2 ms |

| E: 95% scan, 5% insert | **19.1 k**
p99 0.10 ms | 17.4 k
p99 0.63 ms | **58.9 k**
p99 3.7 ms | 16.6 k
p99 10.6 ms |

| F: 50% read, 50% read-modify-write | **192 k**
p99 0.01 ms | 34.6 k
p99 0.18 ms | **121 k**
p99 0.90 ms | 35.5 k
p99 10.4 ms |

**Over the wire: fenec-server against PostgreSQL and MongoDB.** Durable, all three in Docker's virtual machine, which syncs a write in a tenth of a millisecond where the Mac's `F_FULLFSYNC` takes four; fenec-server on the Mac beside them:

| durable, one client | fenec-server, Docker | PostgreSQL, Docker | MongoDB, Docker | fenec-server on the Mac |

| A: 50% read, 50% update | **3.34 k**
p99 1.1 ms | 2.52 k
p99 1.4 ms | 1.97 k
p99 1.7 ms | 0.53 k
p99 7.0 ms |

| B: 95% read, 5% update | **4.50 k**
p99 1.2 ms | 3.09 k
p99 5.5 ms | 2.48 k
p99 1.8 ms | 3.26 k
p99 6.1 ms |

| D: 95% read the latest, 5% insert | **4.86 k**
p99 1.1 ms | 2.77 k
p99 8.2 ms | 2.79 k
p99 2.0 ms | 3.68 k
p99 5.4 ms |

| E: 95% scan, 5% insert | **1.69 k**
p99 2.6 ms | 1.28 k
p99 3.4 ms | 0.84 k
p99 5.1 ms | 3.30 k
p99 4.9 ms |

| F: 50% read, 50% read-modify-write | **2.36 k**
p99 1.7 ms | 1.74 k
p99 3.2 ms | 1.33 k
p99 2.8 ms | 0.49 k
p99 6.4 ms |

| durable, 16 clients | fenec-server, Docker | PostgreSQL, Docker | MongoDB, Docker | fenec-server on the Mac |

| A: 50% read, 50% update | **13.3 k**
p99 5.7 ms | 10.5 k
p99 10.7 ms | 6.56 k
p99 15.5 ms | 3.94 k
p99 18.2 ms |

| B: 95% read, 5% update | **18.2 k**
p99 6.0 ms | 13.0 k
p99 17.7 ms | 11.1 k
p99 12.3 ms | 42.1 k
p99 10.4 ms |

| D: 95% read the latest, 5% insert | **18.0 k**
p99 6.2 ms | 14.8 k
p99 10.2 ms | 12.0 k
p99 11.9 ms | 41.8 k
p99 10.4 ms |

| E: 95% scan, 5% insert | **2.72 k**
p99 25.8 ms | 2.21 k
p99 30.5 ms | 2.01 k
p99 31.9 ms | 40.8 k
p99 9.9 ms |

| F: 50% read, 50% read-modify-write | **9.43 k**
p99 9.9 ms | 7.91 k
p99 12.0 ms | 5.00 k
p99 34.3 ms | 4.05 k
p99 12.5 ms |

Buffered, fenec-server on the Mac against the two in Docker, its network included:

| buffered | fenec-server, 1 thread | PostgreSQL, 1 thread | MongoDB, 1 thread | fenec-server, 16 threads | PostgreSQL, 16 threads | MongoDB, 16 threads |

| A: 50% read, 50% update | **44.2 k**
p99 0.04 ms | 2.98 k
p99 1.3 ms | 2.20 k
p99 1.5 ms | **110 k**
p99 0.70 ms | 13.7 k
p99 4.9 ms | 8.90 k
p99 10.0 ms |

| A, fenec-server in Docker as well | **3.06 k**
p99 1.4 ms |   |   | **16.5 k**
p99 3.7 ms |   |   |

| B: 95% read, 5% update | **44.0 k**
p99 0.04 ms | 2.99 k
p99 1.4 ms | 2.22 k
p99 1.6 ms | **154 k**
p99 0.37 ms | 13.7 k
p99 4.8 ms | 11.8 k
p99 5.5 ms |

| B, fenec-server in Docker as well | **3.08 k**
p99 1.4 ms |   |   | **17.5 k**
p99 3.4 ms |   |   |

| C: 100% read | **44.1 k**
p99 0.03 ms | 3.14 k
p99 1.3 ms | 2.14 k
p99 1.5 ms | **155 k**
p99 0.24 ms | 14.8 k
p99 4.1 ms | 12.9 k
p99 4.6 ms |

| C, fenec-server in Docker as well | 3.07 k
p99 1.3 ms |   |   | **18.4 k**
p99 2.9 ms |   |   |

| D: 95% read the latest, 5% insert | **44.0 k**
p99 0.04 ms | 2.82 k
p99 1.4 ms | 2.22 k
p99 1.5 ms | **149 k**
p99 0.46 ms | 14.4 k
p99 4.6 ms | 13.2 k
p99 4.6 ms |

| D, fenec-server in Docker as well | **3.10 k**
p99 1.4 ms |   |   | **17.7 k**
p99 3.5 ms |   |   |

| E: 95% scan, 5% insert | **17.8 k**
p99 0.10 ms | 1.26 k
p99 3.4 ms | 0.85 k
p99 4.9 ms | **55.2 k**
p99 2.6 ms | 2.07 k
p99 34.3 ms | 2.02 k
p99 33.8 ms |

| E, fenec-server in Docker as well | **1.49 k**
p99 2.8 ms |   |   | **2.59 k**
p99 27.7 ms |   |   |

| F: 50% read, 50% read-modify-write | **30.1 k**
p99 0.06 ms | 2.09 k
p99 2.0 ms | 1.40 k
p99 2.4 ms | **81.3 k**
p99 1.1 ms | 9.22 k
p99 8.8 ms | 6.60 k
p99 14.6 ms |

| F, fenec-server in Docker as well | 2.07 k
p99 1.9 ms |   |   | **11.2 k**
p99 7.3 ms |   |   |

- **Durable writes from many threads share their fsyncs.** At 16
    threads fenecdb's lock is released before the fsync, so one fsync covers
    every write that landed meanwhile; SQLite syncs each commit under its
    write lock and stays at one thread's rate.

- **Durable over the wire is compared inside Docker.** On a Mac,
    PostgreSQL and MongoDB run in Docker's Linux virtual machine, whose
    fsync took 0.08 to 0.10 ms where the Mac's `F_FULLFSYNC`
    took 3.9 ms, so the durable tables put fenec-server in the same
    machine; the server on the Mac is beside them for what a real flush
    to the disk costs it.

- **Buffered over the wire is mostly the network.** Docker's network
    costs a request about a quarter of a millisecond: the emptiest request
    took 0.334 ms to PostgreSQL and 0.342 ms to MongoDB at the
    median, 0.281 ms to fenec-server in Docker and 0.019 ms to
    it on the Mac. The rows fenec-server in Docker as well are the
    like-for-like there: ahead of PostgreSQL at one client in A, B, D and
    E, behind in C and F by 2% and 1%; ahead in every workload at 4 and 16
    clients; and ahead of MongoDB in every cell.

**Where fenecdb loses, and why.**

- **Nowhere in process.** fenecdb is ahead of SQLite in every cell;
    the closest are C at four threads, 1.43 M against 1.33 M, and E durable
    at one, 2.94 k against 2.81 k. It lost the buffered reads after the
    updates until its file compacted on its own (C at one thread 30.1 k
    against 279 k, now 438 k; below).

- **fenec-server in Docker against PostgreSQL, at one client:**
    buffered C 3.07 k against 3.14 k and F 2.07 k against 2.09 k. A read
    is a round trip of about 0.29 ms, and
    [taken apart](#round-trip) 0.25 ms of it is Docker's port
    forwarding, the same for both; fenec-server's own part is 0.018 ms,
    PostgreSQL's bind and execute 0.016, and the answer the same size, 1.3
    KB against 1.2 in one packet. The rest moves with the VM rather than
    with either server, by more than the gap from one day to the next:
    PostgreSQL's C at one client did 3.66 k on 2026-10-05, 3.14 k here,
    fenec-server's 3.19 k and 3.07 k. E was the gap that was fenec-server's: a
    scan of 50 rows spent 213-220 µs inside it, half decoding each row into
    values and half writing them out, and took 37 page faults, musl giving
    the answer's memory back to the system and faulting it in again. Written
    straight from the stored rows into a buffer the connection keeps, it
    takes 50-60 µs and none: on 2026-10-05 E went 1.03 k -> 1.62 k
    buffered and 1.06 k -> 1.82 k durable, and here it does 1.49 k against
    PostgreSQL's 1.26 k buffered, 1.69 k against 1.28 k durable. Durable A
    and F, where it was behind, are ahead too: an fsync of a file that grew
    commits ext4's journal, 0.36 ms against 0.065 in place, and a durable
    write syncs a log of fixed size beside the file instead (A 2.24 k ->
    3.27 k on 2026-10-04; here 3.34 k against PostgreSQL's 2.52 k).

- **fenec-server on the Mac, durable A and F:**
    A 0.53 k at one client and 3.94 k at 16, F 0.49 k and 4.05 k. Half the
    operations are writes,
    and each waits for `F_FULLFSYNC`, 3.9 ms on this disk; it is
    beside the Docker columns, not against them.

### Beside Redis

Redis 7.4, the official `redis:7` image, in a container of its
own in the systems' turns, asked over redis-rs 1.7.1 by a connection a
client thread, in the same cells as fenec-server in Docker. It is another
kind of system from the rest, and the tables read with that in mind:

- **Everything is in memory.** Redis holds the whole data set in its
    memory and answers from it; nothing is read from disk, and a data set
    larger than the memory is not an option. fenec-server reads its file
    through the page cache, PostgreSQL and MongoDB through their buffers.
    What each held is the last table below.

- **One thread runs the commands.** Every command runs on Redis's
    main thread, one after another, however many clients there are
    (`io-threads` is 1 as it comes, and raised it threads the
    sockets' reading and writing, not the commands). fenec-server reads on
    a thread a connection under a shared lock and writes under an exclusive
    one.

- **Its durability is a setting of its own,** and each mode is
    Redis's own terms for it. durable is the append-only file with
    `appendfsync always`: the writes are in the file and fsynced
    before the answers go out, one fsync for the writes of every client
    that landed meanwhile. buffered is `appendfsync
    everysec`, the policy Redis suggests and its default once the file
    is on: a thread fsyncs once a second, and a crash loses up to a second
    of answered writes, two if the disk is slow, where fenec-server's
    `--sync 250` loses 250 ms. snapshot is the image as it
    comes: no append-only file, a snapshot of the data set after 3 600 s
    and a write, 300 s and 100 writes, or 60 s and 10 000, so a crash loses
    every write since the last one, minutes of them. Nothing else here is
    measured that way: it is a column beside the others, not against them.
    The snapshots stay on in the other two modes, as Redis's
    [persistence
    page](https://redis.io/docs/latest/operate/oss_and_stack/management/persistence/) advises both "if you want a degree of data safety comparable
    to what PostgreSQL can provide you"; each snapshot and each rewrite of
    the append-only file is a fork, and its copied pages and its writing
    fall inside the cells.

- **A record is a hash, and a scan needs an index of its own.** Each
    is as YCSB's own Redis binding has it
    ([RedisClient.java](https://github.com/brianfrankcooper/YCSB/blob/master/redis/src/main/java/site/ycsb/db/RedisClient.java)):
    a hash a record, read whole by `HGETALL`, an update an
    `HSET` of its one field, and every key a member of a sorted
    set, `_indices`, from which a scan takes its keys
    (`ZRANGEBYSCORE`) before it reads their records. Three things
    differ, each in Redis's favour or neither's: a key's score is the key,
    where the binding scatters keys by a hash of their text, so a scan reads
    keys in order from its own as every other system's does; an insert's
    `HSET` and `ZADD` are one `MULTI`/`EXEC`,
    one round trip and one fsync, where the binding sends them apart; and a
    scan's records come in one pipeline, where the binding asks for them one
    at a time, so a scan of 50 records is two round trips rather than 51.

- **Both are behind Docker's network,** about a quarter of a
    millisecond a request, most of a read by key: fenec-server in Docker is
    the like-for-like, and the server on the Mac is not set against Redis
    in a container.

Durable: fenec-server's `--sync always` against Redis's `appendfsync always`, an fsync before each answer, both in Docker's virtual machine:

| durable | fenec-server, 1 client | Redis, 1 client | fenec-server, 16 clients | Redis, 16 clients |

| A: 50% read, 50% update | **3.34 k**
p99 1.1 ms | 2.03 k
p99 1.8 ms | **13.3 k**
p99 5.7 ms | 8.53 k
p99 6.5 ms |

| B: 95% read, 5% update | **4.50 k**
p99 1.2 ms | 3.51 k
p99 1.8 ms | **18.2 k**
p99 6.0 ms | 9.25 k
p99 8.8 ms |

| D: 95% read the latest, 5% insert | **4.86 k**
p99 1.1 ms | 3.59 k
p99 2.1 ms | **18.0 k**
p99 6.2 ms | 13.0 k
p99 5.5 ms |

| E: 95% scan, 5% insert | **1.69 k**
p99 2.6 ms | 0.65 k
p99 6.1 ms | **2.72 k**
p99 25.8 ms | 0.93 k
p99 64.2 ms |

| F: 50% read, 50% read-modify-write | **2.36 k**
p99 1.7 ms | 1.60 k
p99 2.4 ms | **9.43 k**
p99 9.9 ms | 4.93 k
p99 16.2 ms |

Buffered, both in Docker as well: `--sync 250` against `appendfsync everysec`, and beside them Redis with its snapshots alone, which nothing here is measured against:

| buffered | fenec-server, 1 client | Redis, 1 client | Redis, snapshots, 1 client | fenec-server, 16 clients | Redis, 16 clients | Redis, snapshots, 16 clients |

| A: 50% read, 50% update | **3.06 k**
p99 1.4 ms | 2.74 k
p99 1.5 ms | 2.80 k
p99 1.4 ms | **16.5 k**
p99 3.7 ms | 8.35 k
p99 7.2 ms | 8.60 k
p99 7.3 ms |

| B: 95% read, 5% update | **3.08 k**
p99 1.4 ms | 2.75 k
p99 1.5 ms | 2.76 k
p99 1.5 ms | **17.5 k**
p99 3.4 ms | 13.5 k
p99 4.2 ms | 14.4 k
p99 3.8 ms |

| C: 100% read | **3.07 k**
p99 1.3 ms | 2.73 k
p99 1.5 ms |   | **18.4 k**
p99 2.9 ms | 14.0 k
p99 4.0 ms |   |

| D: 95% read the latest, 5% insert | **3.10 k**
p99 1.4 ms | 2.85 k
p99 1.5 ms | 2.71 k
p99 1.6 ms | **17.7 k**
p99 3.5 ms | 15.2 k
p99 3.6 ms | 15.5 k
p99 3.3 ms |

| E: 95% scan, 5% insert | **1.49 k**
p99 2.8 ms | 0.62 k
p99 6.2 ms | 0.62 k
p99 6.5 ms | **2.59 k**
p99 27.7 ms | 1.04 k
p99 59.5 ms | 1.04 k
p99 62.1 ms |

| F: 50% read, 50% read-modify-write | **2.07 k**
p99 1.9 ms | 1.52 k
p99 2.3 ms | 1.87 k
p99 2.0 ms | **11.2 k**
p99 7.3 ms | 5.53 k
p99 14.6 ms | 9.20 k
p99 7.8 ms |

**Where fenec-server loses to Redis.** Every cell in which Redis does
more operations a second than fenec-server in Docker, the same mode against
the same mode, at 1, 4 and 16 clients:

None of the 33 cells measured both ways.

**What each holds.** On disk and in memory after A, B, C, F and D:
the memory is the container's own, its processes' anonymous memory and
shared memory from its cgroup, the page cache of its files apart, and
Redis's is read with no snapshot or rewrite running. fenec-server's file
is in the page cache, not in this column, as PostgreSQL's and MongoDB's
files beyond their buffers are; Redis's data set is in it whole.

| after A to F | durable, on disk | durable, in memory | buffered, on disk | buffered, in memory | snapshot, on disk | snapshot, in memory |

| fenec-server in Docker | 1 881 MB | 43 MB | 1 947 MB | 43 MB |   |   |

| Redis | 2 346 MB | 1 874 MB | 2 365 MB | 1 865 MB | 2 238 MB | 1 890 MB |

| PostgreSQL | 1 382 MB | 1 133 MB | 1 392 MB | 1 134 MB |   |   |

| MongoDB | 2 248 MB | 1 398 MB | 2 337 MB | 1 504 MB |   |   |

`--verify` held every Redis answer to what was written, in the
three modes (`crates/fenec-bench/ycsb/verify.tsv`, run
`rv`), and the runs are `crates/fenec-bench/ycsb/run-redis.sh`.

### One request's round trip, taken apart

A YCSB read by key (a 1 KB record of ten fields) and an update of one
field, one client over 100 000 records, the client's parts and the
process's CPU from `getrusage`, fenec-server's phases from a
build with `--features timing` (`GET /_timing`), and
PostgreSQL's bind and execute from its own log; means but the totals, which
are medians (`make roundtrip-bench`):

| a read by key | fenec-server, Mac | fenec-server, Docker | PostgreSQL, Docker |

| round trip, p50 | 0.025 ms | **0.288 ms** | 0.333 ms |

| inside the server | 0.010 ms | 0.023 ms | 0.016 ms, bind and execute |

| its parts | HTTP 1.1 / 2.1 µs, JSON and the parsed statement 0.7 / 1.4, run 2.6 / 5.2, the row as JSON 1.7 / 2.4, the send 3.0 / 9.8 | executor 9.2 µs |

| the client's CPU | 0.006 ms | 0.030 ms | 0.055 ms |

| the server's system calls |   | a receive, a writev | two receives, a send, an epoll_pwait |

| the rest: the kernel and the network | 0.012 ms | about 0.25 ms | about 0.26 ms |

| a scan of 50 records, p50 |   | **0.454-0.479 ms**
0.619-0.630 before | 0.582 ms |

| the scan inside the server |   | **0.050-0.060 ms**
0.213-0.220 before | 0.237 ms, bind and execute |

| the scan's answer, as the container sent it |   | 56.9 KB of JSON, 0 page faults
37 before | 53.4 KB of binary rows |

| a durable update, p50 | 4.0 ms (`F_FULLFSYNC`) | **0.325 ms**
0.624 before the sync log | 0.378 ms |

- **Docker's port forwarding is most of a request,** for both:
    about 0.25 ms of a 0.29 ms read. What either server does is a tenth
    of it, so a statement sent by an id rather than its text, or rows in a
    binary form rather than JSON, would save a microsecond or two of 290
    and are not worth a protocol of their own.

- **The durable gap was an fsync of a file that grew.** On ext4 it
    commits the journal as well: 356 µs for a 300-byte append and its
    `fdatasync` in the VM, 65 µs for the same bytes written where
    the file already reached, which is how PostgreSQL's WAL is written. A
    durable write now syncs a log of fixed size beside the file, and the
    file is synced once the log is full ([File
    format](https://fenecdb.com/docs/file-format)): 392 -> 92 µs of a durable update's server time.

- **A scan was fenec-server's own.** Fifty 1 KB records are 56.9
    KB of JSON against PostgreSQL's 53.4 KB of binary rows, each handed to
    the network card at once, so the bytes were not it: the server spent
    213-220 µs inside on them, half decoding each row's ten fields into
    values one look-up at a time and half writing them out a byte at a time,
    and took 37 page faults a scan, the container's allocator (musl) giving
    the answer's memory back to the system after each and faulting it in
    again. A plain `get` is now written as JSON straight from the
    stored rows under the read lock, eight bytes tested at a time for
    escaping, into a buffer the connection keeps: 50-60 µs, no fault, and a
    read by id 22.4 -> 17.7 µs inside. The answer is the same JSON, byte for
    byte, so no client changed. `make roundtrip-bench RT_ARGS="--measure
    read,scan50"` counts the bytes, packets and page faults.

- **What was the client's:** Python's `Client` opened a
    connection a request, 152 µs a read by id on the Mac against 63 over
    the one it keeps now; Go's, .NET's and JavaScript's clients kept theirs
    already.

Method and details

YCSB's core workloads, written in the bench crate
(`crates/fenec-bench/src/bin/ycsb.rs`) to YCSB's own
definitions rather than run through its Java client: a record is a key and
ten fields of 100 random characters, a read reads the whole record, an
update writes one field, keys are drawn by YCSB's scrambled zipfian (0.99,
hashed over ten billion items) or, for D, its skewed-latest, and a scan
reads 1 to 100 records from a key. The key is an integer from 1
(`insertorder=ordered` without the `user` prefix), so
each system keys by what it keys best by: fenecdb's `id`,
SQLite's `INTEGER PRIMARY KEY`, a `bigint` primary
key, MongoDB's `_id`; E's scan is `get usertable where id
>= $1 limit n`, which walks the id index from the key. A read never
asks for a key whose insert has not been answered. The sequence is YCSB's:
a load, then A, B, C, F and D over that data, then a load afresh for E, at
1, 4 and 16 threads, each cell 30 s after a 5 s warm-up. Latencies go into
a log-linear histogram of 64 steps an octave.

**The systems.** fenecdb 0.1.9 in process, reads under the
shared lock and writes under the exclusive one as the server takes them,
against SQLite 3.53.2 through `rusqlite`, a connection a
thread, WAL, the file mapped. `fenec-server` natively, each
client thread a kept-alive HTTP connection sending `POST /query`
with parameters, against PostgreSQL 17 (`shared_buffers` 1 GB,
prepared statements) and MongoDB 8 (the official Rust driver), each in a
container started for its turn and removed with its volume after, so the
virtual machine (8 cores, 3.8 GB) holds one at a time. fenec-server in
Docker is the same server in a container of its own image.
**Durable** is an fsync a write in each engine's own terms: a flush and
its fsync (`F_FULLFSYNC` on macOS) and `--sync
always`, `synchronous=FULL` with `fullfsync`,
`synchronous_commit=on`, `j: true`; **buffered** is
each one's default: a flush every 250 ms (`--sync 250`),
`synchronous=NORMAL`, `synchronous_commit=off`,
`w: 1, j: false`. C writes nothing and runs once, buffered.
Docker's fsync is the Linux VM's, which does not flush the Mac's disk as
`F_FULLFSYNC` does: a 1 KB append and its
`F_FULLFSYNC` took 3.9 ms on the Mac and its plain
`fsync` 0.023 ms, while in the VM, on a volume as the databases'
data, `pg_test_fsync` gave 0.076 ms for `fdatasync`
and 0.103 for `fsync` (`crates/fenec-bench/ycsb/fsync.txt`).
So the server is measured in Docker as well, at 1, 4 and 16 threads,
durable and buffered, beside it on the Mac.

**Why fenecdb's reads slowed down after the updates, and no longer do.**
An update writes the whole record again at the file's end, and the version
it replaces stays dead until a `compact`. The first three runs
(`r1`-`r3`) were measured before anything compacted on
its own: after A's and B's updates the buffered file had grown to 4.7 to
6.5 GB on this 8 GB machine, against SQLite's 2.1 to 2.5 GB updated in
place, and a profile of C at one thread
(`crates/fenec-bench/ycsb/profile-c1.txt`) put 86% of the
reading thread's time on the first byte of the record in the mapped file,
waiting for its page: about one read in five went to the disk. C at one
thread read 30.1 k a second, where the same cell over a file just loaded
read 443 k. fenecdb now compacts a file beside its queries once half of it
is dead ([Compaction](https://fenecdb.com/docs//docs/server#compaction)), in an app's
process as in the server, and the harness runs it as an app would get it:
the in-process database with the engine's `Compactor`, the
server with its default `--auto-compact`. Runs
`c1`-`c3` measured fenecdb and fenec-server again with
it; the buffered file stood at 3.7 to 4.0 GB after D, mid-way between
compacts of a file A's updates grow by 100 MB a second, and C at one thread
read 438 k. The earlier runs of fenecdb and fenec-server stay in the results,
marked superseded (`crates/fenec-bench/ycsb/superseded.tsv`);
SQLite was not measured again, and the servers were measured again since,
all of them on one day (below).

**And once the file nears what the page cache can hold.** Since these
runs a file is also compacted as it nears the memory the system has for
its pages, or once the process waits for the disk
([Compaction](https://fenecdb.com/docs//docs/server#compaction)); the cells above predate
it. Measured apart on 2026-10-11 and 12, in process, buffered, A, B and F at 1
and 16 threads, the build before it and the one with it in turns, its cells
in none of the medians: with the machine's memory as it stood, two runs
each, A at 16 threads went from 82 and 92 k to 133 k twice and the rest
stayed within the runs' spread; beside 1 GB of memory held by another
program, one run each, B went from 23.5 k to 63.5 k at one thread and from
62.3 k to 112.8 k at 16, F from 23.6 k to 48.5 k at 16, the file at its
largest 1.2 to 1.5 GB against 1.5 to 2.0, and A at one thread from 29.7 k
to 24.1 k.

**The one-client server cells, measured again the same day.** On
2026-10-04 fenec-server natively and in Docker (A to F at one thread,
durable and buffered, A and C at 16) and PostgreSQL beside them, in turns
with three idle minutes between, three times, after the sync log and the
rendering changes of [the round trip](#round-trip): each cell
measured again counts as its run's last line, under the runs' ids,
`c1`-`c3` for the servers and
`r1`-`r3` for PostgreSQL, so every cell compared at
one client is of one day. Comparing them against PostgreSQL's cells of
days before had put the servers behind where Docker's network had been
quicker then: its C at one client was 3.34 k, 3.53 k this time, and 3.00 k
in a check in between. The `--verify` pass over all 84 cells
found no mismatch, and nor did a shorter one after the sync log's fixes
(`crates/fenec-bench/ycsb/verify.tsv`, runs `c` and
`fix`).

**Again after the read path's changes.** On 2026-10-05, with
fenec-server writing a plain `get`'s rows as JSON straight from
the stored documents ([the round trip](#round-trip)),
fenec-server natively and in Docker and PostgreSQL beside them measured
B, C, D and E at 1 and 16 clients, durable and buffered, in turns, three
times, under the same ids: each cell measured again counts as its run's
last line, so these replace those cells of the day before, PostgreSQL's
too, and A and F stay as they were. The first of the three ran while
other work on the machine still held a core, its idle probe taken at a
load of five (65.0 ms against 62.5 idle), and its cells came in up to a quarter
under the other two, PostgreSQL's as much as the servers'; it was
measured again on the quiet machine, and those lines are its cells. A
`--verify` pass over fenecdb, fenec-server and fenec-server in
Docker, A to F at 1 and 16 threads, durable and buffered, 8 s a cell,
checked 42.1 million reads, 1.8 million scans of 89.2 million rows and 6.1
million writes and found no mismatch (`verify.tsv`, run
`gap`).

**Every server on one day.** On the night of 2026-10-13,
fenec-server natively and in Docker, PostgreSQL, MongoDB and Redis measured
every cell, A to F at 1, 4 and 16 clients, durable and buffered, three
times, in turns with three idle minutes between (runs
`rx1`-`rx3`, 8.5 hours). They replace the cells of
the two paragraphs above, the servers' of `c1`-`c3`
and PostgreSQL's and MongoDB's of `r1`-`r3`, so every
cell compared over the wire is of one day; fenecdb's and SQLite's in
process stay as they were. A first attempt (`rd1`-`rd3`)
is in the results and in no median: it read the containers' file sizes
beside the cells, a container started for each look ten times a second,
and the VM's work took fenec-server in Docker's C at one client over
100 000 records from 3.14-3.17 k to 2.33-2.38 k, in turns. A size read through a container is read once
the clients stop now.

**The machine.** A fanless M1 MacBook Air, 8 GB, macOS, which slows
to a third under minutes of load on every core: each cell starts once a
one-core probe runs within 4% of its idle time, the probe's ratio written
beside the cell, and the systems take turns with three idle minutes
between them.

```bash
make ycsb YCSB_ARGS="--runs 3 --out ycsb.tsv"
./target/release/ycsb report ycsb.tsv
```

**The answers are checked.** `--verify` logs every
operation and what it was answered, and after each cell holds each one to
what was written: a read must answer the record it asked for, each field
holding the value of a write that could have been the last one before it
(one that ended before the read began and no other ended after it, or one
that overlapped the read); a scan must answer consecutive keys from its
own, as many as it asked for wherever they existed, each record as
written; a write must report one record written. Every write is logged,
not a sample, because judging a read under concurrent writes takes every
write's span; the log is a push into each thread's own list and the check
runs once the cell is over, so nothing is compared in an operation's time.
The same check holds every system. A pass over all six — 100 000 records,
10 s a cell, workloads A, E and F at 1 and 16 threads, durable and
buffered — checked 13.8 million reads, 2.7 million scans of 134.7 million rows and
9.8 million writes and found no mismatch
(`crates/fenec-bench/ycsb/verify.tsv`); its timings are not
published.

**Held to YCSB itself.** Yahoo's YCSB 0.17.0 — its Java client, the
`jdbc` and `mongodb` bindings, PostgreSQL's JDBC driver
42.7.4 — and this harness ran the same cells against the same containers:
workloads A and C, 1 and 16 threads, 100 000 records, durable, each 30 s
(YCSB's `maxexecutiontime`, which counts its start; the harness
warms up 5 s first). Operations a second and the p99 of the slowest
operation:

|   | this harness | YCSB 0.17.0 | harness against YCSB |

| PostgreSQL, A, 1 thread | 2 814, 1.04 ms | 3 102, 0.93 ms | -9% |

| PostgreSQL, A, 16 threads | 15 469, 3.10 ms | 15 173, 3.62 ms | +2% |

| PostgreSQL, C, 1 thread | 2 701, 1.29 ms | 2 956, 1.20 ms | -9% |

| PostgreSQL, C, 16 threads | 18 075, 2.54 ms | 18 763, 2.77 ms | -4% |

| MongoDB, A, 1 thread | 2 459, 1.12 ms | 2 522, 1.16 ms | -2% |

| MongoDB, A, 16 threads | 10 760, 5.67 ms | 12 489, 4.64 ms | -14% |

| MongoDB, C, 1 thread | 2 279, 1.27 ms | 2 481, 1.30 ms | -8% |

| MongoDB, C, 16 threads | 15 748, 2.54 ms | 15 392, 4.18 ms | +2% |

The harness lands between 14% under and 2% over YCSB's own
client, six of the eight cells within 9%; where it differs by more than 2% it measures
PostgreSQL and MongoDB a little slower than YCSB does, not faster. The
p99s agree to a few tenths of a millisecond but for MongoDB's C at 16
threads, 2.54 ms here against YCSB's 4.18. YCSB keys by strings
(`user…`) where the harness keys by integers, and each side
drives its own client library. The outputs and the script are
`crates/fenec-bench/ycsb/calibration/`.

The full grid, every cell at 1, 4 and 16 threads with each operation's
p50 and p99, is below; the runs themselves are
`crates/fenec-bench/ycsb/results.tsv`.

Every cell

| system | setting | workload | threads | ops/s | p50 / p99 per operation |

| fenecdb | durable | A | 1 | 0.50 k | read 0.02 ms / 0.05 ms; update 4.0 ms / 4.9 ms |

| fenecdb | durable | A | 4 | 1.01 k | read 0.02 ms / 0.10 ms; update 8.0 ms / 11.1 ms |

| fenecdb | durable | A | 16 | 3.22 k | read 0.05 ms / 0.43 ms; update 8.0 ms / 25.0 ms |

| fenecdb | durable | B | 1 | 4.90 k | read 0.01 ms / 0.03 ms; update 3.9 ms / 4.8 ms |

| fenecdb | durable | B | 4 | 10.1 k | read 0.01 ms / 0.06 ms; update 7.7 ms / 10.8 ms |

| fenecdb | durable | B | 16 | 39.4 k | read 0.01 ms / 0.35 ms; update 7.5 ms / 14.7 ms |

| fenecdb | durable | D | 1 | 4.97 k | read 0.01 ms / 0.03 ms; insert 3.9 ms / 4.6 ms |

| fenecdb | durable | D | 4 | 10.1 k | read 0.01 ms / 0.05 ms; insert 7.8 ms / 10.8 ms |

| fenecdb | durable | D | 16 | 44.8 k | read 0.01 ms / 0.31 ms; insert 6.5 ms / 13.8 ms |

| fenecdb | durable | E | 1 | 2.94 k | insert 4.0 ms / 4.7 ms; scan 0.14 ms / 0.32 ms |

| fenecdb | durable | E | 4 | 9.75 k | insert 6.8 ms / 10.8 ms; scan 0.08 ms / 0.17 ms |

| fenecdb | durable | E | 16 | 37.7 k | insert 6.7 ms / 11.5 ms; scan 0.08 ms / 0.42 ms |

| fenecdb | durable | F | 1 | 0.50 k | read 0.02 ms / 0.04 ms; rmw 4.0 ms / 4.9 ms |

| fenecdb | durable | F | 4 | 1.01 k | read 0.02 ms / 0.09 ms; rmw 8.0 ms / 10.9 ms |

| fenecdb | durable | F | 16 | 4.05 k | read 0.03 ms / 0.38 ms; rmw 7.9 ms / 16.1 ms |

| fenecdb | buffered | A | 1 | 212 k | read 0.00 ms / 0.01 ms; update 0.00 ms / 0.01 ms |

| fenecdb | buffered | A | 4 | 178 k | read 0.00 ms / 0.17 ms; update 0.01 ms / 0.33 ms |

| fenecdb | buffered | A | 16 | 151 k | read 0.04 ms / 0.68 ms; update 0.06 ms / 0.71 ms |

| fenecdb | buffered | B | 1 | 410 k | read 0.00 ms / 0.00 ms; update 0.00 ms / 0.01 ms |

| fenecdb | buffered | B | 4 | 553 k | read 0.00 ms / 0.06 ms; update 0.01 ms / 0.21 ms |

| fenecdb | buffered | B | 16 | 545 k | read 0.00 ms / 0.28 ms; update 0.09 ms / 0.61 ms |

| fenecdb | buffered | C | 1 | 438 k | read 0.00 ms / 0.00 ms |

| fenecdb | buffered | C | 4 | 1433 k | read 0.00 ms / 0.00 ms |

| fenecdb | buffered | C | 16 | 1841 k | read 0.00 ms / 0.01 ms |

| fenecdb | buffered | D | 1 | 432 k | read 0.00 ms / 0.00 ms; insert 0.00 ms / 0.01 ms |

| fenecdb | buffered | D | 4 | 625 k | read 0.00 ms / 0.05 ms; insert 0.01 ms / 0.18 ms |

| fenecdb | buffered | D | 16 | 485 k | read 0.00 ms / 0.35 ms; insert 0.10 ms / 0.84 ms |

| fenecdb | buffered | E | 1 | 19.1 k | insert 0.01 ms / 0.01 ms; scan 0.06 ms / 0.10 ms |

| fenecdb | buffered | E | 4 | 42.3 k | insert 0.19 ms / 1.3 ms; scan 0.07 ms / 0.51 ms |

| fenecdb | buffered | E | 16 | 58.9 k | insert 0.17 ms / 3.7 ms; scan 0.10 ms / 2.0 ms |

| fenecdb | buffered | F | 1 | 192 k | read 0.00 ms / 0.01 ms; rmw 0.01 ms / 0.01 ms |

| fenecdb | buffered | F | 4 | 164 k | read 0.00 ms / 0.12 ms; rmw 0.01 ms / 0.30 ms |

| fenecdb | buffered | F | 16 | 121 k | read 0.02 ms / 0.56 ms; rmw 0.13 ms / 0.90 ms |

| SQLite | durable | A | 1 | 0.47 k | read 0.19 ms / 0.61 ms; update 4.0 ms / 8.1 ms |

| SQLite | durable | A | 4 | 0.47 k | read 0.18 ms / 0.63 ms; update 4.0 ms / 20.1 ms |

| SQLite | durable | A | 16 | 0.45 k | read 0.20 ms / 0.78 ms; update 4.0 ms / 2 533 ms |

| SQLite | durable | B | 1 | 3.68 k | read 0.01 ms / 0.44 ms; update 3.9 ms / 8.1 ms |

| SQLite | durable | B | 4 | 3.82 k | read 0.02 ms / 0.57 ms; update 4.0 ms / 388 ms |

| SQLite | durable | B | 16 | 3.80 k | read 0.02 ms / 0.62 ms; update 4.2 ms / 1 669 ms |

| SQLite | durable | D | 1 | 4.04 k | read 0.01 ms / 0.61 ms; insert 3.9 ms / 7.9 ms |

| SQLite | durable | D | 4 | 4.25 k | read 0.01 ms / 0.53 ms; insert 3.9 ms / 367 ms |

| SQLite | durable | D | 16 | 4.07 k | read 0.01 ms / 0.53 ms; insert 3.9 ms / 1 653 ms |

| SQLite | durable | E | 1 | 2.81 k | insert 3.9 ms / 8.0 ms; scan 0.08 ms / 1.00 ms |

| SQLite | durable | E | 4 | 3.37 k | insert 4.9 ms / 275 ms; scan 0.08 ms / 1.3 ms |

| SQLite | durable | E | 16 | 3.60 k | insert 7.7 ms / 1 602 ms; scan 0.08 ms / 1.2 ms |

| SQLite | durable | F | 1 | 0.47 k | read 0.05 ms / 0.73 ms; rmw 4.0 ms / 7.2 ms |

| SQLite | durable | F | 4 | 0.46 k | read 0.19 ms / 1.0 ms; rmw 4.0 ms / 202 ms |

| SQLite | durable | F | 16 | 0.43 k | read 0.21 ms / 1.0 ms; rmw 4.0 ms / 2 122 ms |

| SQLite | buffered | A | 1 | 32.1 k | read 0.00 ms / 0.15 ms; update 0.01 ms / 0.16 ms |

| SQLite | buffered | A | 4 | 35.7 k | read 0.00 ms / 0.43 ms; update 0.01 ms / 1.9 ms |

| SQLite | buffered | A | 16 | 29.4 k | read 0.01 ms / 0.51 ms; update 0.01 ms / 9.9 ms |

| SQLite | buffered | B | 1 | 78.8 k | read 0.00 ms / 0.17 ms; update 0.01 ms / 0.18 ms |

| SQLite | buffered | B | 4 | 79.7 k | read 0.00 ms / 0.42 ms; update 0.01 ms / 4.1 ms |

| SQLite | buffered | B | 16 | 47.9 k | read 0.04 ms / 1.4 ms; update 0.44 ms / 38.0 ms |

| SQLite | buffered | C | 1 | 279 k | read 0.00 ms / 0.01 ms |

| SQLite | buffered | C | 4 | 1326 k | read 0.00 ms / 0.01 ms |

| SQLite | buffered | C | 16 | 606 k | read 0.01 ms / 0.20 ms |

| SQLite | buffered | D | 1 | 183 k | read 0.00 ms / 0.14 ms; insert 0.01 ms / 0.04 ms |

| SQLite | buffered | D | 4 | 126 k | read 0.00 ms / 0.26 ms; insert 0.02 ms / 3.9 ms |

| SQLite | buffered | D | 16 | 71.4 k | read 0.02 ms / 0.99 ms; insert 0.18 ms / 24.2 ms |

| SQLite | buffered | E | 1 | 17.4 k | insert 0.01 ms / 0.04 ms; scan 0.04 ms / 0.63 ms |

| SQLite | buffered | E | 4 | 28.4 k | insert 0.02 ms / 1.5 ms; scan 0.10 ms / 0.62 ms |

| SQLite | buffered | E | 16 | 16.6 k | insert 0.18 ms / 10.6 ms; scan 0.93 ms / 2.7 ms |

| SQLite | buffered | F | 1 | 34.6 k | read 0.00 ms / 0.16 ms; rmw 0.01 ms / 0.18 ms |

| SQLite | buffered | F | 4 | 29.0 k | read 0.01 ms / 1.4 ms; rmw 0.01 ms / 4.1 ms |

| SQLite | buffered | F | 16 | 35.5 k | read 0.01 ms / 1.2 ms; rmw 0.01 ms / 10.4 ms |

| fenec-server | durable | A | 1 | 0.53 k | read 0.14 ms / 0.26 ms; update 3.8 ms / 7.0 ms |

| fenec-server | durable | A | 4 | 1.08 k | read 0.19 ms / 0.81 ms; update 7.0 ms / 11.2 ms |

| fenec-server | durable | A | 16 | 3.94 k | read 0.33 ms / 1.3 ms; update 7.5 ms / 18.2 ms |

| fenec-server | durable | B | 1 | 3.26 k | read 0.09 ms / 0.33 ms; update 3.9 ms / 6.1 ms |

| fenec-server | durable | B | 4 | 10.1 k | read 0.07 ms / 0.22 ms; update 6.5 ms / 9.9 ms |

| fenec-server | durable | B | 16 | 42.1 k | read 0.06 ms / 0.20 ms; update 6.3 ms / 10.4 ms |

| fenec-server | durable | D | 1 | 3.68 k | read 0.08 ms / 0.15 ms; insert 3.9 ms / 5.4 ms |

| fenec-server | durable | D | 4 | 10.0 k | read 0.07 ms / 0.14 ms; insert 6.6 ms / 9.9 ms |

| fenec-server | durable | D | 16 | 41.8 k | read 0.06 ms / 0.20 ms; insert 6.4 ms / 10.4 ms |

| fenec-server | durable | E | 1 | 3.30 k | insert 3.5 ms / 4.9 ms; scan 0.13 ms / 0.24 ms |

| fenec-server | durable | E | 4 | 10.9 k | insert 5.7 ms / 8.7 ms; scan 0.08 ms / 0.18 ms |

| fenec-server | durable | E | 16 | 40.8 k | insert 5.9 ms / 9.9 ms; scan 0.08 ms / 0.32 ms |

| fenec-server | durable | F | 1 | 0.49 k | read 0.14 ms / 0.21 ms; rmw 3.9 ms / 6.4 ms |

| fenec-server | durable | F | 4 | 1.04 k | read 0.16 ms / 0.50 ms; rmw 7.5 ms / 11.9 ms |

| fenec-server | durable | F | 16 | 4.05 k | read 0.30 ms / 0.75 ms; rmw 7.7 ms / 12.5 ms |

| fenec-server | buffered | A | 1 | 44.2 k | read 0.02 ms / 0.03 ms; update 0.02 ms / 0.04 ms |

| fenec-server | buffered | A | 4 | 107 k | read 0.03 ms / 0.09 ms; update 0.03 ms / 0.10 ms |

| fenec-server | buffered | A | 16 | 110 k | read 0.10 ms / 0.54 ms; update 0.13 ms / 0.70 ms |

| fenec-server | buffered | B | 1 | 44.0 k | read 0.02 ms / 0.03 ms; update 0.03 ms / 0.04 ms |

| fenec-server | buffered | B | 4 | 123 k | read 0.03 ms / 0.08 ms; update 0.03 ms / 0.09 ms |

| fenec-server | buffered | B | 16 | 154 k | read 0.09 ms / 0.26 ms; update 0.11 ms / 0.37 ms |

| fenec-server | buffered | C | 1 | 44.1 k | read 0.02 ms / 0.03 ms |

| fenec-server | buffered | C | 4 | 123 k | read 0.03 ms / 0.08 ms |

| fenec-server | buffered | C | 16 | 155 k | read 0.10 ms / 0.24 ms |

| fenec-server | buffered | D | 1 | 44.0 k | read 0.02 ms / 0.03 ms; insert 0.03 ms / 0.04 ms |

| fenec-server | buffered | D | 4 | 118 k | read 0.03 ms / 0.08 ms; insert 0.04 ms / 0.10 ms |

| fenec-server | buffered | D | 16 | 149 k | read 0.09 ms / 0.35 ms; insert 0.12 ms / 0.46 ms |

| fenec-server | buffered | E | 1 | 17.8 k | insert 0.03 ms / 0.07 ms; scan 0.06 ms / 0.10 ms |

| fenec-server | buffered | E | 4 | 47.0 k | insert 0.09 ms / 0.19 ms; scan 0.08 ms / 0.18 ms |

| fenec-server | buffered | E | 16 | 55.2 k | insert 0.24 ms / 2.6 ms; scan 0.19 ms / 2.1 ms |

| fenec-server | buffered | F | 1 | 30.1 k | read 0.02 ms / 0.03 ms; rmw 0.05 ms / 0.06 ms |

| fenec-server | buffered | F | 4 | 72.8 k | read 0.03 ms / 0.09 ms; rmw 0.07 ms / 0.15 ms |

| fenec-server | buffered | F | 16 | 81.3 k | read 0.09 ms / 0.49 ms; rmw 0.23 ms / 1.1 ms |

| fenec-server in Docker | durable | A | 1 | 3.34 k | read 0.24 ms / 0.64 ms; update 0.33 ms / 1.1 ms |

| fenec-server in Docker | durable | A | 4 | 7.88 k | read 0.32 ms / 1.5 ms; update 0.54 ms / 1.9 ms |

| fenec-server in Docker | durable | A | 16 | 13.3 k | read 0.63 ms / 2.4 ms; update 1.4 ms / 5.7 ms |

| fenec-server in Docker | durable | B | 1 | 4.50 k | read 0.20 ms / 0.84 ms; update 0.31 ms / 1.2 ms |

| fenec-server in Docker | durable | B | 4 | 8.21 k | read 0.34 ms / 1.7 ms; update 0.53 ms / 2.0 ms |

| fenec-server in Docker | durable | B | 16 | 18.2 k | read 0.68 ms / 2.9 ms; update 1.1 ms / 6.0 ms |

| fenec-server in Docker | durable | D | 1 | 4.86 k | read 0.18 ms / 0.76 ms; insert 0.29 ms / 1.1 ms |

| fenec-server in Docker | durable | D | 4 | 8.06 k | read 0.34 ms / 1.7 ms; insert 0.55 ms / 2.2 ms |

| fenec-server in Docker | durable | D | 16 | 18.0 k | read 0.69 ms / 3.0 ms; insert 1.2 ms / 6.2 ms |

| fenec-server in Docker | durable | E | 1 | 1.69 k | insert 0.45 ms / 2.5 ms; scan 0.44 ms / 2.6 ms |

| fenec-server in Docker | durable | E | 4 | 2.67 k | insert 1.1 ms / 7.8 ms; scan 1.0 ms / 7.6 ms |

| fenec-server in Docker | durable | E | 16 | 2.72 k | insert 4.0 ms / 21.6 ms; scan 4.2 ms / 25.8 ms |

| fenec-server in Docker | durable | F | 1 | 2.36 k | read 0.23 ms / 0.72 ms; rmw 0.56 ms / 1.7 ms |

| fenec-server in Docker | durable | F | 4 | 5.13 k | read 0.34 ms / 1.6 ms; rmw 0.92 ms / 2.8 ms |

| fenec-server in Docker | durable | F | 16 | 9.43 k | read 0.69 ms / 2.9 ms; rmw 2.2 ms / 9.9 ms |

| fenec-server in Docker | buffered | A | 1 | 3.06 k | read 0.29 ms / 1.3 ms; update 0.30 ms / 1.4 ms |

| fenec-server in Docker | buffered | A | 4 | 7.78 k | read 0.38 ms / 1.8 ms; update 0.39 ms / 1.8 ms |

| fenec-server in Docker | buffered | A | 16 | 16.5 k | read 0.76 ms / 3.5 ms; update 0.82 ms / 3.7 ms |

| fenec-server in Docker | buffered | B | 1 | 3.08 k | read 0.29 ms / 1.4 ms; update 0.30 ms / 1.3 ms |

| fenec-server in Docker | buffered | B | 4 | 8.10 k | read 0.37 ms / 1.8 ms; update 0.38 ms / 1.8 ms |

| fenec-server in Docker | buffered | B | 16 | 17.5 k | read 0.72 ms / 3.2 ms; update 0.79 ms / 3.4 ms |

| fenec-server in Docker | buffered | C | 1 | 3.07 k | read 0.29 ms / 1.3 ms |

| fenec-server in Docker | buffered | C | 4 | 8.22 k | read 0.37 ms / 1.7 ms |

| fenec-server in Docker | buffered | C | 16 | 18.4 k | read 0.69 ms / 2.9 ms |

| fenec-server in Docker | buffered | D | 1 | 3.10 k | read 0.29 ms / 1.3 ms; insert 0.32 ms / 1.4 ms |

| fenec-server in Docker | buffered | D | 4 | 7.94 k | read 0.37 ms / 1.8 ms; insert 0.39 ms / 1.8 ms |

| fenec-server in Docker | buffered | D | 16 | 17.7 k | read 0.71 ms / 3.1 ms; insert 0.79 ms / 3.5 ms |

| fenec-server in Docker | buffered | E | 1 | 1.49 k | insert 0.35 ms / 1.7 ms; scan 0.53 ms / 2.8 ms |

| fenec-server in Docker | buffered | E | 4 | 2.55 k | insert 0.87 ms / 6.1 ms; scan 1.1 ms / 8.8 ms |

| fenec-server in Docker | buffered | E | 16 | 2.59 k | insert 3.7 ms / 22.2 ms; scan 4.3 ms / 27.7 ms |

| fenec-server in Docker | buffered | F | 1 | 2.07 k | read 0.29 ms / 1.3 ms; rmw 0.59 ms / 1.9 ms |

| fenec-server in Docker | buffered | F | 4 | 5.29 k | read 0.38 ms / 1.8 ms; rmw 0.78 ms / 2.5 ms |

| fenec-server in Docker | buffered | F | 16 | 11.2 k | read 0.75 ms / 3.6 ms; rmw 1.6 ms / 7.3 ms |

| PostgreSQL | durable | A | 1 | 2.52 k | read 0.30 ms / 1.1 ms; update 0.42 ms / 1.4 ms |

| PostgreSQL | durable | A | 4 | 5.97 k | read 0.42 ms / 1.7 ms; update 0.72 ms / 3.3 ms |

| PostgreSQL | durable | A | 16 | 10.5 k | read 0.86 ms / 3.2 ms; update 1.7 ms / 10.7 ms |

| PostgreSQL | durable | B | 1 | 3.09 k | read 0.28 ms / 1.1 ms; update 0.41 ms / 5.5 ms |

| PostgreSQL | durable | B | 4 | 7.12 k | read 0.40 ms / 1.8 ms; update 0.67 ms / 7.0 ms |

| PostgreSQL | durable | B | 16 | 13.0 k | read 0.91 ms / 4.4 ms; update 1.6 ms / 17.7 ms |

| PostgreSQL | durable | D | 1 | 2.77 k | read 0.27 ms / 1.3 ms; insert 0.46 ms / 8.2 ms |

| PostgreSQL | durable | D | 4 | 7.07 k | read 0.38 ms / 1.8 ms; insert 0.64 ms / 5.9 ms |

| PostgreSQL | durable | D | 16 | 14.8 k | read 0.86 ms / 3.5 ms; insert 1.5 ms / 10.2 ms |

| PostgreSQL | durable | E | 1 | 1.28 k | insert 0.51 ms / 3.2 ms; scan 0.63 ms / 3.4 ms |

| PostgreSQL | durable | E | 4 | 2.04 k | insert 1.3 ms / 10.9 ms; scan 1.5 ms / 10.4 ms |

| PostgreSQL | durable | E | 16 | 2.21 k | insert 4.8 ms / 26.3 ms; scan 5.5 ms / 30.5 ms |

| PostgreSQL | durable | F | 1 | 1.74 k | read 0.31 ms / 1.1 ms; rmw 0.70 ms / 3.2 ms |

| PostgreSQL | durable | F | 4 | 4.20 k | read 0.42 ms / 1.8 ms; rmw 1.1 ms / 5.5 ms |

| PostgreSQL | durable | F | 16 | 7.91 k | read 0.88 ms / 3.5 ms; rmw 2.5 ms / 12.0 ms |

| PostgreSQL | buffered | A | 1 | 2.98 k | read 0.30 ms / 1.3 ms; update 0.31 ms / 1.2 ms |

| PostgreSQL | buffered | A | 4 | 7.04 k | read 0.42 ms / 1.9 ms; update 0.42 ms / 1.9 ms |

| PostgreSQL | buffered | A | 16 | 13.7 k | read 0.95 ms / 4.8 ms; update 0.96 ms / 4.9 ms |

| PostgreSQL | buffered | B | 1 | 2.99 k | read 0.28 ms / 1.4 ms; update 0.29 ms / 1.4 ms |

| PostgreSQL | buffered | B | 4 | 7.16 k | read 0.40 ms / 1.9 ms; update 0.41 ms / 1.9 ms |

| PostgreSQL | buffered | B | 16 | 13.7 k | read 0.91 ms / 4.8 ms; update 0.95 ms / 4.8 ms |

| PostgreSQL | buffered | C | 1 | 3.14 k | read 0.27 ms / 1.3 ms |

| PostgreSQL | buffered | C | 4 | 7.20 k | read 0.40 ms / 1.9 ms |

| PostgreSQL | buffered | C | 16 | 14.8 k | read 0.89 ms / 4.1 ms |

| PostgreSQL | buffered | D | 1 | 2.82 k | read 0.28 ms / 1.4 ms; insert 0.30 ms / 1.3 ms |

| PostgreSQL | buffered | D | 4 | 7.04 k | read 0.40 ms / 1.9 ms; insert 0.44 ms / 2.0 ms |

| PostgreSQL | buffered | D | 16 | 14.4 k | read 0.90 ms / 4.1 ms; insert 1.00 ms / 4.6 ms |

| PostgreSQL | buffered | E | 1 | 1.26 k | insert 0.30 ms / 1.9 ms; scan 0.63 ms / 3.4 ms |

| PostgreSQL | buffered | E | 4 | 2.00 k | insert 0.90 ms / 8.7 ms; scan 1.5 ms / 11.2 ms |

| PostgreSQL | buffered | E | 16 | 2.07 k | insert 4.4 ms / 28.7 ms; scan 5.9 ms / 34.3 ms |

| PostgreSQL | buffered | F | 1 | 2.09 k | read 0.27 ms / 1.4 ms; rmw 0.54 ms / 2.0 ms |

| PostgreSQL | buffered | F | 4 | 4.76 k | read 0.42 ms / 1.9 ms; rmw 0.89 ms / 3.1 ms |

| PostgreSQL | buffered | F | 16 | 9.22 k | read 0.95 ms / 4.4 ms; rmw 1.9 ms / 8.8 ms |

| MongoDB | durable | A | 1 | 1.97 k | read 0.40 ms / 1.3 ms; update 0.53 ms / 1.7 ms |

| MongoDB | durable | A | 4 | 4.37 k | read 0.51 ms / 2.1 ms; update 0.84 ms / 4.1 ms |

| MongoDB | durable | A | 16 | 6.56 k | read 0.99 ms / 4.5 ms; update 2.2 ms / 15.5 ms |

| MongoDB | durable | B | 1 | 2.48 k | read 0.35 ms / 1.4 ms; update 0.57 ms / 1.8 ms |

| MongoDB | durable | B | 4 | 6.12 k | read 0.47 ms / 1.9 ms; update 0.82 ms / 3.1 ms |

| MongoDB | durable | B | 16 | 11.1 k | read 1.1 ms / 5.0 ms; update 2.2 ms / 12.3 ms |

| MongoDB | durable | D | 1 | 2.79 k | read 0.31 ms / 1.3 ms; insert 0.54 ms / 2.0 ms |

| MongoDB | durable | D | 4 | 6.50 k | read 0.44 ms / 1.9 ms; insert 0.90 ms / 3.1 ms |

| MongoDB | durable | D | 16 | 12.0 k | read 1.0 ms / 4.4 ms; insert 2.4 ms / 11.9 ms |

| MongoDB | durable | E | 1 | 0.84 k | insert 0.78 ms / 4.9 ms; scan 1.0 ms / 5.1 ms |

| MongoDB | durable | E | 4 | 1.94 k | insert 1.7 ms / 12.6 ms; scan 1.5 ms / 10.0 ms |

| MongoDB | durable | E | 16 | 2.01 k | insert 5.7 ms / 29.2 ms; scan 6.0 ms / 31.9 ms |

| MongoDB | durable | F | 1 | 1.33 k | read 0.39 ms / 1.5 ms; rmw 0.92 ms / 2.8 ms |

| MongoDB | durable | F | 4 | 3.12 k | read 0.51 ms / 2.0 ms; rmw 1.4 ms / 5.7 ms |

| MongoDB | durable | F | 16 | 5.00 k | read 1.1 ms / 5.1 ms; rmw 3.5 ms / 34.3 ms |

| MongoDB | buffered | A | 1 | 2.20 k | read 0.41 ms / 1.5 ms; update 0.41 ms / 1.5 ms |

| MongoDB | buffered | A | 4 | 5.22 k | read 0.52 ms / 2.8 ms; update 0.52 ms / 2.8 ms |

| MongoDB | buffered | A | 16 | 8.90 k | read 1.2 ms / 9.6 ms; update 1.3 ms / 10.0 ms |

| MongoDB | buffered | B | 1 | 2.22 k | read 0.41 ms / 1.5 ms; update 0.45 ms / 1.6 ms |

| MongoDB | buffered | B | 4 | 6.06 k | read 0.50 ms / 2.0 ms; update 0.53 ms / 2.1 ms |

| MongoDB | buffered | B | 16 | 11.8 k | read 1.1 ms / 5.2 ms; update 1.2 ms / 5.5 ms |

| MongoDB | buffered | C | 1 | 2.14 k | read 0.42 ms / 1.5 ms |

| MongoDB | buffered | C | 4 | 6.52 k | read 0.49 ms / 1.9 ms |

| MongoDB | buffered | C | 16 | 12.9 k | read 1.0 ms / 4.6 ms |

| MongoDB | buffered | D | 1 | 2.22 k | read 0.40 ms / 1.5 ms; insert 0.43 ms / 1.5 ms |

| MongoDB | buffered | D | 4 | 6.67 k | read 0.48 ms / 1.9 ms; insert 0.48 ms / 1.9 ms |

| MongoDB | buffered | D | 16 | 13.2 k | read 1.0 ms / 4.2 ms; insert 1.0 ms / 4.6 ms |

| MongoDB | buffered | E | 1 | 0.85 k | insert 0.47 ms / 2.3 ms; scan 1.0 ms / 4.9 ms |

| MongoDB | buffered | E | 4 | 1.96 k | insert 0.92 ms / 7.6 ms; scan 1.5 ms / 10.9 ms |

| MongoDB | buffered | E | 16 | 2.02 k | insert 4.4 ms / 26.3 ms; scan 6.0 ms / 33.8 ms |

| MongoDB | buffered | F | 1 | 1.40 k | read 0.43 ms / 1.6 ms; rmw 0.86 ms / 2.4 ms |

| MongoDB | buffered | F | 4 | 3.69 k | read 0.52 ms / 2.7 ms; rmw 1.1 ms / 5.1 ms |

| MongoDB | buffered | F | 16 | 6.60 k | read 1.2 ms / 8.8 ms; rmw 2.4 ms / 14.6 ms |

| Redis | durable | A | 1 | 2.03 k | read 0.28 ms / 1.0 ms; update 0.65 ms / 1.8 ms |

| Redis | durable | A | 4 | 3.88 k | read 0.84 ms / 2.6 ms; update 0.98 ms / 2.9 ms |

| Redis | durable | A | 16 | 8.53 k | read 1.6 ms / 6.5 ms; update 1.6 ms / 6.5 ms |

| Redis | durable | B | 1 | 3.51 k | read 0.24 ms / 1.1 ms; update 0.62 ms / 1.8 ms |

| Redis | durable | B | 4 | 6.26 k | read 0.40 ms / 2.0 ms; update 0.83 ms / 2.9 ms |

| Redis | durable | B | 16 | 9.25 k | read 1.4 ms / 6.8 ms; update 1.8 ms / 8.8 ms |

| Redis | durable | D | 1 | 3.59 k | read 0.23 ms / 1.0 ms; insert 0.69 ms / 2.1 ms |

| Redis | durable | D | 4 | 4.92 k | read 0.55 ms / 3.1 ms; insert 1.2 ms / 5.9 ms |

| Redis | durable | D | 16 | 13.0 k | read 1.1 ms / 4.2 ms; insert 1.4 ms / 5.5 ms |

| Redis | durable | E | 1 | 0.65 k | insert 1.1 ms / 5.5 ms; scan 1.4 ms / 6.1 ms |

| Redis | durable | E | 4 | 1.26 k | insert 1.6 ms / 10.9 ms; scan 2.5 ms / 14.2 ms |

| Redis | durable | E | 16 | 0.93 k | insert 6.5 ms / 47.4 ms; scan 13.6 ms / 64.2 ms |

| Redis | durable | F | 1 | 1.60 k | read 0.28 ms / 1.2 ms; rmw 0.88 ms / 2.4 ms |

| Redis | durable | F | 4 | 3.00 k | read 0.72 ms / 2.4 ms; rmw 1.6 ms / 4.4 ms |

| Redis | durable | F | 16 | 4.93 k | read 1.8 ms / 8.7 ms; rmw 3.6 ms / 16.2 ms |

| Redis | buffered | A | 1 | 2.74 k | read 0.32 ms / 1.4 ms; update 0.32 ms / 1.5 ms |

| Redis | buffered | A | 4 | 6.87 k | read 0.46 ms / 2.0 ms; update 0.45 ms / 2.0 ms |

| Redis | buffered | A | 16 | 8.35 k | read 1.6 ms / 7.2 ms; update 1.6 ms / 7.2 ms |

| Redis | buffered | B | 1 | 2.75 k | read 0.32 ms / 1.5 ms; update 0.34 ms / 1.5 ms |

| Redis | buffered | B | 4 | 4.82 k | read 0.63 ms / 3.1 ms; update 0.69 ms / 3.2 ms |

| Redis | buffered | B | 16 | 13.5 k | read 0.94 ms / 4.2 ms; update 0.96 ms / 4.2 ms |

| Redis | buffered | C | 1 | 2.73 k | read 0.32 ms / 1.5 ms |

| Redis | buffered | C | 4 | 6.74 k | read 0.45 ms / 2.0 ms |

| Redis | buffered | C | 16 | 14.0 k | read 0.92 ms / 4.0 ms |

| Redis | buffered | D | 1 | 2.85 k | read 0.31 ms / 1.4 ms; insert 0.38 ms / 1.5 ms |

| Redis | buffered | D | 4 | 4.95 k | read 0.62 ms / 2.9 ms; insert 0.72 ms / 3.5 ms |

| Redis | buffered | D | 16 | 15.2 k | read 0.88 ms / 3.2 ms; insert 0.91 ms / 3.6 ms |

| Redis | buffered | E | 1 | 0.62 k | insert 0.44 ms / 1.9 ms; scan 1.5 ms / 6.2 ms |

| Redis | buffered | E | 4 | 1.21 k | insert 0.99 ms / 8.8 ms; scan 2.6 ms / 15.7 ms |

| Redis | buffered | E | 16 | 1.04 k | insert 5.1 ms / 35.4 ms; scan 11.9 ms / 59.5 ms |

| Redis | buffered | F | 1 | 1.52 k | read 0.36 ms / 1.6 ms; rmw 0.73 ms / 2.3 ms |

| Redis | buffered | F | 4 | 4.59 k | read 0.45 ms / 2.0 ms; rmw 0.91 ms / 3.3 ms |

| Redis | buffered | F | 16 | 5.53 k | read 1.6 ms / 7.4 ms; rmw 3.4 ms / 14.6 ms |

| Redis | snapshot | A | 1 | 2.80 k | read 0.32 ms / 1.4 ms; update 0.31 ms / 1.4 ms |

| Redis | snapshot | A | 4 | 6.96 k | read 0.44 ms / 2.0 ms; update 0.44 ms / 1.9 ms |

| Redis | snapshot | A | 16 | 8.60 k | read 1.5 ms / 7.3 ms; update 1.5 ms / 7.2 ms |

| Redis | snapshot | B | 1 | 2.76 k | read 0.31 ms / 1.5 ms; update 0.33 ms / 1.4 ms |

| Redis | snapshot | B | 4 | 4.98 k | read 0.61 ms / 3.0 ms; update 0.65 ms / 2.7 ms |

| Redis | snapshot | B | 16 | 14.4 k | read 0.91 ms / 3.7 ms; update 0.92 ms / 3.8 ms |

| Redis | snapshot | D | 1 | 2.71 k | read 0.31 ms / 1.5 ms; insert 0.38 ms / 1.6 ms |

| Redis | snapshot | D | 4 | 7.30 k | read 0.41 ms / 1.9 ms; insert 0.44 ms / 2.0 ms |

| Redis | snapshot | D | 16 | 15.5 k | read 0.87 ms / 3.1 ms; insert 0.90 ms / 3.3 ms |

| Redis | snapshot | E | 1 | 0.62 k | insert 0.41 ms / 2.4 ms; scan 1.5 ms / 6.5 ms |

| Redis | snapshot | E | 4 | 1.22 k | insert 0.99 ms / 7.2 ms; scan 2.6 ms / 15.4 ms |

| Redis | snapshot | E | 16 | 1.04 k | insert 4.9 ms / 38.5 ms; scan 11.9 ms / 62.1 ms |

| Redis | snapshot | F | 1 | 1.87 k | read 0.32 ms / 1.5 ms; rmw 0.63 ms / 2.0 ms |

| Redis | snapshot | F | 4 | 3.58 k | read 0.59 ms / 2.5 ms; rmw 1.2 ms / 4.8 ms |

| Redis | snapshot | F | 16 | 9.20 k | read 0.93 ms / 4.1 ms; rmw 2.0 ms / 7.8 ms |

## Writers and readers at once

fenecdb and SQLite both allow one writer at a time. This measures what
that costs when many want to write, and how fast readers go meanwhile.

|   | fenecdb | SQLite |

| durable writes/s, 1 / 4 / 16 writers | 252 / 487 / **516** | 248 / 248 / 258 |

| writes/s without an fsync, 1 / 4 / 16 writers | **1 497 k** / 821 k / 611 k | 128 k / 128 k / 126 k |

| reads/s from 4 threads, alone | **7 061 k**, p99 0.8 µs | 1 111 k, p99 16.5 µs |

| … beside 4 writers | **701 k**, p99 75 µs | 344 k, p99 79 µs |

| … beside blocks of 1 000 writes | **2 010 k**, p99 2.5 µs, the longest 0.5–1.2 ms | 630 k, p99 59 µs, the longest 0.4–1.9 ms |

- **Durable writes scale with the writers.** Sixteen writers share
    their disk syncs; SQLite's stay flat.

- **A block of writes holds the readers only while it lands.** Its
    rows are built and encoded beside them first, and its buffer is written
    after the lock is let go.

- **A long read does not hold the writers.** It reads what it pinned
    with no lock held, so a write waits milliseconds beside it, not the
    read's whole length.

Writers beside long reads: four threads each putting a row every half a
millisecond into a million events, two reading one by id as often, and a long
read every second — the longest any write waited, under the lock as before
and pinned, over two runs.

|   | the read | under the lock | pinned |

| a read-only batch of three aggregates | 151–158 ms | 183–389 ms | **4.4–5.2 ms** |

| an aggregate of 50 000 groups | 0.66–0.75 s | 800–854 ms | **2.5–9.0 ms** |

| a full scan | 61–65 ms | 118–119 ms | **2.8–10.8 ms** |

Method and details

fenecdb and SQLite both take one writer at a time. What that costs when
many want to write, and what readers wait meanwhile, measured in one process:
fenecdb's threads share one database behind a lock, as `fenec-server`
does; SQLite's hold a connection each, WAL, with a 30 s busy timeout. A row a
write, a row by id a read, one run on an M1 after idle minutes. Durable is an fsync
per commit, `F_FULLFSYNC` for both (Rust's `sync_data`
on macOS; `PRAGMA fullfsync` for SQLite, `synchronous=FULL`);
without one, fenecdb's writes stay in its buffer and SQLite's reach the
operating system (`synchronous=NORMAL`).

```bash
make concurrency-bench
```

**Durable writes scale with the writers.** fenecdb's fsync runs after
its lock is released, and a write an earlier fsync already covered waits for
none, so sixteen writers share their fsyncs; SQLite syncs a commit while it
holds the write lock. Without an fsync, one fenecdb writer is twelve times
SQLite's, and handing the lock between sixteen costs more than half of that. A
write, or a `/batch` of them, holds the
lock only while it runs — microseconds for a row — and there is no
transaction held open across requests to keep the readers or the other
writers waiting. A block of 1 000 writes — a `/batch`'s — held
it while it ran, about half a millisecond, and readers waited for it, where
SQLite's WAL reads the last commit meanwhile: 213 k reads a second, the
longest wait 4.8 ms. Most of that half millisecond had nothing to do with the
database as the lock holds it — each row built from the statement and its
parameters, its values made the fields' types, encoded, and freed once
written — and a block's rows are now built and encoded under the shared
lock, beside the readers, the write lock held only to land them, about
0.15 ms; whatever the database decides, the ids, a value already held, the
indexes, is decided under the lock as before, and the file is the same byte
for byte. A REST insert of 1 000 rows read its body under the write lock as
well: a read beside them waited 0.4 ms at the median for rows of a text and
an int, 3.3 ms with a 128-dim vector each, where the body is now read with
no lock held and its rows built beside the readers, 0.08 and 0.2 ms. The
native library builds an app's write of many rows the same way, and a read
beside writes of 1 000 128-dim rows waits 0.3 ms rather than 0.9. The rest
of the longest waits were the disk: on macOS a write that
takes a file's dirty pages past each 8 MB waits for it, 1.5 to 11 ms, and the
buffer was written under the lock as it filled. It is written once the lock
is let go now, and the handover — the documents written since the open
pointed at their places in the file — runs in pieces of 16 384 writes,
0.3 ms each, as the next block starts, with nothing left to write. Until
it also ran by count the longest wait was 17.4 ms: it ran once the
documents reached 16 MB, which for rows this small was 930 000 of them.

**A long read is pinned.** A read took the shared lock for as long as
it ran, and a write waited for every read, so an aggregate over a million
events held every write — and every read by id behind a waiting write — for
its whole 0.8 s. Now a long read takes the shared lock only to pin what it
reads: each collection's id index copied, 12 bytes a row, its stored
documents shared, 0.26 to 0.87 ms for the million rows. It then reads with
no lock held, on the database as it stood at the pin, and answers what the
read under the lock would have; alone it takes what it took, 151.9 ms
against 151.4 for the batch. The writes beside it waited at most what they
wait with no long read at all, a few milliseconds of handovers and the copy
of the segment the pin shares. A read an index answers is short, and still
reads under the lock: indexes change in place, and copying one would cost
the read it spares. The same over HTTP, a ledger's reconciliation every
second beside four clients sending transfers (`make
recon-bench`), four runs each in turns: the transfers' longest wait
went from 187–512 ms, 36 of them past 50 ms in every run, to 5.4–29 ms and
none in three runs; the fourth met one stall of 385 ms, four transfers past
50 ms, of the kind both servers met in the round with no reconciliation at
all (118–223 ms). Those had two causes, both gone: in three runs since, the
transfers' longest wait was 1.9–10.4 ms, none past 50.

**A write waits for no index's growth and no fsync.** A
`@unique` index holds a key a row, and its table grew whole: the
write that found it full moved every key into one twice its size, under the
write lock. The ledger's journal passing 1.8 million entries held one
transfer, and every request behind it, 120 to 137 ms in every run; four
million puts one at a time waited 7, 19, 49, 109 and 235–267 ms at each
doubling from 229 376 keys (`make growth-bench`). Past 114 688
keys the table is now 256 that grow on their own, and the longest put is
2.5 to 3.6 ms. Each key keeps its hash beside it, so a lookup hashes it
once — 83 to 93 ns over four million keys against 121 to 144 in the
shards before, 24 to 34 under that size against 33 to 42 — and a table
grows without hashing a key again, for 8 bytes a key. A text index's terms grew the same way, 276 to
324 ms at 3.7 million terms, and are 256 tables now too, the longest put
5 ms. And a vector index's table of the vectors it holds, which finds one
written again, was made again from every vector once half full: 242 to
281 ms at a million vectors of 128 dimensions, 1.45 s at 524 288 of 768.
It grows from what it holds now, a 256th at a time, and no put waits for
it. The first write after an open made it by reading every vector once —
275 to 441 ms over 1.1 million of 128 dimensions, 33 to 169 on every core;
it makes it now from four bytes a vector the file keeps beside the graph,
and that first write took 16 to 31 ms there, 4.4 to 27 over 300 000 of
768 against 68 to 312, the open as fast as it was.
And the two writes still made under the lock —
the 1 MB buffer's once it is full, and a handover's — waited for the
disk while the server's periodic fsync held it, 52 ms of macOS's
`F_FULLFSYNC`; they are put off until it is done now. The rest
of those first-round stalls was the bench's own: `--warm` built
an index its warm-up had not read just as the round began, and it reads
that one too now.

## A request over HTTP

What one request to `fenec-server` costs a client on the same
machine, over HTTP. Most of each round trip is the network stack, not the
server's work.

|   | HTTP, p50 | 8 clients |

| a row by id | **0.022 ms** | 149 268/s |

| filter, limit 10 | **0.032 ms** | 107 322/s |

| `near`, top 10 | **0.057 ms** | 66 071/s |

| put a row | **0.022 ms** | 129 476/s |

Method and details

What one request costs a client of `fenec-server`, on loopback: a
row by id, a filter with a limit, a `near` for the top ten and a
put, over 10 000 rows of 128 dimensions, asked by one client at a time — the
round trip's p50 — and by eight at once. Each is `POST /query`
with the parameters beside the text, on a kept-alive connection. One run
on an M1, after idle minutes.

```bash
make pgvector-up      # optional: PostgreSQL's arm, skipped without it
make requests-bench
```

**Most of a round trip is the loopback.** For a row by id the
server's own work is a few microseconds: one read and one write on the
socket, and the statement. `POST /query` keeps the statement a
text parsed to, so it does not read its text again for each request, and an
answer goes out in one write.

PostgreSQL with pgvector, asked the same in SQL, answers a row by id in
0.142 ms and a `near` in 0.653 ms, from
inside Docker's virtual machine on the same Mac, where its empty query alone
takes 0.127 ms. The network, not the engine, is most of that
difference, and the bench prints the empty query beside the rest so that it
reads that way.

## Loading rows

Loading 100 000 rows of a text, an int and a 128-dim vector, every way
a client can send them. With no vector index they go in as JSON arrays
over HTTP at 212 820 rows a second, against PostgreSQL's `COPY`
at 56 232; with an HNSW index kept up to date as they land, at about
18 000 to 19 000, against pgvector's 410.

|   | no vector index, rows/s | HNSW kept |

| in process, `put` of 1 000 | 921 665 | 20 425 |

| HTTP, `POST /docs` of 1 000 | 212 820 | 19 151 |

| HTTP, `POST /query`, a `put` of 1 000 | 94 662 | 17 483 |

| HTTP, `POST /batch` of single puts | 145 811 | 18 325 |

| PostgreSQL + pgvector, `COPY` | 56 232 | 410 |

| PostgreSQL + pgvector, `INSERT` of 1 000 | 29 662 | 403 |

- **Without a vector index, reading the vectors' text is the
    cost.**

- **With one, the index is the cost.** A batch of single puts goes
    as fast as an array.

Method and details

What loading 100 000 rows of a text, an int and a 128-dim vector costs,
each way a client can send them, into an empty collection: with the
vector's HNSW graph kept as the rows land, and with no vector index.
`fenec-server` on loopback, 1 000 rows a request —
`POST /docs` with a JSON array; a `put` of 1 000 sent
to `POST /query`; a `POST /batch` of single puts.
PostgreSQL with pgvector is sent the same rows by `COPY` and by
`INSERT`s of 1 000, with `synchronous_commit = off`, as
fenec-server's writes reach the disk within `--sync 250`. One
run on an M1, after idle minutes.

```bash
make pgvector-up      # optional: PostgreSQL's arm, skipped without it
make load-bench
```

**Without a vector index, the vectors' text is the cost.** Each
row's vector is 128 numbers written out, read straight into its
`f32`s wherever it comes from — a JSON array, a parameter, a
FenecQL literal — in one pass that finds each number's end. In process the
rows arrive parsed, which is what the rest can at most reach. PostgreSQL
answers from inside Docker's virtual machine on the same Mac, whose network
carries every byte.

**With the graph kept, the graph is the cost.** Every way lands
close to linking the vectors in process. A `/batch`'s vectors are
linked together, 512 at a time on every core, whether they came a row a
statement or 1 000 a put: linked a row at a time, as they were before, the
same rows went in at 5 400 rows/s. PostgreSQL's HNSW index takes each row on
its own, on one core.

## Text search

What a search box costs: `match` ranked by BM25, the marks
of what it matched, the facets beside it, and the words a typo or a half
typed word reaches. In one process on an M1.

| 100 000 documents of 30 words | median |

| `match … limit 10`, two words | **0.06 ms** |

| … with `highlight()`, a row's marks | 1–3 µs more |

| … with `snippet()`, a row's | 2–4 µs more |

| … with two facets over what it reaches, about 16 667 rows | 1.6 ms |

| a facet over every row, through `@hash` | **0.01 ms** |

| … read from every row | 6.1 ms |

| `fuzzy` and `prefix` |   |

| a word's neighbours among FiQA's 135 500 words, one edit | **14 µs** |

| … two edits | 120 µs |

| … the words it begins | 4 µs |

| the sorted words, made at the first loose `match`, 60 000 words | 6 ms, 1.3 MB |

- **A query that asks for no marks runs as it did.** The marks read
    the row's text again by the index's own tokenizer; nothing is kept for
    them in the index.

- **How good the answers are** is the next section: BM25 on two BEIR
    datasets, with a typo in each query and half typed, beside the
    vectors.

Method and details

100 000 documents of 30 words each over a vocabulary of 5 000, the
commoner words more common, a `brand` under `@hash`
(40 values) and a `color` with no index (12). Each figure is the
median of 15 runs over 200 two-word queries; a row's marks are the
difference over the ten rows of `match` alone. The words a typo
reaches are timed over FiQA's own words, 2 000 query words with a typo in
each, and the sorted words over 100 000 documents of made-up words, many an
edit or two from another, as a language's are (Typos and words begun, Highlights and snippets, Facets).

```bash
make search-bench
make fuzzy-bench BEIR=path/to/fiqa
```

## Retrieval quality

How good the answers are, not how fast: every way fenecdb ranks
documents, scored by nDCG@10 on two public BEIR datasets. Fusing keyword
and vector search (`fuse`) is the one path near the top of
both.

|   | SciFact | FiQA | FiQA p50 |

| `match`, BM25 | 0.662 | 0.232 | 0.26 ms |

| `near`, the graph | 0.645 | 0.366 | 0.37 ms |

| `near … exact` | 0.645 | 0.369 | 4.99 ms |

| `match … rerank`, 50 candidates | 0.656 | 0.337 | 0.45 ms |

| `match … rerank`, 200 (default) | 0.646 | 0.361 | 0.78 ms |

| `match … rerank`, 1 000 | 0.649 | 0.369 | 1.82 ms |

| `match … near … fuse`, 10 a side | **0.700** | 0.364 | 0.67 ms |

| `match … near … fuse`, 20 (default) | 0.700 | 0.366 | 0.74 ms |

| `match … near … fuse`, 100 | 0.688 | 0.359 | 0.97 ms |

| `match … near … fuse`, 1 000 | 0.690 | 0.364 | 4.03 ms |

| `near`, sparse (SPLADE++) | 0.693 | 0.331 | 0.68 ms |

| `match … near … fuse`, sparse | 0.698 | 0.314 | 1.07 ms |

With a typo in each query, and with each query's last word
half typed, as a search box sends it: `fuzzy` and
`prefix` take back most of what is lost, and leave the queries
written whole where they were.

|   | SciFact | FiQA | FiQA p50 |

| `match`, as written | 0.662 | 0.232 | 0.26 ms |

| `match … fuzzy`, as written | 0.665 | 0.233 | 0.74 ms |

| `match … prefix`, as written | 0.663 | 0.234 | 0.32 ms |

| `match`, a typo | 0.636 | 0.213 | 0.24 ms |

| `match … fuzzy`, a typo | 0.663 | 0.229 | 0.71 ms |

| `match … fuzzy 1`, a typo | **0.666** | 0.231 | 0.53 ms |

| `match`, the last word half typed | 0.616 | 0.188 | 0.26 ms |

| `match … prefix`, half typed | 0.655 | 0.208 | 0.60 ms |

| `match … fuzzy prefix`, half typed | 0.658 | 0.210 | 1.09 ms |

How the typos were made

One typo a query, by a generator seeded by the query's place
(`crates/fenec-bench/src/bin/beir.rs`): in one word of four
letters or more, a letter dropped, a key beside it on the keyboard typed as
well or instead, or two letters swapped — the four kinds Damerau found four
misspellings in five to be — and in the first letter 7% of the time, as
Pollock and Zamora found it. SciFact's 300 test queries hold 297 such
words, FiQA's 648 hold 644. Half typed is the last word cut to half its
letters, rounded up. A word of under five letters takes no edit, which is
most of what `fuzzy` leaves unrecovered; with one typo a query,
`fuzzy 1` does a little better than the two edits nine letters
allow, which pay where a long word holds two. The p50s are the whole
query's, the words found and their lists merged included.

Method and details

```bash
cd crates/fenec-bench/beir && npm install
node embed.mjs path/to/scifact     # dense: all-MiniLM-L6-v2, 90 MB
node splade.mjs path/to/scifact    # sparse: SPLADE++, 532 MB, optional
make beir BEIR=path/to/scifact
```

Two BEIR datasets as BEIR ships them — SciFact, 5 183 documents and 300
test queries, and FiQA, 57 638 and 648 — with every document and query
embedded by all-MiniLM-L6-v2 the way sentence-transformers does it, and
encoded by SPLADE++ (`prithivida/Splade_PP_en_v1`) into a
`sparse<30522>` field, both cut at 256 tokens with the
closing `[SEP]` kept. transformers.js drops it when it cuts a
text, so both scripts cut texts themselves: for SPLADE that was worth 0.46
of SciFact's nDCG@10, for the dense vectors up to 0.003. Each of the
engine's ways of ranking ten documents is then scored by nDCG@10, through
FenecQL: the answers are the engine's own.

The two datasets pull opposite ways. In SciFact a claim shares its words
with the evidence and BM25 alone beats the transformer; in FiQA a question
shares few words with its answer and BM25 is far behind. `fuse`
is the one path near the top of both. The rank offset matters less than the
depth: at 20 a side, `k` from 10 to 120 moves either score by at
most 0.005.

SPLADE sits between the two. On SciFact it comes within 0.006 of the dense
fusion with one index; on FiQA it lifts BM25 by 0.099 but stays under the
dense vectors, and fused with BM25 there it loses 0.016 — both sides are
lexical. Its queries' 37 to 65 terms reach most of the corpus, and its
lists are summed a list at a time rather than walked a document at a time:
0.68 ms, where MaxScore's walk took 2.58 (How it works).

## Against SQLite and pgvector

The next sections are search by meaning: vectors, their index and what
it costs; the sizes and the module's features follow them. The same 100 000 rows of 128 dimensions in fenecdb, SQLite and
PostgreSQL with pgvector, with the same index settings. fenecdb writes
fastest, searches 10 to 330 times faster and uses the least disk; SQLite
opens faster.

|   | fenecdb | SQLite | PostgreSQL + pgvector |

| data write, no index | **936 k rows/s** | 430 k/s | 66 k/s |

| index build | **3.4 s** HNSW | 0.03 s, B-tree only | 8.6 s HNSW |

| total | 3.6 s | **0.30 s**, no ANN | 10.1 s |

| disk size | **59.3 MB** | 59.8 MB | 145.0 MB |

| scalar filter, indexed | **1.25 ms** | 14.1 ms | 7.3 ms |

| vector top-10, **exact** | **2.9 ms** | 30.2 ms | 15.6 ms |

| vector top-10, **approximate** | **0.091 ms** | none | 1.04 ms |

| recall@10 | **100%** | — | **100%** |

| open | 6.0 ms | **0.07 ms** | server, always open |

| open, then a read by id | 6.0 ms | **0.46 ms** | — |

| open, then a vector top-10 | **6.1 ms** | 30.5 ms, exact | — |

| empty query round trip | 0, embedded | 0, embedded | 0.41 ms |

- **SQLite has no vector index.** Its vector search reads every row,
    so the table compares exact search with exact search first.

- **The slower open pays off at once.** fenecdb loads its vectors
    when it opens; from the first vector query after it, it is ahead.

Method and details

Same data, and the same index settings wherever there is an index. SQLite
3.53 and PostgreSQL 17 with pgvector 0.8.6, on an Apple M-series, 100 000 rows
× 128 dimensions with clustered embeddings. Every figure is the median of three
runs.

```bash
make pgvector-up      # start PostgreSQL + pgvector in Docker
make compare          # without it, that arm is skipped
```

**Method.**

- All three follow the same workflow: **bulk load first, index after**.
    That is already the recommended order for SQLite and PostgreSQL; in fenecdb
    `create index` exists for exactly this.

- **SQLite**: WAL with `synchronous=NORMAL`, the bulk write
    in one transaction, the index built afterwards. fenecdb does not
    `fsync` per write.

- **PostgreSQL**: in Docker, bulk load with COPY, then
    `hnsw (m=16, ef_construction=200)` — the same parameters as
    fenecdb — with `max_parallel_maintenance_workers` set so the
    build gets one process per core, as fenecdb's does: seven workers and the
    leader here. At query time `hnsw.ef_search = 100`, fenecdb's
    default.

- fenecdb and SQLite compute vector distance with the same code: SQLite's
    rows are read back and scored by fenecdb's kernel, so what separates the
    two is the cost of fetching the data, not the arithmetic.
    PostgreSQL computes it inside pgvector.

- SQLite's core has no ANN index, so a vector search there is a full scan.
    The comparison is therefore **exact against exact** first, with ANN on a
    separate row.

- The write is split in two stages, rows and index. A single "total" would
    weigh an engine that builds an ANN index and one that does not on the same
    scale.

- The PostgreSQL figures include a 0.41 ms TCP round trip, about 40% of its
    ANN time. That is not a measurement flaw, it is the client-server
    architecture.

**How to read the numbers.**

**fenecdb leads on raw write speed:** 936 k rows/s, more than twice
SQLite's. The append-only segment store reduces a write to appending to an
arena — no B-tree rebalancing, no page splits.

**fenecdb builds the index faster**: 3.4 s against pgvector's 8.6 s, at
the same `m` and `ef_construction` on the same cores. It
was the other way round, 10.2 s against 8.7, while fenecdb linked each batch
of new nodes on one core: pruning the neighbour lists they joined was 70% of
its build, and the lists are now pruned in parallel, grouped by list, into the
same graph. On aarch64 the distances are written out in NEON as well, a
walk measures its neighbours four at a time, to the same bits, and it asks
for the next neighbour list and the neighbours' vectors before it reads
them, so that one cache miss is not waited out before the next. Before that,
this page said fenecdb was faster, 10.2 s against
14.2 s — pgvector was then building with its default two workers while fenecdb
used every core. SQLite's 0.03 s is not an ANN index but a single-column
B-tree; that cost does not vanish, it is deferred to query time, where a
vector search takes 30.2 ms — 330× slower than fenecdb's ANN.

**The clearest loss is the open.** SQLite opens in 0.07 ms and
reads a row by id 0.46 ms after it, because it loads nothing until it is
asked and then reads the pages it needs. fenecdb restores its graph as it
opens: it copies the 51 MB of vectors out of the file into memory as unit
vectors, decodes the graph's links and validates the graph against the
documents, 6.0 ms, more than half of it the copy. That is the other side of
the same coin — it is why queries are 10–330× faster. Asked the same first
question, fenecdb is ahead: a vector top-10 right after the open takes 6.1
ms in all, where SQLite's scan takes 30.5. Each open figure is the median of
31 opens, the file's pages in the page cache as a close leaves them. This
page once gave 9 ms against 0.9, from one open each: fenecdb's open and a
vector query against SQLite's open and a count through its index, two
different first questions.

**fenecdb leads on disk as well**, 2.4× smaller than pgvector — though
not for the reason this page used to give. Of pgvector's 145.0 MB, 83.2 MB is
the HNSW index, which keeps its own copy of every vector so a search reads no
heap tuple. The heap is 58.7 MB, as large as fenecdb's whole file with its
graph, and its MVCC headers are about 2.8 MB of that. fenecdb's graph stores
links and nothing else; the vectors are read from the one arena. The WAL is in
neither figure.

## At scale, against pgvector

Here the server, `fenec-server`, answers over HTTP and
PostgreSQL with pgvector over its own wire, each asked by the same client
program. At a million 128-dim vectors, fenec-server loads and indexes them
2.2 times as fast, on less than half the disk, and
answers 14 to 24 times sooner.

| 1 000 000 × 128 | fenec-server, HTTP | PostgreSQL + pgvector |

| load and index | **49.6 s** | 108.5 s |

| memory the server holds | **738 MB** | 1 131 MB |

| its files' pages in memory besides | **377 MB** | 1 940 MB |

| disk | **612 MB** | 1 432 MB |

| ef 40: recall@10, p50 | 92.5%, 0.087 ms | 94.2%, 1.213 ms |

| ef 100: recall@10, p50 | **99.1%, 0.141 ms** | 98.6%, 2.076 ms |

| ef 200: recall@10, p50 | **99.8%, 0.232 ms** | 99.0%, 3.176 ms |

| a filter keeping 1%, ef 100 | **100%, 0.667 ms** | 99.0%, 16.1 ms |

| a filter keeping 25%, ef 100 | 97.2%, 0.149 ms | 97.3%, 2.146 ms |

| 8 clients, ef 100 | **18 852 queries/s** | 2 289 queries/s |

- **Recall comes from the engine,** not the transport: the same
    graph answers in process and over HTTP.

- **768-dim vectors**: 250 000 of them, in the table below.

Method and details

The [table above](#against-sqlite-and-pgvector) measures
fenecdb in process. Here one client program asks both servers the same
questions: `fenec-server` through a kept-alive HTTP/1.1
connection a client thread, `POST /query` with the vector as a
parameter, and PostgreSQL through the `postgres` crate with
pgvector-rust's types, a statement prepared once.
At the million, fenec-server loads and indexes in 49.6 s against 108.5, takes 612 MB of disk against 1 432, and answers a beam of 100 at 99.1% recall in 0.141 ms against 98.6% in 2.08 ms; eight clients get 18 852 queries a second through, against 2 289. Its memory is what its engine counts, 738 MB; macOS puts the process's physical footprint at 992 MB.

```bash
make pgvector-up
make scale-bench                                   # 1 000 000 x 128
make scale-bench SCALE_ROWS=250000 SCALE_DIM=768
```

| 250 000 × 768 | fenec-server, HTTP | PostgreSQL + pgvector |

| load and index | **39.9 s** | 79.9 s |

| memory the server holds | **835 MB** | 1 137 MB |

| its files' pages in memory besides | **336 MB** | 2 119 MB |

| disk | **787 MB** | 2 082 MB |

| ef 40: recall@10, p50 | **96.9%, 0.302 ms** | 95.8%, 2.387 ms |

| ef 100: recall@10, p50 | **99.8%, 0.431 ms** | 98.1%, 4.223 ms |

| ef 200: recall@10, p50 | **99.9%, 0.664 ms** | 98.6%, 3.852 ms |

| a filter keeping 1%, ef 100 | **100%, 0.626 ms** | 100%, 15.6 ms |

| a filter keeping 25%, ef 100 | **99.3%, 0.458 ms** | 97.7%, 3.017 ms |

| 8 clients, ef 100 | **5 964 queries/s** | 1 991 queries/s |

The vectors are `quant`'s: 64 centres spread along 32
directions, as embeddings vary along far fewer directions than they have,
1 000 held-out queries, and recall against the exact ten found by brute
force. Each row also carries a tag from a hash of its own, which a filter
keeps 1% or 25% of the rows by, with an index on it in both. Both HNSW
indexes take m 16 and ef_construction 64, pgvector's defaults.

Each side gets its best way. `fenec-server` keeps its index as
the rows land, as JSON arrays of rows, a `POST /items` of about 13 MB at a time (10 000 rows of 128 dimensions, 1 666 of 768), the next one written while the last is sent. pgvector builds after a binary
COPY, as it advises, in memory and on every core of Docker's virtual machine
(4 GB, 8 CPUs; its `parallel_workers` is set, since 768-dim
vectors live in TOAST and PostgreSQL planned one process for them), reads
its index into its buffers before the queries (`pg_prewarm`), and
searches a filter with `hnsw.iterative_scan = relaxed_order`.
PostgreSQL answers from inside that virtual machine, so an empty query's
round trip is 0.13 to 0.22 ms against `fenec-server`'s
0.06 to 0.11 ms on the host: it accounts for part of each answer,
not for the rest.

Each server was run on its own after four idle minutes, on an Apple M1
MacBook Air: fanless, it slows under minutes of load on every core, and a
run straight after another took up to three times as long. pgvector's
recall moves from build to build, since its workers insert in parallel:
95.3 to 98.3% at a beam of 100 over four builds of the million. The greedy
descent through its upper layers leaves some queries in another cluster
than their own, and 3 to 23 of the 1 000 found none of their ten. A
million 768-dim vectors does not fit this machine: pgvector's build alone
needs about 4 GB.

The memory a server holds is what the kernel cannot take back without
swapping it out, on Linux for both: PostgreSQL's is its container's
anonymous memory and the shared memory of its buffers, from the container's
cgroup — mostly the buffers, 1 GB as configured here and full once the
index is read into them — measured in a container started afresh for each
run, since the buffers keep what an earlier run left. `fenec-server`'s
is its vector arena, its graph and its indexes, and at most 16 MB of
documents written since it last handed them over to its file (The file is mapped): what its
engine counts (`fenec_memory_bytes`). On a Mac its physical
footprint is larger, since macOS's allocator keeps freed blocks in a cache
of its own. The pages of their files each keeps besides are clean, and the
kernel takes them back as it needs the memory: the pages of
`fenec-server`'s mapped file that the queries read, and
PostgreSQL's page cache of its table and index.

What a load holds at its peak is another matter, and a block of every
row at once shows it. A block holds its writes until it lands, and it used
to hold them twice, in its record and in the stores: 250 000 768-dim rows
in one block peaked at 2 389 MB of anonymous memory against the 820 they
left, measured in process in a Linux container. A block now spills into the
file every 16 MB before it lands ([File
format](https://fenecdb.com/docs/file-format)), and the same load peaks at 904 MB; in blocks of 50 000 rows,
888 against 1 144.

## Scale measurement

fenecdb alone, in one process, with 100 000 documents of 128
dimensions. A vector search takes 0.070 ms at the median and finds all ten
true neighbours.

| write | **28 656 documents/s** (3.5 s), HNSW indexing included, 8 cores |

| storage | 52.7 MB · 527 bytes/document · 7 segments · no page cache |

| ANN k=10 | p50 **0.070 ms** · p95 0.086 ms · p99 0.097 ms |

| ANN with a filter | p50 **0.141 ms**, candidate set n/4 |

| ANN with a filter no row near the query matches | p50 **1.15 ms**, the n/4 measured on every core |

| recall@10 | **100%** against an exact scan |

| snapshot | 29 ms, producing 59.3 MB with the graph and the records' index, +13% |

| reopen | **8 ms**, read into memory, the graph validated and restored |

Method and details

Apple M-series, single thread, `--release`. 100 000 documents ×
128 dimensions, clustered embedding distribution.

```bash
make bench
cargo run --release -p fenec-core --example bench
```

Without a persisted graph, reopening the same data takes 3.4 seconds —
the index is rebuilt from scratch, which would make a page refresh in the
browser unusable. The graph record makes that roughly 340× faster and costs 12%
more space in the image, 52.7 MB to 58.9 MB; the index of the records, which a
mapped open reads rather than walking them, is 0.4 MB more.

A filter that no row near the query matches leaves the walk with nothing
to keep, and the search then measures every row the filter does match: the
answer is exact, and the set is read on every core, the nearest ten kept as
they are measured.

## At a million

The same measurement with a million vectors, of 128 dimensions and of
768. Building takes longer per vector as the graph outgrows the caches.

|   | 1 000 000 × 128 | 1 000 000 × 768, f16 |

| write, HNSW indexing included | 105 s, 9 509 documents/s | 446 s |

| memory | 527 MB of documents | 3.4 GB heap, 1.5 GB of it the arena |

| ANN k=10, p50 | 0.203 ms | 0.64 ms at ef 100, 0.97 at 200 |

| recall@10 | 93.0% at ef 100 | 97.4% at ef 100, 98.4% at 200 |

| ANN with a filter, n/4 | 1.19 ms | — |

| … no row near the query matching | 6.97 ms | — |

| snapshot, reopen | 196 ms, 128 ms | — |

Method and details

The same measurement at 1 000 000 × 128, one run, and the full vectors of
`make quant-bench QUANT_ROWS=1000000` at 1 000 000 × 768, stored as
`f16` and spread along 32 directions, as embeddings are:

```bash
cargo run --release -p fenec-core --example bench -- 1000000 128
cargo run --release -p fenec-core --example quant -- 1000000 768 none --rank 32
```

A build grows faster than the data. Each vector's walk reads more of a graph
that no longer fits in the caches: a node cost 0.27 ms of the cores at 100 000
and 0.63 at a million, with about six of the eight kept busy. `bench`
spreads each vector in every dimension, which no embedding does, and at a
million its recall at the default beam falls to 93%; the 768-dim vectors spread
along 32 directions keep 97.4%, and a beam of 200 buys another point.

## The ef and recall trade-off

A wider search beam (`ef`) finds more of the true neighbours
and takes longer. The default, 100, already finds all of them on clustered
data.

| ef (search) | recall@10 | ANN p50 |

| 64 | 99.0% | 0.060 ms |

| **100** (default) | **100%** | 0.079 ms |

| 128 | 100% | 0.091 ms |

| 160 | 100% | 0.107 ms |

- **Set it per query**: `get docs near embed $1 ef 200 limit
    10`.

- **Ask for `exact`** to read every vector when you want
    ground truth.

Method and details

```bash
cargo run --release -p fenec-core --example bench -- 100000 128 --ef 200
make sweep      # walks the trade-off on both distributions
```

The default is 100: recall is complete and it is still roughly 13× faster
than the engines compared above. It is tunable per query —
`get docs near embed $1 ef 200 limit 10` — and
`exact` performs a full scan when you want ground truth.

`--example sweep` walks the same trade-off on both
  distributions, but its latency column is the mean over 50 cold queries, so
  the two tables are not directly comparable.

On uniformly random vectors — the pathological case for any ANN index, where
in 128 dimensions all distances converge and no neighbourhood structure
survives — recall at the same `ef` is around 82%. Real embedding
models output clustered data. `--uniform` measures that worst case
deliberately.

**Build-side tuning.** 100 000 × 128,
single-threaded measurement:

| ef_construction | build | recall@10 |

| 64 | 6 450 rows/s | 94.0% |

| 100 | 5 849 rows/s | 98.5% |

| **200** (default) | 4 929 rows/s | **99.0%+** |

If write speed matters more than the last point of recall,
`@hnsw(cosine, ef_construction=100)` gains 19%.

## Half precision

Storing vectors as 16-bit floats halves their memory and keeps recall at
100%, for a slightly slower search.

|   | vector<128> | vector<128, f16> |

| vector arena | 51.2 MB | **25.6 MB** |

| file image | 59.3 MB | **33.7 MB** |

| write, index included | **3.5 s** | 4.2 s |

| ANN p50 | **0.070 ms** | 0.096 ms |

| recall@10 | 100% | 100% |

Method and details

Same data, 100 000 × 128, clustered. The trade is explained in
[How it works](https://fenecdb.com/docs/concepts#half-precision).

```bash
cargo run --release -p fenec-core --example bench -- 100000 128 --f16
```

## Memory

How the memory fenecdb counts for itself compares with what the process
really uses at its peak. Use it to choose `--max-memory`.

|   | measured footprint | peak RSS | ratio |

| 100 000 × 128 | 128.2 MB | 172.9 MB | 74% |

| 200 000 × 4 | 57.6 MB | 97.2 MB | 59% |

- **Set `--max-memory` to a third of the container's
    memory.**

Method and details

```bash
make memory       # for calibrating --max-memory
```

The footprint is an early warning rather than a guarantee. A third of the
container memory is a sensible `--max-memory`: it covers both this
60–75% ratio and `compact`'s 3× peak. Details in
[Limits](https://fenecdb.com/docs/limits#memory-and-scale).

## Binary size

How large each build is. For the module and the client together, what
the browser pays is **324 KB brotli**; for an app whose queries run on a
server, the client alone in its bundle.

| Build | On disk | Brotli | gzip |

| `cargo build --release -p fenec-cli`, default | 3013 KB | — | — |

| `--profile cli`, default, as a release builds it | 2400 KB | — | — |

| `--no-default-features`, no import, backup or MCP | 2236 KB | — | — |

| `--profile cli --no-default-features` (`make small`) | **1880 KB** | — | — |

| wasm32, browser | **803 KB** | **273 KB** | 330 KB |

| wasm32, no index or schema check, `make wasm FEATURES=language SCHEMA=0` | 584 KB | 202 KB | 242 KB |

| wasm32, no feature at all, `make wasm FEATURES=none SCHEMA=0` | 524 KB | 179 KB | 214 KB |

| `web/fenec.js` and its modules, the client | 194 KB | 51 KB | 60 KB |

| `@fenecdb/web/client` in an app's bundle, `connect` and a query's rows, esbuild, minified | 29 KB | 9 KB | 10 KB |

| container image, scratch + musl, amd64 | 2.99 MB | — | — |

Method and details

The compressed columns are filled in only where they mean something: the
wasm module and the client are fetched over HTTP, so what the browser pays is
**324 KB brotli** for both together. A binary you execute and an image
whose layers are already compressed are not served that way, hence the
dashes.

**The brotli column is `-q 11`, and the level matters.**
  That is the right number for a file compressed once at build time and served
  as a static asset, which is how this site ships the module. A server
  compressing on the fly does not spend that: the same two files come out at
  306 KB at `-q 5`, nginx's dynamic default, and 325 KB at
  `-q 4`. Quoting the best case without naming it would flatter the
  number by 12–19%.

The on-disk column is not the transfer size and is not trying to be. It is
  what gets decompressed, compiled and held: `WebAssembly.compile`
  and the module's memory footprint track that figure, not the compressed one,
  and so does reading the thing end to end.

`fenec-server` stays on the `release` profile on purpose:
a connection thread that panics unwinds and takes down only its own session,
and the server stays up. The `cli` profile turns panic unwinding
into abort, which is right for a single process with nothing to recover.

**Where the module's bytes go.**
`make size-report` builds the module with its function names
kept and puts each function's code where it is written: a method with its
impl's module, a closure with its function, and an iterator adapter of the
standard library — a `fold`, an `extend` — with the
closure it runs, which it holds inlined. What is left of the standard library
is split by what it is for. `make size-report BASE=main` shows what
a change did against main, down to the ten functions that moved most, and
every pull request's CI run shows the same against the branch it goes into.

| Code, 637.2 KB of the 713.7 | KB | Share |

| **fenecdb's own** | **510.5** | **80.1%** |

| the engine, `fenec_core::engine` | 202.0 | 31.7% |

| FenecQL's parser and lexer | 49.7 | 7.8% |

| the vector index | 41.9 | 6.6% |

| `@sorted` | 20.6 | 3.2% |

| the rest | 196.3 | 30.8% |

| **the standard library's** | **126.6** | **19.9%** |

| `Vec`, `String`, `Box` | 46.3 | 7.3% |

| the sort | 23.0 | 3.6% |

| `HashMap` | 12.1 | 1.9% |

| `core::fmt` | 4.6 | 0.7% |

| the rest | 40.6 | 6.4% |

The data section is another 67.6 KB. Most of the standard library's share
is generic code compiled once for each type it is used with, so the module
sorts few types and keeps few maps: ids as `u64`s, scored ids
through one comparator that the text index's terms share, rows through one
ordering that `lookup` shares, and a collection's indexes by field
in a list rather than a map. That took the sort from eight element types to
five and the maps from eleven entry types to seven, 28.5 KB of code; and
later the stable sort of ids an `in` list's buckets went through
— merged where their runs end now — and the maps of a database's
collections by name and a load's by number, to five, 6.9 KB more. A
function inlined into two callers is two copies of itself, and a generic
one a copy for each type and closure it is called with: the handles'
table each export reached through a closure of its own, `build_index`
and `facet` inlined twice, the standard library's reading of an
integer inlined at each of its three readers, were 8.6 KB, which each
took one copy of. Writing a
float was 20.8 KB more, Grisu and Dragon behind `{}`, until the
module wrote its own: Ryu's shortest digits, the text `{}` writes
to the byte, in 2.1 KB (`num.rs`). And 16.5 KB went with a
panic's formatting of a `char`, kept by every slice of a string
that could panic, the standard library's `to_lowercase`,
`to_uppercase` and `contains` among them: the module
slices without, and puts a string in a case through `case.rs`,
character by character, the same text to the byte. The link writes its
relocations short (`--compress-relocations`), 36 KB of padded
LEBs, and V8 runs the smaller module faster: `near` 12%, a page
of vectors as JSON 15% (`make wasm-speed`). `make size-report
WHY=<pattern>` names what pulls a part in.

## The browser module, feature by feature

A page that does not search text, needs no graph behind its vector
search, or asks no facets, can load a smaller module built without what it
does not use. With every feature the module is 244.1 KB brotli; with none,
149.0 KB, 39% less, and a `near` still answers, every vector
measured. The release ships the module with every feature; any other set
is a build option.

| Feature | Added to none, brotli KB | Left out of every one, brotli KB | On disk, KB saved |

| `vector`, the HNSW graph | 20.4 | 19.0 | 61.4 |

| `text`, `match` with `fuzzy` and `prefix` | 13.1 | 15.2 | 43.5 |

| `sparse`, `@inverted` | 2.4 | 1.4 | 3.9 |

| `sorted`, `@sorted`, and `composite` with it | 8.7 | 23.5 | 76.4 |

| `composite`, an index over several fields, beside `sorted` | 11.6 | 10.9 | 32.8 |

| `geo`, points, and `@geo` where `sorted` is | 2.5 | 6.0 | 16.8 |

| `nocase` | 2.8 | 2.6 | 7.1 |

| `collate`, ICU's order | 8.9 | 8.4 | 19.2 |

| `facets` | 3.6 | 4.5 | 12.5 |

| `highlight`, beside `text` | 3.8 | 3.6 | 10.0 |

| `analytics` | 5.5 | 5.1 | 14.3 |

| the schema check, `SCHEMA=0` without | 9.5 | 9.5 | 28.2 |

Method and details

A page loads a module built with what it uses (A smaller module): `make wasm
FEATURES="text sorted"`, `FEATURES=indexes` or
`language` for a group. `make wasm-sizes` builds the
module with every feature (713.8 KB on disk), with none (432.5 KB), with
each added to none and with every one but each, and measures each under
gzip -9 and brotli -q 11. The two columns differ where features share
code: the sort and the map the sparse index ranks through are the other
indexes' too, and the ordered index's chunks are the index over several
fields' and `@geo`'s, so `sorted` left out takes both
with it.

What stands in for a missing index is a type no value can have, so the
compiler drops the index whole, and the engine's paths through it with it;
a fold of no value does the same for `nocase`, and a function
or an aggregate a feature brings is refused by name. A module without a
feature opens every file: a collection declaring an index it lacks is
answered by the scan, a point is read and written, and a statement over a
collection whose `nocase` or collated field it cannot compare
is refused, the feature named. Making these optional cost the module with
every feature nothing: its code came out 14 bytes smaller, functions moved
and closures renumbered, and its compressed sizes within the noise of a
build's layout. Left in every module, measured: json fields and their
paths, which would save 0.6 KB brotli, and `@ttl`, 0.7 KB.

Without the graph a `near` measures every vector, as
`near ... exact` does in the full module, with the same rows and
scores: the scan cost the modules without it 2.2 KB brotli. Measured with
`make wasm-exact-speed` in Node, the median of 50 queries,
`limit 10`, over clustered vectors under cosine:

| Rows × dimensions | Through the graph | Exact, full module | Built without `vector` |

| 1 000 × 128 | 0.20 ms | 0.17 ms | 0.38 ms |

| 10 000 × 128 | 0.22 ms | 0.49 ms | 2.4 ms |

| 50 000 × 128 | 0.34 ms | 1.9 ms | 11.7 ms |

| 1 000 × 384 | 0.47 ms | 0.43 ms | 0.90 ms |

| 10 000 × 384 | 0.51 ms | 1.1 ms | 6.2 ms |

| 50 000 × 384 | 0.75 ms | 3.9 ms | 29.4 ms |

The full module's exact scan reads its arena, the vectors already made
unit ones; without the graph each is read out of its document and made one
for the query, four lengths summed side by side, which is the rest of the
difference: a `near` 2 to 40 times slower.


---

Source: https://fenecdb.com/docs/file-format

# File format

A fenecdb file is a header followed by records, and an
open reads it from start to end in one pass.

## Records

Every record is `[kind][collection-id][length][body]`. These
are the kinds:

```text
"FENECDB\x01"
  [6][u64 counter][u64 body length]        change counter (optional)
  [1][collection-id][length][schema]       create collection
  [7][collection-id][length][next-id]      id counter (optional),
                           [index]         an image's, and its data's index
  [5][collection-id][length][schema]       schema change (index)
  [12][collection-id][length][change]      a field added, dropped or renamed,
                           [field][name]   the field and its new name,
                           [schema]        and the schema after it
  [2][collection-id][length]               drop collection (empty body)
  [3][collection-id][length][records]      data
  [4][collection-id][length][field][graph] HNSW graph (optional)
  [8][0][length][history]                  replication history (optional)
  [9][0][length][records]                  a block of several records
  [10][0][length][records]                 a block's spill, before it lands
  [11][0][length][spills][records]         a block that spilled, landing
```

Inside a data record, each document is
`[op][doc-id][length][fields]`, each field a tagged value. A
`sparse<N>` field is `[10][N][count]`, then the
indices, each as its distance from the one before, then the weights as
`f32`. An object, in a `json` field, is
`[13][count]`, then each member's key as a length and its
bytes and its value tagged, ascending by key.

**A record cut short is cut off the file, not only skipped.** Left in
  place, the first write appended after it is read back as the rest of it:
  the write was acknowledged, and the next open loses it.

**The length is written even for an empty body** — a
  `drop` record has one — **and the reader must consume it.**
  Otherwise the leftover byte is read as the next record kind and the file
  stops opening.

Details

The reasons for each exception to the plain layout are worth knowing
before you touch the reader; the sections below give them.

A sparse vector's indices are written as gaps because a SPLADE vector's
sit close together, and a small gap takes a byte.

A crash in the middle of an append leaves the last record cut short, in its
header or its body: the open loads everything before it and cuts it off the
file, so there is no separate recovery step after a crash.

On Linux a sync goes to `<file>.sync` beside the
file: a log of fixed size holding what the file took since its last own
fsync. An fsync of a file that grew commits ext4's journal as well, 356 us
against 65 for bytes written where the file already reached, so a durable
write syncs the log and the file is synced once the log is full. An open
writes back into the file what the log holds and the file lost, before it
reads it, then wipes the log's header; an entry carries a hash and the log
names the file it was written for (its inode, and a hash of the bytes
before the log's start), so an entry cut short ends it and a log beside
another file -- one a restore or an import renamed into the place -- is
ignored. A clean close syncs the file and removes the log. A tool that only
reads the file (`fenec types`) applies none: after a power loss,
open the database before you copy or inspect it. The file's format is the
same with it or without.

## The graph record

Kind 4 holds an HNSW graph's links, so an open does not have to build the
graph again. It is derived data: if it does not match the documents, it is
ignored and the graph rebuilt, so it can never lose data.

- **It stores links, not vectors** — except a tombstone's vector.

- **The last one wins,** and the writes after it are applied on
    top.

- **Versions 5 and 6** are a checkpoint with nodes waiting to be
    linked, and a server's record in the tail.

Details

It is derived data, not a cache. On open, the version, the dimension, the
precision and the link bounds are validated, and the live nodes against the
documents: every one's document has to hold a vector, and every document
holding one has to have a node. If any of it does not hold, the record is
ignored and the index is rebuilt from scratch. A corrupt or stale graph
therefore cannot cause data loss.

It is written during `snapshot`, `compact` and
`checkpoint`, and a server appends one to the file's tail now and
then (below) — never on the normal write path. What it stores is the
links, not the vectors: the vectors are already in the document records, and
the expensive part is the build. The exception is a tombstone, the node a
deleted or rewritten document leaves behind until `compact`: it
still routes searches, but its document is gone or holds another vector, so
the record carries the vector the node was linked with.

The graph is restored where its last record is, against the documents as
they stood there: a checkpoint's image holds one after each collection's
data. Everything after that point is applied to it the way the write path
applied it: the node of each document written since is retired, and the
vector the document holds now goes in. A crash therefore costs the writes
after the last graph, not the graph. An earlier record of the same graph is
passed over — the open walks the record heads to find the last one first —
since restoring it would read every vector again for nothing.

A server keeps its graphs in the file as they change, since it writes a
checkpoint only on its way down: once a graph has changed in 10 000 nodes
since it last reached the file, has none waiting to be linked, and the file
has grown by three times its record since, it is appended to the tail, the
writes held off while it is written. The record holds the whole graph,
which is why the growth counts too: a big graph written every few thousand
writes would be written over and over for a little of it. It is written in
version 6, whose layout is version 5's; a binary that does not know 6
rebuilds the graph rather than restore it against the documents the whole
file left, which a vector rewritten after the record would not match. It
is not a write: the change counter does not move, no replica is sent it,
and a record cut short is cut off the file like any other. The next
checkpoint leaves the ones before it out.

A server does not wait for the tail. It opens with the tail's vectors in
the arena but in no neighbour list, measures each of them in every
`near`, and links them on a thread beside its queries. A
checkpoint written before that is done marks those nodes as waiting: a
node's flag byte, 0 for a live node and 1 for a tombstone, is 2 for one,
in a graph record of version 5, which is written only then. The next open
leaves them waiting again, or links them there when it is not a server's.
An older binary does not know version 5 and rebuilds the graph, as it does
from any record it cannot read.

A graph over bit codes (`quant=bit`) also carries the centres
its codes are taken from, since a restored graph has to code its vectors
against the ones it was built with. Where the quantization's code goes, a
bit graph writes 3; the number of centres follows, then each centre as
`dim` little-endian f32s — none while the index still holds its
vectors whole. A tombstone's vector is its code: the signs, the centre's
number in a byte and two f32s. A bit graph written before the centres, with
the vectors' own signs and code 2, is built again on open; a binary from
before them does not know 3, and builds its own.

## Indexes in the schema

A schema names each field's index by a kind byte. Only the graph is
stored; every other index is built from the documents after an open.

| kind | index |

| 0 | none |

| 1 | `@hash` |

| 2 | `@hnsw` with its parameters |

| 3 | `@text` with its parameters |

| 4 | `@sorted` |

| 5 | `@hnsw` over codes (`quant=`) |

| 6 | `@inverted` |

| 7 | `@text` indexing characters as well (`chars`), with the parameters kind 3 has |

| 8 | `@unique`: a hash that refuses a value held twice |

A field ordered in a collation (`name text collate tr`) is
written with type tag 11 and the collation's code in front of its type: 1
for `tr`, 2 for `und`. A text field compared by its
case fold (`email text nocase`) has type tag 17 in front of its
type, after the collation's where it has one. A dropped field's place
(`alter collection ... drop field`) is written where it stands
as a field with no name and type tag 12: the documents written before the
drop hold a value there, which a read passes over, and a document written
after it a `null`. A document that ends before a field's place
holds `null` there: it was written before the field was
added. A `json` field is type tag 14, and after its index kind
come the indexes on paths into it: a count, then each one's keys
(`source.rank`) and its index kind as a field's is written.

Details

For every kind but the graph that byte is all the file holds: the hash,
text, ordered and inverted indexes are built from the documents by the first
statement that reads each after an open, so there is nothing of theirs to
validate or to go stale — and a binary that splits text another way, as one
before runs of characters did, builds its own text index from the same
documents. An old binary cannot open a file whose schema uses a newer kind
and reports `unknown index kind 4`, the same refusal as for a
record kind it does not know.

An older binary meets a type tag it does not know, or a collation code,
and refuses the file, rather than read the field and order it in another
order — or, meeting a `json` field, before it reads an object
(value tag 13) it would have no reading of: unknown type tag 14.
The value and type tags share their numbers, so 13 and 14 are new to both,
where 11 and 12 are a collation's and a dropped place's, and 17 is
`nocase`'s: a binary from before it refuses the file rather than
compare the field's bytes and take a second spelling its
`@unique` refuses. The collation's weights are not in the file: they are the binary's,
and a `@sorted` index over the field is built from the documents
on open.

## The change counter, and why it is at the front

Kind 6 holds the change counter. It sits at the front of the file and is
fixed width; its body length marks where the image's own records end.

Details

Record kind 6 sits at the front of the file and is fixed width. Both are
deliberate.

**At the front**, because at the end a corrupt or half-written tail would
break opening. The end of the file is exactly where the graph stops, and the
graph is derived data — a record that errors out in that region would take that
tolerance back.

**Fixed width**, because the body length is known only after the body is
written. A placeholder can be filled in place, whereas a uvarint would mean
shifting the whole image.

The body length means "this is where this image's own records end". Every
record after it was written after the checkpoint and carries the counter
forward. Old files without the record load fine — the counter is then counted
from the records. The reverse does not hold: an old binary cannot open a new
file and reports `unknown record kind 6`.

## A statement's writes are one record

All the writes of one statement or `/batch` are
appended as one record, so a crash keeps all of them or none. Within one
collection that is a data record (kind 3); otherwise a block (kind 9).

Details

Every write is a block: the documents a `put`, a
`set` or a `del` writes, the statements of a
`/batch` or of the browser module's `run` of several.
Their frames are held back until the block ends and are appended as one
record, so a crash cuts all of them off the file or none; a block that fails
is put back in memory instead, and nothing of it reaches the file. Within one
collection the record is a data record of as many frames — the form an
image's data records have always had, so a binary from before blocks reads
it — and a lone schema change is the record it always was. Across
collections, or with a schema change among other writes, it is record kind
9, holding each write's record: a data record, or a create's, a drop's or an
index's. A binary from before blocks refuses it with `unknown record
kind 9`, and one from before 10.4 a block holding a schema change as
corrupt.

The change counter moves by the writes a record holds, not by one a
record: a block of five writes is numbered by its last, and whatever counts
records by their numbers — a replica, an archive — counts them that way. A
restore to a change inside a block stops before it, since the database never
stood there.

## A large block spills before it lands

A block past 16 MB is written to the file in parts as it goes (kind 10,
a spill), then landed by a record that names those parts (kind 11). A spill
no land names is ignored.

Details

Held back until it lands, a block's frames were in memory twice — in the
record being built and in the stores — so a load of 250 000
768-dim rows in one command peaked 1.57 GB over what it left. Once a block's
frames amount to 16 MB, a server appends them as record kind 10, a spill,
whose body is a block record's, and the stores read those documents from the
file from then on. The block lands as record kind 11, which names its spills
by where their bodies are in the file and how long they are, and holds its
writes since the last. A load applies a spill only where a land names it:
one that a rollback, a lapsed lease or a crash left is dead in the file, and
the ones a file ends with are cut off, as a record cut short is. The land
never travels: a replica and an archive are sent the block as the one kind 9
record it would have been. A binary from before spills refuses kinds 10 and
11 as unknown.

## The history record is not a write

Kind 8 records which [replication](https://fenecdb.com/docs//docs/replication) history
the writes belong to. It is the one record that must not move the change
counter.

Details

Record kind 8 says which history the writes belong to: a list of
`(id, from change)` pairs, one per fork, and whether the file is a
replica's. A replica is fed a primary's writes by their number on the change
counter, so this is the one record that must not move the counter — a
replica would be numbered one change off its primary. It is appended when a
history forks or a replica follows one, and written into every image while
it is set; a file that never had a replica holds none. An old binary cannot
open a file with one and reports `unknown record kind 8`.

## The id counter exists because of compact

Kind 7 keeps a collection's next document id, so a `compact`
can never hand a deleted id out again. In an image it also carries an index
of the data record after it, which makes opening faster.

Details

A document id is normally derived from the records: a replay takes one more
than the largest id it saw. Compaction throws tombstones away, so that
derivation falls short — the highest deleted id disappears from the image
entirely and would be **handed out again** on the next open.

An id coming back silently binds everything still holding it — a link handed
out, a row a subscriber holds — to the wrong document. The record is per
collection and comes right after the schema. Only `snapshot` writes
it; the append path does not need it, because the tombstones are still there.
Old files without the counter still load correctly.

Behind the counter an image writes an index of the collection's data
record, which comes right after it: the record's length, the bytes in it
that are dead, and each live document's id, payload length and place, each
written as its difference from the one before — four bytes a document for
records of 590 bytes. A mapped open takes the offset index from it rather
than walking the record's frames, whose heads each say where the next one
starts: a chain of cache misses that was 5.5 ms of a 14.2 ms open at
100 000 × 128, which opens in 8.4 ms with it. An index that does not fit the
record — another length, ids out of order, a payload past its end — is
passed over and the record walked. Every reader of the record, older ones
included, reads the counter and nothing after it.

## Crates and layout

Where each part of the format and the engine lives in the source.

```text
crates/
  fenec-core/   storage, HNSW, plan executor, plugin registry, JSON,
                calendar arithmetic, change stream          (no dependencies)
  fenec-ql/     FenecQL lexer and parser                    (core only)
  fenec-wasm/   the browser ABI                             (no wasm-bindgen)
  fenec-http/   HTTP/JSON endpoint: REST, raw FenecQL, SSE, replication,
                tenants                                     (core + ql only)
  fenec-wire/   the PostgreSQL client the importer reads with
  fenec-import/ SQLite file reader, PostgreSQL COPY source, --follow
  fenec-server/ the server: HTTP, replication, tenants, the --follow mirror
  fenec-shard/  the tenant router
  fenec-cli/    the `fenec` shell, `fenec import`, `fenec types`
web/            fenec.js (builder + sync layer), fenec.d.ts,
                fenec.test.js and fenec.sync.test.js, the browser console
```

## Maintenance operations

Four operations read or rewrite the whole file. A rewrite goes to a side
file that is renamed into place, so a crash cannot leave it half
written.

| Operation | What it does | Peak memory |

| `snapshot` | Builds the image in memory: what the browser persists, and what a tenant's move and a replica's first stream send | ≈ 1× the file |

| `checkpoint` | Lands the graph in the file so the next open does not rebuild it; written into the new file as it is produced | the indexes |

| `compact` | Drops the dead records, keeping every index but a graph holding tombstones: the documents are the same ones | the indexes |

| open | Maps the file and builds the indexes from the documents | the indexes |

Details

A graph holding tombstones is rebuilt by `compact` from the live
vectors: nothing else takes a tombstone out, and every write that changes a
document's vector, or deletes it, leaves one. A graph without them is kept
as it is.

The rewrite goes to a side file and is renamed, so a half-written checkpoint
cannot corrupt the database — and since an image is only ever whole, a record
cut short inside it, or an image longer than the file, is refused as a damaged
file rather than cut off as a crash's torn tail is. A server's
`compact` writes its image beside the database into
`<file>.fenec.beside`, with no lock held; the documents written
meanwhile go into it as a data record at the end of the image, and its header
takes the counter as it stands when it is renamed into place. A side file left
by a crash is written over by the next compact, and can be deleted.


---

Source: https://fenecdb.com/docs/limits

# Limits

What fenecdb does not do, the ceilings in its code, and the
behaviour worth knowing before it surprises you.

## Out of scope

Before you choose it, at a glance:

- **FenecQL, not SQL.** `select a from t`, `insert
    into t (a) values (1)`, `update t set a = 1`,
    `in (…)`, `between`, `like` and a
    `join` on an indexed key work ([From
    SQL](https://fenecdb.com/docs//docs/sql)); a join of any other condition does not. `lookup`
    covers related rows (below).

- **One writer per file.** Readers run beside it, and a
    `/batch` holds the writer while it runs. Scale out with a file
    per tenant ([Tenants and sharding](https://fenecdb.com/docs//docs/sharding)).

- **Indexes live in memory.** The data stays in the mapped file; the
    indexes are restored or rebuilt on open ([Memory
    and scale](#memory-and-scale)).

- **Typed fields, and `json` where a shape is not
    fixed.** A `json` field holds objects, lists and scalars,
    and a path (`meta.lang`) reads into it
    ([Objects and paths](https://fenecdb.com/docs//docs/fenecql#json)).

- **No TLS.** Put a TLS terminator in front of any port off the
    machine ([Server](https://fenecdb.com/docs//docs/server)).

Deliberately not done: JOIN, SQL as the query language, several writers to
one file, multi-writer replication, a field's type changed in place (it
would rewrite every document under the write lock). Not done yet, and
planned: TLS, in the repository's `ROADMAP.md`. A row's
expiry is [`@ttl`](https://fenecdb.com/docs//docs/fenecql#ttl), a filter by
another collection's rows is
[`in (get …)`](https://fenecdb.com/docs//docs/fenecql#subqueries), a unique field is
[`@unique`](https://fenecdb.com/docs//docs/fenecql#indexes), a field is
added, dropped or renamed with
[`alter collection`](https://fenecdb.com/docs//docs/fenecql#changing-a-collection),
and nested objects are a [`json`
field](https://fenecdb.com/docs//docs/fenecql#json).

Details

These are consequences of the design, hard-coded ceilings, and behaviours
it is cheaper to read about than to discover. The few that are planned say
so.

Relations are [`lookup`](https://fenecdb.com/docs//docs/fenecql#the-lookup-clause)
rather than JOIN: it attaches a collection's matching documents to the row they
belong to, which is one bucket probe per parent and a `limit` that
counts children per parent. It chains up to 8 levels, and
`required` lets the children decide which parents appear. What it
still does not give is a predicate spanning both sides —
`products.price > reviews.cost` cannot be written, because the
positional scoping that keeps qualified names out of the language leaves a
predicate no way to say it reads two collections. That stays a second
query.

Replication exists in **one direction only**: the server is
authoritative and clients read it. A replica's writes wait in a queue kept in
its own file (`_sync_queue`), in the browser and on a phone, and
reach the server one at a time, each once under its
`Idempotency-Key`; the server answers each, and a write it refuses
is put back ([Sync](https://fenecdb.com/docs/sync#refusals-retries-and-offline),
[mobile](https://fenecdb.com/docs/mobile#sync)). Multi-writer merging and CRDTs are out
of scope — that is what fits the single-writer model, and the other would be
a different database.

There are deliberate gaps in the type system too:

- **No decimal.** `float` is binary floating point and unfit
    for money. The answer is an `int` in cents.

- **No UUID type.** `text @hash` is functionally
    enough.

- **Objects only in a `json` field.** A typed field still
    refuses one, and a list of a type holds that type alone. A
    `json` field holds any value JSON does; a path filters,
    orders and is indexed (`@hash`, `@unique`,
    `@sorted`), but an aggregate and `group` read a
    field, and text, vector and sparse indexes stay on typed fields.

## Fixed limits

Ceilings baked into the code. Exceeding any of the first seven makes the
query **error**, because a silently cut result is a wrong answer believed
right; so does going past the two `json` depths.

| Limit | Value | Note |

| `near`, `match`, `fuse` result (`limit + offset`) | **10 000 rows** | a query error when exceeded, by a number written or a parameter's (`limit $2`); without `limit`, the default top-k applies |

| words one word of a [`match ... fuzzy` or `prefix`](https://fenecdb.com/docs//docs/fenecql#typos) reaches | **10 000** | the word itself among them; a query error naming the word past it, never some of them taken — a search box sends its second letter, or its first over a smaller vocabulary (FiQA's 80 426 words: `s` reaches 7 020) |

| edits `fuzzy N` takes | **2** | and only what a word's length allows: none under 5 characters, one up to 8; past 2, a parse or query error |

| `rerank` candidates, `fuse` candidates a side | **10 000** | a query error when exceeded; never fewer than `limit + offset` |

| expression depth | **512 levels** | parentheses and `and`/`or` chains count |

| `lookup` chain depth | **8 levels** | a query error when exceeded; the parser, the check and the engine all recurse per level |

| fields of a [`lookup` key](https://fenecdb.com/docs//docs/fenecql#the-lookup-clause) | **8** | as many as an index over several fields has parts, whose first ones they must be; refused past it, naming the index to declare |

| `in (get …)` values | **100 000** | distinct values, each counted once however many rows hold it; a query error naming the bound when the inner `get` finds more; never a set cut short |

| a policy rule's [`in (get …)`](https://fenecdb.com/docs//docs/http#scoped-access) values | **1 000** | as a list claim's: answered before each statement its token sends, so a query error naming the bound past it, never a list cut short |

| `in (get …)` nesting | **4 levels** | a query error when exceeded; each level runs a query before the one around it |

| values of one `facet` | **10 000** | a query error when a facet without `top` finds more, and for `top` past it, written or a parameter's; never a list cut where it stopped |

| ranges of one `facet ... ranges` | **10 000** | a query error past 10 001 bounds; every range answers |

| `json` value nesting | **64 levels** | objects and lists inside one another; a write or a JSON text deeper is refused, never cut |

| path length | **64 keys** | past its field (`meta.a.b` is two); a query error, since no value nests deeper |

| fields of an [index over several fields](https://fenecdb.com/docs//docs/fenecql#composite) | **8** | refused past it, and under 2: one field's index is its own `@sorted` or `@unique` |

| spans the `in` lists over an index's parts make | 256 | the elements of each, multiplied; past it the list is tested row by row, never cut |

| session stack (`fenec-server`) | 8 MiB | virtual; the deepest expression wants ~750 KiB in release, ~5 MiB in debug |

| segment size | 8 MiB | sealed when full, a new one is opened |

| document payload, segment offset | 4 GiB | the location record is a `u32` |

| vector index node count | 232 | the node id is a `u32`; memory runs out first in practice |

| level-0 degree (`m0 = 2m`) | 65 535 | `u16` counter, so `m` ≤ 32 767 |

| bulk insert batch | 64–512 vectors | `graph/16`, clamped to this range |

| parallel build threshold | 1024 vectors | below it, the serial path |

| gap in an externally given `id` | 4096 | a larger jump falls into a sparse map rather than a dense array |

| `timestamp` | 1 ms, i64 | ±2.9×108 years; microseconds truncated, always UTC |

| [`bucket` by a named zone](https://fenecdb.com/docs//docs/fenecql#functions) | from 2000 | IANA's rules of the build's tzdata (2026c); a time before 2000 is a query error, never answered by rules the build does not hold; an offset written out (`'+03:00'`) answers any time |

| `vector<N, f16>` | ±65 504 | ~3 decimal digits; outside it, values silently become `inf` or 0 |

| `geo` point | ±180°, ±90° | two `f64`s kept as written; past either range a write is refused, never wrapped |

| `sparse<N>` dimension | 109 | pgvector's; the weights are `f32`, finite, and a zero is dropped |

| change ring | **4096 entries** | `--changes`; how far behind a subscriber may fall, 24 bytes each |

| concurrent subscriptions | **64** | `--http-max-streams`, a database's; each is a connection and a thread; on a tenant node, each tenant's own, and one of the `--max-connections` every tenant shares |

| a tenant's file (`--dir`) | none unless set | `--tenant-max-bytes`, a tenant's own `bytes`; a write that would take the file past it is refused (507) before it lands; deletes, drops and `compact` run |

| a tenant's rows | none unless set | `--tenant-max-rows`, its `rows`; its collections together, the server's own (`_`-named) left out; 507 |

| a tenant's streams | a quarter of `--max-connections` unless set | `--tenant-max-streams`, its `streams`: subscriptions, `/_changes` waits and `Fenec-After` waits together; 429 |

| a tenant's held writes | a quarter of `--max-connections` unless set | `--tenant-max-held`, its `held`: writes held until they can write (`Fenec-Wait`); 429 |

| a tenant's requests in flight | none unless set | `--tenant-max-requests`, its `requests`; 429 |

| a tenant's rows written a second | none unless set | `--tenant-writes-per-sec`, its `writes`; charged the rows each write lands, a write of none costing none; 429 while in debt, `retry_after_ms` to the millisecond |

| a tenant's bytes written a second | none unless set | `--tenant-write-bytes-per-sec`, its `write_bytes`; charged as each write lands, 429 while in debt |

| rows an MCP tool answers | **100** | `--mcp-max-rows`; past it the call is refused (413) with what to narrow, never cut ([Agents and MCP](https://fenecdb.com/docs//docs/mcp#limits)) |

| bytes an MCP tool answers | **64K** | `--mcp-max-bytes`, the answer's JSON; refused past it (413) |

| MCP tool calls a second | **20** | `--mcp-calls-per-sec`, a scoped token's subject or an address; 429, with when to call again |

## Memory and scale

The documents stay in the file and the indexes are held in memory, so a
server's memory follows its indexes, not its data. The largest of them is
usually the vectors.

- **A 1 GB file** of 2.3 million rows with two indexes holds 188 MB
    mapped.

- **One million 768-dim vectors** take about 3 GB, 1.5 GB as
    `f16`, 772 MB as int8 codes and 105 MB as bit codes.

- **Set `--max-memory` to a third** of the container's
    memory.

Details

**The documents stay in the file and the indexes are in memory.** The
file is mapped ([The file is
mapped](https://fenecdb.com/docs/concepts#the-file-is-mapped)), so a document is read from the pages the operating system holds
and can drop; what the process holds is the offset index, the hash, ordered
and text indexes, the graph, and the documents written since the open until
they amount to 16 MB and are handed over to the file — a large
`/batch`'s spilled into it as it goes.

- **What a file costs to hold.** A 1 GB file of 2.3 million rows with
    a hash and an ordered index: 188 MB mapped, against 1 095 MB read into
    memory (`--no-mmap`). A 10 GB file of 23 million rows opens on
    an 8 GB machine, which reading it cannot.

- **`compact` peaks at the indexes, not the data.** On that
    1 GB file: 236 MB and 2.08 s mapped, 2 012 MB and 1.52 s read — the
    mapped one writes the live records into the new file and points at it,
    copying nothing and rebuilding no index but a graph holding tombstones.
    It takes longer and holds a ninth of the memory. A
    `checkpoint` is the same path with the dead records kept.

- **Open cost is proportional to data size, not constant.** The graph
    is restored, reading every vector, and the offset index is built: mapped,
    from the index of its records the last checkpoint wrote, and walking the
    records written since; read into memory, walking them all. The hash,
    text, ordered and sparse indexes are built from the documents by the first
    query that reads each. 100 000 × 128 with the graph in the file opens in
    about 9 ms, which a single vector query makes up against SQLite's 30 ms
    scan.

The vector arena is the largest item: one million 768-dimension vectors is
about 3 GB, or 1.5 GB with `f16`, 772 MB as int8 codes and 105 MB
as bit codes ([`quant=`](https://fenecdb.com/docs/concepts#quantized-indexes))
— the documents keep their vectors either way. In the browser nothing is
mapped: there the whole file is in memory, inside WASM32's 4 GB address
space, which is the real ceiling there.

`Database::memory_bytes()` sums the segment bytes, the offset
indexes, the vector arenas, the graph links, and the text, sparse, hash and
ordered indexes, and `--max-memory` uses it before every write
that grows the data. Each index keeps its count as it changes, so asking
costs nothing next to the write: summed over every term instead, a text
index of 200 000 documents took 0.42 ms a call. It is not RSS: allocator
slack, HNSW build buffers, upper-level neighbour allocations (~3% of level
0) and the transient peaks all sit outside it. On the 1 GB file above,
opened mapped, it counts 197 MB where the heap holds 188, 34 of them the
hash index.

|   | measured footprint | peak RSS | ratio |

| 100 000 × 128 | 128.2 MB | 172.9 MB | 74% |

| 200 000 × 4 | 57.6 MB | 97.2 MB | 59% |

So the ceiling is an early warning, not a guarantee. A third of the
container memory is a sensible start: it covers both this 60–75% ratio and
`compact`'s 3× peak.

Past one machine the data is split by tenant, not by row: each tenant is a
file of its own, nodes hold many, and a router forwards each request to the
right one ([Tenants and sharding](https://fenecdb.com/docs//docs/sharding)). The bound moves
from "the database fits one machine" to "each tenant does" — a single tenant
larger than a node is still out of reach. A single file has
[read replicas](https://fenecdb.com/docs//docs/replication), but a tenant node does not yet,
so a node that is down takes its tenants with it.

## Index build, open and maintenance

Indexes are rebuilt or restored when a file opens, and the vector graph is
saved to the file at checkpoints, never on each write. A crash or a skipped checkpoint costs
time at the next open, never data.

- **A server answers while it relinks** vectors written since the last
    checkpoint, and keeps its graphs in the file as they change.

- **A server builds an index or compacts beside the database,** so
    reads and writes wait milliseconds, not the whole build.

- **Deleted or changed vectors stay in the graph** until
    `compact`, slowing `near` a little.

Details

- **The graph is derived data and is written only during
    `snapshot`, `compact` and `checkpoint`**
    — never on the write path. Without it in the file it is rebuilt on open:
    0.90 s every open at 20 000 × 32, against 0.01 s with it.

- Who writes a checkpoint: the `fenec` shell when leaving an
    interactive or piped session, and `fenec-server` on the shutdown
    signal — both only if a vector index exists.
    **A one-shot `fenec file.fenec -c '...'` does not.** Neither
    does `kill -9` or a cgroup OOM. The next open then restores the
    graph the last checkpoint wrote and applies the writes since it, so what
    is lost is time, not data: with 1 000 writes after the checkpoint,
    0.07 s at 100 000 × 128 and 0.29 s at 100 000 × 768. The time grows with
    the writes since the checkpoint, and a server does not wait for it:
    their vectors go in unlinked, `near` measures each of them,
    and a thread beside the queries links them. At 100 000 × 768 never
    checkpointed, `fenec-server` answered its first `near`
    1.23 s after it started, against 67.7 s when the open linked them first,
    which now takes 18 s; `near` takes the exact scan's 10.5 ms
    until the 19 s of linking are done, and 0.31 ms after. A server also keeps
    its graphs in the file as they change, so what waits is what came after
    the last of them: at 100 000 × 768 at most 10 000 vectors, linked in
    2.6 s with `near` at 1.56 ms meanwhile, where every vector took
    19.1 s in the same run.

- **`compact` rebuilds a graph only when it holds
    tombstones.** The dead records go and the other indexes stand; a vector
    graph with a deleted or rewritten document's node in it is built again
    from the live vectors, which is what a `create index` of it
    costs — 50 s at 100 000 × 768.

- **A server builds an index or compacts beside the database.** A
    `create index` or `compact` sent on its own to
    `fenec-server` or `POST /query` copies what it reads
    under the read lock, builds with no lock held, and takes the write lock
    only to apply the writes made meanwhile and put the result in place. At
    100 000 × 128, where either takes about 20 s, reads waited at most 21 ms
    through an HNSW build and 30 ms through a compact — the compact's includes
    rewriting the file — against the full 20 s under the write lock, and
    writes went on at the same pace. Over a mapped file the compact copies no
    record: the graphs holding tombstones are rebuilt beside the database,
    the live records are written into a side file with no lock held, and the
    write lock is taken only to add the documents written meanwhile and
    rename it into place. Over 1.8 million × 128, a 1.1 GB file, writes
    waited at most 24 ms, where the rewrite under the lock held them 1.22 s,
    for 67 MB more heap at the peak. The records those documents superseded
    stay in the new file as dead ones until the next compact. One in a
    batch, or run by the shell or an embedding program, still holds the
    write lock throughout. A schema
    change to the same collection while it builds — another index, a drop —
    fails it with a request to run it again, since what was built no longer
    fits, and so does a second compact while one writes its file. It cannot
    be cancelled once begun.

- **A file compacts itself once half of it is dead.** The server
    (`--auto-compact`, [Compaction](https://fenecdb.com/docs//docs/server#compaction))
    and the native library (`FENEC_OPEN_NO_AUTO_COMPACT` turns it
    off) look every 5 s and compact beside the queries once the dead records
    are half the file and 64 MB, so a file under updates stays under about
    twice its data, and its peak while the compact writes is the old file and
    the new one together. A file that reaches three quarters of the memory
    the system has for its pages is compacted sooner, once a third of what
    the compact writes is dead, so its reads stay in memory: up to four
    bytes written a byte updated against two. Over a million 1 KB records updated at 150 000 a
    second, each compact held the write lock 6 to 13 ms. An embedding
    program calls `compact_when_due` from a thread of its own or
    starts a `Compactor`; the browser module has no thread to run
    one, and drops the dead records from each image it persists.

- A delete is a tombstone and the graph node stays in memory until
    `compact`. Changing a document's vector also marks the old
    node deleted and adds a new one, so in collections whose vectors change,
    the graph grows until compaction — and the tombstones take places in the
    beam
    `near` walks. An update of the other fields leaves the node
    where it is: 0.006 ms for a title at 20 000 × 768, where taking the
    vector out and back in cost 1.89 ms and a tombstone. A page tombstones
    cut short is walked again with the
    beam wider by their number, or searched exactly where that walk would
    cost more than reading every vector, so the answer is whole and only
    slower until the compact.

- Documents inside one batch cannot see each other as neighbours. The batch
    is 1/16 of the graph size (64–512), so there is no measured recall loss —
    but very small collections fall to the serial path below 1024 vectors.

- The build is parallel within a batch, and because writing links is serial
    the speed-up is not linear in core count: about 1.9× on eight cores.

- **Ordered indexes are rebuilt from the documents and cost on
    write.** `@sorted` keeps about 20 bytes a row for a number,
    plus the text for a `text` field, and none of it is in the
    file: two ordered indexes over a million rows took about 40 ms to build,
    which the first query reading each after an open pays, and took a bulk
    load from 0.52 s to 0.88 s. A key that grows with the id, a
    creation time, is appended; a random one is inserted into a sorted run of
    at most 512 entries.

- File writes are buffered. If the process crashes before
    `sync` or `checkpoint`, the last writes can be lost.
    In embedded use calling `sync` is the caller's job.

## Query behaviour

Which filters use an index, what `order` and
`near` cost, and the few ways a query behaves differently from
what you might expect.

- **Only an `and` chain of equalities and ranges reaches an
    index;** `!=`, `~` and anything under
    `or` scan every row.

- **`order` reads every match** unless it is over one
    `@sorted` field with a `limit`.

- **`put` does not check that a vector is finite.**

Details

- **`near` over a `sparse<N>` field ranks
    only the documents sharing a dimension with the query**, by dot product
    and exactly — it takes no `ef`. A document with no dimension in
    common scores nothing and is not returned, as a document holding no word
    of a `match` query is not; with weights of both signs, one that
    shares dimensions can score below zero and still be on the page. The cost
    grows with the query's non-zero entries: SPLADE expands a query to 40 to 65
    of them, which reach most of the corpus.

- **Only an `and` chain reaches an index** — equality
    (`=` or `in [..]`) over a `@hash` field
    or over `id`, and comparisons (`<`,
    `<=`, `>`, `>=`, `=`)
    over a `@sorted` field. `!=`, `~`,
    `has`, a comparison on a field without `@sorted`, and
    anything under an `or` is a full scan: every matching row is
    decoded and evaluated. `~` is unranked substring matching;
    ranked retrieval is `match` over a `@text` field,
    which does reach an index.

- A `@hash` equality answers as the scan does, a float's
    `-0.0` and `0.0` as one value. Inside a list or a
    vector a `-0.0` keeps its own key, so an equality over a
    hashed list or vector holding one can miss what the scan finds.

- A comparison reaches an ordered index only when its literal compares
    exactly as the scan would: `price < 12.5` on an
    `int` field, or an integer past 253 against a
    `timestamp`, is tested row by row. A range wider than half the
    collection is scanned rather than read from the index.

- An `in` list reaches the index only when **every** element
    resolves to a value the field's type can express. One element it cannot —
    `year in [2024, "abc"]` — sends the whole list back to the scan,
    because a union missing an element's rows would be a wrong answer rather
    than a slow one.

- A filtered `near` reads its filter's rows only until more of
    them match than the ANN would measure anyway (`ef × m0`, 3 200
    at the defaults), so a filter matching many rows costs little: over
    200 000 × 128, `year >= 2020`, 23% of the rows, takes
    1.1 ms against the ANN's 0.14. **A selective filter is read in full**
    — 17.9 ms for 1% of the rows — and its set is searched exactly, so recall
    is 10/10. An index on the filtered field takes that scan away: 0.16 ms
    with `@sorted`.

- **`order` reads every match, unless it is over a
    `@sorted` field and has a `limit`.** Only
    `offset + limit` rows are put in order — the rest are
    partitioned away in linear time — but every matching row's key is still
    read, since the first twenty cannot be known without seeing all of them.
    The cost follows the number of matches, not `limit`: over a
    million rows `order created desc limit 20` takes 26.8 ms, and
    0.001 ms once `created` is `@sorted`, because the
    index is walked in order and the walk stops at the page. A filter the
    index cannot narrow is tested on each row the walk passes, and one that
    has not filled the page within an eighth of the collection hands the query
    back to the scan — at worst about 1.25× the scan alone. An order over two
    keys, or over a `float` field holding a `NaN`, which
    has no place in an order, takes the sorting path. Without
    `order`, a scan stops at `offset + limit`
    matches.

- **Two collations: Unicode's root and Turkish.**
    `collate und` orders text as ICU's root collation does, for
    every script, and `collate tr` as its Turkish one; another
    language's own order — Swedish `å ä ö` after `z`,
    German phonebook order — is the root's until its tailoring is added.
    The order is ICU's exactly but for a mark that composes with its letter
    across another mark between them, or marks out of their canonical
    order, which ICU's normalisation finds and fenecdb sorts as marks of
    their own: five in 1 250 000 random pairs. A key collated in the query
    over a field declared in no collation is sorted rather than walked —
    890 ms for a million names, against 374 ms by their bytes — and a
    comparison in `where` over it compares bytes; a field
    declared in the collation has both. In the browser a statement that needs
    a script's weights the module has not been handed is refused until it has
    them ([JavaScript](https://fenecdb.com/docs/javascript#collation)).

- **`nocase` folds case, and only case.** Unicode's full
    case folding (Turkish's in `collate tr`), with no
    normalisation: `é` as one code point and as `e`
    and a combining accent are two texts, as are `résumé` and
    `resume`, and a full-width letter is not its ASCII one. It is
    a text field's alone, not a list's or a `json` field's, and a
    field is made `nocase`, or no longer, by being built again, not
    in place. A scan over one folds each row as far as it is compared — a
    name by the scan 5.5 → 6.8 ms over 100 000 rows — and an order without a
    `@sorted` index folds each row's key once, 8.5 → 16.2 ms
    ([Case-insensitive text](https://fenecdb.com/docs/fenecql#nocase)).

- `near` and `order` cannot be used together;
    `near` determines the ordering by similarity.

- A query vector can be given three ways: a vector, a list, or pgvector's
    text form (`near embed '[1,0,0]'`), which is the natural one
    when typing by hand in the shell. None of the three fails silently — unparsable
    text and a non-numeric component both error.

- **`put` does not check that a vector is finite.** A NaN or
    infinite component enters the index and makes distances meaningless; the
    check exists only on the `fenec import` path. In a
    `vector<N, f16>` field an out-of-range value silently
    becomes `inf` or drops to zero below about 6×10-8.

- The browser module needs WebAssembly SIMD: Chrome 91, Firefox 89, Safari
    16.4 (March 2023) or later. On an engine without it,
    `Fenec.open` fails with a `FenecError` that says
    so.

- `now()` does not work in the browser: wasm32-unknown-unknown
    has no clock and the call errors. Pass the time as a parameter with
    `Date.now()`.

## fenec-server

The server is one process over one file, reached over HTTP, and it has no
encryption of its own.

- **No TLS:** put a TLS terminator in front off the machine.

- **No interactive transactions:** a `/batch` is the unit
    that lands whole.

- **Too many connections are refused,** not queued.

Details

- **No TLS.** A token travels in the clear on an open network.
    Listening on a non-loopback address with no token is refused without
    `--insecure`.

- **No transaction held open across requests.** There is one writer,
    and a transaction a client held open between its requests would hold
    every other write for as long as the client took. A `/batch`
    lands its statements whole or not at all, and an
    `Idempotency-Key` makes it safe to retry. A
    `compact` cannot be put back, so a batch holding one runs each
    statement on its own.

- A running statement, such as a long `compact`, is not cut in
    the middle; a client that goes away does not cancel it.

- **A ceiling, not a pool.** Every connection is an OS thread.
    `--max-connections` rejects the excess with 503 rather than
    queueing it; put a proxy in front if you want a queue.

- A keep-alive connection silent for 60 s is closed
    (`--idle-timeout <s>`, 0 for never).

## fenec-http

The HTTP endpoint runs inside `fenec-server` and keeps to a plain
subset of HTTP.

- **No TLS.** Without a token it binds only to the local
    machine.

- **Bodies up to 64 MiB,** no chunked uploads, and paging by
    `limit` and `offset`.

- **Subscriptions are one-way** server-sent events, not
    WebSockets.

Details

- **Not a separate process.** It opens as a second listener via
    `fenec-server --http`; there is no standalone binary, because two
    processes opening the same file would corrupt it.

- No TLS. Without a token it will not bind a non-loopback address. The
    server's own token, `--http-token`, is a single shared secret
    with full access; per-user access with scopes and expiry is a JSON Web
    Token held to a policy ([Scoped
    access](https://fenecdb.com/docs//docs/http#scoped-access)).

- `Transfer-Encoding: chunked` is unsupported (411). The body is
    capped at 64 MiB and the header block at 64 KiB; above that, 413 or
    431.

- Pagination is `limit` and `offset` — no cursor, no
    `Range` header — each a number or, in FenecQL, a parameter
    (`limit $2 offset $3`), so every page is one statement. The response is built in one piece, so a very
    large `select` holds that much memory on the server.

- A field whose name matches a clause name (`select`,
    `order`, `limit`, `offset`,
    `count`, `where`) cannot be filtered from the query
    string. `POST /query` and `fenec-server` are
    unaffected.

- No `ETag`, no `Last-Modified`, no conditional
    requests. Responses are not marked cacheable.

- **A subscription is one-way.** The server sends changes over SSE and
    the client writes with ordinary `POST`, `PATCH` and
    `DELETE`. There is no WebSocket; for the same job it would
    demand framing, masking and ping/pong.

- A subscription refuses `order`, `limit`,
    `offset` and `count` with a 400: a shape is a set,
    not a window, and an incremental diff cannot express a moving one.

- **A write to any collection wakes every subscriber.** That is a
    necessity rather than noise: the counter and the ring are shared across
    collections, so a subscriber to a quiet collection that did not wake would
    keep its cursor in place and be reseeded once other writes overflowed the
    ring. Waking moves the cursor forward, and no empty batch is sent for a
    change that does not match the shape. The cost is one read lock and an
    empty scan.

- **A batch holding a compact is not one block.** A
    `compact` rewrites the file, which cannot be put back, so a
    `POST /batch` holding one runs each statement on its own,
    stops at the first error and reports how many were applied. Every other
    batch — a `create`, a `drop` or a `create
    index` among its statements — lands whole or not at all.

## Import

`fenec import` copies one table into one collection, from
SQLite or PostgreSQL, and can follow a PostgreSQL table as it changes.

- **Checkpoint a SQLite file's WAL first;** one with an unprocessed
    WAL is not read.

- **A vector with a NaN or an infinity stops the import** at that
    row.

- **`--follow` needs `wal_level = logical`** and
    an integer key.

Details

- One table becomes one collection. Multi-table migration is out of
    scope; a `JOIN` on a key reads as a `lookup`
    afterwards.

- `fenec import` is in the default build but is a separate
    feature. A binary built with `--no-default-features` has no such
    subcommand, and says so before exiting.

- `--where` sees only the built-in functions; plugin functions
    cannot be used. The filter is evaluated without borrowing the target
    database, so that writing can happen in the same loop.

- **A SQLite file with an unprocessed WAL is not read.** The main file
    shows stale data and stopping beats silently returning it. Run
    `sqlite3 <db> "PRAGMA wal_checkpoint(TRUNCATE)"` first.
    `WITHOUT ROWID` tables are not read either.

- No TLS to the PostgreSQL server read from, and no md5 authentication:
    scram-sha-256 or cleartext.

- PostgreSQL `timestamp(tz)` values are truncated to
    milliseconds.

- **A non-finite component in a vector column stops the import** and it
    says which row. A NaN makes a distance incomparable; even if the index
    accepted it, the result would be meaningless.

- An import that stops halfway cannot be rolled back — there are no
    transactions.

- **`--follow` needs `wal_level = logical` and an
    integer key** in the replica identity: a changed row has to be found
    again by id. It holds its file as its only writer, and its slot holds the
    server's log until confirmed — drop the slot when you stop following for
    good ([Following changes](https://fenecdb.com/docs/import#following)).

- A column added to the followed table after the copy is left out; one
    the collection needs that is dropped from it stops the follower.