Serverless and Cloudflare

Run fenecdb inside a Cloudflare Worker, with one small database for each Durable Object.

Two ways to use it

Either the engine runs inside the function, one database per Durable Object, or the function sends its queries to a fenec-server server. Inside is faster; a server holds more data and more writers.

	the engine in the function	the function talks to a server
where the data is	in the object's memory, kept in its storage	in `fenec-server`'s file
a query	no network in the object: 2.5 ms for a request's `near` over 10 000 × 128	a round trip to the server
writers	one an object; a tenant an object	one a file; a node holds many
size	about 60 000 rows of 128 dimensions, 12 000 of 768 (Limits)	the machine's disk, the file mapped
fits	a database a user, a team or a document; RAG over a tenant's own corpus	one large database, many writers, replicas

Details

The browser module runs wherever V8 runs a WebAssembly module, a Cloudflare Worker among them. A Durable Object gives it what a file gives a server: one writer, and storage that outlives the process. So one object holds one database, as a tenant is one file on a fenec-server node.

The second needs nothing new: Fenec.connect(url), the Python Client or fetch, and a function's connections are HTTP requests, so there is no pool to exhaust. The rest of this page is the first.

A database a Durable Object

The object opens the engine on its first request, restores its database from storage, and saves after every write, before it answers.

import { DurableObject } from 'cloudflare:workers';
import { Fenec } from '@fenecdb/web';
import { checkpoint, persist, restore } from '@fenecdb/cloudflare';
import wasm from '@fenecdb/web/fenec.wasm';

interface Env {
  TENANT: DurableObjectNamespace<Tenant>;
}

export class Tenant extends DurableObject<Env> {
  #fenec?: Fenec;

  async #db(): Promise<Fenec> {
    if (!this.#fenec) {
      this.#fenec = await Fenec.open(wasm);       // the module as the Worker imports it
      await restore(this.#fenec, this.ctx.storage);
    }
    return this.#fenec;
  }

  async query(sql: string, params: unknown[]): Promise<unknown> {
    const db = await this.#db();
    const out = db.run(sql, params);
    await persist(db, this.ctx.storage);          // kept before the answer goes
    await this.ctx.storage.setAlarm(Date.now() + 10_000);
    return out;
  }

  async alarm() {
    await checkpoint(await this.#db(), this.ctx.storage);
  }
}

export default {
  async fetch(req, env) {
    const tenant = env.TENANT.get(env.TENANT.idFromName('acme'));
    const { sql, params } = await req.json<{ sql: string; params: unknown[] }>();
    return Response.json(await tenant.query(sql, params));
  },
} satisfies ExportedHandler<Env>;

Details

A Worker imports a .wasm file compiled, and cannot compile bytes at run time; Fenec.open takes the module so. integrations/cloudflare/example is this Worker with a tenant from the path, POST /t/<tenant>/query, and its test runs it under wrangler dev — workerd, as Cloudflare runs it — then stops and starts it over the same storage and reads the rows, a near's answer and each tenant's own database back.

What the storage holds

persist saves a full image once, then only the new writes. If storage stops part way through, the last whole database is still there.

Details

persist keeps what a file would hold: an image the first time, then only the writes since the last call — the engine's journal, as persist in the browser takes it for IndexedDB — until they outgrow half the image and a new image replaces them. Each is cut into pieces under a key-value backed object's 128 KiB a value; a SQLite-backed object takes 2 MB with the key, and {piece} can be set larger there. Every image goes under a generation of its own, and the record naming the whole one is written last, the generation before let go after it: storage that stops part way holds the last whole database. A Durable Object combines a call's puts into one atomic write besides; this does not lean on it, and a test stops the storage after every number of puts, an image's and an append's, and restores the last whole database each time.

A cold start

After an eviction or a deploy, the first request has to load the database again. Run a checkpoint from an alarm once writes stop, and that first query stays fast.

rows	first `near`, as written	after a checkpoint	every `near` after
1 000	22 ms	23 ms	2.5 ms
10 000	792 ms	36 ms	2.5 ms
50 000	1 455 ms	84 ms	2.4 ms

Details

An object evicted, or a deploy, starts the next request with no database in memory: the module instantiated, the image and the writes after it read out of storage and loaded. The graphs the image holds are restored as they were; the writes after it are applied as the write path would, their vectors linked into the graph one after another — the browser module has one thread. So a checkpoint — a new image now, the writes since folded into it — once the writes stop, from an alarm, is what keeps a start short. The first near after a restart, against the ones after it, under wrangler dev on an M1 (make cloudflare-bench, median of five restarts; rows of a text, an int and a 128-dim vector, 1 000 a request, each kept before it was answered) are in the table above.

A request's round trip through wrangler dev is in every figure. The rows went in at 3 000 to 5 700 a second, a request of 1 000 at a time with its vectors linked and kept before the answer.

Limits

An object holds about 60 000 rows of 128 dimensions, or 12 000 of 768. For more, use a fenec-server server or spread a tenant over more objects.

One writer an object. Writes to one tenant go one after another, as on a server; tenants write side by side.
Bundle collation data. Collation data for scripts the module does not carry is fetched from a URL by default; in a Worker, pass {collation: name => ...} to Fenec.open and hand the chunks from your bundle.
Install it from npm. npm install @fenecdb/cloudflare @fenecdb/web; both are published with each release.

Details

A Worker's isolate has 128 MB, the WebAssembly memory in it, and wrangler dev does not hold a Worker to it: the figures above reach past it. What the module holds, measured in Node on the same module (rows of a text, an int and a vector under HNSW):

rows	restored	written, 1 000 a request	then a checkpoint	the image
10 000 × 128	16 MB	24 MB	28 MB	6 MB
20 000 × 128	29 MB	37 MB	42 MB	12 MB
30 000 × 128	41 MB	46 MB	53 MB	18 MB
50 000 × 128	67 MB	73 MB	83 MB	30 MB
60 000 × 128	80 MB	85 MB	97 MB	36 MB
3 000 × 768	21 MB	40 MB	40 MB	9 MB
5 000 × 768	35 MB	58 MB	58 MB	16 MB
10 000 × 768	66 MB	84 MB	84 MB	32 MB
12 000 × 768	80 MB	102 MB	102 MB	38 MB

A WebAssembly memory grows and never gives back, so an object holds the most it ever held. The module keeps an index's vectors in chunks of about a mebibyte and lets a segment of documents grow no further than the record that seals it: grown by doubling, old and new side by side, 5 000 rows of 768 written 1 000 a request held 100 MB, 58 now, and 10 000 of them 150, 84 now. A restore keeps the image it reads and the documents in it, rather than copy each out: 10 000 rows of 768 restored held 97 MB, 66 now. A checkpoint takes the image a mebibyte at a time (snapshotChunks), each chunk made as it is taken out of the documents the stores hold, not a copy of them, and let go once it is in storage: made all at once, the image stood beside the rows, and 50 000 rows of 128 went from 73 to 109 MB at it, 83 now. So an object holds about 60 000 rows of 128 dimensions and 12 000 of 768, however many a request brings; past that, a fenec-server node and Fenec.connect, or a tenant's data spread over more objects.