Ternlight WASM Embeddings: Browser Semantic Search in 7 MB

Updated November 12, 2026: corrected the API examples to match @ternlight/base 0.1.1 (embed() is synchronous and similar() takes strings), fixed the browser-support and SIMD guidance after inspecting the shipped .wasm, replaced unsourced benchmark figures with the project’s published numbers, and moved the Transformers.js example to @huggingface/transformers.

If you’ve ever wired up semantic search on a frontend app, you know what the stack usually looks like: an embeddings API call per query, a vector database subscription, CORS headers to wrangle, and a network round-trip on every keystroke that you can’t optimise past. Ternlight’s own README puts that round-trip at 100–300 ms, plus per-call billing and every network hiccup showing up in search latency.

Ternlight dissolves that whole stack. It’s a WebAssembly package that bundles a distilled MiniLM model, the inference engine, and the tokenizer into a single npm install. Everything runs on the user’s CPU. This post covers how Ternlight works under the hood, an implementation that matches the API that actually ships in the npm package, and the gotchas the launch posts skipped over — including one we confirmed by inspecting the published .wasm file ourselves.

What is Ternlight and how does its 7 MB WASM bundle actually work?

Ternlight is a sentence embedding library built by distilling all-MiniLM-L6-v2 using ternary quantization-aware training (QAT), then compiling the Rust inference engine to WebAssembly with SIMD. The base tier reaches 0.844 Spearman correlation to the MiniLM teacher and 0.465 NDCG@10 on SciFact retrieval. At 7.2 MB gzipped it is roughly a third the size of the 23 MB int8-quantized ONNX export of MiniLM (the fp32 ONNX file is 90.4 MB).

The ternary quantization is the architecturally clever part. Instead of float32 weights, each parameter is constrained to one of three values: −1, 0, or +1, which pack to 2 bits each, while the embedding table is int4. Multiplying by −1, 0 or +1 means the matrix multiplications reduce to additions and subtractions, which a hand-written SIMD forward pass can vectorise well. That is why Ternlight gets to ~5 ms per embedding on a modern laptop without any GPU dependency.

The library ships two tiers with the same API. Figures below come from the project’s repository benchmark table (shipped int4 builds, M-series Mac, Node/V8):

Ternlight tiers, as published by the project (v0.1.x)

Metric	@ternlight/mini	@ternlight/base
Wire size (gzipped wasm)	5.0 MB	7.2 MB
Latency (p50 per embed)	2.5 ms	5.1 ms
Throughput (single thread)	~400 emb/s	~195 emb/s
Spearman vs MiniLM-L6 teacher	0.820	0.844
SciFact NDCG@10	0.439	0.465

We checked the sizes against the packages themselves: gzipping the pkg-web wasm files from the 0.1.1 tarballs (gzip -9c tern_engine_bg.wasm | wc -c) gave 7,170,199 bytes for base and 5,037,560 bytes for mini, in line with the published figures. The model produces 384-dimensional float vectors that are already unit-length, so cosine similarity is just a dot product. One limit to know up front: input is truncated at 128 tokens (about 95 English words), silently — chunk long documents before you embed them.

How do you install and initialise Ternlight in a JavaScript project?

Installation is a single npm command. The package requires no native addons, no Python, and no model download step — the model ships inside the wasm. It runs in Node ≥ 18, browsers via a bundler, Cloudflare Workers, Vercel Edge, Deno, and Bun.

npm install @ternlight/base
# or the ~5 MB variant:
npm install @ternlight/mini

In browsers the wasm is imported as an ES module, so your bundler needs to understand it. Per the project docs, webpack 5 and Next.js need experiments: { asyncWebAssembly: true }; Vite 8.1 and later need nothing, while older Vite versions need vite-plugin-wasm. Without a bundler, import from @ternlight/base/web and call its init() once before embedding.

The API surface is intentionally minimal. embed(text) returns a Float32Array of 384 dimensions and is synchronous — there is no promise to await. cosineSim(a, b) compares two vectors. similar(query, corpus, { topK }) takes an array of strings, embeds the query and every corpus item, and returns { text, sim } rows sorted by similarity (this is what the 0.1.1 type definitions declare):

import { embed, cosineSim, similar } from '@ternlight/base';

const docs = [
  'Reset your password via the account settings page.',
  'Change your billing method under Subscription.',
  'Contact support for account deletion requests.',
];

// Quick path for small lists: similar() embeds the corpus on every call
const results = similar('forgot my password', docs, { topK: 3 });
// [{ text: 'Reset your password via...', sim: ... }, ...]

// Repeated searches: embed the corpus ONCE, then rank with cosineSim()
const index = docs.map((text) => ({ text, v: embed(text) }));
function search(query, topK = 3) {
  const q = embed(query);
  return index
    .map(({ text, v }) => ({ text, sim: cosineSim(q, v) }))
    .sort((a, b) => b.sim - a.sim)
    .slice(0, topK);
}

Because similar() re-embeds the whole corpus on every call, use it only for short lists; for anything you search more than once, keep the precomputed vectors as above. For a sense of the scores you can expect, the README lists 0.88 for “reset my password” vs “I forgot my password” with the base model. The official demo at ternlight.dev/demo lets you pick a tier and index React’s documentation in the browser with zero network calls during search.

How do I build a complete semantic search feature with Ternlight?

The production pattern has two files: a Web Worker that owns all embedding logic, and a main thread file that dispatches and renders. Because embed() is synchronous and CPU-bound, the worker is what keeps the page responsive. Here’s the implementation, written against the 0.1.1 API:

// search.worker.js — run all Ternlight calls off the main thread
import { embed, cosineSim } from '@ternlight/base';

let corpus = [];
let vectors = [];

self.onmessage = ({ data }) => {
  if (data.type === 'index') {
    corpus = data.documents;
    vectors = corpus.map((doc) => embed(doc)); // sync, ~5 ms each
    self.postMessage({ type: 'indexed', count: corpus.length });
  }

  if (data.type === 'search') {
    const q = embed(data.query);
    const results = vectors
      .map((v, i) => ({ text: corpus[i], sim: cosineSim(q, v) }))
      .sort((a, b) => b.sim - a.sim)
      .slice(0, data.topK ?? 5);
    self.postMessage({ type: 'results', results });
  }
};

// App.js — main thread stays fully responsive
const worker = new Worker(new URL('./search.worker.js', import.meta.url), {
  type: 'module',
});

// Index on startup
worker.postMessage({ type: 'index', documents: myDocuments });

// Query on user input
function search(query) {
  worker.postMessage({ type: 'search', query });
}

worker.onmessage = ({ data }) => {
  if (data.type === 'results') renderResults(data.results);
  if (data.type === 'indexed') console.log(`Indexed ${data.count} docs`);
};

The Web Worker pattern is non-negotiable for production. At the project’s published single-thread throughput of ~195 embeddings per second for base, a 500-document corpus is about 2.6 seconds of uninterrupted CPU work on an M-series Mac — and the demo page’s tier picker lists a lower 125 embeddings/sec for base and 210/sec for mini, so plan for slower in real browsers and on older devices. Leaving that on the main thread destroys your Interaction to Next Paint score; moving it to a worker costs a few lines of boilerplate and removes the problem.

The worker file is just an ES module, and Vite and webpack both bundle it via the new URL() Worker constructor syntax shown above — remember the wasm bundler setting from the previous section applies to the worker too. For a complete example wiring an AI feature into a form UI, Building Smart Web Forms with AI & JavaScript covers the surrounding state management patterns.

Does WASM outperform WebGPU for browser embedding models?

Ternlight doesn’t give you the choice: it is deliberately CPU-only WASM, and the project publishes no WebGPU comparison. An earlier version of this section quoted WASM-vs-WebGPU latency ranges that we could not trace to a primary benchmark, so we have removed them rather than repeat unsourced numbers.

The architectural argument still holds as a rule of thumb. Scheduling work on a GPU has fixed per-dispatch costs (buffer uploads, command submission, reading results back) that only pay off when there is enough arithmetic per call. A small embedding model processing one short sentence at a time gives the GPU very little to do, so that overhead is a large share of each call, while a SIMD CPU path has no such setup cost. As the model and batch size grow, the balance shifts toward the GPU — which is why Transformers.js exposes WebGPU as an opt-in by setting device: 'webgpu' rather than forcing it.

Comparison: Browser embedding backends for a small embedding model

Backend	GPU Required	Per-call overhead	Best For
WASM + SIMD (Ternlight)	No	None beyond the CPU work (~5 ms p50 for base, published)	Small embedding models, one query at a time
WebGPU (e.g. Transformers.js)	Yes	GPU dispatch and readback on every call	Larger models and big batches; measure on your target devices
Remote API	N/A	Network round-trip (100–300 ms per Ternlight’s README)	Max accuracy, multilingual, no bundle cost

My recommendation: for an English, sub-10 MB embedding model used for search-as-you-type, start with WASM and only reach for WebGPU after you have measured a real gain on the devices your users actually have. As noted in The 3-Second Rule: JavaScript Performance Checklist for 2026, picking the right execution path for a given payload size is the difference between milliseconds and seconds at the user-perceived layer.

What are the production gotchas nobody warned you about with Ternlight?

These are the ones that will bite you if you miss them.

There is no non-SIMD fallback. Each package ships a single .wasm build per target, and its target_features custom section — which we read straight out of the 0.1.1 binaries with a short Python section parser, without executing anything — declares +simd128 alongside +reference-types, +multivalue, +bulk-memory, +nontrapping-fptoint and +sign-ext. Per the WebAssembly feature table, fixed-width SIMD arrived in Chrome 91, Firefox 89 and Safari 16.4, and reference types in Chrome 96. Safari added 128-bit WebAssembly SIMD in Safari 16.4, released March 27, 2023. On an older engine the module won’t slow down — it won’t compile. Detect support first and fall back to a server endpoint. The maintained way is wasm-feature-detect:

import { simd } from 'wasm-feature-detect';

if (await simd()) {
  // load the Ternlight worker
} else {
  // fall back to your server-side embeddings endpoint
}

If you’d rather not add a dependency, the probe that library uses is a tiny module that must validate:

const simdSupported = WebAssembly.validate(new Uint8Array([
  0,97,115,109,1,0,0,0,1,5,1,96,0,1,123,3,2,1,0,10,10,1,8,0,65,0,253,15,253,98,11
]));

(An earlier version of this post printed a byte sequence that was missing the i32.const 0 operand, so it could never validate and always reported “no SIMD”. The bytes above are copied from wasm-feature-detect 1.10.1.)

It’s single-threaded — and that keeps COOP/COEP out of your way. The published throughput figures are single-thread, and the project documents no threaded build, so you don’t need cross-origin isolation to run it. That matters, because SharedArrayBuffer (which multi-threaded WASM needs) is only available when the page is cross-origin isolated, which MDN says requires both of these headers:

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

Under require-corp, cross-origin no-cors resources — analytics pixels, chat widgets, CDN images — load only if they send a Cross-Origin-Resource-Policy header that allows embedding; COEP: credentialless is the gentler alternative, sending those requests without cookies. If you need more indexing throughput, running two or three ordinary workers over different slices of the corpus parallelises the work without any of this, at the cost of each worker holding its own copy of the model.

Assume English only. The project doesn’t state Ternlight’s language coverage, but its teacher model, all-MiniLM-L6-v2, is listed on its model card as English only, and Ternlight uses the same BERT WordPiece vocabulary (30,522 tokens, per the package’s type definitions). Treat non-English and cross-language queries as unsupported until you’ve evaluated them on your own data. If your app needs cross-language search, use Transformers.js with a dedicated multilingual model instead.

The 128-token cap is silent. Anything past 128 tokens (~95 English words) is dropped without an error. Split documents into paragraph- or section-sized chunks, embed each chunk, and map hits back to the parent document.

Performance varies by device. All published latency and throughput numbers are from M-series Macs, and the project’s own pages already disagree a little (4.9 ms p50 in the npm README, 5.1 ms on the repo and ternlight.dev; 195/s in the repo table vs 125/s in the demo picker). Always measure on your actual target devices before you put performance claims in your product copy.

How do you handle the cold start indexing delay in a real app?

The cold start hit is the biggest practical hurdle: indexing a corpus on first load takes seconds, scaling linearly with document count and device speed. Three patterns cover most cases, each matched to a different corpus type:

Option 1: Pre-embed at build time (best for static corpora). If your documents don’t change at runtime — docs, FAQs, product listings — embed them during your build step in Node and ship the vectors. Since the vectors are fixed-length Float32Arrays, a raw binary file is far smaller than JSON:

// scripts/preembed.mjs — run via: node scripts/preembed.mjs
import { embed } from '@ternlight/base';
import { readFileSync, writeFileSync } from 'node:fs';

const docs = JSON.parse(readFileSync('content/docs.json', 'utf8'));
const out = new Float32Array(docs.length * 384);
docs.forEach((d, i) => out.set(embed(d.text), i * 384));
writeFileSync('public/embeddings.bin', Buffer.from(out.buffer));
writeFileSync('public/docs.json', JSON.stringify(docs.map((d) => d.text)));
console.log(`Pre-embedded ${docs.length} documents`);

// in the worker: load vectors instead of embedding on startup
const buf = await (await fetch('/embeddings.bin')).arrayBuffer();
const all = new Float32Array(buf);
const vectors = [];
for (let i = 0; i < all.length; i += 384) vectors.push(all.subarray(i, i + 384));

The size is simple arithmetic: 384 floats × 4 bytes = 1,536 bytes per document, so 500 documents is about 750 KB of vectors (before compression), versus roughly twice that or more as a JSON array of decimal numbers. Startup becomes one fetch instead of seconds of CPU work, and only the user’s query needs embedding at runtime. Embed with the same package and version you ship to the browser — vectors from different tiers or releases aren’t comparable.

Option 2: Cache to IndexedDB. On first visit, embed and persist vectors to IndexedDB keyed by a content hash plus the Ternlight version. On return visits, load from IndexedDB instead of re-embedding. Invalidate the cache when the content hash or library version changes. This suits dynamic corpora where build-time pre-embedding isn’t practical.

Option 3: Progressive indexing with partial results. Embed documents in small batches after initial page load, posting partial results while the index fills. This is the right approach for large, frequently updated corpora where neither build-time embedding nor a stable cache is viable. If those documents end up in a server-side store as well, pgvector vs MongoDB Atlas Vector Search for Node.js RAG compares the two most common backends.

How does Ternlight compare to Transformers.js for JavaScript semantic search?

Transformers.js is the Swiss Army knife: many model types via ONNX Runtime Web, an opt-in WebGPU backend, multilingual models, image+text capabilities. Ternlight is a scalpel: one model family, one job, aggressively optimised for that job. The tradeoffs fall out clearly once you know your requirements.

Note the package name: current Transformers.js is published as @huggingface/transformers (4.3.2 as of October 27, 2026); the old @xenova/transformers package stopped at 2.17.2 in May 2024. Model precision is chosen with the dtype option, and for MiniLM the 8-bit ONNX file is 23 MB versus 90.4 MB at fp32:

// Transformers.js — more setup, more model choice
import { pipeline } from '@huggingface/transformers';

const extractor = await pipeline(
  'feature-extraction',
  'Xenova/all-MiniLM-L6-v2',
  { dtype: 'q8' } // add device: 'webgpu' to opt into the GPU backend
);
const output = await extractor('your text', {
  pooling: 'mean',
  normalize: true,
});

With Ternlight you call embed(text) directly, skip the async pipeline and model download, and ship a 7.2 MB bundle instead of a 23 MB model file. Because Ternlight is distilled from that exact MiniLM model, its 0.844 Spearman score tells you how closely it tracks the larger model. For the same “one simple component or a flexible system?” reasoning applied to agents, see AI Agent Orchestration: One Agent or Many?.

Comparison: Ternlight vs Transformers.js vs Remote API

Option	Download Size	Published latency	Multilingual	Best For
Ternlight (base 0.1.1)	7.2 MB gzipped (engine + model + tokenizer)	~5 ms p50, M-series CPU	No (English teacher)	English search, zero backend
Transformers.js + all-MiniLM-L6-v2 (q8)	23 MB model file, plus the ONNX Runtime Web wasm	Not published — measure on your devices	Yes, with other models	Multi-model, multilingual, WebGPU option
Remote Embeddings API	~0 KB	100–300 ms round-trip	Yes	Max accuracy, no bundle cost

• Ternlight runs a 384-dim MiniLM-distilled embedding model at ~5 ms p50 per embedding on an M-series CPU, with engine, weights and tokenizer in a 7.2 MB gzipped wasm — no backend required.
• In 0.1.1, embed() is synchronous and similar() takes strings and re-embeds them every call: precompute corpus vectors and rank with cosineSim().
• Treat the Web Worker boundary as mandatory: embedding is synchronous and CPU-bound, and even at the published ~195 embeddings/sec a 500-document corpus is seconds of blocked UI.
• The shipped wasm requires WebAssembly SIMD (Chrome 91+, Firefox 89+, Safari 16.4+) and there’s no fallback build — feature-detect and route older browsers to a server endpoint.
• Inputs over 128 tokens are silently truncated; chunk documents before embedding.
• Pre-embed static corpora at build time into a binary Float32Array file (1,536 bytes per document) so startup is a fetch, not seconds of CPU work.
• Assume Ternlight is English-only; for multilingual search or multi-model needs, use @huggingface/transformers with a dedicated multilingual model.

Frequently Asked Questions

Does Ternlight work without a backend server?

Yes. Ternlight’s WASM bundle (7.2 MB gzipped for base) includes the model engine, weights, and tokenizer. Install it via npm, import embed() and cosineSim(), and all inference runs on the user’s CPU. No server, API key, or network request is needed after the initial bundle download.

How does Ternlight compare to Transformers.js for browser semantic search?

Ternlight is smaller (7.2 MB gzipped versus a 23 MB 8-bit ONNX MiniLM file plus the runtime), synchronous, and has a three-function API. Transformers.js (now @huggingface/transformers) supports many models, multilingual embeddings and an opt-in WebGPU backend. For English-only semantic search, Ternlight wins on bundle size and API simplicity; for multilingual or multi-task needs, use Transformers.js.

Is Ternlight accurate enough for production use?

The base model scores 0.844 Spearman correlation to its all-MiniLM-L6-v2 teacher and 0.465 NDCG@10 on SciFact, per the project’s README. Its README example scores ‘reset my password’ vs ‘I forgot my password’ at 0.88 cosine similarity. That makes it a reasonable fit for English FAQ matching, knowledge-base search and intent routing — but pick your similarity threshold by testing against real queries from your own users.

What browsers support Ternlight’s WebAssembly model?

The shipped wasm is compiled with WebAssembly SIMD and has no fallback build, so it needs SIMD support: Chrome 91+, Firefox 89+ and Safari 16.4+ (Chromium-based Edge follows Chrome), and Chrome 96+ for the reference-types feature it also declares. Older browsers can’t compile the module, so feature-detect with wasm-feature-detect and fall back to a server endpoint. Ternlight is single-threaded, so it does not need COOP/COEP headers.

How do I stop Ternlight from blocking the main thread?

Run Ternlight inside a Web Worker. Create a module worker that imports embed and cosineSim from @ternlight/base, handles ‘index’ and ‘search’ messages, and posts results back to the main thread. Because embed() is synchronous and CPU-bound, indexing hundreds of documents takes seconds; keeping that work off the main thread keeps the UI responsive.

Ternlight is one of those rare tools that collapses a genuinely complex architecture — embedding API, vector store, search backend — into a single npm install. What this walkthrough adds beyond the launch coverage: the 0.1.1 API is synchronous and string-based, the shipped wasm hard-requires SIMD with no fallback, and inputs stop at 128 tokens. Pair that with the Web Worker offload pattern, the binary pre-embedding shortcut, and the IndexedDB cache strategy, and you have a semantic search feature that is fast, private, and ships entirely in your frontend bundle. If you later need to cache LLM answers rather than search documents, the same embeddings idea powers Semantic Caching for LLMs: Architecture, Thresholds, Risks.

If you’ve been deferring semantic search because “it requires a backend,” Ternlight removes that blocker for English content. Start with the official demo at ternlight.dev/demo, then drop the Web Worker pattern above into your app. Drop a comment below or subscribe to NexGismo for posts like this delivered weekly.