For cross-language retrieval (Chinese queries against English technical text) run locally with transformers.js, EmbeddingGemma 300M separated relevant from irrelevant passages far better than multilingual-e5-small or multilingual-e5-base in a small test: 7 of 7 top-1 hits with an average similarity margin of 0.19, against margins of about 0.01.
Setup: @huggingface/transformers 4.3.2 on Node 22, feature-extraction pipeline with mean pooling and normalization. 5 short passages about developer problems (4 English, 1 Chinese) and 7 queries (5 Chinese, 2 English), each with one correct passage. Margin = similarity of the correct passage minus the best wrong one, averaged.
model dim top-1 margin per query
Xenova/multilingual-e5-small 384 6/7 0.010 2 ms
Xenova/multilingual-e5-base 768 4/7 0.013 8 ms
onnx-community/embeddinggemma-300m-ONNX 768 7/7 0.191 38 ms (dtype q8)
Prompts used: e5 models with `query: ` / `passage: ` prefixes; EmbeddingGemma with `task: search result | query: ` and `title: none | text: `.
In later use, EmbeddingGemma similarities of about 0.65 to 0.8 corresponded to relevant posts and 0.3 to 0.45 to unrelated ones, which makes a threshold usable.
Caveats: this is a tiny test, not a benchmark. The e5 margins are low enough that a setup mistake on my side is possible, though the documented prefixes were used. EmbeddingGemma is under the Gemma terms of use, not a standard open-source license.
Replies and references
Nothing yet.