Table of Contents

Class LiteRtEmbeddingEngine

Namespace
LiteRtLmSharp
Assembly
LiteRtLmSharp.dll

An embedding engine: loads an embedding model (such as EmbeddingGemma 2) and turns text into vectors for semantic search, retrieval-augmented generation, clustering or classification.

public sealed class LiteRtEmbeddingEngine : IDisposable
Inheritance
LiteRtEmbeddingEngine
Implements
Inherited Members

Remarks

An embedding engine is independent of LiteRtEngine: it does not count toward the one-live-engine rule, so an app can keep a chat engine and an embedding engine loaded at the same time.

Calls on one embedding engine are serialized internally, so it is safe to share across threads; a call waits for the one in progress. Dispose() also waits for it before releasing the native engine. The runtime does not apply a model's task instructions: prepend them to the text yourself (see docs/embeddings.md). Requires native LiteRT-LM v0.18.0+ (c/embedding_engine.h).

Properties

Dimension

Gets the length of the vectors the model produces before any truncation by OutputDimensions, or null when the model file does not declare it or its metadata could not be read (see Read(string)).

public int? Dimension { get; }

Property Value

int?

Methods

Dispose()

Releases the native engine after any call in progress completes.

public void Dispose()

Embed(string, LiteRtEmbeddingOptions?)

Computes the embedding of text.

public float[] Embed(string text, LiteRtEmbeddingOptions? options = null)

Parameters

text string

The text to embed, with any task instruction the model expects already prepended.

options LiteRtEmbeddingOptions

Per-call options; null uses the runtime defaults (normalized, full length).

Returns

float[]

The embedding vector.

Exceptions

ObjectDisposedException

The engine has been disposed.

LiteRtException

The native computation failed; the message carries the runtime's reason.

EmbedAsync(string, LiteRtEmbeddingOptions?, CancellationToken)

Computes the embedding of text on the thread pool.

public Task<float[]> EmbedAsync(string text, LiteRtEmbeddingOptions? options = null, CancellationToken cancellationToken = default)

Parameters

text string

The text to embed, with any task instruction the model expects already prepended.

options LiteRtEmbeddingOptions

Per-call options; null uses the runtime defaults (normalized, full length).

cancellationToken CancellationToken

Cancels the wait for a call in progress; a computation that has started runs to completion (the native layer cannot interrupt it).

Returns

Task<float[]>

The embedding vector.

Exceptions

ObjectDisposedException

The engine has been disposed.

LiteRtException

The native computation failed; the message carries the runtime's reason.

EmbedBatch(IReadOnlyList<string>, LiteRtEmbeddingOptions?)

Computes the embeddings of several texts in one native call.

public float[][] EmbedBatch(IReadOnlyList<string> texts, LiteRtEmbeddingOptions? options = null)

Parameters

texts IReadOnlyList<string>

The texts to embed, each with any task instruction already prepended.

options LiteRtEmbeddingOptions

Per-call options applied to every text; null uses the runtime defaults.

Returns

float[][]

One vector per text, in the same order. Empty when texts is empty.

Remarks

The runtime embeds the texts one after another, so a batch saves per-call overhead but not compute time. If any text fails (for example, a text too long for the loaded signatures), the whole call fails.

Exceptions

ArgumentException

An element of texts is null.

ObjectDisposedException

The engine has been disposed.

LiteRtException

The native computation failed; the message carries the runtime's reason.

EmbedBatchAsync(IReadOnlyList<string>, LiteRtEmbeddingOptions?, CancellationToken)

Computes the embeddings of several texts in one native call, on the thread pool.

public Task<float[][]> EmbedBatchAsync(IReadOnlyList<string> texts, LiteRtEmbeddingOptions? options = null, CancellationToken cancellationToken = default)

Parameters

texts IReadOnlyList<string>

The texts to embed, each with any task instruction already prepended.

options LiteRtEmbeddingOptions

Per-call options applied to every text; null uses the runtime defaults.

cancellationToken CancellationToken

Cancels the wait for a call in progress; a computation that has started runs to completion.

Returns

Task<float[][]>

One vector per text, in the same order. Empty when texts is empty.

Remarks

See EmbedBatch(IReadOnlyList<string>, LiteRtEmbeddingOptions?): the texts are embedded one after another, and one failing text fails the call.

Exceptions

ArgumentException

An element of texts is null.

ObjectDisposedException

The engine has been disposed.

LiteRtException

The native computation failed; the message carries the runtime's reason.

Load(LiteRtEmbeddingEngineOptions)

Loads an embedding model and creates the engine.

public static LiteRtEmbeddingEngine Load(LiteRtEmbeddingEngineOptions options)

Parameters

options LiteRtEmbeddingEngineOptions

The model path, backend and engine settings.

Returns

LiteRtEmbeddingEngine

The loaded embedding engine. Dispose it when done.

Exceptions

ArgumentException

The model path is empty, the file does not exist, the file is a language model (load those with Load(LiteRtEngineOptions)), or the cache directory does not exist.

LiteRtException

Native engine creation failed; the message carries the runtime's reason (for example, a file that is not a model, or a backend the model does not support).