Class LiteRtEmbeddingEngine
- Namespace
- LiteRtLmSharp
- Assembly
- LiteRtLmSharp.dll
An embedding engine: loads an embedding model (such as EmbeddingGemma 2) and turns text into vectors for semantic search, retrieval-augmented generation, clustering or classification.
public sealed class LiteRtEmbeddingEngine : IDisposable
- Inheritance
-
LiteRtEmbeddingEngine
- Implements
- Inherited Members
Remarks
An embedding engine is independent of LiteRtEngine: it does not count toward the one-live-engine rule, so an app can keep a chat engine and an embedding engine loaded at the same time.
Calls on one embedding engine are serialized internally, so it is safe to share across threads; a call
waits for the one in progress. Dispose() also waits for it before releasing the native
engine. The runtime does not apply a model's task instructions: prepend them to the text yourself (see
docs/embeddings.md). Requires native LiteRT-LM v0.18.0+ (c/embedding_engine.h).
Properties
Dimension
Gets the length of the vectors the model produces before any truncation by
OutputDimensions, or null when the model file does not
declare it or its metadata could not be read (see Read(string)).
public int? Dimension { get; }
Property Value
- int?
Methods
Dispose()
Releases the native engine after any call in progress completes.
public void Dispose()
Embed(string, LiteRtEmbeddingOptions?)
Computes the embedding of text.
public float[] Embed(string text, LiteRtEmbeddingOptions? options = null)
Parameters
textstringThe text to embed, with any task instruction the model expects already prepended.
optionsLiteRtEmbeddingOptionsPer-call options;
nulluses the runtime defaults (normalized, full length).
Returns
- float[]
The embedding vector.
Exceptions
- ObjectDisposedException
The engine has been disposed.
- LiteRtException
The native computation failed; the message carries the runtime's reason.
EmbedAsync(string, LiteRtEmbeddingOptions?, CancellationToken)
Computes the embedding of text on the thread pool.
public Task<float[]> EmbedAsync(string text, LiteRtEmbeddingOptions? options = null, CancellationToken cancellationToken = default)
Parameters
textstringThe text to embed, with any task instruction the model expects already prepended.
optionsLiteRtEmbeddingOptionsPer-call options;
nulluses the runtime defaults (normalized, full length).cancellationTokenCancellationTokenCancels the wait for a call in progress; a computation that has started runs to completion (the native layer cannot interrupt it).
Returns
Exceptions
- ObjectDisposedException
The engine has been disposed.
- LiteRtException
The native computation failed; the message carries the runtime's reason.
EmbedBatch(IReadOnlyList<string>, LiteRtEmbeddingOptions?)
Computes the embeddings of several texts in one native call.
public float[][] EmbedBatch(IReadOnlyList<string> texts, LiteRtEmbeddingOptions? options = null)
Parameters
textsIReadOnlyList<string>The texts to embed, each with any task instruction already prepended.
optionsLiteRtEmbeddingOptionsPer-call options applied to every text;
nulluses the runtime defaults.
Returns
- float[][]
One vector per text, in the same order. Empty when
textsis empty.
Remarks
The runtime embeds the texts one after another, so a batch saves per-call overhead but not compute time. If any text fails (for example, a text too long for the loaded signatures), the whole call fails.
Exceptions
- ArgumentException
An element of
textsisnull.- ObjectDisposedException
The engine has been disposed.
- LiteRtException
The native computation failed; the message carries the runtime's reason.
EmbedBatchAsync(IReadOnlyList<string>, LiteRtEmbeddingOptions?, CancellationToken)
Computes the embeddings of several texts in one native call, on the thread pool.
public Task<float[][]> EmbedBatchAsync(IReadOnlyList<string> texts, LiteRtEmbeddingOptions? options = null, CancellationToken cancellationToken = default)
Parameters
textsIReadOnlyList<string>The texts to embed, each with any task instruction already prepended.
optionsLiteRtEmbeddingOptionsPer-call options applied to every text;
nulluses the runtime defaults.cancellationTokenCancellationTokenCancels the wait for a call in progress; a computation that has started runs to completion.
Returns
Remarks
See EmbedBatch(IReadOnlyList<string>, LiteRtEmbeddingOptions?): the texts are embedded one after another, and one failing text fails the call.
Exceptions
- ArgumentException
An element of
textsisnull.- ObjectDisposedException
The engine has been disposed.
- LiteRtException
The native computation failed; the message carries the runtime's reason.
Load(LiteRtEmbeddingEngineOptions)
Loads an embedding model and creates the engine.
public static LiteRtEmbeddingEngine Load(LiteRtEmbeddingEngineOptions options)
Parameters
optionsLiteRtEmbeddingEngineOptionsThe model path, backend and engine settings.
Returns
- LiteRtEmbeddingEngine
The loaded embedding engine. Dispose it when done.
Exceptions
- ArgumentException
The model path is empty, the file does not exist, the file is a language model (load those with Load(LiteRtEngineOptions)), or the cache directory does not exist.
- LiteRtException
Native engine creation failed; the message carries the runtime's reason (for example, a file that is not a model, or a backend the model does not support).