Table of Contents

Class LiteRtEmbeddingEngineOptions

Namespace
LiteRtLmSharp
Assembly
LiteRtLmSharp.dll

Options for loading a LiteRtEmbeddingEngine.

public sealed record LiteRtEmbeddingEngineOptions : IEquatable<LiteRtEmbeddingEngineOptions>
Inheritance
LiteRtEmbeddingEngineOptions
Implements
Inherited Members

Properties

ActivationDataType

Gets the activation precision. Defaults to Float32, like ActivationDataType: EmbeddingGemma's activations exceed the float16 range, so its model card advises against float16, which is the runtime's fallback on the GPU backend. The CPU backend runs float32 either way. Float16 measured no faster on a desktop GPU but about twice as fast per sentence on a phone GPU (Adreno 650), with vectors slightly further from the CPU ones. Set Float16 to make that trade, or null to let the runtime choose (the model's preferred type, else float16 on GPU).

public LiteRtActivationDataType? ActivationDataType { get; init; }

Property Value

LiteRtActivationDataType?

Backend

Gets the backend to run the model on. Defaults to Cpu.

public LiteRtBackend Backend { get; init; }

Property Value

LiteRtBackend

Remarks

On the GPU backend the vectors differ slightly from the CPU ones (cosine similarity above 0.99 between the two in our measurement), so index and query on the same backend.

Cache

Gets where the engine keeps its compiled-artifact cache. Defaults to Default (next to the model file). A Directory(string) must already exist: the runtime does not create it, so Load(LiteRtEmbeddingEngineOptions) throws ArgumentException.

public LiteRtCache Cache { get; init; }

Property Value

LiteRtCache

MaxInputLength

Gets the longest input, in tokens, the engine prepares for, or null (default) for the limit the model declares (EmbeddingGemma 2: 1024). The engine loads the smallest of the model's input signatures that holds this many tokens, plus the shorter ones (see EmbeddingInputLengths; EmbeddingGemma 2 has 128 to 8192), so texts up to that signature's length are accepted (1500 loads the 2,048 signature). A longer text fails unless OverflowStrategy truncates or chunks it. A value above the longest signature makes Load(LiteRtEmbeddingEngineOptions) fail. Longer signatures need more memory.

public int? MaxInputLength { get; init; }

Property Value

int?

Exceptions

ArgumentOutOfRangeException

The value is zero or negative.

MinInputLength

Gets the shortest input signature, in tokens, the engine loads, or null (default) for the model's own minimum. Signatures shorter than this are not loaded, which saves memory but pads short texts to a longer signature. It must not exceed the effective maximum: MaxInputLength when set (checked by Load(LiteRtEmbeddingEngineOptions)), else the limit the model declares (1024 for EmbeddingGemma 2; the runtime rejects a larger minimum).

public int? MinInputLength { get; init; }

Property Value

int?

Exceptions

ArgumentOutOfRangeException

The value is negative.

ModelPath

Gets the path to the embedding model's .litertlm file. Must be set.

public string ModelPath { get; init; }

Property Value

string

NumThreads

Gets the number of CPU threads, or null (default) for the engine default. CPU backend only.

public int? NumThreads { get; init; }

Property Value

int?

Exceptions

ArgumentOutOfRangeException

The value is zero or negative.