Class LiteRtEngine

Namespace

LiteRtLmSharp

Assembly

LiteRtLmSharp.dll

A LiteRT-LM inference engine. Heavyweight: holds the model weights. Create one per model and spawn lightweight LiteRtConversation objects from it.

public sealed class LiteRtEngine : IDisposable

Inheritance

object

LiteRtEngine

Implements

IDisposable

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Methods

CreateConversation(LiteRtConversationOptions?)

Creates a new stateful conversation from this engine.

public LiteRtConversation CreateConversation(LiteRtConversationOptions? options = null)

Parameters

options LiteRtConversationOptions

Returns

LiteRtConversation

Detokenize(ReadOnlySpan<int>)

Detokenizes token ids back to text with the model's tokenizer — the inverse of Tokenize(string). An empty span returns the empty string without calling native.

public string Detokenize(ReadOnlySpan<int> tokens)

Parameters

tokens ReadOnlySpan<int>

Returns

string

Exceptions

LiteRtException

The native detokenizer call failed (e.g. an out-of-range id).

EntryPointNotFoundException

The native binary predates the tokenizer API.

Dispose()

Disposes the engine, freeing the native model weights and releasing the process-wide one-engine slot so another model can be loaded. Dispose every conversation first.

public void Dispose()

GetStartToken()

The model's configured start (BOS) token, or null when the model declares none. Reported either as a literal string or a token-id sequence — see Kind.

public LiteRtTokenUnion? GetStartToken()

Returns

LiteRtTokenUnion

GetStopTokens()

The model's configured stop (EOS) tokens, or an empty list when none. Generation halts when the model emits any of these; each is a literal string or a token-id sequence — see Kind.

public IReadOnlyList<LiteRtTokenUnion> GetStopTokens()

Returns

IReadOnlyList<LiteRtTokenUnion>

Load(LiteRtEngineOptions)

Loads a model and creates the engine. Only ONE engine may be alive at a time (a second concurrent engine hangs in the native layer). To switch model or backend, dispose every conversation and the engine first, then call Load(LiteRtEngineOptions) again.

public static LiteRtEngine Load(LiteRtEngineOptions options)

Parameters

options LiteRtEngineOptions

Returns

LiteRtEngine

Exceptions

ArgumentException

The model file does not exist.

InvalidOperationException

Another engine is still alive in this process.

DllNotFoundException

The native LiteRT-LM library could not be loaded; the message names the fix (the missing LiteRtLmSharp.runtime.<rid> package, or a system prerequisite such as the VC++ Redistributable on Windows).

LiteRtException

Native engine creation failed.

SetMinLogLevel(int)

Sets the global minimum log level (0=VERBOSE … 5=FATAL, 1000=SILENT).

public static void SetMinLogLevel(int level)

Parameters

level int

Exceptions

DllNotFoundException

The native LiteRT-LM library could not be loaded; the message names the fix (the missing LiteRtLmSharp.runtime.<rid> package, or a system prerequisite such as the VC++ Redistributable on Windows).

Tokenize(string)

Tokenizes text with the model's own tokenizer and returns the token ids, without running inference. Use it to measure a prompt's exact token cost or to budget a conversation against MaxNumTokens before sending. The ids are the raw tokenizer output (no chat template is applied); pair with Detokenize(ReadOnlySpan<int>) for the round trip.

public int[] Tokenize(string text)

Parameters

text string

Returns

int[]

Exceptions

LiteRtException

The native tokenizer call failed.

EntryPointNotFoundException

The native binary predates the tokenizer API.