Class LiteRtEngine
- Namespace
- LiteRtLmSharp
- Assembly
- LiteRtLmSharp.dll
A LiteRT-LM inference engine. Heavyweight: holds the model weights. Create one per model and spawn lightweight LiteRtConversation objects from it.
public sealed class LiteRtEngine : IDisposable
- Inheritance
-
LiteRtEngine
- Implements
- Inherited Members
Methods
CreateConversation(LiteRtConversationOptions?)
Creates a new stateful conversation from this engine.
public LiteRtConversation CreateConversation(LiteRtConversationOptions? options = null)
Parameters
optionsLiteRtConversationOptions
Returns
Detokenize(ReadOnlySpan<int>)
Detokenizes token ids back to text with the model's tokenizer — the inverse of Tokenize(string). An empty span returns the empty string without calling native.
public string Detokenize(ReadOnlySpan<int> tokens)
Parameters
tokensReadOnlySpan<int>
Returns
Exceptions
- LiteRtException
The native detokenizer call failed (e.g. an out-of-range id).
- EntryPointNotFoundException
The native binary predates the tokenizer API.
Dispose()
Disposes the engine, freeing the native model weights and releasing the process-wide one-engine slot so another model can be loaded. Dispose every conversation first.
public void Dispose()
GetStartToken()
The model's configured start (BOS) token, or null when the model declares none. Reported
either as a literal string or a token-id sequence — see Kind.
public LiteRtTokenUnion? GetStartToken()
Returns
GetStopTokens()
The model's configured stop (EOS) tokens, or an empty list when none. Generation halts when the model emits any of these; each is a literal string or a token-id sequence — see Kind.
public IReadOnlyList<LiteRtTokenUnion> GetStopTokens()
Returns
Load(LiteRtEngineOptions)
Loads a model and creates the engine. Only ONE engine may be alive at a time (a second concurrent engine hangs in the native layer); a LiteRtEmbeddingEngine does not count. To switch model or backend, dispose every conversation and the engine first, then call Load(LiteRtEngineOptions) again.
public static LiteRtEngine Load(LiteRtEngineOptions options)
Parameters
optionsLiteRtEngineOptions
Returns
Exceptions
- ArgumentException
The model file does not exist.
- InvalidOperationException
Another engine is still alive in this process.
- DllNotFoundException
The native LiteRT-LM library could not be loaded; the message names the fix (the missing
LiteRtLmSharp.runtime.<rid>package, a runtime package for another architecture, or a missing system library).- LiteRtException
Native engine creation failed.
SetMinLogLevel(int)
Sets the global minimum log level (0=VERBOSE … 5=FATAL, 1000=SILENT).
public static void SetMinLogLevel(int level)
Parameters
levelint
Remarks
Above 5 (SILENT included) LiteRT also stops recording the reason of the errors its status
macros raise, so a LiteRtException then names the status (for example
INVALID_ARGUMENT) without saying why. Use 3 (ERROR) to keep the console quiet and the
reasons.
Exceptions
- DllNotFoundException
The native LiteRT-LM library could not be loaded; the message names the fix (the missing
LiteRtLmSharp.runtime.<rid>package, a runtime package for another architecture, or a missing system library).
Tokenize(string)
Tokenizes text with the model's own tokenizer and returns the token ids,
without running inference. Use it to measure a prompt's exact token cost or to budget a
conversation against MaxNumTokens before sending. The ids are
the raw tokenizer output (no chat template is applied); pair with Detokenize(ReadOnlySpan<int>) for
the round trip.
public int[] Tokenize(string text)
Parameters
textstring
Returns
- int[]
Exceptions
- LiteRtException
The native tokenizer call failed.
- EntryPointNotFoundException
The native binary predates the tokenizer API.