Class LiteRtConversationOptions
- Namespace
- LiteRtLmSharp
- Assembly
- LiteRtLmSharp.dll
Per-conversation options: system prompt, sampler, output limit, and tools (function calling).
Requires native binaries version-matched to the bindings (the official builds from this repo's
releases are); see docs/native-abi.md for the ABI history.
public sealed record LiteRtConversationOptions : IEquatable<LiteRtConversationOptions>
- Inheritance
-
LiteRtConversationOptions
- Implements
- Inherited Members
Properties
AudioLoraPath
Path to an audio LoRA adapter (weights file) to apply to this conversation. null
(default) = no adapter. Requires an audio-capable, LoRA-enabled model and a matching
AudioLoraRank. Opened at conversation creation, so a bad path fails
fast like LoraPath. Not validated against a real audio adapter in this binding. Maps
to the C API session_config_set_audio_lora_path.
public string? AudioLoraPath { get; init; }
Property Value
ConstraintProvider
Enables custom constrained decoding for this conversation: with a provider set, each
send may carry a Constraint (regex / JSON Schema) that the
provider enforces during sampling. null (default) = no custom constraints.
public LiteRtConstraintProvider? ConstraintProvider { get; init; }
Property Value
Remarks
Mutually exclusive with EnableConstrainedDecoding (the tool-calling
constrained-decoding path) — the native runtime supports one per conversation, and
CreateConversation(LiteRtConversationOptions?) throws ArgumentException when both
are set. Maps to the C API conversation_config_set_constraint_provider (native
v0.15.0+). The LlGuidance provider, like the tool-calling one, is embedded in the native library.
EnableConstrainedDecoding
Force the model to emit valid (schema-constrained) output. Strongly recommended when Tools are set so tool-call arguments parse reliably.
public bool EnableConstrainedDecoding { get; init; }
Property Value
Remarks
Supported on every platform with the official LiteRT-LM v0.16.0 prebuilts, whose constraint
provider is embedded in the native library. Releases before 1.2.0 threw
PlatformNotSupportedException on linux-x64, where the separately shipped
provider crashed the process (google-ai-edge/LiteRT-LM#2149); that guard is gone. Tools also
work with this left false; arguments are then not grammar-constrained.
EnableSpeculativeDecoding
Speculative decoding for this conversation, overriding
EnableSpeculativeDecoding. null (default) uses the engine
setting.
public bool? EnableSpeculativeDecoding { get; init; }
Property Value
- bool?
Remarks
true on an engine loaded without speculative decoding makes the runtime load the model's
Multi-Token-Prediction (MTP) drafter lazily, on this conversation's first send, so you can enable it
only where it pays off. false turns it off for this conversation even when the engine enables
it. The runtime keeps a lazily loaded drafter on the engine and would use it for every later
conversation that leaves this unset, so from then on the binding passes the engine setting to those
conversations explicitly. Requires a model that ships an MTP drafter (such as the Gemma 4 E-series):
on a model without one, true makes the first send fail with LiteRtException.
Maps to the C API session_config_set_enable_speculative_decoding (native LiteRT-LM v0.18.0+).
EnableThinking
Toggles the model's reasoning ("thinking") mode by setting enable_thinking in the
conversation's ExtraContext. null (default) leaves it unset so the
model uses its own default; true/false force reasoning on/off.
public bool? EnableThinking { get; init; }
Property Value
- bool?
Remarks
This is the canonical use of extra context for the Gemma reasoning builds: the chat
template branches on {% if enable_thinking %}. On a model whose template does not
reference the key it is a harmless no-op. Pair with FilterThinkingFromKvCache
to keep the (often long) reasoning out of the KV cache on later turns. When both this and
ExtraContext set enable_thinking, this flag wins.
ExtraContext
Extra context merged into the conversation preface and passed to the prompt-template
renderer — a raw JSON object string, e.g. {"user_name":"Alice"}. Template
variables are referenced Jinja-style ({{ user_name }}). Null/empty = none.
public string? ExtraContext { get; init; }
Property Value
Remarks
For the common enable_thinking toggle prefer the typed EnableThinking;
this is the general escape hatch for arbitrary template variables. Must be a JSON object —
CreateConversation(LiteRtConversationOptions?) throws ArgumentException otherwise
(validated when the conversation is created, not when this property is set). Maps to the C API
conversation_config_set_extra_context.
FilterThinkingFromKvCache
Drop channel content — in practice the thinking channel — from the KV cache, so a long
reasoning block does not eat into the context window on subsequent turns. Default
false. Only meaningful alongside EnableThinking.
public bool FilterThinkingFromKvCache { get; init; }
Property Value
Remarks
Maps to the C API conversation_config_set_filter_channel_content_from_kv_cache.
History
Conversation history to restore (or a few-shot preface) — the prior turns are re-prefilled into the KV cache when the conversation is created, so the model continues as if they had just happened. Build the list with LiteRtMessage factories and capture assistant turns with ToMessage(bool); persist and reload via Serialize(IEnumerable<LiteRtMessage>) / Deserialize(string).
public IReadOnlyList<LiteRtMessage>? History { get; init; }
Property Value
Remarks
The C API has no history getter, so the round-trip is caller-owned: record each turn yourself.
Restoring is a replay through prefill, not a zero-cost snapshot — it costs a prefill of the
history and counts against MaxNumTokens. Keep the system prompt
in SystemMessage OR as a leading System message, not
both (the native side prepends SystemMessage before the history). Takes precedence
over HistoryJson when it is non-empty. Maps to the C API
conversation_config_set_messages.
HistoryJson
Raw escape hatch for History: the messages as a JSON array string (the format Serialize(IEnumerable<LiteRtMessage>) produces). Use it to pass persisted history verbatim without re-parsing into typed messages, or to include content the typed model does not cover yet (e.g. image/audio parts). Validated as a JSON array when the conversation is created — throws ArgumentException otherwise. Ignored when History is non-empty.
public string? HistoryJson { get; init; }
Property Value
LoraPath
Path to a text LoRA adapter (weights file) to apply to this conversation. null
(default) = no adapter. Requires a LoRA-enabled model, and the engine loaded with a matching
LoraRank.
public string? LoraPath { get; init; }
Property Value
Remarks
The native side opens the file when the conversation is created, so a missing or unreadable path
fails fast with LiteRtException at CreateConversation(LiteRtConversationOptions?),
not silently at first send. Despite the per-conversation setting, the runtime applies the adapter to
the whole engine: see LoraRank for what that means and for the
published Gemma 4 bundles, which accept an adapter without effect (google-ai-edge/LiteRT-LM#3173).
Maps to the C API session_config_set_lora_path.
MaxOutputTokens
Maximum output tokens per response. 0 = engine default.
public int MaxOutputTokens { get; init; }
Property Value
PromptTemplate
Overrides the chat prompt template for this conversation with a custom Jinja template string.
null (default) = the template the model file (or engine) provides. For advanced use —
a template that does not match what the model was trained on degrades output quality.
public string? PromptTemplate { get; init; }
Property Value
Remarks
Maps to the C API conversation_config_set_prompt_template (native v0.15.0+).
Inspect the result with RenderMessage(string) /
RenderPreface() before relying on it.
Sampler
Optional sampler parameters. Null = engine default.
public LiteRtSamplerParams? Sampler { get; init; }
Property Value
StreamToolCalls
Stream the raw text of a tool call while the model is generating it. Default false:
during a tool-call block the stream goes silent until the block completes and the parsed
ToolCall chunk arrives whole. With this on,
SendStreamingAsync(string, CancellationToken)
additionally yields ToolCallDelta chunks — incremental,
unparsed fragments of the tool-call text — as they are produced, followed by the usual complete
ToolCall chunk. Use the deltas for progress display only;
act on the final parsed chunk. Only meaningful when Tools are set.
public bool StreamToolCalls { get; init; }
Property Value
Remarks
Maps to the C API conversation_config_set_stream_tool_calls (requires native LiteRT-LM
v0.14.0+; older binaries throw EntryPointNotFoundException at conversation
creation). The deltas do not affect the blocking Send(string) path.
SystemMessage
Optional system prompt applied to the conversation.
public string? SystemMessage { get; init; }
Property Value
ThinkingTokenBudget
Caps how many tokens the model may spend on its thinking block per reply: the decode loop
tracks the model's thinking start/end token ids and cuts the block off at the budget.
null (default) = no cap; -1 = explicitly infinite; 0 is treated by the
native layer as "no budget" (NOT "no thinking"). Setting a budget with
EnableThinking unset turns thinking ON (a budget only makes sense with a
thinking block; the native thinking config defaults to enabled) — set
EnableThinking = false explicitly if you want a budget configured but thinking off.
public int? ThinkingTokenBudget { get; init; }
Property Value
- int?
Remarks
Unlike MaxOutputTokens — which caps the whole reply and starves the answer when
thinking runs long — this bounds only the reasoning share. Maps to the C API thinking config
(litert_lm_thinking_config_*, native v0.15.0+; older binaries throw
EntryPointNotFoundException at conversation creation). The per-send
ThinkingTokenBudget overrides this for one send.
Exceptions
- ArgumentOutOfRangeException
The value is negative and not -1.
Tools
Tools the model may call (function calling). Null/empty = no tools.
public IReadOnlyList<LiteRtTool>? Tools { get; init; }
Property Value
VisualTokenBudget
Tokens each image attachment may expand to. 0 (default) = the model's default size. The runtime scales each image down to about the budget in tokens (never above the image's default size), so a lower budget leaves more of the context window for text at the cost of image detail. Only sends that carry an image use it.
public int VisualTokenBudget { get; init; }
Property Value
Remarks
The Gemma 4 E-series bundles carry vision signatures of 70, 140 and 280 tokens and default to
about 256 tokens per image. Measured on gemma-4-E2B-it, one image costs 260 tokens by default, and
68, 104 and 200 tokens with budgets of 70, 100 and 200. The budget must not exceed the engine's
per-image maximum: MaxVisionTokensPerImage when the engine sets
it, else the model's own (MaxVisionTokenBudget, 280 on the Gemma 4
E-series). Above it, image sends fail with InvalidArgument; the
binding attaches the budget only to sends that carry an image, so text-only sends are unaffected.
Applied per send via the C API conversation_optional_args_set_visual_token_budget;
VisualTokenBudget overrides it for one send.