LiteRtLmSharp

.NET 10 bindings for Google's LiteRT-LM — on-device LLM inference (e.g. Gemma) for any .NET app, including MAUI. P/Invoke over LiteRT-LM's C API, with native binaries distributed per-RID as NuGet packages.

Chat (blocking, awaitable + cancellable, streaming), function calling, multimodal (image/audio), conversation restore/clone, reasoning mode, tokenizer, speculative decoding, benchmarking and text embeddings for semantic search — all running locally, no server.

Streaming chat on-device in the MAUI sample (Android) The same app on Windows — multimodal chat on GPU at 68 tok/s

The same MAUI sample app, on an Android phone and on Windows (GPU) — both fully on-device.

Install

<PackageReference Include="LiteRtLmSharp" Version="1.3.0" />
<PackageReference Include="LiteRtLmSharp.runtime.win-x64" Version="1.3.0" />
<!-- or LiteRtLmSharp.runtime.linux-x64 / linux-arm64 / android-arm64 / android-x64 (emulator) / osx-arm64, per target -->

Install the managed package plus the runtime package for your platform, always with the same version number. Optional integrations: LiteRtLmSharp.Extensions.AI (IChatClient and IEmbeddingGenerator — Microsoft Agent Framework, MEAI) and LiteRtLmSharp.SemanticKernel (IChatCompletionService and an embedding generator).

First tokens

using LiteRtLmSharp;

using var engine = LiteRtEngine.Load(new LiteRtEngineOptions
{
    ModelPath = "gemma-4-E2B-it.litertlm",   // from huggingface.co/litert-community
    Backend = LiteRtBackend.Cpu,              // or .Gpu (WebGPU -> D3D12/Vulkan/Metal)
    MaxNumTokens = 4096,                       // total context window
});

using var chat = engine.CreateConversation();

await foreach (var chunk in chat.SendStreamingAsync("Tell me a joke"))
    Console.Write(chunk.Text);

The repository README walks through every feature with runnable snippets; the guides here go deeper per topic.

Where to go next

  • Chat & generation — function calling, reasoning mode, multimodal (image/audio), token counting.
  • Embeddings — on-device text embeddings (EmbeddingGemma 2) for semantic search and RAG, also as an IEmbeddingGenerator.
  • Microsoft.Extensions.AI integration — plug the model into the .NET AI ecosystem as an IChatClient and an IEmbeddingGenerator (Agent Framework, Semantic Kernel, MEAI middleware).
  • Semantic Kernel connector — IChatCompletionService and an embedding generator over the same bridge.
  • Conversation state — persist/restore chats, clone live conversations.
  • Engine tuning — precision, prefill chunking, thread counts, benchmarking.
  • Speculative decoding — the MTP drafter: when it helps and what it needs.
  • Android — device setup, backends, and MAUI notes.
  • API Reference — the full public surface, generated from the XML docs.

Samples

  • samples/Console — chat loop with --tools, --spec, and --thinking demos.
  • samples/Maui — full Android/Windows chat app: model download, streaming, multimodal attachments, function calling.
  • samples/SemanticKernel — kernel registration, streaming, and a [KernelFunction] plugin.

Project status

The roadmap is the status source of truth; release history is in the changelog.