Class LiteRtBenchmarkInfo

Namespace

LiteRtLmSharp

Assembly

LiteRtLmSharp.dll

A snapshot of the engine's benchmark counters for one conversation, returned by GetBenchmarkInfo(). Only populated when the engine was created with EnableBenchmark = true.

Prefill and decode metrics accumulate one entry per turn; the Last* properties report the most recent turn (the headline number for "how fast was that reply"). Decode throughput is the metric that EnableSpeculativeDecoding accelerates.

public sealed record LiteRtBenchmarkInfo : IEquatable<LiteRtBenchmarkInfo>

Inheritance

object

LiteRtBenchmarkInfo

Implements

IEquatable<LiteRtBenchmarkInfo>

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

object.ToString()

Properties

LastDecodeTokenCount

Decode token count of the most recent decode turn (0 if none).

public int LastDecodeTokenCount { get; init; }

Property Value

int

LastDecodeTokensPerSecond

Decode throughput (tokens/sec) of the most recent decode turn (0 if none).

public double LastDecodeTokensPerSecond { get; init; }

Property Value

double

LastPrefillTokenCount

Prefill token count of the most recent prefill turn (0 if none).

public int LastPrefillTokenCount { get; init; }

Property Value

int

LastPrefillTokensPerSecond

Prefill throughput (tokens/sec) of the most recent prefill turn (0 if none).

public double LastPrefillTokensPerSecond { get; init; }

Property Value

double

NumDecodeTurns

Number of decode turns recorded so far.

public int NumDecodeTurns { get; init; }

Property Value

int

NumPrefillTurns

Number of prefill turns recorded so far.

public int NumPrefillTurns { get; init; }

Property Value

int

TimeToFirstTokenSeconds

Seconds from the start of prefill to the first decoded token (excludes init).

public double TimeToFirstTokenSeconds { get; init; }

Property Value

double

TotalInitTimeSeconds

Total engine initialization time in seconds.

public double TotalInitTimeSeconds { get; init; }

Property Value

double