Class LiteRtTokenUnion

Namespace

LiteRtLmSharp

Assembly

LiteRtLmSharp.dll

A single model start (BOS) or stop (EOS) token, as reported by GetStartToken() / GetStopTokens(). A model expresses each such token either as a literal Text string or as a sequence of token Ids; Kind says which. Use it to recognize when generated output reaches a stop token, or to compare against Tokenize(string) output.

public sealed class LiteRtTokenUnion

Inheritance

object

LiteRtTokenUnion

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.ReferenceEquals(object, object)

Properties

Ids

The token ids when Kind is Ids; otherwise empty.

public IReadOnlyList<int> Ids { get; }

Property Value

IReadOnlyList<int>

Kind

Whether this token is a String or a set of Ids.

public LiteRtTokenKind Kind { get; }

Property Value

LiteRtTokenKind

Text

The literal string when Kind is String; otherwise null.

public string? Text { get; }

Property Value

string

Methods

ToString()

A readable form: the string itself, or the id list rendered as [a, b, c].

public override string ToString()

Returns

string