xiiToken Struct Reference

Represents one piece of tokenized text in a document. More...

#include <Tokenizer.h>

Public Attributes
xiiInt32	m_iType
	Typically of type xiiTokenType, but users can put anything in there, that they like.
xiiUInt32	m_uiLine
	The line in which the token appeared.
xiiUInt32	m_uiColumn
	The column in the line, at which the token string started.
xiiStringView	m_DataView
xiiUInt32	m_uiCustomFlags
	For users to be able to store additional info for a token.
xiiHashedString	m_File
	The file in which the token appeared.

Detailed Description

Represents one piece of tokenized text in a document.

Member Data Documentation

m_DataView

xiiStringView xiiToken::m_DataView

The actual string data that represents the token. Note that this is a view to a substring of some larger text data. To get only the relevant piece as one zero-terminated string, assign m_DataView to a xiiStringBuilder and read that instead.

---

The documentation for this struct was generated from the following file:

• /home/runner/work/XII/XII/Source/Engine/Foundation/CodeUtils/Tokenizer.h