Specification

Tokens

This chapter describes the tokens specification. All of the tokens in the design are described in this section.

Token format

Token

#![allow(unused)]
fn main() {
pub struct Token {
    /// Token Type
    pub kind: TokenKind,

    /// Start offset in source
    pub start: usize,

    /// End offset in source
    pub end: usize,

    pub value: TokenValue,
}
}

TokenValue

#![allow(unused)]
fn main() {
pub enum TokenValue {
    None,
    Number(f64),
    String(String),
}
}

Token Kinds

This chapter describes the token kinds used in the spec.

Kinds

#![allow(unused)]
fn main() {
pub enum TokenKind {
    Eof, // end of file
    Comment,
    Unexpected,

    Semicolon,
    Comma,
    Colon,
    Dot,

    // Operators
    Plus,
    Minus,
    Star,
    Slash,
    Power,
    Range,
    Modulo,

    // Bitwise operators (Keyword2Operator)
    BitAnd,
    BitOr,
    BitNot,
    BitXor,
    BitLeftShift,
    BitRightShift,

    // Unary operators
    PlusPlus,
    MinusMinus,
    Question,

    // Logic operators (Keyword2Operator)
    And,
    Or,
    Xor,
    Not,

    /// Assignments operators (+=, -=, *=, /=...)
    Assign,
    FormulaAssign,
    PlusAssign,
    MinusAssign,
    MultiplyAssign,
    DivideAssign,
    PowerAssign,
    ModuloAssign,

    // Comparison operators
    Equal,
    NotEqual,
    Less,
    LessEqual,
    Greater,
    GreaterEqual,

    // Brackets
    LParen,
    RParen,
    LBrace,
    RBrace,
    LBracket,
    RBracket,

    // Identifiers
    Identifier,

    // Literals
    Number,
    Str,

    // --- Keywords ---

    // Keyword literals
    True,
    False,

    // Keywords
    If,
    Else,
    While,
    For,
    Loop,
    Let,
    Return,
    Break,
    Continue,
    Function,
    In,
}
}

Operators

Keywords

Identifiers

Literals

In SymboScript exists only three kinds of token literals: strings, numbers, booleans.

Strings

Strings are enclosed in double quotes, single quotes or backticks.

"hello"
"hello"
`hello`
"hello\nworld"
'hello"world'
"hello ' world"

Numbers

1 .1
1.1
1e1
1E1
1.1e1
1.1E1

Booleans

true
false

Brackets

Parser

Expressionss

Operator priority

Examples

This is chapter about how the code in SymboScript looks.