Specification

Tokens

This chapter describes the tokens specification. All of the tokens in the design are described in this section.

Token format

#![allow(unused)]
fn main() {
pub struct Token {
    /// Token Type
    pub kind: TokenKind,

    /// Start offset in source
    pub start: usize,

    /// End offset in source
    pub end: usize,

    pub value: TokenValue,
}
}

TokenValue

#![allow(unused)]
fn main() {
pub enum TokenValue {
    None,
    Number(f64),
    String(String),
}
}

Token Kinds

This chapter describes the token kinds used in the spec.

Kinds

#![allow(unused)]
fn main() {
pub enum TokenKind {
    Eof, // end of file
    Comment,
    Unexpected,

    Semicolon,
    Comma,
    Colon,
    Dot,

    // Operators
    Plus,
    Minus,
    Star,
    Slash,
    Power,
    Range,
    Modulo,

    // Bitwise operators (Keyword2Operator)
    BitAnd,
    BitOr,
    BitNot,
    BitXor,
    BitLeftShift,
    BitRightShift,

    // Unary operators
    PlusPlus,
    MinusMinus,
    Question,

    // Logic operators (Keyword2Operator)
    And,
    Or,
    Xor,
    Not,

    /// Assignments operators (+=, -=, *=, /=...)
    Assign,
    FormulaAssign,
    PlusAssign,
    MinusAssign,
    MultiplyAssign,
    DivideAssign,
    PowerAssign,
    ModuloAssign,

    // Comparison operators
    Equal,
    NotEqual,
    Less,
    LessEqual,
    Greater,
    GreaterEqual,

    // Brackets
    LParen,
    RParen,
    LBrace,
    RBrace,
    LBracket,
    RBracket,

    // Identifiers
    Identifier,

    // Literals
    Number,
    Str,

    // --- Keywords ---

    // Keyword literals
    True,
    False,

    // Keywords
    If,
    Else,
    While,
    For,
    Loop,
    Let,
    Return,
    Break,
    Continue,
    Function,
    In,
}
}

Operators

Keywords

Identifiers

Literals

In SymboScript exists only three kinds of token literals: strings, numbers, booleans.

Strings

Strings are enclosed in double quotes, single quotes or backticks.

"hello"
"hello"
`hello`
"hello\nworld"
'hello"world'
"hello ' world"

Numbers

1 .1
1.1
1e1
1E1
1.1e1
1.1E1

Booleans

true
false

Brackets

Parser

Expressionss

Operator priority

Operator/Expression	Associativity	Parser fn name	Impl
Calls, Identifiers, Literals	Left to Right	call	✅
Members	Left to Right	dot	✅
new	Right to Left	new_expr	✅
delete	Right to Left	delete_expr	✅
await	Right to Left	await_expr	✅
`!` `++` `--` `()` `~` (unary `-` `+`) `[]` `{}`	Left to Right	factor	✅
`^`	Left to Left	power	✅
`*` `/` `%`	Left to Right	term	✅
`+` `-`	Left to Right	add_sub	✅
`>>` `<<`	Left to Right	shift	✅
`&`	Left to Right	bit_and	✅
`bxor`	Left to Right	bit_xor	✅
`\|`	Left to Right	bit_or	✅
`==` `!=` `<` `<=` `>` `>=`	Left to Right	cmp	✅
`&&`	Left to Right	logical_and	✅
`\|\|`	Left to Right	logical_or	✅
`..`	Left to Right	range	✅
`?:`	Right to Left	ternary	✅
`=` `:=` `+=` `-=` `*=` `/=` `^=` `%=`	Right to Left	assign	✅
`,`	None	comma	✅

Statements

Statement	Parser fn name	Trigger token	Impl
Variable declaration	var_decl	Let	✅
Async function declaration	async_fn_decl	Async	❌
Function declaration	fn_decl	Function	✅
Return statement	return_stmt	Return	✅
Yield statement	yield_stmt	Yield	✅
Break statement	break_stmt	Break	✅
Continue statement	continue_stmt	Continue	✅
If statement	if_stmt	If	❌
While statement	while_stmt	While	❌
For statement	for_stmt	For	❌
Loop statement	loop_stmt	Loop	❌
Try statement	try_stmt	Try	❌
Throw statement	throw_stmt	Throw	❌
Expression statement	expr_stmt	Other	✅

Examples

This is chapter about how the code in SymboScript looks.

SymboScript book