Tokens

Tokenizer splits every source line into tokens. There are several kinds of tokens:

Symbol

Single character that has grammatical meaning, like , : . & ^ *

Identifier

Identifiers are used to name various objects: labels, instructions, macros, namespaces, preprocessor directives and compiler commands.

Valid identifier can contain only letters, digits and underline character _ and cannot begin with a digit. Identifiers are case insensitive. Examples of valid identifiers:

AX MyName123 __FILE__

Number

Integer numbers can be written ordinarily. Plus + and preceding zeros have no meaning.

23 +1024 -007

Besides decimal numbers, you can also write binary, octal and hexadecimal numbers. Because decimal is the default system, d has no meaning. Integer numbers written in other systems can be also negative. Numbers below are equal:

1111011b 173o 123d 7Bh

Instead of h at the end, you can write 0x at the begin of hexadecimal numbers. It is useful because number cannot being with letter though it is hexadecimal digit.

0xFFFFFFFF

You can also write real (floating point) numbers:

10.5 -0.3333333333 2.5e-10

All the letters in numbers are case insensitive.

String

String are the characters embraced with quotation-marks or apostrophes:

'String1' "The Second String"

Special characters can written using escape sequences that begin with backslash \:

Examples:

"And then he said: \"Blablabla\"!"
'First line\nSecond line'
"\xFF"