This token processor allows you to use classes and other syntax additions.
Features:
Table of contents:
this
variableThe built-in types are:
Dynamic
- the dynamic type is the implicit type compatible with any other type (often omitted)Void
- used only for return types to enforce that the function doesn't return anythingByte
- just an alias for Integer
for documentation of the intentShort
- just an alias for Integer
for documentation of the intentInteger
- 32-bit signed integer (but can also refer to 8bit and 16bit unsigned integers in arrays)Float
- 32-bit floatBoolean
- boolean type (zero = false, non-zero = true)String
- string type
You can also use array, hash and class types. Array types use brackets after the type,
for example Float[]
describes an array of floats. Hash types contains the type
of the key in the brackets, for example Integer[String]
describes a hash map
with a String
key and an Integer
value.
The syntax for constructors is just a syntax sugar for static methods that create the object and return it. In case you need more flexibility when creating an object (eg. using a different underlying type) you can use static methods directly.
Classes can either extend an existing class type or optionally extend a special common
class type named Object
. This class allows to provide automatic implementation
of to_string
methods. You can also provide your own implementations
of this method for custom string representations.
Some class types can't extend from Object, for example when they need serialization
or are backed by other type than an array. In such cases you can also register
a to_string method, albeit in a less efficient way. This is achieved by using the
Object::set_to_string(obj, func)
static method. It simply registers
the provided function to a global (self-clearing) hash map with a weak reference key.
Another alternative is to simply put a function reference as a first entry in an
array, the function must take one argument and must end with _to_string
suffix.
You can then either call the to_string
methods directly, or use
dump
or to_string
(with newlines), these are automatically
replaced with versions that know about class types. You can of course use the
original functions by using @dump
and @to_string
functions.
This functionality is completely optional. You have to put object.fix
file in the root of the scripts and extend the class from the Object
type.
Interfaces provide common type for otherwise different classes. This is achieved by creating a wrapper class that contains reference to the original class and a set of function references specific to that class so it can be called in a common way.
The original class contains some method, called as_interface
(when
the interface class is called Interface
) that returns a new instance
of this wrapper class. It can also cache it when desirable (using weak references
if the same existing interface instance should be always used).
Here is an example:
class Class1 { var @field1; var @field2; var @field3; function as_some_interface(): SomeInterface { return SomeInterface::create(this, Class1::common_method#1); } function common_method() { log("Class1"); } } class Class2 { function as_some_interface(): SomeInterface { return SomeInterface::create(this, Class2::common_method#1); } function common_method() { log("Class2"); } } class SomeInterface { var @data; var @common_method_func; constructor create(data, common_method_func) { this.data = data; this.common_method_func = common_method_func; } function common_method() { common_method_func(data); } }
Classes can provide implementations of operators. This makes the code much more readable for classes that represent numbers or something close enough. For example strings can be concatenated by adding. However a care must be taken to not misuse this feature.
The operators are declared as special static methods with two parameters. At least one of the parameters must be the class itself. The implied return type is the class or a Boolean in case of comparison operators.
For the in-place modification operators (+=
, -=
, etc.)
the methods have just one parameter and are instance methods. The implied return
type is the class itself. The implementation must return this
.
The special universal comparison operator <=>
is used when a
concrete comparison operator is not defined. The return value is an Integer (the
result must be zero when the values are equal, less than zero if the left value is
lesser than the right value or greater than zero if the left value is greater than
the right value).
Here is an example:
class IntRef { var value: Integer; constructor create(value: Integer) { this.value = value; } operator + (op1: IntRef, op2: IntRef) { return create(op1.value + op2.value); } operator + (op1: IntRef, op2: Integer) { return create(op1.value + op2); } operator + (op1: Integer, op2: IntRef) { return create(op1 + op2.value); } // same for other operators and combinations }
Structure is a special kind of a class that allow to store multiple packed instances in an array. This has an advantage for bulk data processing and multithreading (when used with shared arrays). Usage of shared arrays also avoids the GC overhead.
Internally the reference to a structure is just an offset to the backing array.
This offset is pointing just after the last field (end of the structure) to allow
usage of zero offset as a null
value. This also results into an
error when the null
is accessed because it will be out of the bounds
of the backing array.
The structure reference can be cast to Integer
to get the offset
at the beginning of the struct (internally the size will be subtracted). Doing
the opposite (casting from Integer
) will internally add the size.
Casting between different sized struct references adjusts the offset so it points
to the end of the new struct type. Comparing the structs uses the subtracted
offset (the beginning of the structure).
The structures are declared in the same way as classes with the difference that
the struct
keyword is used instead of a class
.
Accessing of fields requires a reference to the backing array and uses this syntax:
array[some_struct].field
.
You can have methods in structs, in the case of instance methods the this
variable refers to the offset only so in most cases the instance methods are required
to have the first parameter be an array or other object that contain a reference to
such array. There is a direct syntax for calling the methods:
array[some_struct].method()
(the array is passed as the first parameter
internally).
The syntax for backing array type is [Struct]
(or [Dynamic]
in case it contains different kinds of structs). You can also use Dynamic[]
or Dynamic
to also allow arbitrary direct array access.
You can do pointer arithmetic on structs. Any addition or subtraction is multiplied by the size of the structure to allow processing of the adjacent entries.
You can use foreach
on struct arrays. In such case the value is the
structure pointer.
When using structs backed by a shared array there is a behavior (of shared arrays) where floats are stored with their raw value. This is not a problem on assign, but normally when retrieving you will get just the raw value as an integer instead of a float. The implementation of structs automatically handles it, the float is restored to the normal float type when obtained from the struct (by multiplying it with 1.0).
You can also set or append the whole struct at once. This has the advantage of setting the unspecified fields with a zero value.
Here is an example:
struct Base { var base_field: Integer; } struct Test: Base { var @parent: Test; var @field1: Integer; var @field2: Integer; function update(owner: TestOwner, field1: Integer, field2: Integer) { var array = owner.array; array[this].field1 = field1; array[this].field2 = field2; } function replace(owner: TestOwner, field1: Integer, field2: Integer) { var array = owner.array; array[this] = { .field1: field1, .field2: field2 }; // you can also use explicit struct type like this: var dynarr = owner.array as Dynamic; dynarr[this] = Test { .field1: field1, .field2: field2 }; } function to_string(owner: TestOwner) { var array = owner.array; return "Test(parent=#"+array[this].parent+")"; } } class TestOwner { var @common: Integer; var @array: [Test]; function create_test(parent: Test): Test { var test = length(array) as Test; array[] = { .base_field: 123, .parent: parent }; return test; } function process_all() { for (var i=0 as Test; i<length(array); i++) { array[i].base_field = 100; this[i].update(1, 2); } // or just: foreach (var i in array) { array[i].base_field = 100; this[i].update(1, 2); } } }
this
variable
The this
variable used in instance functions and constructors is
a normal variable. It can be assigned, or it can even contain a null
value (in fact any kind of value). This is because the concept of classes is
separated from the underlying implementation type.
This can be used for various things, for example in callbacks you can pass
an array of multiple values instead of just the object and unpack it in the
code by assigning the object instance into the this
variable.
Another example is the usage of weak references (again typically in callbacks).
Some methods can check explicitly for null
s in this
.
For example comparison functions or to_string
implementations.
A slight downside of this approach is that when a null
is passed
to an instance method, there is no check at the time of method call, it will
error only when the object is accessed in the method, for some methods it can
even succeed if it's not accessed at all.
You can get the field offset with SomeClass::field_name
and the
size with SomeClass::SIZE
. This is useful when working with
the underlying array representation directly.
You can also get references to methods with SomeClass::method#1
.
The number of arguments must count with the this
parameter
for instance methods.
Sometimes you may want to define classes without actually using the classes token processor. For example if you're not using classes (or want it optional) but still provide the comfort of using them. Another example is the ability to make multiple versions of classes token processor (or other token processors) to work together.
The format is simple, to declare a class use a @class_SomeClass
private constant with a string value (can be empty). The string can contain
these attributes:
prefix
- function prefix (without the _
at the end)struct
- prefix for the constants (without the _
at the end)static
- specifies a list of methods that are static (the name includes a '#
' followed by the number of parameters)extend
- specifies a name of the super class
The value is delimited by '=
', multiple attributes are separated
by ',
' and when the attribute can contain multiple values they're
separated by a ':
'. No whitespace is allowed.
You can define methods by declaring a @method_SomeClass_method_name_1
private constant, where the last part is a number of parameters (including the
implied 'this
' parameter). You can either specify static methods
using the static
attribute for the class, or just use static
instead of method
in the private constant name. Undefined methods
are still recognized based on the prefix (all parameters and the return type are Dynamic
in that case).
You can define global functions by using a @global_some_func_1
private constant
to define a function named some_func
with a single parameter.
Every method uses the '(SomeType, Integer, Float): Boolean
' format to specify
the parameters. This means specifying the types of the parameters (the parameter for
'this
' is omitted for non-static methods), optionally followed by a return type.
Whitespace is allowed.
You can define operators by using a @operator_SomeClass_add_1
private
constant. The value is in the format 'method_name (Type1, Type2)
' (the
inplace
variants omit the first type as it is always the class type).
The trailing number can have any value, it is used to allow multiple constants when
there are multiple variants of the same operator. The names for the operators are
as follows:
|
|
|
|
|
This is an example how it can look:
const @class_SomeClass = "extend=BaseClass,prefix=somecls,struct=SCLS,static=create#2:static#0"; const @field_SomeClass_field1 = "Integer"; const @field_SomeClass_field2 = "Boolean"; const @static_SomeClass_create_2 = "(Integer, Boolean): SomeClass"; const @method_SomeClass_instance_function_3 = "(Integer, Boolean)"; const @operator_SomeClass_add_1 = "add (SomeClass, SomeClass)"; const @operator_SomeClass_add_2 = "add_int (SomeClass, Integer)"; const @operator_SomeClass_add_inplace_1 = "add_int_inplace (Integer)"; const @global_some_func_2 = "(SomeClass, Integer): Boolean";
The API allows other token processors to integrate with classes processing. To
get access to the API, just import classes
script and obtain the
context. Then you can register various hooks. The hooks are added based on actual
needs.
You can also use the API without classes, just use the prefixes class_context_
and class_type_
for the methods and pass the objects as a first parameter.
const { TYPE_DYNAMIC, TYPE_VOID, TYPE_INTEGER, TYPE_FLOAT, TYPE_BOOLEAN, TYPE_STRING }; const { EXT_TYPE_CLASS, EXT_TYPE_ARRAY, EXT_TYPE_HASH, EXT_TYPE_STRUCT, EXT_TYPE_STRUCT_ARRAY }; class ClassContext { static function get(fname: String): ClassContext; function register_function_call(name: String, get_types_func, adjust_call_func, data); function register_postprocess(func, data); function get_class(name: String): ClassType; function get_const_type(name: String): ClassType; function get_local_type(name: String): ClassType; function get_variable_type(name: String): ClassType; } class ClassType { static function create_array(base: ClassType): ClassType; static function create_hash(base: ClassType, index: ClassType): ClassType; function is_class(): Boolean; function is_array(): Boolean; function is_hash(): Boolean; function is_struct(): Boolean; function is_struct_array(): Boolean; function is_assignable_from(other: ClassType): Boolean; function get_class_name(): String; function get_parent_class(): ClassType; function get_method_real_name(name: String, types: ClassType[], is_static: Boolean): String function get_base(): ClassType; function get_index(): ClassType; function to_string(): String; static function dump_list(list: ClassType[]); }
The types are divided into two enumerations. The simple types are just integers
and you can directly use them (cast from/to ClassType
). The extended
types are contained in an array and the first entry is the type.
static function get(fname: String): ClassContext
function register_function_call(name: String, get_types_func, adjust_call_func, data)
null
to match all names) and the callbacks
with these signatures:function get_types(data, name: String, num_params: Integer, line: Integer): ClassType[]
null
if there is no
function for given number of parameters. The returned array is modified and passed
to the adjust_call
callback.
function adjust_call(data, name: String, types: ClassType[], tokens, src: String, start: Integer, end: Integer): Integer
function register_postprocess(func, data)
function postprocess(data, fname: String, tokens, src: String)
function get_class(name: String): ClassType
function get_const_type(name: String): ClassType
-1
if the constant
with given name is not defined. Must be called within processing.
function get_local_type(name: String): ClassType
-1
if the
local variable with given name is not defined. Must be called within
processing.
function get_variable_type(name: String): ClassType
-1
if the variable
with given name is not defined. Must be called within processing.
static function create_array(base_type: ClassType): ClassType
static function create_hash(base_type: ClassType, index_type: ClassType): ClassType
function is_class(): Boolean
function is_array(): Boolean
function is_hash(): Boolean
function is_struct(): Boolean
function is_struct_array(): Boolean
function is_assignable_from(other: ClassType): Boolean
function get_class_name(): String
function get_parent_class(): ClassType
function get_method_real_name(name: String, types: ClassType[], is_static: Boolean): String
null
) with given types
(the first entry is a return value).
function get_base(): ClassType
function get_index(): ClassType
function to_string(): String
static function dump_list(list: ClassType[])