1 Evaluation Model🔗ℹ

Rhombus evaluation can be viewed as the simplification of expressions to obtain values. For example, just as an elementary-school student simplifies

Rhombus evaluation simplifies

1 + 1 → 2

The arrow → replaces the more traditional = to emphasize that evaluation proceeds in a particular direction toward simpler expressions. In particular, a value, such as the number 2, is an expression that evaluation simplifies no further.

1.1 Subexpression Evaluation and Continuations🔗ℹ

Some simplifications require more than one step. For example:

4 - (1 + 1) → 4 - 2 → 2

An expression that is not a value can always be partitioned into two parts: a redex (“reducible expression”), which is the part that can change in a single-step simplification (highlighted), and the continuation, which is the evaluation context surrounding the redex. In 4 - (1 + 1), the redex is (1 + 1), and the continuation is 4 - [], where [] takes the place of the redex as it is reduced. That is, the continuation says how to “continue” after the redex is reduced to a value.

Before some expressions can be evaluated, some or all of their sub-expressions must be evaluated. For example, in the expression 4 - (1 + 1), the use of - cannot be reduced until the subexpression (1 + 1) is reduced. Thus, the specification of each syntactic form specifies how (some of) its sub-expressions are evaluated and then how the results are combined to reduce the form away.

The dynamic extent of an expression is the sequence of evaluation steps during which the expression contains the redex.

1.2 Tail Position🔗ℹ

An expression expr1 is in tail position with respect to an enclosing expression expr2 if, whenever expr1 becomes a redex, its continuation is the same as was the enclosing expr2’s continuation.

For example, the (1 + 1) expression is not in tail position with respect to 4 - (1 + 1). To illustrate, we use the notation C[expr] to mean the expression that is produced by substituting expr in place of [] in some continuation C:

C[4 - (1 + 1)] → C[4 - 2]

In this case, the continuation for reducing (1 + 1) is C[(4 - [])], not just C. The requirement specified in the first paragraph above is not met.

In contrast, (1 + 1) is in tail position with respect to if 0 == 0 | (1 + 1) | 3 because, for any continuation C,

C[if 0 == 0 | (1 + 1) | 3] → C[if #true | (1 + 1) | 3] → C[1 + 1]

The requirement specified in the first paragraph is met. The steps in this reduction sequence are driven by the definition of if, and they do not depend on the continuation C. The “then” branch of an if form is always in tail position with respect to the if form. Due to a similar reduction rule for if and #false, the “else” branch of an if form is also in tail position.

Tail-position specifications provide a guarantee about the asymptotic space consumption of a computation. In general, the specification of tail positions accompanies the description of each syntactic form, such as if.

1.3 Multiple Return Values🔗ℹ

A Rhombus expression can evaluate to multiple values, to provide symmetry with the fact that a function can accept multiple arguments.

Most continuations expect a certain number of result values, although some continuations can accept an arbitrary number. Indeed, most continuations, such as [] + 1, expect a single value. The continuation

block:

let (x, y) = []

body

expects two result values; the first result replaces x in body, and the second replaces y in body. The continuation

block:

[]

1 + 2

accepts any number of result values, because it ignores the result(s).

In general, the specification of a syntactic form indicates the number of values that it produces and the number that it expects from each of its sub-expressions. In addition, some functions (notably values) produce multiple values, and some functions (notably call_with_values) create continuations internally that accept a certain number of values.

1.4 Top-Level Variables🔗ℹ

Given

then an algebra student simplifies x + 1 as follows:

Rhombus works much the same way, in that a set of top-level variables (see also Variables and Locations) are available for substitutions on demand during evaluation. For example, given

def x = 10

then

x + 1 → 10 + 1 → 11

In Rhombus, the way definitions are created is just as important as the way they are used. Rhombus evaluation thus keeps track of both definitions and the current expression, and it extends the set of definitions in response to evaluating forms such as def.

Each evaluation step, then, transforms the current set of definitions and program into a new set of definitions and program. Before a def can be moved into the set of definitions, its expression (i.e., its right-hand side) must be reduced to a value. (The left-hand side is not an expression position, and so it is not evaluated.)

Using def again, a program can change the value associated with an existing top-level variable:

1.5 Objects and Imperative Update🔗ℹ

In addition to def for imperative update of top-level variables, various functions and operators enable the modification of elements within a mutable compound data structure. For example, [] with := modifies the content of an array.

To explain such modifications to data, we must distinguish between values, which are the results of expressions, and objects, which actually hold data.

A few kinds of objects can serve directly as values, including booleans, #void, and small exact integers. More generally, however, a value is a reference to an object stored somewhere else. For example, a value can refer to a particular array that currently holds the value 10 in its first slot. If an object is modified via one value, then the modification is visible through all the values that reference the object.

In the evaluation model, a set of objects must be carried along with each step in evaluation, just like the definition set. Operations that create objects, such as Array, add to the set of objects:

The distinction between a top-level variable and an object reference is crucial. A top-level variable is not a value, so it must be evaluated. Each time a variable expression is evaluated, the value of the variable is extracted from the current set of definitions. An object reference, in contrast, is a value and therefore needs no further evaluation. The evaluation steps above use ⟨o1⟩ for an object reference to distinguish it from a variable name.

An object reference can never appear directly in a text-based source program. A program representation created with Syntax.make, however, can embed direct references to existing objects.

1.6 Garbage Collection🔗ℹ

See Memory Management for functions related to garbage collection.

In the program state

evaluation cannot depend on ⟨o2⟩, because it is not part of the program to evaluate, and it is not referenced by any definition that is accessible by the program. The object is said to not be reachable. The object ⟨o2⟩ may therefore be removed from the program state by garbage collection.

A few special compound datatypes hold weak references to objects. Such weak references are treated specially by the garbage collector in determining which objects are reachable for the remainder of the computation. If an object is reachable only via a weak reference, then the object can be reclaimed, and the weak reference is replaced by a different value (typically #false).

As a special case, a fixnum is always considered reachable by the garbage collector. Many other values are always reachable due to the way they are implemented and used: A character in the Latin-1 range is always reachable, because == Latin-1 characters are always ===, and all of the Latin-1 characters are referenced by an internal module. Similarly, PairList[], #true, #false, Port.eof, and #void are always reachable. Values produced by #%literal remain reachable when the #%literal expression itself is reachable.

1.7 Function Calls and Local Variables🔗ℹ

Given

an algebra student simplifies f(7) as follows:

The key step in this simplification is to take the body of the defined function f and replace each x with the actual value 7.

Rhombus function calls work much the same way. A function is an object, so evaluating f(7) starts with a variable lookup:

If a variable like x is made mutable, however, the value associated with the variable can be changed in the body of a function by using :=, as in the example fun (mutable x): x := 3; x. Since the value associated with a mutable argument variable x should be able to change, we cannot just substitute the value in for x when we first call the function.

Instead, a new location is created for each variable on each function call. The argument value is placed in the location, and each instance of the variable in the function body is replaced with the new location:

A location is the same as a top-level variable, but when a location is generated, it (conceptually) uses a name that has not been used before and that cannot be generated again or accessed directly.

Generating a location in this way means that := evaluates for local variables, including argument variables, in the same way as for top-level variables, because the local variable is always replaced with a location by the time the := form is evaluated:

The location-generation and substitution step of function call requires that the argument is a value. Therefore, in (fun (mutable x): x + 10)(1 + 2), the 1 + 2 subexpression must be simplified to the value 3, and then 3 can be placed into a location for x. In other words, Rhombus is a call-by-value language.

Evaluation of a local-variable form, such as

block:

let mutable x = 1 + 2

expr

is the same as for a function call. After 1 + 2 produces a value, it is stored in a fresh location that replaces every instance of x in expr.

1.8 Variables and Locations🔗ℹ

A variable is a placeholder for a value, and expressions in an initial program refer to variables. A top-level variable is both a variable and a location. Any other variable is always replaced by a location at run-time—conceptually, even in the case of variables that are not mutable. Thus, evaluation of expressions involves only locations. A single local variable (i.e., a non-top-level, non-module-level variable), such as an argument variable, can correspond to different locations during different calls.

For example, in the program

def y = (block: let x = 5; x) + 6

both y and x are variables. The y variable is a top-level variable, and the x is a local variable. When this code is evaluated, a location is created for x to hold the value 5, and a location is also created for y to hold the value 11.

The replacement of a variable with a location during evaluation implements Rhombus’s lexical scoping. For the purposes of substituting xloc for x, all variable bindings must use distinct names, so no x that is really a different variable will get replaced. Ensuring that distinction is one of the jobs of the macro expander; see Syntax Model. For example, when an argument variable x is replaced by the location xloc, it is replaced throughout the body of the function, including any nested fun forms. As a result, future references to the variable always access the same location.

1.9 Modules and Module-Level Variables🔗ℹ

Most definitions in Rhombus are in modules. In terms of evaluation, a module is essentially a prefix on a defined name, so that different modules can define the same name. That is, a module-level variable is like a top-level variable from the perspective of evaluation.

One difference between a module and a top-level definition is that a module can be declared without instantiating its module-level definitions. Evaluation of a import instantiates (i.e., triggers the instantiation of) the declared module, which creates variables that correspond to its module-level definitions.

For example, given the module declaration

// in "m.rhm"

#lang rhombus

def x = 10

the evaluation of import "m.rkt" creates the variable x and installs 10 as its value. This x is unrelated to any top-level definition of x (as if it were given a unique, module-specific prefix).

1.9.1 Phases🔗ℹ

The purpose of phases is to address the necessary separation of names defined at execution time versus names defined at expansion time.

A module can be instantiated in multiple phases. A phase is an integer that, like a module name, is effectively a prefix on the names of module-level definitions. Phase 0 is the run-time phase.

A top-level import instantiates a module at phase 0, if the module is not already instantiated at phase 0. A top-level import meta instantiates a module at phase 1 (if it is not already instantiated at that phase); meta also has a different binding effect on further program parsing, as described in Section to Appear.

Within a module, some definitions are already shifted by a phase: the meta form shifts expressions and definitions by a relative phase +1. Thus, if the module is instantiated at phase 1, the variables defined with meta are created at phase 2, and so on. Moreover, this relative phase acts as another layer of prefixing, so that x defined with def and x defined with meta def can co-exist in a module without colliding. A meta form can be nested within a meta form, in which case the inner definitions and expressions are in relative phase +2, and so on. Higher phases are mainly related to program parsing instead of normal evaluation.

If a module instantiated at phase n imports another module, then the imported module is first instantiated at phase n, and so on transitively. (Module imports cannot form cycles.) If a module instantiated at phase n imports another module M with meta, then M becomes available at phase n+1, and it later may be instantiated at phase n+1. If a module that is available at phase n (for n>0) imports another module M with meta -1, then M becomes available at phase n-1, and so on. Instantiations of available modules above phase 0 are triggered on demand as described in Module Expansion, Phases, and Visits.

A final distinction among module instantiations is that multiple instantiations may exist at phase 1 and higher. These instantiations are created by the parsing of module forms (see Module Expansion, Phases, and Visits), and are, again, conceptually distinguished by prefixes.

Top-level variables can exist in multiple phases in the same way as within modules. For example, def within meta creates a phase 1 variable. Furthermore, reflective operations like Evaluator.make_rhombus and eval provide access to top-level variables in higher phases, while module instantiations (triggered by import) relative to such top-levels are in correspondingly higher phases.

1.9.2 Module Redeclarations🔗ℹ

When a module is declared using a name with which a module is already declared, the new declaration’s definitions replace and extend the old declarations. If a variable in the old declaration has no counterpart in the new declaration, the old variable continues to exist, but its binding is not included in the lexical information for the module body. If a new variable definition has a counterpart in the old declaration, it effectively assigns to the old variable.

If a module is instantiated in the current namespace’s base phase before the module is redeclared, the redeclaration of the module is immediately instantiated in that phase.

If the current inspector does not manage a module’s declaration inspector (see Section to Appear), then the module cannot be redeclared. Even if redeclaration succeeds, instantiation of a module that is previously instantiated may fail if instantiation for the redeclaration attempts to modify variables that are constant.

1.9.3 Submodules🔗ℹ

A module form within a module declares a submodule. A submodule is accessed relative to its enclosing module, usually with the ! operator. Submodules can be nested to any depth.

Although a submodule is lexically nested within a module, a subsmodule declared with ~lang cannot access the bindings of its enclosing module directly. In that case, unless a submodule imports from its enclosing module or vice versa, then visits or instantiations of the two modules are independent, and their implementations may even be loaded from a compiled form at different times.

See module for more information.

1.10 Continuation Frames and Marks🔗ℹ

See Continuation.Marks for continuation-mark forms and functions.

Every continuation C can be partitioned into continuation frames C1, C2, ..., Cn such that C = C1[C2[...[Cn]]], and no frame Ci can be itself partitioned into smaller continuations. Evaluation steps add frames to and remove frames from the current continuation, typically one at a time.

Each frame is conceptually annotated with a set of continuation marks. A mark consists of a key and its value. The key is an arbitrary value, and each frame includes at most one mark for any given key. Various operations set and extract marks from continuations, so that marks can be used to attach information to a dynamic extent. For example, marks can be used to record information for a “stack trace” to be presented when an exception is raised, or to implement dynamic scope.

1.11 Prompts, Delimited Continuations, and Barriers🔗ℹ

See Continuations for continuation and prompt functions.

A prompt is a special kind of continuation frame that is annotated with a specific prompt tag (essentially a continuation mark). Various operations allow the capture of frames in the continuation from the redex position out to the nearest enclosing prompt with a particular prompt tag; such a continuation is sometimes called a delimited continuation. Other operations allow the current continuation to be extended with a captured continuation (specifically, a composable continuation). Yet other operations abort the computation to the nearest enclosing prompt with a particular tag, or replace the continuation to the nearest enclosing prompt with another one. When a delimited continuation is captured, the marks associated with the relevant frames are also captured.

A continuation barrier is another kind of continuation frame that prohibits certain replacements of the current continuation with another. Specifically, a continuation can be replaced by another only when the replacement does not introduce any continuation barriers. A continuation barrier thus prevents “downward jumps” into a continuation that is protected by a barrier. Certain operations install barriers automatically; in particular, when an exception handler is called, a continuation barrier prohibits the continuation of the handler from capturing the continuation past the exception point.

An escape continuation is essentially a derived concept. It combines a prompt for escape purposes with a continuation for mark-gathering purposes. As the name implies, escape continuations are used only to abort to the point of capture.

1.12 Threads🔗ℹ

See Threads and Concurrency for thread and synchronization functions.

Rhombus supports multiple threads of evaluation. Threads run concurrently, in the sense that one thread can preempt another without its cooperation, but threads currently all run on the same processor (i.e., the same underlying operating system process and thread).

Threads are created explicitly by forms such as thread. In terms of the evaluation model, each step in evaluation actually deals with multiple concurrent expressions, up to one per thread, rather than a single expression. The expressions all share the same objects and top-level variables, so that they can communicate through shared state, and sequential consistency is guaranteed (i.e., the result is consistent with some global sequence imposed on all evaluation steps across threads). Most evaluation steps involve a single step in a single thread, but certain synchronization primitives require multiple threads to progress together in one step; for example, an exchange of a value through a channel progresses in two threads simultaneously.

Unless otherwise noted, all constant-time functions and operations provided by Rhombus are thread-safe in the sense that they are atomic: they happen as a single evaluation step. For example, := assigns to a variable as an atomic action with respect to all threads, so that no thread can see a “half-assigned” variable. Similarly, [] with := assigns to an array atomically. Note that the evaluation of a := expression with its subexpression is not necessarily atomic, because evaluating the subexpression involves a separate step of evaluation. Only the assignment action itself (which takes after the subexpression is evaluated to obtain a value) is atomic. Similarly, a function call can involve multiple steps that are not atomic, even if the function itself performs an atomic action.

The [] plus := combination is not atomic on a MutableMap, but the map is protected by a lock; see Maps for more information. Port operations are generally not atomic, but they are thread-safe in the sense that a byte consumed by one thread from an input port will not be returned also to another thread, and methods like Port.Input.Progress.commit and Port.Output.write_bytes offer specific concurrency guarantees.

In addition to the state that is shared among all threads, each thread has its own private state that is accessed through thread cells. A thread cell is similar to a normal mutable object, but a change to the value inside a thread cell is seen only when extracting a value from that cell in the same thread. A thread cell can be preserved; when a new thread is created, the creating thread’s value for a preserved thread cell serves as the initial value for the cell in the created thread. For a non-preserved thread cell, a new thread sees the same initial value (specified when the thread cell is created) as all other threads.

1.13 Context Parameters🔗ℹ

See Context Parameters for context-parameter forms and functions.

Context parameters are essentially a derived concept in Rhombus; they are defined in terms of continuation marks and thread cells. However, parameters are also “built in,” due to the fact that some primitive functions consult parameter values. For example, the default output stream for primitive output operations is specified by a parameter.

A parameter is a setting that is both thread-specific and continuation-specific. In the empty continuation, each parameter corresponds to a preserved thread cell; a corresponding parameter function accesses and sets the thread cell’s value for the current thread.

In a non-empty continuation, a parameter’s value is determined through a parameterization that is associated with the nearest enclosing continuation frame via a continuation mark (whose key is not directly accessible). A parameterization maps each parameter to a preserved thread cell, and the combination of the thread cell and the current thread yields the parameter’s value. A parameter function sets or accesses the relevant thread cell for its parameter.

Various operations, such as parameterize, install a parameterization into the current continuation’s frame.

1.14 Exceptions🔗ℹ

See Exceptions for exception forms, functions, and types.

Exceptions are essentially a derived concept in Rhombus; they are defined in terms of continuations, prompts, and continuation marks. However, exceptions are also “built in,” due to the fact that primitive forms and functions may throw exceptions.

An exception handler to catch exceptions can be associated with a continuation frame though a continuation mark (whose key is not directly accessible). When an exception is raised, the current continuation’s marks determine a chain of exception handler functions that are consulted to handle the exception. A handler for uncaught exceptions is designated through a built-in context parameter.

One potential action of an exception handler is to abort the current continuation up to an enclosing prompt with a particular prompt tag. The default handler for uncaught exceptions, in particular, aborts to a particular tag for which a prompt is always present, because the prompt is installed in the outermost frame of the continuation for any new thread.

1.15 Custodians🔗ℹ

See Custodians for custodian functions.

A custodian manages a collection of objects such as threads, Port.FileStream objects, TCPListener objects, and UDP objects. Whenever a thread, etc., is created, it is placed under the management of the current custodian as determined by the Custodian.current context parameter.

Except for the root custodian, every custodian itself is managed by a custodian, so that custodians form a hierarchy. Every object managed by a subordinate custodian is also managed by the custodian’s owner.

When a custodian is shut down via Custodian.shutdown_all, it forcibly and immediately closes the ports, TCP connections, etc., that it manages, as well as terminating (or suspending) its threads. A custodian that has been shut down cannot manage new objects. After the current custodian is shut down, if a function is called that attempts to create a managed resource (e.g., Port.Input.open_file, thread), then the Exn.Fail.Contract exception is thrown.

The values managed by a custodian are semi-weakly held by the custodian: a will can be executed for a value that is managed by a custodian. A custodian only weakly references its subordinate custodians; if a subordinate custodian is unreferenced but has its own subordinates, then the custodian may be garbage collected, at which point its subordinates become immediately subordinate to the collected custodian’s superordinate (owner) custodian.

In addition to the other entities managed by a custodian, a custodian box created with Custodian.Box strongly holds onto a value placed in the box until the box’s custodian is shut down. However, the custodian only weakly retains the box itself, so the box and its content can be collected if there are no other references to them.