1 Background: Scope and Macros
An essential consequence of hygienic macro expansion is to enable
macro definitions via
patterns and templates—also known as
macros by example (Kohlbecker and Wand 1987; Clinger and Rees 1991). Although pattern-based macros are limited in various
ways, a treatment of binding that can accommodate patterns and
templates is key to the overall expressiveness of a hygienic macro
system, even for macros that are implemented with more general
constructs.
As an example of a pattern-based macro, suppose that a Racket library
implements a Java-like object system and provides a
send form,
where
evaluates
a-point to an object, locates a function mapped to
the symbol
'rotate within the object, and calls the function
as a method by providing the object itself followed by the argument
90. Assuming a
lookup-method function that locates a method
within an object, the
send form can be implemented by a
pattern-based macro as follows:
The
define-syntax-rule form declares a
pattern that
is keyed on an initial identifier; in this case, the
pattern is keyed on
send. The remaining identifiers in the
parenthesized
send pattern are
pattern variables. The second
part of the definition specifies a
template that replaces a
match of the pattern, where each use of a pattern variable in the
template is replaced with the corresponding part of the match.
With this definition, the example use of
send above matches
the pattern with
a-point as
obj-expr,
rotate as
method-name, and
90 as
arg, so the
send use expands to
| (let ([obj a-point]) |
| ((lookup-method obj 'rotate) obj 90)) |
Hygienic macro expansion ensures that the
identifier
obj is not accidentally referenced
in an expression that replaces
arg in a use of
send (
Kohlbecker et al. 1986). For example, the body of
must call the
same? method of
a-point with the function argument
obj,
and not with
a-point itself as bound to
obj in the
macro template for
send. Along similar lines, a local binding of
lookup-method at a use site of
send must
not affect the meaning of
lookup-method in
send’s
template. That is,
| (let ([lookup-method #f]) |
| (send a-point rotate 90)) |
should still call the rotate method of a-point.
Macros can be bound locally, and macros can even expand to definitions
of macros. For example, suppose that the library also provides
a
with-method form that performs a method lookup just once
for multiple sends:
| (with-method ([rotate-a-point (a-point rotate)]) ; find rotate once |
| (for ([i 1000000]) |
| (rotate-a-point 90))) ; send rotate to point many times |
The implementation of
with-method can make
rotate-a-point a local macro binding, where a use of
rotate-a-point expands to a function call with
a-point added as the first argument to the function. That is,
the full expansion is
| (let ([obj a-point]) |
| (let ([rotate-a-point-method (lookup-method obj 'rotate)]) |
| (for ([i 1000000]) |
| (rotate-a-point-method obj 90)))) |
but the intermediate expansion is
| (let ([obj a-point]) |
| (let ([rotate-a-point-method (lookup-method obj 'rotate)]) |
| (let-syntax ([rotate-a-point (syntax-rules () |
| [(rotate-a-point arg) |
| (rotate-a-point-method obj arg)])]) |
| (for ([i 1000000]) |
| (rotate-a-point 90))))) |
where
let-syntax locally binds the macro
rotate-a-point. The macro is implemented by a
syntax-rules form that produces an anonymous pattern-based
macro (in the same way that
lambda produces an anonymous
function).
In other words,
with-method is a binding form, it is a
macro-generating macro, it relies on local-macro binding, and the
macro that it generates refers to a private binding
obj that
is also macro-introduced.
Nevertheless,
with-method is straightforwardly implemented as
a pattern-based macro:
Note that the
obj binding cannot be given a permanently
distinct name within
with-method. A distinct name must be
generated for each use of
with-method, so that nested uses
create local macros that reference the correct
obj.
In general, the necessary bindings or even the binding structure of a
macro’s expansion cannot be predicted in advance of expanding the
macro. For example, the let identifier that starts the
with-method template could be replaced with a macro argument,
so that either let or, say, a lazy variant of let
could be supplied to the macro. The expander must accommodate such
macros by delaying binding decisions as long as possible. Meanwhile,
the expander must accumulate information about the origin of
identifiers to enable correct binding decisions.
Even with additional complexities—where the macro-generated macro is
itself a binding form, where uses can be nested so the different uses
of the generated macro must have distinct bindings, and so
on—pattern-based macros support implementations that are essentially
specifications (Kohlbecker and Wand 1987). A naive approach to macros and
binding fails to accommodate the
specifications (Adams 2015, sections 4.2-4.5), while existing
formalizations of suitable binding rules detour into concepts of marks
and renamings that are distant from the programmer’s sense of the
specification.
The details of a formalization matter more when moving beyond
pattern-matching macros to
procedural macros, where the
expansion of a macro can be implemented by an arbitrary compile-time
function. The
syntax-case and
syntax forms provide
the pattern-matching and template-construction facilities,
respectively, of
syntax-rules, but they work as expressions
within a compile-time function (
Dybvig et al. 1993). This combination
allows a smooth transition from pattern-based macros to procedural
macros for cases where more flexibility is needed. In fact,
syntax-rules
is itself simply a macro that expands to a procedure:
Besides allowing arbitrary computation mixed with pattern matching and
template construction, the
syntax-case system provides
operations for manipulating program representations as
syntax
objects. Those operations include “bending” hygiene by attaching
the binding context of one syntax object to another. For example, a
macro might accept an identifier
point and synthesize the
identifier
make-point, giving the new identifier the same
context as
point so that
make-point behaves as if it
appeared in the same source location with respect to binding.
Racket provides an especially rich set of operations on syntax objects
to enable macros that compose and cooperate (Flatt et al. 2012). Racket’s
macro system also relies on a module layer that prevents interference
between run-time and compile-time phases of a program, since interference would
make macros compose less reliably (Flatt 2002). Finally, modules
can be nested and macro-generated, which enables macros and modules to
implement facets of a program that have different instantiation
times—such as the program’s run-time code, its tests, and its
configuration metadata (Flatt 2013). The module-level facets of
Racket’s macro system are, at best, awkwardly accommodated by existing
models of macro binding; those models are designed for
expression-level binding, where α-renaming is straightforward, while
modules address a more global space of mutually recursive macro and
variable definitions. A goal of our new binding model is to more
simply and directly account for such definition contexts.