1 Background: Scope and Macros

An essential consequence of hygienic macro expansion is to enable macro definitions via patterns and templatesalso known as macros by example (Kohlbecker and Wand 1987; Clinger and Rees 1991). Although pattern-based macros are limited in various ways, a treatment of binding that can accommodate patterns and templates is key to the overall expressiveness of a hygienic macro system, even for macros that are implemented with more general constructs.

As an example of a pattern-based macro, suppose that a Racket library implements a Java-like object system and provides a send form, where

(send a-point rotate 90)

evaluates a-point to an object, locates a function mapped to the symbol 'rotate within the object, and calls the function as a method by providing the object itself followed by the argument 90. Assuming a lookup-method function that locates a method within an object, the send form can be implemented by a pattern-based macro as follows:
(define-syntax-rule (send obj-expr method-name arg)
  (let ([obj obj-expr])
    ((lookup-method obj 'method-name) obj arg)))
The define-syntax-rule form declares a pattern that is keyed on an initial identifier; in this case, the pattern is keyed on send. The remaining identifiers in the parenthesized send pattern are pattern variables. The second part of the definition specifies a template that replaces a match of the pattern, where each use of a pattern variable in the template is replaced with the corresponding part of the match.

With this definition, the example use of send above matches the pattern with a-point as obj-expr, rotate as method-name, and 90 as arg, so the send use expands to
(let ([obj a-point])
  ((lookup-method obj 'rotate) obj 90))
Hygienic macro expansion ensures that the identifier obj is not accidentally referenced in an expression that replaces arg in a use of send (Kohlbecker et al. 1986). For example, the body of
(lambda (obj)
  (send a-point same? obj))
must call the same? method of a-point with the function argument obj, and not with a-point itself as bound to obj in the macro template for send. Along similar lines, a local binding of lookup-method at a use site of send must not affect the meaning of lookup-method in send’s template. That is,
(let ([lookup-method #f])
  (send a-point rotate 90))
should still call the rotate method of a-point.

Macros can be bound locally, and macros can even expand to definitions of macros. For example, suppose that the library also provides a with-method form that performs a method lookup just once for multiple sends:
(with-method ([rotate-a-point (a-point rotate)]) ; find rotate once
  (for ([i 1000000])
    (rotate-a-point 90))) ; send rotate to point many times
The implementation of with-method can make rotate-a-point a local macro binding, where a use of rotate-a-point expands to a function call with a-point added as the first argument to the function. That is, the full expansion is
(let ([obj a-point])
  (let ([rotate-a-point-method (lookup-method obj 'rotate)])
    (for ([i 1000000])
      (rotate-a-point-method obj 90))))
but the intermediate expansion is
(let ([obj a-point])
  (let ([rotate-a-point-method (lookup-method obj 'rotate)])
    (let-syntax ([rotate-a-point (syntax-rules ()
                                  [(rotate-a-point arg)
                                   (rotate-a-point-method obj arg)])])
      (for ([i 1000000])
        (rotate-a-point 90)))))
where let-syntax locally binds the macro rotate-a-point. The macro is implemented by a syntax-rules form that produces an anonymous pattern-based macro (in the same way that lambda produces an anonymous function).

In other words, with-method is a binding form, it is a macro-generating macro, it relies on local-macro binding, and the macro that it generates refers to a private binding obj that is also macro-introduced. Nevertheless, with-method is straightforwardly implemented as a pattern-based macro:
(define-syntax-rule (with-method ([local-id (obj-expr method-name)])
  (let ([obj obj-expr])
    (let ([method (lookup-method obj 'method-name)])
      (let-syntax ([local-id (syntax-rules ()
                                [(local-id arg)
                                 (method obj arg)])])
Note that the obj binding cannot be given a permanently distinct name within with-method. A distinct name must be generated for each use of with-method, so that nested uses create local macros that reference the correct obj.

In general, the necessary bindings or even the binding structure of a macro’s expansion cannot be predicted in advance of expanding the macro. For example, the let identifier that starts the with-method template could be replaced with a macro argument, so that either let or, say, a lazy variant of let could be supplied to the macro. The expander must accommodate such macros by delaying binding decisions as long as possible. Meanwhile, the expander must accumulate information about the origin of identifiers to enable correct binding decisions.

Even with additional complexities—where the macro-generated macro is itself a binding form, where uses can be nested so the different uses of the generated macro must have distinct bindings, and so on—pattern-based macros support implementations that are essentially specifications (Kohlbecker and Wand 1987). A naive approach to macros and binding fails to accommodate the specifications (Adams 2015, sections 4.2-4.5), while existing formalizations of suitable binding rules detour into concepts of marks and renamings that are distant from the programmer’s sense of the specification.

The details of a formalization matter more when moving beyond pattern-matching macros to procedural macros, where the expansion of a macro can be implemented by an arbitrary compile-time function. The syntax-case and syntax forms provide the pattern-matching and template-construction facilities, respectively, of syntax-rules, but they work as expressions within a compile-time function (Dybvig et al. 1993). This combination allows a smooth transition from pattern-based macros to procedural macros for cases where more flexibility is needed. In fact, syntax-rules is itself simply a macro that expands to a procedure:
(define-syntax-rule (syntax-rules literals
                      [pattern template] ...)
  (lambda (stx)
    (syntax-case stx literals
      [pattern #'template] ; #'_  is short for  (syntax _)
Besides allowing arbitrary computation mixed with pattern matching and template construction, the syntax-case system provides operations for manipulating program representations as syntax objects. Those operations include “bending” hygiene by attaching the binding context of one syntax object to another. For example, a macro might accept an identifier point and synthesize the identifier make-point, giving the new identifier the same context as point so that make-point behaves as if it appeared in the same source location with respect to binding.

Racket provides an especially rich set of operations on syntax objects to enable macros that compose and cooperate (Flatt et al. 2012). Racket’s macro system also relies on a module layer that prevents interference between run-time and compile-time phases of a program, since interference would make macros compose less reliably (Flatt 2002). Finally, modules can be nested and macro-generated, which enables macros and modules to implement facets of a program that have different instantiation times—such as the program’s run-time code, its tests, and its configuration metadata (Flatt 2013). The module-level facets of Racket’s macro system are, at best, awkwardly accommodated by existing models of macro binding; those models are designed for expression-level binding, where α-renaming is straightforward, while modules address a more global space of mutually recursive macro and variable definitions. A goal of our new binding model is to more simply and directly account for such definition contexts.