Making a function that defines functions in GNU Emacs ELisp

September 18, 2023

Suppose that for some reason you're trying to create a number of functions that follow a fixed template; for example, they should all be called 'mh-visit-<name>' that will all use mh-visit-folder to visit the (N)MH folder '+inbox/<name>'. In my last installment I did this with an Emacs Lisp macro, but it turns out there are reasons to prefer a function over a macro. For example, you apparently can't use a macro in dolist, which means you have to write all of the macro invocations for all of the functions you want to create by hand, instead of having a list of all of the names and dolist'ing over it.

There are two ways to write this function to create functions, a simpler version that doesn't necessarily work and a more complicated version that always does (as far as I know). I'll start with the simple version and describe the problem:

(defun make-visit-func (fname)
  (let ((folder-name (concat "+inbox/" fname))
        (func (concat "mh-visit-" fname)))
    (defalias (intern func)
      (lambda ()
        (mh-visit-folder folder-name))
      (format "Visit MH folder %s." folder-name))))

If you try this in an Emacs *scratch* buffer, it may well work. If you put this into a .el file (one that has no special adornment) and use it to create a bunch of functions in that file, then try to use one of them, Emacs will tell you 'mh-visit-<name>: Symbol’s value as variable is void: folder-name'. This is because folder-name is dynamically scoped, and so is not captured by the lambda we've created here; the 'folder-name' in the lambda is just a free-floating variable. As far as I know, there is no way to create a lexically bound variable and a closure without making all elisp code in the file use lexical binding instead of dynamic scoping.

(As of Emacs 29.1, .el files that aren't specifically annotated on their first lines still use dynamic scoping, so your personal .el files are probably set up this way. If you innocently create a .el file and start pushing code from your .emacs into it, dynamic scoping is what you get.)

Fortunately we can use a giant hammer, basically imitating the structure of our macro version and directly calling apply. This version looks like this:

(defun make-visit-func (fname)
  (let ((folder-name (concat "+inbox/" fname))
        (func (concat "mh-visit-" fname)))
    (apply `(defalias ,(intern func)
              (lambda ()
                ,(format "Visit MH folder %s." folder-name)
                (mh-visit-folder ,folder-name))))))

(This time I've attached the docstring to the lambda, not the alias, which is really the right thing but which seems to be hard in the other version.)

As I understand it, we are effectively doing what a macro would be; we are creating the S-expression version of the function we want, with our let created variables being directly spliced in by value, not by their (dynamically bound) names.

PS: I probably should switch my .el files over to lexical binding and fix anything that breaks, especially since I only have one right now. But the whole thing irritates me a bit. And I think the apply-based version is still a tiny bit more efficient and better, since it directly substitutes in the values (and puts the docstring on the lambda).

Comments on this page:

Not being able to call a macro in a (dolist ...) form is a bit strange at first glance. The nasty part is a bit of subtle nuance. The solution is wrapping the macro call in a (lambda ...) form like so:

(dolist ...
        (lambda (fn-arg1 fn-arg2)
          (macro-name fn-arg1 fn-arg2))

Here is how:

A function call and a macro call are supposed to look the same. In one sense they are similar and in another sense a world apart. They are similar in their superficial interpretation that the form (foo-fn ...) and (foo-macro ...) both take parameters and return a value.

They are a world apart with regards to how they get that value. A function evaluates all arguments eagerly from left to right and exactly once. Then the function is called with all argument values.

A macro, on the other hand, is more of a user-defined special form. This simply means that at least one of its "arguments" is not eagerly evaluated exactly once as if going from left to right. This could mean that it is not evaluated at all. This could mean that they are evaluated out of order. This could mean that it is evaluated multiple times. This could mean, that the body -- the "..." -- is not even a valid lisp form (read "lisp code") as far as the core language is concerned.

The special form "if"

(if (print "foobarbaz")
    (print "then-branch")
    (print "else-branch"))

does this. Haskell achieves a similar effect by lazy evaluation.

Looping constructs may evaluate a form multiple times, for example the increment step or their body.

Special forms are (special-form ...) forms that don't follow the once-only left-to-right evaluation that function calls do and they are part of the core language. Macros look like function calls, but behave like special forms. Macros are user-defined special forms.

As far as the core language is concerned, it does a depth-first traversal from left to right. This part matches with evaluating function arguments. Whenever it hits a macro, traversal stops. It cannot directly descend into that subtree. The macro defines the root of a subtree where the syntax of the core language does not have to hold. The macro needs to be expanded. If the new form (expanded-form ...) is another macro call, then it is expanded again and so on, until it is a value or a function call. The subtree gets replaced again and again until the core language knows how to descend into it. If there is another macro call hidden in that subtree, it gets recursively expanded until it is not a macro call anymore when it is encountered during depth-first descent.

This way of expanding and descending also means, that arguments to a macro call that look like function calls are treated as a list of lists-and-atoms, instead of being evaluated. Confer Common Lisp (and a b):

(and (+ 1 2) (* 3 4))

almost becomes

 (if (+ 1 2)      ;; Literally (+ 1 2) instead of 3.
     (if (* 3 4)  ;; Literally (* 3 4) instead of 12.
         (* 3 4)  ;; Literally (* 3 4) instead of 12.

To avoid multiple evaluation, a gensym and let-binding are needed.


(and 999
     (progn (print "hello")
            (* 3 4)))


 (if 999
     (let ((#:my-non-collidable-gensym-name-5
              (progn (print "hello")
                     (* 3 4))))
       (if #:my-non-collidable-gensym-name-5

The code for defining a macroexpansion like this looks like so

(defmacro and (a b)
  ;; gensym. Evaluated during macro expansion, creates a unique variable every time
  (let ((gs-for-b (gensym "my-non-collidable-gensym-name-"))) ;; missing "5"
    ;; the (if ... (if ...)) template
    `(if ,a
         ;; evaluate parameter b only once. Confer (once-only ...)
         (let ((,gs-for-b ,b))
           (if ,gs-for-b

The funny-looking ",gs-for-b" dance makes sure, that those symbols -- read variable names -- are the same. The lisp reader must botch reading #:a as identical to another #:a to make the gensyms uncollidable. Parsing a lisp source code string lacks the information to determine which gensyms are the same variable and which gensyms happen to have the same printed name.

* * *

To "use" a macro in something like (dolist ...) -- or (mapcar ...) for that matter -- you have to wrap it in a (lambda ...).

Here is why:

A simpler example is using the short-circuiting macro (and ...) from Common Lisp.

Form (and a b) basically becomes (if a (if b b nil) nil) -- see above for the nitty-gritty details. Contrast this with (+ 1 2) which becomes 3. The form (and a b) is not a function call.

(mapcar (and x y)

does not work, because the core language sees

(mapcar (if x
            (if y

What works is

(mapcar (lambda (x y)
          (and x y))

It works because it is equivalent to writing

(macpar (lambda (x y)
          (if x
              (if y

The macro (and ...) gets expanded in the body of the (lambda ...).

* * *

The upside of macros being so close to functions and yet a world apart, are macros like Common Lisp's (cl:with-open-file ...). You will never forget to close a file again, at least in single-threaded code.

Try that with a C preprocessor macro. C fails to have gensyms. C fails to have the special form (cl:unwind-protect ...). And best of all, a user of the core language a.k.a. you, the programmer, can define macros as powerful as (cl:with-open-file ...).

* * *

Side note: The backtick syntax is for list templates. It is irrelevant whether or not lists are returned from a macro, a function or something else. They just happend to be used in most macro bodies, because macro bodies return lists == lisp code. List templates can also be used with functions, for example in (apply ...) forms.

Side note: The source code is a string. Reading the string yields symbolic expressions, so lists and atoms. The symbolic expressions that follow the rules of the core language are known as lisp forms. This is where evaluation by the core language interpreter comes into play. There are self-evaluating atoms. There are lists. Lists can be either function calls or special forms. The two differ in how their arguments are treated, refer towards the top of this text. Macros are basically user-defined special forms.

From at 2023-09-19 13:46:52:

The simpler version does what you want if you splice the values into the lambda, i.e.

   `(lambda ... ,... ...)

(Which is what your second version ends up doing in a more complicated way.)

By Derek Upham at 2023-09-20 10:24:08:

As far as I know, there is no way to create a lexically bound variable and a closure without making all elisp code in the file use lexical binding instead of dynamic scoping.

Use macro lexical-let, which pre-dates the file-based mechanism and still exists.

Written on 18 September 2023.
« Unix shells are generally not viable access control mechanisms any more
How Unix shells used to be used as an access control mechanism »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Sep 18 22:02:24 2023
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.