Let over Lambda

Let over Lambda

nil

Doug Hoyte. Let over Lambda. HCSW and Hoytech. ISBN 978-1-4357-1275-1. 2008.

A gloriously opinionated (in the best way) take on Lisp macro programming. The author treats Lisp – one of the first languages – as the ultimate, final language to which all others can (and should) be translated, since they’re simply special cases of Lisp forms. And he makes a surprisingly compelling case.

The material builds extensively on On Lisp, re-presenting the idea of anaphoric macros (those that bind lexical variables for use in their bodies) and extending them with pandoric macros that allow introspection into an otherwise closed lexical environment. A lot of these examples feel contrived as presented, but the techniques are anything but: powerful ways to build abstractions.

The finale is the implementation of Forth in Lisp, and then a reflection to allow Forth programs to define Lisp functions – converting a (transparent) threaded-interpreted structure into an (opaque) code block. It’s a slightly non-standard Forth, but it’s true to the basics and exposes issues like the implementations of interpretation, compilation, and tail recursion really well. It’s only another step to explore Lisp implemented in Forth through the medium of macro programming, which deserves a book itself.

On Lisp

On Lisp

Paul Graham. On Lisp: Advanced Techniques for Common Lisp. Prentice Hall. ISBN 0-13-030552-9. 1993.

An ideal world is left as an exercise to the reader.

One of the most famous advanced treatments of Lisp. Essentially it’s a book in two parts. The first deals with the traditional advanced topics of functional programming such as identifying patterns that can be factored-out into higher-order functions and re-used. But the second, better-known, part deals with macro programming in extraordinary detail and with enormous creativity, including developing macros for continuations and several embedded languages.

Debugging Lisp

Debugging Lisp

Debugging Lisp

A four-part blog post series on debugging Lisp, covering:

  1. Live recompilation of code from inside a debugging session, together with re-executing changed code from within a running stack trace
  2. Inspecting objects and interacting with traces of function calls
  3. Redefining classes, and how to ensure that existing instances are upgraded to be compatible with the new definition
  4. Restarts, the neglected part of the condition system that controls how programs continue after encountering errors

The first two are essential, and show how different Lisp programming is from using other languages. In fact it requires a considerable mental shift to re-acquire the right reflexes for dealing with errors and debugging in a fully interactive environment: well, it did for me, anyway. We’re not used to interactivity in large development environments. There is seldom any need to close down a running Lisp session and start again, as everything can usually be changed and adapted within a session. This is very unlike the usual compile-edit-debug cycles we’ve become accustomed to.

The third post – on redefining classes – shows how one can upgrade a program that simply has to keep running, because its live state upgrade can be programmed too.

The most significant part of the mental shift is to realise that the debugger is written in Lisp itself, and makes use of restarts and other features to provide the interface. This is a consequence of the degree of exposure of the Lisp run-time structures into to language itself, where they can be examined and manipulated using the full power of the language – and then be re-started or discarded as required.

Ultra-Processed People: The Science Behind Food That Isn’t Food

Chris van Tulleken (2023)

Food versus food-shaped industrial products.

Ultra-processed food (UPF) is pervasive in modern diets, and with it comes a litany of actual, conjectured, and supposed harms. UPF itself is a strange beast, taking food crops and turning them into pure components that can then be re-mixed to construct new products. This has some perverse consequences, such as flavours being removed from purified and “modified” oils and starches in order to make them more broadly usable – and then having those same flavours re-introduced later in the process, again in “modified” form. That sounds insane – how can it be cheaper than using the original oil? – but in fact it makes perfet commercial sense for companies wanting fungible raw materials to produce unvarying, known-ahead-of-time tastes and textures.

The resulting products (I’m now reluctant to call them foods), being created in labs, can be re-worked in pursuit of particular commercial goals, for example by adding powdered soya when looking to create a “high-protein” snack. They can also be re-engineered to be far more pleasurable and addictive for consumers, for example by hacking the body’s responses to food (which are themselves coming to be understood as far more subtle and complicated than we used to think).

van Tulleken has a science background, and it shows in the writing: most of the claims are carefully framed and evidenced. His background also saves him from falling for the industry’s faux-refutations about there not being definitive causal links to specific harms: randomised controlled trials aren’t the “scientific gold standard” in situations where they’re impossible to conduct in the real world, and epidemiological evidence coupled with some knowledge of the possible harm pathways can provide sufficient evidence. Having said that, he does sometimes deviate from this careful path, and there are a few instances of words like “may” and “could” doing a lot of heavy lifting.

One of the most powerful elements of the book is that it simultaneously doesn’t preach or prescribe, but does offer suggestions for ways forward. UPF is very difficult to precisely define, and is therefore difficult to legislate for or avoid. Does a single stabiliser in a product render it UPF? – because if so literally anything in a packet would be included. van Tulleken also takes aim at some of the wider social drivers of UPF, notably its cheapness and ease of preparation compared to “real” food, reinforcing poor diet as a consequence of poverty. He also suggests some interesting policy options, while also taking aim at a policy infrastructure that’s heavily co-opted by the UPF industry. Regulators and the food industry are not partners and have goals that are mutually irreconcilable within the current framework of pre-eminent shareholder value.

5/5. Finished Thursday 4 July, 2024.

(Originally published on Goodreads.)

Class slots that work with classes and instances in CLOS

Class slots that work with classes and instances in CLOS

I recently had a use case where I wanted to associate a constant value with a class and its instances – but I needed to be able to get the value without having an instance to hand. This turns out to be solvable in CLOS.

In languages like Java you can associate class variables with classes, which can then be accessed without having an instance of the class. CLOS also has class-allocated slots, for example:

    (defclass A ()
      ((instance-slot
        :initform 1)
       (class-slot
        :allocation :class
        :initform 2))
      (:documentation "A class with instance- and class-allocated slots."))

An instance of A has two slots: instance-slot stored per-instance, and class-slot stored only once and shared amongst all instances. This is close to Java’s notion of class variables, but one still needs an instance against which to call the method. (Seibel makes this point in chapter 17 of “Practical Common Lisp”.)

One could just create a basic object and retrieve the slot:

 (slot-value (make-instance 'A) 'class-slot)

but that’s inelegant and could potentially trigger a lot of unnecessary execution (and errors) if there are constructors (overridden initialize-instance methods) for A. One could use the metaobject protocol to introspect on the slot, but that’s quite involved and still allows the slot to be changed, which isn’t part of this use case.

What I really want is to be able to define a generic function such as class-slot – but specialised against the class A rather than against the instances of A. I thought this would need a metaclass to define the method on, but it turned out that generic functions are powerful enough on their own.

The trick is to first define a generic method:

    (defgeneric class-slot (classname)
      "Access the class slot on class.")

As the argument name suggests, we’re planning on passing a class name to this method, not an instance. To set the value for A, we specialise the method as working on exactly the class A:

    (defmethod class-slot ((classname (eql 'A)))
      2)

The eql specialiser selects this method only when exactly this object is passed in – that is to say, the name of A.

But what if we have an instance of A? The same generic function can still be used, but instead we specialise it against objects of class A in the usual way:

    (defmethod class-slot ((a A))
      (class-slot (class-name (class-of a))))

If we now pass an instance of A, we extract its class name and then re-call the same generic function, passing it the class name instead of the object itself (which it doesn’t need, because the slot value is independent of the actual object). This will select the correct specialisation and return the slot value.

This approach works if we generate sub-classes of A: we just use eql to specialise the generic function to the class we’re interested in. It also works fine with packages, since the undecorated symbol passed to the specialiser will be expanded correctly according to what symbols are in scope. However, the value is only associated with a single class, and isn’t inherited. That’s not a massive limitation for my current use case, but would be in general, I think.

This approach critically relies on an easily-forgotten property of Lisp: values have types, but variables don’t, and we can specialise the same generic function against any value or type. The pattern makes use of this to avoid actually storing the value of class-slot anywhere, which as a side effect avoids the problem of someone accidentally assigning a new value to it. It’s an example of how powerful generic functions are: more so than the method tables and messages found in most O-O languages. And it’s sufficiently structured that it’s crying-out for a couple of macros to define these kinds of class slots.

UPDATED 2024-06-29: Fixed the typo in the class definition to use :initform and not :initarg. Thanks to @vindarel for pointing this out to me.