Trying to refute some criticisms of Lisp

I recently had a discussion with someone on Mastodon about Lisp and its perceived (by them) deficiencies as a language. There were some interesting points, but I felt I had to try to refute them, at least partially.

I should say from the start the I’m not blind to Lisp’s many inadequacies and anachronisms, merely pointing out that it has a context like everything else.

There seemed to be two main issues:

  • Poor design decisions throughout, and especially a lack of static typing
  • The shadows of really early machines in car and cadr

These points are tied together, but let’s try to unpack them.

Design

Let’s start with design. Lisp is over half a century old. I’d argue it was exceptionally well-designed – when it was designed. It lacks most modern advances in types because … well, they didn’t exist, many of them arose as solutions to perceived problems in Lisp (and Fortran), and many of those “solutions” still aren’t universally accepted, such as static typing itself.

What we’ve actually learned is that many aspects of programming lack any really universal solutions. If static typing were such an obvious and unarguable route to efficiency and quality, all new software would be being written in Haskell.

Typing and features

And the lack of modern types isn’t really as clear-cut as it appears. The argument about the lack of features in Lisp also ignores the presence of other features that are absent from almost all other languages.

Lisp’s numeric types are surprisingly flexible. Indeed, Common Lisp is still, in the 21st century, just about the only language in which one can write modern crypto algorithms like Diffie-Hellman key exchange without recourse to additional libraries, because it has arbitrary-precision integer arithmetic built-in to the standard operators. It also has rational numbers, so no loss of precision on division either.

The Common Lisp Object System (CLOS) is vastly more flexible than any modern object-oriented language. Sub-class methods can specify their relationship with the methods they override, such as being called after or just filtering the return values. Methods themselves are multiple-dispatch and so can be selected based on the types of their arguments as well as their target. The basic mechanisms can be overridden or extended using a meta-object protocol.

Then there are macros. It’s easy to underestimate these: after all, C has macros, doesn’t it? Well, yes – and no. A C macro is a function from strings to strings that can do literal string substitution of its arguments. A Lisp macro is a function from code to code that can perform arbitrary computation. They’re really not the same things at all, and it’s misleading that the same word is used for both. (C++ templates are a closer analogy, but still limited in comparison.)

The persistence of hardware 1: Stupid operation names

The complaints about car and cdr are long established: they were originally derived from machine-language instructions on the IBM 704 that was used for the first Lisp implementations. They’re a terrible hold-over from that terrible decision … aren’t they?

Well, yes – and no. Of course they’re terrible in one sense. But car and cdr are basically nouns as far as Lisp programmers are concerned. One could replace them with more modern usages like head and tail (and indeed many Lisps define these using macros).

But it’s important to remember that even “head” and “tail” are analogies, sanctified by familiarity in the computer science literature but still inexplicable to anyone outside. (If you doubt that, try explaining to someone who isn’t a programmer that a shopping list has a “head” consisting of the first entry, and a “tail” consisting of another, shorter, shopping list, is “in fact” a recursive type, and you have to acquire each item of shopping sequentially by working your way down the list from the head.) car and cdr are artificial nouns, and cons is an artificial verb – but really no more artificial that head, tail, and append, their rough equivalents in other languages.

One can argue that the persistence of car and cdr drives the persistence of compounds like caaddr. But those are unnecessary and seldom used: barely anyone would mind if they were removed.

The persistence of hardware 2: It happens a lot

The suggestion that Lisp has hardware holdovers that should be removed also neglects these holdovers in other languages.

As an example, check the definition of std::memcpy in C++. It doesn’t work with overlapping memory areas. Why is that? – why is it so fast, but so dangerous? Does it relate to underlying machine features, such as machine code move instructions on particular machines with particular restrictions? Doesn’t this introduce the risk of security flaws like buffer overruns?

Languages with more abstracted machine models don’t have these issues. I struggle to think of how one could even introduce the concept of a buffer overrun into Lisp, other than by using some external raw-memory-access library: the language itself is immune, as far as I know.

The different choices

For the sake of argument, let’s turn the argument around and ask: give that early Lisps had proper macros, arbitrary-precision integers, and so on, why did these features disappear from what we now consider to be “the mainstream” of programming language design?

Lisp’s designers had a goal of building a powerful machine in which to think: indeed, they intended it to eventually have its own hardware designed specifically for it to run on. They therefore didn’t buy into the necessity of immediate performance, and as their applications were largely symbolic AI they didn’t need numerical performance at all. They chose instead to create high-level constructs even if these couldn’t be compiled efficiently, and explored using these to create more code as they identified more and more abstract patterns whose details could be automated away. (Paul Graham has a great essay on this.)

Other language designers had other priorities. Often they needed to do numerical simulation, and needed both performance and scale. So they chose a different design pathway, emphasising efficient compilation to the hardware they had available, and made the compromises needed to get it. These have persisted, and that’s why we have languages with fixed-width integers scaled to fit into a single machine register, and compilers that generate – but don’t directly execute – the code of programs, which limits our ability to abstract and automate code generation without recourse to complicated external tools.

It’s interesting to explore these choices. They’re at one level “just” historical: accidents that shaped the present. But at another level they’re still very much present in the hardware and software landscape we inhabit. I think it’s important that we remind ourselves, continuously, that much of that landscape is a choice, not a given, and one we can question and change as we wish.