Trying to refute some criticisms of Lisp
I recently had a discussion with someone on Mastodon about Lisp and its perceived (by them) deficiencies as a language. There were some interesting points, but I felt I had to try to refute them, at least partially.
I should say from the start the I’m not blind to Lisp’s many inadequacies and anachronisms, merely pointing out that it has a context like everything else.
There seemed to be two main issues:
- Poor design decisions throughout, and especially a lack of static typing
- The shadows of really early machines in
These points are tied together, but let’s try to unpack them.
Let’s start with design. Lisp is over half a century old. I’d argue it was exceptionally well-designed – when it was designed. It lacks most modern advances in types because … well, they didn’t exist, many of them arose as solutions to perceived problems in Lisp (and Fortran), and many of those “solutions” still aren’t universally accepted, such as static typing itself.
What we’ve actually learned is that many aspects of programming lack any really universal solutions. If static typing were such an obvious and unarguable route to efficiency and quality, all new software would be being written in Haskell.
Typing and features
And the lack of modern types isn’t really as clear-cut as it appears. The argument about the lack of features in Lisp also ignores the presence of other features that are absent from almost all other languages.
Lisp’s numeric types are surprisingly flexible. Indeed, Common Lisp is still, in the 21st century, just about the only language in which one can write modern crypto algorithms like Diffie-Hellman key exchange without recourse to additional libraries, because it has arbitrary-precision integer arithmetic built-in to the standard operators. It also has rational numbers, so no loss of precision on division either.
The Common Lisp Object System (CLOS) is vastly more flexible than any modern object-oriented language. Sub-class methods can specify their relationship with the methods they override, such as being called after or just filtering the return values. Methods themselves are multiple-dispatch and so can be selected based on the types of their arguments as well as their target. The basic mechanisms can be overridden or extended using a meta-object protocol.
Then there are macros. It’s easy to underestimate these: after all, C has macros, doesn’t it? Well, yes – and no. A C macro is a function from strings to strings that can do literal string substitution of its arguments. A Lisp macro is a function from code to code that can perform arbitrary computation. They’re really not the same things at all, and it’s misleading that the same word is used for both. (C++ templates are a closer analogy, but still limited in comparison.)
The persistence of hardware 1: Stupid operation names
The complaints about
cdr are long established: they
were originally derived from machine-language instructions on the
IBM 704 that was used for the first Lisp implementations. They’re
a terrible hold-over from that terrible decision … aren’t they?
Well, yes – and no. Of course they’re terrible in one sense. But
cdr are basically nouns as far as Lisp programmers are
concerned. One could replace them with more modern usages like
tail (and indeed many Lisps define these using macros).
But it’s important to remember that even “head” and “tail” are
analogies, sanctified by familiarity in the computer science
literature but still inexplicable to anyone outside. (If you doubt
that, try explaining to someone who isn’t a programmer that a
shopping list has a “head” consisting of the first entry, and a
“tail” consisting of another, shorter, shopping list, is “in fact”
a recursive type, and you have to acquire each item of shopping
sequentially by working your way down the list from the head.)
cdr are artificial nouns, and
cons is an artificial
verb – but really no more artificial that
append, their rough equivalents in other languages.
One can argue that the persistence of
cdr drives the
persistence of compounds like
caaddr. But those are unnecessary
and seldom used: barely anyone would mind if they were removed.
The persistence of hardware 2: It happens a lot
The suggestion that Lisp has hardware holdovers that should be removed also neglects these holdovers in other languages.
As an example, check the definition of
std::memcpy in C++. It
doesn’t work with overlapping memory areas. Why is that? – why is
it so fast, but so dangerous? Does it relate to underlying machine
features, such as machine code move instructions on particular
machines with particular restrictions? Doesn’t this introduce the
risk of security flaws like buffer overruns?
Languages with more abstracted machine models don’t have these issues. I struggle to think of how one could even introduce the concept of a buffer overrun into Lisp, other than by using some external raw-memory-access library: the language itself is immune, as far as I know.
The different choices
For the sake of argument, let’s turn the argument around and ask: give that early Lisps had proper macros, arbitrary-precision integers, and so on, why did these features disappear from what we now consider to be “the mainstream” of programming language design?
Lisp’s designers had a goal of building a powerful machine in which to think: indeed, they intended it to eventually have its own hardware designed specifically for it to run on. They therefore didn’t buy into the necessity of immediate performance, and as their applications were largely symbolic AI they didn’t need numerical performance at all. They chose instead to create high-level constructs even if these couldn’t be compiled efficiently, and explored using these to create more code as they identified more and more abstract patterns whose details could be automated away. (Paul Graham has a great essay on this.)
Other language designers had other priorities. Often they needed to do numerical simulation, and needed both performance and scale. So they chose a different design pathway, emphasising efficient compilation to the hardware they had available, and made the compromises needed to get it. These have persisted, and that’s why we have languages with fixed-width integers scaled to fit into a single machine register, and compilers that generate – but don’t directly execute – the code of programs, which limits our ability to abstract and automate code generation without recourse to complicated external tools.
It’s interesting to explore these choices. They’re at one level “just” historical: accidents that shaped the present. But at another level they’re still very much present in the hardware and software landscape we inhabit. I think it’s important that we remind ourselves, continuously, that much of that landscape is a choice, not a given, and one we can question and change as we wish.