Matthew Butterick. Beautiful Racket: An Introduction to Language-Oriented Programming Using Racket. 2024.

Racket positions itself as a “programmable programming language” – more so than Lisp’s usual claim to this title – which makes it attractive for exploring multi-paradigm and multi-language programming.

Racket’s main claim to a niche is allowing a programmer to define new domain-specific languages (DSLs), not necessarily Lisp-like, that can be combined together to let a program be expressed in the “most appropriate” form for the problem. This books is structured around exactly this idea, defining a succession of languages of increasing sophistication to demonstrate Racket’s capabilities. Each language has a “reader” to convert strings into Lisp S-expressions and an “expander” to translate these forms into Racket (or another Racket-defined language). The examples include a simple stack-based calculator, an embedded JSON parser, and a Basic interpreter that exercises the power of a full parser.

The description of Racket itself is very thorough, to the extent that this book could be treated as a language definition itself, complementing the article about it. Certainly it provides the most accessible way to learn the language and explore what it means to explore “language-oriented” programming.

It’s possible to define DSLs in Common Lisp or Scheme without Racket’s extra functionality, and in many cases that will be sufficient – especially when the DSL is a Lisp itself. But there will certainly be applications for which Racket is absolutely the right tool.

Matthias Felleisen, Robert Bruce Findler, Matthew Flatt, Shriram Krishnamurthi, Eli Barzilay, Jay McCarthy, and Sam Tobin-Hochstadt. A Programmable Programming Language. Communications of the ACM 61, pp.62–71. 2018.

Lisp has been described as a “programmable programming language”: the fact that Lisp code is valid Lisp data and can be manipulated as such, coupled with macros, makes it easy to construct new Lisp variants in Lisp, and to combine them with “real” Lisp in interesting ways.

Racket is a Lisp that takes this to its logical conclusion, making domain-specific sub-languages into first-class objects that can be freely interleaved within programs and can be matched to the “natural” expression of the target domain. The languages don’t even have to be Lisp-like.

In the ideal world, software developers would analyze each problem in the language of its domain and then articulate solutions in matching terms.

Racket languages are formed of two parts: a reader that turns strings into properly bracketed S-expressions, and an expander that turns these expressions into “real” Racket (or some other Racket-defined language).

This flexibility adds a whole new level to Lisp programming. For example, it allows languages that don’t readily translate to Lisp style to be implemented and integrated into Lisp. This may simply be for user convenience, or may have some deeper semantic structure. Racket also defines clean interfaces between languages, which is what allows them to be combined. It’s a deeper version of the usual Lisp multi-paradigm approach, reaching out beyond macros working on Lisp syntax.

https://www.winestockwebdesign.com/Essays/Lisp_Curse.html

The power of Lisp is its own worst enemy.

An interesting essay on the “Lisp curse”, the idea that Lisp’s power can sometimes be a disadvantage for developers. The core idea expressed as a question: how hard is it to add object-oriented features to a language? In the C world it’s happened twice, giving rise to C++ and Objective-C, an no-one would even think about defining their own. In the Lisp world, by contrast, building that style of object-oriented extension is a weekend’s work (assuming we didn’t already have CLOS, which is in many ways superior anyway). The point is that it’s relatively easy to build even complex features as libraries, meaning there’s less reason to standardise, and this leads to a proliferation of mutually incompatible designs that solve 80% of the problem – but all a different 80%. This then makes interoperability difficult.

In fact the essay goes farther, to argue that the sorts of people who write Lisp tend not to be the sorts of people of work well in the political and compromise-driven world of standardisation. I’m not sure how true that is, given the stability of the Common Lisp standard itself since its adoption in 1994, but it’s an interesting reflection on how increased language expressiveness need not automatically be a strength.

(Some of the links in the article are now broken, or point to what looks like an AI link farm.)

Vsevolod Domkin. Programming Algorithms in Lisp. Apress. 2021.

This is exactly what it sounds like: an algorithms textbook that uses Lisp as its implementation language. It has the strengths and weaknesses of all such textbooks: it shows programmers, especially new programmers, how to write algorithms that are both correct and efficient, and the places where incorrectness and inefficiency will hide; and in doing this it tempts them to write algorithms that are already implemented in libraries, when they should really seek out and re-use the already-developed and -debugged code.

But of course there are always new algorithms that don’t have an existing library and therefore need to be written from scratch. This book is excellent preparation for that. It would give a new-ish programmer the confidence to tackle a challenging algorithm design and implementation job. It ranges well away from the common algorithms, to cover things like approximation and (especially importantly) parallelism and lock-free data structures, and I can imagine it providing templates that could be used as starting points for a huge range of new algorithms. Just remember not to re-invent the wheel.

https://fset.common-lisp.dev/Modern-CL/Top_html/index.html

A book-length description of a library, FSet, that provides functional collections for Lisp. By “functional collections” the author means collections – sets, trees, maps, and others – where update always produces a new collection rather than mutating the original collection in place. This is an attractive property for a language, and one that’s hard to do efficiently. As with Purely functional data structures, it provides a change of emphasis that makes for more referentially transparent code.

The book contains a lot of discussion about the library’s rationale and design, which makes it useful as a reference for others designing large libraries (or new languages, which of course in Lisp can mean much the same thing). There’s an equally important discussion about what’s not provided (for example lazy collections) which is really interesting by giving an example of when to stop with a codebase that could otherwise probably grow indefinitely.