Simon Dobson (Posts about python)

The different energy footprints of different programming languages

Simon Dobson — Fri, 27 Sep 2024 15:15:23 GMT

The different energy footprints of different programming languages

I've recently been thinking about low-power computing, from AI and data centres down to sensors, as part of a new initiative the University is considering. It's easy to forget the resource footprint of our computing systems – especially those that are "out of sight, out of mind" in the cloud – but there's growing evidence that their growth threatens the renewable energy transition. Some of the figures for AI electricity and water usage are astonishing.

One aspect of this is the effect of choice of programming language. I can across some work from 2017 on this:

Rui Pereira, Marco Couto, Francisco Ribeiro, Rui Rua, Cunha Jácome, João Paulo Fernandes, and João Saraiva. Energy Efficiency across Programming Languages: How Do Energy, Time, and Memory Relate? In Proceedings of the 10th ACM SIGPLAN International Conference on Software Language Engineering. 2017.

The authors compare 13 benchmarks run in 27 different languages, with the benchmarks being chosen widely to avoid being biased by numeric performance. I was expecting some patterns: compiled languages doing better on performance, memory, and energy usage, for example. But I wasn't expecting exactly how widely the figures diverged, or some of the details.

The following table is from the paper, annotated by me. The figures are normalised against the best result in each category (so the top language has value 1, and so on).

The two most-used languages for web application, Python and JavaScript, perform uniformly pretty badly: 75 times C's energy usage, in Python's case. But although JavaScript does substantially better on energy (only a factor of 4), TypeScript – which is usually thought of as JavaScript with type pre-processing – requires 21 times C's energy, or 5 times JavaScript's. Why is that? – I can't think of a reason.

But the real surprise was that "research" languages like Haskell and Lisp both hold up well: twice C's energy, in Lisp's case. I don't think that would surprise modern Lisp programmers, who are used to their compilers' efficiencies – but it would surprise someone used only to the "hearsay" about Lisp. The same for Haskell, actually, whose modern compilers really leverage the extra structure. When you consider that both those languages are pretty much dependent on garbage collection and so are doing substantially more work than the equivalent C program, it's impressive.

(Also look in the table for Racket, consistently lower than Lisp despite their close similarities. I suspect this is a compiler optimisation issue more than anything else.)

This work clearly isn't complete or definitive. Clojure is entirely missing, as is Scala, and there will have been compiler improvements since 2017 for the languages with the most active developer communities. But it's still quite sobering that the differences are so wide, and that we've chosen to push languages that exacerbate energy usage rather than managing it.

Notes on using Jupyter in the cloud

Simon Dobson — Fri, 02 Dec 2022 16:36:55 GMT

Notes on using Jupyter in the cloud

I've been thinking about running Jupyter notebooks in the cloud for some fairly compute-intensive simulation. Specifically I want to do epidemic and other simulations over complex networks. These are CPU-intensive and don't make use of GPU acceleration (yet, anyway). Using the cloud would make things easier to scale-out, especially for those without access to a local compute cluster.

Adding filters to a Jinja2 template for Nikola

Simon Dobson — Wed, 19 Oct 2022 11:08:59 GMT

Adding filters to a Jinja2 template for Nikola

The Jinja2 template engine defines a suite of filters that can be used to transform text as it's rendered. You can define extra custom filters, and also make them available within Nikola templates (as long as the theme uses Jinja as its template engine, of course.)

Read more… (1 min remaining to read)

Talk at UK Systems on unit testing stochastic code

Simon Dobson — Mon, 06 Dec 2021 16:49:30 GMT

Last week I went to my first scientific meeting since February 2021 to talk about unit testing of stochastic code.

Read more… (1 min remaining to read)

New version of epyc released

Simon Dobson — Mon, 06 Sep 2021 12:50:48 GMT

I released a new version of my epyc experimental management library.

Read more… (1 min remaining to read)

Generating functions for epydemic

Simon Dobson — Thu, 10 Jun 2021 17:22:16 GMT

The next version of epydemic has just been released. It now includes a generating functions library.

Read more… (1 min remaining to read)

"Epidemic modelling" added to the executable books gallery

Simon Dobson — Fri, 04 Jun 2021 07:24:05 GMT

"Epidemic modelling: Some notes, maths, and code", my book about using network science to model epidemics, is now featured in the gallery of the Executable Books project.

Read more… (1 min remaining to read)

New, faster, release of epydemic

Simon Dobson — Fri, 14 May 2021 08:33:17 GMT

I just made a new release of epydemic.

Read more… (1 min remaining to read)

IDE convergence

Simon Dobson — Tue, 23 Feb 2021 12:21:05 GMT

I recently tried out a new development environment for my Python development, and noticed an unexpected convergence in the designs of the two tools.

I've been a long-time Emacs user. I periodically get a desire to try something new, something less old-school, just to see whether there are advantages. There always are advantages, of course -- but often significant disadvantages as well, which often keep me coming back to my comfort zone.

My most recent excursion was to try Microsoft's VS Code.

This is handily cross-platform, being built in Javascript on top of Electron. It's got a lot of nice features: a tree view of the project in the left-hand pane, syntax colouring, code style linting, integrated debugging and unit test running, integrated connection to git, and so on. Looking a little closer there are all sorts of status markers around the code and in the ribbons at the bottom of panes and the window overall to show status that might be important.

But it's so slow. That's a feature of VS Code, not of Electron (as I first suspected), because other Electron-based editors like Atom aren't as slow. And my development box isn't the latest, but it also isn't that old.

So I reverted to Emacs, but upgraded it a little to more modern standards. Specifically, I installed the elpy Python IDE, with assorted other packages suggested by various sites. The result is this:

Now for anyone who's used Emacs for a while, it's definitely still Emacs -- not least with the convoluted keystrokes and infinite customisation you either love or hate. But it's striking how similar the two IDEs now are, and striking how VS Code has inherited some ideas from Emacs: resizeable panes, modelines in the ribbon, markers in pane gutters, and so forth -- things that Emacs-based applications have had for years, which have now migrated into "the mainstream". Both the feature sets and the visuals of the two systems are very similar indeed. Both are entirely cross-platform and extensible. For VS Code you write extensions in Javascript; for Emacs you write them in Lisp; and that's about it. And Emacs is a lot faster on my set-up. There are some limitations -- I've yet to get the hang of using pdb as a debugger, for example, especially for modules and from within tests -- but the functionality is really quite comparable.

I think it's safe to say there's been cross-fertilisation between VS Code (and other IDEs) and Emacs over the years. A lot of the developers of the former quite possibly used the latter. But I strongly suspect that most of the traffic has gone from Emacs to the other systems: the similarities are just too great to be accidental. It's interesting to think that a system that emerged at the dawn of the free-software movement has had -- and is still having -- such an influence on modern development tools. And I'm happily back in my comfort zone.

Backporting Python type annotations

Simon Dobson — Wed, 09 Dec 2020 10:45:43 GMT

I recently added type annotations to my epyc and epydemic libraries. Making these work while not sacrificing interoperability a wide range of Python versions is quite delicate.