The old-school approach to email involves several elements, each potentially provided by a different tool:

• a client program or mail user agent (MUA) that presents email to you and lets you search, delete, store, etc;
• a retrieval program or mail delivery agent (MDA) that retrieves mail from the providers and manages local email directories
• a sending program or mail transfer agent (MTA) that takes; and locally-created messages and transfer them to their intended recipients.

Modern GUI email clients like Thunderbird typically wrap-up all three services into one program that’s easier to deploy and manage, but that therefore forces certain choices on the user. By reverting to the older architecture we regain flexibility and choice, at the expense of making our lives harder.

All these tools need to authenticate against other services. Traditionally this used usernames and passwords, which are clearly inadequate for the modern web. Instead we need a system based around stronger encryption.

OAUTH2 is a an authorisation delegation protocol that lets a site grant access to someone who’s authenticated against another, without getting sight of their credentials. The typical use case is for a web site to allow users to sign-in using social media services like Facebook or Google, which reduces the number of passwords a user needs to remember or manage – and, totally incidentally I’m sure, improves the social media services’ ability to track users’ activities across the web.

In our case, the OAUTH2 “flow” interacts with the authentication provider and acquires a bearer token that can then be presented to authorise access to the various email services.

In outline the solution is as follows:

1. Install mbsync as MDA
2. Set up OAUTH2 authentication for Office365
3. Use the to authenticate mbsync against Office365 to allow retrieval
4. Install msmtp as MTA, using the same authentication scheme
5. Install mu4e as MUA, since I want to read my email from inside Emacs

Under Arch Linux we need the isync package for synchronisation and the cyrus-sasl-xoauth2 provider for OAUTH authentication.

   sudo pacman -S isync
   yay -S cyrus-sasl-xoauth2

The same packages are available for other distros under similar names. Note that the actual synchronisation tool is called mbsync, even though the package that contains it is called isync.

We want to use OAUTH2 to authenticate an IMAP transaction, so that no additional passwords are needed. To this we need a script to manage the OAUTH2 flow.

Weirdly for an operation that’s becoming so common on the web, there doesn’t seem to be a package that offers OAUTH2 from the command line. However, there is a script that does it that’s included as an example with the mutt MUA, and we can use that. It can be found (in Arch) in the mutt package.

   sudo pacman -S mutt
   cp /usr/share/doc/mutt/samples/mutt_oauth2.py .

This puts a copy of the script into the current directory, which we can then edit in two ways:

• add the internal application identification and client secrets for accessing Office365; and
• set up the security for the OAUTH2 access tokens when they’re downloaded and held locally.

The client secret and app id need to be “proper”, in the sense that Office365 knows about them – but weirdly they don’t have to be related to your email domain or cloud tenancy. It’s perfectly fine to use credentials available in the public domain, for example those of Thunderbird:

   AppID = "08162f7c-0fd2-4200-a84a-f25a4db0b584"
   ClientSecret = "TxRBilcHdC6WGBee]fs?QR:SJ8nI[g82"

(I got these from here, but other than that have no idea where they come from: they’re not the same as those in the Thunderbird source code, as far as I can tell.)

The mutt_oauth2.py script stores the tokens it manages in a gpg-encrypted file. You therefore need to provide your gpg keypair identification, and I’m assuming anyone wanting to get local email has one of those! Mine is “simoninireland”.

   GPGKey = "simoninireland"

I edited the file to look like this, with some details elided:

   MSAppID = "08162f7c-0fd2-4200-a84a-f25a4db0b584"
   MSClientSecret = "TxRBilcHdC6WGBee]fs?QR:SJ8nI[g82"
   GPGKey = "simoninireland"

   ENCRYPTION_PIPE = ['gpg', '--encrypt', '--recipient', GPGKey]
   DECRYPTION_PIPE = ['gpg', '--decrypt']

   registrations = {
       'google': {
           ...
           'client_id': '',
           'client_secret': '',
       },
       'microsoft': {
           ...
           'client_id': MSAppID,
           'client_secret': MSClientSecret,
       },
   }

Put resulting script into /usr/local/bin and make it executable. Then run it in “authorisation” mode. The token file can go anywhere: I put it in the directory used by pass to allow for an alternative access route:

   mutt_oauth2.py -t .password-store/email/work.gpg --authorize

This will ask some questions:

• we want “microsoft” authentication
• and a “localhostauthcode” flow
• enter your email address (the actual user, not any alias)

and it prints out a URL to copy into a browser to authenticate against Office365’s web interface. In my case this involved interacting with the university’s single sign-on and two-factor authentication (2FA) system. Doing this successfully put the necessary OAUTH2 tokens, encrypted, into the specified file. Running:

   mutt_oauth2.py -t .password-store/email/work.gpg

will output the token, refreshing it automatically if it’s expired. This may ask for the GPG key’s passphrase, if it has one, and if it’s not available from a local key agent.

(All this security means that the bearer tokens are stored encryoted at rest. It’s a little inconvenient, though, as it means you need to enter a gpg passphrase periodically, and makes it hard to run mbsync in a cron job. This is fine if, like me, your level of security paranoia is such that you accept the minor inconvenience in exchange for not having plain-text access tokens lying around; on the other hand, you may decide that using, for example, a machine with full-disc encryption is secure enough, in which case you need to edit the ENCRYPTION_PIPE and DECRYPTION_PIPE commands in the script to not do encryption: they can basically just use cat to store and retrieve the token information.)

We now have OAUTH2 tokens for accessing Office365, which we can integrate with our MDA. mbsync has four main concepts:

• Accounts, typically using IMAP
• IMAP message stores, which are remote
• Maildir stores, which are local
• Channels, which tie local and remote together

Maildir is a file format for storing email in a directory structure, and is a long-running standard that’s supported by lots of tools. A maildir is typically presented in the MUA to a user as a folder, and represented to the MDA as a directory.

For Office365 we have:

   IMAPAccount work
   Host outlook.office365.com
   Port 993
   User <<work-email>>
   PassCmd "mutt_oauth2.py -t ~/.password-store/email/work.gpg"
   AuthMechs XOAUTH2
   SSLType IMAPS

   IMAPStore work-remote
   Account work

   MaildirStore work-local
   Subfolders Verbatim
   Path ~/Maildir/Work/
   Inbox ~/Maildir/Work/Inbox

   Channel Work
   Far :work-remote:
   Near :work-local:
   Patterns * !"Conversation History" !Calendar !Archive !Archives !Clutter !Drafts
   Create Both
   Expunge Both
   SyncState *

(See the mbsync man pages for the details of its configuration. <<work-email>> should be a a proper username, not an alias.) For our purposes the important line is the PassCmd that calls our edited script to retrieve the OAUTH2 bearer token. Email will be downloaded into a maildir tree rooted at ~/Maildir/Work: you need to create this before sync-ing.

   mkdir -p ~/Maildir/Work

For a full sync of all maildirs just run:

   mbsync -a

That can be time-consuming, as all the maildirs (i.e., folders) have to be visited – and I have several hundred. A faster option is to normally just look at (for example) the inbox:

   mbsync Work:INBOX

This will ignore everything else, which means they’ll drift – but can be re-sync’ed periodically by running a full sync. One could also set up a cron job to do a full sync early every morning, for example, as long as the access token was held unencrypted (see above).

You’ll almost certainly now want to index your newly-downloaded trove of messages. There are two common tools for this mu and notmuch. Both do basically the same job of maintaining a structured and full-text index of messages that can be queried by an appropriate MUA. I chose mu, for no particular reason: some people swear by notmuch, which is based on extensive tagging of messages and so might be more familiar to Gmail users.

To install mu, we first grab the package:

   pacman -S mu

We then initialise the index by running the indexer over the maildir. If we also provide our own email address (or more than one) it knows to index these differently.

   mu init -m ~/Maildir --my-address=<<work-email>>

All of the above sets up the MDA to get mail: we now need to be able to send mail. Fortunately we’ve already done most of the hard work needed to get this working.

We need a local MTA, for which I chose msmtp. It understands OAUTH2 natively. Installation in Arch is easy:

   sudo pacman -S msmtp

It needs to be pointed at the Office365 SMTP server and provided with the OAUTH2 tokens, which are the same as we used above:

   defaults
   auth           on
   tls            on
   tls_starttls   on
   tls_trust_file /etc/ssl/certs/ca-certificates.crt
   logfile        ~/.msmtp.log

   account        work
   host           smtp.office365.com
   port           587
   auth           xoauth2
   user           <<work-email>>
   from           <<work-email>>
   passwordeval   "mutt_oauth2.py -t ~/.password-store/email/work.gpg"

   account default : work

Again, see the msmtp man pages for the details of this, and replace <<work-email>> as appropriate: the only interesting part from our current perspective is that the passwordeval line calls exactly the same script as we used above.

Finally we’re ready to read email. I’ll leave this to you: there are lots of text-based email clients around, notably mutt that we encountered earlier. There’s also mu4e for reading email in Emacs, making use of the mu index; and notmuch also has an Emacs interface.

I use mu4e. There’s a lot of documentation on the web for setting this up, all of which applies immediately to our new set-up: the MUA is entirely independent of the MDA and MTA, and simply needs to be pointed at the right directories and accounts.

Gmail lets one use “app passwords” for accessing using IMAP, but also supports OAUTH2, which is obviously more secure. The same approach as above works for Gmail too. The initial credentials are:

   GAppID = '406964657835-aq8lmia8j95dhl1a2bvharmfk3t1hgqj.apps.googleusercontent.com'
   GClientSecret = 'kSmqreRr0qwBWJgbf5Y-PjSU'

(Same source as above.) Edit these into the script and change the entries in the config files to call it to authenticate with an appropriate store, for example:

   mutt_oauth2.py -t .password-store/email/personal.gpg --authorize

and similarly in the configurations of mbsync and msmtp.

If you’re still with me: congratulations, but you must really want to read your email old-school!

For me, this has completely changed my relationship with email in ways I didn’t expect. Using Emacs means typically not having the client visible all the time, which reduces the temptation to check all the time. Instead I can adopt a more structured approach and only check my email when I want to, which often means only three or four times a day. It’s also made email easier to manage, for example by adding hyperlinks in my to-do list straight to messages that need attention, and adding some integrations with org mode to simplify email processing. Those are matters for another time, though.

There are many resources on using mbsync, mu, mu4e, and the rest on the web. I found these covered all the topics in great detail, with the exception of the OAUTH2 integration I’ve detailed here. In particular I’d like to acknowledge the following:

• Growing pains, Casper van Elteren’s mu4e setup that pointed me in the right direction
• Microsoft OAuth authentication with mu4e in Emacs, which uses offlineimap, a different MDA, and needs closer integration with Microsoft Azure that often isn’t possible for corporate email
• Configure Mutt to work with OAuth 2.0 that focuses on using mutt and Gmail