Skip to content

A modest solution to a simple problem: Filter on X-Trigger headers in Gmail

I have a very simple problem. My Gmail is receiving a mail with an X-Trigger header and I need to filter these messages (mark them as Archived, as Read an label them into the “filtered” category).

Here is a sample:

$ cat t
X-Trigger: test
Subject: a test
From: kris@koehntopp.de (Kristian Koehntopp)
To: kristian.koehntopp@booking.com

Testing
$ mutt -H t
...

Now, generating filters in Gmail is very easy for various capabilities, but for some reason filters on arbitray header lines are not possible.

Here is what Gmail offers for the trivial cases it supports: Newsletters can be automatically unsubscribed from, using the “Unsubscribe” link shown in the left hand red circle.

If you want to tag the mailing list instead, the “Filter messages like these” from the “More” menu will take care of it. You most likely want to filter on the “list-id” header, if present, or on the “from” header line.

A full list of supported search operators can be found in the Gmail Documentation. But all that search goodness is for supported headers and the bodies of mail, not arbitrary header lines.

Time for some overkill: here is script.google.com. It’s a thing where you can use Javascript to automate any of G.suite, Gmail and a bunch of other Google Cloud offerings.

So this looks vaguely cool, but what can I do here? Tutorial Time. The default script will create a Google Docs document in your Gdrive, and then mail you the link.

function createAndSendDocument() {
  var doc = DocumentApp.create('Hello, world!');
  doc.getBody().appendParagraph('This document was created by Google Apps Script.');
  var url = doc.getUrl();
 
  var email = Session.getActiveUser().getEmail();
  var subject = doc.getName();
  var body = 'Link to your doc: ' + url;
 
  GmailApp.sendEmail(email, subject, body);
}

So, can we talk to Gmail? Turns out, we can, there is documentation.

So let’s do that:

function filterByHeader(search, hdr_regex, labelname) {
  var label = GmailApp.getUserLabelByName(labelname);
  var cnt   = 0;
 
  var threads = GmailApp.search(search);
 
  for (var j=0; j<threads.length; j++) {
    thd  = threads[j];
    msgs = thd.getMessages();
 
    for (var i=0; i< msgs.length; i++) { 
      var m = msgs[i]; 
      var raw = m.getRawContent(); 
      var headers = raw.substring(0, raw.indexOf("\r\n\r\n")); 
 
      if (headers.search(hdr_regex) >= 0) {
        cnt++;
        thd.markUnimportant();
        thd.markRead();
        thd.moveToArchive();
        thd.addLabel(label);
      }
    }
  }
 
  return "We had "+cnt+" matches.";
}

With filterByHeader(“is:unread label:inbox newer_than:1d”, /^X-Trigger:/m, “filtered”) we are getting precisely what we want.

Now, we need to let that run every time a mail is coming in. How do we run code, besides pressing [>] in the editor?

If we add a doGet(e) function to our script, it will be running after we are publishing it as a web app.

So:

function doGet(e) {
  var search    = "is:unread label:inbox newer_than:1d";
  var hdr_regex = /^X-Trigger:/m;
  var labelname = "kris-test";
  var url = filterByHeader(search, hdr_regex, labelname);
 
  return ContentService.createTextOutput(url);
}

Now, how do we run Webhooks on mail receipt?

Well, in theory, we can go to the Google Cloud Console, and enable PubSub, and then have a watch request post a message to a topic of our choice. The consumer/client for that topic is then a push client pointing to our webhook.

In practice, generating and posting these events works, and calling the webhook manually works, but when I want to set up the webhook in PubSub, I get an INVALID ARGUMENT.

I get no details to debug on, so at the moment I am stuck.

See also: https://www.youtube.com/watch?v=ICP1-w593NE#t=26s

Published inComputer ScienceErklärbär

4 Comments

  1. Christoph

    Wouldn’t it be easier to use procmail or just set up your own imap server? Note that is is not a statement on the usability of procmail or the easiness of running an imap server.

    • kris kris

      IMAP and Google Mail do not play well together, conceptually. Google Mail has tags, a single mail can have many of them. These do not map well to folders, where a mail can only be in a single folder at a time.

      Procmail assumes local delivery, which is not the case. It is not even applicable to the problem.

      • Christoph

        Yes, exactly. I prefer the pain of running an IMAP server over being stuck with a web interface, the mobile mail app from hell and mail filtering like it was 1990.

        And you could run an IMAP server on top of the user’s maildirs, combining the worst of local delivery (with procmail!) and running some server. I’m not saying you’d even want to try to do that. Been there, done that, bought a spade.

Leave a Reply

Your email address will not be published. Required fields are marked *