Measuring Inbox/Spam placement

One common use case for syncing an IMAP mailbox with IMAP API is to measure if emails are going to INBOX (or in Gmail's case, to promotions) or to the Spam folder.

IMAP API is a self hosted application to access IMAP accounts over a REST API. IMAP API also sends webhooks about any changes on the account.

One common use case for syncing an IMAP mailbox with IMAP API is to measure if emails are going to INBOX (or in Gmail's case, to promotions) or to the Spam folder. This way we can send emails to that account and see what the email service provider thinks of us.

Such an analysis is a bit different from normal syncing. Usually there is no noticeable delay when an email is put into one of these target folders and we would like to get the results immediately as well. It is simpler with INBOX placement as IMAP API is already following that folder but to detect new messages in the Spam folder IMAP API would have to perform periodic polling checks which always happen with a noticeable delay.

The solution is to add multiple IMAP API checks, one for each target folder.

At first we should check which kind of folders there even are on the account. We can use /v1/verifyAccount endpoint with this by setting mailboxes option to true

curl -XPOST "http://127.0.0.1:3000/v1/verifyAccount" -H "content-type: application/json" -d '{
    "imap": {
        "host": "imap.ethereal.email",
        "port": 993,
        "secure": true,
        "auth": {
            "user": "pansy.watsica@ethereal.email",
            "pass": "h1dW7F8GVfyfjn8ENR"
        }
    },
    "mailboxes": true
}'

If IMAP settings check out then we should get a success result together with a folder listing

{
  "imap": {
    "success": true
  },
  "mailboxes": [
    {
      "path": "INBOX",
      "delimiter": "/",
      "listed": true,
      "specialUse": "\\Inbox",
      "name": "INBOX",
      "subscribed": true
    },
    ...
    {
      "path": "Junk",
      "delimiter": "/",
      "listed": true,
      "name": "Junk",
      "specialUse": "\\Junk",
      "subscribed": true
    },
    ...
  ]
}

So now we know that there are folders INBOX and "Junk" that interests us.

Next we should register 2 IMAP API accounts, one for each folder. Normally IMAP API syncs all the folders on an account but in this case we are only interested in the contents of these specific folders and want to ignore everything else.

When registering the account we should use the path property – this ensures that IMAP API is syncing data only from that specific path and does not touch other folders.

First we register an account for the INBOX folder:

curl -XPOST "http://127.0.0.1:3000/v1/account" -H "content-type: application/json" -d '{
    "account": "pansy_inbox",
    "name": "Pansy Watsica INBOX",
    "path": "INBOX",
    "imap": {
        "host": "imap.ethereal.email",
        "port": 993,
        "secure": true,
        "auth": {
            "user": "pansy.watsica@ethereal.email",
            "pass": "h1dW7F8GVfyfjn8ENR"
        }
    }
}'

and then the same but for the Spam folder. Also be aware that even though it's the same IMAP account we have to use a different account id for IMAP API, otherwise we would be only modifying the already existing INBOX handling account.

curl -XPOST "http://127.0.0.1:3000/v1/account" -H "content-type: application/json" -d '{
    "account": "pansy_spam",
    "name": "Pansy Watsica SPAM",
    "path": "Junk",
    "imap": {
        "host": "imap.ethereal.email",
        "port": 993,
        "secure": true,
        "auth": {
            "user": "pansy.watsica@ethereal.email",
            "pass": "h1dW7F8GVfyfjn8ENR"
        }
    }
}'

Now whenever a message lands in INBOX or Spam folder we immediately get a webhook notification. If the email was sent by us and the webhook notification is from the account "pansy_inbox" then the email ended up in INBOX, if it's from "pansy_spam" then we failed and were deemed as a spammer.

It's a bit more complicated in case of Gmail if we want to get the promotion tab info as well and not just the INBOX info. In this case we can register an account not for the INBOX folder but for the All Mail folder and when receiving notifications about new emails we should check the Gmail Labels property to determine the actual placement of that email.