Merge branch 'mail_rework' into dev

Pipfile

@@ -29,6 +29,7 @@ watchdog = "*"
 pathvalidate = "*"
 django-q = "*"
 redis = "*"
+imap-tools = "*"
 
 [dev-packages]
 coveralls = "*"

Pipfile.lock

@@ -1,7 +1,7 @@
 {
     "_meta": {
         "hash": {
-            "sha256": "c0dfeedbac2e9b705267336349e6f72ba650ff9184affae06046db32299e2c87"
+            "sha256": "d6416e6844126b09200b9839a3abdcf3c24ef5cf70052b8f134d8bc804552c17"
         },
         "pipfile-spec": 6,
         "requires": {},
@@ -123,6 +123,14 @@
             "index": "pypi",
             "version": "==20.0.4"
         },
+        "imap-tools": {
+            "hashes": [
+                "sha256:070929b8ec429c0aad94588a37a2962eed656a119ab61dcf91489f20fe983f5d",
+                "sha256:6232cd43748741496446871e889eb137351fc7a7e7f4c7888cd8c0fa28e20cda"
+            ],
+            "index": "pypi",
+            "version": "==0.31.0"
+        },
         "joblib": {
             "hashes": [
                 "sha256:698c311779f347cf6b7e6b8a39bb682277b8ee4aba8cf9507bc0cf4cd4737b72",

docs/changelog.rst

@@ -36,6 +36,12 @@ paperless-ng 0.9.0
   multi user solution, however, it allows more than one user to access the website
   and set some basic permissions / renew passwords.
 
+* **Modified [breaking]:** All new mail consumer with customizable filters, actions and
+  multiple account support. Replaces the old mail consumer. The new mail consumer
+  needs different configuration but can be configured to act exactly like the old
+  consumer.
+
+
 * **Modified:** Changes to the consumer:
 
   * Now uses the excellent watchdog library that should make sure files are

docs/screenshots.rst

@@ -36,6 +36,10 @@ The old admin is still there and accessible!
 
 .. image:: _static/paperless-9-admin.png
 
+Fancy mail filters!
+
+.. image:: _static/paperless-11-mail-filters.png
+
 Mobile support in the future? This doesn't really work yet.
 
 .. image:: _static/paperless-10-mobile.png

docs/usage_overview.rst

@@ -86,49 +86,48 @@ files from the scanner.  Typically, you're looking at an FTP server like
 IMAP (Email)
 ============
 
-Another handy way to get documents into your database is to email them to
+You can tell paperless-ng to consume documents from your email accounts.
-yourself.  The typical use-case would be to be out for lunch and want to send a
+This is a very flexible and powerful feature, if you regularly received documents
-copy of the receipt back to your system at home.  Paperless can be taught to
+via mail that you need to archive. The mail consumer can be configured by using the
-pull emails down from an arbitrary account and dump them into the consumption
+admin interface in the following manner:
-directory where the consumer will follow the
-usual pattern on consuming the document.
 
-.. hint::
+1.  Define e-mail accounts.
+2.  Define mail rules for your account.
 
-    It's disabled by default. By setting the values below it will be enabled.
+These rules perform the following:
 
-    It's been tested in a limited environment, so it may not work for you (please
+1.  Connect to the mail server.
-    submit a pull request if you can!)
+2.  Fetch all matching mails (as defined by folder, maximum age and the filters)
+3.  Check if there are any consumable attachments.
+4.  If so, instruct paperless to consume the attachments and optionally
+    use the metadata provided in the rule for the new document.
+5.  If documents were consumed from a mail, the rule action is performed
+    on that mail.
 
-.. danger::
+Paperless will completely ignore mails that do not match your filters. It will also
+only perform the action on mails that it has consumed documents from.
 
-    It's designed to **delete mail from the server once consumed**.  So don't go
+The actions all ensure that the same mail is not consumed twice by different means.
-    pointing this to your personal email account and wonder where all your stuff
+These are as follows:
-    went.
 
-.. hint::
+*   **Delete:** Immediately deletes mail that paperless has consumed documents from.
+    Use with caution.
+*   **Mark as read:** Mark consumed mail as read. Paperless will not consume documents
+    from already read mails. If you read a mail before paperless sees it, it will be
+    ignored.
+*   **Flag:** Sets the 'important' flag on mails with consumed documents. Paperless
+    will not consume flagged mails.
+*   **Move to folder:** Moves consumed mails out of the way so that paperless wont
+    consume them again.
 
-    Currently, only one photo (attachment) per email will work.
+.. caution::
 
-So, with all that in mind, here's what you do to get it running:
+    The mail consumer will perform these actions on all mails it has consumed
+    documents from. Keep in mind that the actual consumption process may fail
+    for some reason, leaving you with missing documents in paperless.
 
-1. Setup a new email account somewhere, or if you're feeling daring, create a
+Paperless is set up to check your mails every 10 minutes. this can be configured on the
-   folder in an existing email box and note the path to that folder.
+'Scheduled tasks' page in the admin.
-2. In ``/etc/paperless.conf`` set all of the appropriate values in
-   ``PATHS AND FOLDERS`` and ``SECURITY``.
-   If you decided to use a subfolder of an existing account, then make sure you
-   set ``PAPERLESS_CONSUME_MAIL_INBOX`` accordingly here.  You also have to set
-   the ``PAPERLESS_EMAIL_SECRET`` to something you can remember 'cause you'll
-   have to include that in every email you send.
-3. Restart paperless.  Paperless will check
-   the configured email account at startup and from then on every 10 minutes
-   for something new and pulls down whatever it finds.
-4. Send yourself an email!  Note that the subject is treated as the file name,
-   so if you set the subject to ``Correspondent - Title - tag,tag,tag``, you'll
-   get what you expect.  Also, you must include the aforementioned secret
-   string in every email so the fetcher knows that it's safe to import.
-   Note that Paperless only allows the email title to consist of safe characters
-   to be imported. These consist of alpha-numeric characters and ``-_ ,.'``.
 
 
 REST API

paperless.conf.example

@@ -59,22 +59,6 @@ PAPERLESS_CONSUMPTION_DIR="../consume"
 #PAPERLESS_STATIC_URL="/static/"
 
 
-# These values are required if you want paperless to check a particular email
-# box every 10 minutes and attempt to consume documents from there.  If you
-# don't define a HOST, mail checking will just be disabled.
-#PAPERLESS_CONSUME_MAIL_HOST=""
-#PAPERLESS_CONSUME_MAIL_PORT=""
-#PAPERLESS_CONSUME_MAIL_USER=""
-#PAPERLESS_CONSUME_MAIL_PASS=""
-
-# Override the default IMAP inbox here. If not set Paperless defaults to
-# "INBOX".
-#PAPERLESS_CONSUME_MAIL_INBOX="INBOX"
-
-# Any email sent to the target account that does not contain this text will be
-# ignored.
-PAPERLESS_EMAIL_SECRET=""
-
 # Specify a filename format for the document (directories are supported)
 # Use the following placeholders:
 # * {correspondent}

src/documents/mail.py

@@ -1,249 +0,0 @@
-import datetime
-import imaplib
-import logging
-import os
-import re
-import time
-import uuid
-from base64 import b64decode
-from email import policy
-from email.parser import BytesParser
-
-from dateutil import parser
-from django.conf import settings
-
-from .models import Correspondent
-
-
-class MailFetcherError(Exception):
-    pass
-
-
-class InvalidMessageError(MailFetcherError):
-    pass
-
-
-class Loggable(object):
-
-    def __init__(self, group=None):
-        self.logger = logging.getLogger(__name__)
-        self.logging_group = group or uuid.uuid4()
-
-    def log(self, level, message):
-        getattr(self.logger, level)(message, extra={
-            "group": self.logging_group
-        })
-
-
-class Message(Loggable):
-    """
-    A crude, but simple email message class.  We assume that there's a subject
-    and n attachments, and that we don't care about the message body.
-    """
-
-    SECRET = os.getenv("PAPERLESS_EMAIL_SECRET")
-
-    def __init__(self, data, group=None):
-        """
-        Cribbed heavily from
-        https://www.ianlewis.org/en/parsing-email-attachments-python
-        """
-
-        Loggable.__init__(self, group=group)
-
-        self.subject = None
-        self.time = None
-        self.attachment = None
-
-        message = BytesParser(policy=policy.default).parsebytes(data)
-        self.subject = str(message["Subject"]).replace("\r\n", "")
-        self.body = str(message.get_body())
-
-        self.check_subject()
-        self.check_body()
-
-        self._set_time(message)
-
-        self.log("info", 'Importing email: "{}"'.format(self.subject))
-
-        attachments = []
-        for part in message.walk():
-
-            content_disposition = part.get("Content-Disposition")
-            if not content_disposition:
-                continue
-
-            dispositions = content_disposition.strip().split(";")
-            if len(dispositions) < 2:
-                continue
-
-            if not dispositions[0].lower() == "attachment" and \
-               "filename" not in dispositions[1].lower():
-                continue
-
-            file_data = part.get_payload()
-
-            attachments.append(Attachment(
-                b64decode(file_data), content_type=part.get_content_type()))
-
-        if len(attachments) == 0:
-            raise InvalidMessageError(
-                "There don't appear to be any attachments to this message")
-
-        if len(attachments) > 1:
-            raise InvalidMessageError(
-                "There's more than one attachment to this message. It cannot "
-                "be indexed automatically."
-            )
-
-        self.attachment = attachments[0]
-
-    def __bool__(self):
-        return bool(self.attachment)
-
-    def check_subject(self):
-        if self.subject is None:
-            raise InvalidMessageError("Message does not have a subject")
-        if not Correspondent.SAFE_REGEX.match(self.subject):
-            raise InvalidMessageError("Message subject is unsafe: {}".format(
-                self.subject))
-
-    def check_body(self):
-        if self.SECRET not in self.body:
-            raise InvalidMessageError("The secret wasn't in the body")
-
-    def _set_time(self, message):
-        self.time = datetime.datetime.now()
-        message_time = message.get("Date")
-        if message_time:
-            try:
-                self.time = parser.parse(message_time)
-            except (ValueError, AttributeError):
-                pass  # We assume that "now" is ok
-
-    @property
-    def file_name(self):
-        return "{}.{}".format(self.subject, self.attachment.suffix)
-
-
-class Attachment(object):
-
-    SAFE_SUFFIX_REGEX = re.compile(
-        r"^(application/(pdf))|(image/(png|jpeg|gif|tiff))$")
-
-    def __init__(self, data, content_type):
-
-        self.content_type = content_type
-        self.data = data
-        self.suffix = None
-
-        m = self.SAFE_SUFFIX_REGEX.match(self.content_type)
-        if not m:
-            raise MailFetcherError(
-                "Not-awesome file type: {}".format(self.content_type))
-        self.suffix = m.group(2) or m.group(4)
-
-    def read(self):
-        return self.data
-
-
-class MailFetcher(Loggable):
-
-    def __init__(self, consume=settings.CONSUMPTION_DIR):
-
-        Loggable.__init__(self)
-
-        self._connection = None
-        self._host = os.getenv("PAPERLESS_CONSUME_MAIL_HOST")
-        self._port = os.getenv("PAPERLESS_CONSUME_MAIL_PORT")
-        self._username = os.getenv("PAPERLESS_CONSUME_MAIL_USER")
-        self._password = os.getenv("PAPERLESS_CONSUME_MAIL_PASS")
-        self._inbox = os.getenv("PAPERLESS_CONSUME_MAIL_INBOX", "INBOX")
-
-        self._enabled = bool(self._host)
-        if self._enabled and Message.SECRET is None:
-            raise MailFetcherError("No PAPERLESS_EMAIL_SECRET defined")
-
-        self.last_checked = time.time()
-        self.consume = consume
-
-    def pull(self):
-        """
-        Fetch all available mail at the target address and store it locally in
-        the consumption directory so that the file consumer can pick it up and
-        do its thing.
-        """
-
-        if self._enabled:
-
-            # Reset the grouping id for each fetch
-            self.logging_group = uuid.uuid4()
-
-            self.log("debug", "Checking mail")
-
-            for message in self._get_messages():
-
-                self.log("info", 'Storing email: "{}"'.format(message.subject))
-
-                t = int(time.mktime(message.time.timetuple()))
-                file_name = os.path.join(self.consume, message.file_name)
-                with open(file_name, "wb") as f:
-                    f.write(message.attachment.data)
-                    os.utime(file_name, times=(t, t))
-
-        self.last_checked = time.time()
-
-    def _get_messages(self):
-
-        r = []
-        try:
-
-            self._connect()
-            self._login()
-
-            for message in self._fetch():
-                if message:
-                    r.append(message)
-
-            self._connection.expunge()
-            self._connection.close()
-            self._connection.logout()
-
-        except MailFetcherError as e:
-            self.log("error", str(e))
-
-        return r
-
-    def _connect(self):
-        try:
-            self._connection = imaplib.IMAP4_SSL(self._host, self._port)
-        except OSError as e:
-            msg = "Problem connecting to {}: {}".format(self._host, e.strerror)
-            raise MailFetcherError(msg)
-
-    def _login(self):
-
-        login = self._connection.login(self._username, self._password)
-        if not login[0] == "OK":
-            raise MailFetcherError("Can't log into mail: {}".format(login[1]))
-
-        inbox = self._connection.select(self._inbox)
-        if not inbox[0] == "OK":
-            raise MailFetcherError("Can't find the inbox: {}".format(inbox[1]))
-
-    def _fetch(self):
-
-        for num in self._connection.search(None, "ALL")[1][0].split():
-
-            __, data = self._connection.fetch(num, "(RFC822)")
-
-            message = None
-            try:
-                message = Message(data[0][1], self.logging_group)
-            except InvalidMessageError as e:
-                self.log("error", str(e))
-            else:
-                self._connection.store(num, "+FLAGS", "\\Deleted")
-
-            if message:
-                yield message

src/documents/management/commands/document_consumer.py

@@ -34,7 +34,7 @@ class Handler(FileSystemEventHandler):
 class Command(BaseCommand):
     """
     On every iteration of an infinite loop, consume what we can from the
-    consumption directory, and fetch any mail available.
+    consumption directory.
     """
 
     def __init__(self, *args, **kwargs):

src/documents/migrations/1001_auto_20201109_1636.py

@@ -9,13 +9,11 @@ from django_q.tasks import schedule
 def add_schedules(apps, schema_editor):
     schedule('documents.tasks.train_classifier', name="Train the classifier", schedule_type=Schedule.HOURLY)
     schedule('documents.tasks.index_optimize', name="Optimize the index", schedule_type=Schedule.DAILY)
-    schedule('documents.tasks.consume_mail', name="Check E-Mail", schedule_type=Schedule.MINUTES, minutes=10)
 
 
 def remove_schedules(apps, schema_editor):
     Schedule.objects.filter(func='documents.tasks.train_classifier').delete()
     Schedule.objects.filter(func='documents.tasks.index_optimize').delete()
-    Schedule.objects.filter(func='documents.tasks.consume_mail').delete()
 
 
 class Migration(migrations.Migration):

src/documents/tasks.py

@@ -7,14 +7,9 @@ from documents import index
 from documents.classifier import DocumentClassifier, \
     IncompatibleClassifierVersionError
 from documents.consumer import Consumer, ConsumerError
-from documents.mail import MailFetcher
 from documents.models import Document
 
 
-def consume_mail():
-    MailFetcher().pull()
-
-
 def index_optimize():
     index.open_index().optimize()
 

src/documents/tests/samples/mail.txt

@@ -1,208 +0,0 @@
-Return-Path: <sender@example.com>
-X-Original-To: sender@mailbox4.mailhost.com
-Delivered-To: sender@mailbox4.mailhost.com
-Received: from mx8.mailhost.com (mail8.mailhost.com [75.126.24.68])
-	by mailbox4.mailhost.com (Postfix) with ESMTP id B62BD5498001
-	for <sender@mailbox4.mailhost.com>; Thu,  4 Feb 2016 22:01:17 +0000 (UTC)
-Received: from localhost (localhost.localdomain [127.0.0.1])
-	by mx8.mailhost.com (Postfix) with ESMTP id B41796F190D
-	for <sender@mailbox4.mailhost.com>; Thu,  4 Feb 2016 22:01:17 +0000 (UTC)
-X-Spam-Flag: NO
-X-Spam-Score: 0
-X-Spam-Level: 
-X-Spam-Status: No, score=0 tagged_above=-999 required=3
-	tests=[RCVD_IN_DNSWL_NONE=-0.0001]
-Received: from mx8.mailhost.com ([127.0.0.1])
-	by localhost (mail8.mailhost.com [127.0.0.1]) (amavisd-new, port 10024)
-	with ESMTP id 3cj6d28FXsS3 for <sender@mailbox4.mailhost.com>;
-	Thu,  4 Feb 2016 22:01:17 +0000 (UTC)
-Received: from smtp.mailhost.com (smtp.mailhost.com [74.55.86.74])
-	by mx8.mailhost.com (Postfix) with ESMTP id 527D76F1529
-	for <paperless@example.com>; Thu,  4 Feb 2016 22:01:17 +0000 (UTC)
-Received: from [10.114.0.19] (nl3x.mullvad.net [46.166.136.162])
-	by smtp.mailhost.com (Postfix) with ESMTP id 9C52420C6FDA
-	for <paperless@example.com>; Thu,  4 Feb 2016 22:01:16 +0000 (UTC)
-To: paperless@example.com
-From: Daniel Quinn <sender@example.com>
-Subject: Test 0
-Message-ID: <56B3CA2A.6030806@example.com>
-Date: Thu, 4 Feb 2016 22:01:14 +0000
-User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
- Thunderbird/38.5.0
-MIME-Version: 1.0
-Content-Type: multipart/mixed;
- boundary="------------090701020702030809070008"
-
-This is a multi-part message in MIME format.
---------------090701020702030809070008
-Content-Type: text/plain; charset=utf-8
-Content-Transfer-Encoding: 7bit
-
-The secret word is "paperless" :-)
-
---------------090701020702030809070008
-Content-Type: application/pdf;
- name="test0.pdf"
-Content-Transfer-Encoding: base64
-Content-Disposition: attachment;
- filename="test0.pdf"
-
-JVBERi0xLjQKJcOkw7zDtsOfCjIgMCBvYmoKPDwvTGVuZ3RoIDMgMCBSL0ZpbHRlci9GbGF0
-ZURlY29kZT4+CnN0cmVhbQp4nFWLQQvCMAyF7/kVOQutSdeuHZSA0+3gbVDwIN6c3gR38e/b
-bF4kkPfyvReyjB94IyFVF7pgG0ze4TLDZYevLamzPKEvEFqbMEZfq+WO+5GRHZbHNROLy+So
-UfFi6g7/RyusEpUl9VsQxQTlHR2oV3wUEzOdhOnXG1aw/o1yK2cYCkww4RdbUCevCmVuZHN0
-cmVhbQplbmRvYmoKCjMgMCBvYmoKMTM5CmVuZG9iagoKNSAwIG9iago8PC9MZW5ndGggNiAw
-IFIvRmlsdGVyL0ZsYXRlRGVjb2RlL0xlbmd0aDEgMTA4MjQ+PgpzdHJlYW0KeJzlOWt0G9WZ
-95uRbNmWLckPWY4SaRTFedmybI8T4rw8sS3ZiZ1YfqWSCbFkS7YEtiQkJSE8GlNeOQ5pUmh5
-Zkt2l+XQNl3GhLaBpcWw0D19UGALLRRS0gM9nD0lxVBK9wCx97tXI0UJAc727L8d+c587/u9
-7p0rOZXYEyJaMkV4Io1OBuLOqmqBEPJLQqB0dG9K2NRTsQHhM4Rw/zkWH5+870e7PiRE9Rgh
-+Y+NT+wf+/b3e4YI0YYJKX41HAoEfxj6vUjIIgltrA0jYef8/nzEr0F8WXgydY2bP7QO8WOI
-SxOx0cDxxbUmxN9AfOlk4Jr4apWLI8SMKBGigcmQpYXrRBx9KtobjyVTQbJsgZDl91B+PBGK
-d9838hzipwjhjyIN8EMvLYJ5FOd4lTovX1NQWKQtLtGR/3eX+jCpIJ3qTURH4ux+wcWfIFXk
-XkIW3qXY+ft898LH/5deaNKPe8hD5DFymLxGrlAYbuIhEbIHKbnX0+QlpNLLQ4bId8n055g9
-QU4hPy3nJ0doJJe8PORucpL8xwWzeMgkuQ59+QF5DRrIz7BVYuQD0JAbyXNo9QOkbb+UKa4E
-b2MMHMuhvk7u5w6RbdzbiNxLOZyT05NnyTHYjZZTGOfhbMQbP2P0NnID3vtJmOxFmF3qTZ/+
-jhQs/AWjuoFsI18jW8hEjsaT8ABfiPUbIA9gTp9mNGeGmd/JX8n9kOPO3YnIN8g4jgBg7Nxh
-fsvnZOh/ffGDpBhW8dWk4FJcrono5j/mGhc+5JeRQjK4MJehLXQt/IUPzEdVw6rF6k2qX3zR
-HHnfUE2iNln44/x180H1DvVDWK2HcePouHzI5x0c6O/r9fTs2N7dtW1rZ4fb1d7WukVq2bxp
-44b1zesuW7umod5Z56hduWJ59TL7UpvVVG7Q60qKiwoLNPl5ahXPAakVZPC7ZL5aMLgDdpc9
-0OmoFVymcLuj1mV3+2UhIMj4UC23d3Yykj0gC35BXo6PQA7ZL0soOXaRpJSWlLKSoBc2ko10
-CrsgP99uF07BUK8X4cPtdp8gn2XwdgarljOkGBGbDTWYV9RbwSW794anXX70EWaKCtvsbaFC
-Ry2ZKSxCsAgheaU9PgMrNwMDuJWu9TMc0RTTaTFSVyAoe3q9rnazzeZz1G6VS+ztjEXamEk5
-r03OZyaFCHWdHBJmamenbz+lJyP+Gm3QHgzs8sp8AHWnedf09G2yoUZeZW+XV137tgkjD8m1
-9naXXEOtdvVl5+k6PyXI6mq9XZj+K8Fw7GffvZASUCh51fq/EgrKXJsMfV4bvcxuzPX0tNsu
-uKf904FTC1MjdkFvn57RaqfjLkw38XjRxKmFJw6ZZfftPlnvD8N6nxK6u69LLuu93Ctz1W4h
-HEAK/rXYbevMNkNWxvN5bIJpweRghm02moZDpyQygog81etN4wIZMT9KJGeNT+b8lDOb4VQM
-Us5UhpNV99uxtl393mlZVb01aHdhxg8F5KkR7K4raWHsernkI7PNPl1qEJqdPiYroFdbgxFB
-Vi/HJKFWrgL2DVWZ1jOk5KP046wZJ1huKBWa7WiG2nHZXX7lb2/YhAYETHRnTboRBryy1I6A
-FFAq5pqpd6JGwI8Fi7SzYspOe1wut7dmq0vdckX6vUxFUZPL22TiH1W0ZKeLrSvBNe1vT7tA
-bdl7vY8TceHMTJNgPimSJuJrp8LGNuyy5a5pb3BMtvrNQVx3Y4LXbJMlH1bYZ/eGfLTtMEOr
-zphZc/hYrwx4u/rtXb1D3nWKI2kGNaeqdl1kxu41p81gA8qaao3g5cy8DwX1SBDcCNhbN+Jd
-zq/W4NBjwhmVNm7rRsELZpKRRjfkVYIr1K7IUfwCo2raTm2dGWt5FEU7bZ1mm8+Wvhy1HLIF
-ZWLU0NCkdmZYuE0hQ4P92dbJSDSXJtr0gtcesvvsYUGWPF4aG00Py7KSDJZzpVYDF2A5ycI0
-ERuyMwhNpuyuMecmV+5geBbtvIi9NcMWpjX2rv5patyuGCTo+VaZ0BaW1hnMbC+gC9qOe6+g
-xyXNFvT0jCTRxRxeT43Ytwan7f3ejUwa95MbzNfSuUpJF3QNtDpqcWtrnbHDwd4ZCQ72D3kf
-1+O58OCA91EOuDZ/q29mGfK8jwv40mBUjlIpkSICRailPkQ0TN78uETIFOOqGIHho6eAMJom
-QwMyeopL0/TpiZaziSTCIUeV5kgZaRXSNGnaFKOxa4bQlEmFakkjFUharpgzzwAlPYqUJ/Ac
-WwDkpBaKwTyDWn2MfAqmZgokc1piCiWktIcHB89PPTjkPanFt7OZ3XGiVnphu5jCWGx8rbiE
-IG2U633hab+PLjZixNLgH8hg34xlsm9GR/K0cqE91CoX2VspvYXSW9L0PErPxxYFI6D6FNbe
-IwPtgMu9NlySwqKfmaf1Z2mlfLipTOv/6MCMVeP3hqfxDFoOG6XTpVwRp+ErjFqigQJeoykw
-8AW831fAl3KEG/aR0hYj6IxwxghPGeGIEQ4YYdgISBQY/ao5I7xghOOMFzdCjxGsjJGmy0Z4
-gLFiTE0yQj0TIEZ4k3GnGL2eUTYssHnSakcYo4fx5hhdzsyRVhCYzhwzNMummWJcdM2ZmeOK
-7HV15koo1+6L6J/hUB5pqTEQ0cTuBtHkHN59hWgohcpmg9hQb1tzmcG+VAd2g81gX1EHNWCo
-rIANr4jnrjC3qY61my0/v6bhlTVm1d3lL8GG+edeyi/65CrzGnqgAlKOJ7c/4neCJeQJaT8p
-L68qLikpqCqwWJcs8viWkHJEKqs8Pm1lRRnHqdWGPp9af9wKZ6wwawW9FYgVmhE5aoW4FfxW
-8FhBskK9FQQrWBkbWVMZLrJeZJqyFY7n0HOTk0hckAAldoy6RaSAyNJQCs0Ye/rTUA/l+ZtB
-bDRWYOA0G032pfkKuGKNDdz5nT9qufb6xPxVNzy0+6YD88F9t0Mj/1G4btXGr9927q4qh6OK
-231iybkyCqk5kwMXTg2eT0vV3aQIvy39gzRGtNo8g6HSyBf0+wgPep6vkCpKPb4KndagM3h8
-uorySlBVQvOHlXC0Erh4JfgrwVMJUiXMVoJcCccZKlSCvhJIJcwxCormSl7YIzQFwywL2fKT
-RSb9r7D4LAEGUQk+z750+ZqmtZgA/nzQ10mOWkmqdUiF/zhfdfwWqFG9mcalT9bTOHmhiq7B
-gYV3uV/zz5GVxCc12fLLFxVjS6xaXWzjKystHp+5Us8XeXz5vHFqNcRXg381eFaDsBoeWQ3D
-q6FnNWT8JVgewmpUSrA26QKhg1kPV6wRK41i45omJ9RxzN3KCvuK5faleRXlxkoLz/165vvu
-79Q7GrqueeZeX2hX43eOjt/vXL0m0Tu4fcedQy120Nx+dEnpOze1P3Rt0xJb+6j7+iPW5yed
-nvbmHYsa69p20q8ZpHPhXf5q/mlixt1lUmoxaKqrVYJWW6Xi8di/tHBpr89UYTAsxooZrAZO
-yxsMRFNozFdhjBWkwuMj+qkVMLwCpBWAwBVYBEw+MbEhljY708knzawn0yvQoESp9N8KDNbQ
-tBlaYE3TcrYu16yF/BKoKBcb114GL933jT3z82WJmfe3Hr/ncMe2YP/Sdf8E5KZbh4+0jzby
-T3/1a+duqXLsToBp93VbeNWdgV3OPc/b5y0q9e6obDWxNYs1c6huJEbSIa0oLCnJL+P5SpNK
-W6T1+Aryi3S4pg29PmJ8wASyCVpM4DTRMiUybSSKivfNpc2NjbSH1NhABvuaFhArxAq7oRzr
-dFlFCcAO//B1N4RafvvbDfXr++03lyfGuTsdK155ZeDcgS2t+i0mK8u5B3Puxh6qIIvJYWmo
-CkC3SFOhq1hiqSKY6CprFSa6qkpbWmr0+Er1WnWvT2uctYBsgeMWOGqBKQvELeC3gMcCxAKb
-8SFZoN4CggX0FphjciiU2R2yO+MVSnFoRUzOzMJINx5bGxXlFqBpx2CwBQ3YdYKhArDlbE3L
-QbXpwPjab9bX/8vO13/xq6cgMn93OAZ37ILXSqfv9ZQWrbPWvQvqjz6YH+uDYw8/ePJeGus2
-jPUd3C/LcMecknrKVUWkqkqv0lusZXqPrwz3A4yY5GOD5eurUIGr7PVxRtwGO3J3RsI2wSlG
-SQN+RldWvxLk+Z0v04HnNz4WXnWeXTA0leJKWr4JcNHT9gNWPMNyu8D9+uq75w/87uWJWN63
-oT01/9/z1qmbrx7yJeY/dQ/BH/4GUGm75UOT4+PHqxzw/E/+bQX3joHVcwfG+CjWsxA77Anp
-RoO6iKhJpUlT4vFp9Fy5BwMSTEBMcMYEHhPUm0BvgjmGvmiCWdZ1x01w1ARTJoibwG8CyQRp
-lQ0PMJKHkeoZVc8YufrHmWZaDe9XfO6bMbtdZpdpNkFYfL0tsy/mNyn7DPYC/+h858uvvvrG
-b3732FdvvWnPvhtvnoLX5w3z7//507/95dVnnjjz1o+fTb8baR52YB6MxC9txCwY1UbMgg7f
-hhq9sZwv7/XxRvR8c24kcyyGdABIf8QEw3TxZd3fnd3MxVxfq7E/BQPbFA10UxTSa5Df0XBi
-aP6y/3rttuOX1fSn5j/85+/dMdG8bBW8/6dz1vmPH3LOh1/+gY36akZfT/Mn0NdvScOktFil
-KigtqDSpy4xl2IpGnQqPpX2+Yr1RW4D+Vxxn2Z7NJL/5TE49CCtgtm5yJpw0RTBBbtpzX9NE
-eUUrj5yXNH0H0K5UenQFXY1VtGOh+fj1E18Hcd/8nzUdT7TMXQMW0J6wcu9UOT69r8rRvaIZ
-yrkxfFPRGPGdnFeF9WiAR6UFgzZv8WIbWbnS4bBpebGxoc7ja9CttC02aB01Do/PqqupqMrL
-Kygo7/MV6FfgMYev7vPx+r0i7BRhrQjLRDCKkCfCRyK8LcLLIvxUhAdFuEuEERHAI0K7CPVM
-rlwElQjhuYzgYyKkRJBEaGJs5H0owusizIogMxs3ixAUFRNpGX1G7EURnhXheyIcZWJXibBB
-BCEzx7r0BMdF8IswkJmjnGm+zTS/KcIUTi/V5PDNTPdt5gAnM4E4mx5n1YmgUdbL8BcfMy88
-heYcxM6r5wjlbE6Z45lyPsuc0CqzJzTWAOyEVknvVZA9ppVw+edPbcsvOrZ1PSy59izZ/kL7
-3P75wduPL3K5WioMh+dbDw0Oem86PL9z3z4o4/0165uaa1rn/6Qc5LwnNIXFqrVbMmi/b8m5
-quyBh/WRE5vhD9hHi8msdAMpKzMVabX5pvwllsV40l2sK0PEaPL4Co0VpbRt9LRtHrTA2xZ4
-1gL4QlFZoBmRb1ogZYGgBQYs0G6BJgsss4CZsfHNxuW+1/Bt9qIFsq+8LD03o8N/18n3wnPv
-RRls3/6v69Pn3t7BITz4Xnn11aDl/bXN2WOvt39YOfcq58HbFt6C/eQVPPeapCKSl6ct5gvu
-v5wvIy3KmRP3qpwDJ+x3NTW53KLo3tXQ2dkgut3s/y30Pzblq28Z1m38K2dN/9b/yzuXdJ7/
-JXfhrbwqNf0FXJMloV6+bd5FvpJLueDS5zXjN8a3SLWKkHKumdTwS8gAR397Pkw6ES/Hpwd5
-23DsQHgHPs2oU4NPJ0eUX9KfgR3wDLcaP8e4t/kh/pcqj+ohtSlvY97P895VZtWTRhoDi0SP
-/bILgX/nf0p4xrVANOvbzqyfgJI7FZgj+WRMgXk8i04qsAplDiqwmpSQexQ4j+jIQwqcT64l
-P1BgDX43dipwASmBNgUuhCj0KnARWcw9lf0vVx33ugIXkzV8gQKXkEX8Zuq9iv46f4L3KjAQ
-QaVSYI6UqJYpME/WqhoVWIUyYQVWk8WqgwqcRyyqBxU4n3yoekaBNWSl+ocKXEAWq3+vwIXc
-G+qPFbiIrNP8RoG1ZFdBiQIXkysLrlTgEtJU8HJ7ZDySilwbCgrBQCogjMbi+xOR8XBKWDm6
-Smisb6gXOmKx8YmQ0BZLxGOJQCoSi9YVtl0s1ij0oYnOQKpW2BodreuOjITSskJ/KBEZ6wuN
-75kIJLYkR0PRYCghOISLJS7Gd4YSSYo01tXX1zWc514sHEkKASGVCARDk4HEVUJs7EJHhERo
-PJJMhRJIjESFwbr+OsETSIWiKSEQDQoDWcWesbHIaIgRR0OJVACFY6kwunrlnkQkGYyM0tmS
-ddkIctLRnwrtDQnbA6lUKBmLtgaSOBd6NhCJxpK1wr5wZDQs7AskhWAoGRmPInNkv3ChjoDc
-AMYSjcb2osm9oVr0eywRSoYj0XEhSUNWtIVUOJCiQU+GUonIaGBiYj/WbDKOWiNYpH2RVBgn
-ngwlhR2hfUJfbDIQ/W5d2hXMzRgmVYhMxhOxvcxHR3I0EQpFcbJAMDASmYik0Fo4kAiMYsYw
-bZHRJMsIJkKIB6IO155ELB5CT7/S0X1eEB1MZzMZm9iLM1PpaCgUpDOi23tDE6iEE0/EYlfR
-eMZiCXQ0mAo7cjwfi0VTqBoTAsEgBo7Zio3umaR1wjSnMs4FRhMx5MUnAim0MpmsC6dS8fVO
-5759++oCSmlGsTJ1aNn5RbzU/nhIqUeCWpmc6MbyR2np9rD60iD6t3YLPXHMjxudExSBWiHT
-mg11DcoUmMZIPJWsS0Ym6mKJcWePu5u0kwgZx5HCcS0JkSARcAQQDyA0SmIkTvaTBJMKI1Ug
-K5G6Cp+NpJ404BBIB0rFkD+B+gJpQziBWvQeYHZjJErq8FtE25daa0SoT/Gik2nXIrQV9UfR
-QjfqjSA3165A+hklgvss1Rwne9CPAFK2kCRqhVAmyCQE4sDxZTa+jL+TQckspxH9qsdPHXp/
-Kd0vsxxBWwLLdYpxqK+TzP+rkBZDvS/KiIByIVa/JHJCDAsyq9T2IEr0MykP06S5SLHZokxq
-4BIz9uCMY6g/ymqZkRxltmlPpC3HEA4rWb0SM55gHgSZXia2JM782Rpcujv6mXd72ZzbGZ3i
-ScZrRTypxJXO2QDzIoZUmot96AmdN8zgAMtnkGnTLosqmiPYd8IXziMougGlLlE2x17FS6pT
-q+R7jN2TbN4oziEw/9JVvnBugeUpwLKervQkclNMdhTpE/jZr6yzScxKeq4RZSXtY+syrEQ8
-yewKZAc+97GuiLG6RW1LWY3PZyXdN2NKpwpMN45wjEWRyaOD1YZGEmKeUijA1v4IakywudO+
-hVl3BFhtQ0qtUyyCTL6CSqTU6zijOIiL9QVd8SElp1/BnaL7khbTGcztTVqTCeZvMsd2lHkb
-zMaYzjaVmlBmSkc8wXakq7L1GWP9ls5okFlzfE7Ox1huUsqsMeZRED/piqd7K4a6e1g90usp
-3c2pz2QuwPIbU/TibF9KKb5MsvURZh0YJ+vxbOlE7+injvVh7qoZVdZMneKz8+/Wo37FWQZz
-10ci68sk+titrP5odtXtyVm/mUr04x7UzfaLuNI/biVzwkUW6Kq5eNdsYPvlhVGkuzGCeIr5
-k2S5rGMxjCO/B2foZufo9DcHG/p0iWumwLNlBEIEIAzjpIxYwU92wDAZhC1kE0j4lJDXis82
-xOmzDjaRKZTbhPTNiG9E+gbcPK14b8HRg+MIDhWOtEQ9Sjjx6VRwB+K1qPEC3oENSm1BKn1u
-Q7wTnx3K0410Fz5dCr4VcXwSP+TjQbyF3Z8ClXQSzpyDF86BcA4OfAKeT2Dqg6MfcO/PrbI+
-MvfUHNfz3vB7j7zH178HuvdAQ87qz3rO+s/Gzx4/m1eoexe05E9geOvMOuubm04P/n7TG4Pk
-NEZ2uv605/TUafm0+jTwg2/wRqt+Vpitn43PTs2+OHtmdm5WM/WToz/hfvyk06p70vokZz3Z
-c/LASd7/MOgetj7Mee73388dPQa6Y9ZjzmP8fffWWe/tsFjvvmuF9cxdc3dxpxZmT95VbHA/
-CT3QTTZhDnec5Besj2ypgO0Ylg7vVhxOHD04YjiO4MDvPShuxeGEbmkdP/wtKLrDfEfNHdfd
-cegOdfzWqVuP3spP3XL0Fu6RvU/t5ZKeVdZYtMYa7VhtrRJNg/kiP5iH0+Ds0taR6pVu/7Bk
-HUahy4fqrUMdq6xlYumgGgNWoaCOt/ItfA8f44/wT/H5mj6PxdqL44xnzsNJngKtW9dj7XH2
-8KcWzkihLhta2xbfNrWN3+peZe3sWGfVdVg7nB0vdLzZ8V5H3nAHPIB/7kfcT7l5yb3K6Zbc
-Fpt7cad50ChWDBpAN6gXdYMcYKFFMujULeg4nW5Yd0DH60gL4aaMoIZTcHRmoL+mputU/kJf
-l6zxXC7DQbm6n96l3iE576BMBocu984AfN13y+HDpHVJl9zY75X9S3xdchABiQJTCOiXzBhJ
-qy+ZTNWwC2pqEN6Dd1KzpwaJu5NpKsnySU0SkrhHJZkS1FCBNA54r6E8JFA9QO3dSUJvlFmT
-VqLaScUcU07fGGDa/T/LhW2oCmVuZHN0cmVhbQplbmRvYmoKCjYgMCBvYmoKNjI5MQplbmRv
-YmoKCjcgMCBvYmoKPDwvVHlwZS9Gb250RGVzY3JpcHRvci9Gb250TmFtZS9CQUFBQUErTGli
-ZXJhdGlvblNlcmlmCi9GbGFncyA0Ci9Gb250QkJveFstNTQzIC0zMDMgMTI3NyA5ODFdL0l0
-YWxpY0FuZ2xlIDAKL0FzY2VudCA4OTEKL0Rlc2NlbnQgLTIxNgovQ2FwSGVpZ2h0IDk4MQov
-U3RlbVYgODAKL0ZvbnRGaWxlMiA1IDAgUgo+PgplbmRvYmoKCjggMCBvYmoKPDwvTGVuZ3Ro
-IDI5Mi9GaWx0ZXIvRmxhdGVEZWNvZGU+PgpzdHJlYW0KeJxdkctuwyAQRfd8Bct0EfmROA/J
-spQmseRFH6rbD3BgnCLVGGGy8N+XmUlbqQvQmZl7BxiSY3NqrAnJqx9VC0H2xmoP03jzCuQF
-rsaKLJfaqHCPaFdD50QSve08BRga249lKZK3WJuCn+XioMcLPIjkxWvwxl7l4uPYxri9OfcF
-A9ggU1FVUkMf+zx17rkbICHXstGxbMK8jJY/wfvsQOYUZ3wVNWqYXKfAd/YKokzTSpZ1XQmw
-+l8tK9hy6dVn56M0i9I0LdZV5Jx4s0NeMe+R18TbFXJBnKfIG9ZkyFvWUJ8d5wvkPTPlD8w1
-8iMz9Tyyl/Qnzp+Qz8xn5JrPPdOj7rfH5+H8f8Ym1c37ODL6JJoVTslY+P1HNzp00foG7l+O
-gwplbmRzdHJlYW0KZW5kb2JqCgo5IDAgb2JqCjw8L1R5cGUvRm9udC9TdWJ0eXBlL1RydWVU
-eXBlL0Jhc2VGb250L0JBQUFBQStMaWJlcmF0aW9uU2VyaWYKL0ZpcnN0Q2hhciAwCi9MYXN0
-Q2hhciAxNQovV2lkdGhzWzc3NyA2MTAgNTAwIDI3NyAzODkgMjUwIDQ0MyAyNzcgNDQzIDUw
-MCA1MDAgNDQzIDUwMCA3NzcgNTAwIDI1MApdCi9Gb250RGVzY3JpcHRvciA3IDAgUgovVG9V
-bmljb2RlIDggMCBSCj4+CmVuZG9iagoKMTAgMCBvYmoKPDwvRjEgOSAwIFIKPj4KZW5kb2Jq
-CgoxMSAwIG9iago8PC9Gb250IDEwIDAgUgovUHJvY1NldFsvUERGL1RleHRdCj4+CmVuZG9i
-agoKMSAwIG9iago8PC9UeXBlL1BhZ2UvUGFyZW50IDQgMCBSL1Jlc291cmNlcyAxMSAwIFIv
-TWVkaWFCb3hbMCAwIDU5NSA4NDJdL0dyb3VwPDwvUy9UcmFuc3BhcmVuY3kvQ1MvRGV2aWNl
-UkdCL0kgdHJ1ZT4+L0NvbnRlbnRzIDIgMCBSPj4KZW5kb2JqCgo0IDAgb2JqCjw8L1R5cGUv
-UGFnZXMKL1Jlc291cmNlcyAxMSAwIFIKL01lZGlhQm94WyAwIDAgNTk1IDg0MiBdCi9LaWRz
-WyAxIDAgUiBdCi9Db3VudCAxPj4KZW5kb2JqCgoxMiAwIG9iago8PC9UeXBlL0NhdGFsb2cv
-UGFnZXMgNCAwIFIKL09wZW5BY3Rpb25bMSAwIFIgL1hZWiBudWxsIG51bGwgMF0KL0xhbmco
-ZW4tR0IpCj4+CmVuZG9iagoKMTMgMCBvYmoKPDwvQ3JlYXRvcjxGRUZGMDA1NzAwNzIwMDY5
-MDA3NDAwNjUwMDcyPgovUHJvZHVjZXI8RkVGRjAwNEMwMDY5MDA2MjAwNzIwMDY1MDA0RjAw
-NjYwMDY2MDA2OTAwNjMwMDY1MDAyMDAwMzUwMDJFMDAzMD4KL0NyZWF0aW9uRGF0ZShEOjIw
-MTYwMjA0MjIwMDAyWicpPj4KZW5kb2JqCgp4cmVmCjAgMTQKMDAwMDAwMDAwMCA2NTUzNSBm
-IAowMDAwMDA3NTA5IDAwMDAwIG4gCjAwMDAwMDAwMTkgMDAwMDAgbiAKMDAwMDAwMDIyOSAw
-MDAwMCBuIAowMDAwMDA3NjUyIDAwMDAwIG4gCjAwMDAwMDAyNDkgMDAwMDAgbiAKMDAwMDAw
-NjYyNSAwMDAwMCBuIAowMDAwMDA2NjQ2IDAwMDAwIG4gCjAwMDAwMDY4NDEgMDAwMDAgbiAK
-MDAwMDAwNzIwMiAwMDAwMCBuIAowMDAwMDA3NDIyIDAwMDAwIG4gCjAwMDAwMDc0NTQgMDAw
-MDAgbiAKMDAwMDAwNzc1MSAwMDAwMCBuIAowMDAwMDA3ODQ4IDAwMDAwIG4gCnRyYWlsZXIK
-PDwvU2l6ZSAxNC9Sb290IDEyIDAgUgovSW5mbyAxMyAwIFIKL0lEIFsgPDRFN0ZCMEZCMjA4
-ODBCNURBQkIzQTNEOTQxNDlBRTQ3Pgo8NEU3RkIwRkIyMDg4MEI1REFCQjNBM0Q5NDE0OUFF
-NDc+IF0KL0RvY0NoZWNrc3VtIC8yQTY0RDMzNzRFQTVEODMwNTRDNEI2RDFEMUY4QzU1RQo+
-PgpzdGFydHhyZWYKODAxOAolJUVPRgo=
---------------090701020702030809070008--

src/documents/tests/test_mail.py

@@ -1,90 +0,0 @@
-import base64
-import os
-from hashlib import md5
-from unittest import mock
-
-import magic
-from django.conf import settings
-from django.test import TestCase
-
-from ..mail import Message, Attachment
-
-
-class TestMessage(TestCase):
-
-    def __init__(self, *args, **kwargs):
-
-        TestCase.__init__(self, *args, **kwargs)
-        self.sample = os.path.join(
-            settings.BASE_DIR,
-            "documents",
-            "tests",
-            "samples",
-            "mail.txt"
-        )
-
-    def test_init(self):
-
-        with open(self.sample, "rb") as f:
-
-            with mock.patch("logging.StreamHandler.emit") as __:
-                message = Message(f.read())
-
-            self.assertTrue(message)
-            self.assertEqual(message.subject, "Test 0")
-
-            data = message.attachment.read()
-
-            self.assertEqual(
-                md5(data).hexdigest(), "7c89655f9e9eb7dd8cde8568e8115d59")
-
-            self.assertEqual(
-                message.attachment.content_type, "application/pdf")
-            with magic.Magic(flags=magic.MAGIC_MIME_TYPE) as m:
-                self.assertEqual(m.id_buffer(data), "application/pdf")
-
-
-class TestInlineMessage(TestCase):
-
-    def __init__(self, *args, **kwargs):
-
-        TestCase.__init__(self, *args, **kwargs)
-        self.sample = os.path.join(
-            settings.BASE_DIR,
-            "documents",
-            "tests",
-            "samples",
-            "inline_mail.txt"
-        )
-
-    def test_init(self):
-
-        with open(self.sample, "rb") as f:
-
-            with mock.patch("logging.StreamHandler.emit") as __:
-                message = Message(f.read())
-
-            self.assertTrue(message)
-            self.assertEqual(message.subject, "Paperless Inline Image")
-
-            data = message.attachment.read()
-
-            self.assertEqual(
-                md5(data).hexdigest(), "30c00a7b42913e65f7fdb0be40b9eef3")
-
-            self.assertEqual(
-                message.attachment.content_type, "image/png")
-            with magic.Magic(flags=magic.MAGIC_MIME_TYPE) as m:
-                self.assertEqual(m.id_buffer(data), "image/png")
-
-
-class TestAttachment(TestCase):
-
-    def test_init(self):
-        data = base64.encodebytes(b"0")
-        self.assertEqual(Attachment(data, "application/pdf").suffix, "pdf")
-        self.assertEqual(Attachment(data, "image/png").suffix, "png")
-        self.assertEqual(Attachment(data, "image/jpeg").suffix, "jpeg")
-        self.assertEqual(Attachment(data, "image/gif").suffix, "gif")
-        self.assertEqual(Attachment(data, "image/tiff").suffix, "tiff")
-        self.assertEqual(Attachment(data, "image/png").read(), data)

src/paperless/settings.py

@@ -80,6 +80,7 @@ INSTALLED_APPS = [
     "documents.apps.DocumentsConfig",
     "paperless_tesseract.apps.PaperlessTesseractConfig",
     "paperless_text.apps.PaperlessTextConfig",
+    "paperless_mail.apps.PaperlessMailConfig",
 
     "django.contrib.admin",
 

src/paperless_mail/admin.py

@@ -0,0 +1,27 @@
+from django.contrib import admin
+from django import forms
+
+from paperless_mail.models import MailAccount, MailRule
+
+
+class MailAccountForm(forms.ModelForm):
+
+    password = forms.CharField(widget=forms.PasswordInput)
+
+    class Meta:
+        fields = '__all__'
+        model = MailAccount
+
+
+class MailAccountAdmin(admin.ModelAdmin):
+
+    list_display = ("name", "imap_server", "username")
+
+
+class MailRuleAdmin(admin.ModelAdmin):
+
+    list_display = ("name", "account", "folder", "action")
+
+
+admin.site.register(MailAccount, MailAccountAdmin)
+admin.site.register(MailRule, MailRuleAdmin)

src/paperless_mail/apps.py

@@ -0,0 +1,7 @@
+from django.apps import AppConfig
+
+
+class PaperlessMailConfig(AppConfig):
+    name = 'paperless_mail'
+
+    verbose_name = 'Paperless Mail'

src/paperless_mail/mail.py

@@ -0,0 +1,227 @@
+import os
+import tempfile
+from datetime import timedelta, date
+
+from django.conf import settings
+from django.utils.text import slugify
+from django_q.tasks import async_task
+from imap_tools import MailBox, MailBoxUnencrypted, AND, MailMessageFlags, \
+    MailboxFolderSelectError
+
+from documents.models import Correspondent
+from paperless_mail.models import MailAccount, MailRule
+
+
+class MailError(Exception):
+    pass
+
+
+class BaseMailAction:
+
+    def get_criteria(self):
+        return {}
+
+    def post_consume(self, M, message_uids, parameter):
+        pass
+
+
+class DeleteMailAction(BaseMailAction):
+
+    def post_consume(self, M, message_uids, parameter):
+        M.delete(message_uids)
+
+
+class MarkReadMailAction(BaseMailAction):
+
+    def get_criteria(self):
+        return {'seen': False}
+
+    def post_consume(self, M, message_uids, parameter):
+        M.seen(message_uids, True)
+
+
+class MoveMailAction(BaseMailAction):
+
+    def post_consume(self, M, message_uids, parameter):
+        M.move(message_uids, parameter)
+
+
+class FlagMailAction(BaseMailAction):
+
+    def get_criteria(self):
+        return {'flagged': False}
+
+    def post_consume(self, M, message_uids, parameter):
+        M.flag(message_uids, [MailMessageFlags.FLAGGED], True)
+
+
+def get_rule_action(rule):
+    if rule.action == MailRule.ACTION_FLAG:
+        return FlagMailAction()
+    elif rule.action == MailRule.ACTION_DELETE:
+        return DeleteMailAction()
+    elif rule.action == MailRule.ACTION_MOVE:
+        return MoveMailAction()
+    elif rule.action == MailRule.ACTION_MARK_READ:
+        return MarkReadMailAction()
+    else:
+        raise ValueError("Unknown action.")
+
+
+def make_criterias(rule):
+    maximum_age = date.today() - timedelta(days=rule.maximum_age)
+    criterias = {
+        "date_gte": maximum_age
+    }
+    if rule.filter_from:
+        criterias["from_"] = rule.filter_from
+    if rule.filter_subject:
+        criterias["subject"] = rule.filter_subject
+    if rule.filter_body:
+        criterias["body"] = rule.filter_body
+
+    return {**criterias, **get_rule_action(rule).get_criteria()}
+
+
+def handle_mail_account(account):
+
+    if account.imap_security == MailAccount.IMAP_SECURITY_NONE:
+        mailbox = MailBoxUnencrypted(account.imap_server, account.imap_port)
+    elif account.imap_security == MailAccount.IMAP_SECURITY_STARTTLS:
+        mailbox = MailBox(account.imap_server, account.imap_port, starttls=True)
+    elif account.imap_security == MailAccount.IMAP_SECURITY_SSL:
+        mailbox = MailBox(account.imap_server, account.imap_port)
+    else:
+        raise ValueError("Unknown IMAP security")
+
+    total_processed_files = 0
+
+    with mailbox as M:
+
+        try:
+            M.login(account.username, account.password)
+        except Exception:
+            raise MailError(
+                f"Error while authenticating account {account.name}")
+
+        for rule in account.rules.all():
+
+            try:
+                M.folder.set(rule.folder)
+            except MailboxFolderSelectError:
+                raise MailError(
+                    f"Rule {rule.name}: Folder {rule.folder} does not exist "
+                    f"in account {account.name}")
+
+            criterias = make_criterias(rule)
+
+            try:
+                messages = M.fetch(criteria=AND(**criterias), mark_seen=False)
+            except Exception:
+                raise MailError(
+                    f"Rule {rule.name}: Error while fetching folder "
+                    f"{rule.folder} of account {account.name}")
+
+            post_consume_messages = []
+
+            for message in messages:
+                try:
+                    processed_files = handle_message(message, rule)
+                except Exception:
+                    raise MailError(
+                        f"Rule {rule.name}: Error while processing mail "
+                        f"{message.uid} of account {account.name}")
+                if processed_files > 0:
+                    post_consume_messages.append(message.uid)
+
+                total_processed_files += processed_files
+            try:
+                get_rule_action(rule).post_consume(
+                    M,
+                    post_consume_messages,
+                    rule.action_parameter)
+
+            except Exception:
+                raise MailError(
+                    f"Rule {rule.name}: Error while processing post-consume "
+                    f"actions for account {account.name}")
+
+    return total_processed_files
+
+
+def get_title(message, att, rule):
+    if rule.assign_title_from == MailRule.TITLE_FROM_SUBJECT:
+        title = message.subject
+    elif rule.assign_title_from == MailRule.TITLE_FROM_FILENAME:
+        title = os.path.splitext(os.path.basename(att.filename))[0]
+    else:
+        raise ValueError("Unknown title selector.")
+
+    return title
+
+
+def get_correspondent(message, rule):
+    if rule.assign_correspondent_from == MailRule.CORRESPONDENT_FROM_NOTHING:
+        correspondent = None
+    elif rule.assign_correspondent_from == MailRule.CORRESPONDENT_FROM_EMAIL:
+        correspondent_name = message.from_
+        correspondent = Correspondent.objects.get_or_create(
+            name=correspondent_name, defaults={
+                "slug": slugify(correspondent_name)
+            })[0]
+    elif rule.assign_correspondent_from == MailRule.CORRESPONDENT_FROM_NAME:
+        if message.from_values and \
+           'name' in message.from_values \
+           and message.from_values['name']:
+            correspondent_name = message.from_values['name']
+        else:
+            correspondent_name = message.from_
+
+        correspondent = Correspondent.objects.get_or_create(
+            name=correspondent_name, defaults={
+                "slug": slugify(correspondent_name)
+            })[0]
+    elif rule.assign_correspondent_from == MailRule.CORRESPONDENT_FROM_CUSTOM:
+        correspondent = rule.assign_correspondent
+    else:
+        raise ValueError("Unknwown correspondent selector")
+
+    return correspondent
+
+
+def handle_message(message, rule):
+    if not message.attachments:
+        return 0
+
+    correspondent = get_correspondent(message, rule)
+    tag = rule.assign_tag
+    doc_type = rule.assign_document_type
+
+    processed_attachments = 0
+
+    for att in message.attachments:
+
+        title = get_title(message, att, rule)
+
+        # TODO: check with parsers what files types are supported
+        if att.content_type == 'application/pdf':
+
+            os.makedirs(settings.SCRATCH_DIR, exist_ok=True)
+            _, temp_filename = tempfile.mkstemp(prefix="paperless-mail-", dir=settings.SCRATCH_DIR)
+            with open(temp_filename, 'wb') as f:
+                f.write(att.payload)
+
+            async_task(
+                "documents.tasks.consume_file",
+                path=temp_filename,
+                override_filename=att.filename,
+                override_title=title,
+                override_correspondent_id=correspondent.id if correspondent else None,
+                override_document_type_id=doc_type.id if doc_type else None,
+                override_tag_ids=[tag.id] if tag else None,
+                task_name=f"Mail: {att.filename}"
+            )
+
+            processed_attachments += 1
+
+    return processed_attachments

src/paperless_mail/management/commands/mail_fetcher.py

@@ -0,0 +1,13 @@
+from django.core.management.base import BaseCommand
+
+from paperless_mail import mail, tasks
+
+
+class Command(BaseCommand):
+
+    help = """
+    """.replace("    ", "")
+
+    def handle(self, *args, **options):
+
+        tasks.process_mail_accounts()

src/paperless_mail/migrations/0001_initial.py

@@ -0,0 +1,48 @@
+# Generated by Django 3.1.3 on 2020-11-15 22:54
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    initial = True
+
+    dependencies = [
+        ('documents', '1002_auto_20201111_1105'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='MailAccount',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('name', models.CharField(max_length=256, unique=True)),
+                ('imap_server', models.CharField(max_length=256)),
+                ('imap_port', models.IntegerField(blank=True, null=True)),
+                ('imap_security', models.PositiveIntegerField(choices=[(1, 'No encryption'), (2, 'Use SSL'), (3, 'Use STARTTLS')], default=2)),
+                ('username', models.CharField(max_length=256)),
+                ('password', models.CharField(max_length=256)),
+            ],
+        ),
+        migrations.CreateModel(
+            name='MailRule',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('name', models.CharField(max_length=256)),
+                ('folder', models.CharField(default='INBOX', max_length=256)),
+                ('filter_from', models.CharField(blank=True, max_length=256, null=True)),
+                ('filter_subject', models.CharField(blank=True, max_length=256, null=True)),
+                ('filter_body', models.CharField(blank=True, max_length=256, null=True)),
+                ('maximum_age', models.PositiveIntegerField(default=30)),
+                ('action', models.PositiveIntegerField(choices=[(1, 'Delete'), (2, 'Move to specified folder'), (3, "Mark as read, don't process read mails"), (4, "Flag the mail, don't process flagged mails")], default=3, help_text='The action applied to the mail. This action is only performed when documents were consumed from the mail. Mails without attachments will remain entirely untouched.')),
+                ('action_parameter', models.CharField(blank=True, help_text='Additional parameter for the action selected above, i.e., the target folder of the move to folder action.', max_length=256, null=True)),
+                ('assign_title_from', models.PositiveIntegerField(choices=[(1, 'Use subject as title'), (2, 'Use attachment filename as title')], default=1)),
+                ('assign_correspondent_from', models.PositiveIntegerField(choices=[(1, 'Do not assign a correspondent'), (2, 'Use mail address'), (3, 'Use name (or mail address if not available)'), (4, 'Use correspondent selected below')], default=1)),
+                ('account', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='rules', to='paperless_mail.mailaccount')),
+                ('assign_correspondent', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='documents.correspondent')),
+                ('assign_document_type', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='documents.documenttype')),
+                ('assign_tag', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, to='documents.tag')),
+            ],
+        ),
+    ]

src/paperless_mail/migrations/0002_auto_20201117_1334.py

@@ -0,0 +1,32 @@
+# Generated by Django 3.1.3 on 2020-11-17 13:34
+
+from django.db import migrations
+from django.db.migrations import RunPython
+from django_q.models import Schedule
+from django_q.tasks import schedule
+
+
+def add_schedules(apps, schema_editor):
+    schedule('paperless_mail.tasks.process_mail_accounts',
+             name="Check all e-mail accounts",
+             schedule_type=Schedule.MINUTES,
+             minutes=10)
+
+
+def remove_schedules(apps, schema_editor):
+    Schedule.objects.filter(
+        func='paperless_mail.tasks.process_mail_accounts').delete()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('paperless_mail', '0001_initial'),
+        ('django_q', '0013_task_attempt_count'),
+    ]
+
+    operations = [
+        RunPython(add_schedules, remove_schedules)
+    ]
+
+

src/paperless_mail/models.py

@@ -0,0 +1,137 @@
+from django.db import models
+
+# Create your models here.
+from django.db import models
+
+import documents.models as document_models
+
+
+class MailAccount(models.Model):
+
+    IMAP_SECURITY_NONE = 1
+    IMAP_SECURITY_SSL = 2
+    IMAP_SECURITY_STARTTLS = 3
+
+    IMAP_SECURITY_OPTIONS = (
+        (IMAP_SECURITY_NONE, "No encryption"),
+        (IMAP_SECURITY_SSL, "Use SSL"),
+        (IMAP_SECURITY_STARTTLS, "Use STARTTLS"),
+    )
+
+    name = models.CharField(max_length=256, unique=True)
+
+    imap_server = models.CharField(max_length=256)
+
+    imap_port = models.IntegerField(blank=True, null=True)
+
+    imap_security = models.PositiveIntegerField(
+        choices=IMAP_SECURITY_OPTIONS,
+        default=IMAP_SECURITY_SSL
+    )
+
+    username = models.CharField(max_length=256)
+
+    password = models.CharField(max_length=256)
+
+    def __str__(self):
+        return self.name
+
+
+class MailRule(models.Model):
+
+    ACTION_DELETE = 1
+    ACTION_MOVE = 2
+    ACTION_MARK_READ = 3
+    ACTION_FLAG = 4
+
+    ACTIONS = (
+        (ACTION_DELETE, "Delete"),
+        (ACTION_MOVE, "Move to specified folder"),
+        (ACTION_MARK_READ, "Mark as read, don't process read mails"),
+        (ACTION_FLAG, "Flag the mail, don't process flagged mails")
+    )
+
+    TITLE_FROM_SUBJECT = 1
+    TITLE_FROM_FILENAME = 2
+
+    TITLE_SELECTOR = (
+        (TITLE_FROM_SUBJECT, "Use subject as title"),
+        (TITLE_FROM_FILENAME, "Use attachment filename as title")
+    )
+
+    CORRESPONDENT_FROM_NOTHING = 1
+    CORRESPONDENT_FROM_EMAIL = 2
+    CORRESPONDENT_FROM_NAME = 3
+    CORRESPONDENT_FROM_CUSTOM = 4
+
+    CORRESPONDENT_SELECTOR = (
+        (CORRESPONDENT_FROM_NOTHING, "Do not assign a correspondent"),
+        (CORRESPONDENT_FROM_EMAIL, "Use mail address"),
+        (CORRESPONDENT_FROM_NAME, "Use name (or mail address if not available)"),
+        (CORRESPONDENT_FROM_CUSTOM, "Use correspondent selected below")
+    )
+
+    name = models.CharField(max_length=256)
+
+    account = models.ForeignKey(
+        MailAccount,
+        related_name="rules",
+        on_delete=models.CASCADE
+    )
+
+    folder = models.CharField(default='INBOX', max_length=256)
+
+    filter_from = models.CharField(max_length=256, null=True, blank=True)
+    filter_subject = models.CharField(max_length=256, null=True, blank=True)
+    filter_body = models.CharField(max_length=256, null=True, blank=True)
+
+    maximum_age = models.PositiveIntegerField(default=30)
+
+    action = models.PositiveIntegerField(
+        choices=ACTIONS,
+        default=ACTION_MARK_READ,
+        help_text="The action applied to the mail. This action is only "
+                  "performed when documents were consumed from the mail. "
+                  "Mails without attachments will remain entirely "
+                  "untouched."
+    )
+
+    action_parameter = models.CharField(
+        max_length=256, blank=True, null=True,
+        help_text="Additional parameter for the action selected above, i.e., "
+                  "the target folder of the move to folder action."
+    )
+
+    assign_title_from = models.PositiveIntegerField(
+        choices=TITLE_SELECTOR,
+        default=TITLE_FROM_SUBJECT
+    )
+
+    assign_tag = models.ForeignKey(
+        document_models.Tag,
+        null=True,
+        blank=True,
+        on_delete=models.SET_NULL
+    )
+
+    assign_document_type = models.ForeignKey(
+        document_models.DocumentType,
+        null=True,
+        blank=True,
+        on_delete=models.SET_NULL
+    )
+
+    assign_correspondent_from = models.PositiveIntegerField(
+        choices=CORRESPONDENT_SELECTOR,
+        default=CORRESPONDENT_FROM_NOTHING
+    )
+
+    assign_correspondent = models.ForeignKey(
+        document_models.Correspondent,
+        null=True,
+        blank=True,
+        on_delete=models.SET_NULL
+    )
+
+    def __str__(self):
+        return self.name

src/paperless_mail/tasks.py

@@ -0,0 +1,23 @@
+import logging
+
+from paperless_mail import mail
+from paperless_mail.models import MailAccount
+
+
+def process_mail_accounts():
+    total_new_documents = 0
+    for account in MailAccount.objects.all():
+        total_new_documents += mail.handle_mail_account(account)
+
+    if total_new_documents > 0:
+        return f"Added {total_new_documents} document(s)."
+    else:
+        return "No new documents were added."
+
+
+def process_mail_account(name):
+    account = MailAccount.objects.find(name=name)
+    if account:
+        mail.handle_mail_account(account)
+    else:
+        logging.error("Unknown mail acccount: {}".format(name))

src/paperless_mail/tests/test_mail.py

@@ -0,0 +1,352 @@
+import uuid
+from collections import namedtuple
+from typing import ContextManager
+from unittest import mock
+
+from django.test import TestCase
+from imap_tools import MailMessageFlags, MailboxFolderSelectError
+
+from documents.models import Correspondent
+from paperless_mail.mail import get_correspondent, get_title, handle_message, handle_mail_account, MailError
+from paperless_mail.models import MailRule, MailAccount
+
+
+class BogusFolderManager:
+
+    current_folder = "INBOX"
+
+    def set(self, new_folder):
+        if new_folder not in ["INBOX", "spam"]:
+            raise MailboxFolderSelectError(None, "uhm")
+        self.current_folder = new_folder
+
+
+class BogusMailBox(ContextManager):
+    def __enter__(self):
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        pass
+
+    def __init__(self):
+        self.messages = []
+        self.messages_spam = []
+
+    def login(self, username, password):
+        if not (username == 'admin' and password == 'secret'):
+            raise Exception()
+
+    folder = BogusFolderManager()
+
+    def fetch(self, criteria, mark_seen):
+        msg = self.messages
+
+        criteria = str(criteria).strip('()').split(" ")
+
+        if 'UNSEEN' in criteria:
+            msg = filter(lambda m: not m.seen, msg)
+
+        if 'SUBJECT' in criteria:
+            subject = criteria[criteria.index('SUBJECT') + 1].strip('"')
+            msg = filter(lambda m: subject in m.subject, msg)
+
+        if 'BODY' in criteria:
+            body = criteria[criteria.index('BODY') + 1].strip('"')
+            msg = filter(lambda m: body in m.body, msg)
+
+        if 'FROM' in criteria:
+            from_ = criteria[criteria.index('FROM') + 1].strip('"')
+            msg = filter(lambda m: from_ in m.from_, msg)
+
+        if 'UNFLAGGED' in criteria:
+            msg = filter(lambda m: not m.flagged, msg)
+
+        return list(msg)
+
+    def seen(self, uid_list, seen_val):
+        for message in self.messages:
+            if message.uid in uid_list:
+                message.seen = seen_val
+
+    def delete(self, uid_list):
+        self.messages = list(filter(lambda m: m.uid not in uid_list, self.messages))
+
+    def flag(self, uid_list, flag_set, value):
+        for message in self.messages:
+            if message.uid in uid_list:
+                for flag in flag_set:
+                    if flag == MailMessageFlags.FLAGGED:
+                        message.flagged = value
+
+    def move(self, uid_list, folder):
+        if folder == "spam":
+            self.messages_spam.append(
+                filter(lambda m: m.uid in uid_list, self.messages)
+            )
+            self.messages = list(
+                filter(lambda m: m.uid not in uid_list, self.messages)
+            )
+        else:
+            raise Exception()
+
+
+def create_message(num_attachments=1, body="", subject="the suject", from_="noone@mail.com", seen=False, flagged=False):
+    message = namedtuple('MailMessage', [])
+
+    message.uid = uuid.uuid4()
+    message.subject = subject
+    message.attachments = []
+    message.from_ = from_
+    message.body = body
+    for i in range(num_attachments):
+        attachment = namedtuple('Attachment', [])
+        attachment.filename = 'some_file.pdf'
+        attachment.content_type = 'application/pdf'
+        attachment.payload = b'content of the attachment'
+        message.attachments.append(attachment)
+
+    message.seen = seen
+    message.flagged = flagged
+
+    return message
+
+
+class TestMail(TestCase):
+
+    def setUp(self):
+        patcher = mock.patch('paperless_mail.mail.MailBox')
+        m = patcher.start()
+        self.bogus_mailbox = BogusMailBox()
+        m.return_value = self.bogus_mailbox
+        self.addCleanup(patcher.stop)
+
+        patcher = mock.patch('paperless_mail.mail.async_task')
+        self.async_task = patcher.start()
+        self.addCleanup(patcher.stop)
+
+        self.reset_bogus_mailbox()
+
+    def reset_bogus_mailbox(self):
+        self.bogus_mailbox.messages = []
+        self.bogus_mailbox.messages_spam = []
+        self.bogus_mailbox.messages.append(create_message(subject="Invoice 1", from_="amazon@amazon.de", body="cables", seen=True, flagged=False))
+        self.bogus_mailbox.messages.append(create_message(subject="Invoice 2", body="from my favorite electronic store", seen=False, flagged=True))
+        self.bogus_mailbox.messages.append(create_message(subject="Claim your $10M price now!", from_="amazon@amazon-some-indian-site.org", seen=False))
+
+    def test_get_correspondent(self):
+        message = namedtuple('MailMessage', [])
+        message.from_ = "someone@somewhere.com"
+        message.from_values = {'name': "Someone!", 'email': "someone@somewhere.com"}
+
+        message2 = namedtuple('MailMessage', [])
+        message2.from_ = "me@localhost.com"
+        message2.from_values = {'name': "", 'email': "fake@localhost.com"}
+
+        me_localhost = Correspondent.objects.create(name=message2.from_)
+        someone_else = Correspondent.objects.create(name="someone else")
+
+        rule = MailRule(assign_correspondent_from=MailRule.CORRESPONDENT_FROM_NOTHING)
+        self.assertIsNone(get_correspondent(message, rule))
+
+        rule = MailRule(assign_correspondent_from=MailRule.CORRESPONDENT_FROM_EMAIL)
+        c = get_correspondent(message, rule)
+        self.assertIsNotNone(c)
+        self.assertEqual(c.name, "someone@somewhere.com")
+        c = get_correspondent(message2, rule)
+        self.assertIsNotNone(c)
+        self.assertEqual(c.name, "me@localhost.com")
+        self.assertEqual(c.id, me_localhost.id)
+
+        rule = MailRule(assign_correspondent_from=MailRule.CORRESPONDENT_FROM_NAME)
+        c = get_correspondent(message, rule)
+        self.assertIsNotNone(c)
+        self.assertEqual(c.name, "Someone!")
+        c = get_correspondent(message2, rule)
+        self.assertIsNotNone(c)
+        self.assertEqual(c.id, me_localhost.id)
+
+        rule = MailRule(assign_correspondent_from=MailRule.CORRESPONDENT_FROM_CUSTOM, assign_correspondent=someone_else)
+        c = get_correspondent(message, rule)
+        self.assertEqual(c, someone_else)
+
+    def test_get_title(self):
+        message = namedtuple('MailMessage', [])
+        message.subject = "the message title"
+        att = namedtuple('Attachment', [])
+        att.filename = "this_is_the_file.pdf"
+        rule = MailRule(assign_title_from=MailRule.TITLE_FROM_FILENAME)
+        self.assertEqual(get_title(message, att, rule), "this_is_the_file")
+        rule = MailRule(assign_title_from=MailRule.TITLE_FROM_SUBJECT)
+        self.assertEqual(get_title(message, att, rule), "the message title")
+
+    def test_handle_message(self):
+        message = namedtuple('MailMessage', [])
+        message.subject = "the message title"
+
+        att = namedtuple('Attachment', [])
+        att.filename = "test1.pdf"
+        att.content_type = 'application/pdf'
+        att.payload = b"attachment contents"
+
+        att2 = namedtuple('Attachment', [])
+        att2.filename = "test2.pdf"
+        att2.content_type = 'application/pdf'
+        att2.payload = b"attachment contents"
+
+        att3 = namedtuple('Attachment', [])
+        att3.filename = "test3.pdf"
+        att3.content_type = 'application/invalid'
+        att3.payload = b"attachment contents"
+
+        message.attachments = [att, att2, att3]
+
+        rule = MailRule(assign_title_from=MailRule.TITLE_FROM_FILENAME)
+
+        result = handle_message(message, rule)
+
+        self.assertEqual(result, 2)
+
+        self.assertEqual(len(self.async_task.call_args_list), 2)
+
+        args1, kwargs1 = self.async_task.call_args_list[0]
+        args2, kwargs2 = self.async_task.call_args_list[1]
+
+        self.assertEqual(kwargs1['override_title'], "test1")
+        self.assertEqual(kwargs1['override_filename'], "test1.pdf")
+
+        self.assertEqual(kwargs2['override_title'], "test2")
+        self.assertEqual(kwargs2['override_filename'], "test2.pdf")
+
+    @mock.patch("paperless_mail.mail.async_task")
+    def test_handle_empty_message(self, m):
+        message = namedtuple('MailMessage', [])
+
+        message.attachments = []
+        rule = MailRule()
+
+        result = handle_message(message, rule)
+
+        self.assertFalse(m.called)
+        self.assertEqual(result, 0)
+
+    def test_handle_mail_account_mark_read(self):
+
+        account = MailAccount.objects.create(name="test", imap_server="", username="admin", password="secret")
+
+        rule = MailRule.objects.create(name="testrule", account=account, action=MailRule.ACTION_MARK_READ)
+
+        self.assertEqual(self.async_task.call_count, 0)
+        self.assertEqual(len(self.bogus_mailbox.fetch("UNSEEN", False)), 2)
+        handle_mail_account(account)
+        self.assertEqual(self.async_task.call_count, 2)
+        self.assertEqual(len(self.bogus_mailbox.fetch("UNSEEN", False)), 0)
+
+    def test_handle_mail_account_delete(self):
+
+        account = MailAccount.objects.create(name="test", imap_server="", username="admin", password="secret")
+
+        rule = MailRule.objects.create(name="testrule", account=account, action=MailRule.ACTION_DELETE, filter_subject="Invoice")
+
+        self.assertEqual(self.async_task.call_count, 0)
+        self.assertEqual(len(self.bogus_mailbox.messages), 3)
+        handle_mail_account(account)
+        self.assertEqual(self.async_task.call_count, 2)
+        self.assertEqual(len(self.bogus_mailbox.messages), 1)
+
+    def test_handle_mail_account_flag(self):
+        account = MailAccount.objects.create(name="test", imap_server="", username="admin", password="secret")
+
+        rule = MailRule.objects.create(name="testrule", account=account, action=MailRule.ACTION_FLAG, filter_subject="Invoice")
+
+        self.assertEqual(self.async_task.call_count, 0)
+        self.assertEqual(len(self.bogus_mailbox.fetch("UNFLAGGED", False)), 2)
+        handle_mail_account(account)
+        self.assertEqual(self.async_task.call_count, 1)
+        self.assertEqual(len(self.bogus_mailbox.fetch("UNFLAGGED", False)), 1)
+
+    def test_handle_mail_account_move(self):
+        account = MailAccount.objects.create(name="test", imap_server="", username="admin", password="secret")
+
+        rule = MailRule.objects.create(name="testrule", account=account, action=MailRule.ACTION_MOVE, action_parameter="spam", filter_subject="Claim")
+
+        self.assertEqual(self.async_task.call_count, 0)
+        self.assertEqual(len(self.bogus_mailbox.messages), 3)
+        self.assertEqual(len(self.bogus_mailbox.messages_spam), 0)
+        handle_mail_account(account)
+        self.assertEqual(self.async_task.call_count, 1)
+        self.assertEqual(len(self.bogus_mailbox.messages), 2)
+        self.assertEqual(len(self.bogus_mailbox.messages_spam), 1)
+
+    def test_errors(self):
+        account = MailAccount.objects.create(name="test", imap_server="", username="admin", password="wrong")
+
+        try:
+            handle_mail_account(account)
+        except MailError as e:
+            self.assertTrue(str(e).startswith("Error while authenticating account"))
+        else:
+            self.fail("Should raise exception")
+
+        account = MailAccount.objects.create(name="test2", imap_server="", username="admin", password="secret")
+        rule = MailRule.objects.create(name="testrule", account=account, folder="uuuh")
+
+        try:
+            handle_mail_account(account)
+        except MailError as e:
+            self.assertTrue("uuuh does not exist" in str(e))
+        else:
+            self.fail("Should raise exception")
+
+        account = MailAccount.objects.create(name="test3", imap_server="", username="admin", password="secret")
+
+        rule = MailRule.objects.create(name="testrule", account=account, action=MailRule.ACTION_MOVE, action_parameter="doesnotexist", filter_subject="Claim")
+
+        try:
+            handle_mail_account(account)
+        except MailError as e:
+            self.assertTrue("Error while processing post-consume actions" in str(e))
+        else:
+            self.fail("Should raise exception")
+
+    def test_filters(self):
+
+        account = MailAccount.objects.create(name="test3", imap_server="", username="admin", password="secret")
+        rule = MailRule.objects.create(name="testrule", account=account, action=MailRule.ACTION_DELETE, filter_subject="Claim")
+
+        self.assertEqual(self.async_task.call_count, 0)
+
+        self.assertEqual(len(self.bogus_mailbox.messages), 3)
+        handle_mail_account(account)
+        self.assertEqual(len(self.bogus_mailbox.messages), 2)
+        self.assertEqual(self.async_task.call_count, 1)
+
+        self.reset_bogus_mailbox()
+
+        rule.filter_subject = None
+        rule.filter_body = "electronic"
+        rule.save()
+        self.assertEqual(len(self.bogus_mailbox.messages), 3)
+        handle_mail_account(account)
+        self.assertEqual(len(self.bogus_mailbox.messages), 2)
+        self.assertEqual(self.async_task.call_count, 2)
+
+        self.reset_bogus_mailbox()
+
+        rule.filter_from = "amazon"
+        rule.filter_body = None
+        rule.save()
+        self.assertEqual(len(self.bogus_mailbox.messages), 3)
+        handle_mail_account(account)
+        self.assertEqual(len(self.bogus_mailbox.messages), 1)
+        self.assertEqual(self.async_task.call_count, 4)
+
+        self.reset_bogus_mailbox()
+
+        rule.filter_from = "amazon"
+        rule.filter_body = "cables"
+        rule.filter_subject = "Invoice"
+        rule.save()
+        self.assertEqual(len(self.bogus_mailbox.messages), 3)
+        handle_mail_account(account)
+        self.assertEqual(len(self.bogus_mailbox.messages), 2)
+        self.assertEqual(self.async_task.call_count, 5)

src/paperless_mail/views.py

@@ -0,0 +1,3 @@
+from django.shortcuts import render
+
+# Create your views here.