paradizelost/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2025-07-30 18:27:45 -05:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'dev'

Browse Source 
commit fb9d3f736b
Merge: 4d4d5453 049dc179
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Dec 4 16:47:58 2022 -0800

    Merge pull request #2000 from paperless-ngx/feature-frontend-paperless-mail

    Feature: frontend paperless mail

commit 4d4d545343
Merge: 2704bcb9 4b31e5d0
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Sun Dec 4 16:34:41 2022 -0800

    Merge pull request #848 from p-h-a-i-l/feature-consume-eml

    Feature ability to consume mails and eml files

commit 049dc17902
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Dec 4 16:33:07 2022 -0800

    Moves where the mail views live and puts the ordering on those

commit 4b31e5d0b4
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Dec 4 14:00:59 2022 -0800

    Fixes my broken formatting

commit 8076ebd78c
Merge: fe2db4db 2704bcb9
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Dec 4 13:55:46 2022 -0800

    Merge remote-tracking branch 'upstream/dev' into feature-consume-eml

commit 2704bcb979
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Dec 4 13:06:18 2022 -0800

    Resets to -dev versioning

commit 59f6074093
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Dec 4 12:58:03 2022 -0800

    Bumps version to 1.10.2

commit b1da7f3491
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Dec 4 12:57:19 2022 -0800

    Probably fixes the changelog step not working

commit adde88e7b9
Merge: a8f3c4be 8e876ef2
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Dec 4 12:55:55 2022 -0800

    Merge branch 'dev'

commit 8e876ef2d1
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Dec 4 10:20:48 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit 2ea0f83a91
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 05:12:02 2022 -0800

    New translations messages.xlf (Italian)
    [ci skip]

commit 05d8ea5a9d
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:09:05 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit 967248233f
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:09:04 2022 -0800

    New translations messages.xlf (French)
    [ci skip]

commit b4c4b9fb6a
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:09:03 2022 -0800

    New translations messages.xlf (Spanish)
    [ci skip]

commit adb6483abc
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:09:01 2022 -0800

    New translations messages.xlf (Arabic)
    [ci skip]

commit 908db55bb7
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:09:00 2022 -0800

    New translations messages.xlf (Belarusian)
    [ci skip]

commit 610f20de28
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:59 2022 -0800

    New translations messages.xlf (Czech)
    [ci skip]

commit b2513a5cde
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:57 2022 -0800

    New translations messages.xlf (Danish)
    [ci skip]

commit bfa1c13d01
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:56 2022 -0800

    New translations messages.xlf (Finnish)
    [ci skip]

commit 12aaff431f
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:55 2022 -0800

    New translations messages.xlf (Hebrew)
    [ci skip]

commit 547e5ea55e
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:54 2022 -0800

    New translations messages.xlf (Italian)
    [ci skip]

commit c301127096
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:52 2022 -0800

    New translations messages.xlf (Dutch)
    [ci skip]

commit 19147855e7
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:51 2022 -0800

    New translations messages.xlf (Romanian)
    [ci skip]

commit 4e7c7ea1d6
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:49 2022 -0800

    New translations messages.xlf (Norwegian)
    [ci skip]

commit fcf8a49160
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:48 2022 -0800

    New translations messages.xlf (Portuguese)
    [ci skip]

commit c6d658a954
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:47 2022 -0800

    New translations messages.xlf (Russian)
    [ci skip]

commit a78cd6526c
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:45 2022 -0800

    New translations messages.xlf (Slovenian)
    [ci skip]

commit bf895b54f4
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:44 2022 -0800

    New translations messages.xlf (Swedish)
    [ci skip]

commit e5f84ef583
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:43 2022 -0800

    New translations messages.xlf (Turkish)
    [ci skip]

commit 8c690a9a51
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:41 2022 -0800

    New translations messages.xlf (Chinese Simplified)
    [ci skip]

commit 56526b970a
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:40 2022 -0800

    New translations messages.xlf (Portuguese, Brazilian)
    [ci skip]

commit 94fbf92916
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:38 2022 -0800

    New translations messages.xlf (Croatian)
    [ci skip]

commit 37f5e46d09
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:37 2022 -0800

    New translations messages.xlf (Luxembourgish)
    [ci skip]

commit 38be817637
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:36 2022 -0800

    New translations messages.xlf (Polish)
    [ci skip]

commit 17303f41da
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Dec 3 02:08:34 2022 -0800

    New translations messages.xlf (German)
    [ci skip]

commit 55ef0d4a1b
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Dec 4 08:44:35 2022 -0800

    Fixes language code checks around two part languages

commit a8f3c4be54
Merge: fa62ae82 1b9de2be
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 18:47:09 2022 -0800

    Merge pull request #1947 from alexander-bauer/helm

    Take ownership of k8s-at-home Helm chart,

commit 1b9de2be5a
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 18:46:19 2022 -0800

    Use checkout v3

commit 0e8265f1ae
Merge: ea38eb01 5b45a140
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 18:44:28 2022 -0800

    Merge pull request #2109 from paperless-ngx/fix/redis-socket-parsing

    Bugfix: Redis socket compatibility didn't handle URLs with ports

commit 5b45a140b9
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sat Dec 3 18:30:21 2022 -0800

    Fixes issue when the Redis URL also specifies a port

commit 72fb9a475d
Author: Alexander Bauer <sasha@linux.com>
Date:   Thu Nov 10 02:24:04 2022 +0000

    Ignore end-of-lines on generated Chart README

commit bf97f5807f
Author: Alexander Bauer <sasha@linux.com>
Date:   Thu Nov 10 02:21:29 2022 +0000

    Ignore non-yaml Helm chart template

commit a707818b4d
Author: Alexander Bauer <sasha@linux.com>
Date:   Thu Nov 10 01:54:35 2022 +0000

    Change Helm chart releaser to use version tags only

commit fb46c1b96a
Author: Alexander Bauer <sasha@linux.com>
Date:   Thu Nov 10 01:52:02 2022 +0000

    Ignore generated Helm chart README from prettier

commit 3226d8b25b
Author: Alexander Bauer <sasha@linux.com>
Date:   Tue Nov 8 00:46:27 2022 +0000

    fixup! Add mostly-unchanged Helm chart from k8s-at-home

commit 5c4363cbea
Author: Alexander Bauer <sasha@linux.com>
Date:   Tue Nov 8 00:24:04 2022 +0000

    Add mostly-unchanged Helm chart from k8s-at-home

    - Add the chart from k8s-at-home with some modifications
    - Add the Apache 2.0 license to the new charts/paperless-ngx
      subdirectory, the license under which the chart was distributed by
      k8s-at-home. I believe the chart will have to maintain this license.

    - Update the maintainers section and contact information to point to
      Paperless-ngx.
    - Regenerate the README (using helm-docs)

    - Add a GitHub actions configuration to publish the chart using GitHub
      pages. This makes the GitHub Pages page rendered by this repository
      usable as a Helm repository, without affecting potential future uses
      of the Pages site.

    These are in response to discussion #1790.

commit fa62ae820b
Merge: bcc029a2 17891baf
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 14:19:36 2022 -0800

    Merge pull request #2106 from tooomm/docs/edit

    Docs: Some more small MkDocs updates

commit 17891bafaf
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Sat Dec 3 20:02:40 2022 +0100

    lint

commit 15fdadadef
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Sat Dec 3 19:36:49 2022 +0100

    open demo in new page

commit ce9f604d81
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 09:29:34 2022 -0800

    Explicit default ordering for rule / account views

commit 4f876db5d1
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Nov 28 21:38:52 2022 -0800

    prevent loss of unsaved changes to settings on tab nav

commit 5e5f56dc67
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Nov 28 20:39:03 2022 -0800

    Re-org where some of the new classes are found

commit 93fab8bb95
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Nov 28 12:53:20 2022 -0800

    Apply code suggestions from @stumpylog

commit 35ca2195fe
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Nov 28 15:51:39 2022 -0800

    frontend mail rule validation

    Display non-field validation errors, hide action param field if not needed

commit 7ace66d7fd
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Nov 28 13:58:37 2022 -0800

    fix edit dialog getters

commit 4f9a31244b
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Nov 18 20:23:40 2022 -0800

    Add settings routing

commit 14cf4f7095
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Nov 18 19:38:49 2022 -0800

    Update frontend strings

commit 8bd7c27826
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Nov 18 17:11:15 2022 -0800

    Hide order parameter, fix imap port

commit 8c4f486fe9
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Nov 18 14:22:07 2022 -0800

    API mail rule & account tests

    and fix use of assign_tags

commit 2849414445
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Nov 18 14:21:31 2022 -0800

    one-way imap password setting via API, ObfuscatedPasswordField

commit ea1ea0816f
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Nov 18 14:10:17 2022 -0800

    Fix mail account / rule delete

commit 52d3a8703c
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Nov 12 15:14:58 2022 -0800

    Dynamically load mail rules / accounts settings

commit 4cb4d6adcd
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Nov 12 15:15:59 2022 -0800

    update settings tests to not wait on data which is now on-demand

commit 24444237f2
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Nov 12 14:46:57 2022 -0800

    dynamic loading of settings tab contents

commit 40c8629aef
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Nov 10 21:04:29 2022 -0800

    Update welcome tour, move admin button

commit 98cdf614a5
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Nov 9 19:59:35 2022 -0800

    Mail form tweaks

    Include add button

    Include add button

commit 2eb2d99a91
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Nov 9 03:43:57 2022 -0800

    Update frontend fixtures & tests for compatibility

commit 18ad9bcbf2
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 8 12:18:47 2022 -0800

    Working mail rule & account edit

commit 997bff4917
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Nov 9 02:40:45 2022 -0800

    Update deprecated edit-dialog rxjs

commit 78f9a80895
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 8 12:09:16 2022 -0800

    mail account + rule deletion

commit 9231df7a4a
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 8 11:50:57 2022 -0800

    Mail rule edit dialog

commit 6f25917c86
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 8 11:11:35 2022 -0800

    Mail account edit dialog

commit c41d1a78a8
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 8 10:53:41 2022 -0800

    remove unused toastService from edit dialogs and add confirmation

commit c3331086d5
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 8 03:39:54 2022 -0800

    Basic data retrieval

commit 6bd9ccd8f6
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Sat Dec 3 18:30:02 2022 +0100

    update default edit path

commit 68c7cecb07
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Sat Dec 3 18:29:14 2022 +0100

    add mkdocs site to gitignore

commit bcc029a2c7
Merge: 9d418055 1727eb00
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 09:00:02 2022 -0800

    Merge pull request #2104 from paperless-ngx/docs-cleanup

    Chore: Cleans up documentation links

commit ea38eb01b2
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Fri Dec 2 13:54:15 2022 -0800

    Adds support for database number specification

commit 01d070b882
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Fri Dec 2 09:34:59 2022 -0800

    Adds a layer to translate between differing formats of socket based Redis URLs

commit 1727eb00cc
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Fri Dec 2 19:34:43 2022 -0800

    Cleans up a number of internal links

commit 9d4180553c
Merge: 7c614264 8049af4b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 07:58:12 2022 -0800

    Merge pull request #2102 from tooomm/patch-1

    Fix doc links in contributing

commit 8049af4b22
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Sat Dec 3 14:38:55 2022 +0100

    Fix doc links in contributing

commit 7c6142643d
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 02:08:51 2022 -0800

    Update more docs references

commit 2e8706f4e2
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 02:05:49 2022 -0800

    Update frontend translation strings

commit d39d32d555
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 02:04:30 2022 -0800

    Fix docs references

commit 6f52945449
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 01:47:04 2022 -0800

    docs index formatting error

commit 37025297b5
Merge: aa023ea2 dc9e9e3b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 01:33:31 2022 -0800

    Merge pull request #2095 from paperless-ngx/update-readme-doc-links

    Documentation: Update docs links and screenshot in readme, add favicon

commit aa023ea2e3
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Dec 3 01:30:07 2022 -0800

    correct docs deploy domain

commit 78bf0b63a5
Merge: 29391c1c a96ecd67
Author: Felix E <felix@eckhofer.com>
Date:   Sat Dec 3 09:10:20 2022 +0100

    Merge pull request #2087 from Ricks-ha/main

    Add examples to URL and TIME_ZONE

commit dc9e9e3b48
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Dec 2 20:06:51 2022 -0800

    add favicon

commit ab29c49b7a
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Dec 2 19:09:19 2022 -0800

    Update docs links and screenshot in readme

commit 1c0ac474b8
Merge: 25fb8d9c 29391c1c
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Fri Dec 2 19:05:28 2022 -0800

    Merge branch 'main' into dev

commit 29391c1c7b
Merge: 58a01a57 69383497
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Dec 2 15:22:39 2022 -0800

    Merge pull request #2067 from paperless-ngx/material-docs

    [WIP] Feature: Move docs to material-mkdocs

commit 693834971c
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 20:00:23 2022 -0800

    Add v1.10.1 changelog

commit 97376d4b72
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Dec 2 09:09:29 2022 -0800

    update ci for documentation build vs deploy

commit 3ee1d2a9a9
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 29 21:20:45 2022 -0800

    Add changes from #2069

commit 605f885e19
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 29 12:49:23 2022 -0800

    Move docs to material-mkdocs

commit 25fb8d9c3b
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Dec 2 08:30:42 2022 -0800

    Update dev version string

commit a96ecd673b
Author: Ricks-ha <ricks@ricksha.eu>
Date:   Fri Dec 2 13:27:57 2022 +0100

    Add examples to URL and TIME_ZONE

commit 58a01a57ee
Merge: a96f79f6 c18fc03e
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Dec 2 03:36:55 2022 -0800

    Merge pull request #2082 from paperless-ngx/v1.10.1-changelog

    Chore: Add v1.10.1 changelong

commit c18fc03ef3
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 20:00:23 2022 -0800

    Add v1.10.1 changelong

commit a96f79f6a3
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 18:54:00 2022 -0800

    Bump version to 1.10.1

commit d6f1d004a3
Merge: 88cf6ef8 da72d357
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 18:52:35 2022 -0800

    Merge branch 'dev'

commit da72d3571b
Merge: 8241da0e 86592928
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 18:52:06 2022 -0800

    Merge pull request #2050 from paperless-ngx/l10n_dev

    New Crowdin updates

commit 8241da0eb3
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 17:01:22 2022 -0800

    fix broken npm package-lock

commit 51562667bf
Merge: 1aee2988 97eeae65
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 16:40:07 2022 -0800

    Merge pull request #2076 from paperless-ngx/dependabot/npm_and_yarn/src-ui/dev/tslib-2.4.1

    Bump tslib from 2.4.0 to 2.4.1 in /src-ui

commit 97eeae65a3
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Fri Dec 2 00:39:26 2022 +0000

    Bump tslib from 2.4.0 to 2.4.1 in /src-ui

    Bumps [tslib](https://github.com/Microsoft/tslib) from 2.4.0 to 2.4.1.
    - [Release notes](https://github.com/Microsoft/tslib/releases)
    - [Commits](https://github.com/Microsoft/tslib/compare/2.4.0...2.4.1)

    ---
    updated-dependencies:
    - dependency-name: tslib
      dependency-type: direct:production
      update-type: version-update:semver-patch
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit 1aee2988f7
Merge: a63a8dd4 6f0077ef
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 16:39:24 2022 -0800

    Merge pull request #2079 from paperless-ngx/dependabot/npm_and_yarn/src-ui/dev/angular-builders/jest-14.1.0

    Bump @angular-builders/jest from 14.0.1 to 14.1.0 in /src-ui

commit a63a8dd488
Merge: 06a9df6d 39be68a1
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 16:38:38 2022 -0800

    Merge pull request #2078 from paperless-ngx/dependabot/npm_and_yarn/src-ui/dev/jest-preset-angular-12.2.3

    Bump jest-preset-angular from 12.2.2 to 12.2.3 in /src-ui

commit 06a9df6dbd
Merge: 49933bb5 7d7d9630
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 16:37:25 2022 -0800

    Merge pull request #2080 from paperless-ngx/dependabot/npm_and_yarn/src-ui/dev/ngx-file-drop-14.0.2

    Bump ngx-file-drop from 14.0.1 to 14.0.2 in /src-ui

commit 49933bb5a8
Merge: 02c782a1 ac69babf
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 16:18:07 2022 -0800

    Merge pull request #2077 from paperless-ngx/dependabot/npm_and_yarn/src-ui/dev/ngneat/dirty-check-forms-3.0.3

    Bump @ngneat/dirty-check-forms from 3.0.2 to 3.0.3 in /src-ui

commit 7d7d9630c1
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Thu Dec 1 20:03:32 2022 +0000

    Bump ngx-file-drop from 14.0.1 to 14.0.2 in /src-ui

    Bumps [ngx-file-drop](https://github.com/georgipeltekov/ngx-file-drop) from 14.0.1 to 14.0.2.
    - [Release notes](https://github.com/georgipeltekov/ngx-file-drop/releases)
    - [Changelog](https://github.com/georgipeltekov/ngx-file-drop/blob/master/CHANGELOG.md)
    - [Commits](https://github.com/georgipeltekov/ngx-file-drop/compare/v14.0.1...v14.0.2)

    ---
    updated-dependencies:
    - dependency-name: ngx-file-drop
      dependency-type: direct:production
      update-type: version-update:semver-patch
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit 6f0077efac
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Thu Dec 1 20:03:05 2022 +0000

    Bump @angular-builders/jest from 14.0.1 to 14.1.0 in /src-ui

    Bumps [@angular-builders/jest](https://github.com/just-jeb/angular-builders/tree/HEAD/packages/jest) from 14.0.1 to 14.1.0.
    - [Release notes](https://github.com/just-jeb/angular-builders/releases)
    - [Changelog](https://github.com/just-jeb/angular-builders/blob/master/packages/jest/CHANGELOG.md)
    - [Commits](https://github.com/just-jeb/angular-builders/commits/@angular-builders/jest@14.1.0/packages/jest)

    ---
    updated-dependencies:
    - dependency-name: "@angular-builders/jest"
      dependency-type: direct:development
      update-type: version-update:semver-minor
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit 39be68a1a4
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Thu Dec 1 20:02:39 2022 +0000

    Bump jest-preset-angular from 12.2.2 to 12.2.3 in /src-ui

    Bumps [jest-preset-angular](https://github.com/thymikee/jest-preset-angular) from 12.2.2 to 12.2.3.
    - [Release notes](https://github.com/thymikee/jest-preset-angular/releases)
    - [Changelog](https://github.com/thymikee/jest-preset-angular/blob/main/CHANGELOG.md)
    - [Commits](https://github.com/thymikee/jest-preset-angular/compare/v12.2.2...v12.2.3)

    ---
    updated-dependencies:
    - dependency-name: jest-preset-angular
      dependency-type: direct:development
      update-type: version-update:semver-patch
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit ac69babfce
Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Date:   Thu Dec 1 20:02:11 2022 +0000

    Bump @ngneat/dirty-check-forms from 3.0.2 to 3.0.3 in /src-ui

    Bumps [@ngneat/dirty-check-forms](https://github.com/ngneat/dirty-check-forms) from 3.0.2 to 3.0.3.
    - [Release notes](https://github.com/ngneat/dirty-check-forms/releases)
    - [Changelog](https://github.com/ngneat/dirty-check-forms/blob/master/CHANGELOG.md)
    - [Commits](https://github.com/ngneat/dirty-check-forms/compare/v3.0.2...v3.0.3)

    ---
    updated-dependencies:
    - dependency-name: "@ngneat/dirty-check-forms"
      dependency-type: direct:production
      update-type: version-update:semver-patch
    ...

    Signed-off-by: dependabot[bot] <support@github.com>

commit 02c782a127
Merge: 4e90fda8 9b602a4b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Dec 1 08:19:59 2022 -0800

    Merge pull request #2073 from paperless-ngx/fix-frontend-tasks-display

    Fix: frontend tasks display in 1.10.0

commit 4e90fda80f
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Tue Nov 29 20:06:56 2022 -0800

    Expands documentation around the permissions of the custom scripts and the folder

commit 88e3e556a1
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Tue Nov 29 20:05:08 2022 -0800

    Fixes the custom scripts not running as root

commit 88cf6ef843
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Nov 30 15:14:21 2022 -0800

    add demo badge

commit 9b602a4bf0
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Nov 30 13:55:51 2022 -0800

    Fix frontend tasks display

commit fe2db4dbf7
Author: phail <phail@hacknology.de>
Date:   Wed Nov 30 10:16:39 2022 +0100

    adapt compose file for eml parsing

commit 47c88a6bdd
Merge: 4aa31859 a3bc3b78
Author: phail <phail@hacknology.de>
Date:   Wed Nov 30 10:10:57 2022 +0100

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit a3bc3b78d5
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Nov 29 14:34:12 2022 -0800

    Also display the container logs

commit fed7d3e993
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Nov 29 13:59:48 2022 -0800

    Use docker compose to start and stop containers which match directly to our command overrides

commit 3a74f24e49
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Nov 29 12:44:53 2022 -0800

    Adds libatomic1 for supporting armv7 better

commit 52afab39cf
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Nov 29 12:16:51 2022 -0800

    Organizes the system packages a little bit more

commit 8659292852
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Tue Nov 29 00:29:45 2022 -0800

    New translations django.po (Norwegian)
    [ci skip]

commit ce73f159fd
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Mon Nov 28 14:13:54 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit 71382e9c62
Merge: 5d3a6e23 a1a802fc
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Nov 28 14:05:57 2022 -0800

    Merge pull request #2062 from paperless-ngx/fix/2053-long-names-too-quiet

    Bugfix: Don't silence an exception when trying to handle file naming

commit a1a802fc92
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Mon Nov 28 13:44:17 2022 -0800

    Don't silence an exception when trying to handle file naming

commit 4200fc610d
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Mon Nov 28 13:17:22 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit 32d212cd9f
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Mon Nov 28 11:17:07 2022 -0800

    New translations messages.xlf (German)
    [ci skip]

commit 5d3a6e230d
Merge: b33fcc11 f0497e77
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Nov 28 11:04:44 2022 -0800

    Merge pull request #2057 from paperless-ngx/fix/2044-lang-code-diffs

    Bugfix: Some tesseract languages aren't detected as installed.

commit b33fcc117e
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Mon Nov 28 09:25:03 2022 -0800

    Transition to a maintained upload release assert

commit e96d65f945
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Mon Nov 14 15:38:35 2022 -0800

    Allows parsing of WebP format images

commit cfeed0ce6e
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Mon Nov 28 08:30:32 2022 -0800

    New translations django.po (Polish)
    [ci skip]

commit b89ecf7d77
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Mon Nov 28 05:28:39 2022 -0800

    New translations messages.xlf (Luxembourgish)
    [ci skip]

commit 5ca25d44ba
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Mon Nov 28 04:27:31 2022 -0800

    New translations messages.xlf (Luxembourgish)
    [ci skip]

commit 1e11c12d96
Merge: e74d7dad 3e22e8e0
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Nov 27 21:16:50 2022 -0800

    Merge branch 'main' into dev

commit 3e22e8e0b9
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 27 19:22:59 2022 -0800

    prepends the latest changelog

commit dba45f93a4
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 27 19:22:03 2022 -0800

    Fixes the pre-commit command

commit 18f3f44ae9
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:25 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit 85a6a271dc
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:23 2022 -0800

    New translations messages.xlf (French)
    [ci skip]

commit abb515d4ea
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:22 2022 -0800

    New translations messages.xlf (Spanish)
    [ci skip]

commit 309d1f2b67
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:21 2022 -0800

    New translations messages.xlf (Arabic)
    [ci skip]

commit fa2f09bc4b
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:19 2022 -0800

    New translations messages.xlf (Belarusian)
    [ci skip]

commit c51590cd12
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:18 2022 -0800

    New translations messages.xlf (Czech)
    [ci skip]

commit 8e01406acf
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:17 2022 -0800

    New translations messages.xlf (Danish)
    [ci skip]

commit 7cce2f0fe6
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:16 2022 -0800

    New translations messages.xlf (Finnish)
    [ci skip]

commit 95091c2f39
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:14 2022 -0800

    New translations messages.xlf (Hebrew)
    [ci skip]

commit 4a0aa12bd9
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:13 2022 -0800

    New translations messages.xlf (Italian)
    [ci skip]

commit 9a0329746a
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:11 2022 -0800

    New translations messages.xlf (Dutch)
    [ci skip]

commit 8392a6fd4a
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:10 2022 -0800

    New translations messages.xlf (Romanian)
    [ci skip]

commit 8fa18bb8a6
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:09 2022 -0800

    New translations messages.xlf (Norwegian)
    [ci skip]

commit 0095b593fb
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:08 2022 -0800

    New translations messages.xlf (Portuguese)
    [ci skip]

commit b1e5135e21
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:05 2022 -0800

    New translations messages.xlf (Russian)
    [ci skip]

commit e88755e7ac
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:04 2022 -0800

    New translations messages.xlf (Slovenian)
    [ci skip]

commit c582947291
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:02 2022 -0800

    New translations messages.xlf (Swedish)
    [ci skip]

commit 98fe3a2cb7
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:58:01 2022 -0800

    New translations messages.xlf (Turkish)
    [ci skip]

commit 61647606fa
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:57:59 2022 -0800

    New translations messages.xlf (Chinese Simplified)
    [ci skip]

commit 95a1e5c645
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:57:58 2022 -0800

    New translations messages.xlf (Portuguese, Brazilian)
    [ci skip]

commit 8ead77f128
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:57:57 2022 -0800

    New translations messages.xlf (Croatian)
    [ci skip]

commit b9e9e82f33
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:57:55 2022 -0800

    New translations messages.xlf (Luxembourgish)
    [ci skip]

commit 487fd3a5dd
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:57:54 2022 -0800

    New translations messages.xlf (Polish)
    [ci skip]

commit 657786a2fe
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sun Nov 27 17:57:52 2022 -0800

    New translations messages.xlf (German)
    [ci skip]

commit e74d7dadfb
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 27 17:43:46 2022 -0800

    Adds the -dev back to the UI version

commit a2937cd54d
Merge: 9b01aa92 7b3ce628
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 27 17:42:41 2022 -0800

    Merge branch 'main' into dev

commit 7b3ce6289f
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 27 17:21:12 2022 -0800

    Bumps version number to 1.10.0

commit a16e8324be
Merge: 34a0111f 39de531d
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 27 17:18:08 2022 -0800

    Merge pull request #1960 from paperless-ngx/beta

    [Beta] Paperless-ngx v1.10.0 Release Candidate

commit 39de531df5
Merge: 4764d4fd c9d6c208
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Nov 27 14:47:14 2022 -0800

    Merge pull request #2041 from paperless-ngx/l10n_dev

    New Crowdin updates

commit 4764d4fd2b
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Nov 26 12:28:51 2022 -0800

    New translations django.po (German)
    [ci skip]

commit e147d4571f
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Nov 26 09:54:00 2022 -0800

    New translations django.po (German)
    [ci skip]

commit dc9aaa6472
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Tue Nov 22 15:21:31 2022 -0800

    New translations django.po (German)
    [ci skip]

commit 8a061c4ac2
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Tue Nov 22 15:21:30 2022 -0800

    New translations messages.xlf (German)
    [ci skip]

commit d051c5c282
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Nov 12 08:48:48 2022 -0800

    Remove ar-SA

commit 9e60810a8b
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Nov 12 08:33:08 2022 -0800

    New translations messages.xlf (Arabic)
    [ci skip]

commit 96ee7990b2
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Fri Nov 11 13:59:54 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit 224bfeb72e
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Fri Nov 11 13:04:01 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit f0497e7744
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 27 08:28:22 2022 -0800

    Fixes how a language code like chi-sim is treated in the checks

commit c9d6c208af
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Nov 26 12:28:51 2022 -0800

    New translations django.po (German)
    [ci skip]

commit 9f2b8b1734
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Nov 26 09:54:00 2022 -0800

    New translations django.po (German)
    [ci skip]

commit a04b9e3755
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Tue Nov 22 15:21:31 2022 -0800

    New translations django.po (German)
    [ci skip]

commit a81d4c5e9d
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Tue Nov 22 15:21:30 2022 -0800

    New translations messages.xlf (German)
    [ci skip]

commit 2140d42098
Merge: a5283525 2a5dc4de
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 22 14:35:34 2022 -0800

    Merge pull request #2025 from paperless-ngx/fix-redo-ocr-message

    Add info that re-do OCR doesnt automatically refresh content

commit 43325371fc
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Nov 12 08:48:48 2022 -0800

    Remove ar-SA

commit d10721089e
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Nov 12 08:33:08 2022 -0800

    New translations messages.xlf (Arabic)
    [ci skip]

commit f1a1a2da8b
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Fri Nov 11 13:59:54 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit 612e0a1163
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Fri Nov 11 13:04:01 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit 2a5dc4de38
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 22 14:16:04 2022 -0800

    Add info that re-do OCR doesnt automatically refresh content

commit a5283525bc
Merge: f0155565 de98d748
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Nov 22 13:53:08 2022 -0800

    Merge pull request #2023 from paperless-ngx/fix/2019-create-date

    Bugfix: Fix created_date being a string

commit de98d748a9
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Nov 22 10:11:27 2022 -0800

    If override_date is provided, coerce it into a datetime

commit f015556562
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Mon Nov 21 14:56:14 2022 -0800

    Adds a test to cover this edge case

commit b897d6de2e
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Mon Nov 21 14:45:20 2022 -0800

    Don't use the sidecar file when redoing the OCR, it only contains new text

commit 54f20b381e
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Mon Nov 21 12:59:14 2022 -0800

    Documents some issues and the required manual fixes for MariaDB

commit c0d4248021
Merge: 27f7f0a9 870e295a
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Nov 21 22:19:02 2022 -0800

    Merge pull request #1973 from paperless-ngx/l10n_dev

    New Crowdin updates

commit 870e295aae
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Mon Nov 21 21:43:54 2022 -0800

    New translations messages.xlf (German)
    [ci skip]

commit 4aa318598f
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 23:26:20 2022 +0100

    add test comments

commit 00f39d8b58
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 22:49:42 2022 +0100

    add test comments

commit 0b1a16908f
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 20:33:07 2022 +0100

    Include .eml reference in docs

commit d9796e5003
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 20:24:36 2022 +0100

    change order of elements in parsed Texts

commit 3599bb52c0
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 20:12:41 2022 +0100

    minor test improvements

commit af8a6c3764
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 19:53:57 2022 +0100

    fix filenames

commit 6d37ebf79e
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 20 09:15:06 2022 -0800

    Fixes one more place which used manual size formatting

commit f6a70b85f4
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 20 09:13:08 2022 -0800

    Use Django templating engine

commit 538a4219bd
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 20 09:10:44 2022 -0800

    Fixes missing return

commit 85c41b79be
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Sun Nov 20 08:02:06 2022 -0800

    Adds the new packages without updating other dependencies

commit 25d014d8ef
Merge: 9ec89762 27f7f0a9
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Nov 20 07:48:55 2022 -0800

    Merge branch 'beta' into l10n_dev

commit 9b01aa9202
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Mon Nov 14 15:47:22 2022 -0800

    Fixes the link for flake8 to the new (?) GitHub repo

commit df101f5e7a
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 16:09:46 2022 +0100

    split handle_message function

commit 1fa735eb23
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 15:44:43 2022 +0100

    use imagehash instead of bitwise hashing

commit ebe21a0114
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 14:22:30 2022 +0100

    eml parsing requires tika

commit d132eba143
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 12:48:03 2022 +0100

    optimize regex

commit 073c3c8fed
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 12:36:49 2022 +0100

    use html.escape instead of some self build functions

commit e3c1bde793
Author: phail <phail@hacknology.de>
Date:   Sun Nov 20 12:06:35 2022 +0100

    remove log mocking, replace pytest raises, use humanfriendly

commit 27f7f0a941
Merge: 9f5fd6c3 914661fd
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Nov 17 14:31:46 2022 -0800

    Merge pull request #1998 from paperless-ngx/fix/1993-date-overflow

    Bugfix: Don't allow exceptions during date parsing to fail consume

commit 9f5fd6c3ba
Merge: 0ae82005 3dfeee93
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Nov 17 14:30:23 2022 -0800

    Merge pull request #1967 from paperless-ngx/feature-scripts-output

    Feature: Capture stdout & stderr of the pre/post consume scripts

commit 914661fdbb
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Thu Nov 17 13:37:37 2022 -0800

    Don't allow an exception when trying to parse a date cause complete failure

commit 0ae8200593
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Mon Nov 14 15:47:22 2022 -0800

    Fixes the link for flake8 to the new (?) GitHub repo

commit b68906b14e
Author: phail <phail@hacknology.de>
Date:   Sun Nov 13 22:49:52 2022 +0100

    merge pipfile

commit 681eecc46e
Merge: 1578e8de d4712234
Author: phail <phail@hacknology.de>
Date:   Sun Nov 13 22:43:55 2022 +0100

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit 1578e8de2d
Author: phail <phail@hacknology.de>
Date:   Sun Nov 13 22:33:26 2022 +0100

    fix live tests

commit 023c931401
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Nov 13 07:11:45 2022 -0800

    Fix top search not working due to missing button type

commit 9ec89762a3
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Nov 12 09:31:54 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit fa47595ac8
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Nov 12 08:58:01 2022 -0800

    remove ar_SA

    [ci skip]

commit 79f5019b40
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Sat Nov 12 08:53:52 2022 -0800

    New Crowdin updates (#1971)

    * New translations messages.xlf (Serbian (Latin))
    [ci skip]

    * New translations messages.xlf (Serbian (Latin))
    [ci skip]

    * New translations messages.xlf (Italian)
    [ci skip]

    * New translations django.po (Italian)
    [ci skip]

    * New translations django.po (Serbian (Latin))
    [ci skip]

    * New translations messages.xlf (Russian)
    [ci skip]

    * New translations messages.xlf (Polish)
    [ci skip]

    * New translations messages.xlf (Serbian (Latin))
    [ci skip]

    * New translations messages.xlf (Luxembourgish)
    [ci skip]

    * New translations messages.xlf (Croatian)
    [ci skip]

    * New translations messages.xlf (Portuguese, Brazilian)
    [ci skip]

    * New translations messages.xlf (Chinese Simplified)
    [ci skip]

    * New translations messages.xlf (Turkish)
    [ci skip]

    * New translations messages.xlf (Swedish)
    [ci skip]

    * New translations messages.xlf (Slovenian)
    [ci skip]

    * New translations messages.xlf (Portuguese)
    [ci skip]

    * New translations messages.xlf (Norwegian)
    [ci skip]

    * New translations messages.xlf (German)
    [ci skip]

    * New translations messages.xlf (Dutch)
    [ci skip]

    * New translations messages.xlf (Italian)
    [ci skip]

    * New translations messages.xlf (Hebrew)
    [ci skip]

    * New translations messages.xlf (Finnish)
    [ci skip]

    * New translations messages.xlf (Danish)
    [ci skip]

    * New translations messages.xlf (Czech)
    [ci skip]

    * New translations messages.xlf (Belarusian)
    [ci skip]

    * New translations messages.xlf (Spanish)
    [ci skip]

    * New translations messages.xlf (French)
    [ci skip]

    * New translations messages.xlf (Romanian)
    [ci skip]

    * New translations messages.xlf (Arabic)
    [ci skip]

    * Remove ar-SA

    * remote ar other than ar-ar

    Co-authored-by: Michael Shamoon <4887959+shamoon@users.noreply.github.com>

commit 756ce2f9d8
Merge: 50a211f3 d4712234
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Nov 12 08:33:43 2022 -0800

    Merge branch 'dev' into beta

commit d47122340a
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Nov 12 08:31:25 2022 -0800

    Add translation strings for welcome tour buttons

commit b01cbc9aa0
Author: phail <phail@hacknology.de>
Date:   Sat Nov 12 15:48:30 2022 +0100

    add conditions to unittests

commit 3dfeee9332
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Fri Nov 11 10:09:56 2022 -0800

    Don't do decoding work if not needed

commit 057f6016cc
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Fri Nov 11 08:58:49 2022 -0800

    Adds further testing to cover scripts with non-zero exit codes

commit c4965580de
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Thu Nov 10 17:40:36 2022 -0800

    Fixes stderr appearing to have content when it doesn't

commit 9a47963fd5
Author: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
Date:   Wed Nov 9 20:11:36 2022 -0800

    Captures the stdout and stderr of the pre/post scripts into the log

commit 50a211f367
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Thu Nov 10 17:25:39 2022 -0800

    Fixes an issue with the install of languages and read-only variable

commit 5f278d7fbb
Merge: e5106bdc a17d2519
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Nov 9 21:37:29 2022 -0800

    Merge pull request #1956 from paperless-ngx/l10n_dev

    New Crowdin updates

commit a17d251913
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:37 2022 -0800

    New translations django.po (Serbian (Latin))
    [ci skip]

commit 1cbf088656
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:36 2022 -0800

    New translations messages.xlf (Serbian (Latin))
    [ci skip]

commit d3254d6bcf
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:35 2022 -0800

    New translations messages.xlf (Luxembourgish)
    [ci skip]

commit 1543729c7b
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:34 2022 -0800

    New translations messages.xlf (Croatian)
    [ci skip]

commit ef2a96c34b
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:32 2022 -0800

    New translations messages.xlf (Portuguese, Brazilian)
    [ci skip]

commit 656b1e150f
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:31 2022 -0800

    New translations messages.xlf (Chinese Simplified)
    [ci skip]

commit e0f61003cf
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:30 2022 -0800

    New translations messages.xlf (Turkish)
    [ci skip]

commit 1ca98678cd
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:28 2022 -0800

    New translations messages.xlf (Swedish)
    [ci skip]

commit 9919cc1956
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:27 2022 -0800

    New translations messages.xlf (Slovenian)
    [ci skip]

commit d2096e3c05
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:26 2022 -0800

    New translations messages.xlf (Portuguese)
    [ci skip]

commit 5f2b508b7a
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:24 2022 -0800

    New translations messages.xlf (Polish)
    [ci skip]

commit 752d4f4249
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:23 2022 -0800

    New translations django.po (German)
    [ci skip]

commit 72e7d5150e
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:22 2022 -0800

    New translations messages.xlf (Norwegian)
    [ci skip]

commit 42a9e05a7f
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:21 2022 -0800

    New translations messages.xlf (Italian)
    [ci skip]

commit b4add2ed55
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:19 2022 -0800

    New translations messages.xlf (Hebrew)
    [ci skip]

commit ed7d9295bd
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:18 2022 -0800

    New translations messages.xlf (Finnish)
    [ci skip]

commit 5b7b1b2349
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:16 2022 -0800

    New translations messages.xlf (Danish)
    [ci skip]

commit d5c930acc9
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:15 2022 -0800

    New translations messages.xlf (Czech)
    [ci skip]

commit 4c93d6d7e6
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:14 2022 -0800

    New translations messages.xlf (Belarusian)
    [ci skip]

commit 066f3264fb
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:12 2022 -0800

    New translations messages.xlf (Spanish)
    [ci skip]

commit 88a803f949
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:11 2022 -0800

    New translations messages.xlf (French)
    [ci skip]

commit e69615dc06
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:10 2022 -0800

    New translations messages.xlf (Romanian)
    [ci skip]

commit a1e0840e24
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:08 2022 -0800

    New translations messages.xlf (Dutch)
    [ci skip]

commit d814353e83
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:07 2022 -0800

    New translations messages.xlf (German)
    [ci skip]

commit 06d7845eca
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:04 2022 -0800

    New translations django.po (Dutch)
    [ci skip]

commit ae8682c7a5
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:03 2022 -0800

    New translations django.po (Romanian)
    [ci skip]

commit c9c0b3d430
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:02 2022 -0800

    New translations django.po (Luxembourgish)
    [ci skip]

commit cc46fc7e4b
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:01 2022 -0800

    New translations django.po (Croatian)
    [ci skip]

commit d1b1ba21cd
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:12:00 2022 -0800

    New translations django.po (Portuguese, Brazilian)
    [ci skip]

commit a009417a99
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:59 2022 -0800

    New translations django.po (Chinese Simplified)
    [ci skip]

commit 775da720ec
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:58 2022 -0800

    New translations django.po (Turkish)
    [ci skip]

commit aeae6ea0d3
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:57 2022 -0800

    New translations django.po (Swedish)
    [ci skip]

commit 0ae46d2269
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:56 2022 -0800

    New translations django.po (Slovenian)
    [ci skip]

commit 0e7f1ec0de
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:55 2022 -0800

    New translations django.po (Russian)
    [ci skip]

commit 13cd55b96f
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:53 2022 -0800

    New translations django.po (Portuguese)
    [ci skip]

commit 9139e807ec
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:52 2022 -0800

    New translations messages.xlf (Arabic)
    [ci skip]

commit 53616f6625
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:51 2022 -0800

    New translations django.po (Polish)
    [ci skip]

commit 526fdf1153
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:50 2022 -0800

    New translations django.po (Italian)
    [ci skip]

commit fc4aceb0ee
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:49 2022 -0800

    New translations django.po (Hebrew)
    [ci skip]

commit 3d8421b718
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:48 2022 -0800

    New translations django.po (Finnish)
    [ci skip]

commit 6cebceda15
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:47 2022 -0800

    New translations django.po (Danish)
    [ci skip]

commit e1fd6bda19
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:46 2022 -0800

    New translations django.po (Czech)
    [ci skip]

commit fd34414b17
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:45 2022 -0800

    New translations django.po (Belarusian)
    [ci skip]

commit 3ce1886a54
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:44 2022 -0800

    New translations django.po (Arabic)
    [ci skip]

commit 8ed43779a8
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:43 2022 -0800

    New translations django.po (Spanish)
    [ci skip]

commit a7949b3e22
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:42 2022 -0800

    New translations django.po (French)
    [ci skip]

commit 19c293c3e6
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:41 2022 -0800

    New translations django.po (Norwegian)
    [ci skip]

commit ccb1ec4ff5
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Wed Nov 9 15:11:40 2022 -0800

    New translations messages.xlf (Russian)
    [ci skip]

commit e5106bdca0
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Wed Nov 9 14:00:09 2022 -0800

    Updates the version strings to 1.10.0

commit ba1366f49a
Merge: 34a0111f f3b3db30
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Wed Nov 9 13:51:10 2022 -0800

    Merge branch 'dev' into beta

commit acd3832417
Author: phail <phail@hacknology.de>
Date:   Thu Nov 3 21:08:15 2022 +0100

    merge Pipfile.lock

commit 82b2ba3cc2
Merge: 3de6e0bc 7e3e0a0f
Author: phail <phail@hacknology.de>
Date:   Thu Nov 3 21:00:01 2022 +0100

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit 3de6e0bcf1
Author: phail <phail@hacknology.de>
Date:   Thu Nov 3 00:58:36 2022 +0100

    put parser into setup
    make test using convert optional
    Gotenberg live testing

commit 34a0111ff5
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Oct 31 13:06:17 2022 -0700

    update logs section

commit b511b084d0
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Oct 30 06:48:41 2022 -0700

    Update matrix url

    [ci skip]

commit 6df73ae940
Author: phail <phail@hacknology.de>
Date:   Sat Oct 29 23:20:35 2022 +0200

    gotenberg with modified cmd

commit 4a24ba51c5
Merge: 87472b31 d5fb98b7
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Oct 28 22:22:08 2022 -0700

    Merge pull request #1876 from astubenbord/main

    Added new application to list of affiliated projects

commit d5fb98b7c4
Author: Anton Stubenbord <79228196+astubenbord@users.noreply.github.com>
Date:   Fri Oct 28 11:07:42 2022 +0200

    Added new application to list of affiliated projects

commit 2204090151
Author: phail <phail@hacknology.de>
Date:   Thu Oct 27 23:53:47 2022 +0200

    fix string

commit 3c81a7468b
Author: phail <phail@hacknology.de>
Date:   Thu Oct 27 23:41:29 2022 +0200

    replace thumbnail creation with mock

commit 5ef86f9489
Merge: 90cb0836 9b82ab95
Author: phail <phail@hacknology.de>
Date:   Thu Oct 27 23:12:51 2022 +0200

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit 90cb0836bb
Author: phail <phail@hacknology.de>
Date:   Thu Oct 27 23:11:41 2022 +0200

    Downgrade pdf validation to text only

commit ef1d4264b5
Author: phail <phail@hacknology.de>
Date:   Thu Oct 27 00:27:15 2022 +0200

    improve test coverage a little

commit e1fa59122d
Merge: 5bf26369 3357fa19
Author: phail <phail@hacknology.de>
Date:   Wed Oct 26 20:59:49 2022 +0200

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit 5bf26369e2
Author: phail <phail@hacknology.de>
Date:   Tue Oct 25 21:17:40 2022 +0200

    remove erroring paramerter

commit 36239ba09f
Author: phail <phail@hacknology.de>
Date:   Mon Oct 24 22:15:33 2022 +0200

    rename help text

commit 318c1d2fbd
Merge: e7c40fc3 f8ce6285
Author: phail <phail@hacknology.de>
Date:   Mon Oct 24 21:12:35 2022 +0200

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit e7c40fc3dc
Author: phail <phail@hacknology.de>
Date:   Sun Oct 23 22:02:11 2022 +0200

    Update Pipfile

commit 0da0b1c062
Author: phail <phail@hacknology.de>
Date:   Sun Oct 23 21:39:15 2022 +0200

    update variable names

commit 08988e11f8
Merge: 30372b0e 8be6c707
Author: phail <phail@hacknology.de>
Date:   Sun Oct 23 20:37:22 2022 +0200

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit 30372b0e85
Author: phail <phail@hacknology.de>
Date:   Sun Oct 23 17:18:10 2022 +0200

    add tests for mail_to_html and generate_pdf_from_mail

commit 567e89d1c7
Author: phail <phail@hacknology.de>
Date:   Sat Oct 22 02:25:23 2022 +0200

    test for broken eml, add test_generate_pdf

commit f1f5227ccd
Author: phail <phail@hacknology.de>
Date:   Sat Oct 22 00:44:32 2022 +0200

    add unittest for external images

commit 09b5bd17f2
Author: phail <phail@hacknology.de>
Date:   Wed Oct 19 23:19:33 2022 +0200

    add unittest for generate_pdf_from_html

commit e384bd78c5
Author: phail <phail@hacknology.de>
Date:   Tue Oct 18 23:48:07 2022 +0200

    add unittest for transform_inline_html

commit fda844f64c
Author: phail <phail@hacknology.de>
Date:   Sat Oct 15 15:41:43 2022 +0200

    add unittest for parse

commit daf90399bd
Author: phail <phail@hacknology.de>
Date:   Sat Oct 15 13:13:29 2022 +0200

    Add unitest for tika_parse()

commit 3d37e49c1a
Author: phail <phail@hacknology.de>
Date:   Fri Oct 14 15:43:43 2022 +0200

    add 2 more tests

commit 261c6fb990
Author: phail <phail@hacknology.de>
Date:   Thu Oct 13 01:03:09 2022 +0200

    add unittest for get_thumbnail

commit 87472b31d2
Merge: 430c5c3b 1024d7e6
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Oct 11 19:59:46 2022 -0700

    Merge pull request #1780 from paperless-ngx/fix/issue-1647

    Documentation: Add note re MS exchange servers

commit 1024d7e6e2
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Oct 11 15:12:22 2022 -0700

    Add note re MS exchange servers

commit 430c5c3b87
Merge: 0b5c6d35 b7c33550
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Oct 7 23:45:51 2022 -0700

    Merge pull request #1761 from paperless-ngx/docs/lsio-tweak

    Documentation: Tweak LinuxServer

commit b7c335507f
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Thu Oct 6 18:24:25 2022 -0700

    Fixes the LSIO migration setting for the media root

commit 0b5c6d3532
Merge: fdac108c 5fd39472
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Oct 6 13:22:58 2022 -0700

    Merge pull request #1731 from paperless-ngx/fix/1624

    Documentation: Adds troubleshooting note about Kubernetes and ports

commit fdac108cab
Merge: 5639659b 821c14fb
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Oct 3 17:07:06 2022 -0700

    Merge pull request #1733 from paperless-ngx/docs-lsio-migrate

    Documentation: LinuxServer.io Migration

commit 821c14fbce
Author: Trenton H <holmes.trenton@gmail.com>
Date:   Mon Oct 3 11:14:18 2022 -0700

    Corrects how the link to example compose files looks

commit 8c03d9c638
Author: Trenton H <holmes.trenton@gmail.com>
Date:   Mon Oct 3 11:11:51 2022 -0700

    Corrects a re-numbered step

commit 174a609449
Author: Trenton H <holmes.trenton@gmail.com>
Date:   Mon Oct 3 10:16:53 2022 -0700

    Adds a few steps for migration from the LinuxServer.io image

commit 5fd394726e
Author: Trenton H <holmes.trenton@gmail.com>
Date:   Mon Oct 3 09:01:07 2022 -0700

    Adds troubleshooting note for Kubernetes about needing to set the port again

commit 5639659b63
Merge: 807b7130 7ba9cdbe
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Sep 28 09:48:20 2022 -0700

    Merge pull request #1683 from paperless-ngx/fix/issue-1660

    Mariadb compose files should use `PAPERLESS_DBPASS`

commit 7ba9cdbe23
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Sep 28 09:07:30 2022 -0700

    Mariadb compose files should use `PAPERLESS_DBPASS`

commit 807b7130e5
Merge: 9d117ee1 e2d593c0
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Sep 27 16:50:23 2022 -0700

    Merge pull request #1671 from paperless-ngx/v1.9.2-changelog

    [Documentation] Add v1.9.2 changelog

commit e2d593c023
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Sep 27 10:04:41 2022 -0700

    Fix formatting, add note about 1.9.1 version string

commit 7455963124
Author: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Date:   Tue Sep 27 17:01:37 2022 +0000

    Changelog  - GHA

commit cdd2b99b6b
Merge: d0a0ae91 72ce4405
Author: phail <phail@hacknology.de>
Date:   Mon Jul 11 23:58:21 2022 +0200

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit d0a0ae91c4
Merge: c04b9fd7 feaf2da8
Author: phail <phail@hacknology.de>
Date:   Fri May 20 19:29:52 2022 +0200

    Merge branch 'dev' into feature-consume-eml

commit c04b9fd7f6
Author: phail <phail@hacknology.de>
Date:   Thu May 19 22:51:17 2022 +0200

    switch to From: Header instead of date to compensate for older libmagic versions

commit 6809b15ce1
Author: phail <phail@hacknology.de>
Date:   Sat May 14 16:47:12 2022 +0200

    workaround for wrong mime detection of .eml files

commit c317eca1ca
Author: phail <phel@hacknology.de>
Date:   Thu May 5 00:20:32 2022 +0200

    add attachment size to pdf

commit 466afa8203
Author: phail <phel@hacknology.de>
Date:   Wed May 4 23:42:59 2022 +0200

    fix consumption of mails without html
    split pdf generation functions

commit c2e3dc76d9
Author: phail <phel@hacknology.de>
Date:   Tue May 3 23:21:33 2022 +0200

    add parsing of inline attachments
    remove insecure chromium option
    fix html parsing

commit 5a899664f8
Author: phail <phel@hacknology.de>
Date:   Tue May 3 18:02:08 2022 +0200

    remove .eml parser from tika

commit 990e905a04
Merge: 6b7155a8 98ebb095
Author: phail <phel@hacknology.de>
Date:   Tue May 3 17:42:56 2022 +0200

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit 6b7155a849
Author: phail <phel@hacknology.de>
Date:   Sat Apr 30 17:33:12 2022 +0200

    merge migrations

commit 47851ddd3f
Merge: 47189643 0f1e3164
Author: phail <phel@hacknology.de>
Date:   Fri Apr 29 23:52:56 2022 +0200

    Merge remote-tracking branch 'paperless/dev' into feature-consume-eml

commit 47189643ff
Author: phail <phel@hacknology.de>
Date:   Fri Apr 29 22:58:11 2022 +0200

    add eml parser to paperless_mail

commit c1efe11cf3
Author: phail <phel@hacknology.de>
Date:   Wed Apr 27 23:32:10 2022 +0200

    improve pdf generation

commit 0e40ef5f35
Author: phail <phel@hacknology.de>
Date:   Wed Apr 27 19:52:59 2022 +0200

    add css for pdf generation

commit c8081595c4
Author: phail <phel@hacknology.de>
Date:   Tue Apr 26 23:25:48 2022 +0200

    improve pdf generation

commit a2b5b3b253
Author: phail <phel@hacknology.de>
Date:   Tue Apr 26 23:12:36 2022 +0200

    moved files

commit 790bcf05ed
Author: phail <phel@hacknology.de>
Date:   Mon Apr 25 20:55:00 2022 +0200

    add prototype archive pdf

commit d8d2d53c59
Author: phail <phel@hacknology.de>
Date:   Tue Apr 19 20:14:31 2022 +0200

    fix Mail actions mixup

commit 027897ff03
Author: phail <phel@hacknology.de>
Date:   Tue Apr 19 00:39:00 2022 +0200

    work in progress Mail parsing

commit cca576f518
Author: phail <phel@hacknology.de>
Date:   Fri Apr 15 14:40:02 2022 +0200

    add feature to consume imap mail als .eml

commit 5fcf1b5434
Author: phail <phel@hacknology.de>
Date:   Thu Apr 14 00:19:30 2022 +0200

    remove uneeded print and fix merge fail

commit 942b5aa9df
Merge: c05b39a0 cc936160
Author: phail <phel@hacknology.de>
Date:   Wed Apr 13 23:55:38 2022 +0200

    Merge branch 'dev' into fix-mail-starttls

commit c05b39a056
Author: phail <phel@hacknology.de>
Date:   Wed Apr 13 23:37:21 2022 +0200

    fix unittest

commit 3c8196527f
Author: phail <phel@hacknology.de>
Date:   Sat Apr 9 13:07:14 2022 +0200

    adapt to starttls interface change in imap_tools
    pin imap-tools version to avoid breaking changes
    improve mail log

Update settings.component.ts

Update settings.component.ts
This commit is contained in:
Michael Shamoon
2022-12-05 01:25:11 -08:00
parent 4279ba13e9
commit b21970fd53
229 changed files with 39451 additions and 16009 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
src/paperless_mail/admin.py
View File

@@ -56,6 +56,7 @@ class MailRuleAdmin(admin.ModelAdmin):
"filter_body",
"filter_attachment_filename",
"maximum_age",
"consumption_scope",
"attachment_type",
),
},
@@ -65,8 +66,8 @@ class MailRuleAdmin(admin.ModelAdmin):
{
"description": _(
"The action applied to the mail. This action is only "
"performed when documents were consumed from the mail. "
"Mails without attachments will remain entirely untouched.",
"performed when the mail body or attachments were "
"consumed from the mail.",
),
"fields": ("action", "action_parameter"),
},

9
src/paperless_mail/apps.py
View File

@@ -1,8 +1,17 @@
from django.apps import AppConfig
from django.conf import settings
from django.utils.translation import gettext_lazy as _
from paperless_mail.signals import mail_consumer_declaration
class PaperlessMailConfig(AppConfig):
name = "paperless_mail"
verbose_name = _("Paperless mail")
def ready(self):
from documents.signals import document_consumer_declaration
if settings.TIKA_ENABLED:
document_consumer_declaration.connect(mail_consumer_declaration)
AppConfig.ready(self)

104
src/paperless_mail/mail.py
View File

@@ -350,9 +350,16 @@ class MailAccountHandler(LoggingMixin):
return total_processed_files
def handle_message(self, message, rule) -> int:
if not message.attachments:
return 0
def handle_message(self, message, rule: MailRule) -> int:
processed_elements = 0
# Skip Message handling when only attachments are to be processed but
# message doesn't have any.
if (
not message.attachments
and rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
):
return processed_elements
self.log(
"debug",
@@ -365,8 +372,41 @@ class MailAccountHandler(LoggingMixin):
tag_ids = [tag.id for tag in rule.assign_tags.all()]
doc_type = rule.assign_document_type
processed_attachments = 0
if (
rule.consumption_scope == MailRule.ConsumptionScope.EML_ONLY
or rule.consumption_scope == MailRule.ConsumptionScope.EVERYTHING
):
processed_elements += self.process_eml(
message,
rule,
correspondent,
tag_ids,
doc_type,
)
if (
rule.consumption_scope == MailRule.ConsumptionScope.ATTACHMENTS_ONLY
or rule.consumption_scope == MailRule.ConsumptionScope.EVERYTHING
):
processed_elements += self.process_attachments(
message,
rule,
correspondent,
tag_ids,
doc_type,
)
return processed_elements
def process_attachments(
self,
message: MailMessage,
rule: MailRule,
correspondent,
tag_ids,
doc_type,
):
processed_attachments = 0
for att in message.attachments:
if (
@@ -436,5 +476,59 @@ class MailAccountHandler(LoggingMixin):
f"since guessed mime type {mime_type} is not supported "
f"by paperless",
)
return processed_attachments
def process_eml(
self,
message: MailMessage,
rule: MailRule,
correspondent,
tag_ids,
doc_type,
):
os.makedirs(settings.SCRATCH_DIR, exist_ok=True)
_, temp_filename = tempfile.mkstemp(
prefix="paperless-mail-",
dir=settings.SCRATCH_DIR,
suffix=".eml",
)
with open(temp_filename, "wb") as f:
# Move "From"-header to beginning of file
# TODO: This ugly workaround is needed because the parser is
# chosen only by the mime_type detected via magic
# (see documents/consumer.py "mime_type = magic.from_file")
# Unfortunately magic sometimes fails to detect the mime
# type of .eml files correctly as message/rfc822 and instead
# detects text/plain.
# This also effects direct file consumption of .eml files
# which are not treated with this workaround.
from_element = None
for i, header in enumerate(message.obj._headers):
if header[0] == "From":
from_element = i
if from_element:
new_headers = [message.obj._headers.pop(from_element)]
new_headers += message.obj._headers
message.obj._headers = new_headers
f.write(message.obj.as_bytes())
self.log(
"info",
f"Rule {rule}: "
f"Consuming eml from mail "
f"{message.subject} from {message.from_}",
)
consume_file.delay(
path=temp_filename,
override_filename=pathvalidate.sanitize_filename(
message.subject + ".eml",
),
override_title=message.subject,
override_correspondent_id=correspondent.id if correspondent else None,
override_document_type_id=doc_type.id if doc_type else None,
override_tag_ids=tag_ids,
)
processed_elements = 1
return processed_elements

32
src/paperless_mail/migrations/0016_mailrule_consumption_scope.py Normal file
View File

@@ -0,0 +1,32 @@
# Generated by Django 4.0.4 on 2022-07-11 22:02
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("paperless_mail", "0015_alter_mailrule_action"),
]
operations = [
migrations.AddField(
model_name="mailrule",
name="consumption_scope",
field=models.PositiveIntegerField(
choices=[
(1, "Only process attachments."),
(
2,
"Process full Mail (with embedded attachments in file) as .eml",
),
(
3,
"Process full Mail (with embedded attachments in file) as .eml + process attachments as separate documents",
),
],
default=1,
verbose_name="consumption scope",
),
),
]

14
src/paperless_mail/models.py
View File

@@ -56,6 +56,14 @@ class MailRule(models.Model):
verbose_name = _("mail rule")
verbose_name_plural = _("mail rules")
class ConsumptionScope(models.IntegerChoices):
ATTACHMENTS_ONLY = 1, _("Only process attachments.")
EML_ONLY = 2, _("Process full Mail (with embedded attachments in file) as .eml")
EVERYTHING = 3, _(
"Process full Mail (with embedded attachments in file) as .eml "
"+ process attachments as separate documents",
)
class AttachmentProcessing(models.IntegerChoices):
ATTACHMENTS_ONLY = 1, _("Only process attachments.")
EVERYTHING = 2, _("Process all files, including 'inline' " "attachments.")
@@ -145,6 +153,12 @@ class MailRule(models.Model):
),
)
consumption_scope = models.PositiveIntegerField(
_("consumption scope"),
choices=ConsumptionScope.choices,
default=ConsumptionScope.ATTACHMENTS_ONLY,
)
action = models.PositiveIntegerField(
_("action"),
choices=MailAction.choices,

333
src/paperless_mail/parsers.py Normal file
View File

@@ -0,0 +1,333 @@
import os
import re
from html import escape
from io import BytesIO
from io import StringIO
import requests
from bleach import clean
from bleach import linkify
from django.conf import settings
from documents.parsers import DocumentParser
from documents.parsers import make_thumbnail_from_pdf
from documents.parsers import ParseError
from humanfriendly import format_size
from imap_tools import MailMessage
from tika import parser
class MailDocumentParser(DocumentParser):
"""
This parser uses imap_tools to parse .eml files, generates pdf using
gotenbergs and sends the html part to a local tika server for text extraction.
"""
gotenberg_server = settings.TIKA_GOTENBERG_ENDPOINT
tika_server = settings.TIKA_ENDPOINT
logging_name = "paperless.parsing.mail"
_parsed = None
def get_parsed(self, document_path) -> MailMessage:
if not self._parsed:
try:
with open(document_path, "rb") as eml:
self._parsed = MailMessage.from_bytes(eml.read())
except Exception as err:
raise ParseError(
f"Could not parse {document_path}: {err}",
)
if not self._parsed.from_values:
self._parsed = None
raise ParseError(
f"Could not parse {document_path}: Missing 'from'",
)
return self._parsed
def get_thumbnail(self, document_path, mime_type, file_name=None):
if not self.archive_path:
self.archive_path = self.generate_pdf(document_path)
return make_thumbnail_from_pdf(
self.archive_path,
self.tempdir,
self.logging_group,
)
def extract_metadata(self, document_path, mime_type):
result = []
try:
mail = self.get_parsed(document_path)
except ParseError as e:
self.log(
"warning",
f"Error while fetching document metadata for " f"{document_path}: {e}",
)
return result
for key, value in mail.headers.items():
value = ", ".join(i for i in value)
result.append(
{
"namespace": "",
"prefix": "header",
"key": key,
"value": value,
},
)
result.append(
{
"namespace": "",
"prefix": "",
"key": "attachments",
"value": ", ".join(
f"{attachment.filename}"
f"({format_size(attachment.size, binary=True)})"
for attachment in mail.attachments
),
},
)
result.append(
{
"namespace": "",
"prefix": "",
"key": "date",
"value": mail.date.strftime("%Y-%m-%d %H:%M:%S %Z"),
},
)
result.sort(key=lambda item: (item["prefix"], item["key"]))
return result
def parse(self, document_path, mime_type, file_name=None):
def strip_text(text: str):
text = re.sub(r"\s+", " ", text)
text = re.sub(r"(\n *)+", "\n", text)
return text.strip()
mail = self.get_parsed(document_path)
self.text = f"Subject: {mail.subject}\n\n"
self.text += f"From: {mail.from_values.full}\n\n"
self.text += f"To: {', '.join(address.full for address in mail.to_values)}\n\n"
if len(mail.cc_values) >= 1:
self.text += (
f"CC: {', '.join(address.full for address in mail.cc_values)}\n\n"
)
if len(mail.bcc_values) >= 1:
self.text += (
f"BCC: {', '.join(address.full for address in mail.bcc_values)}\n\n"
)
if len(mail.attachments) >= 1:
att = []
for a in mail.attachments:
att.append(f"{a.filename} ({format_size(a.size, binary=True)})")
self.text += f"Attachments: {', '.join(att)}\n\n"
if mail.html != "":
self.text += "HTML content: " + strip_text(self.tika_parse(mail.html))
self.text += f"\n\n{strip_text(mail.text)}"
self.date = mail.date
self.archive_path = self.generate_pdf(document_path)
def tika_parse(self, html: str):
self.log("info", "Sending content to Tika server")
try:
parsed = parser.from_buffer(html, self.tika_server)
except Exception as err:
raise ParseError(
f"Could not parse content with tika server at "
f"{self.tika_server}: {err}",
)
if parsed["content"]:
return parsed["content"]
else:
return ""
def generate_pdf(self, document_path):
pdf_collection = []
url_merge = self.gotenberg_server + "/forms/pdfengines/merge"
pdf_path = os.path.join(self.tempdir, "merged.pdf")
mail = self.get_parsed(document_path)
pdf_collection.append(("1_mail.pdf", self.generate_pdf_from_mail(mail)))
if mail.html == "":
with open(pdf_path, "wb") as file:
file.write(pdf_collection[0][1])
file.close()
return pdf_path
else:
pdf_collection.append(
(
"2_html.pdf",
self.generate_pdf_from_html(mail.html, mail.attachments),
),
)
files = {}
for name, content in pdf_collection:
files[name] = (name, BytesIO(content))
headers = {}
try:
response = requests.post(url_merge, files=files, headers=headers)
response.raise_for_status() # ensure we notice bad responses
except Exception as err:
raise ParseError(f"Error while converting document to PDF: {err}")
with open(pdf_path, "wb") as file:
file.write(response.content)
file.close()
return pdf_path
@staticmethod
def mail_to_html(mail: MailMessage) -> StringIO:
data = {}
def clean_html(text: str):
if isinstance(text, list):
text = "\n".join([str(e) for e in text])
if type(text) != str:
text = str(text)
text = escape(text)
text = clean(text)
text = linkify(text, parse_email=True)
text = text.replace("\n", "<br>")
return text
data["subject"] = clean_html(mail.subject)
if data["subject"] != "":
data["subject_label"] = "Subject"
data["from"] = clean_html(mail.from_values.full)
if data["from"] != "":
data["from_label"] = "From"
data["to"] = clean_html(", ".join(address.full for address in mail.to_values))
if data["to"] != "":
data["to_label"] = "To"
data["cc"] = clean_html(", ".join(address.full for address in mail.cc_values))
if data["cc"] != "":
data["cc_label"] = "CC"
data["bcc"] = clean_html(", ".join(address.full for address in mail.bcc_values))
if data["bcc"] != "":
data["bcc_label"] = "BCC"
att = []
for a in mail.attachments:
att.append(f"{a.filename} ({format_size(a.size, binary=True)})")
data["attachments"] = clean_html(", ".join(att))
if data["attachments"] != "":
data["attachments_label"] = "Attachments"
data["date"] = clean_html(mail.date.astimezone().strftime("%Y-%m-%d %H:%M"))
data["content"] = clean_html(mail.text.strip())
html = StringIO()
from django.template.loader import render_to_string
rendered = render_to_string("email_msg_template.html", context=data)
html.write(rendered)
html.seek(0)
return html
def generate_pdf_from_mail(self, mail):
url = self.gotenberg_server + "/forms/chromium/convert/html"
self.log("info", "Converting mail to PDF")
css_file = os.path.join(os.path.dirname(__file__), "templates/output.css")
with open(css_file, "rb") as css_handle:
files = {
"html": ("index.html", self.mail_to_html(mail)),
"css": ("output.css", css_handle),
}
headers = {}
data = {
"marginTop": "0.1",
"marginBottom": "0.1",
"marginLeft": "0.1",
"marginRight": "0.1",
"paperWidth": "8.27",
"paperHeight": "11.7",
"scale": "1.0",
}
try:
response = requests.post(
url,
files=files,
headers=headers,
data=data,
)
response.raise_for_status() # ensure we notice bad responses
except Exception as err:
raise ParseError(f"Error while converting document to PDF: {err}")
return response.content
@staticmethod
def transform_inline_html(html, attachments):
def clean_html_script(text: str):
compiled_open = re.compile(re.escape("<script"), re.IGNORECASE)
text = compiled_open.sub("<div hidden ", text)
compiled_close = re.compile(re.escape("</script"), re.IGNORECASE)
text = compiled_close.sub("</div", text)
return text
html_clean = clean_html_script(html)
files = []
for a in attachments:
name_cid = "cid:" + a.content_id
name_clean = "".join(e for e in name_cid if e.isalnum())
files.append((name_clean, BytesIO(a.payload)))
html_clean = html_clean.replace(name_cid, name_clean)
files.append(("index.html", StringIO(html_clean)))
return files
def generate_pdf_from_html(self, orig_html, attachments):
url = self.gotenberg_server + "/forms/chromium/convert/html"
self.log("info", "Converting html to PDF")
files = {}
for name, file in self.transform_inline_html(orig_html, attachments):
files[name] = (name, file)
headers = {}
data = {
"marginTop": "0.1",
"marginBottom": "0.1",
"marginLeft": "0.1",
"marginRight": "0.1",
"paperWidth": "8.27",
"paperHeight": "11.7",
"scale": "1.0",
}
try:
response = requests.post(
url,
files=files,
headers=headers,
data=data,
)
response.raise_for_status() # ensure we notice bad responses
except Exception as err:
raise ParseError(f"Error while converting document to PDF: {err}")
return response.content

110
src/paperless_mail/serialisers.py Normal file
View File

@@ -0,0 +1,110 @@
from documents.serialisers import CorrespondentField
from documents.serialisers import DocumentTypeField
from documents.serialisers import TagsField
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from rest_framework import serializers
class ObfuscatedPasswordField(serializers.Field):
"""
Sends *** string instead of password in the clear
"""
def to_representation(self, value):
return "*" * len(value)
def to_internal_value(self, data):
return data
class MailAccountSerializer(serializers.ModelSerializer):
password = ObfuscatedPasswordField()
class Meta:
model = MailAccount
depth = 1
fields = [
"id",
"name",
"imap_server",
"imap_port",
"imap_security",
"username",
"password",
"character_set",
]
def update(self, instance, validated_data):
if "password" in validated_data:
if len(validated_data.get("password").replace("*", "")) == 0:
validated_data.pop("password")
super().update(instance, validated_data)
return instance
def create(self, validated_data):
mail_account = MailAccount.objects.create(**validated_data)
return mail_account
class AccountField(serializers.PrimaryKeyRelatedField):
def get_queryset(self):
return MailAccount.objects.all().order_by("-id")
class MailRuleSerializer(serializers.ModelSerializer):
account = AccountField(required=True)
action_parameter = serializers.CharField(
allow_null=True,
required=False,
default="",
)
assign_correspondent = CorrespondentField(allow_null=True, required=False)
assign_tags = TagsField(many=True, allow_null=True, required=False)
assign_document_type = DocumentTypeField(allow_null=True, required=False)
order = serializers.IntegerField(required=False)
class Meta:
model = MailRule
depth = 1
fields = [
"id",
"name",
"account",
"folder",
"filter_from",
"filter_subject",
"filter_body",
"filter_attachment_filename",
"maximum_age",
"action",
"action_parameter",
"assign_title_from",
"assign_tags",
"assign_correspondent_from",
"assign_correspondent",
"assign_document_type",
"order",
"attachment_type",
]
def update(self, instance, validated_data):
super().update(instance, validated_data)
return instance
def create(self, validated_data):
if "assign_tags" in validated_data:
assign_tags = validated_data.pop("assign_tags")
mail_rule = MailRule.objects.create(**validated_data)
if assign_tags:
mail_rule.assign_tags.set(assign_tags)
return mail_rule
def validate(self, attrs):
if (
attrs["action"] == MailRule.MailAction.TAG
or attrs["action"] == MailRule.MailAction.MOVE
) and attrs["action_parameter"] is None:
raise serializers.ValidationError("An action parameter is required.")
return attrs

14
src/paperless_mail/signals.py Normal file
View File

@@ -0,0 +1,14 @@
def get_parser(*args, **kwargs):
from .parsers import MailDocumentParser
return MailDocumentParser(*args, **kwargs)
def mail_consumer_declaration(sender, **kwargs):
return {
"parser": get_parser,
"weight": 20,
"mime_types": {
"message/rfc822": ".eml",
},
}

48
src/paperless_mail/templates/email_msg_template.html Normal file
View File

@@ -0,0 +1,48 @@
{% autoescape off %}
<!doctype html>
<html>
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="output.css" rel="stylesheet">
</head>
<body class="bg-white w-screen flex flex-col items-center">
<div class="container max-w-4xl">
<!-- Header -->
<div class="grid gap-x-2 bg-slate-200 p-4">
<div class="col-start-9 col-span-4 row-start-1 text-right">{{ date }}</div>
<div class="col-start-1 row-start-1 text-slate-400 text-right">{{ from_label }}</div>
<div class="col-start-2 col-span-7 row-start-1">{{ from }}</div>
<div class="col-start-1 row-start-2 text-slate-400 text-right">{{ subject_label }}</div>
<div class=" col-start-2 col-span-10 row-start-2 font-bold">{{ subject }}</div>
<div class="col-start-1 row-start-3 text-slate-400 text-right">{{ to_label }}</div>
<div class="col-start-2 col-span-10 row-start-3 text-sm my-0.5">{{ to }}</div>
<div class="col-start-1 row-start-4 text-slate-400 text-right">{{ cc_label }}</div>
<div class="col-start-2 col-span-10 row-start-4 text-sm my-0.5">{{ cc }}</div>
<div class="col-start-1 row-start-5 text-slate-400 text-right">{{ bcc_label }}</div>
<div class="col-start-2 col-span-10 row-start-5" text-sm my-0.5>{{ bcc }}</div>
<div class="col-start-1 row-start-6 text-slate-400 text-right">{{ attachments_label }}</div>
<div class="col-start-2 col-span-10 row-start-6">{{ attachments }}</div>
</div>
<!-- Separator-->
<div class="border-t border-solid border-b w-full h-[1px] box-content border-black mb-5 bg-slate-200"></div>
<!-- Content-->
<div class="w-full break-words">{{ content }}</div>
</div>
</body>
</html>
{% endautoescape %}

3
src/paperless_mail/templates/input.css Normal file
View File

@@ -0,0 +1,3 @@
@tailwind base;
@tailwind components;
@tailwind utilities;

706
src/paperless_mail/templates/output.css Normal file
View File

@@ -0,0 +1,706 @@
/*
! tailwindcss v3.0.24 | MIT License | https://tailwindcss.com
*/
/*
1. Prevent padding and border from affecting element width. (https://github.com/mozdevs/cssremedy/issues/4)
2. Allow adding a border to an element by just adding a border-width. (https://github.com/tailwindcss/tailwindcss/pull/116)
*/
*,
::before,
::after {
box-sizing: border-box;
/* 1 */
border-width: 0;
/* 2 */
border-style: solid;
/* 2 */
border-color: #e5e7eb;
/* 2 */
}
::before,
::after {
--tw-content: '';
}
/*
1. Use a consistent sensible line-height in all browsers.
2. Prevent adjustments of font size after orientation changes in iOS.
3. Use a more readable tab size.
4. Use the user's configured `sans` font-family by default.
*/
html {
line-height: 1.5;
/* 1 */
-webkit-text-size-adjust: 100%;
/* 2 */
-moz-tab-size: 4;
/* 3 */
-o-tab-size: 4;
tab-size: 4;
/* 3 */
font-family: ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, "Helvetica Neue", Arial, "Noto Sans", sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji";
/* 4 */
}
/*
1. Remove the margin in all browsers.
2. Inherit line-height from `html` so users can set them as a class directly on the `html` element.
*/
body {
margin: 0;
/* 1 */
line-height: inherit;
/* 2 */
}
/*
1. Add the correct height in Firefox.
2. Correct the inheritance of border color in Firefox. (https://bugzilla.mozilla.org/show_bug.cgi?id=190655)
3. Ensure horizontal rules are visible by default.
*/
hr {
height: 0;
/* 1 */
color: inherit;
/* 2 */
border-top-width: 1px;
/* 3 */
}
/*
Add the correct text decoration in Chrome, Edge, and Safari.
*/
abbr:where([title]) {
-webkit-text-decoration: underline dotted;
text-decoration: underline dotted;
}
/*
Remove the default font size and weight for headings.
*/
h1,
h2,
h3,
h4,
h5,
h6 {
font-size: inherit;
font-weight: inherit;
}
/*
Reset links to optimize for opt-in styling instead of opt-out.
*/
a {
color: inherit;
text-decoration: inherit;
}
/*
Add the correct font weight in Edge and Safari.
*/
b,
strong {
font-weight: bolder;
}
/*
1. Use the user's configured `mono` font family by default.
2. Correct the odd `em` font sizing in all browsers.
*/
code,
kbd,
samp,
pre {
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", "Courier New", monospace;
/* 1 */
font-size: 1em;
/* 2 */
}
/*
Add the correct font size in all browsers.
*/
small {
font-size: 80%;
}
/*
Prevent `sub` and `sup` elements from affecting the line height in all browsers.
*/
sub,
sup {
font-size: 75%;
line-height: 0;
position: relative;
vertical-align: baseline;
}
sub {
bottom: -0.25em;
}
sup {
top: -0.5em;
}
/*
1. Remove text indentation from table contents in Chrome and Safari. (https://bugs.chromium.org/p/chromium/issues/detail?id=999088, https://bugs.webkit.org/show_bug.cgi?id=201297)
2. Correct table border color inheritance in all Chrome and Safari. (https://bugs.chromium.org/p/chromium/issues/detail?id=935729, https://bugs.webkit.org/show_bug.cgi?id=195016)
3. Remove gaps between table borders by default.
*/
table {
text-indent: 0;
/* 1 */
border-color: inherit;
/* 2 */
border-collapse: collapse;
/* 3 */
}
/*
1. Change the font styles in all browsers.
2. Remove the margin in Firefox and Safari.
3. Remove default padding in all browsers.
*/
button,
input,
optgroup,
select,
textarea {
font-family: inherit;
/* 1 */
font-size: 100%;
/* 1 */
line-height: inherit;
/* 1 */
color: inherit;
/* 1 */
margin: 0;
/* 2 */
padding: 0;
/* 3 */
}
/*
Remove the inheritance of text transform in Edge and Firefox.
*/
button,
select {
text-transform: none;
}
/*
1. Correct the inability to style clickable types in iOS and Safari.
2. Remove default button styles.
*/
button,
[type='button'],
[type='reset'],
[type='submit'] {
-webkit-appearance: button;
/* 1 */
background-color: transparent;
/* 2 */
background-image: none;
/* 2 */
}
/*
Use the modern Firefox focus style for all focusable elements.
*/
:-moz-focusring {
outline: auto;
}
/*
Remove the additional `:invalid` styles in Firefox. (https://github.com/mozilla/gecko-dev/blob/2f9eacd9d3d995c937b4251a5557d95d494c9be1/layout/style/res/forms.css#L728-L737)
*/
:-moz-ui-invalid {
box-shadow: none;
}
/*
Add the correct vertical alignment in Chrome and Firefox.
*/
progress {
vertical-align: baseline;
}
/*
Correct the cursor style of increment and decrement buttons in Safari.
*/
::-webkit-inner-spin-button,
::-webkit-outer-spin-button {
height: auto;
}
/*
1. Correct the odd appearance in Chrome and Safari.
2. Correct the outline style in Safari.
*/
[type='search'] {
-webkit-appearance: textfield;
/* 1 */
outline-offset: -2px;
/* 2 */
}
/*
Remove the inner padding in Chrome and Safari on macOS.
*/
::-webkit-search-decoration {
-webkit-appearance: none;
}
/*
1. Correct the inability to style clickable types in iOS and Safari.
2. Change font properties to `inherit` in Safari.
*/
::-webkit-file-upload-button {
-webkit-appearance: button;
/* 1 */
font: inherit;
/* 2 */
}
/*
Add the correct display in Chrome and Safari.
*/
summary {
display: list-item;
}
/*
Removes the default spacing and border for appropriate elements.
*/
blockquote,
dl,
dd,
h1,
h2,
h3,
h4,
h5,
h6,
hr,
figure,
p,
pre {
margin: 0;
}
fieldset {
margin: 0;
padding: 0;
}
legend {
padding: 0;
}
ol,
ul,
menu {
list-style: none;
margin: 0;
padding: 0;
}
/*
Prevent resizing textareas horizontally by default.
*/
textarea {
resize: vertical;
}
/*
1. Reset the default placeholder opacity in Firefox. (https://github.com/tailwindlabs/tailwindcss/issues/3300)
2. Set the default placeholder color to the user's configured gray 400 color.
*/
input::-moz-placeholder, textarea::-moz-placeholder {
opacity: 1;
/* 1 */
color: #9ca3af;
/* 2 */
}
input:-ms-input-placeholder, textarea:-ms-input-placeholder {
opacity: 1;
/* 1 */
color: #9ca3af;
/* 2 */
}
input::placeholder,
textarea::placeholder {
opacity: 1;
/* 1 */
color: #9ca3af;
/* 2 */
}
/*
Set the default cursor for buttons.
*/
button,
[role="button"] {
cursor: pointer;
}
/*
Make sure disabled buttons don't get the pointer cursor.
*/
:disabled {
cursor: default;
}
/*
1. Make replaced elements `display: block` by default. (https://github.com/mozdevs/cssremedy/issues/14)
2. Add `vertical-align: middle` to align replaced elements more sensibly by default. (https://github.com/jensimmons/cssremedy/issues/14#issuecomment-634934210)
This can trigger a poorly considered lint error in some tools but is included by design.
*/
img,
svg,
video,
canvas,
audio,
iframe,
embed,
object {
display: block;
/* 1 */
vertical-align: middle;
/* 2 */
}
/*
Constrain images and videos to the parent width and preserve their intrinsic aspect ratio. (https://github.com/mozdevs/cssremedy/issues/14)
*/
img,
video {
max-width: 100%;
height: auto;
}
/*
Ensure the default browser behavior of the `hidden` attribute.
*/
[hidden] {
display: none;
}
*, ::before, ::after {
--tw-translate-x: 0;
--tw-translate-y: 0;
--tw-rotate: 0;
--tw-skew-x: 0;
--tw-skew-y: 0;
--tw-scale-x: 1;
--tw-scale-y: 1;
--tw-pan-x: ;
--tw-pan-y: ;
--tw-pinch-zoom: ;
--tw-scroll-snap-strictness: proximity;
--tw-ordinal: ;
--tw-slashed-zero: ;
--tw-numeric-figure: ;
--tw-numeric-spacing: ;
--tw-numeric-fraction: ;
--tw-ring-inset: ;
--tw-ring-offset-width: 0px;
--tw-ring-offset-color: #fff;
--tw-ring-color: rgb(59 130 246 / 0.5);
--tw-ring-offset-shadow: 0 0 #0000;
--tw-ring-shadow: 0 0 #0000;
--tw-shadow: 0 0 #0000;
--tw-shadow-colored: 0 0 #0000;
--tw-blur: ;
--tw-brightness: ;
--tw-contrast: ;
--tw-grayscale: ;
--tw-hue-rotate: ;
--tw-invert: ;
--tw-saturate: ;
--tw-sepia: ;
--tw-drop-shadow: ;
--tw-backdrop-blur: ;
--tw-backdrop-brightness: ;
--tw-backdrop-contrast: ;
--tw-backdrop-grayscale: ;
--tw-backdrop-hue-rotate: ;
--tw-backdrop-invert: ;
--tw-backdrop-opacity: ;
--tw-backdrop-saturate: ;
--tw-backdrop-sepia: ;
}
.container {
width: 100%;
}
@media (min-width: 640px) {
.container {
max-width: 640px;
}
}
@media (min-width: 768px) {
.container {
max-width: 768px;
}
}
@media (min-width: 1024px) {
.container {
max-width: 1024px;
}
}
@media (min-width: 1280px) {
.container {
max-width: 1280px;
}
}
@media (min-width: 1536px) {
.container {
max-width: 1536px;
}
}
.col-span-2 {
grid-column: span 2 / span 2;
}
.col-span-8 {
grid-column: span 8 / span 8;
}
.col-span-10 {
grid-column: span 10 / span 10;
}
.col-span-3 {
grid-column: span 3 / span 3;
}
.col-span-4 {
grid-column: span 4 / span 4;
}
.col-span-7 {
grid-column: span 7 / span 7;
}
.col-start-11 {
grid-column-start: 11;
}
.col-start-1 {
grid-column-start: 1;
}
.col-start-2 {
grid-column-start: 2;
}
.col-start-10 {
grid-column-start: 10;
}
.col-start-9 {
grid-column-start: 9;
}
.row-start-1 {
grid-row-start: 1;
}
.row-start-2 {
grid-row-start: 2;
}
.row-start-3 {
grid-row-start: 3;
}
.row-start-4 {
grid-row-start: 4;
}
.row-start-5 {
grid-row-start: 5;
}
.row-start-6 {
grid-row-start: 6;
}
.my-1 {
margin-top: 0.25rem;
margin-bottom: 0.25rem;
}
.my-0\.5 {
margin-top: 0.125rem;
margin-bottom: 0.125rem;
}
.my-0 {
margin-top: 0px;
margin-bottom: 0px;
}
.mb-5 {
margin-bottom: 1.25rem;
}
.box-content {
box-sizing: content-box;
}
.flex {
display: flex;
}
.grid {
display: grid;
}
.h-\[1px\] {
height: 1px;
}
.w-screen {
width: 100vw;
}
.w-full {
width: 100%;
}
.max-w-4xl {
max-width: 56rem;
}
.grid-cols-12 {
grid-template-columns: repeat(12, minmax(0, 1fr));
}
.grid-rows-5 {
grid-template-rows: repeat(5, minmax(0, 1fr));
}
.flex-col {
flex-direction: column;
}
.items-center {
align-items: center;
}
.gap-x-2 {
-moz-column-gap: 0.5rem;
column-gap: 0.5rem;
}
.whitespace-pre-line {
white-space: pre-line;
}
.break-words {
overflow-wrap: break-word;
}
.border-t {
border-top-width: 1px;
}
.border-b {
border-bottom-width: 1px;
}
.border-solid {
border-style: solid;
}
.border-black {
--tw-border-opacity: 1;
border-color: rgb(0 0 0 / var(--tw-border-opacity));
}
.bg-white {
--tw-bg-opacity: 1;
background-color: rgb(255 255 255 / var(--tw-bg-opacity));
}
.bg-slate-200 {
--tw-bg-opacity: 1;
background-color: rgb(226 232 240 / var(--tw-bg-opacity));
}
.p-4 {
padding: 1rem;
}
.text-right {
text-align: right;
}
.text-sm {
font-size: 0.875rem;
line-height: 1.25rem;
}
.font-bold {
font-weight: 700;
}
.text-slate-400 {
--tw-text-opacity: 1;
color: rgb(148 163 184 / var(--tw-text-opacity));
}
.text-blue-600 {
--tw-text-opacity: 1;
color: rgb(37 99 235 / var(--tw-text-opacity));
}
.underline {
-webkit-text-decoration-line: underline;
text-decoration-line: underline;
}

1260
src/paperless_mail/templates/package-lock.json generated Normal file
View File

File diff suppressed because it is too large Load Diff

5
src/paperless_mail/templates/package.json Normal file
View File

@@ -0,0 +1,5 @@
{
"devDependencies": {
"tailwindcss": "^3.0.24"
}
}

7
src/paperless_mail/templates/tailwind.config.js Normal file
View File

@@ -0,0 +1,7 @@
module.exports = {
content: ['./*.html'],
theme: {
extend: {},
},
plugins: [],
}

1
src/paperless_mail/tests/samples/broken.eml Normal file
View File

@@ -0,0 +1 @@
This is not a valid eml.

BIN
src/paperless_mail/tests/samples/first.pdf Normal file
View File

Binary file not shown.

11599
src/paperless_mail/tests/samples/html.eml Normal file
View File

File diff suppressed because it is too large Load Diff

45
src/paperless_mail/tests/samples/html.eml.html Normal file
View File

@@ -0,0 +1,45 @@
<!doctype html>
<html>
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="output.css" rel="stylesheet">
</head>
<body class="bg-white w-screen flex flex-col items-center">
<div class="container max-w-4xl">
<!-- Header -->
<div class="grid gap-x-2 bg-slate-200 p-4">
<div class="col-start-9 col-span-4 row-start-1 text-right">2022-10-15 09:23</div>
<div class="col-start-1 row-start-1 text-slate-400 text-right">From</div>
<div class="col-start-2 col-span-7 row-start-1">Name &lt;<a href="mailto:someone@example.de">someone@example.de</a>&gt;</div>
<div class="col-start-1 row-start-2 text-slate-400 text-right">Subject</div>
<div class=" col-start-2 col-span-10 row-start-2 font-bold">HTML Message</div>
<div class="col-start-1 row-start-3 text-slate-400 text-right">To</div>
<div class="col-start-2 col-span-10 row-start-3 text-sm my-0.5"><a href="mailto:someone@example.de">someone@example.de</a></div>
<div class="col-start-1 row-start-4 text-slate-400 text-right"></div>
<div class="col-start-2 col-span-10 row-start-4 text-sm my-0.5"></div>
<div class="col-start-1 row-start-5 text-slate-400 text-right"></div>
<div class="col-start-2 col-span-10 row-start-5" text-sm my-0.5></div>
<div class="col-start-1 row-start-6 text-slate-400 text-right">Attachments</div>
<div class="col-start-2 col-span-10 row-start-6">IntM6gnXFm00FEV5.png (6.89 KiB), 600+kbfile.txt (600.24 KiB)</div>
</div>
<!-- Separator-->
<div class="border-t border-solid border-b w-full h-[1px] box-content border-black mb-5 bg-slate-200"></div>
<!-- Content-->
<div class="w-full break-words">Some Text<br><br>and an embedded image.</div>
</div>
</body>
</html>

BIN
src/paperless_mail/tests/samples/html.eml.pdf Normal file
View File

Binary file not shown.

BIN
src/paperless_mail/tests/samples/html.eml.pdf.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.0 KiB

19
src/paperless_mail/tests/samples/sample.html Normal file
View File

@@ -0,0 +1,19 @@
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Some Text</p>
<p>
<img src="cid:part1.pNdUSz0s.D3NqVtPg@example.de" alt="Has to be rewritten to work..">
<img src="https://upload.wikimedia.org/wikipedia/en/f/f7/RickRoll.png" alt="This image should not be shown.">
</p>
<p>and an embedded image.<br>
</p>
<p id="changeme">Paragraph unchanged.</p>
<scRipt>
document.getElementById("changeme").innerHTML = "Paragraph changed via Java Script.";
</script>
</body>
</html>

BIN
src/paperless_mail/tests/samples/sample.html.pdf Normal file
View File

Binary file not shown.

BIN
src/paperless_mail/tests/samples/sample.html.pdf.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.8 KiB

BIN
src/paperless_mail/tests/samples/sample.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.9 KiB

BIN
src/paperless_mail/tests/samples/second.pdf Normal file
View File

Binary file not shown.

25
src/paperless_mail/tests/samples/simple_text.eml Normal file
View File

@@ -0,0 +1,25 @@
Return-Path: <mail@someserver.de>
Delivered-To: mail@someserver.de
Received: from mail.someserver.org ([::1])
by e1acdba3bd07 with LMTP
id KBKZGD2YR2NTCgQAjubtDA
(envelope-from <mail@someserver.de>)
for <mail@someserver.de>; Wed, 10 Oct 2022 11:40:46 +0200
Received: from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 2BC9064C1616
for <some@one.de>; Wed, 12 Oct 2022 21:40:46 +0200 (CEST)
Message-ID: <6e99e34d-e20a-80c4-ea61-d8234b612be9@someserver.de>
Date: Wed, 12 Oct 2022 21:40:43 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
Thunderbird/102.3.1
Content-Language: en-US
To: some@one.de
Cc: asdasd@æsdasd.de, asdadasdasdasda.asdasd@æsdasd.de
Bcc: fdf@fvf.de
From: Some One <mail@someserver.de>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Last-TLS-Session-Version: TLSv1.3
Subject: Simple Text Mail
This is just a simple Text Mail.

BIN
src/paperless_mail/tests/samples/simple_text.eml.pdf Normal file
View File

Binary file not shown.

BIN
src/paperless_mail/tests/samples/simple_text.eml.pdf.webp Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.2 KiB

429
src/paperless_mail/tests/test_api.py Normal file
View File

@@ -0,0 +1,429 @@
from django.contrib.auth.models import User
from documents.models import Correspondent
from documents.models import DocumentType
from documents.models import Tag
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from rest_framework.test import APITestCase
class TestAPIMailAccounts(APITestCase):
ENDPOINT = "/api/mail_accounts/"
def setUp(self):
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user)
def test_get_mail_accounts(self):
"""
GIVEN:
- Configured mail accounts
WHEN:
- API call is made to get mail accounts
THEN:
- Configured mail accounts are provided
"""
account1 = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.data["count"], 1)
returned_account1 = response.data["results"][0]
self.assertEqual(returned_account1["name"], account1.name)
self.assertEqual(returned_account1["username"], account1.username)
self.assertEqual(
returned_account1["password"],
"*" * len(account1.password),
)
self.assertEqual(returned_account1["imap_server"], account1.imap_server)
self.assertEqual(returned_account1["imap_port"], account1.imap_port)
self.assertEqual(returned_account1["imap_security"], account1.imap_security)
self.assertEqual(returned_account1["character_set"], account1.character_set)
def test_create_mail_account(self):
"""
WHEN:
- API request is made to add a mail account
THEN:
- A new mail account is created
"""
account1 = {
"name": "Email1",
"username": "username1",
"password": "password1",
"imap_server": "server.example.com",
"imap_port": 443,
"imap_security": MailAccount.ImapSecurity.SSL,
"character_set": "UTF-8",
}
response = self.client.post(
self.ENDPOINT,
data=account1,
)
self.assertEqual(response.status_code, 201)
returned_account1 = MailAccount.objects.get(name="Email1")
self.assertEqual(returned_account1.name, account1["name"])
self.assertEqual(returned_account1.username, account1["username"])
self.assertEqual(returned_account1.password, account1["password"])
self.assertEqual(returned_account1.imap_server, account1["imap_server"])
self.assertEqual(returned_account1.imap_port, account1["imap_port"])
self.assertEqual(returned_account1.imap_security, account1["imap_security"])
self.assertEqual(returned_account1.character_set, account1["character_set"])
def test_delete_mail_account(self):
"""
GIVEN:
- Existing mail account
WHEN:
- API request is made to delete a mail account
THEN:
- Account is deleted
"""
account1 = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
response = self.client.delete(
f"{self.ENDPOINT}{account1.pk}/",
)
self.assertEqual(response.status_code, 204)
self.assertEqual(len(MailAccount.objects.all()), 0)
def test_update_mail_account(self):
"""
GIVEN:
- Existing mail accounts
WHEN:
- API request is made to update mail account
THEN:
- The mail account is updated, password only updated if not '****'
"""
account1 = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
response = self.client.patch(
f"{self.ENDPOINT}{account1.pk}/",
data={
"name": "Updated Name 1",
"password": "******",
},
)
self.assertEqual(response.status_code, 200)
returned_account1 = MailAccount.objects.get(pk=account1.pk)
self.assertEqual(returned_account1.name, "Updated Name 1")
self.assertEqual(returned_account1.password, account1.password)
response = self.client.patch(
f"{self.ENDPOINT}{account1.pk}/",
data={
"name": "Updated Name 2",
"password": "123xyz",
},
)
self.assertEqual(response.status_code, 200)
returned_account2 = MailAccount.objects.get(pk=account1.pk)
self.assertEqual(returned_account2.name, "Updated Name 2")
self.assertEqual(returned_account2.password, "123xyz")
class TestAPIMailRules(APITestCase):
ENDPOINT = "/api/mail_rules/"
def setUp(self):
super().setUp()
self.user = User.objects.create_superuser(username="temp_admin")
self.client.force_authenticate(user=self.user)
def test_get_mail_rules(self):
"""
GIVEN:
- Configured mail accounts and rules
WHEN:
- API call is made to get mail rules
THEN:
- Configured mail rules are provided
"""
account1 = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
rule1 = MailRule.objects.create(
name="Rule1",
account=account1,
folder="INBOX",
filter_from="from@example.com",
filter_subject="subject",
filter_body="body",
filter_attachment_filename="file.pdf",
maximum_age=30,
action=MailRule.MailAction.MARK_READ,
assign_title_from=MailRule.TitleSource.FROM_SUBJECT,
assign_correspondent_from=MailRule.CorrespondentSource.FROM_NOTHING,
order=0,
attachment_type=MailRule.AttachmentProcessing.ATTACHMENTS_ONLY,
)
response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.data["count"], 1)
returned_rule1 = response.data["results"][0]
self.assertEqual(returned_rule1["name"], rule1.name)
self.assertEqual(returned_rule1["account"], account1.pk)
self.assertEqual(returned_rule1["folder"], rule1.folder)
self.assertEqual(returned_rule1["filter_from"], rule1.filter_from)
self.assertEqual(returned_rule1["filter_subject"], rule1.filter_subject)
self.assertEqual(returned_rule1["filter_body"], rule1.filter_body)
self.assertEqual(
returned_rule1["filter_attachment_filename"],
rule1.filter_attachment_filename,
)
self.assertEqual(returned_rule1["maximum_age"], rule1.maximum_age)
self.assertEqual(returned_rule1["action"], rule1.action)
self.assertEqual(returned_rule1["assign_title_from"], rule1.assign_title_from)
self.assertEqual(
returned_rule1["assign_correspondent_from"],
rule1.assign_correspondent_from,
)
self.assertEqual(returned_rule1["order"], rule1.order)
self.assertEqual(returned_rule1["attachment_type"], rule1.attachment_type)
def test_create_mail_rule(self):
"""
GIVEN:
- Configured mail account exists
WHEN:
- API request is made to add a mail rule
THEN:
- A new mail rule is created
"""
account1 = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
tag = Tag.objects.create(
name="t",
)
correspondent = Correspondent.objects.create(
name="c",
)
document_type = DocumentType.objects.create(
name="dt",
)
rule1 = {
"name": "Rule1",
"account": account1.pk,
"folder": "INBOX",
"filter_from": "from@example.com",
"filter_subject": "subject",
"filter_body": "body",
"filter_attachment_filename": "file.pdf",
"maximum_age": 30,
"action": MailRule.MailAction.MARK_READ,
"assign_title_from": MailRule.TitleSource.FROM_SUBJECT,
"assign_correspondent_from": MailRule.CorrespondentSource.FROM_NOTHING,
"order": 0,
"attachment_type": MailRule.AttachmentProcessing.ATTACHMENTS_ONLY,
"action_parameter": "parameter",
"assign_tags": [tag.pk],
"assign_correspondent": correspondent.pk,
"assign_document_type": document_type.pk,
}
response = self.client.post(
self.ENDPOINT,
data=rule1,
)
self.assertEqual(response.status_code, 201)
response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, 200)
self.assertEqual(response.data["count"], 1)
returned_rule1 = response.data["results"][0]
self.assertEqual(returned_rule1["name"], rule1["name"])
self.assertEqual(returned_rule1["account"], account1.pk)
self.assertEqual(returned_rule1["folder"], rule1["folder"])
self.assertEqual(returned_rule1["filter_from"], rule1["filter_from"])
self.assertEqual(returned_rule1["filter_subject"], rule1["filter_subject"])
self.assertEqual(returned_rule1["filter_body"], rule1["filter_body"])
self.assertEqual(
returned_rule1["filter_attachment_filename"],
rule1["filter_attachment_filename"],
)
self.assertEqual(returned_rule1["maximum_age"], rule1["maximum_age"])
self.assertEqual(returned_rule1["action"], rule1["action"])
self.assertEqual(
returned_rule1["assign_title_from"],
rule1["assign_title_from"],
)
self.assertEqual(
returned_rule1["assign_correspondent_from"],
rule1["assign_correspondent_from"],
)
self.assertEqual(returned_rule1["order"], rule1["order"])
self.assertEqual(returned_rule1["attachment_type"], rule1["attachment_type"])
self.assertEqual(returned_rule1["action_parameter"], rule1["action_parameter"])
self.assertEqual(
returned_rule1["assign_correspondent"],
rule1["assign_correspondent"],
)
self.assertEqual(
returned_rule1["assign_document_type"],
rule1["assign_document_type"],
)
self.assertEqual(returned_rule1["assign_tags"], rule1["assign_tags"])
def test_delete_mail_rule(self):
"""
GIVEN:
- Existing mail rule
WHEN:
- API request is made to delete a mail rule
THEN:
- Rule is deleted
"""
account1 = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
rule1 = MailRule.objects.create(
name="Rule1",
account=account1,
folder="INBOX",
filter_from="from@example.com",
filter_subject="subject",
filter_body="body",
filter_attachment_filename="file.pdf",
maximum_age=30,
action=MailRule.MailAction.MARK_READ,
assign_title_from=MailRule.TitleSource.FROM_SUBJECT,
assign_correspondent_from=MailRule.CorrespondentSource.FROM_NOTHING,
order=0,
attachment_type=MailRule.AttachmentProcessing.ATTACHMENTS_ONLY,
)
response = self.client.delete(
f"{self.ENDPOINT}{rule1.pk}/",
)
self.assertEqual(response.status_code, 204)
self.assertEqual(len(MailRule.objects.all()), 0)
def test_update_mail_rule(self):
"""
GIVEN:
- Existing mail rule
WHEN:
- API request is made to update mail rule
THEN:
- The mail rule is updated
"""
account1 = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
rule1 = MailRule.objects.create(
name="Rule1",
account=account1,
folder="INBOX",
filter_from="from@example.com",
filter_subject="subject",
filter_body="body",
filter_attachment_filename="file.pdf",
maximum_age=30,
action=MailRule.MailAction.MARK_READ,
assign_title_from=MailRule.TitleSource.FROM_SUBJECT,
assign_correspondent_from=MailRule.CorrespondentSource.FROM_NOTHING,
order=0,
attachment_type=MailRule.AttachmentProcessing.ATTACHMENTS_ONLY,
)
response = self.client.patch(
f"{self.ENDPOINT}{rule1.pk}/",
data={
"name": "Updated Name 1",
"action": MailRule.MailAction.DELETE,
},
)
self.assertEqual(response.status_code, 200)
returned_rule1 = MailRule.objects.get(pk=rule1.pk)
self.assertEqual(returned_rule1.name, "Updated Name 1")
self.assertEqual(returned_rule1.action, MailRule.MailAction.DELETE)

688
src/paperless_mail/tests/test_parsers.py Normal file
View File

@@ -0,0 +1,688 @@
import datetime
import os
from unittest import mock
from django.test import TestCase
from documents.parsers import ParseError
from paperless_mail.parsers import MailDocumentParser
class TestParser(TestCase):
SAMPLE_FILES = os.path.join(os.path.dirname(__file__), "samples")
def setUp(self) -> None:
self.parser = MailDocumentParser(logging_group=None)
def tearDown(self) -> None:
self.parser.cleanup()
def test_get_parsed_missing_file(self):
"""
GIVEN:
- Fresh parser
WHEN:
- A nonexistent file should be parsed
THEN:
- An Exception is thrown
"""
# Check if exception is raised when parsing fails.
self.assertRaises(
ParseError,
self.parser.get_parsed,
os.path.join(self.SAMPLE_FILES, "na"),
)
def test_get_parsed_broken_file(self):
"""
GIVEN:
- Fresh parser
WHEN:
- A faulty file should be parsed
THEN:
- An Exception is thrown
"""
# Check if exception is raised when the mail is faulty.
self.assertRaises(
ParseError,
self.parser.get_parsed,
os.path.join(self.SAMPLE_FILES, "broken.eml"),
)
def test_get_parsed_simple_text_mail(self):
"""
GIVEN:
- Fresh parser
WHEN:
- A .eml file should be parsed
THEN:
- The content of the mail should be available in the parse result.
"""
# Parse Test file and check relevant content
parsed1 = self.parser.get_parsed(
os.path.join(self.SAMPLE_FILES, "simple_text.eml"),
)
self.assertEqual(parsed1.date.year, 2022)
self.assertEqual(parsed1.date.month, 10)
self.assertEqual(parsed1.date.day, 12)
self.assertEqual(parsed1.date.hour, 21)
self.assertEqual(parsed1.date.minute, 40)
self.assertEqual(parsed1.date.second, 43)
self.assertEqual(parsed1.date.tzname(), "UTC+02:00")
self.assertEqual(parsed1.from_, "mail@someserver.de")
self.assertEqual(parsed1.subject, "Simple Text Mail")
self.assertEqual(parsed1.text, "This is just a simple Text Mail.\n")
self.assertEqual(parsed1.to, ("some@one.de",))
def test_get_parsed_reparse(self):
"""
GIVEN:
- An E-Mail was parsed
WHEN:
- Another .eml file should be parsed
THEN:
- The parser should not retry to parse and return the old results
"""
# Parse Test file and check relevant content
parsed1 = self.parser.get_parsed(
os.path.join(self.SAMPLE_FILES, "simple_text.eml"),
)
# Check if same parsed object as before is returned, even if another file is given.
parsed2 = self.parser.get_parsed(
os.path.join(os.path.join(self.SAMPLE_FILES, "html.eml")),
)
self.assertEqual(parsed1, parsed2)
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf")
@mock.patch("paperless_mail.parsers.make_thumbnail_from_pdf")
def test_get_thumbnail(
self,
mock_make_thumbnail_from_pdf: mock.MagicMock,
mock_generate_pdf: mock.MagicMock,
):
"""
GIVEN:
- An E-Mail was parsed
WHEN:
- The Thumbnail is requested
THEN:
- The parser should call the functions which generate the thumbnail
"""
mocked_return = "Passing the return value through.."
mock_make_thumbnail_from_pdf.return_value = mocked_return
mock_generate_pdf.return_value = "Mocked return value.."
thumb = self.parser.get_thumbnail(
os.path.join(self.SAMPLE_FILES, "simple_text.eml"),
"message/rfc822",
)
self.assertEqual(
self.parser.archive_path,
mock_make_thumbnail_from_pdf.call_args_list[0].args[0],
)
self.assertEqual(
self.parser.tempdir,
mock_make_thumbnail_from_pdf.call_args_list[0].args[1],
)
self.assertEqual(mocked_return, thumb)
@mock.patch("documents.loggers.LoggingMixin.log")
def test_extract_metadata_fail(self, m: mock.MagicMock):
"""
GIVEN:
- Fresh start
WHEN:
- Metadata extraction is triggered for nonexistent file
THEN:
- A log warning should be generated
"""
# Validate if warning is logged when parsing fails
self.assertEqual([], self.parser.extract_metadata("na", "message/rfc822"))
self.assertEqual("warning", m.call_args[0][0])
def test_extract_metadata(self):
"""
GIVEN:
- Fresh start
WHEN:
- Metadata extraction is triggered
THEN:
- metadata is returned
"""
# Validate Metadata parsing returns the expected results
metadata = self.parser.extract_metadata(
os.path.join(self.SAMPLE_FILES, "simple_text.eml"),
"message/rfc822",
)
self.assertIn(
{"namespace": "", "prefix": "", "key": "attachments", "value": ""},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "",
"key": "date",
"value": "2022-10-12 21:40:43 UTC+02:00",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "content-language",
"value": "en-US",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "content-type",
"value": "text/plain; charset=UTF-8; format=flowed",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "date",
"value": "Wed, 12 Oct 2022 21:40:43 +0200",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "delivered-to",
"value": "mail@someserver.de",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "from",
"value": "Some One <mail@someserver.de>",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "message-id",
"value": "<6e99e34d-e20a-80c4-ea61-d8234b612be9@someserver.de>",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "mime-version",
"value": "1.0",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "received",
"value": "from mail.someserver.org ([::1])\n\tby e1acdba3bd07 with LMTP\n\tid KBKZGD2YR2NTCgQAjubtDA\n\t(envelope-from <mail@someserver.de>)\n\tfor <mail@someserver.de>; Wed, 10 Oct 2022 11:40:46 +0200, from [127.0.0.1] (localhost [127.0.0.1]) by localhost (Mailerdaemon) with ESMTPSA id 2BC9064C1616\n\tfor <some@one.de>; Wed, 12 Oct 2022 21:40:46 +0200 (CEST)",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "return-path",
"value": "<mail@someserver.de>",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "subject",
"value": "Simple Text Mail",
},
metadata,
)
self.assertIn(
{"namespace": "", "prefix": "header", "key": "to", "value": "some@one.de"},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "user-agent",
"value": "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101\n Thunderbird/102.3.1",
},
metadata,
)
self.assertIn(
{
"namespace": "",
"prefix": "header",
"key": "x-last-tls-session-version",
"value": "TLSv1.3",
},
metadata,
)
def test_parse_na(self):
"""
GIVEN:
- Fresh start
WHEN:
- parsing is attempted with nonexistent file
THEN:
- Exception is thrown
"""
# Check if exception is raised when parsing fails.
self.assertRaises(
ParseError,
self.parser.parse,
os.path.join(self.SAMPLE_FILES, "na"),
"message/rfc822",
)
@mock.patch("paperless_mail.parsers.MailDocumentParser.tika_parse")
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf")
def test_parse_html_eml(self, n, mock_tika_parse: mock.MagicMock):
"""
GIVEN:
- Fresh start
WHEN:
- parsing is done with html mail
THEN:
- Tika is called, parsed information from non html parts is available
"""
# Validate parsing returns the expected results
text_expected = "Subject: HTML Message\n\nFrom: Name <someone@example.de>\n\nTo: someone@example.de\n\nAttachments: IntM6gnXFm00FEV5.png (6.89 KiB), 600+kbfile.txt (600.24 KiB)\n\nHTML content: tika return\n\nSome Text and an embedded image."
mock_tika_parse.return_value = "tika return"
self.parser.parse(os.path.join(self.SAMPLE_FILES, "html.eml"), "message/rfc822")
self.assertEqual(text_expected, self.parser.text)
self.assertEqual(
datetime.datetime(
2022,
10,
15,
11,
23,
19,
tzinfo=datetime.timezone(datetime.timedelta(seconds=7200)),
),
self.parser.date,
)
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf")
def test_parse_simple_eml(self, n):
"""
GIVEN:
- Fresh start
WHEN:
- parsing is done with non html mail
THEN:
- parsed information is available
"""
# Validate parsing returns the expected results
self.parser.parse(
os.path.join(self.SAMPLE_FILES, "simple_text.eml"),
"message/rfc822",
)
text_expected = "Subject: Simple Text Mail\n\nFrom: Some One <mail@someserver.de>\n\nTo: some@one.de\n\nCC: asdasd@æsdasd.de, asdadasdasdasda.asdasd@æsdasd.de\n\nBCC: fdf@fvf.de\n\n\n\nThis is just a simple Text Mail."
self.assertEqual(text_expected, self.parser.text)
self.assertEqual(
datetime.datetime(
2022,
10,
12,
21,
40,
43,
tzinfo=datetime.timezone(datetime.timedelta(seconds=7200)),
),
self.parser.date,
)
# Just check if file exists, the unittest for generate_pdf() goes deeper.
self.assertTrue(os.path.isfile(self.parser.archive_path))
@mock.patch("paperless_mail.parsers.parser.from_buffer")
def test_tika_parse_unsuccessful(self, mock_from_buffer: mock.MagicMock):
"""
GIVEN:
- Fresh start
WHEN:
- tika parsing fails
THEN:
- the parser should return an empty string
"""
# Check unsuccessful parsing
mock_from_buffer.return_value = {"content": None}
parsed = self.parser.tika_parse(None)
self.assertEqual("", parsed)
@mock.patch("paperless_mail.parsers.parser.from_buffer")
def test_tika_parse(self, mock_from_buffer: mock.MagicMock):
"""
GIVEN:
- Fresh start
WHEN:
- tika parsing is called
THEN:
- a web request to tika shall be done and the reply es returned
"""
html = '<html><head><meta http-equiv="content-type" content="text/html; charset=UTF-8"></head><body><p>Some Text</p></body></html>'
expected_text = "Some Text"
# Check successful parsing
mock_from_buffer.return_value = {"content": expected_text}
parsed = self.parser.tika_parse(html)
self.assertEqual(expected_text, parsed.strip())
mock_from_buffer.assert_called_with(html, self.parser.tika_server)
@mock.patch("paperless_mail.parsers.parser.from_buffer")
def test_tika_parse_exception(self, mock_from_buffer: mock.MagicMock):
"""
GIVEN:
- Fresh start
WHEN:
- tika parsing is called and an exception is thrown on the request
THEN:
- a ParseError Exception is thrown
"""
html = '<html><head><meta http-equiv="content-type" content="text/html; charset=UTF-8"></head><body><p>Some Text</p></body></html>'
# Check ParseError
def my_side_effect():
raise Exception("Test")
mock_from_buffer.side_effect = my_side_effect
self.assertRaises(ParseError, self.parser.tika_parse, html)
def test_tika_parse_unreachable(self):
"""
GIVEN:
- Fresh start
WHEN:
- tika parsing is called but tika is not available
THEN:
- a ParseError Exception is thrown
"""
html = '<html><head><meta http-equiv="content-type" content="text/html; charset=UTF-8"></head><body><p>Some Text</p></body></html>'
# Check if exception is raised when Tika cannot be reached.
self.parser.tika_server = ""
self.assertRaises(ParseError, self.parser.tika_parse, html)
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf_from_mail")
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf_from_html")
def test_generate_pdf_parse_error(self, m: mock.MagicMock, n: mock.MagicMock):
"""
GIVEN:
- Fresh start
WHEN:
- pdf generation is requested but gotenberg can not be reached
THEN:
- a ParseError Exception is thrown
"""
m.return_value = b""
n.return_value = b""
# Check if exception is raised when the pdf can not be created.
self.parser.gotenberg_server = ""
self.assertRaises(
ParseError,
self.parser.generate_pdf,
os.path.join(self.SAMPLE_FILES, "html.eml"),
)
def test_generate_pdf_exception(self):
"""
GIVEN:
- Fresh start
WHEN:
- pdf generation is requested but parsing throws an exception
THEN:
- a ParseError Exception is thrown
"""
# Check if exception is raised when the mail can not be parsed.
self.assertRaises(
ParseError,
self.parser.generate_pdf,
os.path.join(self.SAMPLE_FILES, "broken.eml"),
)
@mock.patch("paperless_mail.parsers.requests.post")
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf_from_mail")
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf_from_html")
def test_generate_pdf(
self,
mock_generate_pdf_from_html: mock.MagicMock,
mock_generate_pdf_from_mail: mock.MagicMock,
mock_post: mock.MagicMock,
):
"""
GIVEN:
- Fresh start
WHEN:
- pdf generation is requested
THEN:
- gotenberg is called and the resulting file is returned
"""
mock_generate_pdf_from_mail.return_value = b"Mail Return"
mock_generate_pdf_from_html.return_value = b"HTML Return"
mock_response = mock.MagicMock()
mock_response.content = b"Content"
mock_post.return_value = mock_response
pdf_path = self.parser.generate_pdf(os.path.join(self.SAMPLE_FILES, "html.eml"))
self.assertTrue(os.path.isfile(pdf_path))
mock_generate_pdf_from_mail.assert_called_once_with(
self.parser.get_parsed(None),
)
mock_generate_pdf_from_html.assert_called_once_with(
self.parser.get_parsed(None).html,
self.parser.get_parsed(None).attachments,
)
self.assertEqual(
self.parser.gotenberg_server + "/forms/pdfengines/merge",
mock_post.call_args.args[0],
)
self.assertEqual({}, mock_post.call_args.kwargs["headers"])
self.assertEqual(
b"Mail Return",
mock_post.call_args.kwargs["files"]["1_mail.pdf"][1].read(),
)
self.assertEqual(
b"HTML Return",
mock_post.call_args.kwargs["files"]["2_html.pdf"][1].read(),
)
mock_response.raise_for_status.assert_called_once()
with open(pdf_path, "rb") as file:
self.assertEqual(b"Content", file.read())
def test_mail_to_html(self):
"""
GIVEN:
- Fresh start
WHEN:
- conversion from eml to html is requested
THEN:
- html should be returned
"""
mail = self.parser.get_parsed(os.path.join(self.SAMPLE_FILES, "html.eml"))
html_handle = self.parser.mail_to_html(mail)
html_received = html_handle.read()
with open(
os.path.join(self.SAMPLE_FILES, "html.eml.html"),
) as html_expected_handle:
html_expected = html_expected_handle.read()
self.assertHTMLEqual(html_expected, html_received)
@mock.patch("paperless_mail.parsers.requests.post")
@mock.patch("paperless_mail.parsers.MailDocumentParser.mail_to_html")
def test_generate_pdf_from_mail(
self,
mock_mail_to_html: mock.MagicMock,
mock_post: mock.MagicMock,
):
"""
GIVEN:
- Fresh start
WHEN:
- conversion of PDF from .eml is requested
THEN:
- gotenberg should be called with valid intermediary html files, the resulting pdf is returned
"""
mock_response = mock.MagicMock()
mock_response.content = b"Content"
mock_post.return_value = mock_response
mock_mail_to_html.return_value = "Testresponse"
mail = self.parser.get_parsed(os.path.join(self.SAMPLE_FILES, "html.eml"))
retval = self.parser.generate_pdf_from_mail(mail)
self.assertEqual(b"Content", retval)
mock_mail_to_html.assert_called_once_with(mail)
self.assertEqual(
self.parser.gotenberg_server + "/forms/chromium/convert/html",
mock_post.call_args.args[0],
)
self.assertEqual({}, mock_post.call_args.kwargs["headers"])
self.assertEqual(
{
"marginTop": "0.1",
"marginBottom": "0.1",
"marginLeft": "0.1",
"marginRight": "0.1",
"paperWidth": "8.27",
"paperHeight": "11.7",
"scale": "1.0",
},
mock_post.call_args.kwargs["data"],
)
self.assertEqual(
"Testresponse",
mock_post.call_args.kwargs["files"]["html"][1],
)
self.assertEqual(
"output.css",
mock_post.call_args.kwargs["files"]["css"][0],
)
mock_response.raise_for_status.assert_called_once()
def test_transform_inline_html(self):
"""
GIVEN:
- Fresh start
WHEN:
- transforming of html content from an email with an inline image attachment is requested
THEN:
- html is returned and sanitized
"""
class MailAttachmentMock:
def __init__(self, payload, content_id):
self.payload = payload
self.content_id = content_id
result = None
with open(os.path.join(self.SAMPLE_FILES, "sample.html")) as html_file:
with open(os.path.join(self.SAMPLE_FILES, "sample.png"), "rb") as png_file:
html = html_file.read()
png = png_file.read()
attachments = [
MailAttachmentMock(png, "part1.pNdUSz0s.D3NqVtPg@example.de"),
]
result = self.parser.transform_inline_html(html, attachments)
resulting_html = result[-1][1].read()
self.assertTrue(result[-1][0] == "index.html")
self.assertIn(result[0][0], resulting_html)
self.assertNotIn("<script", resulting_html.lower())
@mock.patch("paperless_mail.parsers.requests.post")
def test_generate_pdf_from_html(self, mock_post: mock.MagicMock):
"""
GIVEN:
- Fresh start
WHEN:
- generating pdf from html with inline attachments is attempted
THEN:
- gotenberg is called with the correct parameters and the resulting pdf is returned
"""
class MailAttachmentMock:
def __init__(self, payload, content_id):
self.payload = payload
self.content_id = content_id
mock_response = mock.MagicMock()
mock_response.content = b"Content"
mock_post.return_value = mock_response
result = None
with open(os.path.join(self.SAMPLE_FILES, "sample.html")) as html_file:
with open(os.path.join(self.SAMPLE_FILES, "sample.png"), "rb") as png_file:
html = html_file.read()
png = png_file.read()
attachments = [
MailAttachmentMock(png, "part1.pNdUSz0s.D3NqVtPg@example.de"),
]
result = self.parser.generate_pdf_from_html(html, attachments)
self.assertEqual(
self.parser.gotenberg_server + "/forms/chromium/convert/html",
mock_post.call_args.args[0],
)
self.assertEqual({}, mock_post.call_args.kwargs["headers"])
self.assertEqual(
{
"marginTop": "0.1",
"marginBottom": "0.1",
"marginLeft": "0.1",
"marginRight": "0.1",
"paperWidth": "8.27",
"paperHeight": "11.7",
"scale": "1.0",
},
mock_post.call_args.kwargs["data"],
)
# read to assert it is a file like object.
mock_post.call_args.kwargs["files"]["cidpart1pNdUSz0sD3NqVtPgexamplede"][
1
].read()
mock_post.call_args.kwargs["files"]["index.html"][1].read()
mock_response.raise_for_status.assert_called_once()
self.assertEqual(b"Content", result)

361
src/paperless_mail/tests/test_parsers_live.py Normal file
View File

@@ -0,0 +1,361 @@
import os
from unittest import mock
from urllib.error import HTTPError
from urllib.request import urlopen
import pytest
from django.test import TestCase
from documents.parsers import ParseError
from documents.parsers import run_convert
from imagehash import average_hash
from paperless_mail.parsers import MailDocumentParser
from pdfminer.high_level import extract_text
from PIL import Image
class TestParserLive(TestCase):
SAMPLE_FILES = os.path.join(os.path.dirname(__file__), "samples")
def setUp(self) -> None:
self.parser = MailDocumentParser(logging_group=None)
def tearDown(self) -> None:
self.parser.cleanup()
@staticmethod
def imagehash(file, hash_size=18):
return f"{average_hash(Image.open(file), hash_size)}"
# Only run if convert is available
@pytest.mark.skipif(
"PAPERLESS_TEST_SKIP_CONVERT" in os.environ,
reason="PAPERLESS_TEST_SKIP_CONVERT set, skipping Test",
)
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf")
def test_get_thumbnail(self, mock_generate_pdf: mock.MagicMock):
"""
GIVEN:
- Fresh start
WHEN:
- The Thumbnail is requested
THEN:
- The returned thumbnail image file is as expected
"""
mock_generate_pdf.return_value = os.path.join(
self.SAMPLE_FILES,
"simple_text.eml.pdf",
)
thumb = self.parser.get_thumbnail(
os.path.join(self.SAMPLE_FILES, "simple_text.eml"),
"message/rfc822",
)
self.assertTrue(os.path.isfile(thumb))
expected = os.path.join(self.SAMPLE_FILES, "simple_text.eml.pdf.webp")
self.assertEqual(
self.imagehash(thumb),
self.imagehash(expected),
f"Created Thumbnail {thumb} differs from expected file {expected}",
)
@pytest.mark.skipif(
"TIKA_LIVE" not in os.environ,
reason="No tika server",
)
def test_tika_parse_successful(self):
"""
GIVEN:
- Fresh start
WHEN:
- tika parsing is called
THEN:
- a web request to tika shall be done and the reply es returned
"""
html = '<html><head><meta http-equiv="content-type" content="text/html; charset=UTF-8"></head><body><p>Some Text</p></body></html>'
expected_text = "Some Text"
# Check successful parsing
parsed = self.parser.tika_parse(html)
self.assertEqual(expected_text, parsed.strip())
@pytest.mark.skipif(
"TIKA_LIVE" not in os.environ,
reason="No tika server",
)
def test_tika_parse_unsuccessful(self):
"""
GIVEN:
- Fresh start
WHEN:
- tika parsing fails
THEN:
- the parser should return an empty string
"""
# Check unsuccessful parsing
parsed = self.parser.tika_parse(None)
self.assertEqual("", parsed)
@pytest.mark.skipif(
"GOTENBERG_LIVE" not in os.environ,
reason="No gotenberg server",
)
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf_from_mail")
@mock.patch("paperless_mail.parsers.MailDocumentParser.generate_pdf_from_html")
def test_generate_pdf_gotenberg_merging(
self,
mock_generate_pdf_from_html: mock.MagicMock,
mock_generate_pdf_from_mail: mock.MagicMock,
):
"""
GIVEN:
- Intermediary pdfs to be merged
WHEN:
- pdf generation is requested with html file requiring merging of pdfs
THEN:
- gotenberg is called to merge files and the resulting file is returned
"""
with open(os.path.join(self.SAMPLE_FILES, "first.pdf"), "rb") as first:
mock_generate_pdf_from_mail.return_value = first.read()
with open(os.path.join(self.SAMPLE_FILES, "second.pdf"), "rb") as second:
mock_generate_pdf_from_html.return_value = second.read()
pdf_path = self.parser.generate_pdf(os.path.join(self.SAMPLE_FILES, "html.eml"))
self.assertTrue(os.path.isfile(pdf_path))
extracted = extract_text(pdf_path)
expected = (
"first\tPDF\tto\tbe\tmerged.\n\n\x0csecond\tPDF\tto\tbe\tmerged.\n\n\x0c"
)
self.assertEqual(expected, extracted)
@pytest.mark.skipif(
"GOTENBERG_LIVE" not in os.environ,
reason="No gotenberg server",
)
def test_generate_pdf_from_mail_no_convert(self):
"""
GIVEN:
- Fresh start
WHEN:
- pdf generation from simple eml file is requested
THEN:
- gotenberg is called and the resulting file is returned and contains the expected text.
"""
mail = self.parser.get_parsed(os.path.join(self.SAMPLE_FILES, "html.eml"))
pdf_path = os.path.join(self.parser.tempdir, "html.eml.pdf")
with open(pdf_path, "wb") as file:
file.write(self.parser.generate_pdf_from_mail(mail))
extracted = extract_text(pdf_path)
expected = extract_text(os.path.join(self.SAMPLE_FILES, "html.eml.pdf"))
self.assertEqual(expected, extracted)
@pytest.mark.skipif(
"GOTENBERG_LIVE" not in os.environ,
reason="No gotenberg server",
)
# Only run if convert is available
@pytest.mark.skipif(
"PAPERLESS_TEST_SKIP_CONVERT" in os.environ,
reason="PAPERLESS_TEST_SKIP_CONVERT set, skipping Test",
)
def test_generate_pdf_from_mail(self):
"""
GIVEN:
- Fresh start
WHEN:
- pdf generation from simple eml file is requested
THEN:
- gotenberg is called and the resulting file is returned and look as expected.
"""
mail = self.parser.get_parsed(os.path.join(self.SAMPLE_FILES, "html.eml"))
pdf_path = os.path.join(self.parser.tempdir, "html.eml.pdf")
with open(pdf_path, "wb") as file:
file.write(self.parser.generate_pdf_from_mail(mail))
converted = os.path.join(
self.parser.tempdir,
"html.eml.pdf.webp",
)
run_convert(
density=300,
scale="500x5000>",
alpha="remove",
strip=True,
trim=False,
auto_orient=True,
input_file=f"{pdf_path}", # Do net define an index to convert all pages.
output_file=converted,
logging_group=None,
)
self.assertTrue(os.path.isfile(converted))
thumb_hash = self.imagehash(converted)
# The created pdf is not reproducible. But the converted image should always look the same.
expected_hash = self.imagehash(
os.path.join(self.SAMPLE_FILES, "html.eml.pdf.webp"),
)
self.assertEqual(
thumb_hash,
expected_hash,
f"PDF looks different. Check if {converted} looks weird.",
)
@pytest.mark.skipif(
"GOTENBERG_LIVE" not in os.environ,
reason="No gotenberg server",
)
def test_generate_pdf_from_html_no_convert(self):
"""
GIVEN:
- Fresh start
WHEN:
- pdf generation from html eml file is requested
THEN:
- gotenberg is called and the resulting file is returned and contains the expected text.
"""
class MailAttachmentMock:
def __init__(self, payload, content_id):
self.payload = payload
self.content_id = content_id
result = None
with open(os.path.join(self.SAMPLE_FILES, "sample.html")) as html_file:
with open(os.path.join(self.SAMPLE_FILES, "sample.png"), "rb") as png_file:
html = html_file.read()
png = png_file.read()
attachments = [
MailAttachmentMock(png, "part1.pNdUSz0s.D3NqVtPg@example.de"),
]
result = self.parser.generate_pdf_from_html(html, attachments)
pdf_path = os.path.join(self.parser.tempdir, "sample.html.pdf")
with open(pdf_path, "wb") as file:
file.write(result)
extracted = extract_text(pdf_path)
expected = extract_text(os.path.join(self.SAMPLE_FILES, "sample.html.pdf"))
self.assertEqual(expected, extracted)
@pytest.mark.skipif(
"GOTENBERG_LIVE" not in os.environ,
reason="No gotenberg server",
)
# Only run if convert is available
@pytest.mark.skipif(
"PAPERLESS_TEST_SKIP_CONVERT" in os.environ,
reason="PAPERLESS_TEST_SKIP_CONVERT set, skipping Test",
)
def test_generate_pdf_from_html(self):
"""
GIVEN:
- Fresh start
WHEN:
- pdf generation from html eml file is requested
THEN:
- gotenberg is called and the resulting file is returned and look as expected.
"""
class MailAttachmentMock:
def __init__(self, payload, content_id):
self.payload = payload
self.content_id = content_id
result = None
with open(os.path.join(self.SAMPLE_FILES, "sample.html")) as html_file:
with open(os.path.join(self.SAMPLE_FILES, "sample.png"), "rb") as png_file:
html = html_file.read()
png = png_file.read()
attachments = [
MailAttachmentMock(png, "part1.pNdUSz0s.D3NqVtPg@example.de"),
]
result = self.parser.generate_pdf_from_html(html, attachments)
pdf_path = os.path.join(self.parser.tempdir, "sample.html.pdf")
with open(pdf_path, "wb") as file:
file.write(result)
converted = os.path.join(self.parser.tempdir, "sample.html.pdf.webp")
run_convert(
density=300,
scale="500x5000>",
alpha="remove",
strip=True,
trim=False,
auto_orient=True,
input_file=f"{pdf_path}", # Do net define an index to convert all pages.
output_file=converted,
logging_group=None,
)
self.assertTrue(os.path.isfile(converted))
thumb_hash = self.imagehash(converted)
# The created pdf is not reproducible. But the converted image should always look the same.
expected_hash = self.imagehash(
os.path.join(self.SAMPLE_FILES, "sample.html.pdf.webp"),
)
self.assertEqual(
thumb_hash,
expected_hash,
f"PDF looks different. Check if {converted} looks weird. "
f"If Rick Astley is shown, Gotenberg loads from web which is bad for Mail content.",
)
@pytest.mark.skipif(
"GOTENBERG_LIVE" not in os.environ,
reason="No gotenberg server",
)
def test_online_image_exception_on_not_available(self):
"""
GIVEN:
- Fresh start
WHEN:
- nonexistent image is requested
THEN:
- An exception shall be thrown
"""
"""
A public image is used in the html sample file. We have no control
whether this image stays online forever, so here we check if we can detect if is not
available anymore.
"""
# Start by Testing if nonexistent URL really throws an Exception
self.assertRaises(
HTTPError,
urlopen,
"https://upload.wikimedia.org/wikipedia/en/f/f7/nonexistent.png",
)
@pytest.mark.skipif(
"GOTENBERG_LIVE" not in os.environ,
reason="No gotenberg server",
)
def test_is_online_image_still_available(self):
"""
GIVEN:
- Fresh start
WHEN:
- A public image used in the html sample file is requested
THEN:
- No exception shall be thrown
"""
"""
A public image is used in the html sample file. We have no control
whether this image stays online forever, so here we check if it is still there
"""
# Now check the URL used in samples/sample.html
urlopen("https://upload.wikimedia.org/wikipedia/en/f/f7/RickRoll.png")

41
src/paperless_mail/views.py Normal file
View File

@@ -0,0 +1,41 @@
from paperless.views import StandardPagination
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from paperless_mail.serialisers import MailAccountSerializer
from paperless_mail.serialisers import MailRuleSerializer
from rest_framework.permissions import IsAuthenticated
from rest_framework.viewsets import ModelViewSet
class MailAccountViewSet(ModelViewSet):
model = MailAccount
queryset = MailAccount.objects.all().order_by("pk")
serializer_class = MailAccountSerializer
pagination_class = StandardPagination
permission_classes = (IsAuthenticated,)
# TODO: user-scoped
# def get_queryset(self):
# user = self.request.user
# return MailAccount.objects.filter(user=user)
# def perform_create(self, serializer):
# serializer.save(user=self.request.user)
class MailRuleViewSet(ModelViewSet):
model = MailRule
queryset = MailRule.objects.all().order_by("pk")
serializer_class = MailRuleSerializer
pagination_class = StandardPagination
permission_classes = (IsAuthenticated,)
# TODO: user-scoped
# def get_queryset(self):
# user = self.request.user
# return MailRule.objects.filter(user=user)
# def perform_create(self, serializer):
# serializer.save(user=self.request.user)