37 Commits

Author SHA1 Message Date
Trenton H
0b1523f4e5
Fix: Test metadata items for Unicode issues (#5707)
Test each key for unicode issues and reject ones which will fail inside DRF
2024-02-09 20:08:23 +00:00
Trenton H
061f33fb05
Feature: Allow setting backend configuration settings via the UI (#5126)
* Saving some start on this

* At least partially working for the tesseract parser

* Problems with migration testing need to figure out

* Work around that error

* Fixes max m_pixels

* Moving the settings to main paperless application

* Starting some consumer options

* More fixes and work

* Fixes these last tests

* Fix max_length on OcrSettings.mode field

* Fix all fields on Common & Ocr settings serializers

* Umbrellla config view

* Revert "Umbrellla config view"

This reverts commit fbaf9f4be30f89afeb509099180158a3406416a5.

* Updates to use a single configuration object for all settings

* Squashed commit of the following:

commit 8a0a49dd5766094f60462fbfbe62e9921fbd2373
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 23:02:47 2023 -0800

    Fix formatting

commit 66b2d90c507b8afd9507813ff555e46198ea33b9
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 22:36:35 2023 -0800

    Refactor frontend data models

commit 5723bd8dd823ee855625e250df39393e26709d48
Author: Adam Bogdał <adam@bogdal.pl>
Date:   Wed Dec 20 01:17:43 2023 +0100

    Fix: speed up admin panel for installs with a large number of documents (#5052)

commit 9b08ce176199bf9011a6634bb88f616846150d2b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:18:51 2023 -0800

    Update PULL_REQUEST_TEMPLATE.md

commit a6248bec2d793b7690feed95fcaf5eb34a75bfb6
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:02:05 2023 -0800

    Chore: Update Angular to v17 (#4980)

commit b1f6f52486d5ba5c04af99b41315eb6428fd1fa8
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:53:56 2023 -0800

    Fix: Dont allow null custom_fields property via API (#5063)

commit 638d9970fd468d8c02c91d19bd28f8b0796bdcb1
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:43:50 2023 -0800

    Enhancement: symmetric document links (#4907)

commit 5e8de4c1da6eb4eb8f738b20962595c7536b30ec
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 12:45:04 2023 -0800

    Enhancement: shared icon & shared by me filter (#4859)

commit 088bad90306025d3f6b139cbd0ad264a1cbecfe5
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Dec 19 12:04:03 2023 -0800

    Bulk updates all the backend libraries (#5061)

* Saving some work on frontend config

* Very basic but dynamically-generated config form

* Saving work on slightly less ugly frontend config

* JSON validation for user_args field

* Fully dynamic config form

* Adds in some additional validators for a nicer error message

* Cleaning up the testing and coverage more

* Reverts unintentional change

* Adds documentation about the settings and the precedence

* Couple more commenting and style fixes

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2023-12-29 15:42:56 -08:00
Trenton H
771c1fab92
Chore: Raise Gotenberg container version (#4815)
* Updates the Gotenberg version to use 7.10 and gotenberg-client to match
* Fixes a long standing bug in this test where a whole page was missing from the expected
2023-12-05 15:36:25 +00:00
Trenton H
999ae678c2
Feature: Switches to a new client to handle communication with Gotenberg (#4391)
Switches to a new client to handle communication with Gotenberg for merging and generating PDFs
2023-10-20 00:27:29 +00:00
Trenton Holmes
650c816a7b Removes support for Python 3.8 and lower from the code base 2023-09-10 11:42:59 -07:00
Trenton H
d532913d56 Unlocks all things which were locked due to ARMv7, etc 2023-09-08 11:21:43 -07:00
Trenton Holmes
fe1f88ce5d Sets the http timeouts equal to the task timeout, so it's either done or really done 2023-08-23 18:40:22 -07:00
Trenton H
e05b3441de Updates tika client library and handle the changes to it 2023-06-26 10:41:05 -06:00
Trenton H
0d6e79cb93 Fixes generation of thumbnails when the archive file hasn't already been created 2023-06-26 10:36:50 -06:00
Trenton H
74fe7c586b Updates the httpx timeout to be 30s for all operations 2023-06-19 08:59:51 -07:00
Trenton Holmes
4782b4da07 Adds better error handling/checking around getting content of a document via Tika
Signed-off-by: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
2023-06-18 08:39:17 -07:00
Trenton H
2c1cd25be4 Rewrites the email parsing to be more clear and concise.
Adds testing to use httpx mocked responses to stand in as a server even offline
2023-06-06 09:05:26 -07:00
Trenton H
6e65558ea4 Swapping out the tika and replaces requests with httpx 2023-06-06 09:05:26 -07:00
Trenton H
452c79f9a1 Improves the logging mixin and allows it to be typed better 2023-05-23 17:16:39 -07:00
Trenton H
6f163111ce Upgrades black to v23, upgrades ruff 2023-04-26 09:35:27 -07:00
Trenton H
3bcbd05252 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Trenton H
ce41ac9158 Configures ruff as the one stop linter and resolves warnings it raised 2023-04-01 17:03:52 -07:00
Trenton H
1b2cb13a21 Adds setting to Gotenberg API call for outputting the correct PDF/A format 2023-01-27 11:05:23 -08:00
Trenton Holmes
28b26eb4c7 Check for and handle the mail's date being naive 2022-12-31 12:35:42 -08:00
phail
d9796e5003 change order of elements in parsed Texts 2022-11-20 20:24:36 +01:00
phail
af8a6c3764 fix filenames 2022-11-20 19:53:57 +01:00
Trenton Holmes
6d37ebf79e Fixes one more place which used manual size formatting 2022-11-20 09:15:06 -08:00
Trenton Holmes
f6a70b85f4 Use Django templating engine 2022-11-20 09:13:16 -08:00
phail
ebe21a0114 eml parsing requires tika 2022-11-20 14:22:30 +01:00
phail
d132eba143 optimize regex 2022-11-20 12:48:03 +01:00
phail
073c3c8fed use html.escape instead of some self build functions 2022-11-20 12:36:49 +01:00
phail
e3c1bde793 remove log mocking, replace pytest raises, use humanfriendly 2022-11-20 12:06:35 +01:00
phail
0da0b1c062 update variable names 2022-10-23 21:39:15 +02:00
phail
30372b0e85 add tests for mail_to_html and generate_pdf_from_mail 2022-10-23 17:18:10 +02:00
phail
567e89d1c7 test for broken eml, add test_generate_pdf 2022-10-22 02:25:23 +02:00
phail
e384bd78c5 add unittest for transform_inline_html 2022-10-18 23:48:07 +02:00
phail
daf90399bd Add unitest for tika_parse() 2022-10-15 13:13:29 +02:00
phail
c317eca1ca add attachment size to pdf 2022-05-05 00:20:32 +02:00
phail
466afa8203 fix consumption of mails without html
split pdf generation functions
2022-05-04 23:42:59 +02:00
phail
c2e3dc76d9 add parsing of inline attachments
remove insecure chromium option
fix html parsing
2022-05-03 23:21:33 +02:00
phail
5a899664f8 remove .eml parser from tika 2022-05-03 18:02:08 +02:00
phail
47189643ff add eml parser to paperless_mail 2022-04-29 22:58:11 +02:00