89 Commits

Author SHA1 Message Date
Trenton H
1d3f525f92 Ensure all creations of directories create the parents too (#5711) 2024-02-10 11:02:40 -08:00
Henning B
5638871934 Enhancement: Respect PDF cropbox for thumbnail generation (#5531)
* Use the convert argument '-define "pdf:use-cropbox=true"' when creating thumbnails. That way cropboxes, if present, gets respected for thumbnail generation
2024-02-01 11:25:57 -08:00
shamoon
9792514d8d Fix: Modify one of date regexes (#5540) 2024-01-25 21:29:22 +00:00
shamoon
f52ba29baf Chore: add pre-commit hook for codespell (#5324) 2024-01-08 13:03:05 -08:00
Trenton H
eb2caa5118 Fix: Allows pre-consume scripts to modify the working path again (#5260)
* Allows pre-consume scripts to modify the working path again and generally cleans up some confusion about working copy vs original
2024-01-05 21:01:57 -08:00
Trenton H
c8a62715ec Feature: Allow setting backend configuration settings via the UI (#5126)
* Saving some start on this

* At least partially working for the tesseract parser

* Problems with migration testing need to figure out

* Work around that error

* Fixes max m_pixels

* Moving the settings to main paperless application

* Starting some consumer options

* More fixes and work

* Fixes these last tests

* Fix max_length on OcrSettings.mode field

* Fix all fields on Common & Ocr settings serializers

* Umbrellla config view

* Revert "Umbrellla config view"

This reverts commit fbaf9f4be30f89afeb509099180158a3406416a5.

* Updates to use a single configuration object for all settings

* Squashed commit of the following:

commit 8a0a49dd5766094f60462fbfbe62e9921fbd2373
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 23:02:47 2023 -0800

    Fix formatting

commit 66b2d90c507b8afd9507813ff555e46198ea33b9
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 22:36:35 2023 -0800

    Refactor frontend data models

commit 5723bd8dd823ee855625e250df39393e26709d48
Author: Adam Bogdał <adam@bogdal.pl>
Date:   Wed Dec 20 01:17:43 2023 +0100

    Fix: speed up admin panel for installs with a large number of documents (#5052)

commit 9b08ce176199bf9011a6634bb88f616846150d2b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:18:51 2023 -0800

    Update PULL_REQUEST_TEMPLATE.md

commit a6248bec2d793b7690feed95fcaf5eb34a75bfb6
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:02:05 2023 -0800

    Chore: Update Angular to v17 (#4980)

commit b1f6f52486d5ba5c04af99b41315eb6428fd1fa8
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:53:56 2023 -0800

    Fix: Dont allow null custom_fields property via API (#5063)

commit 638d9970fd468d8c02c91d19bd28f8b0796bdcb1
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:43:50 2023 -0800

    Enhancement: symmetric document links (#4907)

commit 5e8de4c1da6eb4eb8f738b20962595c7536b30ec
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 12:45:04 2023 -0800

    Enhancement: shared icon & shared by me filter (#4859)

commit 088bad90306025d3f6b139cbd0ad264a1cbecfe5
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Dec 19 12:04:03 2023 -0800

    Bulk updates all the backend libraries (#5061)

* Saving some work on frontend config

* Very basic but dynamically-generated config form

* Saving work on slightly less ugly frontend config

* JSON validation for user_args field

* Fully dynamic config form

* Adds in some additional validators for a nicer error message

* Cleaning up the testing and coverage more

* Reverts unintentional change

* Adds documentation about the settings and the precedence

* Couple more commenting and style fixes

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2023-12-29 15:42:56 -08:00
Trenton H
dc5ea43aed Use the original image file for the checksum, not the maybe alpha removed version (#4781) 2023-12-02 16:18:06 -08:00
Trenton H
ab9e561317 Chore: Backend bulk updates (#4509) 2023-11-13 17:09:56 +00:00
Trenton H
6317fc2294 Logs the errors during thumbnail generation, updates default to be WebP 2023-09-12 12:16:22 -07:00
Trenton Holmes
34b80a4d8e Removes support for Python 3.8 and lower from the code base 2023-09-10 11:42:59 -07:00
brainrecursion
b6afccd8cd Remove debug 2023-07-30 08:34:33 -07:00
brainrecursion
7900d31485 Add support for more date formats 2023-07-30 08:34:33 -07:00
Trenton H
e160580c8b Fixes issues with copy2 or copystat and SELinux see #3665 2023-07-22 06:27:49 -07:00
Johannes Plunien
3a58a5f123 Copy default thumbnail if thumbnail generation fails
Fix #3631
2023-06-20 11:28:46 -07:00
Trenton H
bad8d304cb Improves the logging mixin and allows it to be typed better 2023-05-23 17:16:39 -07:00
Trenton Holmes
2f12206911 Changes the error mode to replace instead of ignore, to better highlight where a problem happened 2023-05-13 09:29:18 -07:00
Trenton H
6722b6e31c Adds better handling for files with invalid utf8 content 2023-05-13 09:29:18 -07:00
Trenton H
aabcc9a1c4 Upgrades black to v23, upgrades ruff 2023-04-26 09:35:27 -07:00
Trenton H
30655f1b73 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Trenton H
d2c02b9102 Configures ruff as the one stop linter and resolves warnings it raised 2023-04-01 17:03:52 -07:00
Trenton H
d58747c912 relock with Python 3.8.15 2023-01-06 17:59:39 -08:00
Trenton H
8504b6f7da Cleans up and improves parser discovery testing, simplifies the determination of supported or not supported extensions and mime types 2023-01-05 08:39:48 -08:00
Trenton H
cdfcbff529 Don't allow an exception when trying to parse a date cause complete failure 2022-11-17 13:37:37 -08:00
Matthias Eck
05d97d2cf1 fix(parsers|test_api): fix failed tests 2022-08-06 19:19:10 +02:00
Matthias Eck
1195fb9afe feat(parsers): add generator for date parsing 2022-08-06 13:03:20 +02:00
Trenton Holmes
ef6ebf9888 Entirely removes the optipng, updates ghostscript fall back to also use WebP. Updates the conversion to use a multiprocessing pool 2022-06-11 08:38:49 -07:00
Michael Shamoon
f208f89179 webp thumbnail support with png fallback 2022-06-10 02:28:13 -07:00
shamoon
3ccf143c0b Merge pull request #721 from paperless-ngx/bug-fix-date-ignore
Fix Ignore Date Parsing
2022-05-10 16:45:58 -07:00
Trenton Holmes
304d5b0d5a Updates the ignore date parsing to utilize the settings defined date order, instead of guessing a bit 2022-05-08 16:57:35 -07:00
Trenton Holmes
a944ef1ca6 Adds additional testing for both date parsing and consumed document created date 2022-05-08 16:57:35 -07:00
Trenton Holmes
f62193099c Runs pyupgrade to Python 3.8+ and adds a hook for it 2022-05-06 09:04:08 -07:00
Fantasticle
6982641398 update new regex pattern for second boundary 2022-03-31 09:37:15 +02:00
fantasticle
95fdcab953 Update regex date match patterns 2022-03-30 12:19:30 +02:00
Simon Siebert
5aea4da8b2 Update parsers.py and test_consumer.py 2022-03-14 19:03:09 +01:00
Trenton Holmes
6635fa5f0d Runs the pre-commit hooks over all the Python files 2022-03-11 11:34:28 -08:00
kpj
c56cb25b5f Format Python code with black 2022-02-27 15:26:41 +01:00
jonaswinkler
3a67462396 fixes #631 2021-03-14 14:42:48 +01:00
jonaswinkler
f8f49bac75 only import dateparser when required 2021-02-15 11:52:46 +01:00
jonaswinkler
b04d91d68c fix a bug with thumbnail generation when TIKA was enabled 2021-02-09 22:12:43 +01:00
jonaswinkler
e5a7dc0cc7 rework most of the logging 2021-02-05 01:10:29 +01:00
jonaswinkler
eeff7b3bdb code style 2021-02-02 23:58:25 +01:00
jonaswinkler
5f7d817d69 localization for websockets 2021-01-28 22:06:02 +01:00
jonaswinkler
c0f185fe7e bug fixes, test case fixes 2021-01-26 15:19:56 +01:00
jonaswinkler
044aa55d74 Merge branch 'dev' into feature-websockets-status 2021-01-23 22:22:17 +01:00
Jonas Winkler
22f45ac619 Merge pull request #251 from jayme-github/ignore-date
Add option to ignore certain dates in parse_date
2021-01-05 00:19:13 +01:00
jonaswinkler
179b53d373 Merge branch 'dev' into feature-websockets-status 2021-01-04 22:45:56 +01:00
jonaswinkler
e2680b7113 code style 2021-01-02 15:26:09 +01:00
jayme-github
cd15490e91 Add option to ignore certain dates in parse_date
PAPERLESS_IGNORE_DATES allows to specify a comma separated list of dates
to ignore during date parsing (from filename and content). This can be
used so specify dates that do appear often in documents but are usually
not the documents creation date (like your date of birth).
2021-01-02 15:20:49 +01:00
jonaswinkler
755f950cd2 supply file_name for tika parser 2021-01-01 22:19:43 +01:00
jonaswinkler
f1e9b414f9 remove duplicate code 2021-01-01 21:50:45 +01:00