110 Commits

Author SHA1 Message Date
Trenton H
a6a51e4a98 Chore: Convert the consumer to a plugin (#6361) 2024-04-18 02:59:14 +00:00
Trenton H
75bc6b3ba8 Chore: Standardize subprocess running and logging (#6275) 2024-04-04 13:11:43 -07:00
shamoon
aabe4c48fc Enhancement: title assignment placeholder error handling, fallback (#5282) 2024-01-10 10:18:55 -08:00
shamoon
f52ba29baf Chore: add pre-commit hook for codespell (#5324) 2024-01-08 13:03:05 -08:00
Trenton H
eb2caa5118 Fix: Allows pre-consume scripts to modify the working path again (#5260)
* Allows pre-consume scripts to modify the working path again and generally cleans up some confusion about working copy vs original
2024-01-05 21:01:57 -08:00
Trenton H
c8a62715ec Feature: Allow setting backend configuration settings via the UI (#5126)
* Saving some start on this

* At least partially working for the tesseract parser

* Problems with migration testing need to figure out

* Work around that error

* Fixes max m_pixels

* Moving the settings to main paperless application

* Starting some consumer options

* More fixes and work

* Fixes these last tests

* Fix max_length on OcrSettings.mode field

* Fix all fields on Common & Ocr settings serializers

* Umbrellla config view

* Revert "Umbrellla config view"

This reverts commit fbaf9f4be30f89afeb509099180158a3406416a5.

* Updates to use a single configuration object for all settings

* Squashed commit of the following:

commit 8a0a49dd5766094f60462fbfbe62e9921fbd2373
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 23:02:47 2023 -0800

    Fix formatting

commit 66b2d90c507b8afd9507813ff555e46198ea33b9
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 22:36:35 2023 -0800

    Refactor frontend data models

commit 5723bd8dd823ee855625e250df39393e26709d48
Author: Adam Bogdał <adam@bogdal.pl>
Date:   Wed Dec 20 01:17:43 2023 +0100

    Fix: speed up admin panel for installs with a large number of documents (#5052)

commit 9b08ce176199bf9011a6634bb88f616846150d2b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:18:51 2023 -0800

    Update PULL_REQUEST_TEMPLATE.md

commit a6248bec2d793b7690feed95fcaf5eb34a75bfb6
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:02:05 2023 -0800

    Chore: Update Angular to v17 (#4980)

commit b1f6f52486d5ba5c04af99b41315eb6428fd1fa8
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:53:56 2023 -0800

    Fix: Dont allow null custom_fields property via API (#5063)

commit 638d9970fd468d8c02c91d19bd28f8b0796bdcb1
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:43:50 2023 -0800

    Enhancement: symmetric document links (#4907)

commit 5e8de4c1da6eb4eb8f738b20962595c7536b30ec
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 12:45:04 2023 -0800

    Enhancement: shared icon & shared by me filter (#4859)

commit 088bad90306025d3f6b139cbd0ad264a1cbecfe5
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Dec 19 12:04:03 2023 -0800

    Bulk updates all the backend libraries (#5061)

* Saving some work on frontend config

* Very basic but dynamically-generated config form

* Saving work on slightly less ugly frontend config

* JSON validation for user_args field

* Fully dynamic config form

* Adds in some additional validators for a nicer error message

* Cleaning up the testing and coverage more

* Reverts unintentional change

* Adds documentation about the settings and the precedence

* Couple more commenting and style fixes

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2023-12-29 15:42:56 -08:00
shamoon
084c31b37a Enhancement: support assigning custom fields via consumption templates (#4727) 2023-12-03 15:35:30 -08:00
Trenton H
dc642152d1 Standarizes the imports across all the files and modules (#4248) 2023-09-23 20:17:01 -07:00
shamoon
54783f706f Feature: consumption templates (#4196)
* Initial implementation of consumption templates

* Frontend implementation of consumption templates

Testing

* Support consumption template source

* order templates, automatically add permissions

* Support title assignment in consumption templates

* Refactoring, filters to and, show sources on list

Show sources on template list, update some translation strings

Make filters and

minor testing

* Update strings

* Only update django-multiselectfield

* Basic docs, document some methods

* Improve testing coverage, template multi-assignment merges
2023-09-22 16:53:13 -07:00
Trenton Holmes
34b80a4d8e Removes support for Python 3.8 and lower from the code base 2023-09-10 11:42:59 -07:00
Trenton H
128890a350 Merge pull request #4037 from andreheuer/dev
Enhancement: add task id to pre/post consume script as env
2023-09-08 10:00:05 -07:00
shamoon
516d1fcb38 Fix consumer error typo 2023-09-01 00:11:32 -07:00
shamoon
9ed9dbb369 Fix: ghostscript rendering error doesnt trigger frontend failure message (#4092)
* Raise ParseError from gs rendering error

* catch all parser errors as generic exception

* Differentiate generic vs parse errors during consumption
2023-08-31 19:49:00 -07:00
André Heuer
7d53f8d7d3 Removed parameter, added documentation 2023-08-29 23:09:47 -07:00
André Heuer
d8ff6dc692 Added task id to pre/post consume script as env 2023-08-29 23:09:47 -07:00
Trenton Holmes
41839728fe Small improvement to the consumer status with stronger typing 2023-07-26 07:03:43 -07:00
Trenton H
f05d58827d Also handles confirming returned predictions are still automatic matching, in case the classifier hasn't been run since a type was changed 2023-07-24 12:31:56 -07:00
Trenton H
bad8d304cb Improves the logging mixin and allows it to be typed better 2023-05-23 17:16:39 -07:00
Trenton H
aabcc9a1c4 Upgrades black to v23, upgrades ruff 2023-04-26 09:35:27 -07:00
Trenton H
30655f1b73 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Trenton H
d2c02b9102 Configures ruff as the one stop linter and resolves warnings it raised 2023-04-01 17:03:52 -07:00
Trenton Holmes
acfa7d633d Creates a mix-in for asserting file system states 2023-02-20 10:25:21 -08:00
Trenton Holmes
7fbcc4f97e Fixes up some minor warnings from test code 2023-02-11 14:35:16 -08:00
Trenton H
5941c22c33 Changes the consumer to work on a temporary copy and provies that copy to the pre-consume script for modifications 2023-01-29 08:37:32 -08:00
Trenton H
6f23cfe78c Resolves minor flake8 warnings in the test suite 2023-01-05 08:39:48 -08:00
Trenton H
2228cc292d Adds further testing to cover scripts with non-zero exit codes 2022-11-11 08:58:49 -08:00
Trenton Holmes
81c98103a8 Captures the stdout and stderr of the pre/post scripts into the log 2022-11-10 17:26:47 -08:00
Trenton H
cce0d363e4 Refresh the instance from the database before doing any file handling with it 2022-11-03 11:32:27 -07:00
Trenton Holmes
ee4e3cebe2 Removes last vestiges of PNG from the tests, code, docs and samples 2022-06-11 14:20:50 -07:00
Trenton Holmes
ef6ebf9888 Entirely removes the optipng, updates ghostscript fall back to also use WebP. Updates the conversion to use a multiprocessing pool 2022-06-11 08:38:49 -07:00
Markus
dd3b5c129c Feature: Dynamic document storage pathes (#916)
* Added devcontainer

* Add feature storage pathes

* Exclude tests and add versioning

* Check escaping

* Check escaping

* Check quoting

* Echo

* Escape

* Escape :

* Double escape \

* Escaping

* Remove if

* Escape colon

* Missing \

* Esacpe :

* Escape all

* test

* Remove sed

* Fix exclude

* Remove SED command

* Add LD_LIBRARY_PATH

* Adjusted to v1.7

* Updated test-cases

* Remove devcontainer

* Removed internal build-file

* Run pre-commit

* Corrected flak8 error

* Adjusted to v1.7

* Updated test-cases

* Corrected flak8 error

* Adjusted to new plural translations

* Small adjustments due to code-review backend

* Adjusted line-break

* Removed PAPERLESS prefix from settings variables

* Corrected style change due to search+replace

* First documentation draft

* Revert changes to Pipfile

* Add sphinx-autobuild with keep-outdated

* Revert merge error that results in wrong storage path is evaluated

* Adjust styles of generated files ...

* Adds additional testing to cover dynamic storage path functionality

* Remove unnecessary condition

* Add hint to edit storage path dialog

* Correct spelling of pathes to paths

* Minor documentation tweaks

* Minor typo

* improving wrapping of filter editor buttons with new storage path button

* Update .gitignore

* Fix select border radius in non input-groups

* Better storage path edit hint

* Add note to edit storage path dialog re document_renamer

* Add note to bulk edit storage path re document_renamer

* Rename FILTER_STORAGE_DIRECTORY to PATH

* Fix broken filter rule parsing

* Show default storage if unspecified

* Remove note re storage path on bulk edit

* Add basic validation of filename variables

Co-authored-by: Markus Kling <markus@markus-kling.net>
Co-authored-by: Trenton Holmes <holmes.trenton@gmail.com>
Co-authored-by: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Co-authored-by: Quinn Casey <quinn@quinncasey.com>
2022-05-19 14:42:25 -07:00
shamoon
3ccf143c0b Merge pull request #721 from paperless-ngx/bug-fix-date-ignore
Fix Ignore Date Parsing
2022-05-10 16:45:58 -07:00
Trenton Holmes
a944ef1ca6 Adds additional testing for both date parsing and consumed document created date 2022-05-08 16:57:35 -07:00
Trenton Holmes
f62193099c Runs pyupgrade to Python 3.8+ and adds a hook for it 2022-05-06 09:04:08 -07:00
Trenton Holmes
5a4741dcab Bumps django to 4.0 and fixes associated issues 2022-03-24 07:39:11 -07:00
Simon Siebert
58fbfb2e17 Fixing Test 2022-03-14 19:03:09 +01:00
Simon Siebert
4bdbf98c64 Fixing code formating 2022-03-14 19:03:09 +01:00
Simon Siebert
5aea4da8b2 Update parsers.py and test_consumer.py 2022-03-14 19:03:09 +01:00
Trenton Holmes
6635fa5f0d Runs the pre-commit hooks over all the Python files 2022-03-11 11:34:28 -08:00
kpj
c56cb25b5f Format Python code with black 2022-02-27 15:26:41 +01:00
jonaswinkler
56bbe644b8 codestyle 2021-05-19 20:26:12 +02:00
jonaswinkler
5d5afb210a delete macOS specific files 2021-05-19 20:02:47 +02:00
jonaswinkler
b04d91d68c fix a bug with thumbnail generation when TIKA was enabled 2021-02-09 22:12:43 +01:00
jonaswinkler
866d6f814d update all test cases to address the archive filename changes 2021-02-09 19:51:16 +01:00
jonaswinkler
4a7ccd1bae another way to make the test case fail 2021-02-09 02:13:25 +01:00
jonaswinkler
d5ecc5c8e5 make the test case fail 2021-02-07 19:53:08 +01:00
jonaswinkler
817bb299a9 added a test case that replicates #511 2021-02-07 18:23:54 +01:00
jonaswinkler
fce76d0671 Merge branch 'dev' into feature-websockets-status 2021-01-30 16:08:50 +01:00
jonaswinkler
a37e41ef0c centralized classifier loading, better error handling, no error messages when auto matching is not used 2021-01-30 14:22:23 +01:00
jonaswinkler
00cdd2b59b test cases 2021-01-28 19:28:48 +01:00