paradizelost/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2025-04-02 13:45:10 -05:00
Code Issues Packages Projects Releases Wiki Activity
7,646 Commits 29 Branches 180 Tags
Commit Graph

57 Commits

Author SHA1 Message Date
Trenton H
6f163111ce Upgrades black to v23, upgrades ruff 2023-04-26 09:35:27 -07:00
Trenton H
3bcbd05252 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Trenton H
ce41ac9158 Configures ruff as the one stop linter and resolves warnings it raised 2023-04-01 17:03:52 -07:00
Trenton H
41bcfcaffe Changes out the settings and a decent amount of test code to be pathlib compatible 2023-03-06 09:16:07 -08:00
Trenton Holmes
6b939f7567 Returns to using hashing against primary keys, at least for fields. Improves testing coverage 2023-02-28 08:13:10 -08:00
Trenton Holmes
c958a7c593 Changes from a hash based system to a time based system to prevent extra retrains 2023-02-28 08:13:10 -08:00
Trenton H
8709ea4df0 Changes classifier training to hold less data in memory at the same time 2023-02-28 08:13:10 -08:00
Trenton H
1e891414a3 Allows disabling NLTK, adds it as a consideration for low power devices 2022-10-10 08:58:23 -07:00
Trenton Holmes
c44c914d3d Changes the NLTK language to be based on the Tesseract OCR language, with fallback to the default processing 2022-10-10 08:58:23 -07:00
Trenton H
d10d2f5a54 Allows configuration of the NLTK processing language 2022-10-10 08:58:23 -07:00
Trenton Holmes
6523cf0c4b Fixes the download and usage of the downloaded data 2022-10-10 08:58:23 -07:00
Trenton Holmes
d856e48045 Updates the pre-processing of document content to be much more robust, with tokenization, stemming and stop word removal 2022-10-10 08:58:23 -07:00
Trenton Holmes
b70e21a6d5 When raising an exception during exception handling, chain them together for slightly cleaner logs 2022-08-03 09:00:56 -07:00
Trenton Holmes
55dadea98e No need for a branch here, the loop takes care of it 2022-07-05 08:20:35 +02:00
Trenton Holmes
77fbbe95ff Updates the classifier to catch warnings from scikit-learn and rebuild the model file when this happens 2022-07-05 08:20:35 +02:00
Markus
69ef26dab0
Feature: Dynamic document storage pathes (#916) 
* Added devcontainer

* Add feature storage pathes

* Exclude tests and add versioning

* Check escaping

* Check escaping

* Check quoting

* Echo

* Escape

* Escape :

* Double escape \

* Escaping

* Remove if

* Escape colon

* Missing \

* Esacpe :

* Escape all

* test

* Remove sed

* Fix exclude

* Remove SED command

* Add LD_LIBRARY_PATH

* Adjusted to v1.7

* Updated test-cases

* Remove devcontainer

* Removed internal build-file

* Run pre-commit

* Corrected flak8 error

* Adjusted to v1.7

* Updated test-cases

* Corrected flak8 error

* Adjusted to new plural translations

* Small adjustments due to code-review backend

* Adjusted line-break

* Removed PAPERLESS prefix from settings variables

* Corrected style change due to search+replace

* First documentation draft

* Revert changes to Pipfile

* Add sphinx-autobuild with keep-outdated

* Revert merge error that results in wrong storage path is evaluated

* Adjust styles of generated files ...

* Adds additional testing to cover dynamic storage path functionality

* Remove unnecessary condition

* Add hint to edit storage path dialog

* Correct spelling of pathes to paths

* Minor documentation tweaks

* Minor typo

* improving wrapping of filter editor buttons with new storage path button

* Update .gitignore

* Fix select border radius in non input-groups

* Better storage path edit hint

* Add note to edit storage path dialog re document_renamer

* Add note to bulk edit storage path re document_renamer

* Rename FILTER_STORAGE_DIRECTORY to PATH

* Fix broken filter rule parsing

* Show default storage if unspecified

* Remove note re storage path on bulk edit

* Add basic validation of filename variables

Co-authored-by: Markus Kling <markus@markus-kling.net>
Co-authored-by: Trenton Holmes <holmes.trenton@gmail.com>
Co-authored-by: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Co-authored-by: Quinn Casey <quinn@quinncasey.com>
 2022-05-19 14:42:25 -07:00
Trenton Holmes
3003bdd507 Runs pyupgrade to Python 3.8+ and adds a hook for it 2022-05-06 09:04:08 -07:00
Trenton Holmes
9bb5568d8e Un-pickle and re-pickle the test models to resolve the version difference warning 2022-03-22 09:37:17 +01:00
Johann Bauer
cffdaefe2f Fix model test 2022-03-21 18:53:53 +01:00
Johann Bauer
9de4ca61e8 Increase FORMAT_VERSION to force model re-creation 2022-03-21 18:11:18 +01:00
Trenton Holmes
1771d18a21 Runs the pre-commit hooks over all the Python files 2022-03-11 11:34:28 -08:00
kpj
fc695896dd Format Python code with black 2022-02-27 15:26:41 +01:00
jonaswinkler
a3dae02cfb write classifier model to temporary file before copying to final location 2021-06-13 12:03:20 +02:00
jonaswinkler
635c96accf better exception handling 2021-05-19 23:11:24 +02:00
jonaswinkler
ca1e838c52 catch another exception regarding classifier loading 2021-05-19 22:57:52 +02:00
Jonas Winkler
61b47e358f correct file mode 2021-05-16 01:22:51 +02:00
jonaswinkler
12235cc853 fixes #689 2021-03-03 23:35:26 +01:00
jonaswinkler
7e88085377 load sklearn modules only when training data has changed 2021-02-15 11:25:25 +01:00
jonaswinkler
b48e67d714 revert a faulty change that caused memory usage to explode #537 2021-02-13 19:51:04 +01:00
jonaswinkler
ed0b1fe115 better exception logging 2021-02-11 22:16:41 +01:00
jonaswinkler
7702f5012b classifier cache timeout 2021-02-06 21:03:32 +01:00
jonaswinkler
ffe96c8fff classifier caching 2021-02-06 20:54:58 +01:00
jonaswinkler
431d4fd8e4 rework most of the logging 2021-02-05 01:10:29 +01:00
jonaswinkler
d8e0ef257e don't load sklearn libraries unless needed 2021-02-04 15:15:11 +01:00
jonaswinkler
4c6a02aee7 pycodestyle 2021-01-30 15:22:51 +01:00
jonaswinkler
87a18eae2d centralized classifier loading, better error handling, no error messages when auto matching is not used 2021-01-30 14:22:23 +01:00
jonaswinkler
bc4192e7d1 more tests and bugfixes. 2020-11-27 15:36:32 +01:00
Jonas Winkler
30acfdd3f1 tests for the classifier and fixes for edge cases with minimal data. 2020-11-26 14:18:34 +01:00
Jonas Winkler
450fb877f6 code cleanup 2020-11-21 15:34:00 +01:00
Jonas Winkler
2e04ba1c04 code style fixes 2020-11-12 21:09:45 +01:00
Jonas Winkler
fe6568e3b3 fixes #31 2020-11-12 10:04:01 +01:00
Jonas Winkler
296c113b16 updated the classifier. Its now much faster and does not retrain when data hasnt changed. 2020-11-06 14:46:06 +01:00
Jonas Winkler
68df1cf4ee replaced usages of .id with .pk, fixed filename issue in exporter 2020-11-03 12:37:37 +01:00
Jonas Winkler
05f20c19c3 the document classifier is now stateless 2020-10-29 14:33:42 +01:00
Jonas Winkler
11af74ba36 unified document matching, legacy and automatching work alongside now 2020-10-28 11:45:11 +01:00
Jonas Winkler
7257cece30 Code style changes 2018-09-26 10:51:42 +02:00
Jonas Winkler
60618381f8 Code style adjustments 2018-09-25 16:09:33 +02:00
Jonas Winkler
909586bf25 Code style changed 2018-09-13 14:15:16 +02:00
Jonas Winkler
46a5bc00d7 Merge branch 'machine-learning' into dev 2018-09-11 14:36:21 +02:00
Jonas Winkler
d46ee11143 The classifier works with ids now, not names. Minor changes. 2018-09-11 14:30:18 +02:00