291 Commits

Author SHA1 Message Date
shamoon
3b6ce16f1c
Feature: Workflows (#5121) 2024-01-03 08:19:19 +00:00
Trenton H
061f33fb05
Feature: Allow setting backend configuration settings via the UI (#5126)
* Saving some start on this

* At least partially working for the tesseract parser

* Problems with migration testing need to figure out

* Work around that error

* Fixes max m_pixels

* Moving the settings to main paperless application

* Starting some consumer options

* More fixes and work

* Fixes these last tests

* Fix max_length on OcrSettings.mode field

* Fix all fields on Common & Ocr settings serializers

* Umbrellla config view

* Revert "Umbrellla config view"

This reverts commit fbaf9f4be30f89afeb509099180158a3406416a5.

* Updates to use a single configuration object for all settings

* Squashed commit of the following:

commit 8a0a49dd5766094f60462fbfbe62e9921fbd2373
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 23:02:47 2023 -0800

    Fix formatting

commit 66b2d90c507b8afd9507813ff555e46198ea33b9
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 22:36:35 2023 -0800

    Refactor frontend data models

commit 5723bd8dd823ee855625e250df39393e26709d48
Author: Adam Bogdał <adam@bogdal.pl>
Date:   Wed Dec 20 01:17:43 2023 +0100

    Fix: speed up admin panel for installs with a large number of documents (#5052)

commit 9b08ce176199bf9011a6634bb88f616846150d2b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:18:51 2023 -0800

    Update PULL_REQUEST_TEMPLATE.md

commit a6248bec2d793b7690feed95fcaf5eb34a75bfb6
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:02:05 2023 -0800

    Chore: Update Angular to v17 (#4980)

commit b1f6f52486d5ba5c04af99b41315eb6428fd1fa8
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:53:56 2023 -0800

    Fix: Dont allow null custom_fields property via API (#5063)

commit 638d9970fd468d8c02c91d19bd28f8b0796bdcb1
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:43:50 2023 -0800

    Enhancement: symmetric document links (#4907)

commit 5e8de4c1da6eb4eb8f738b20962595c7536b30ec
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 12:45:04 2023 -0800

    Enhancement: shared icon & shared by me filter (#4859)

commit 088bad90306025d3f6b139cbd0ad264a1cbecfe5
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Dec 19 12:04:03 2023 -0800

    Bulk updates all the backend libraries (#5061)

* Saving some work on frontend config

* Very basic but dynamically-generated config form

* Saving work on slightly less ugly frontend config

* JSON validation for user_args field

* Fully dynamic config form

* Adds in some additional validators for a nicer error message

* Cleaning up the testing and coverage more

* Reverts unintentional change

* Adds documentation about the settings and the precedence

* Couple more commenting and style fixes

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2023-12-29 15:42:56 -08:00
Trenton H
7289c4ea56
Feature: Allow deletion of documents via the fuzzy matching command (#4957)
* Adds new flag allowing deletion of one of a document pair which is over the match ratio

* Documents the new command option
2023-12-17 18:37:38 -08:00
Trenton H
be2de4f15d
Fixes export of custom field instances during a split manifest export (#4984) 2023-12-14 19:23:39 -08:00
shamoon
90f90dc9b4
Fix: export consumption templates & custom fields in exporter (#4825) 2023-12-04 21:33:15 -08:00
Trenton H
ca355d5855
Adds additional warnings during an import if it might fail due to reasons (#4814) 2023-12-05 03:39:59 +00:00
Josiah Outram Halstead
a320bfa425
Fix: Add missing spaces to document retagger help string (#4674) 2023-11-24 07:25:38 -08:00
frozenbrain
a14796cf90
Fix: Delay consumption after MODIFY inotify events (#4626) 2023-11-19 14:18:43 -08:00
Trenton H
a94e5d2e47
Only delete the ContentTypes and Permissions once, not the loop (#4623) 2023-11-17 12:24:49 -08:00
Trenton H
caddcaf807
Forces JSON files to be written as UTF-8, and disables the ensure ASCII option which escapes non-ASCII chars (#4574) 2023-11-13 14:18:21 -08:00
Trenton H
e8527ba723
Chore: Cleanup command arguments and standardize process count handling (#4541)
Cleans up some command help text and adds more control over process count for command with a Pool
2023-11-09 11:46:37 -08:00
Trenton H
cb271deb8f
Fixes the returning of a set, change to a list instead (#4442) 2023-10-25 16:18:13 -07:00
Trenton H
612e6341a3
Ensures all old connections are closed in certain longer running locations (#4265) 2023-09-25 12:52:22 -07:00
Trenton H
085e6da1f2
Handle when INotify fails to import but the polling is set to 0 still (#4230) 2023-09-20 14:04:42 -07:00
Trenton H
ec9ebd3026
Allow the user the specifiy the zip file name (#4189) 2023-09-15 16:33:28 -07:00
Trenton H
81b9f2d4e0 Cleans up the docs, adds validation of the process count, include the test descriptions 2023-09-12 08:17:12 -07:00
Trenton H
a03a745295 Adds multiprocessing to the fuzzy matching for a speedup 2023-09-12 08:17:12 -07:00
shamoon
ce8bf90663 Add progress bar to document_fuzzy_match 2023-09-12 08:17:12 -07:00
Trenton Holmes
e2ae919a84 Implements a new command for fuzzy matching document content and reporting potential duplicates 2023-09-12 08:17:12 -07:00
Trenton Holmes
650c816a7b Removes support for Python 3.8 and lower from the code base 2023-09-10 11:42:59 -07:00
Trenton H
d532913d56 Unlocks all things which were locked due to ARMv7, etc 2023-09-08 11:21:43 -07:00
Kamil Kosek
b238ba054d
Enhancement: add --id-range for document_retagger (#4080)
---------

Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
2023-09-08 17:33:24 +00:00
Trenton H
9f5d47c320 Fixes issues with copy2 or copystat and SELinux see #3665 2023-07-22 06:27:49 -07:00
shamoon
bbd4659fbf Include global and object-level permissions in export / import
adds test for transaction
2023-06-23 23:33:36 -07:00
shamoon
243598ae50 Exclude consumer & AnonymousUser users from export manifest 2023-05-30 20:51:25 -07:00
Trenton Holmes
b7577038a0 Replace usages of os.rename with shutil.move to properly handle cases where the source and dest arent't on the same filesystem 2023-05-01 07:28:52 -07:00
Trenton H
6f163111ce Upgrades black to v23, upgrades ruff 2023-04-26 09:35:27 -07:00
Trenton H
3bcbd05252 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Trenton H
a2d4d16867 Make the importer a little more robust against some types of errors 2023-04-26 07:08:50 -07:00
Trenton H
ce41ac9158 Configures ruff as the one stop linter and resolves warnings it raised 2023-04-01 17:03:52 -07:00
Trenton H
3c2bbf244d Creates a data model for the document consumption, allowing stronger typing of arguments and setting of some information about the file only once 2023-04-01 11:05:34 -07:00
Trenton H
4a5f21dd87 Limit the number of threads waiting for files to be ready during polling 2023-03-21 07:46:57 -07:00
shamoon
bf8ae22f3f Rename comments --> notes 2023-03-18 13:59:17 -07:00
Trenton H
c422a081bf Be sure the scratch directory exists before using it as temporary directory 2023-03-01 07:13:31 -08:00
Trenton H
c08f0054da Updates ignore path filtering so files in a folder in an ignored folder will be ignored correctly 2023-02-16 09:05:11 -08:00
Trenton H
06e2500443 Moves the mktime call into the if block where it is used, preventing exceptions during rare cases 2023-02-02 07:25:32 -08:00
Matthieu Helleboid
02a40055f5 replace --use-filename-prefix with --use-folder-prefix 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
aeecc10e45 sort exporter option by alphabetical order 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
2b3edbaa46 use pathlib glob 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
270f8677a7 add document comments to dedicated manifest file when using 'split-manifest' 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
20763e7c26 Fix split_manifest default value 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
b33ba4c902 fix json serialization bug after migration after to Pathlib) 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
fae5e834b9 fix bug on administration exporter when using -d, --delete option 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
4cb4bd13ad add split-manifest option to administration exporter 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
896304ccaa add prefix option to administration exporter 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
9ae186e6f9 add no-archive and no-thumbnail options to administration exporter and importer 2023-01-24 11:06:49 -08:00
Trenton H
b25f083687 Updates the exporter to use pathlib and add a few more tests for coverage 2023-01-14 06:33:12 -08:00
Trenton Holmes
a6b7beaf6b Adds option to allow a user to export directory to a zipfile 2022-12-04 16:38:25 -08:00
Michael Shamoon
91a2dedfec Celery JSON serializable object for override_tag_ids 2022-11-07 07:41:23 -08:00
Trenton Holmes
9247300230 Transitions the backend to celery and celery beat 2022-09-26 11:25:34 -07:00