309 Commits

Author SHA1 Message Date
shamoon
78d8f9a9ff Fix: correct serializing of auth tokens for export (#8100) 2024-10-29 17:02:32 +00:00
shamoon
5fa436c9f0 Fix: handle uuid fields created under mariadb and Django 4 (#8034) 2024-10-28 13:54:16 +00:00
shamoon
417c33230d Fix: disable custom field signals during import in 2.13.0 (#8065) 2024-10-27 18:43:24 -07:00
shamoon
24c05dfe78 Enhancement / fix: include social accounts and api tokens in export (#8016) 2024-10-26 06:51:22 -07:00
Trenton H
e3bc680bf1 Chore: Drop Python 3.9 support (#7774) 2024-09-26 12:22:24 -07:00
Trenton H
1cd45b322b Prefer the metadata JSON file over the version JSON file (#7048) 2024-06-20 12:49:54 -07:00
Trenton H
ada9ac7752 Feature: Allow encrypting sensitive fields in export (#6927)
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2024-06-09 14:41:18 +00:00
Trenton H
5bcd7c300f Feature: Allow a data only export/import cycle (#6871) 2024-06-01 18:22:59 -07:00
Trenton H
16584328f1 Chore: Change the code formatter to Ruff (#6756)
* Changing the formatting to ruff-format

* Replaces references to black to ruff or ruff format, removes black from dependencies
2024-05-18 02:26:50 +00:00
Trenton H
8b215af7e7 Moves additional auditlog imports into protected blocks (#6638) 2024-05-08 09:04:32 -07:00
shamoon
c875770731 Change: enable auditlog by default, fix import / export (#6267) 2024-04-04 18:51:15 +00:00
shamoon
3be37b7b37 Fix: document_renamer fails with audit_log enabled (#6175) 2024-03-24 07:26:25 -07:00
grembo
b425a69795 Fix: inotify read timeout not in ms (#5876)
* Fix inotify read timeout

This was off by a factor of 1000, leading to a lot more invocations
of the loop body than necessary.

* Incorporate review feedback
2024-02-26 00:46:47 +00:00
Trenton H
1d3f525f92 Ensure all creations of directories create the parents too (#5711) 2024-02-10 11:02:40 -08:00
Trenton H
9943f36be3 Chore: Adds additional rules for Ruff linter (#5660) 2024-02-05 21:46:59 +00:00
Trenton H
821e2d8569 Ensure the scratch directory exists before consuming this source (#5579) 2024-01-28 09:07:04 -08:00
luzpaz
97f4690236 Documentation: Fix typos with automated tool (#5319)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2024-01-08 16:58:41 +00:00
Trenton H
f95f8320f8 Fix: zip exports not respecting the --delete option (#5245) 2024-01-04 19:58:58 +00:00
shamoon
1141b767a0 Feature: Workflows (#5121) 2024-01-03 08:19:19 +00:00
Trenton H
c8a62715ec Feature: Allow setting backend configuration settings via the UI (#5126)
* Saving some start on this

* At least partially working for the tesseract parser

* Problems with migration testing need to figure out

* Work around that error

* Fixes max m_pixels

* Moving the settings to main paperless application

* Starting some consumer options

* More fixes and work

* Fixes these last tests

* Fix max_length on OcrSettings.mode field

* Fix all fields on Common & Ocr settings serializers

* Umbrellla config view

* Revert "Umbrellla config view"

This reverts commit fbaf9f4be30f89afeb509099180158a3406416a5.

* Updates to use a single configuration object for all settings

* Squashed commit of the following:

commit 8a0a49dd5766094f60462fbfbe62e9921fbd2373
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 23:02:47 2023 -0800

    Fix formatting

commit 66b2d90c507b8afd9507813ff555e46198ea33b9
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 22:36:35 2023 -0800

    Refactor frontend data models

commit 5723bd8dd823ee855625e250df39393e26709d48
Author: Adam Bogdał <adam@bogdal.pl>
Date:   Wed Dec 20 01:17:43 2023 +0100

    Fix: speed up admin panel for installs with a large number of documents (#5052)

commit 9b08ce176199bf9011a6634bb88f616846150d2b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:18:51 2023 -0800

    Update PULL_REQUEST_TEMPLATE.md

commit a6248bec2d793b7690feed95fcaf5eb34a75bfb6
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:02:05 2023 -0800

    Chore: Update Angular to v17 (#4980)

commit b1f6f52486d5ba5c04af99b41315eb6428fd1fa8
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:53:56 2023 -0800

    Fix: Dont allow null custom_fields property via API (#5063)

commit 638d9970fd468d8c02c91d19bd28f8b0796bdcb1
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:43:50 2023 -0800

    Enhancement: symmetric document links (#4907)

commit 5e8de4c1da6eb4eb8f738b20962595c7536b30ec
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 12:45:04 2023 -0800

    Enhancement: shared icon & shared by me filter (#4859)

commit 088bad90306025d3f6b139cbd0ad264a1cbecfe5
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Dec 19 12:04:03 2023 -0800

    Bulk updates all the backend libraries (#5061)

* Saving some work on frontend config

* Very basic but dynamically-generated config form

* Saving work on slightly less ugly frontend config

* JSON validation for user_args field

* Fully dynamic config form

* Adds in some additional validators for a nicer error message

* Cleaning up the testing and coverage more

* Reverts unintentional change

* Adds documentation about the settings and the precedence

* Couple more commenting and style fixes

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2023-12-29 15:42:56 -08:00
Trenton H
b4e3334646 Feature: Allow deletion of documents via the fuzzy matching command (#4957)
* Adds new flag allowing deletion of one of a document pair which is over the match ratio

* Documents the new command option
2023-12-17 18:37:38 -08:00
Trenton H
fff8674b56 Fixes export of custom field instances during a split manifest export (#4984) 2023-12-14 19:23:39 -08:00
shamoon
3c71a6cdb6 Fix: export consumption templates & custom fields in exporter (#4825) 2023-12-04 21:33:15 -08:00
Trenton H
66481297a5 Adds additional warnings during an import if it might fail due to reasons (#4814) 2023-12-05 03:39:59 +00:00
Josiah Outram Halstead
804a9d5dde Fix: Add missing spaces to document retagger help string (#4674) 2023-11-24 07:25:38 -08:00
frozenbrain
27edc4e1c5 Fix: Delay consumption after MODIFY inotify events (#4626) 2023-11-19 14:18:43 -08:00
Trenton H
49d80406cb Only delete the ContentTypes and Permissions once, not the loop (#4623) 2023-11-17 12:24:49 -08:00
Trenton H
37ae30ccda Forces JSON files to be written as UTF-8, and disables the ensure ASCII option which escapes non-ASCII chars (#4574) 2023-11-13 14:18:21 -08:00
Trenton H
efe779b634 Chore: Cleanup command arguments and standardize process count handling (#4541)
Cleans up some command help text and adds more control over process count for command with a Pool
2023-11-09 11:46:37 -08:00
Trenton H
7769b8968c Fixes the returning of a set, change to a list instead (#4442) 2023-10-25 16:18:13 -07:00
Trenton H
7291f11be7 Ensures all old connections are closed in certain longer running locations (#4265) 2023-09-25 12:52:22 -07:00
Trenton H
b162d7afc8 Handle when INotify fails to import but the polling is set to 0 still (#4230) 2023-09-20 14:04:42 -07:00
Trenton H
af9de2838b Allow the user the specifiy the zip file name (#4189) 2023-09-15 16:33:28 -07:00
Trenton H
9e333bf91d Cleans up the docs, adds validation of the process count, include the test descriptions 2023-09-12 08:17:12 -07:00
Trenton H
e82aaa7d85 Adds multiprocessing to the fuzzy matching for a speedup 2023-09-12 08:17:12 -07:00
shamoon
060966aa22 Add progress bar to document_fuzzy_match 2023-09-12 08:17:12 -07:00
Trenton Holmes
e4bf3c6ac4 Implements a new command for fuzzy matching document content and reporting potential duplicates 2023-09-12 08:17:12 -07:00
Trenton Holmes
34b80a4d8e Removes support for Python 3.8 and lower from the code base 2023-09-10 11:42:59 -07:00
Trenton H
ac8f5eaa7d Unlocks all things which were locked due to ARMv7, etc 2023-09-08 11:21:43 -07:00
Kamil Kosek
addc116f55 Enhancement: add --id-range for document_retagger (#4080)
---------

Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
2023-09-08 17:33:24 +00:00
Trenton H
e160580c8b Fixes issues with copy2 or copystat and SELinux see #3665 2023-07-22 06:27:49 -07:00
shamoon
6e9be0ba9b Include global and object-level permissions in export / import
adds test for transaction
2023-06-23 23:33:36 -07:00
shamoon
2b9d30cb68 Exclude consumer & AnonymousUser users from export manifest 2023-05-30 20:51:25 -07:00
Trenton Holmes
e34b8c4b1f Replace usages of os.rename with shutil.move to properly handle cases where the source and dest arent't on the same filesystem 2023-05-01 07:28:52 -07:00
Trenton H
aabcc9a1c4 Upgrades black to v23, upgrades ruff 2023-04-26 09:35:27 -07:00
Trenton H
30655f1b73 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Trenton H
98811df439 Make the importer a little more robust against some types of errors 2023-04-26 07:08:50 -07:00
Trenton H
d2c02b9102 Configures ruff as the one stop linter and resolves warnings it raised 2023-04-01 17:03:52 -07:00
Trenton H
36a6df0bae Creates a data model for the document consumption, allowing stronger typing of arguments and setting of some information about the file only once 2023-04-01 11:05:34 -07:00
Trenton H
8d0e65d369 Limit the number of threads waiting for files to be ready during polling 2023-03-21 07:46:57 -07:00