Commit Graph

97 Commits

Author SHA1 Message Date
Sebastian Steinbeißer
d2064a2535 Chore: switch from os.path to pathlib.Path (#10539) 2025-09-03 08:12:41 -07:00
Trenton H
f4413e0a08 Fix: Always clean up INotify (#9359) 2025-03-11 10:01:00 -07:00
Sebastian Steinbeißer
e560fa3be0 Chore: Enable ruff FBT (#8645) 2025-02-07 09:12:03 -08:00
dependabot[bot]
00485138f9 Chore(deps-dev): Bump the development group with 4 updates (#8352)
Bumps the development group with 4 updates: [ruff](https://github.com/astral-sh/ruff), [pytest-httpx](https://github.com/Colin-b/pytest_httpx), [pytest-rerunfailures](https://github.com/pytest-dev/pytest-rerunfailures) and [mkdocs-material](https://github.com/squidfunk/mkdocs-material).


Updates `ruff` from 0.7.3 to 0.8.0
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.7.3...0.8.0)

Updates `pytest-httpx` from 0.33.0 to 0.34.0
- [Release notes](https://github.com/Colin-b/pytest_httpx/releases)
- [Changelog](https://github.com/Colin-b/pytest_httpx/blob/develop/CHANGELOG.md)
- [Commits](https://github.com/Colin-b/pytest_httpx/compare/v0.33.0...v0.34.0)

Updates `pytest-rerunfailures` from 14.0 to 15.0
- [Changelog](https://github.com/pytest-dev/pytest-rerunfailures/blob/master/CHANGES.rst)
- [Commits](https://github.com/pytest-dev/pytest-rerunfailures/compare/14.0...15.0)

Updates `mkdocs-material` from 9.5.44 to 9.5.46
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.5.44...9.5.46)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: development
- dependency-name: pytest-httpx
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: development
- dependency-name: pytest-rerunfailures
  dependency-type: direct:development
  update-type: version-update:semver-major
  dependency-group: development
- dependency-name: mkdocs-material
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-01 18:56:54 -08:00
Trenton H
e6f59472e4 Chore: Drop Python 3.9 support (#7774) 2024-09-26 12:22:24 -07:00
grembo
76064178f5 Fix: inotify read timeout not in ms (#5876)
* Fix inotify read timeout

This was off by a factor of 1000, leading to a lot more invocations
of the loop body than necessary.

* Incorporate review feedback
2024-02-26 00:46:47 +00:00
Trenton H
2aea220c6d Ensure the scratch directory exists before consuming this source (#5579) 2024-01-28 09:07:04 -08:00
luzpaz
58bf9c552b Documentation: Fix typos with automated tool (#5319)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2024-01-08 16:58:41 +00:00
Trenton H
061f33fb05 Feature: Allow setting backend configuration settings via the UI (#5126)
* Saving some start on this

* At least partially working for the tesseract parser

* Problems with migration testing need to figure out

* Work around that error

* Fixes max m_pixels

* Moving the settings to main paperless application

* Starting some consumer options

* More fixes and work

* Fixes these last tests

* Fix max_length on OcrSettings.mode field

* Fix all fields on Common & Ocr settings serializers

* Umbrellla config view

* Revert "Umbrellla config view"

This reverts commit fbaf9f4be30f89afeb509099180158a3406416a5.

* Updates to use a single configuration object for all settings

* Squashed commit of the following:

commit 8a0a49dd57
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 23:02:47 2023 -0800

    Fix formatting

commit 66b2d90c50
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 22:36:35 2023 -0800

    Refactor frontend data models

commit 5723bd8dd8
Author: Adam Bogdał <adam@bogdal.pl>
Date:   Wed Dec 20 01:17:43 2023 +0100

    Fix: speed up admin panel for installs with a large number of documents (#5052)

commit 9b08ce1761
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:18:51 2023 -0800

    Update PULL_REQUEST_TEMPLATE.md

commit a6248bec2d
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:02:05 2023 -0800

    Chore: Update Angular to v17 (#4980)

commit b1f6f52486
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:53:56 2023 -0800

    Fix: Dont allow null custom_fields property via API (#5063)

commit 638d9970fd
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:43:50 2023 -0800

    Enhancement: symmetric document links (#4907)

commit 5e8de4c1da
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 12:45:04 2023 -0800

    Enhancement: shared icon & shared by me filter (#4859)

commit 088bad9030
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Dec 19 12:04:03 2023 -0800

    Bulk updates all the backend libraries (#5061)

* Saving some work on frontend config

* Very basic but dynamically-generated config form

* Saving work on slightly less ugly frontend config

* JSON validation for user_args field

* Fully dynamic config form

* Adds in some additional validators for a nicer error message

* Cleaning up the testing and coverage more

* Reverts unintentional change

* Adds documentation about the settings and the precedence

* Couple more commenting and style fixes

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2023-12-29 15:42:56 -08:00
frozenbrain
a14796cf90 Fix: Delay consumption after MODIFY inotify events (#4626) 2023-11-19 14:18:43 -08:00
Trenton H
cb271deb8f Fixes the returning of a set, change to a list instead (#4442) 2023-10-25 16:18:13 -07:00
Trenton H
612e6341a3 Ensures all old connections are closed in certain longer running locations (#4265) 2023-09-25 12:52:22 -07:00
Trenton H
085e6da1f2 Handle when INotify fails to import but the polling is set to 0 still (#4230) 2023-09-20 14:04:42 -07:00
Trenton Holmes
650c816a7b Removes support for Python 3.8 and lower from the code base 2023-09-10 11:42:59 -07:00
Trenton H
3bcbd05252 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Trenton H
ce41ac9158 Configures ruff as the one stop linter and resolves warnings it raised 2023-04-01 17:03:52 -07:00
Trenton H
3c2bbf244d Creates a data model for the document consumption, allowing stronger typing of arguments and setting of some information about the file only once 2023-04-01 11:05:34 -07:00
Trenton H
4a5f21dd87 Limit the number of threads waiting for files to be ready during polling 2023-03-21 07:46:57 -07:00
Trenton H
c08f0054da Updates ignore path filtering so files in a folder in an ignored folder will be ignored correctly 2023-02-16 09:05:11 -08:00
Trenton H
b25f083687 Updates the exporter to use pathlib and add a few more tests for coverage 2023-01-14 06:33:12 -08:00
Michael Shamoon
91a2dedfec Celery JSON serializable object for override_tag_ids 2022-11-07 07:41:23 -08:00
Trenton Holmes
9247300230 Transitions the backend to celery and celery beat 2022-09-26 11:25:34 -07:00
Trenton Holmes
ad1f5ae081 Corrects the timeout's interaction with files waiting to be consumed 2022-08-14 17:47:59 -07:00
Trenton Holmes
86358d5561 Attempts to make production consumer more event driven while still allowing unit testing 2022-08-14 17:47:59 -07:00
Trenton Holmes
3aca576a0d Includes the actual OSError string in the log, instead of assuming it's a busy file 2022-07-11 11:57:02 -07:00
Trenton Holmes
f07cfd4f51 Adds configuration variable to the inotify debounce timing 2022-05-15 11:48:12 -07:00
Trenton Holmes
7cbb73be7a Minor improvements for quality of life 2022-05-09 12:05:29 -07:00
Trenton Holmes
2414dad656 Adds additional checking for both inotify and polling around document still being busy before consuming it 2022-04-10 12:21:34 -07:00
Trenton Holmes
1771d18a21 Runs the pre-commit hooks over all the Python files 2022-03-11 11:34:28 -08:00
kpj
fc695896dd Format Python code with black 2022-02-27 15:26:41 +01:00
Daniel Albers
3ebe6d5aef Make ignores configurable
Adds config file setting PAPERLESS_CONSUMER_IGNORE_PATTERNS.
2021-08-18 22:23:18 +02:00
jonaswinkler
f8afbae2cd ignore macOS specific files 2021-05-19 19:56:01 +02:00
jonaswinkler
bac4a63cc8 run the polling file change checks on individual threads to speed up queueing of new files 2021-02-21 12:43:55 +01:00
jonaswinkler
0453787d38 increased default delay when waiting for file changes with polling 2021-02-21 12:14:54 +01:00
jonaswinkler
ed0b1fe115 better exception logging 2021-02-11 22:16:41 +01:00
jonaswinkler
431d4fd8e4 rework most of the logging 2021-02-05 01:10:29 +01:00
jonaswinkler
bdc247ce49 code style 2021-02-02 23:58:25 +01:00
jonaswinkler
227f7b6946 revert a change 2021-01-21 22:29:47 +01:00
jonaswinkler
08046cb83f revert changes for #351 2021-01-20 11:56:09 +01:00
jonaswinkler
5355f2b027 fixes #351 2021-01-19 14:43:55 +01:00
jonaswinkler
81dee38e4a fixes #300 2021-01-09 01:54:51 +01:00
jonaswinkler
0a0d462938 tags from folders: case insensitive 2020-12-09 00:07:22 +01:00
jonaswinkler
74a99cf330 removed slugs entirely, since their only purpose was purely cosmetic anyway. 2020-12-09 00:04:37 +01:00
jonaswinkler
834352130c checking file types against parsers in the consumer. 2020-12-01 15:26:05 +01:00
jayme-github
fa9a5cc247 Create tags from sub directories
The names of sub directories in the consumer directory will be added as
tags for the document to be consumed.
To enable this, set:
PAPERLESS_CONSUMER_RECURSIVE=1
PAPERLESS_CONSUMER_SUBDIRS_AS_TAGS=1

Fixes #50
2020-11-30 14:22:35 +01:00
jonaswinkler
60ac1ddbb9 fix warnings about unclosed files. 2020-11-27 13:19:58 +01:00
jonaswinkler
20c1139632 inotify: cleanup descriptor when done 2020-11-27 13:12:34 +01:00
jonaswinkler
d04b54140c moved consumption dir check into the correct spot 2020-11-27 13:12:13 +01:00
jonaswinkler
d5ec762954 couple changes to the consumer. 2020-11-26 18:55:05 +01:00
jonaswinkler
75390693b9 Apparently there was a very good reason to use inotify. fixes #46 complete with test cases for inotify and polling. 2020-11-26 17:57:03 +01:00