mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2025-08-18 00:46:25 +00:00
Compare commits
29 Commits
v2.18.0
...
feature-re
Author | SHA1 | Date | |
---|---|---|---|
![]() |
07381d48e6 | ||
![]() |
521fd1c957 | ||
![]() |
d878bc153a | ||
![]() |
f5e6951910 | ||
![]() |
91102d0335 | ||
![]() |
82ec1be622 | ||
![]() |
01a8cf6f36 | ||
![]() |
6bdb365f87 | ||
![]() |
696e591a3b | ||
![]() |
3c2782c3a9 | ||
![]() |
a68800d53c | ||
![]() |
dd0ffaf312 | ||
![]() |
264504affc | ||
![]() |
4feedf2add | ||
![]() |
2f76cf9831 | ||
![]() |
1002d37f6b | ||
![]() |
d260a94740 | ||
![]() |
88c69b83ea | ||
![]() |
2557ee2014 | ||
![]() |
3c75deed80 | ||
![]() |
d05343c927 | ||
![]() |
e7972b7eaf | ||
![]() |
75a091cc0d | ||
![]() |
dca74803fd | ||
![]() |
3cf3d868d0 | ||
![]() |
bf4fc6604a | ||
![]() |
e8c1eb86fa | ||
![]() |
c3dad3cf69 | ||
![]() |
811bd66088 |
@@ -37,6 +37,8 @@ Before you can run `pytest`, ensure to [properly set up your local environment](
|
||||
|
||||
Once you have submitted a **P**ull **R**equest it will be reviewed, approved, and merged by one or more community members of any team. Automated code tests and formatting checks must be passed.
|
||||
|
||||
Important: Pull requests that implement a new feature or enhancement _should almost always target an existing feature request_ with evidence of community interest and discussion. This is in order to balance the work of implementing and maintaining new features / enhancements. Instead of opening a PR which does not meet this requirement, please open a feature request instead, to gather feedback from both users and the project maintainers.
|
||||
|
||||
## Non-Trivial Requests
|
||||
|
||||
PRs deemed `non-trivial` will go through a stricter review process before being merged into `dev`. This is to ensure code quality and complete functionality (free of side effects).
|
||||
@@ -109,28 +111,12 @@ Paperless-ngx is a community project. We do our best to delegate permission and
|
||||
|
||||
## Structure
|
||||
|
||||
As of writing, there are 21 members in paperless-ngx. 4 of these people have complete administrative privileges to the repo:
|
||||
There are currently 2 members in paperless-ngx with complete administrative privileges to the repo:
|
||||
|
||||
- [@shamoon](https://github.com/shamoon)
|
||||
- [@bauerj](https://github.com/bauerj)
|
||||
- [@qcasey](https://github.com/qcasey)
|
||||
- [@FrankStrieter](https://github.com/FrankStrieter)
|
||||
- [@stumpylog](https://github.com/stumpylog)
|
||||
|
||||
There are 5 teams collaborating on specific tasks within paperless-ngx:
|
||||
|
||||
- @paperless-ngx/backend (Python / django)
|
||||
- @paperless-ngx/frontend (JavaScript / Typescript)
|
||||
- @paperless-ngx/ci-cd (GitHub Actions / Deployment)
|
||||
- @paperless-ngx/issues (Issue triage)
|
||||
- @paperless-ngx/test (General testing for larger PRs)
|
||||
|
||||
## Permissions
|
||||
|
||||
All team members are notified when mentioned or assigned to a relevant issue or pull request. Additionally, each team has slightly different access to paperless-ngx:
|
||||
|
||||
- The **test** team has no special permissions.
|
||||
- The **issues** team has `triage` access. This means they can organize issues and pull requests.
|
||||
- The **backend**, **frontend**, and **ci-cd** teams have `write` access. This means they can approve PRs and push code, containers, releases, and more.
|
||||
There are other members who occasionally contribute but we are actively seeking more dedicated maintainers of the project. Please reach out if you are interested.
|
||||
|
||||
## Joining
|
||||
|
||||
|
@@ -1,5 +1,139 @@
|
||||
# Changelog
|
||||
|
||||
## paperless-ngx 2.18.0
|
||||
|
||||
### Notable Changes
|
||||
|
||||
- Feature: PDF editor [@shamoon](https://github.com/shamoon) ([#10318](https://github.com/paperless-ngx/paperless-ngx/pull/10318))
|
||||
|
||||
### Features / Enhancements
|
||||
|
||||
- Feature: Add filter to localize dates for filepath templating [@stumpylog](https://github.com/stumpylog) ([#10559](https://github.com/paperless-ngx/paperless-ngx/pull/10559))
|
||||
- Feature: PDF editor [@shamoon](https://github.com/shamoon) ([#10318](https://github.com/paperless-ngx/paperless-ngx/pull/10318))
|
||||
- Enhancement: support webhook restrictions [@shamoon](https://github.com/shamoon) ([#10555](https://github.com/paperless-ngx/paperless-ngx/pull/10555))
|
||||
- Performance: Classifier performance optimizations [@Merinorus](https://github.com/Merinorus) ([#10363](https://github.com/paperless-ngx/paperless-ngx/pull/10363))
|
||||
- Performance: add setting to enable DB connection pooling for PostgreSQL [@Merinorus](https://github.com/Merinorus) ([#10354](https://github.com/paperless-ngx/paperless-ngx/pull/10354))
|
||||
- Fixhancement: improve text thumbnail generation for large files [@shamoon](https://github.com/shamoon) ([#10483](https://github.com/paperless-ngx/paperless-ngx/pull/10483))
|
||||
- Enhancement: disable auto spellcheck on filtering dropdowns [@TheDodger](https://github.com/TheDodger) ([#10487](https://github.com/paperless-ngx/paperless-ngx/pull/10487))
|
||||
- Enhancement: display saved view counts [@shamoon](https://github.com/shamoon) ([#10246](https://github.com/paperless-ngx/paperless-ngx/pull/10246))
|
||||
- Fixhancement: add missing exact operator for boolean CF queries [@shamoon](https://github.com/shamoon) ([#10402](https://github.com/paperless-ngx/paperless-ngx/pull/10402))
|
||||
- Feature: add Vietnamese translation [@shamoon](https://github.com/shamoon) ([#10352](https://github.com/paperless-ngx/paperless-ngx/pull/10352))
|
||||
- Performance: Add support for configuring date parser languages [@Merinorus](https://github.com/Merinorus) ([#10181](https://github.com/paperless-ngx/paperless-ngx/pull/10181))
|
||||
- Enhancement: Add a database caching for improved performance [@Merinorus](https://github.com/Merinorus) ([#9784](https://github.com/paperless-ngx/paperless-ngx/pull/9784))
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
- Fix: include ignore for config logos in sanity checker [@shamoon](https://github.com/shamoon) ([#10473](https://github.com/paperless-ngx/paperless-ngx/pull/10473))
|
||||
- Fix: track and restore changed document fields from session storage [@shamoon](https://github.com/shamoon) ([#10468](https://github.com/paperless-ngx/paperless-ngx/pull/10468))
|
||||
- Fix: Make some natural keyword date searches timezone-aware [@shamoon](https://github.com/shamoon) ([#10416](https://github.com/paperless-ngx/paperless-ngx/pull/10416))
|
||||
- Fixhancement: follow redirects in curl health check [@V0idC0de](https://github.com/V0idC0de) ([#10415](https://github.com/paperless-ngx/paperless-ngx/pull/10415))
|
||||
- Fix: dont use translated verbose_name for getting object perms [@shamoon](https://github.com/shamoon) ([#10399](https://github.com/paperless-ngx/paperless-ngx/pull/10399))
|
||||
- Fix: fix date format for 'today' in DateComponent [@shamoon](https://github.com/shamoon) ([#10369](https://github.com/paperless-ngx/paperless-ngx/pull/10369))
|
||||
- Fix: default to empty permissions for group creation [@shamoon](https://github.com/shamoon) ([#10337](https://github.com/paperless-ngx/paperless-ngx/pull/10337))
|
||||
- Fix: correct api created coercion with timezone [@shamoon](https://github.com/shamoon) ([#10287](https://github.com/paperless-ngx/paperless-ngx/pull/10287))
|
||||
- Fix: reset search query for preview on reset filter [@shamoon](https://github.com/shamoon) ([#10279](https://github.com/paperless-ngx/paperless-ngx/pull/10279))
|
||||
- Chore: reject absurd max age values [@shamoon](https://github.com/shamoon) ([#10243](https://github.com/paperless-ngx/paperless-ngx/pull/10243))
|
||||
- Chore: add tasks task_id param to openapi spec [@shamoon](https://github.com/shamoon) ([#10469](https://github.com/paperless-ngx/paperless-ngx/pull/10469))
|
||||
- Chore: include advanced search query param in API spec [@shamoon](https://github.com/shamoon) ([#10449](https://github.com/paperless-ngx/paperless-ngx/pull/10449))
|
||||
|
||||
### Security
|
||||
|
||||
- Address XSS vulnerability GHSA-6p53-hqqw-8j62
|
||||
|
||||
### Maintenance
|
||||
|
||||
- docker(deps): Bump astral-sh/uv from 0.8.4-python3.12-bookworm-slim to 0.8.8-python3.12-bookworm-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#10564](https://github.com/paperless-ngx/paperless-ngx/pull/10564))
|
||||
- docker(deps): Bump astral-sh/uv from 0.7.9-python3.12-bookworm-slim to 0.7.19-python3.12-bookworm-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#10343](https://github.com/paperless-ngx/paperless-ngx/pull/10343))
|
||||
- Chore(deps): Bump the small-changes group across 1 directory with 7 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10347](https://github.com/paperless-ngx/paperless-ngx/pull/10347))
|
||||
- Chore(deps-dev): Bump @types/node from 22.15.29 to 24.0.10 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10306](https://github.com/paperless-ngx/paperless-ngx/pull/10306))
|
||||
- Chore(deps): Bump the small-changes group across 1 directory with 8 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10481](https://github.com/paperless-ngx/paperless-ngx/pull/10481))
|
||||
- docker(deps): bump astral-sh/uv from 0.7.19-python3.12-bookworm-slim to 0.8.3-python3.12-bookworm-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#10465](https://github.com/paperless-ngx/paperless-ngx/pull/10465))
|
||||
- Chore: switch from os.path to pathlib.Path [@gothicVI](https://github.com/gothicVI) ([#10397](https://github.com/paperless-ngx/paperless-ngx/pull/10397))
|
||||
- Chore(deps): Bump the small-changes group with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10528](https://github.com/paperless-ngx/paperless-ngx/pull/10528))
|
||||
- Chore(deps): Bump the django group across 1 directory with 9 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10538](https://github.com/paperless-ngx/paperless-ngx/pull/10538))
|
||||
- Chore(deps): Bump stefanzweifel/git-auto-commit-action from 5 to 6 in the actions group @[dependabot[bot]](https://github.com/apps/dependabot) ([#10302](https://github.com/paperless-ngx/paperless-ngx/pull/10302))
|
||||
|
||||
### Dependencies
|
||||
|
||||
<details>
|
||||
<summary>23 changes</summary>
|
||||
|
||||
- chore: Small targeted upgrades to dependencies [@stumpylog](https://github.com/stumpylog) ([#10561](https://github.com/paperless-ngx/paperless-ngx/pull/10561))
|
||||
- docker(deps): Bump astral-sh/uv from 0.8.4-python3.12-bookworm-slim to 0.8.8-python3.12-bookworm-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#10564](https://github.com/paperless-ngx/paperless-ngx/pull/10564))
|
||||
- Chore(deps): Bump the django group across 1 directory with 9 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10538](https://github.com/paperless-ngx/paperless-ngx/pull/10538))
|
||||
- Chore(deps): Bump the small-changes group with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10528](https://github.com/paperless-ngx/paperless-ngx/pull/10528))
|
||||
- Chore(deps-dev): Bump the frontend-jest-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10497](https://github.com/paperless-ngx/paperless-ngx/pull/10497))
|
||||
- Chore(deps-dev): Bump the frontend-eslint-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10498](https://github.com/paperless-ngx/paperless-ngx/pull/10498))
|
||||
- Chore(deps-dev): Bump @playwright/test from 1.53.2 to 1.54.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10499](https://github.com/paperless-ngx/paperless-ngx/pull/10499))
|
||||
- Chore(deps-dev): Bump webpack from 5.99.9 to 5.101.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10501](https://github.com/paperless-ngx/paperless-ngx/pull/10501))
|
||||
- Chore(deps-dev): Bump prettier-plugin-organize-imports from 4.1.0 to 4.2.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10500](https://github.com/paperless-ngx/paperless-ngx/pull/10500))
|
||||
- Chore(deps-dev): Bump @types/node from 24.0.10 to 24.1.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10502](https://github.com/paperless-ngx/paperless-ngx/pull/10502))
|
||||
- Chore(deps): Bump the frontend-angular-dependencies group in /src-ui with 16 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10496](https://github.com/paperless-ngx/paperless-ngx/pull/10496))
|
||||
- Chore(deps): Bump the small-changes group across 1 directory with 8 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10481](https://github.com/paperless-ngx/paperless-ngx/pull/10481))
|
||||
- docker(deps): bump astral-sh/uv from 0.7.19-python3.12-bookworm-slim to 0.8.3-python3.12-bookworm-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#10465](https://github.com/paperless-ngx/paperless-ngx/pull/10465))
|
||||
- docker(deps): Bump astral-sh/uv from 0.7.9-python3.12-bookworm-slim to 0.7.19-python3.12-bookworm-slim @[dependabot[bot]](https://github.com/apps/dependabot) ([#10343](https://github.com/paperless-ngx/paperless-ngx/pull/10343))
|
||||
- Chore(deps): Bump the small-changes group across 1 directory with 7 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10347](https://github.com/paperless-ngx/paperless-ngx/pull/10347))
|
||||
- Chore(deps): Bump stefanzweifel/git-auto-commit-action from 5 to 6 in the actions group @[dependabot[bot]](https://github.com/apps/dependabot) ([#10302](https://github.com/paperless-ngx/paperless-ngx/pull/10302))
|
||||
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10311](https://github.com/paperless-ngx/paperless-ngx/pull/10311))
|
||||
- Chore(deps-dev): Bump @types/node from 22.15.29 to 24.0.10 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10306](https://github.com/paperless-ngx/paperless-ngx/pull/10306))
|
||||
- Chore(deps): Bump bootstrap from 5.3.6 to 5.3.7 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10308](https://github.com/paperless-ngx/paperless-ngx/pull/10308))
|
||||
- Chore(deps-dev): Bump webpack from 5.98.0 to 5.99.9 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10309](https://github.com/paperless-ngx/paperless-ngx/pull/10309))
|
||||
- Chore(deps-dev): Bump @playwright/test from 1.51.1 to 1.53.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10307](https://github.com/paperless-ngx/paperless-ngx/pull/10307))
|
||||
- Chore(deps): Bump the frontend-angular-dependencies group in /src-ui with 13 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10303](https://github.com/paperless-ngx/paperless-ngx/pull/10303))
|
||||
- Chore: update to Angular 20 [@shamoon](https://github.com/shamoon) ([#10273](https://github.com/paperless-ngx/paperless-ngx/pull/10273))
|
||||
</details>
|
||||
|
||||
### All App Changes
|
||||
|
||||
<details>
|
||||
<summary>44 changes</summary>
|
||||
|
||||
- chore: Small targeted upgrades to dependencies [@stumpylog](https://github.com/stumpylog) ([#10561](https://github.com/paperless-ngx/paperless-ngx/pull/10561))
|
||||
- Feature: Add filter to localize dates for filepath templating [@stumpylog](https://github.com/stumpylog) ([#10559](https://github.com/paperless-ngx/paperless-ngx/pull/10559))
|
||||
- Chore: Removes duplication and spread out config for codespell [@stumpylog](https://github.com/stumpylog) ([#10560](https://github.com/paperless-ngx/paperless-ngx/pull/10560))
|
||||
- Chore(deps): Bump the django group across 1 directory with 9 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10538](https://github.com/paperless-ngx/paperless-ngx/pull/10538))
|
||||
- Feature: PDF editor [@shamoon](https://github.com/shamoon) ([#10318](https://github.com/paperless-ngx/paperless-ngx/pull/10318))
|
||||
- Enhancement: support webhook restrictions [@shamoon](https://github.com/shamoon) ([#10555](https://github.com/paperless-ngx/paperless-ngx/pull/10555))
|
||||
- Performance: Classifier performance optimizations [@Merinorus](https://github.com/Merinorus) ([#10363](https://github.com/paperless-ngx/paperless-ngx/pull/10363))
|
||||
- Chore: switch from os.path to pathlib.Path [@gothicVI](https://github.com/gothicVI) ([#10397](https://github.com/paperless-ngx/paperless-ngx/pull/10397))
|
||||
- Chore(deps): Bump the small-changes group with 3 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10528](https://github.com/paperless-ngx/paperless-ngx/pull/10528))
|
||||
- Performance: add setting to enable DB connection pooling for PostgreSQL [@Merinorus](https://github.com/Merinorus) ([#10354](https://github.com/paperless-ngx/paperless-ngx/pull/10354))
|
||||
- Chore(deps-dev): Bump the frontend-jest-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10497](https://github.com/paperless-ngx/paperless-ngx/pull/10497))
|
||||
- Chore(deps-dev): Bump the frontend-eslint-dependencies group in /src-ui with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10498](https://github.com/paperless-ngx/paperless-ngx/pull/10498))
|
||||
- Chore(deps-dev): Bump @playwright/test from 1.53.2 to 1.54.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10499](https://github.com/paperless-ngx/paperless-ngx/pull/10499))
|
||||
- Chore(deps-dev): Bump webpack from 5.99.9 to 5.101.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10501](https://github.com/paperless-ngx/paperless-ngx/pull/10501))
|
||||
- Chore(deps-dev): Bump prettier-plugin-organize-imports from 4.1.0 to 4.2.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10500](https://github.com/paperless-ngx/paperless-ngx/pull/10500))
|
||||
- Chore(deps-dev): Bump @types/node from 24.0.10 to 24.1.0 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10502](https://github.com/paperless-ngx/paperless-ngx/pull/10502))
|
||||
- Chore(deps): Bump the frontend-angular-dependencies group in /src-ui with 16 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10496](https://github.com/paperless-ngx/paperless-ngx/pull/10496))
|
||||
- Fixhancement: improve text thumbnail generation for large files [@shamoon](https://github.com/shamoon) ([#10483](https://github.com/paperless-ngx/paperless-ngx/pull/10483))
|
||||
- Enhancement: disable auto spellcheck on filtering dropdowns [@TheDodger](https://github.com/TheDodger) ([#10487](https://github.com/paperless-ngx/paperless-ngx/pull/10487))
|
||||
- Chore(deps): Bump the small-changes group across 1 directory with 8 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10481](https://github.com/paperless-ngx/paperless-ngx/pull/10481))
|
||||
- Fix: include ignore for config logos in sanity checker [@shamoon](https://github.com/shamoon) ([#10473](https://github.com/paperless-ngx/paperless-ngx/pull/10473))
|
||||
- Chore: add tasks task_id param to openapi spec [@shamoon](https://github.com/shamoon) ([#10469](https://github.com/paperless-ngx/paperless-ngx/pull/10469))
|
||||
- Fix: track and restore changed document fields from session storage [@shamoon](https://github.com/shamoon) ([#10468](https://github.com/paperless-ngx/paperless-ngx/pull/10468))
|
||||
- Chore: include advanced search query param in API spec [@shamoon](https://github.com/shamoon) ([#10449](https://github.com/paperless-ngx/paperless-ngx/pull/10449))
|
||||
- Enhancement: display saved view counts [@shamoon](https://github.com/shamoon) ([#10246](https://github.com/paperless-ngx/paperless-ngx/pull/10246))
|
||||
- Fix: Make some natural keyword date searches timezone-aware [@shamoon](https://github.com/shamoon) ([#10416](https://github.com/paperless-ngx/paperless-ngx/pull/10416))
|
||||
- Fixhancement: add missing exact operator for boolean CF queries [@shamoon](https://github.com/shamoon) ([#10402](https://github.com/paperless-ngx/paperless-ngx/pull/10402))
|
||||
- Fix: dont use translated verbose_name for getting object perms [@shamoon](https://github.com/shamoon) ([#10399](https://github.com/paperless-ngx/paperless-ngx/pull/10399))
|
||||
- Fix: fix date format for 'today' in DateComponent [@shamoon](https://github.com/shamoon) ([#10369](https://github.com/paperless-ngx/paperless-ngx/pull/10369))
|
||||
- Feature: add Vietnamese translation [@shamoon](https://github.com/shamoon) ([#10352](https://github.com/paperless-ngx/paperless-ngx/pull/10352))
|
||||
- Chore(deps): Bump the small-changes group across 1 directory with 7 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10347](https://github.com/paperless-ngx/paperless-ngx/pull/10347))
|
||||
- Fix: default to empty permissions for group creation [@shamoon](https://github.com/shamoon) ([#10337](https://github.com/paperless-ngx/paperless-ngx/pull/10337))
|
||||
- Chore(deps-dev): Bump the frontend-eslint-dependencies group across 1 directory with 4 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10311](https://github.com/paperless-ngx/paperless-ngx/pull/10311))
|
||||
- Chore(deps-dev): Bump @types/node from 22.15.29 to 24.0.10 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10306](https://github.com/paperless-ngx/paperless-ngx/pull/10306))
|
||||
- Chore(deps): Bump bootstrap from 5.3.6 to 5.3.7 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10308](https://github.com/paperless-ngx/paperless-ngx/pull/10308))
|
||||
- Chore(deps-dev): Bump webpack from 5.98.0 to 5.99.9 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10309](https://github.com/paperless-ngx/paperless-ngx/pull/10309))
|
||||
- Chore(deps-dev): Bump @playwright/test from 1.51.1 to 1.53.2 in /src-ui @[dependabot[bot]](https://github.com/apps/dependabot) ([#10307](https://github.com/paperless-ngx/paperless-ngx/pull/10307))
|
||||
- Chore(deps): Bump the frontend-angular-dependencies group in /src-ui with 13 updates @[dependabot[bot]](https://github.com/apps/dependabot) ([#10303](https://github.com/paperless-ngx/paperless-ngx/pull/10303))
|
||||
- Performance: Add support for configuring date parser languages [@Merinorus](https://github.com/Merinorus) ([#10181](https://github.com/paperless-ngx/paperless-ngx/pull/10181))
|
||||
- Enhancement: Add a database caching for improved performance [@Merinorus](https://github.com/Merinorus) ([#9784](https://github.com/paperless-ngx/paperless-ngx/pull/9784))
|
||||
- Fix: correct api created coercion with timezone [@shamoon](https://github.com/shamoon) ([#10287](https://github.com/paperless-ngx/paperless-ngx/pull/10287))
|
||||
- Fix: reset search query for preview on reset filter [@shamoon](https://github.com/shamoon) ([#10279](https://github.com/paperless-ngx/paperless-ngx/pull/10279))
|
||||
- Chore: update to Angular 20 [@shamoon](https://github.com/shamoon) ([#10273](https://github.com/paperless-ngx/paperless-ngx/pull/10273))
|
||||
- Chore: reject absurd max age values [@shamoon](https://github.com/shamoon) ([#10243](https://github.com/paperless-ngx/paperless-ngx/pull/10243))
|
||||
</details>
|
||||
|
||||
## paperless-ngx 2.17.1
|
||||
|
||||
### Bug Fixes
|
||||
|
@@ -1800,3 +1800,23 @@ password. All of these options come from their similarly-named [Django settings]
|
||||
#### [`PAPERLESS_EMAIL_USE_SSL=<bool>`](#PAPERLESS_EMAIL_USE_SSL) {#PAPERLESS_EMAIL_USE_SSL}
|
||||
|
||||
: Defaults to false.
|
||||
|
||||
## Remote OCR
|
||||
|
||||
#### [`PAPERLESS_REMOTE_OCR_ENGINE=<str>`](#PAPERLESS_REMOTE_OCR_ENGINE) {#PAPERLESS_REMOTE_OCR_ENGINE}
|
||||
|
||||
: The remote OCR engine to use. Currently only Azure AI is supported as "azureai".
|
||||
|
||||
Defaults to None, which disables remote OCR.
|
||||
|
||||
#### [`PAPERLESS_REMOTE_OCR_API_KEY=<str>`](#PAPERLESS_REMOTE_OCR_API_KEY) {#PAPERLESS_REMOTE_OCR_API_KEY}
|
||||
|
||||
: The API key to use for the remote OCR engine.
|
||||
|
||||
Defaults to None.
|
||||
|
||||
#### [`PAPERLESS_REMOTE_OCR_ENDPOINT=<str>`](#PAPERLESS_REMOTE_OCR_ENDPOINT) {#PAPERLESS_REMOTE_OCR_ENDPOINT}
|
||||
|
||||
: The endpoint to use for the remote OCR engine. This is required for Azure AI.
|
||||
|
||||
Defaults to None.
|
||||
|
@@ -25,9 +25,10 @@ physical documents into a searchable online archive so you can keep, well, _less
|
||||
## Features
|
||||
|
||||
- **Organize and index** your scanned documents with tags, correspondents, types, and more.
|
||||
- _Your_ data is stored locally on _your_ server and is never transmitted or shared in any way.
|
||||
- _Your_ data is stored locally on _your_ server and is never transmitted or shared in any way, unless you explicitly choose to do so.
|
||||
- Performs **OCR** on your documents, adding searchable and selectable text, even to documents scanned with only images.
|
||||
- Utilizes the open-source Tesseract engine to recognize more than 100 languages.
|
||||
- Utilizes the open-source Tesseract engine to recognize more than 100 languages.
|
||||
- _New!_ Supports remote OCR with Azure AI (opt-in).
|
||||
- Documents are saved as PDF/A format which is designed for long term storage, alongside the unaltered originals.
|
||||
- Uses machine-learning to automatically add tags, correspondents and document types to your documents.
|
||||
- Supports PDF documents, images, plain text files, Office documents (Word, Excel, PowerPoint, and LibreOffice equivalents)[^1] and more.
|
||||
|
@@ -850,6 +850,18 @@ how regularly you intend to scan documents and use paperless.
|
||||
performed the task associated with the document, move it to the
|
||||
inbox.
|
||||
|
||||
## Remote OCR
|
||||
|
||||
!!! important
|
||||
|
||||
This feature is disabled by default and will always remain strictly "opt-in".
|
||||
|
||||
Paperless-ngx supports performing OCR on documents using remote services. At the moment, this is limited to
|
||||
[Microsoft's Azure "Document Intelligence" service](https://azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence).
|
||||
This is of course a paid service (with a free tier) which requires an Azure account and subscription. Azure AI is not affiliated with
|
||||
Paperless-ngx in any way. When enabled, Paperless-ngx will automatically send appropriate documents to Azure for OCR processing, bypassing
|
||||
the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details.
|
||||
|
||||
## Architecture
|
||||
|
||||
Paperless-ngx consists of the following components:
|
||||
|
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "paperless-ngx"
|
||||
version = "2.18.0"
|
||||
description = "A community-supported supercharged version of paperless: scan, index and archive all your physical documents"
|
||||
description = "A community-supported supercharged document management system: scan, index and archive all your physical documents"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10"
|
||||
classifiers = [
|
||||
@@ -15,6 +15,7 @@ classifiers = [
|
||||
# This will allow testing to not install a webserver, mysql, etc
|
||||
|
||||
dependencies = [
|
||||
"azure-ai-documentintelligence>=1.0.2",
|
||||
"babel>=2.17",
|
||||
"bleach~=6.2.0",
|
||||
"celery[redis]~=5.5.1",
|
||||
@@ -239,6 +240,7 @@ testpaths = [
|
||||
"src/paperless_tesseract/tests/",
|
||||
"src/paperless_tika/tests",
|
||||
"src/paperless_text/tests/",
|
||||
"src/paperless_remote/tests/",
|
||||
]
|
||||
addopts = [
|
||||
"--pythonwarnings=all",
|
||||
|
@@ -5549,14 +5549,14 @@
|
||||
<source>Copy metadata</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/common/pdf-editor/pdf-editor.component.html</context>
|
||||
<context context-type="linenumber">93</context>
|
||||
<context context-type="linenumber">94</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="6684403463658676119" datatype="html">
|
||||
<source>Delete original</source>
|
||||
<context-group purpose="location">
|
||||
<context context-type="sourcefile">src/app/components/common/pdf-editor/pdf-editor.component.html</context>
|
||||
<context context-type="linenumber">97</context>
|
||||
<context context-type="linenumber">98</context>
|
||||
</context-group>
|
||||
</trans-unit>
|
||||
<trans-unit id="7940755769131903278" datatype="html">
|
||||
|
@@ -145,7 +145,7 @@ export class AppFrameComponent
|
||||
}
|
||||
|
||||
get versionString(): string {
|
||||
return `${environment.appTitle} v${this.settingsService.get(SETTINGS_KEYS.VERSION)}${environment.production ? '' : ` #${environment.tag}`}`
|
||||
return `${environment.appTitle} v${this.settingsService.get(SETTINGS_KEYS.VERSION)}${environment.tag === 'prod' ? '' : ` #${environment.tag}`}`
|
||||
}
|
||||
|
||||
get customAppTitle(): string {
|
||||
|
@@ -25,7 +25,7 @@
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div cdkDropList (cdkDropListDropped)="drop($event)" cdkDropListOrientation="mixed" class="d-flex flex-wrap row-cols-5">
|
||||
<div cdkDropList (cdkDropListDropped)="drop($event)" cdkDropListOrientation="mixed" class="d-flex flex-wrap row-cols-2 row-cols-md-5">
|
||||
@for (p of pages; track p.page; let i = $index) {
|
||||
<div class="page-item rounded p-2" cdkDrag (click)="toggleSelection(i)" [class.selected]="p.selected">
|
||||
<div class="btn-toolbar hover-actions z-10">
|
||||
@@ -73,8 +73,8 @@
|
||||
}
|
||||
</div>
|
||||
</div>
|
||||
<div class="modal-footer flex-column">
|
||||
<div class="d-flex w-100 justify-content-between align-items-center">
|
||||
<div class="modal-footer">
|
||||
<div class="d-flex flex-column flex-md-row w-100 gap-3 align-items-center">
|
||||
<div class="btn-group" role="group">
|
||||
<input type="radio" class="btn-check" [(ngModel)]="editMode" [value]="PdfEditorEditMode.Create" id="editModeCreate" name="editmode">
|
||||
<label for="editModeCreate" class="btn btn-outline-primary btn-sm">
|
||||
@@ -88,16 +88,20 @@
|
||||
</label>
|
||||
</div>
|
||||
@if (editMode === PdfEditorEditMode.Create) {
|
||||
<div class="form-check ms-3">
|
||||
<input class="form-check-input" type="checkbox" id="copyMeta" [(ngModel)]="includeMetadata">
|
||||
<label class="form-check-label" for="copyMeta" i18n>Copy metadata</label>
|
||||
</div>
|
||||
<div class="form-check ms-3">
|
||||
<input class="form-check-input" type="checkbox" id="deleteOriginal" [(ngModel)]="deleteOriginal">
|
||||
<label class="form-check-label" for="deleteOriginal" i18n>Delete original</label>
|
||||
<div class="form-group d-flex">
|
||||
<div class="form-check">
|
||||
<input class="form-check-input" type="checkbox" id="copyMeta" [(ngModel)]="includeMetadata">
|
||||
<label class="form-check-label" for="copyMeta" i18n>Copy metadata</label>
|
||||
</div>
|
||||
<div class="form-check ms-3">
|
||||
<input class="form-check-input" type="checkbox" id="deleteOriginal" [(ngModel)]="deleteOriginal">
|
||||
<label class="form-check-label" for="deleteOriginal" i18n>Delete original</label>
|
||||
</div>
|
||||
</div>
|
||||
}
|
||||
<button type="button" class="btn ms-auto me-2" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled">{{ cancelBtnCaption }}</button>
|
||||
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="pages.length === 0">{{ btnCaption }}</button>
|
||||
<div class="form-group ms-md-auto">
|
||||
<button type="button" class="btn me-2" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled">{{ cancelBtnCaption }}</button>
|
||||
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="pages.length === 0">{{ btnCaption }}</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
@@ -209,7 +209,7 @@ class ApplicationConfigurationSerializer(serializers.ModelSerializer):
|
||||
return super().update(instance, validated_data)
|
||||
|
||||
def validate_app_logo(self, file):
|
||||
if magic.from_buffer(file.read(2048), mime=True) == "image/svg+xml":
|
||||
if file and magic.from_buffer(file.read(2048), mime=True) == "image/svg+xml":
|
||||
reject_dangerous_svg(file)
|
||||
return file
|
||||
|
||||
|
@@ -324,6 +324,7 @@ INSTALLED_APPS = [
|
||||
"paperless_tesseract.apps.PaperlessTesseractConfig",
|
||||
"paperless_text.apps.PaperlessTextConfig",
|
||||
"paperless_mail.apps.PaperlessMailConfig",
|
||||
"paperless_remote.apps.PaperlessRemoteParserConfig",
|
||||
"django.contrib.admin",
|
||||
"rest_framework",
|
||||
"rest_framework.authtoken",
|
||||
@@ -1443,3 +1444,10 @@ WEBHOOKS_ALLOW_INTERNAL_REQUESTS = __get_boolean(
|
||||
"PAPERLESS_WEBHOOKS_ALLOW_INTERNAL_REQUESTS",
|
||||
"true",
|
||||
)
|
||||
|
||||
###############################################################################
|
||||
# Remote Parser #
|
||||
###############################################################################
|
||||
REMOTE_OCR_ENGINE = os.getenv("PAPERLESS_REMOTE_OCR_ENGINE")
|
||||
REMOTE_OCR_API_KEY = os.getenv("PAPERLESS_REMOTE_OCR_API_KEY")
|
||||
REMOTE_OCR_ENDPOINT = os.getenv("PAPERLESS_REMOTE_OCR_ENDPOINT")
|
||||
|
@@ -1,6 +1,7 @@
|
||||
import dataclasses
|
||||
import email.contentmanager
|
||||
import random
|
||||
import time
|
||||
import uuid
|
||||
from collections import namedtuple
|
||||
from contextlib import AbstractContextManager
|
||||
@@ -385,6 +386,20 @@ class MailMocker(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
|
||||
apply_mail_action([], rule.pk, message.uid, message.subject, message.date)
|
||||
|
||||
|
||||
def assert_eventually_equals(getter_fn, expected_value, timeout=1.0, interval=0.05):
|
||||
"""
|
||||
Repeatedly calls `getter_fn()` until the result equals `expected_value`,
|
||||
or times out after `timeout` seconds.
|
||||
"""
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
if getter_fn() == expected_value:
|
||||
return
|
||||
time.sleep(interval)
|
||||
actual = getter_fn()
|
||||
raise AssertionError(f"Expected {expected_value}, but got {actual}")
|
||||
|
||||
|
||||
@mock.patch("paperless_mail.mail.magic.from_buffer", fake_magic_from_buffer)
|
||||
class TestMail(
|
||||
DirectoriesMixin,
|
||||
@@ -818,7 +833,7 @@ class TestMail(
|
||||
self.mail_account_handler.handle_mail_account(account)
|
||||
self.mailMocker.apply_mail_actions()
|
||||
|
||||
self.assertEqual(len(self.mailMocker.bogus_mailbox.messages), 1)
|
||||
assert_eventually_equals(lambda: len(self.mailMocker.bogus_mailbox.messages), 1)
|
||||
|
||||
def test_handle_mail_account_delete_no_filters(self):
|
||||
account = MailAccount.objects.create(
|
||||
|
4
src/paperless_remote/__init__.py
Normal file
4
src/paperless_remote/__init__.py
Normal file
@@ -0,0 +1,4 @@
|
||||
# this is here so that django finds the checks.
|
||||
from paperless_remote.checks import check_remote_parser_configured
|
||||
|
||||
__all__ = ["check_remote_parser_configured"]
|
14
src/paperless_remote/apps.py
Normal file
14
src/paperless_remote/apps.py
Normal file
@@ -0,0 +1,14 @@
|
||||
from django.apps import AppConfig
|
||||
|
||||
from paperless_remote.signals import remote_consumer_declaration
|
||||
|
||||
|
||||
class PaperlessRemoteParserConfig(AppConfig):
|
||||
name = "paperless_remote"
|
||||
|
||||
def ready(self):
|
||||
from documents.signals import document_consumer_declaration
|
||||
|
||||
document_consumer_declaration.connect(remote_consumer_declaration)
|
||||
|
||||
AppConfig.ready(self)
|
15
src/paperless_remote/checks.py
Normal file
15
src/paperless_remote/checks.py
Normal file
@@ -0,0 +1,15 @@
|
||||
from django.conf import settings
|
||||
from django.core.checks import Error
|
||||
from django.core.checks import register
|
||||
|
||||
|
||||
@register()
|
||||
def check_remote_parser_configured(app_configs, **kwargs):
|
||||
if settings.REMOTE_OCR_ENGINE == "azureai" and not settings.REMOTE_OCR_ENDPOINT:
|
||||
return [
|
||||
Error(
|
||||
"Azure AI remote parser requires endpoint to be configured.",
|
||||
),
|
||||
]
|
||||
|
||||
return []
|
113
src/paperless_remote/parsers.py
Normal file
113
src/paperless_remote/parsers.py
Normal file
@@ -0,0 +1,113 @@
|
||||
from pathlib import Path
|
||||
|
||||
from django.conf import settings
|
||||
|
||||
from paperless_tesseract.parsers import RasterisedDocumentParser
|
||||
|
||||
|
||||
class RemoteEngineConfig:
|
||||
def __init__(
|
||||
self,
|
||||
engine: str,
|
||||
api_key: str | None = None,
|
||||
endpoint: str | None = None,
|
||||
):
|
||||
self.engine = engine
|
||||
self.api_key = api_key
|
||||
self.endpoint = endpoint
|
||||
|
||||
def engine_is_valid(self):
|
||||
valid = self.engine in ["azureai"] and self.api_key is not None
|
||||
if self.engine == "azureai":
|
||||
valid = valid and self.endpoint is not None
|
||||
return valid
|
||||
|
||||
|
||||
class RemoteDocumentParser(RasterisedDocumentParser):
|
||||
"""
|
||||
This parser uses a remote OCR engine to parse documents. Currently, it supports Azure AI Vision
|
||||
as this is the only service that provides a remote OCR API with text-embedded PDF output.
|
||||
"""
|
||||
|
||||
logging_name = "paperless.parsing.remote"
|
||||
|
||||
def get_settings(self) -> RemoteEngineConfig:
|
||||
"""
|
||||
Returns the configuration for the remote OCR engine, loaded from Django settings.
|
||||
"""
|
||||
return RemoteEngineConfig(
|
||||
engine=settings.REMOTE_OCR_ENGINE,
|
||||
api_key=settings.REMOTE_OCR_API_KEY,
|
||||
endpoint=settings.REMOTE_OCR_ENDPOINT,
|
||||
)
|
||||
|
||||
def supported_mime_types(self):
|
||||
if self.settings.engine_is_valid():
|
||||
return {
|
||||
"application/pdf": ".pdf",
|
||||
"image/png": ".png",
|
||||
"image/jpeg": ".jpg",
|
||||
"image/tiff": ".tiff",
|
||||
"image/bmp": ".bmp",
|
||||
"image/gif": ".gif",
|
||||
"image/webp": ".webp",
|
||||
}
|
||||
else:
|
||||
return {}
|
||||
|
||||
def azure_ai_vision_parse(
|
||||
self,
|
||||
file: Path,
|
||||
) -> str | None:
|
||||
"""
|
||||
Uses Azure AI Vision to parse the document and return the text content.
|
||||
It requests a searchable PDF output with embedded text.
|
||||
The PDF is saved to the archive_path attribute.
|
||||
Returns the text content extracted from the document.
|
||||
If the parsing fails, it returns None.
|
||||
"""
|
||||
from azure.ai.documentintelligence import DocumentIntelligenceClient
|
||||
from azure.ai.documentintelligence.models import AnalyzeDocumentRequest
|
||||
from azure.ai.documentintelligence.models import AnalyzeOutputOption
|
||||
from azure.ai.documentintelligence.models import DocumentContentFormat
|
||||
from azure.core.credentials import AzureKeyCredential
|
||||
|
||||
client = DocumentIntelligenceClient(
|
||||
endpoint=self.settings.endpoint,
|
||||
credential=AzureKeyCredential(self.settings.api_key),
|
||||
)
|
||||
|
||||
with file.open("rb") as f:
|
||||
analyze_request = AnalyzeDocumentRequest(bytes_source=f.read())
|
||||
poller = client.begin_analyze_document(
|
||||
model_id="prebuilt-read",
|
||||
body=analyze_request,
|
||||
output_content_format=DocumentContentFormat.TEXT,
|
||||
output=[AnalyzeOutputOption.PDF], # request searchable PDF output
|
||||
content_type="application/json",
|
||||
)
|
||||
|
||||
poller.wait()
|
||||
result_id = poller.details["operation_id"]
|
||||
result = poller.result()
|
||||
|
||||
# Download the PDF with embedded text
|
||||
self.archive_path = Path(self.tempdir) / "archive.pdf"
|
||||
with self.archive_path.open("wb") as f:
|
||||
for chunk in client.get_analyze_result_pdf(
|
||||
model_id="prebuilt-read",
|
||||
result_id=result_id,
|
||||
):
|
||||
f.write(chunk)
|
||||
|
||||
return result.content
|
||||
|
||||
def parse(self, document_path: Path, mime_type, file_name=None):
|
||||
if not self.settings.engine_is_valid():
|
||||
self.log.warning(
|
||||
"No valid remote parser engine is configured, content will be empty.",
|
||||
)
|
||||
self.text = ""
|
||||
return
|
||||
elif self.settings.engine == "azureai":
|
||||
self.text = self.azure_ai_vision_parse(document_path)
|
18
src/paperless_remote/signals.py
Normal file
18
src/paperless_remote/signals.py
Normal file
@@ -0,0 +1,18 @@
|
||||
def get_parser(*args, **kwargs):
|
||||
from paperless_remote.parsers import RemoteDocumentParser
|
||||
|
||||
return RemoteDocumentParser(*args, **kwargs)
|
||||
|
||||
|
||||
def get_supported_mime_types():
|
||||
from paperless_remote.parsers import RemoteDocumentParser
|
||||
|
||||
return RemoteDocumentParser(None).supported_mime_types()
|
||||
|
||||
|
||||
def remote_consumer_declaration(sender, **kwargs):
|
||||
return {
|
||||
"parser": get_parser,
|
||||
"weight": 5,
|
||||
"mime_types": get_supported_mime_types(),
|
||||
}
|
0
src/paperless_remote/tests/__init__.py
Normal file
0
src/paperless_remote/tests/__init__.py
Normal file
BIN
src/paperless_remote/tests/samples/simple-digital.pdf
Normal file
BIN
src/paperless_remote/tests/samples/simple-digital.pdf
Normal file
Binary file not shown.
29
src/paperless_remote/tests/test_checks.py
Normal file
29
src/paperless_remote/tests/test_checks.py
Normal file
@@ -0,0 +1,29 @@
|
||||
from django.test import TestCase
|
||||
from django.test import override_settings
|
||||
|
||||
from paperless_remote import check_remote_parser_configured
|
||||
|
||||
|
||||
class TestChecks(TestCase):
|
||||
@override_settings(REMOTE_OCR_ENGINE=None)
|
||||
def test_no_engine(self):
|
||||
msgs = check_remote_parser_configured(None)
|
||||
self.assertEqual(len(msgs), 0)
|
||||
|
||||
@override_settings(REMOTE_OCR_ENGINE="azureai")
|
||||
@override_settings(REMOTE_OCR_API_KEY="somekey")
|
||||
@override_settings(REMOTE_OCR_ENDPOINT=None)
|
||||
def test_azure_no_endpoint(self):
|
||||
msgs = check_remote_parser_configured(None)
|
||||
self.assertEqual(len(msgs), 1)
|
||||
self.assertTrue(
|
||||
msgs[0].msg.startswith(
|
||||
"Azure AI remote parser requires endpoint to be configured.",
|
||||
),
|
||||
)
|
||||
|
||||
@override_settings(REMOTE_OCR_ENGINE="something")
|
||||
@override_settings(REMOTE_OCR_API_KEY="somekey")
|
||||
def test_valid_configuration(self):
|
||||
msgs = check_remote_parser_configured(None)
|
||||
self.assertEqual(len(msgs), 0)
|
101
src/paperless_remote/tests/test_parser.py
Normal file
101
src/paperless_remote/tests/test_parser.py
Normal file
@@ -0,0 +1,101 @@
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
from django.test import TestCase
|
||||
from django.test import override_settings
|
||||
|
||||
from documents.tests.utils import DirectoriesMixin
|
||||
from documents.tests.utils import FileSystemAssertsMixin
|
||||
from paperless_remote.parsers import RemoteDocumentParser
|
||||
from paperless_remote.signals import get_parser
|
||||
|
||||
|
||||
class TestParser(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
|
||||
SAMPLE_FILES = Path(__file__).resolve().parent / "samples"
|
||||
|
||||
def assertContainsStrings(self, content, strings):
|
||||
# Asserts that all strings appear in content, in the given order.
|
||||
indices = []
|
||||
for s in strings:
|
||||
if s in content:
|
||||
indices.append(content.index(s))
|
||||
else:
|
||||
self.fail(f"'{s}' is not in '{content}'")
|
||||
self.assertListEqual(indices, sorted(indices))
|
||||
|
||||
@mock.patch("paperless_tesseract.parsers.run_subprocess")
|
||||
@mock.patch("azure.ai.documentintelligence.DocumentIntelligenceClient")
|
||||
def test_get_text_with_azure(self, mock_client_cls, mock_subprocess):
|
||||
# Arrange mock Azure client
|
||||
mock_client = mock.Mock()
|
||||
mock_client_cls.return_value = mock_client
|
||||
|
||||
# Simulate poller result and its `.details`
|
||||
mock_poller = mock.Mock()
|
||||
mock_poller.wait.return_value = None
|
||||
mock_poller.details = {"operation_id": "fake-op-id"}
|
||||
mock_client.begin_analyze_document.return_value = mock_poller
|
||||
mock_poller.result.return_value.content = "This is a test document."
|
||||
|
||||
# Return dummy PDF bytes
|
||||
mock_client.get_analyze_result_pdf.return_value = [
|
||||
b"%PDF-",
|
||||
b"1.7 ",
|
||||
b"FAKEPDF",
|
||||
]
|
||||
|
||||
# Simulate pdftotext by writing dummy text to sidecar file
|
||||
def fake_run(cmd, *args, **kwargs):
|
||||
with Path(cmd[-1]).open("w", encoding="utf-8") as f:
|
||||
f.write("This is a test document.")
|
||||
|
||||
mock_subprocess.side_effect = fake_run
|
||||
|
||||
with override_settings(
|
||||
REMOTE_OCR_ENGINE="azureai",
|
||||
REMOTE_OCR_API_KEY="somekey",
|
||||
REMOTE_OCR_ENDPOINT="https://endpoint.cognitiveservices.azure.com",
|
||||
):
|
||||
parser = get_parser(uuid.uuid4())
|
||||
parser.parse(
|
||||
self.SAMPLE_FILES / "simple-digital.pdf",
|
||||
"application/pdf",
|
||||
)
|
||||
|
||||
self.assertContainsStrings(
|
||||
parser.text.strip(),
|
||||
["This is a test document."],
|
||||
)
|
||||
|
||||
@override_settings(
|
||||
REMOTE_OCR_ENGINE="azureai",
|
||||
REMOTE_OCR_API_KEY="key",
|
||||
REMOTE_OCR_ENDPOINT="https://endpoint.cognitiveservices.azure.com",
|
||||
)
|
||||
def test_supported_mime_types_valid_config(self):
|
||||
parser = RemoteDocumentParser(uuid.uuid4())
|
||||
expected_types = {
|
||||
"application/pdf": ".pdf",
|
||||
"image/png": ".png",
|
||||
"image/jpeg": ".jpg",
|
||||
"image/tiff": ".tiff",
|
||||
"image/bmp": ".bmp",
|
||||
"image/gif": ".gif",
|
||||
"image/webp": ".webp",
|
||||
}
|
||||
self.assertEqual(parser.supported_mime_types(), expected_types)
|
||||
|
||||
def test_supported_mime_types_invalid_config(self):
|
||||
parser = get_parser(uuid.uuid4())
|
||||
self.assertEqual(parser.supported_mime_types(), {})
|
||||
|
||||
@override_settings(
|
||||
REMOTE_OCR_ENGINE=None,
|
||||
REMOTE_OCR_API_KEY=None,
|
||||
REMOTE_OCR_ENDPOINT=None,
|
||||
)
|
||||
def test_parse_with_invalid_config(self):
|
||||
parser = get_parser(uuid.uuid4())
|
||||
parser.parse(self.SAMPLE_FILES / "simple-digital.pdf", "application/pdf")
|
||||
self.assertEqual(parser.text, "")
|
39
uv.lock
generated
39
uv.lock
generated
@@ -95,6 +95,34 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/af/cc/55a32a2c98022d88812b5986d2a92c4ff3ee087e83b712ebc703bba452bf/Automat-24.8.1-py3-none-any.whl", hash = "sha256:bf029a7bc3da1e2c24da2343e7598affaa9f10bf0ab63ff808566ce90551e02a", size = 42585, upload-time = "2024-08-19T17:31:56.729Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "azure-ai-documentintelligence"
|
||||
version = "1.0.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "azure-core", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
{ name = "isodate", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
{ name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/44/7b/8115cd713e2caa5e44def85f2b7ebd02a74ae74d7113ba20bdd41fd6dd80/azure_ai_documentintelligence-1.0.2.tar.gz", hash = "sha256:4d75a2513f2839365ebabc0e0e1772f5601b3a8c9a71e75da12440da13b63484", size = 170940 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d9/75/c9ec040f23082f54ffb1977ff8f364c2d21c79a640a13d1c1809e7fd6b1a/azure_ai_documentintelligence-1.0.2-py3-none-any.whl", hash = "sha256:e1fb446abbdeccc9759d897898a0fe13141ed29f9ad11fc705f951925822ed59", size = 106005 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "azure-core"
|
||||
version = "1.33.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "requests", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
{ name = "six", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
{ name = "typing-extensions", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/75/aa/7c9db8edd626f1a7d99d09ef7926f6f4fb34d5f9fa00dc394afdfe8e2a80/azure_core-1.33.0.tar.gz", hash = "sha256:f367aa07b5e3005fec2c1e184b882b0b039910733907d001c20fb08ebb8c0eb9", size = 295633 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/07/b7/76b7e144aa53bd206bf1ce34fa75350472c3f69bf30e5c8c18bc9881035d/azure_core-1.33.0-py3-none-any.whl", hash = "sha256:9b5b6d0223a1d38c37500e6971118c1e0f13f54951e6893968b38910bc9cda8f", size = 207071 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "babel"
|
||||
version = "2.17.0"
|
||||
@@ -1402,6 +1430,15 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c7/fc/4e5a141c3f7c7bed550ac1f69e599e92b6be449dd4677ec09f325cad0955/inotifyrecursive-0.3.5-py3-none-any.whl", hash = "sha256:7e5f4a2e1dc2bef0efa3b5f6b339c41fb4599055a2b54909d020e9e932cc8d2f", size = 8009, upload-time = "2020-11-20T12:38:46.981Z" },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "isodate"
|
||||
version = "0.7.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/54/4d/e940025e2ce31a8ce1202635910747e5a87cc3a6a6bb2d00973375014749/isodate-0.7.2.tar.gz", hash = "sha256:4cd1aa0f43ca76f4a6c6c0292a85f40b35ec2e43e315b59f06e6d32171a953e6", size = 29705 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/15/aa/0aca39a37d3c7eb941ba736ede56d689e7be91cab5d9ca846bde3999eba6/isodate-0.7.2-py3-none-any.whl", hash = "sha256:28009937d8031054830160fce6d409ed342816b543597cece116d966c6d99e15", size = 22320 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jinja2"
|
||||
version = "3.1.6"
|
||||
@@ -2009,6 +2046,7 @@ name = "paperless-ngx"
|
||||
version = "2.18.0"
|
||||
source = { virtual = "." }
|
||||
dependencies = [
|
||||
{ name = "azure-ai-documentintelligence", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
{ name = "babel", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
{ name = "bleach", marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
{ name = "celery", extra = ["redis"], marker = "sys_platform == 'darwin' or sys_platform == 'linux'" },
|
||||
@@ -2143,6 +2181,7 @@ typing = [
|
||||
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "azure-ai-documentintelligence", specifier = ">=1.0.2" },
|
||||
{ name = "babel", specifier = ">=2.17.0" },
|
||||
{ name = "bleach", specifier = "~=6.2.0" },
|
||||
{ name = "celery", extras = ["redis"], specifier = "~=5.5.1" },
|
||||
|
Reference in New Issue
Block a user