Compare commits

..

34 Commits

Author SHA1 Message Date
Trenton H
367f47b2fd Enables testing against 3.13 for fun 2025-10-01 08:50:33 -07:00
Trenton H
80fabb0b56 Fixes extra dependency on psycopg-pool which should only have been in the Postgres group 2025-10-01 08:33:43 -07:00
guleyupo
af1c235af5 Documentation: fix typo (#10974) 2025-10-01 06:11:45 +00:00
dependabot[bot]
92ee906701 docker-compose(deps): Bump library/postgres in /docker/compose (#10965)
Bumps library/postgres from 17 to 18.

---
updated-dependencies:
- dependency-name: library/postgres
  dependency-version: '18'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-30 20:25:02 +00:00
shamoon
d6710de486 Chore: refactor for clarity 2025-09-30 12:16:22 -07:00
GitHub Actions
f71b13b82a Auto translate strings 2025-09-30 16:51:15 +00:00
Antoine Mérino
3df43d828a Performance: Cache django-guardian permissions when counting documents (#10657)
Fixes N+1 queries in tag, correspondent, storage path, custom field,
and document type list views.
Reduces SQL queries from 160 to 9.
2025-09-30 09:48:44 -07:00
Trenton H
643e2b4a8e Chore(deps): Bulk upgrade backend dependencies (#10971)
* Stop this dependabot nonsense with a bulk upgrade and lock

* Downgrades fido2 as allauth MFA doesn't support it yet
2025-09-30 08:51:06 -07:00
dependabot[bot]
6fa896df39 Chore(deps): Bump the major-versions group with 2 updates (#10960)
Bumps the major-versions group with 2 updates: [pytest-rerunfailures](https://github.com/pytest-dev/pytest-rerunfailures) and [types-setuptools](https://github.com/typeshed-internal/stub_uploader).


Updates `pytest-rerunfailures` from 15.1 to 16.0.1
- [Changelog](https://github.com/pytest-dev/pytest-rerunfailures/blob/master/CHANGES.rst)
- [Commits](https://github.com/pytest-dev/pytest-rerunfailures/compare/15.1...16.0.1)

Updates `types-setuptools` from 75.8.2.20250301 to 80.9.0.20250822
- [Commits](https://github.com/typeshed-internal/stub_uploader/commits)

---
updated-dependencies:
- dependency-name: pytest-rerunfailures
  dependency-version: 16.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: major-versions
- dependency-name: types-setuptools
  dependency-version: 80.9.0.20250822
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: major-versions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-30 07:09:22 -07:00
dependabot[bot]
6aeb5a5503 Chore(deps): Bump types-colorama from 0.4.15.20240311 to 0.4.15.20250801 (#10961)
Bumps [types-colorama](https://github.com/typeshed-internal/stub_uploader) from 0.4.15.20240311 to 0.4.15.20250801.
- [Commits](https://github.com/typeshed-internal/stub_uploader/commits)

---
updated-dependencies:
- dependency-name: types-colorama
  dependency-version: 0.4.15.20250801
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-29 15:24:36 -07:00
dependabot[bot]
86dbeb3a27 Chore(deps): Bump django-guardian from 3.1.3 to 3.2.0 (#10909)
Bumps [django-guardian](https://github.com/django-guardian/django-guardian) from 3.1.3 to 3.2.0.
- [Release notes](https://github.com/django-guardian/django-guardian/releases)
- [Commits](https://github.com/django-guardian/django-guardian/compare/3.1.3...3.2.0)

---
updated-dependencies:
- dependency-name: django-guardian
  dependency-version: 3.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-29 11:54:56 -07:00
dependabot[bot]
e97217f267 Chore(deps): Bump django-soft-delete from 1.0.19 to 1.0.21 (#10908)
Bumps [django-soft-delete](https://github.com/san4ezy/django_softdelete) from 1.0.19 to 1.0.21.
- [Changelog](https://github.com/san4ezy/django_softdelete/blob/master/CHANGELOG.md)
- [Commits](https://github.com/san4ezy/django_softdelete/commits)

---
updated-dependencies:
- dependency-name: django-soft-delete
  dependency-version: 1.0.21
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-29 09:38:39 -07:00
dependabot[bot]
05d5d7e796 Chore(deps): Bump whitenoise from 6.10.0 to 6.11.0 (#10910)
Bumps [whitenoise](https://github.com/evansd/whitenoise) from 6.10.0 to 6.11.0.
- [Changelog](https://github.com/evansd/whitenoise/blob/main/docs/changelog.rst)
- [Commits](https://github.com/evansd/whitenoise/compare/6.10.0...6.11.0)

---
updated-dependencies:
- dependency-name: whitenoise
  dependency-version: 6.11.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-29 08:38:35 -07:00
GitHub Actions
e8957de4a7 Auto translate strings 2025-09-26 20:49:13 +00:00
shamoon
1717517e70 Tweakhancement: reorganize some list & bulk editing buttons (#10944) 2025-09-26 13:47:24 -07:00
dependabot[bot]
af544177d4 Chore(deps): Bump django-cors-headers from 4.8.0 to 4.9.0 (#10907)
Bumps [django-cors-headers](https://github.com/adamchainz/django-cors-headers) from 4.8.0 to 4.9.0.
- [Changelog](https://github.com/adamchainz/django-cors-headers/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/adamchainz/django-cors-headers/compare/4.8.0...4.9.0)

---
updated-dependencies:
- dependency-name: django-cors-headers
  dependency-version: 4.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-26 12:21:18 -07:00
dependabot[bot]
766af6a48a docker(deps): bump astral-sh/uv (#10906)
Bumps [astral-sh/uv](https://github.com/astral-sh/uv) from 0.8.17-python3.12-bookworm-slim to 0.8.19-python3.12-bookworm-slim.
- [Release notes](https://github.com/astral-sh/uv/releases)
- [Changelog](https://github.com/astral-sh/uv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/uv/compare/0.8.17...0.8.19)

---
updated-dependencies:
- dependency-name: astral-sh/uv
  dependency-version: 0.8.19-python3.12-bookworm-slim
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-26 18:42:28 +00:00
shamoon
e985051890 Chore: remove Codecov token from CI workflow (#10941) 2025-09-26 10:01:31 -07:00
shamoon
764ad059d1 Revert "Chore: Enable SonarQube scanning (#10904)" (#10934)
This reverts commit 8d1f23e9d6.
2025-09-25 00:45:36 -07:00
shamoon
5e47069934 Fix select option removal and pagination update (#10933) 2025-09-25 00:42:43 -07:00
DerRockWolf
4ff09c4cf4 Enhancement: support workflow path matching of barcode-split documents (#10723) 2025-09-24 21:03:03 +00:00
shamoon
53b393dab5 Chore: remove conditional from pre-commit job in CI (#10916) 2025-09-24 13:43:09 -07:00
shamoon
6119c215e7 Fix: skip fuzzy matching for empty document content (#10914) 2025-09-22 23:30:24 -07:00
Trenton H
8d1f23e9d6 Chore: Enable SonarQube scanning (#10904)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2025-09-22 19:53:32 +00:00
GitHub Actions
c8850fa752 Auto translate strings 2025-09-22 18:21:26 +00:00
shamoon
19a54b3b23 Feature: processed mail UI (#10866) 2025-09-22 18:17:42 +00:00
shamoon
1cdd8d9ba8 Clarify repo maintenance rules 2025-09-21 16:32:21 -07:00
shamoon
4449dbadb5 Merge branch 'main' into dev 2025-09-21 16:10:00 -07:00
shamoon
43b4f36026 Documentation: add note about logo file visibility and exif data 2025-09-21 16:07:29 -07:00
shamoon
0e35acaef5 Fix: add extra error handling to _consume for file checks (#10897) 2025-09-21 13:21:40 -07:00
shamoon
19ff339804 Fix: show children in tag list when filtering (#10899) 2025-09-21 10:09:05 -07:00
shamoon
6b868a5ecb Fix: restore str celery beat schedule filename (#10893) 2025-09-20 18:54:56 -07:00
shamoon
3e4aa87cc5 Fix formatting 2025-09-12 15:55:22 -07:00
shamoon
fc95d42b35 Documentation: add guidance for feature PRs in CONTRIBUTING.md 2025-09-12 15:51:49 -07:00
53 changed files with 3119 additions and 1620 deletions

View File

@@ -17,11 +17,52 @@ env:
DEFAULT_PYTHON_VERSION: "3.11"
NLTK_DATA: "/usr/share/nltk_data"
jobs:
detect-duplicate:
name: Detect Duplicate Run
runs-on: ubuntu-24.04
outputs:
should_run: ${{ steps.check.outputs.should_run }}
steps:
- name: Check if workflow should run
id: check
uses: actions/github-script@v7
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
if (context.eventName !== 'push') {
core.info('Not a push event; running workflow.');
core.setOutput('should_run', 'true');
return;
}
const ref = context.ref || '';
if (!ref.startsWith('refs/heads/')) {
core.info('Push is not to a branch; running workflow.');
core.setOutput('should_run', 'true');
return;
}
const branch = ref.substring('refs/heads/'.length);
const { owner, repo } = context.repo;
const prs = await github.paginate(github.rest.pulls.list, {
owner,
repo,
state: 'open',
head: `${owner}:${branch}`,
per_page: 100,
});
if (prs.length === 0) {
core.info(`No open PR found for ${branch}; running workflow.`);
core.setOutput('should_run', 'true');
} else {
core.info(`Found ${prs.length} open PR(s) for ${branch}; skipping duplicate push run.`);
core.setOutput('should_run', 'false');
}
pre-commit:
# We want to run on external PRs, but not on our own internal PRs as they'll be run
# by the push to the branch. Without this if check, checks are duplicated since
# internal PRs match both the push and pull_request events.
if: github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository
needs:
- detect-duplicate
if: needs.detect-duplicate.outputs.should_run == 'true'
name: Linting Checks
runs-on: ubuntu-24.04
steps:
@@ -86,7 +127,7 @@ jobs:
- pre-commit
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12']
python-version: ['3.10', '3.11', '3.12', '3.13']
fail-fast: false
steps:
- name: Checkout
@@ -142,13 +183,11 @@ jobs:
if: always()
uses: codecov/test-results-action@v1
with:
token: ${{ secrets.CODECOV_TOKEN }}
flags: backend-python-${{ matrix.python-version }}
files: junit.xml
- name: Upload backend coverage to Codecov
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
flags: backend-python-${{ matrix.python-version }}
files: coverage.xml
- name: Stop containers
@@ -224,13 +263,11 @@ jobs:
uses: codecov/test-results-action@v1
if: always()
with:
token: ${{ secrets.CODECOV_TOKEN }}
flags: frontend-node-${{ matrix.node-version }}
directory: src-ui/
- name: Upload frontend coverage to Codecov
uses: codecov/codecov-action@v5
with:
token: ${{ secrets.CODECOV_TOKEN }}
flags: frontend-node-${{ matrix.node-version }}
directory: src-ui/coverage/
tests-frontend-e2e:

View File

@@ -241,6 +241,7 @@ jobs:
) {
nodes {
id,
createdAt,
number,
updatedAt,
upvoteCount,

View File

@@ -49,7 +49,7 @@ repos:
- 'prettier-plugin-organize-imports@4.1.0'
# Python hooks
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.13.0
rev: v0.13.2
hooks:
- id: ruff-check
- id: ruff-format
@@ -59,7 +59,7 @@ repos:
- id: pyproject-fmt
# Dockerfile hooks
- repo: https://github.com/AleksaC/hadolint-py
rev: v2.12.1b3
rev: v2.14.0
hooks:
- id: hadolint
# Shell script hooks

View File

@@ -2,9 +2,11 @@
If you feel like contributing to the project, please do! Bug fixes and improvements are always welcome.
⚠️ Please note: Pull requests that implement a new feature or enhancement _should almost always target an existing feature request_ with evidence of community interest and discussion. This is in order to balance the work of implementing and maintaining new features / enhancements. Pull requests that are opened without meeting this requirement may not be merged.
If you want to implement something big:
- Please start a discussion about that in the issues! Maybe something similar is already in development and we can make it happen together.
- As above, please start with a discussion! Maybe something similar is already in development and we can make it happen together.
- When making additions to the project, consider if the majority of users will benefit from your change. If not, you're probably better of forking the project.
- Also consider if your change will get in the way of other users. A good change is a change that enhances the experience of some users who want that change and does not affect users who do not care about the change.
- Please see the [paperless-ngx merge process](#merging-prs) below.
@@ -133,7 +135,7 @@ community members. That said, in an effort to keep the repository organized and
- Issues, pull requests and discussions that are closed will be locked after 30 days of inactivity.
- Discussions with a marked answer will be automatically closed.
- Discussions in the 'General' or 'Support' categories will be closed after 180 days of inactivity.
- Feature requests that do not meet the following thresholds will be closed: 180 days of inactivity, < 5 "up-votes" after 180 days, < 20 "up-votes" after 1 year or < 80 "up-votes" at 2 years.
- Feature requests that do not meet the following thresholds will be closed: 180 days of inactivity with less than 80 "up-votes", < 5 "up-votes" after 180 days, < 20 "up-votes" after 1 year or < 40 "up-votes" at 2 years.
In all cases, threads can be re-opened by project maintainers and, of course, users can always create a new discussion for related concerns.
Finally, remember that all information remains searchable and 'closed' feature requests can still serve as inspiration for new features.

View File

@@ -32,7 +32,7 @@ RUN set -eux \
# Purpose: Installs s6-overlay and rootfs
# Comments:
# - Don't leave anything extra in here either
FROM ghcr.io/astral-sh/uv:0.8.17-python3.12-bookworm-slim AS s6-overlay-base
FROM ghcr.io/astral-sh/uv:0.8.22-python3.12-bookworm-slim AS s6-overlay-base
WORKDIR /usr/src/s6

View File

@@ -32,7 +32,7 @@ services:
volumes:
- redisdata:/data
db:
image: docker.io/library/postgres:17
image: docker.io/library/postgres:18
restart: unless-stopped
volumes:
- pgdata:/var/lib/postgresql/data

View File

@@ -35,7 +35,7 @@ services:
volumes:
- redisdata:/data
db:
image: docker.io/library/postgres:17
image: docker.io/library/postgres:18
restart: unless-stopped
volumes:
- pgdata:/var/lib/postgresql/data

View File

@@ -31,7 +31,7 @@ services:
volumes:
- redisdata:/data
db:
image: docker.io/library/postgres:17
image: docker.io/library/postgres:18
restart: unless-stopped
volumes:
- pgdata:/var/lib/postgresql/data

View File

@@ -1759,6 +1759,11 @@ started by the container.
: Path to an image file in the /media/logo directory, must include 'logo', e.g. `/logo/Atari_logo.svg`
!!! note
The logo file will be viewable by anyone with access to the Paperless instance login page,
so consider your choice of logo carefully and removing exif data from images before uploading.
#### [`PAPERLESS_ENABLE_UPDATE_CHECK=<bool>`](#PAPERLESS_ENABLE_UPDATE_CHECK) {#PAPERLESS_ENABLE_UPDATE_CHECK}
!!! note

View File

@@ -261,6 +261,10 @@ different means. These are as follows:
Paperless is set up to check your mails every 10 minutes. This can be
configured via [`PAPERLESS_EMAIL_TASK_CRON`](configuration.md#PAPERLESS_EMAIL_TASK_CRON)
#### Processed Mail
Paperless keeps track of emails it has processed in order to avoid processing the same mail multiple times. This uses the message `UID` provided by the mail server, which should be unique for each message. You can view and manage processed mails from the web UI under Mail > Processed Mails. If you need to re-process a message, you can delete the corresponding processed mail entry, which will allow Paperless-ngx to process the email again the next time the mail fetch task runs.
#### OAuth Email Setup
Paperless-ngx supports OAuth2 authentication for Gmail and Outlook email accounts. To set up an email account with OAuth2, you will need to create a 'developer' app with the respective provider and obtain the client ID and client secret and set the appropriate [configuration variables](configuration.md#email_oauth). You will also need to set either [`PAPERLESS_OAUTH_CALLBACK_BASE_URL`](configuration.md#PAPERLESS_OAUTH_CALLBACK_BASE_URL) or [`PAPERLESS_URL`](configuration.md#PAPERLESS_URL) to the correct value for the OAuth2 flow to work correctly.
@@ -633,7 +637,7 @@ When you first delete a document it is moved to the 'trash' until either it is e
You can set how long documents remain in the trash before being automatically deleted with [`PAPERLESS_EMPTY_TRASH_DELAY`](configuration.md#PAPERLESS_EMPTY_TRASH_DELAY), which defaults
to 30 days. Until the file is actually deleted (e.g. the trash is emptied), all files and database content remains intact and can be restored at any point up until that time.
Additionally you may configure a directory where deleted files are moved to when they the trash is emptied with [`PAPERLESS_EMPTY_TRASH_DIR`](configuration.md#PAPERLESS_EMPTY_TRASH_DIR).
Additionally you may configure a directory where deleted files are moved to when the trash is emptied with [`PAPERLESS_EMPTY_TRASH_DIR`](configuration.md#PAPERLESS_EMPTY_TRASH_DIR).
Note that the empty trash directory only stores the original file, the archive file and all database information is permanently removed once a document is fully deleted.
## Best practices {#basic-searching}

View File

@@ -30,10 +30,10 @@ dependencies = [
"django-cachalot~=2.8.0",
"django-celery-results~=2.6.0",
"django-compression-middleware~=0.5.0",
"django-cors-headers~=4.8.0",
"django-cors-headers~=4.9.0",
"django-extensions~=4.1",
"django-filter~=25.1",
"django-guardian~=3.1.2",
"django-guardian~=3.2.0",
"django-multiselectfield~=1.0.1",
"django-soft-delete~=1.0.18",
"django-treenode>=0.23.2",
@@ -54,7 +54,6 @@ dependencies = [
"ocrmypdf~=16.11.0",
"pathvalidate~=3.3.1",
"pdf2image~=1.17.0",
"psycopg-pool",
"python-dateutil~=2.9.0",
"python-dotenv~=1.1.0",
"python-gnupg~=0.5.4",

View File

@@ -174,7 +174,7 @@ test('bulk edit', async ({ page }) => {
await expect(page.locator('pngx-document-list')).toHaveText(
/Selected 61 of 61 documents/i
)
await page.getByRole('button', { name: 'Cancel' }).click()
await page.getByRole('button', { name: 'None' }).click()
await page.locator('pngx-document-card-small').nth(1).click()
await page.locator('pngx-document-card-small').nth(2).click()

File diff suppressed because it is too large Load Diff

View File

@@ -166,13 +166,10 @@
</div>
<div class="nav-group mt-3 mb-1">
<h6 class="sidebar-heading px-3 text-muted d-flex align-items-center">
<h6 class="sidebar-heading px-3 text-muted">
<span i18n>Manage</span>
<button class="btn btn-link p-2 py-0" (click)="manageCollapse.toggle()">
<i-bs width="0.9em" height="0.9em" [name]="isManageMenuCollapsed ? 'chevron-down' : 'chevron-up'"></i-bs>
</button>
</h6>
<ul class="nav flex-column mb-2" #manageCollapse="ngbCollapse" [(ngbCollapse)]="isManageMenuCollapsed">
<ul class="nav flex-column mb-2">
<li class="nav-item app-link"
*pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Correspondent }">
<a class="nav-link" routerLink="correspondents" routerLinkActive="active" (click)="closeMenu()"
@@ -246,14 +243,10 @@
</div>
<div class="nav-group mt-auto mb-1">
<h6 class="sidebar-heading px-3 pt-4 text-muted d-flex align-items-center">
<h6 class="sidebar-heading px-3 pt-4 text-muted">
<span i18n>Administration</span>
<button class="btn btn-link p-2 py-0" (click)="adminCollapse.toggle()">
<i-bs width="0.9em" height="0.9em" [name]="isAdminMenuCollapsed ? 'chevron-down' : 'chevron-up'"></i-bs>
</button>
</h6>
<div class="mb-2">
<ul class="nav flex-column" #adminCollapse="ngbCollapse" [(ngbCollapse)]="isAdminMenuCollapsed">
<ul class="nav flex-column mb-2">
<li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }"
tourAnchor="tour.settings">
<a class="nav-link" routerLink="settings" routerLinkActive="active" (click)="closeMenu()"
@@ -299,8 +292,6 @@
</a>
</li>
}
</ul>
<ul class="nav flex-column">
<li class="nav-item mt-2" tourAnchor="tour.outro">
<a class="px-3 py-2 text-muted small d-flex align-items-center flex-wrap text-decoration-none"
target="_blank" rel="noopener noreferrer" href="https://docs.paperless-ngx.com" ngbPopover="Documentation"
@@ -365,7 +356,6 @@
</ul>
</div>
</div>
</div>
</nav>
<main role="main" class="ms-sm-auto px-md-4"

View File

@@ -89,8 +89,6 @@ export class AppFrameComponent
appRemoteVersion: AppRemoteVersion
isMenuCollapsed: boolean = true
isManageMenuCollapsed: boolean = false
isAdminMenuCollapsed: boolean = false
slimSidebarAnimating: boolean = false

View File

@@ -177,10 +177,16 @@ export class CustomFieldEditDialogComponent
}
public removeSelectOption(index: number) {
this.selectOptions.removeAt(index)
this._allSelectOptions.splice(
index + (this.selectOptionsPage - 1) * SELECT_OPTION_PAGE_SIZE,
1
const globalIndex =
index + (this.selectOptionsPage - 1) * SELECT_OPTION_PAGE_SIZE
this._allSelectOptions.splice(globalIndex, 1)
const totalPages = Math.max(
1,
Math.ceil(this._allSelectOptions.length / SELECT_OPTION_PAGE_SIZE)
)
const targetPage = Math.min(this.selectOptionsPage, totalPages)
this.selectOptionsPage = targetPage
}
}

View File

@@ -1,21 +1,5 @@
<div class="d-flex flex-wrap gap-4">
<div class="d-flex align-items-center" role="group" aria-label="Select">
<button class="btn btn-sm btn-outline-secondary" (click)="list.selectNone()">
<i-bs name="slash-circle"></i-bs>&nbsp;<ng-container i18n>Cancel</ng-container>
</button>
</div>
<div class="d-flex align-items-center gap-2" role="group" aria-label="Select">
<label class="me-2" i18n>Select:</label>
<div class="btn-group">
<button class="btn btn-sm btn-outline-primary" (click)="list.selectPage()">
<i-bs name="file-earmark-check"></i-bs>&nbsp;<ng-container i18n>Page</ng-container>
</button>
<button class="btn btn-sm btn-outline-primary" (click)="list.selectAll()">
<i-bs name="check-all"></i-bs>&nbsp;<ng-container i18n>All</ng-container>
</button>
</div>
</div>
<div class="d-flex align-items-center gap-2" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.Document }">
<div class="d-flex flex-wrap align-items-center gap-2" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.Document }">
<label class="me-2" i18n>Edit:</label>
@if (permissionService.currentUserCan(PermissionAction.View, PermissionType.Tag)) {
<pngx-filterable-dropdown title="Tags" icon="tag-fill" i18n-title
@@ -89,14 +73,14 @@
(apply)="setCustomFields($event)">
</pngx-filterable-dropdown>
}
</div>
<div class="d-flex align-items-center gap-2 ms-auto">
<div class="btn-toolbar">
<div class="btn-group">
<button type="button" class="btn btn-sm btn-outline-primary me-2" (click)="setPermissions()" [disabled]="!userOwnsAll || !userCanEditAll">
<i-bs name="person-fill-lock"></i-bs><div class="d-none d-sm-inline">&nbsp;<ng-container i18n>Permissions</ng-container></div>
</button>
</div>
</div>
<div class="d-flex align-items-center gap-2 ms-auto">
<div class="btn-toolbar">
<div ngbDropdown>
<button class="btn btn-sm btn-outline-primary" id="dropdownSelect" [disabled]="!userCanEdit && !userCanAdd" ngbDropdownToggle>
<i-bs name="three-dots"></i-bs>
@@ -115,7 +99,6 @@
</div>
</div>
</div>
<div class="btn-group btn-group-sm">
<button class="btn btn-sm btn-outline-primary" [disabled]="awaitingDownload" (click)="downloadSelected()">
@if (!awaitingDownload) {
@@ -158,4 +141,4 @@
</button>
</div>
</div>
</div>
</div>

View File

@@ -5,3 +5,7 @@
.dropdown-menu{
--bs-dropdown-min-width: 12rem;
}
.btn-group .btn {
white-space: nowrap;
}

View File

@@ -1,16 +1,36 @@
<pngx-page-header [title]="getTitle()">
<div ngbDropdown class="btn-group flex-fill">
<button class="btn btn-sm btn-outline-primary" id="dropdownSelect" ngbDropdownToggle>
<div ngbDropdown class="btn-group flex-fill d-sm-none">
<button class="btn btn-sm btn-outline-primary" id="dropdownSelectMobile" ngbDropdownToggle>
<i-bs name="text-indent-left"></i-bs>
<div class="d-none d-sm-inline">&nbsp;<ng-container i18n>Select</ng-container></div>
@if (list.selected.size > 0) {
<pngx-clearable-badge [selected]="list.selected.size > 0" [number]="list.selected.size" (cleared)="list.selectNone()"></pngx-clearable-badge><span class="visually-hidden">selected</span>
}
</button>
<div ngbDropdownMenu aria-labelledby="dropdownSelect" class="shadow">
<div ngbDropdownMenu aria-labelledby="dropdownSelectMobile" class="shadow">
<button ngbDropdownItem (click)="list.selectNone()" i18n>Select none</button>
<button ngbDropdownItem (click)="list.selectPage()" i18n>Select page</button>
<button ngbDropdownItem (click)="list.selectAll()" i18n>Select all</button>
</div>
</div>
<div class="d-none d-sm-flex flex-fill me-3">
<div class="input-group input-group-sm">
<span class="input-group-text border-0">Select:</span>
</div>
<div class="btn-group btn-group-sm flex-nowrap">
@if (list.selected.size > 0) {
<button class="btn btn-sm btn-outline-secondary" (click)="list.selectNone()">
<i-bs name="slash-circle"></i-bs>&nbsp;<ng-container i18n>None</ng-container>
</button>
}
<button class="btn btn-sm btn-outline-primary" (click)="list.selectPage()">
<i-bs name="file-earmark-check"></i-bs>&nbsp;<ng-container i18n>Page</ng-container>
</button>
<button class="btn btn-sm btn-outline-primary" (click)="list.selectAll()">
<i-bs name="check-all"></i-bs>&nbsp;<ng-container i18n>All</ng-container>
</button>
</div>
</div>
<div ngbDropdown class="btn-group flex-fill">
<button class="btn btn-sm btn-outline-primary" id="dropdownDisplayFields" ngbDropdownToggle>
<i-bs name="card-heading"></i-bs>
@@ -128,6 +148,11 @@
<i-bs width="1em" height="1em" name="x"></i-bs><small i18n>Reset filters</small>
</button>
}
@if (!list.isReloading && list.selected.size > 0) {
<button class="btn btn-link py-0" (click)="list.selectNone()">
<i-bs width="1em" height="1em" name="slash-circle" class="me-1"></i-bs><small i18n>Clear selection</small>
</button>
}
</div>
@if (list.collectionSize) {
<ngb-pagination [pageSize]="list.pageSize" [collectionSize]="list.collectionSize" [(page)]="list.currentPage" [maxSize]="5"

View File

@@ -56,6 +56,7 @@ import {
filterRulesDiffer,
isFullTextFilterRule,
} from 'src/app/utils/filter-rules'
import { ClearableBadgeComponent } from '../common/clearable-badge/clearable-badge.component'
import { CustomFieldDisplayComponent } from '../common/custom-field-display/custom-field-display.component'
import { PageHeaderComponent } from '../common/page-header/page-header.component'
import { PreviewPopupComponent } from '../common/preview-popup/preview-popup.component'
@@ -72,6 +73,7 @@ import { SaveViewConfigDialogComponent } from './save-view-config-dialog/save-vi
templateUrl: './document-list.component.html',
styleUrls: ['./document-list.component.scss'],
imports: [
ClearableBadgeComponent,
CustomFieldDisplayComponent,
PageHeaderComponent,
BulkEditorComponent,

View File

@@ -109,10 +109,11 @@
<li class="list-group-item">
<div class="row">
<div class="col" i18n>Name</div>
<div class="col d-none d-sm-block" i18n>Sort Order</div>
<div class="col" i18n>Account</div>
<div class="col d-none d-sm-block" i18n>Status</div>
<div class="col" i18n>Actions</div>
<div class="col-1 d-none d-sm-block" i18n>Sort Order</div>
<div class="col-2" i18n>Account</div>
<div class="col-2 d-none d-sm-block" i18n>Status</div>
<div class="col d-none d-sm-block" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.ProcessedMail }">Processed Mail</div>
<div class="col-3" i18n>Actions</div>
</div>
</li>
@@ -127,9 +128,9 @@
<li class="list-group-item">
<div class="row fade" [class.show]="showRules">
<div class="col d-flex align-items-center"><button class="btn btn-link p-0 text-start" type="button" (click)="editMailRule(rule)" [disabled]="!permissionsService.currentUserCan(PermissionAction.Change, PermissionType.MailRule) || !userCanEdit(rule)">{{rule.name}}</button></div>
<div class="col d-flex align-items-center d-none d-sm-flex">{{rule.order}}</div>
<div class="col d-flex align-items-center">{{(mailAccountService.getCached(rule.account) | async)?.name}}</div>
<div class="col d-flex align-items-center d-none d-sm-flex">
<div class="col-1 d-flex align-items-center d-none d-sm-flex">{{rule.order}}</div>
<div class="col-2 d-flex align-items-center">{{(mailAccountService.getCached(rule.account) | async)?.name}}</div>
<div class="col-2 d-flex align-items-center d-none d-sm-flex">
<div class="form-check form-switch mb-0">
<input #inputField type="checkbox" class="form-check-input cursor-pointer" [id]="rule.id+'_enable'" [(ngModel)]="rule.enabled" (change)="onMailRuleEnableToggled(rule)" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.MailRule }">
<label class="form-check-label cursor-pointer" [for]="rule.id+'_enable'">
@@ -137,7 +138,12 @@
</label>
</div>
</div>
<div class="col">
<div class="col d-flex align-items-center d-none d-sm-flex" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.ProcessedMail }">
<button class="btn btn-sm btn-outline-secondary" type="button" (click)="viewProcessedMail(rule)">
<i-bs width="1em" height="1em" name="clock-history"></i-bs>&nbsp;<ng-container i18n>View Processed Mail</ng-container>
</button>
</div>
<div class="col-3">
<div class="btn-group d-block d-sm-none">
<div ngbDropdown container="body" class="d-inline-block">
<button type="button" class="btn btn-link" id="actionsMenuMobile" (click)="$event.stopPropagation()" ngbDropdownToggle>

View File

@@ -409,4 +409,13 @@ describe('MailComponent', () => {
jest.advanceTimersByTime(200)
expect(editSpy).toHaveBeenCalled()
})
it('should open processed mails dialog', () => {
completeSetup()
let modal: NgbModalRef
modalService.activeInstances.subscribe((refs) => (modal = refs[0]))
component.viewProcessedMail(mailRules[0] as MailRule)
const dialog = modal.componentInstance as any
expect(dialog.rule).toEqual(mailRules[0])
})
})

View File

@@ -27,6 +27,7 @@ import { MailRuleEditDialogComponent } from '../../common/edit-dialog/mail-rule-
import { PageHeaderComponent } from '../../common/page-header/page-header.component'
import { PermissionsDialogComponent } from '../../common/permissions-dialog/permissions-dialog.component'
import { ComponentWithPermissions } from '../../with-permissions/with-permissions.component'
import { ProcessedMailDialogComponent } from './processed-mail-dialog/processed-mail-dialog.component'
@Component({
selector: 'pngx-mail',
@@ -347,6 +348,14 @@ export class MailComponent
)
}
viewProcessedMail(rule: MailRule) {
const modal = this.modalService.open(ProcessedMailDialogComponent, {
backdrop: 'static',
size: 'xl',
})
modal.componentInstance.rule = rule
}
userCanEdit(obj: ObjectWithPermissions): boolean {
return this.permissionsService.currentUserHasObjectPermissions(
PermissionAction.Change,

View File

@@ -0,0 +1,107 @@
<div class="modal-header">
<h6 class="modal-title" id="modal-basic-title" i18n>Processed Mail for <em>{{ rule.name }}</em></h6>
<button class="btn btn-sm btn-link text-muted me-auto p-0 p-md-2" title="What's this?" i18n-title type="button" [ngbPopover]="infoPopover" [autoClose]="true">
<i-bs name="question-circle"></i-bs>
</button>
<ng-template #infoPopover>
<a href="https://docs.paperless-ngx.com/usage#processed-mail" target="_blank" referrerpolicy="noopener noreferrer" i18n>Read more</a>
<i-bs class="ms-1" width=".8em" height=".8em" name="box-arrow-up-right"></i-bs>
</ng-template>
<button type="button" class="btn-close" aria-label="Close" (click)="close()"></button>
</div>
<div class="modal-body">
@if (loading) {
<div class="text-center my-5">
<div class="spinner-border" role="status">
<span class="visually-hidden" i18n>Loading...</span>
</div>
</div>
} @else if (processedMails.length === 0) {
<span i18n>No processed email messages found.</span>
} @else {
<div class="table-responsive">
<table class="table table-hover table-sm align-middle">
<thead>
<tr>
<th scope="col" style="width: 40px;">
<div class="form-check m-0 ms-2 me-n2">
<input type="checkbox" class="form-check-input" id="all-objects" [(ngModel)]="toggleAllEnabled" [disabled]="processedMails.length === 0" (click)="toggleAll($event); $event.stopPropagation();">
<label class="form-check-label" for="all-objects"></label>
</div>
</th>
<th scope="col" i18n>Subject</th>
<th scope="col" i18n>Received</th>
<th scope="col" i18n>Processed</th>
<th scope="col" i18n>Status</th>
<th scope="col" i18n>Error</th>
</tr>
</thead>
<tbody>
@for (mail of processedMails; track mail.id) {
<ng-template #statusTooltip>
<div class="small text-light font-monospace">
{{mail.status}}
</div>
</ng-template>
<tr>
<td>
<div class="form-check m-0 ms-2 me-n2">
<input type="checkbox" class="form-check-input" [id]="mail.id" [checked]="selectedMailIds.has(mail.id)" (click)="toggleSelected(mail); $event.stopPropagation();">
<label class="form-check-label" [for]="mail.id"></label>
</div>
</td>
<td>{{ mail.subject }}</td>
<td>{{ mail.received | customDate:'longDate' }}</td>
<td>{{ mail.processed | customDate:'longDate' }}</td>
<td>
@switch (mail.status) {
@case ('SUCCESS') {
<i-bs name="check-circle" title="SUCCESS" class="text-success" [ngbTooltip]="statusTooltip"></i-bs>
}
@case ('FAILED') {
<i-bs name="exclamation-triangle" title="FAILED" class="text-danger" [ngbTooltip]="statusTooltip"></i-bs>
}
@default {
<i-bs name="slash-circle" title="{{ mail.status }}" class="text-muted" [ngbTooltip]="statusTooltip"></i-bs>
}
}
</td>
<td>
<ng-template #errorPopover>
<pre class="small text-light">
{{ mail.error }}
</pre>
</ng-template>
@if (mail.error) {
<span class="text-danger" triggers="mouseenter:mouseleave" [ngbPopover]="errorPopover">{{ mail.error | slice:0:20 }}</span>
}
</td>
</tr>
}
</tbody>
</table>
</div>
<div class="btn-toolbar">
<button type="button" class="btn btn-outline-secondary me-2" (click)="clearSelection()" [disabled]="selectedMailIds.size === 0" i18n>Clear</button>
<pngx-confirm-button
label="Delete selected"
i18n-label
title="Delete selected"
i18n-title
buttonClasses="btn-outline-danger"
iconName="trash"
[disabled]="selectedMailIds.size === 0"
(confirm)="deleteSelected()">
</pngx-confirm-button>
<div class="ms-auto">
<ngb-pagination
[collectionSize]="processedMails.length"
[(page)]="page"
[pageSize]="50"
[maxSize]="5"
(pageChange)="loadProcessedMails()">
</ngb-pagination>
</div>
</div>
}
</div>

View File

@@ -0,0 +1,8 @@
::ng-deep .popover {
max-width: 350px;
pre {
white-space: pre-wrap;
word-break: break-word;
}
}

View File

@@ -0,0 +1,150 @@
import { DatePipe } from '@angular/common'
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import {
HttpTestingController,
provideHttpClientTesting,
} from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { FormsModule } from '@angular/forms'
import { By } from '@angular/platform-browser'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { ToastService } from 'src/app/services/toast.service'
import { environment } from 'src/environments/environment'
import { ProcessedMailDialogComponent } from './processed-mail-dialog.component'
describe('ProcessedMailDialogComponent', () => {
let component: ProcessedMailDialogComponent
let fixture: ComponentFixture<ProcessedMailDialogComponent>
let httpTestingController: HttpTestingController
let toastService: ToastService
const rule: any = { id: 10, name: 'Mail Rule' } // minimal rule object for tests
const mails = [
{
id: 1,
rule: rule.id,
folder: 'INBOX',
uid: 111,
subject: 'A',
received: new Date().toISOString(),
processed: new Date().toISOString(),
status: 'SUCCESS',
error: null,
},
{
id: 2,
rule: rule.id,
folder: 'INBOX',
uid: 222,
subject: 'B',
received: new Date().toISOString(),
processed: new Date().toISOString(),
status: 'FAILED',
error: 'Oops',
},
]
beforeEach(async () => {
await TestBed.configureTestingModule({
imports: [
ProcessedMailDialogComponent,
FormsModule,
NgxBootstrapIconsModule.pick(allIcons),
],
providers: [
DatePipe,
NgbActiveModal,
provideHttpClient(withInterceptorsFromDi()),
provideHttpClientTesting(),
],
}).compileComponents()
httpTestingController = TestBed.inject(HttpTestingController)
toastService = TestBed.inject(ToastService)
fixture = TestBed.createComponent(ProcessedMailDialogComponent)
component = fixture.componentInstance
component.rule = rule
})
afterEach(() => {
httpTestingController.verify()
})
function expectListRequest(ruleId: number) {
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}processed_mail/?page=1&page_size=50&ordering=-processed_at&rule=${ruleId}`
)
expect(req.request.method).toEqual('GET')
return req
}
it('should load processed mails on init', () => {
fixture.detectChanges()
const req = expectListRequest(rule.id)
req.flush({ count: 2, results: mails })
expect(component.loading).toBeFalsy()
expect(component.processedMails).toEqual(mails)
})
it('should delete selected mails and reload', () => {
fixture.detectChanges()
// initial load
const initialReq = expectListRequest(rule.id)
initialReq.flush({ count: 0, results: [] })
// select a couple of mails and delete
component.selectedMailIds.add(5)
component.selectedMailIds.add(6)
const toastInfoSpy = jest.spyOn(toastService, 'showInfo')
component.deleteSelected()
const delReq = httpTestingController.expectOne(
`${environment.apiBaseUrl}processed_mail/bulk_delete/`
)
expect(delReq.request.method).toEqual('POST')
expect(delReq.request.body).toEqual({ mail_ids: [5, 6] })
delReq.flush({})
// reload after delete
const reloadReq = expectListRequest(rule.id)
reloadReq.flush({ count: 0, results: [] })
expect(toastInfoSpy).toHaveBeenCalled()
})
it('should toggle all, toggle selected, and clear selection', () => {
fixture.detectChanges()
// initial load with two mails
const req = expectListRequest(rule.id)
req.flush({ count: 2, results: mails })
fixture.detectChanges()
// toggle all via header checkbox
const inputs = fixture.debugElement.queryAll(
By.css('input.form-check-input')
)
const header = inputs[0].nativeElement as HTMLInputElement
header.dispatchEvent(new Event('click'))
header.checked = true
header.dispatchEvent(new Event('click'))
expect(component.selectedMailIds.size).toEqual(mails.length)
// toggle a single mail
component.toggleSelected(mails[0] as any)
expect(component.selectedMailIds.has(mails[0].id)).toBeFalsy()
component.toggleSelected(mails[0] as any)
expect(component.selectedMailIds.has(mails[0].id)).toBeTruthy()
// clear selection
component.clearSelection()
expect(component.selectedMailIds.size).toEqual(0)
expect(component.toggleAllEnabled).toBeFalsy()
})
it('should close the dialog', () => {
const activeModal = TestBed.inject(NgbActiveModal)
const closeSpy = jest.spyOn(activeModal, 'close')
component.close()
expect(closeSpy).toHaveBeenCalled()
})
})

View File

@@ -0,0 +1,96 @@
import { SlicePipe } from '@angular/common'
import { Component, inject, Input, OnInit } from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import {
NgbActiveModal,
NgbPagination,
NgbPopoverModule,
NgbTooltipModule,
} from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { ConfirmButtonComponent } from 'src/app/components/common/confirm-button/confirm-button.component'
import { MailRule } from 'src/app/data/mail-rule'
import { ProcessedMail } from 'src/app/data/processed-mail'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { ProcessedMailService } from 'src/app/services/rest/processed-mail.service'
import { ToastService } from 'src/app/services/toast.service'
@Component({
selector: 'pngx-processed-mail-dialog',
imports: [
ConfirmButtonComponent,
CustomDatePipe,
NgbPagination,
NgbPopoverModule,
NgbTooltipModule,
NgxBootstrapIconsModule,
FormsModule,
ReactiveFormsModule,
SlicePipe,
],
templateUrl: './processed-mail-dialog.component.html',
styleUrl: './processed-mail-dialog.component.scss',
})
export class ProcessedMailDialogComponent implements OnInit {
private readonly activeModal = inject(NgbActiveModal)
private readonly processedMailService = inject(ProcessedMailService)
private readonly toastService = inject(ToastService)
public processedMails: ProcessedMail[] = []
public loading: boolean = true
public toggleAllEnabled: boolean = false
public readonly selectedMailIds: Set<number> = new Set<number>()
public page: number = 1
@Input() rule: MailRule
ngOnInit(): void {
this.loadProcessedMails()
}
public close() {
this.activeModal.close()
}
private loadProcessedMails(): void {
this.loading = true
this.clearSelection()
this.processedMailService
.list(this.page, 50, 'processed_at', true, { rule: this.rule.id })
.subscribe((result) => {
this.processedMails = result.results
this.loading = false
})
}
public deleteSelected(): void {
this.processedMailService
.bulk_delete(Array.from(this.selectedMailIds))
.subscribe(() => {
this.toastService.showInfo($localize`Processed mail(s) deleted`)
this.loadProcessedMails()
})
}
public toggleAll(event: PointerEvent) {
if ((event.target as HTMLInputElement).checked) {
this.selectedMailIds.clear()
this.processedMails.forEach((mail) => this.selectedMailIds.add(mail.id))
} else {
this.clearSelection()
}
}
public clearSelection() {
this.toggleAllEnabled = false
this.selectedMailIds.clear()
}
public toggleSelected(mail: ProcessedMail) {
this.selectedMailIds.has(mail.id)
? this.selectedMailIds.delete(mail.id)
: this.selectedMailIds.add(mail.id)
}
}

View File

@@ -71,4 +71,20 @@ describe('TagListComponent', () => {
'Do you really want to delete the tag "Tag1"?'
)
})
it('should filter out child tags if name filter is empty, otherwise show all', () => {
const tags = [
{ id: 1, name: 'Tag1', parent: null },
{ id: 2, name: 'Tag2', parent: 1 },
{ id: 3, name: 'Tag3', parent: null },
]
component['_nameFilter'] = null // Simulate empty name filter
const filtered = component.filterData(tags as any)
expect(filtered.length).toBe(2)
expect(filtered.find((t) => t.id === 2)).toBeUndefined()
component['_nameFilter'] = 'Tag2' // Simulate non-empty name filter
const filteredWithName = component.filterData(tags as any)
expect(filteredWithName.length).toBe(3)
})
})

View File

@@ -62,6 +62,8 @@ export class TagListComponent extends ManagementListComponent<Tag> {
}
filterData(data: Tag[]) {
return data.filter((tag) => !tag.parent)
return this.nameFilter?.length
? [...data]
: data.filter((tag) => !tag.parent)
}
}

View File

@@ -0,0 +1,12 @@
import { ObjectWithId } from './object-with-id'
export interface ProcessedMail extends ObjectWithId {
rule: number // MailRule.id
folder: string
uid: number
subject: string
received: Date
processed: Date
status: string
error: string
}

View File

@@ -28,6 +28,7 @@ export enum PermissionType {
ShareLink = '%s_sharelink',
CustomField = '%s_customfield',
Workflow = '%s_workflow',
ProcessedMail = '%s_processedmail',
}
@Injectable({

View File

@@ -0,0 +1,39 @@
import { HttpTestingController } from '@angular/common/http/testing'
import { TestBed } from '@angular/core/testing'
import { Subscription } from 'rxjs'
import { environment } from 'src/environments/environment'
import { commonAbstractPaperlessServiceTests } from './abstract-paperless-service.spec'
import { ProcessedMailService } from './processed-mail.service'
let httpTestingController: HttpTestingController
let service: ProcessedMailService
let subscription: Subscription
const endpoint = 'processed_mail'
// run common tests
commonAbstractPaperlessServiceTests(endpoint, ProcessedMailService)
describe('Additional service tests for ProcessedMailService', () => {
beforeEach(() => {
// Dont need to setup again
httpTestingController = TestBed.inject(HttpTestingController)
service = TestBed.inject(ProcessedMailService)
})
afterEach(() => {
subscription?.unsubscribe()
httpTestingController.verify()
})
it('should call appropriate api endpoint for bulk delete', () => {
const ids = [1, 2, 3]
subscription = service.bulk_delete(ids).subscribe()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/bulk_delete/`
)
expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({ mail_ids: ids })
req.flush({})
})
})

View File

@@ -0,0 +1,19 @@
import { Injectable } from '@angular/core'
import { ProcessedMail } from 'src/app/data/processed-mail'
import { AbstractPaperlessService } from './abstract-paperless-service'
@Injectable({
providedIn: 'root',
})
export class ProcessedMailService extends AbstractPaperlessService<ProcessedMail> {
constructor() {
super()
this.resourceName = 'processed_mail'
}
public bulk_delete(mailIds: number[]) {
return this.http.post(`${this.getResourceUrl()}bulk_delete/`, {
mail_ids: mailIds,
})
}
}

View File

@@ -51,17 +51,17 @@ import {
check,
check2All,
checkAll,
checkCircle,
checkCircleFill,
checkLg,
chevronDoubleLeft,
chevronDoubleRight,
chevronDown,
chevronRight,
chevronUp,
clipboard,
clipboardCheck,
clipboardCheckFill,
clipboardFill,
clockHistory,
dash,
dashCircle,
diagram3,
@@ -265,17 +265,17 @@ const icons = {
check,
check2All,
checkAll,
checkCircle,
checkCircleFill,
checkLg,
chevronDoubleLeft,
chevronDoubleRight,
chevronDown,
chevronRight,
chevronUp,
clipboard,
clipboardCheck,
clipboardCheckFill,
clipboardFill,
clockHistory,
dash,
dashCircle,
diagram3,

View File

@@ -164,6 +164,9 @@ class BarcodePlugin(ConsumeTaskPlugin):
mailrule_id=self.input_doc.mailrule_id,
# Can't use same folder or the consume might grab it again
original_file=(tmp_dir / new_document.name).resolve(),
# Adding optional original_path for later uses in
# workflow matching
original_path=self.input_doc.original_file,
),
# All the same metadata
self.metadata,

View File

@@ -156,6 +156,7 @@ class ConsumableDocument:
source: DocumentSource
original_file: Path
original_path: Path | None = None
mailrule_id: int | None = None
mime_type: str = dataclasses.field(init=False, default=None)

View File

@@ -82,6 +82,13 @@ def _is_ignored(filepath: Path) -> bool:
def _consume(filepath: Path) -> None:
# Check permissions early
try:
filepath.stat()
except (PermissionError, OSError):
logger.warning(f"Not consuming file {filepath}: Permission denied.")
return
if filepath.is_dir() or _is_ignored(filepath):
return
@@ -323,7 +330,12 @@ class Command(BaseCommand):
# Also make sure the file exists still, some scanners might write a
# temporary file first
try:
file_still_exists = filepath.exists() and filepath.is_file()
except (PermissionError, OSError): # pragma: no cover
# If we can't check, let it fail in the _consume function
file_still_exists = True
continue
if waited_long_enough and file_still_exists:
_consume(filepath)

View File

@@ -92,6 +92,9 @@ class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
# doc to doc is obviously not useful
if first_doc.pk == second_doc.pk:
continue
# Skip empty documents (e.g. password-protected)
if first_doc.content.strip() == "" or second_doc.content.strip() == "":
continue
# Skip matching which have already been matched together
# doc 1 to doc 2 is the same as doc 2 to doc 1
doc_1_to_doc_2 = (first_doc.pk, second_doc.pk)

View File

@@ -314,11 +314,19 @@ def consumable_document_matches_workflow(
trigger_matched = False
# Document path vs trigger path
# Use the original_path if set, else us the original_file
match_against = (
document.original_path
if document.original_path is not None
else document.original_file
)
if (
trigger.filter_path is not None
and len(trigger.filter_path) > 0
and not fnmatch(
document.original_file,
match_against,
trigger.filter_path,
)
):

View File

@@ -6,6 +6,7 @@ import re
from datetime import datetime
from decimal import Decimal
from typing import TYPE_CHECKING
from typing import Literal
import magic
from celery import states
@@ -252,6 +253,35 @@ class OwnedObjectSerializer(
except KeyError:
pass
def _get_perms(self, obj, codename: str, target: Literal["users", "groups"]):
"""
Get the given permissions from context or from django-guardian.
:param codename: The permission codename, e.g. 'view' or 'change'
:param target: 'users' or 'groups'
"""
key = f"{target}_{codename}_perms"
cached = self.context.get(key, {}).get(obj.pk)
if cached is not None:
return list(cached)
# Permission not found in the context, get it from guardian
if target == "users":
return list(
get_users_with_perms(
obj,
only_with_perms_in=[f"{codename}_{obj.__class__.__name__.lower()}"],
with_group_users=False,
).values_list("id", flat=True),
)
else: # groups
return list(
get_groups_with_only_permission(
obj,
codename=f"{codename}_{obj.__class__.__name__.lower()}",
).values_list("id", flat=True),
)
@extend_schema_field(
field={
"type": "object",
@@ -286,31 +316,14 @@ class OwnedObjectSerializer(
},
)
def get_permissions(self, obj) -> dict:
view_codename = f"view_{obj.__class__.__name__.lower()}"
change_codename = f"change_{obj.__class__.__name__.lower()}"
return {
"view": {
"users": get_users_with_perms(
obj,
only_with_perms_in=[view_codename],
with_group_users=False,
).values_list("id", flat=True),
"groups": get_groups_with_only_permission(
obj,
codename=view_codename,
).values_list("id", flat=True),
"users": self._get_perms(obj, "view", "users"),
"groups": self._get_perms(obj, "view", "groups"),
},
"change": {
"users": get_users_with_perms(
obj,
only_with_perms_in=[change_codename],
with_group_users=False,
).values_list("id", flat=True),
"groups": get_groups_with_only_permission(
obj,
codename=change_codename,
).values_list("id", flat=True),
"users": self._get_perms(obj, "change", "users"),
"groups": self._get_perms(obj, "change", "groups"),
},
}

View File

@@ -614,14 +614,16 @@ class TestBarcodeNewConsume(
self.assertIsNotFile(temp_copy)
# Check the split files exist
# Check the original_path is set
# Check the source is unchanged
# Check the overrides are unchanged
for (
new_input_doc,
new_doc_overrides,
) in self.get_all_consume_delay_call_args():
self.assertEqual(new_input_doc.source, DocumentSource.ConsumeFolder)
self.assertIsFile(new_input_doc.original_file)
self.assertEqual(new_input_doc.original_path, temp_copy)
self.assertEqual(new_input_doc.source, DocumentSource.ConsumeFolder)
self.assertEqual(overrides, new_doc_overrides)

View File

@@ -209,6 +209,26 @@ class TestConsumer(DirectoriesMixin, ConsumerThreadMixin, TransactionTestCase):
# assert that we have an error logged with this invalid file.
error_logger.assert_called_once()
@mock.patch("documents.management.commands.document_consumer.logger.warning")
def test_permission_error_on_prechecks(self, warning_logger):
filepath = Path(self.dirs.consumption_dir) / "selinux.txt"
filepath.touch()
original_stat = Path.stat
def raising_stat(self, *args, **kwargs):
if self == filepath:
raise PermissionError("Permission denied")
return original_stat(self, *args, **kwargs)
with mock.patch("pathlib.Path.stat", new=raising_stat):
document_consumer._consume(filepath)
warning_logger.assert_called_once()
(args, _) = warning_logger.call_args
self.assertIn("Permission denied", args[0])
self.consume_file_mock.assert_not_called()
@override_settings(CONSUMPTION_DIR="does_not_exist")
def test_consumption_directory_invalid(self):
self.assertRaises(CommandError, call_command, "document_consumer", "--oneshot")

View File

@@ -206,3 +206,29 @@ class TestFuzzyMatchCommand(TestCase):
self.assertEqual(Document.objects.count(), 2)
self.assertIsNotNone(Document.objects.get(pk=1))
self.assertIsNotNone(Document.objects.get(pk=2))
def test_empty_content(self):
"""
GIVEN:
- 2 documents exist, content is empty (pw-protected)
WHEN:
- Command is called
THEN:
- No matches are found
"""
Document.objects.create(
checksum="BEEFCAFE",
title="A",
content="",
mime_type="application/pdf",
filename="test.pdf",
)
Document.objects.create(
checksum="DEADBEAF",
title="A",
content="",
mime_type="application/pdf",
filename="other_test.pdf",
)
stdout, _ = self.call_command()
self.assertIn("No matches found", stdout)

View File

@@ -1,17 +1,23 @@
import json
import tempfile
from datetime import timedelta
from pathlib import Path
from django.conf import settings
from django.contrib.auth.models import Group
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.db import connection
from django.test import TestCase
from django.test import override_settings
from django.test.utils import CaptureQueriesContext
from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import status
from documents.models import Document
from documents.models import ShareLink
from documents.models import Tag
from documents.tests.utils import DirectoriesMixin
from paperless.models import ApplicationConfiguration
@@ -154,3 +160,113 @@ class TestViews(DirectoriesMixin, TestCase):
response.render()
self.assertEqual(response.request["PATH_INFO"], "/accounts/login/")
self.assertContains(response, b"Share link has expired")
def test_list_with_full_permissions(self):
"""
GIVEN:
- Tags with different permissions
WHEN:
- Request to get tag list with full permissions is made
THEN:
- Tag list is returned with the right permission information
"""
user2 = User.objects.create(username="user2")
user3 = User.objects.create(username="user3")
group1 = Group.objects.create(name="group1")
group2 = Group.objects.create(name="group2")
group3 = Group.objects.create(name="group3")
t1 = Tag.objects.create(name="invoice", pk=1)
assign_perm("view_tag", self.user, t1)
assign_perm("view_tag", user2, t1)
assign_perm("view_tag", user3, t1)
assign_perm("view_tag", group1, t1)
assign_perm("view_tag", group2, t1)
assign_perm("view_tag", group3, t1)
assign_perm("change_tag", self.user, t1)
assign_perm("change_tag", user2, t1)
assign_perm("change_tag", group1, t1)
assign_perm("change_tag", group2, t1)
Tag.objects.create(name="bank statement", pk=2)
d1 = Document.objects.create(
title="Invoice 1",
content="This is the invoice of a very expensive item",
checksum="A",
)
d1.tags.add(t1)
d2 = Document.objects.create(
title="Invoice 2",
content="Internet invoice, I should pay it to continue contributing",
checksum="B",
)
d2.tags.add(t1)
view_permissions = Permission.objects.filter(
codename__contains="view_tag",
)
self.user.user_permissions.add(*view_permissions)
self.user.save()
self.client.force_login(self.user)
response = self.client.get("/api/tags/?page=1&full_perms=true")
results = json.loads(response.content)["results"]
for tag in results:
if tag["name"] == "invoice":
assert tag["permissions"] == {
"view": {
"users": [self.user.pk, user2.pk, user3.pk],
"groups": [group1.pk, group2.pk, group3.pk],
},
"change": {
"users": [self.user.pk, user2.pk],
"groups": [group1.pk, group2.pk],
},
}
elif tag["name"] == "bank statement":
assert tag["permissions"] == {
"view": {"users": [], "groups": []},
"change": {"users": [], "groups": []},
}
else:
assert False, f"Unexpected tag found: {tag['name']}"
def test_list_no_n_plus_1_queries(self):
"""
GIVEN:
- Tags with different permissions
WHEN:
- Request to get tag list with full permissions is made
THEN:
- Permissions are not queried in database tag by tag,
i.e. there are no N+1 queries
"""
view_permissions = Permission.objects.filter(
codename__contains="view_tag",
)
self.user.user_permissions.add(*view_permissions)
self.user.save()
self.client.force_login(self.user)
# Start by a small list, and count the number of SQL queries
for i in range(2):
Tag.objects.create(name=f"tag_{i}")
with CaptureQueriesContext(connection) as ctx_small:
response_small = self.client.get("/api/tags/?full_perms=true")
assert response_small.status_code == 200
num_queries_small = len(ctx_small.captured_queries)
# Complete the list, and count the number of SQL queries again
for i in range(2, 50):
Tag.objects.create(name=f"tag_{i}")
with CaptureQueriesContext(connection) as ctx_large:
response_large = self.client.get("/api/tags/?full_perms=true")
assert response_large.status_code == 200
num_queries_large = len(ctx_large.captured_queries)
# A few additional queries are allowed, but not a linear explosion
assert num_queries_large <= num_queries_small + 5, (
f"Possible N+1 queries detected: {num_queries_small} queries for 2 tags, "
f"but {num_queries_large} queries for 50 tags"
)

View File

@@ -5,9 +5,11 @@ import platform
import re
import tempfile
import zipfile
from collections import defaultdict
from datetime import datetime
from pathlib import Path
from time import mktime
from typing import Literal
from unicodedata import normalize
from urllib.parse import quote
from urllib.parse import urlparse
@@ -19,6 +21,7 @@ from celery import states
from django.conf import settings
from django.contrib.auth.models import Group
from django.contrib.auth.models import User
from django.contrib.contenttypes.models import ContentType
from django.db import connections
from django.db.migrations.loader import MigrationLoader
from django.db.migrations.recorder import MigrationRecorder
@@ -56,6 +59,8 @@ from drf_spectacular.utils import OpenApiParameter
from drf_spectacular.utils import extend_schema
from drf_spectacular.utils import extend_schema_view
from drf_spectacular.utils import inline_serializer
from guardian.utils import get_group_obj_perms_model
from guardian.utils import get_user_obj_perms_model
from langdetect import detect
from packaging import version as packaging_version
from redis import Redis
@@ -254,7 +259,104 @@ class PassUserMixin(GenericAPIView):
return super().get_serializer(*args, **kwargs)
class PermissionsAwareDocumentCountMixin(PassUserMixin):
class BulkPermissionMixin:
"""
Prefetch Django-Guardian permissions for a list before serialization, to avoid N+1 queries.
"""
def _get_object_perms(
self,
objects: list,
perm_codenames: list[str],
actor: Literal["users", "groups"],
) -> dict[int, dict[str, list[int]]]:
"""
Collect object-level permissions for either users or groups.
"""
model = self.queryset.model
obj_perm_model = (
get_user_obj_perms_model(model)
if actor == "users"
else get_group_obj_perms_model(model)
)
id_field = "user_id" if actor == "users" else "group_id"
ctype = ContentType.objects.get_for_model(model)
object_pks = [obj.pk for obj in objects]
perms_qs = obj_perm_model.objects.filter(
content_type=ctype,
object_pk__in=object_pks,
permission__codename__in=perm_codenames,
).values_list("object_pk", id_field, "permission__codename")
perms: dict[int, dict[str, list[int]]] = defaultdict(lambda: defaultdict(list))
for object_pk, actor_id, codename in perms_qs:
perms[int(object_pk)][codename].append(actor_id)
# Ensure that all objects have all codenames, even if empty
for pk in object_pks:
for codename in perm_codenames:
perms[pk][codename]
return perms
def get_serializer_context(self):
"""
Get all permissions of the current list of objects at once and pass them to the serializer.
This avoid fetching permissions object by object in database.
"""
context = super().get_serializer_context()
try:
full_perms = get_boolean(
str(self.request.query_params.get("full_perms", "false")),
)
except ValueError:
full_perms = False
if not full_perms:
return context
# Check which objects are being paginated
page = getattr(self, "paginator", None)
if page and hasattr(page, "page"):
queryset = page.page.object_list
elif hasattr(self, "page"):
queryset = self.page
else:
queryset = self.filter_queryset(self.get_queryset())
model_name = self.queryset.model.__name__.lower()
permission_name_view = f"view_{model_name}"
permission_name_change = f"change_{model_name}"
user_perms = self._get_object_perms(
objects=queryset,
perm_codenames=[permission_name_view, permission_name_change],
actor="users",
)
group_perms = self._get_object_perms(
objects=queryset,
perm_codenames=[permission_name_view, permission_name_change],
actor="groups",
)
context["users_view_perms"] = {
pk: user_perms[pk][permission_name_view] for pk in user_perms
}
context["users_change_perms"] = {
pk: user_perms[pk][permission_name_change] for pk in user_perms
}
context["groups_view_perms"] = {
pk: group_perms[pk][permission_name_view] for pk in group_perms
}
context["groups_change_perms"] = {
pk: group_perms[pk][permission_name_change] for pk in group_perms
}
return context
class PermissionsAwareDocumentCountMixin(BulkPermissionMixin, PassUserMixin):
"""
Mixin to add document count to queryset, permissions-aware if needed
"""

View File

@@ -2,7 +2,7 @@ msgid ""
msgstr ""
"Project-Id-Version: paperless-ngx\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2025-09-17 22:44+0000\n"
"POT-Creation-Date: 2025-09-30 16:50+0000\n"
"PO-Revision-Date: 2022-02-17 04:17\n"
"Last-Translator: \n"
"Language-Team: English\n"
@@ -1191,44 +1191,44 @@ msgstr ""
msgid "workflow runs"
msgstr ""
#: documents/serialisers.py:140
#: documents/serialisers.py:141
#, python-format
msgid "Invalid regular expression: %(error)s"
msgstr ""
#: documents/serialisers.py:594
#: documents/serialisers.py:607
msgid "Invalid color."
msgstr ""
#: documents/serialisers.py:623
#: documents/serialisers.py:636
msgid "Invalid parent tag."
msgstr ""
#: documents/serialisers.py:1780
#: documents/serialisers.py:1793
#, python-format
msgid "File type %(type)s not supported"
msgstr ""
#: documents/serialisers.py:1824
#: documents/serialisers.py:1837
#, python-format
msgid "Custom field id must be an integer: %(id)s"
msgstr ""
#: documents/serialisers.py:1831
#: documents/serialisers.py:1844
#, python-format
msgid "Custom field with id %(id)s does not exist"
msgstr ""
#: documents/serialisers.py:1848 documents/serialisers.py:1858
#: documents/serialisers.py:1861 documents/serialisers.py:1871
msgid ""
"Custom fields must be a list of integers or an object mapping ids to values."
msgstr ""
#: documents/serialisers.py:1853
#: documents/serialisers.py:1866
msgid "Some custom fields don't exist or were specified twice."
msgstr ""
#: documents/serialisers.py:1923
#: documents/serialisers.py:1936
msgid "Invalid variable detected."
msgstr ""
@@ -1827,7 +1827,7 @@ msgstr ""
msgid "Chinese Traditional"
msgstr ""
#: paperless/urls.py:368
#: paperless/urls.py:370
msgid "Paperless-ngx administration"
msgstr ""

View File

@@ -922,7 +922,7 @@ CELERY_ACCEPT_CONTENT = ["application/json", "application/x-python-serialize"]
CELERY_BEAT_SCHEDULE = _parse_beat_schedule()
# https://docs.celeryq.dev/en/stable/userguide/configuration.html#beat-schedule-filename
CELERY_BEAT_SCHEDULE_FILENAME = DATA_DIR / "celerybeat-schedule.db"
CELERY_BEAT_SCHEDULE_FILENAME = str(DATA_DIR / "celerybeat-schedule.db")
# Cachalot: Database read cache.

View File

@@ -57,6 +57,7 @@ from paperless.views import UserViewSet
from paperless_mail.views import MailAccountViewSet
from paperless_mail.views import MailRuleViewSet
from paperless_mail.views import OauthCallbackView
from paperless_mail.views import ProcessedMailViewSet
api_router = DefaultRouter()
api_router.register(r"correspondents", CorrespondentViewSet)
@@ -77,6 +78,7 @@ api_router.register(r"workflow_actions", WorkflowActionViewSet)
api_router.register(r"workflows", WorkflowViewSet)
api_router.register(r"custom_fields", CustomFieldViewSet)
api_router.register(r"config", ApplicationConfigurationViewSet)
api_router.register(r"processed_mail", ProcessedMailViewSet)
urlpatterns = [

View File

@@ -0,0 +1,12 @@
from django_filters import FilterSet
from paperless_mail.models import ProcessedMail
class ProcessedMailFilterSet(FilterSet):
class Meta:
model = ProcessedMail
fields = {
"rule": ["exact"],
"status": ["exact"],
}

View File

@@ -6,6 +6,7 @@ from documents.serialisers import OwnedObjectSerializer
from documents.serialisers import TagsField
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from paperless_mail.models import ProcessedMail
class ObfuscatedPasswordField(serializers.CharField):
@@ -130,3 +131,20 @@ class MailRuleSerializer(OwnedObjectSerializer):
if value > 36500: # ~100 years
raise serializers.ValidationError("Maximum mail age is unreasonably large.")
return value
class ProcessedMailSerializer(OwnedObjectSerializer):
class Meta:
model = ProcessedMail
fields = [
"id",
"owner",
"rule",
"folder",
"uid",
"subject",
"received",
"processed",
"status",
"error",
]

View File

@@ -3,6 +3,7 @@ from unittest import mock
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.utils import timezone
from guardian.shortcuts import assign_perm
from rest_framework import status
from rest_framework.test import APITestCase
@@ -13,6 +14,7 @@ from documents.models import Tag
from documents.tests.utils import DirectoriesMixin
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from paperless_mail.models import ProcessedMail
from paperless_mail.tests.test_mail import BogusMailBox
@@ -721,3 +723,285 @@ class TestAPIMailRules(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn("maximum_age", response.data)
class TestAPIProcessedMails(DirectoriesMixin, APITestCase):
ENDPOINT = "/api/processed_mail/"
def setUp(self):
super().setUp()
self.user = User.objects.create_user(username="temp_admin")
self.user.user_permissions.add(*Permission.objects.all())
self.user.save()
self.client.force_authenticate(user=self.user)
def test_get_processed_mails_owner_aware(self):
"""
GIVEN:
- Configured processed mails with different users
WHEN:
- API call is made to get processed mails
THEN:
- Only unowned, owned by user or granted processed mails are provided
"""
user2 = User.objects.create_user(username="temp_admin2")
account = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
rule = MailRule.objects.create(
name="Rule1",
account=account,
folder="INBOX",
filter_from="from@example.com",
order=0,
)
pm1 = ProcessedMail.objects.create(
rule=rule,
folder="INBOX",
uid="1",
subject="Subj1",
received=timezone.now(),
processed=timezone.now(),
status="SUCCESS",
error=None,
)
pm2 = ProcessedMail.objects.create(
rule=rule,
folder="INBOX",
uid="2",
subject="Subj2",
received=timezone.now(),
processed=timezone.now(),
status="FAILED",
error="err",
owner=self.user,
)
ProcessedMail.objects.create(
rule=rule,
folder="INBOX",
uid="3",
subject="Subj3",
received=timezone.now(),
processed=timezone.now(),
status="SUCCESS",
error=None,
owner=user2,
)
pm4 = ProcessedMail.objects.create(
rule=rule,
folder="INBOX",
uid="4",
subject="Subj4",
received=timezone.now(),
processed=timezone.now(),
status="SUCCESS",
error=None,
)
pm4.owner = user2
pm4.save()
assign_perm("view_processedmail", self.user, pm4)
response = self.client.get(self.ENDPOINT)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["count"], 3)
returned_ids = {r["id"] for r in response.data["results"]}
self.assertSetEqual(returned_ids, {pm1.id, pm2.id, pm4.id})
def test_get_processed_mails_filter_by_rule(self):
"""
GIVEN:
- Processed mails belonging to two different rules
WHEN:
- API call is made with rule filter
THEN:
- Only processed mails for that rule are returned
"""
account = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
rule1 = MailRule.objects.create(
name="Rule1",
account=account,
folder="INBOX",
filter_from="from1@example.com",
order=0,
)
rule2 = MailRule.objects.create(
name="Rule2",
account=account,
folder="INBOX",
filter_from="from2@example.com",
order=1,
)
pm1 = ProcessedMail.objects.create(
rule=rule1,
folder="INBOX",
uid="r1-1",
subject="R1-A",
received=timezone.now(),
processed=timezone.now(),
status="SUCCESS",
error=None,
owner=self.user,
)
pm2 = ProcessedMail.objects.create(
rule=rule1,
folder="INBOX",
uid="r1-2",
subject="R1-B",
received=timezone.now(),
processed=timezone.now(),
status="FAILED",
error="e",
)
ProcessedMail.objects.create(
rule=rule2,
folder="INBOX",
uid="r2-1",
subject="R2-A",
received=timezone.now(),
processed=timezone.now(),
status="SUCCESS",
error=None,
)
response = self.client.get(f"{self.ENDPOINT}?rule={rule1.pk}")
self.assertEqual(response.status_code, status.HTTP_200_OK)
returned_ids = {r["id"] for r in response.data["results"]}
self.assertSetEqual(returned_ids, {pm1.id, pm2.id})
def test_bulk_delete_processed_mails(self):
"""
GIVEN:
- Processed mails belonging to two different rules and different users
WHEN:
- API call is made to bulk delete some of the processed mails
THEN:
- Only the specified processed mails are deleted, respecting ownership and permissions
"""
user2 = User.objects.create_user(username="temp_admin2")
account = MailAccount.objects.create(
name="Email1",
username="username1",
password="password1",
imap_server="server.example.com",
imap_port=443,
imap_security=MailAccount.ImapSecurity.SSL,
character_set="UTF-8",
)
rule = MailRule.objects.create(
name="Rule1",
account=account,
folder="INBOX",
filter_from="from@example.com",
order=0,
)
# unowned and owned by self, and one with explicit object perm
pm_unowned = ProcessedMail.objects.create(
rule=rule,
folder="INBOX",
uid="u1",
subject="Unowned",
received=timezone.now(),
processed=timezone.now(),
status="SUCCESS",
error=None,
)
pm_owned = ProcessedMail.objects.create(
rule=rule,
folder="INBOX",
uid="u2",
subject="Owned",
received=timezone.now(),
processed=timezone.now(),
status="FAILED",
error="e",
owner=self.user,
)
pm_granted = ProcessedMail.objects.create(
rule=rule,
folder="INBOX",
uid="u3",
subject="Granted",
received=timezone.now(),
processed=timezone.now(),
status="SUCCESS",
error=None,
owner=user2,
)
assign_perm("delete_processedmail", self.user, pm_granted)
pm_forbidden = ProcessedMail.objects.create(
rule=rule,
folder="INBOX",
uid="u4",
subject="Forbidden",
received=timezone.now(),
processed=timezone.now(),
status="SUCCESS",
error=None,
owner=user2,
)
# Success for allowed items
response = self.client.post(
f"{self.ENDPOINT}bulk_delete/",
data={
"mail_ids": [pm_unowned.id, pm_owned.id, pm_granted.id],
},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["result"], "OK")
self.assertSetEqual(
set(response.data["deleted_mail_ids"]),
{pm_unowned.id, pm_owned.id, pm_granted.id},
)
self.assertFalse(ProcessedMail.objects.filter(id=pm_unowned.id).exists())
self.assertFalse(ProcessedMail.objects.filter(id=pm_owned.id).exists())
self.assertFalse(ProcessedMail.objects.filter(id=pm_granted.id).exists())
self.assertTrue(ProcessedMail.objects.filter(id=pm_forbidden.id).exists())
# 403 and not deleted
response = self.client.post(
f"{self.ENDPOINT}bulk_delete/",
data={
"mail_ids": [pm_forbidden.id],
},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
self.assertTrue(ProcessedMail.objects.filter(id=pm_forbidden.id).exists())
# missing mail_ids
response = self.client.post(
f"{self.ENDPOINT}bulk_delete/",
data={"mail_ids": "not-a-list"},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)

View File

@@ -3,8 +3,10 @@ import logging
from datetime import timedelta
from django.http import HttpResponseBadRequest
from django.http import HttpResponseForbidden
from django.http import HttpResponseRedirect
from django.utils import timezone
from django_filters.rest_framework import DjangoFilterBackend
from drf_spectacular.types import OpenApiTypes
from drf_spectacular.utils import extend_schema
from drf_spectacular.utils import extend_schema_view
@@ -12,23 +14,29 @@ from drf_spectacular.utils import inline_serializer
from httpx_oauth.oauth2 import GetAccessTokenError
from rest_framework import serializers
from rest_framework.decorators import action
from rest_framework.filters import OrderingFilter
from rest_framework.generics import GenericAPIView
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
from rest_framework.viewsets import ModelViewSet
from rest_framework.viewsets import ReadOnlyModelViewSet
from documents.filters import ObjectOwnedOrGrantedPermissionsFilter
from documents.permissions import PaperlessObjectPermissions
from documents.permissions import has_perms_owner_aware
from documents.views import PassUserMixin
from paperless.views import StandardPagination
from paperless_mail.filters import ProcessedMailFilterSet
from paperless_mail.mail import MailError
from paperless_mail.mail import get_mailbox
from paperless_mail.mail import mailbox_login
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from paperless_mail.models import ProcessedMail
from paperless_mail.oauth import PaperlessMailOAuth2Manager
from paperless_mail.serialisers import MailAccountSerializer
from paperless_mail.serialisers import MailRuleSerializer
from paperless_mail.serialisers import ProcessedMailSerializer
from paperless_mail.tasks import process_mail_accounts
@@ -126,6 +134,34 @@ class MailAccountViewSet(ModelViewSet, PassUserMixin):
return Response({"result": "OK"})
class ProcessedMailViewSet(ReadOnlyModelViewSet, PassUserMixin):
permission_classes = (IsAuthenticated, PaperlessObjectPermissions)
serializer_class = ProcessedMailSerializer
pagination_class = StandardPagination
filter_backends = (
DjangoFilterBackend,
OrderingFilter,
ObjectOwnedOrGrantedPermissionsFilter,
)
filterset_class = ProcessedMailFilterSet
queryset = ProcessedMail.objects.all().order_by("-processed")
@action(methods=["post"], detail=False)
def bulk_delete(self, request):
mail_ids = request.data.get("mail_ids", [])
if not isinstance(mail_ids, list) or not all(
isinstance(i, int) for i in mail_ids
):
return HttpResponseBadRequest("mail_ids must be a list of integers")
mails = ProcessedMail.objects.filter(id__in=mail_ids)
for mail in mails:
if not has_perms_owner_aware(request.user, "delete_processedmail", mail):
return HttpResponseForbidden("Insufficient permissions")
mail.delete()
return Response({"result": "OK", "deleted_mail_ids": mail_ids})
class MailRuleViewSet(ModelViewSet, PassUserMixin):
model = MailRule

2480
uv.lock generated

File diff suppressed because it is too large Load Diff