Compare commits

...

13 Commits

Author SHA1 Message Date
Trenton H
02d4fcfe0b Cleans up config formatting 2026-02-23 14:49:08 -08:00
Trenton H
5db46cbc3a Finshes the writeup and migration guide 2026-02-23 14:25:56 -08:00
Trenton H
3c9bbb50b3 Relaxes the typing here 2026-02-23 13:27:19 -08:00
Trenton H
480a406e8a Makes the system checks return multiples, not mashed together 2026-02-23 12:58:06 -08:00
Trenton H
b26546c806 Missing the required parameters 2026-02-23 12:32:27 -08:00
Trenton H
7dc5fdc1d8 Smaller changeset, keep the rest for later 2026-02-23 12:32:27 -08:00
Trenton H
bd1dce2412 Adds the checks for depracated env vars 2026-02-23 12:32:27 -08:00
Trenton H
e9de03303e Moves to the new parser module 2026-02-23 12:32:27 -08:00
Trenton H
d0164e68cf Renames the settings to a module for the next steps 2026-02-23 12:32:27 -08:00
Trenton H
814f57b099 Allows the typing job to error and still pass, so we get results, but not failures for now (#12147) 2026-02-23 09:44:35 -08:00
GitHub Actions
be7f1c6233 Auto translate strings 2026-02-22 23:18:50 +00:00
shamoon
d6cd6d0311 Tweakhancement: reset to page 1 on reset filters (#12143) 2026-02-22 15:17:02 -08:00
Daniel Herrmann
095ea3cbd3 Documentation: clarify behaviour around document splitting (#12137)
Co-Authored-By: shamoon <4887959+shamoon@users.noreply.github.com>
2026-02-22 08:26:38 -08:00
20 changed files with 1094 additions and 239 deletions

View File

@@ -39,3 +39,6 @@ max_line_length = off
[Dockerfile*] [Dockerfile*]
indent_style = space indent_style = space
[*.toml]
indent_style = space

View File

@@ -129,6 +129,7 @@ jobs:
run: | run: |
uv pip list uv pip list
- name: Check typing (pyrefly) - name: Check typing (pyrefly)
continue-on-error: true
run: | run: |
uv run pyrefly \ uv run pyrefly \
check \ check \
@@ -143,6 +144,7 @@ jobs:
${{ runner.os }}-mypy-py${{ env.DEFAULT_PYTHON }}- ${{ runner.os }}-mypy-py${{ env.DEFAULT_PYTHON }}-
${{ runner.os }}-mypy- ${{ runner.os }}-mypy-
- name: Check typing (mypy) - name: Check typing (mypy)
continue-on-error: true
run: | run: |
uv run mypy \ uv run mypy \
--show-error-codes \ --show-error-codes \

View File

@@ -784,9 +784,17 @@ below.
### Document Splitting {#document-splitting} ### Document Splitting {#document-splitting}
When enabled, Paperless will look for a barcode with the configured value and create a new document If document splitting is enabled, Paperless splits _after_ a separator barcode by default.
starting from the next page. The page with the barcode on it will _not_ be retained. It This means:
is expected to be a page existing only for triggering the split.
- any page containing the configured separator barcode starts a new document, starting with the **next** page
- pages containing the separator barcode are discarded
This is intended for dedicated separator sheets such as PATCH-T pages.
If [`PAPERLESS_CONSUMER_BARCODE_RETAIN_SPLIT_PAGES`](configuration.md#PAPERLESS_CONSUMER_BARCODE_RETAIN_SPLIT_PAGES)
is enabled, the page containing the separator barcode is retained instead. In this mode,
each page containing the separator barcode becomes the **first** page of a new document.
### Archive Serial Number Assignment ### Archive Serial Number Assignment
@@ -795,8 +803,9 @@ archive serial number, allowing quick reference back to the original, paper docu
If document splitting via barcode is also enabled, documents will be split when an ASN If document splitting via barcode is also enabled, documents will be split when an ASN
barcode is located. However, differing from the splitting, the page with the barcode is located. However, differing from the splitting, the page with the
barcode _will_ be retained. This allows application of a barcode to any page, including barcode _will_ be retained. Each detected ASN barcode starts a new document _starting with
one which holds data to keep in the document. that page_. This allows placing ASN barcodes on content pages that should remain part of
the document.
### Tag Assignment ### Tag Assignment

View File

@@ -51,137 +51,163 @@ matcher.
### Database ### Database
By default, Paperless uses **SQLite** with a database stored at `data/db.sqlite3`. By default, Paperless uses **SQLite** with a database stored at `data/db.sqlite3`.
To switch to **PostgreSQL** or **MariaDB**, set [`PAPERLESS_DBHOST`](#PAPERLESS_DBHOST) and optionally configure other For multi-user or higher-throughput deployments, **PostgreSQL** (recommended) or
database-related environment variables. **MariaDB** can be used instead by setting [`PAPERLESS_DBENGINE`](#PAPERLESS_DBENGINE)
and the relevant connection variables.
#### [`PAPERLESS_DBENGINE=<engine>`](#PAPERLESS_DBENGINE) {#PAPERLESS_DBENGINE}
: Specifies the database engine to use. Accepted values are `sqlite`, `postgresql`,
and `mariadb`.
Defaults to `sqlite` if not set.
PostgreSQL and MariaDB both require [`PAPERLESS_DBHOST`](#PAPERLESS_DBHOST) to be
set. SQLite does not use any other connection variables; the database file is always
located at `<PAPERLESS_DATA_DIR>/db.sqlite3`.
!!! warning
Using MariaDB comes with some caveats.
See [MySQL Caveats](advanced_usage.md#mysql-caveats).
#### [`PAPERLESS_DBHOST=<hostname>`](#PAPERLESS_DBHOST) {#PAPERLESS_DBHOST} #### [`PAPERLESS_DBHOST=<hostname>`](#PAPERLESS_DBHOST) {#PAPERLESS_DBHOST}
: If unset, Paperless uses **SQLite** by default. : Hostname of the PostgreSQL or MariaDB database server. Required when
`PAPERLESS_DBENGINE` is `postgresql` or `mariadb`.
Set `PAPERLESS_DBHOST` to switch to PostgreSQL or MariaDB instead.
#### [`PAPERLESS_DBENGINE=<engine_name>`](#PAPERLESS_DBENGINE) {#PAPERLESS_DBENGINE}
: Optional. Specifies the database engine to use when connecting to a remote database.
Available options are `postgresql` and `mariadb`.
Defaults to `postgresql` if `PAPERLESS_DBHOST` is set.
!!! warning
Using MariaDB comes with some caveats. See [MySQL Caveats](advanced_usage.md#mysql-caveats).
#### [`PAPERLESS_DBPORT=<port>`](#PAPERLESS_DBPORT) {#PAPERLESS_DBPORT} #### [`PAPERLESS_DBPORT=<port>`](#PAPERLESS_DBPORT) {#PAPERLESS_DBPORT}
: Port to use when connecting to PostgreSQL or MariaDB. : Port to use when connecting to PostgreSQL or MariaDB.
Default is `5432` for PostgreSQL and `3306` for MariaDB. Defaults to `5432` for PostgreSQL and `3306` for MariaDB.
#### [`PAPERLESS_DBNAME=<name>`](#PAPERLESS_DBNAME) {#PAPERLESS_DBNAME} #### [`PAPERLESS_DBNAME=<name>`](#PAPERLESS_DBNAME) {#PAPERLESS_DBNAME}
: Name of the database to connect to when using PostgreSQL or MariaDB. : Name of the PostgreSQL or MariaDB database to connect to.
Defaults to "paperless". Defaults to `paperless`.
#### [`PAPERLESS_DBUSER=<name>`](#PAPERLESS_DBUSER) {#PAPERLESS_DBUSER} #### [`PAPERLESS_DBUSER=<user>`](#PAPERLESS_DBUSER) {#PAPERLESS_DBUSER}
: Username for authenticating with the PostgreSQL or MariaDB database. : Username for authenticating with the PostgreSQL or MariaDB database.
Defaults to "paperless". Defaults to `paperless`.
#### [`PAPERLESS_DBPASS=<password>`](#PAPERLESS_DBPASS) {#PAPERLESS_DBPASS} #### [`PAPERLESS_DBPASS=<password>`](#PAPERLESS_DBPASS) {#PAPERLESS_DBPASS}
: Password for the PostgreSQL or MariaDB database user. : Password for the PostgreSQL or MariaDB database user.
Defaults to "paperless". Defaults to `paperless`.
#### [`PAPERLESS_DBSSLMODE=<mode>`](#PAPERLESS_DBSSLMODE) {#PAPERLESS_DBSSLMODE} #### [`PAPERLESS_DB_OPTIONS=<options>`](#PAPERLESS_DB_OPTIONS) {#PAPERLESS_DB_OPTIONS}
: SSL mode to use when connecting to PostgreSQL or MariaDB. : Advanced database connection options as a semicolon-delimited key-value string.
Keys and values are separated by `=`. Dot-notation produces nested option
dictionaries; for example, `pool.max_size=20` sets
`OPTIONS["pool"]["max_size"] = 20`.
See [the official documentation about Options specified here are merged over the engine defaults. Unrecognised keys
sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html). are passed through to the underlying database driver without validation, so a
typo will be silently ignored rather than producing an error.
See [the official documentation about Refer to your database driver's documentation for the full set of accepted keys:
sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-mode).
*Note*: SSL mode values differ between PostgreSQL and MariaDB. - PostgreSQL: [libpq connection parameters](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS)
- MariaDB: [MariaDB Connector/Python](https://mariadb.com/kb/en/mariadb-connector-python/)
- SQLite: [SQLite PRAGMA statements](https://www.sqlite.org/pragma.html)
Default is `prefer` for PostgreSQL and `PREFERRED` for MariaDB. **Examples:**
#### [`PAPERLESS_DBSSLROOTCERT=<ca-path>`](#PAPERLESS_DBSSLROOTCERT) {#PAPERLESS_DBSSLROOTCERT} ```bash
# PostgreSQL: require SSL, set a custom CA certificate, and limit the pool size
PAPERLESS_DB_OPTIONS="sslmode=require;sslrootcert=/certs/ca.pem;pool.max_size=5"
: Path to the SSL root certificate used to verify the database server. # MariaDB: require SSL with a custom CA certificate
PAPERLESS_DB_OPTIONS="ssl_mode=REQUIRED;ssl.ca=/certs/ca.pem"
See [the official documentation about # PostgreSQL: set a connection timeout
sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html). PAPERLESS_DB_OPTIONS="connect_timeout=10"
Changes the location of `root.crt`. ```
See [the official documentation about !!! note "PostgreSQL connection pooling"
sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-ca). Pool size is controlled via `pool.min_size` and `pool.max_size`. When
configuring pooling, ensure your PostgreSQL `max_connections` is large enough
to handle all pool connections across all workers:
`(web_workers + celery_workers) * pool.max_size + safety_margin`.
Defaults to unset, using the standard location in the home directory. #### ~~[`PAPERLESS_DBSSLMODE`](#PAPERLESS_DBSSLMODE)~~ {#PAPERLESS_DBSSLMODE}
#### [`PAPERLESS_DBSSLCERT=<client-cert-path>`](#PAPERLESS_DBSSLCERT) {#PAPERLESS_DBSSLCERT} !!! failure "Removed in v3"
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
: Path to the client SSL certificate used when connecting securely. ```bash
# PostgreSQL
PAPERLESS_DB_OPTIONS="sslmode=require"
See [the official documentation about # MariaDB
sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html). PAPERLESS_DB_OPTIONS="ssl_mode=REQUIRED"
```
See [the official documentation about #### ~~[`PAPERLESS_DBSSLROOTCERT`](#PAPERLESS_DBSSLROOTCERT)~~ {#PAPERLESS_DBSSLROOTCERT}
sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-cert).
Changes the location of `postgresql.crt`. !!! failure "Removed in v3"
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
Defaults to unset, using the standard location in the home directory. ```bash
# PostgreSQL
PAPERLESS_DB_OPTIONS="sslrootcert=/path/to/ca.pem"
#### [`PAPERLESS_DBSSLKEY=<client-cert-key>`](#PAPERLESS_DBSSLKEY) {#PAPERLESS_DBSSLKEY} # MariaDB
PAPERLESS_DB_OPTIONS="ssl.ca=/path/to/ca.pem"
```
: Path to the client SSL private key used when connecting securely. #### ~~[`PAPERLESS_DBSSLCERT`](#PAPERLESS_DBSSLCERT)~~ {#PAPERLESS_DBSSLCERT}
See [the official documentation about !!! failure "Removed in v3"
sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html). Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
See [the official documentation about ```bash
sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-key). # PostgreSQL
PAPERLESS_DB_OPTIONS="sslcert=/path/to/client.crt"
Changes the location of `postgresql.key`. # MariaDB
PAPERLESS_DB_OPTIONS="ssl.cert=/path/to/client.crt"
```
Defaults to unset, using the standard location in the home directory. #### ~~[`PAPERLESS_DBSSLKEY`](#PAPERLESS_DBSSLKEY)~~ {#PAPERLESS_DBSSLKEY}
#### [`PAPERLESS_DB_TIMEOUT=<int>`](#PAPERLESS_DB_TIMEOUT) {#PAPERLESS_DB_TIMEOUT} !!! failure "Removed in v3"
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
: Sets how long a database connection should wait before timing out. ```bash
# PostgreSQL
PAPERLESS_DB_OPTIONS="sslkey=/path/to/client.key"
For SQLite, this sets how long to wait if the database is locked. # MariaDB
For PostgreSQL or MariaDB, this sets the connection timeout. PAPERLESS_DB_OPTIONS="ssl.key=/path/to/client.key"
```
Defaults to unset, which uses Djangos built-in defaults. #### ~~[`PAPERLESS_DB_TIMEOUT`](#PAPERLESS_DB_TIMEOUT)~~ {#PAPERLESS_DB_TIMEOUT}
#### [`PAPERLESS_DB_POOLSIZE=<int>`](#PAPERLESS_DB_POOLSIZE) {#PAPERLESS_DB_POOLSIZE} !!! failure "Removed in v3"
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
: Defines the maximum number of database connections to keep in the pool. ```bash
# SQLite
PAPERLESS_DB_OPTIONS="timeout=30"
Only applies to PostgreSQL. This setting is ignored for other database engines. # PostgreSQL or MariaDB
PAPERLESS_DB_OPTIONS="connect_timeout=30"
```
The value must be greater than or equal to 1 to be used. #### ~~[`PAPERLESS_DB_POOLSIZE`](#PAPERLESS_DB_POOLSIZE)~~ {#PAPERLESS_DB_POOLSIZE}
Defaults to unset, which disables connection pooling.
!!! note !!! failure "Removed in v3"
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
A pool of 8-10 connections per worker is typically sufficient. ```bash
If you encounter error messages such as `couldn't get a connection` PAPERLESS_DB_OPTIONS="pool.max_size=10"
or database connection timeouts, you probably need to increase the pool size. ```
!!! warning
Make sure your PostgreSQL `max_connections` setting is large enough to handle the connection pools:
`(NB_PAPERLESS_WORKERS + NB_CELERY_WORKERS) × POOL_SIZE + SAFETY_MARGIN`. For example, with
4 Paperless workers and 2 Celery workers, and a pool size of 8:``(4 + 2) × 8 + 10 = 58`,
so `max_connections = 60` (or even more) is appropriate.
This assumes only Paperless-ngx connects to your PostgreSQL instance. If you have other applications,
you should increase `max_connections` accordingly.
#### [`PAPERLESS_DB_READ_CACHE_ENABLED=<bool>`](#PAPERLESS_DB_READ_CACHE_ENABLED) {#PAPERLESS_DB_READ_CACHE_ENABLED} #### [`PAPERLESS_DB_READ_CACHE_ENABLED=<bool>`](#PAPERLESS_DB_READ_CACHE_ENABLED) {#PAPERLESS_DB_READ_CACHE_ENABLED}

View File

@@ -48,3 +48,58 @@ The `CONSUMER_BARCODE_SCANNER` setting has been removed. zxing-cpp is now the on
reliability. reliability.
- The `libzbar0` / `libzbar-dev` system packages are no longer required and can be removed from any custom Docker - The `libzbar0` / `libzbar-dev` system packages are no longer required and can be removed from any custom Docker
images or host installations. images or host installations.
## Database Engine
`PAPERLESS_DBENGINE` is now required to use PostgreSQL or MariaDB. Previously, the
engine was inferred from the presence of `PAPERLESS_DBHOST`, with `PAPERLESS_DBENGINE`
only needed to select MariaDB over PostgreSQL.
SQLite users require no changes, though they may explicitly set their engine if desired.
#### Action Required
PostgreSQL and MariaDB users must add `PAPERLESS_DBENGINE` to their environment:
```yaml
# v2 (PostgreSQL inferred from PAPERLESS_DBHOST)
PAPERLESS_DBHOST: postgres
# v3 (engine must be explicit)
PAPERLESS_DBENGINE: postgresql
PAPERLESS_DBHOST: postgres
```
See [`PAPERLESS_DBENGINE`](configuration.md#PAPERLESS_DBENGINE) for accepted values.
## Database Advanced Options
The individual SSL, timeout, and pooling variables have been removed in favour of a
single [`PAPERLESS_DB_OPTIONS`](configuration.md#PAPERLESS_DB_OPTIONS) string. This
consolidates a growing set of engine-specific variables into one place, and allows
any option supported by the underlying database driver to be set without requiring a
dedicated environment variable for each.
The removed variables and their replacements are:
| Removed Variable | Replacement in `PAPERLESS_DB_OPTIONS` |
| ------------------------- | ---------------------------------------------------------------------------- |
| `PAPERLESS_DBSSLMODE` | `sslmode=<value>` (PostgreSQL) or `ssl_mode=<value>` (MariaDB) |
| `PAPERLESS_DBSSLROOTCERT` | `sslrootcert=<path>` (PostgreSQL) or `ssl.ca=<path>` (MariaDB) |
| `PAPERLESS_DBSSLCERT` | `sslcert=<path>` (PostgreSQL) or `ssl.cert=<path>` (MariaDB) |
| `PAPERLESS_DBSSLKEY` | `sslkey=<path>` (PostgreSQL) or `ssl.key=<path>` (MariaDB) |
| `PAPERLESS_DB_POOLSIZE` | `pool.max_size=<value>` (PostgreSQL only) |
| `PAPERLESS_DB_TIMEOUT` | `timeout=<value>` (SQLite) or `connect_timeout=<value>` (PostgreSQL/MariaDB) |
The deprecated variables will continue to function for now but will be removed in a
future release. A deprecation warning is logged at startup for each deprecated variable
that is still set.
#### Action Required
Users with any of the deprecated variables set should migrate to `PAPERLESS_DB_OPTIONS`.
Multiple options are combined in a single value:
```bash
PAPERLESS_DB_OPTIONS="sslmode=require;sslrootcert=/certs/ca.pem;pool.max_size=10"
```

View File

@@ -8490,7 +8490,7 @@
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">315</context> <context context-type="linenumber">323</context>
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context> <context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context>
@@ -8505,7 +8505,7 @@
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">308</context> <context context-type="linenumber">316</context>
</context-group> </context-group>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context> <context context-type="sourcefile">src/app/components/manage/document-attributes/document-attributes.component.html</context>
@@ -8771,49 +8771,49 @@
<source>Reset filters / selection</source> <source>Reset filters / selection</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">296</context> <context context-type="linenumber">304</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="4135055128446167640" datatype="html"> <trans-unit id="4135055128446167640" datatype="html">
<source>Open first [selected] document</source> <source>Open first [selected] document</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">324</context> <context context-type="linenumber">332</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="3629960544875360046" datatype="html"> <trans-unit id="3629960544875360046" datatype="html">
<source>Previous page</source> <source>Previous page</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">340</context> <context context-type="linenumber">348</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="3337301694210287595" datatype="html"> <trans-unit id="3337301694210287595" datatype="html">
<source>Next page</source> <source>Next page</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">352</context> <context context-type="linenumber">360</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="2155249406916744630" datatype="html"> <trans-unit id="2155249406916744630" datatype="html">
<source>View &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot; saved successfully.</source> <source>View &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot; saved successfully.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">385</context> <context context-type="linenumber">393</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="4646273665293421938" datatype="html"> <trans-unit id="4646273665293421938" datatype="html">
<source>Failed to save view &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot;.</source> <source>Failed to save view &quot;<x id="PH" equiv-text="this.list.activeSavedViewTitle"/>&quot;.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">391</context> <context context-type="linenumber">399</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="6837554170707123455" datatype="html"> <trans-unit id="6837554170707123455" datatype="html">
<source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully.</source> <source>View &quot;<x id="PH" equiv-text="savedView.name"/>&quot; created successfully.</source>
<context-group purpose="location"> <context-group purpose="location">
<context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context> <context context-type="sourcefile">src/app/components/document-list/document-list.component.ts</context>
<context context-type="linenumber">437</context> <context context-type="linenumber">445</context>
</context-group> </context-group>
</trans-unit> </trans-unit>
<trans-unit id="739880801667335279" datatype="html"> <trans-unit id="739880801667335279" datatype="html">

View File

@@ -117,7 +117,7 @@
</pngx-page-header> </pngx-page-header>
<div class="row sticky-top py-3 mt-n2 mt-md-n3 bg-body"> <div class="row sticky-top py-3 mt-n2 mt-md-n3 bg-body">
<pngx-filter-editor [hidden]="isBulkEditing" [disabled]="isBulkEditing" [(filterRules)]="list.filterRules" [unmodifiedFilterRules]="unmodifiedFilterRules" [selectionData]="list.selectionData" #filterEditor></pngx-filter-editor> <pngx-filter-editor [hidden]="isBulkEditing" [disabled]="isBulkEditing" [filterRules]="list.filterRules" (filterRulesChange)="onFilterRulesChange($event)" (resetFilterRules)="onFilterRulesReset($event)" [unmodifiedFilterRules]="unmodifiedFilterRules" [selectionData]="list.selectionData" #filterEditor></pngx-filter-editor>
<pngx-bulk-editor [hidden]="!isBulkEditing" [disabled]="!isBulkEditing"></pngx-bulk-editor> <pngx-bulk-editor [hidden]="!isBulkEditing" [disabled]="!isBulkEditing"></pngx-bulk-editor>
</div> </div>

View File

@@ -147,21 +147,21 @@ describe('DocumentListComponent', () => {
}) })
it('should show score sort fields on fulltext queries', () => { it('should show score sort fields on fulltext queries', () => {
documentListService.filterRules = [ documentListService.setFilterRules([
{ {
rule_type: FILTER_HAS_TAGS_ANY, rule_type: FILTER_HAS_TAGS_ANY,
value: '10', value: '10',
}, },
] ])
fixture.detectChanges() fixture.detectChanges()
expect(component.getSortFields()).toEqual(documentListService.sortFields) expect(component.getSortFields()).toEqual(documentListService.sortFields)
documentListService.filterRules = [ documentListService.setFilterRules([
{ {
rule_type: FILTER_FULLTEXT_QUERY, rule_type: FILTER_FULLTEXT_QUERY,
value: 'foo', value: 'foo',
}, },
] ])
fixture.detectChanges() fixture.detectChanges()
expect(component.getSortFields()).toEqual( expect(component.getSortFields()).toEqual(
documentListService.sortFieldsFullText documentListService.sortFieldsFullText
@@ -170,12 +170,12 @@ describe('DocumentListComponent', () => {
it('should determine if filtered, support reset', () => { it('should determine if filtered, support reset', () => {
fixture.detectChanges() fixture.detectChanges()
documentListService.filterRules = [ documentListService.setFilterRules([
{ {
rule_type: FILTER_HAS_TAGS_ANY, rule_type: FILTER_HAS_TAGS_ANY,
value: '10', value: '10',
}, },
] ])
documentListService.isReloading = false documentListService.isReloading = false
fixture.detectChanges() fixture.detectChanges()
expect(component.isFiltered).toBeTruthy() expect(component.isFiltered).toBeTruthy()
@@ -185,6 +185,20 @@ describe('DocumentListComponent', () => {
expect(fixture.nativeElement.textContent.match(/Reset/g)).toHaveLength(1) expect(fixture.nativeElement.textContent.match(/Reset/g)).toHaveLength(1)
}) })
it('should apply filter rule changes via list service', () => {
const setFilterRulesSpy = jest.spyOn(documentListService, 'setFilterRules')
const rules = [{ rule_type: FILTER_HAS_TAGS_ANY, value: '10' }]
component.onFilterRulesChange(rules)
expect(setFilterRulesSpy).toHaveBeenCalledWith(rules)
})
it('should reset filter rules to page one via list service', () => {
const setFilterRulesSpy = jest.spyOn(documentListService, 'setFilterRules')
const rules = [{ rule_type: FILTER_HAS_TAGS_ANY, value: '10' }]
component.onFilterRulesReset(rules)
expect(setFilterRulesSpy).toHaveBeenCalledWith(rules, true)
})
it('should load saved view from URL', () => { it('should load saved view from URL', () => {
const view: SavedView = { const view: SavedView = {
id: 10, id: 10,
@@ -217,7 +231,7 @@ describe('DocumentListComponent', () => {
.spyOn(activatedRoute, 'paramMap', 'get') .spyOn(activatedRoute, 'paramMap', 'get')
.mockReturnValue(of(convertToParamMap(queryParams))) .mockReturnValue(of(convertToParamMap(queryParams)))
activatedRoute.snapshot.queryParams = queryParams activatedRoute.snapshot.queryParams = queryParams
fixture.detectChanges() component.ngOnInit()
expect(getSavedViewSpy).toHaveBeenCalledWith(view.id) expect(getSavedViewSpy).toHaveBeenCalledWith(view.id)
expect(activateSavedViewSpy).toHaveBeenCalledWith( expect(activateSavedViewSpy).toHaveBeenCalledWith(
view, view,

View File

@@ -212,6 +212,14 @@ export class DocumentListComponent
this.list.setSort(event.column, event.reverse) this.list.setSort(event.column, event.reverse)
} }
onFilterRulesChange(filterRules: FilterRule[]) {
this.list.setFilterRules(filterRules)
}
onFilterRulesReset(filterRules: FilterRule[]) {
this.list.setFilterRules(filterRules, true)
}
get isBulkEditing(): boolean { get isBulkEditing(): boolean {
return this.list.selected.size > 0 return this.list.selected.size > 0
} }
@@ -300,7 +308,7 @@ export class DocumentListComponent
if (this.list.selected.size > 0) { if (this.list.selected.size > 0) {
this.list.selectNone() this.list.selectNone()
} else if (this.isFiltered) { } else if (this.isFiltered) {
this.filterEditor.resetSelected() this.resetFilters()
} }
}) })

View File

@@ -2107,6 +2107,22 @@ describe('FilterEditorComponent', () => {
expect(component.filterRules).toEqual(rules) expect(component.filterRules).toEqual(rules)
}) })
it('should emit reset filter rules when resetting', () => {
const rules = [{ rule_type: FILTER_HAS_TAGS_ANY, value: '2' }]
component.unmodifiedFilterRules = rules
component.filterRules = [
{ rule_type: FILTER_DOES_NOT_HAVE_TAG, value: '2' },
]
const resetFilterRulesSpy = jest.spyOn(component.resetFilterRules, 'next')
const filterRulesChangeSpy = jest.spyOn(component.filterRulesChange, 'next')
component.resetSelected()
expect(resetFilterRulesSpy).toHaveBeenCalledWith(rules)
expect(filterRulesChangeSpy).not.toHaveBeenCalled()
})
it('should support resetting text field', () => { it('should support resetting text field', () => {
component.textFilter = 'foo' component.textFilter = 'foo'
component.resetTextField() component.resetTextField()

View File

@@ -1101,6 +1101,9 @@ export class FilterEditorComponent
@Output() @Output()
filterRulesChange = new EventEmitter<FilterRule[]>() filterRulesChange = new EventEmitter<FilterRule[]>()
@Output()
resetFilterRules = new EventEmitter<FilterRule[]>()
@Input() @Input()
set selectionData(selectionData: SelectionData) { set selectionData(selectionData: SelectionData) {
this.tagDocumentCounts = selectionData?.selected_tags ?? null this.tagDocumentCounts = selectionData?.selected_tags ?? null
@@ -1244,7 +1247,7 @@ export class FilterEditorComponent
this.textFilterTarget = TEXT_FILTER_TARGET_TITLE_CONTENT this.textFilterTarget = TEXT_FILTER_TARGET_TITLE_CONTENT
this.documentService.searchQuery = '' this.documentService.searchQuery = ''
this.filterRules = this._unmodifiedFilterRules this.filterRules = this._unmodifiedFilterRules
this.updateRules() this.resetFilterRules.next(this.filterRules)
} }
toggleTag(tagId: number) { toggleTag(tagId: number) {

View File

@@ -164,7 +164,7 @@ describe('DocumentListViewService', () => {
value: tags__id__in, value: tags__id__in,
}, },
] ]
documentListViewService.filterRules = filterRulesAny documentListViewService.setFilterRules(filterRulesAny)
let req = httpTestingController.expectOne( let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__in=${tags__id__in}` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__in=${tags__id__in}`
) )
@@ -178,7 +178,7 @@ describe('DocumentListViewService', () => {
) )
expect(req.request.method).toEqual('GET') expect(req.request.method).toEqual('GET')
// reset the list // reset the list
documentListViewService.filterRules = [] documentListViewService.setFilterRules([])
req = httpTestingController.expectOne( req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
) )
@@ -210,7 +210,7 @@ describe('DocumentListViewService', () => {
value: tags__id__in, value: tags__id__in,
}, },
] ]
documentListViewService.filterRules = filterRulesAny documentListViewService.setFilterRules(filterRulesAny)
let req = httpTestingController.expectOne( let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__in=${tags__id__in}` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__in=${tags__id__in}`
) )
@@ -218,7 +218,7 @@ describe('DocumentListViewService', () => {
req.flush('Generic error', { status: 404, statusText: 'Unexpected error' }) req.flush('Generic error', { status: 404, statusText: 'Unexpected error' })
expect(documentListViewService.error).toEqual('Generic error') expect(documentListViewService.error).toEqual('Generic error')
// reset the list // reset the list
documentListViewService.filterRules = [] documentListViewService.setFilterRules([])
req = httpTestingController.expectOne( req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
) )
@@ -295,13 +295,41 @@ describe('DocumentListViewService', () => {
}) })
it('should use filter rules to update query params', () => { it('should use filter rules to update query params', () => {
documentListViewService.filterRules = filterRules documentListViewService.setFilterRules(filterRules)
const req = httpTestingController.expectOne( const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&tags__id__all=${tags__id__all}` `${environment.apiBaseUrl}documents/?page=${documentListViewService.currentPage}&page_size=${documentListViewService.pageSize}&ordering=-created&truncate_content=true&tags__id__all=${tags__id__all}`
) )
expect(req.request.method).toEqual('GET') expect(req.request.method).toEqual('GET')
}) })
it('should support setting filter rules and resetting to page one', () => {
documentListViewService.currentPage = 2
let req = httpTestingController.expectOne((request) =>
request.urlWithParams.startsWith(
`${environment.apiBaseUrl}documents/?page=2&page_size=50&ordering=-created&truncate_content=true`
)
)
expect(req.request.method).toEqual('GET')
req.flush(full_results)
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/selection_data/`
)
req.flush([])
documentListViewService.setFilterRules(filterRules, true)
const filteredReqs = httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__all=${tags__id__all}`
)
expect(filteredReqs).toHaveLength(1)
filteredReqs[0].flush(full_results)
req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/selection_data/`
)
req.flush([])
expect(documentListViewService.currentPage).toEqual(1)
})
it('should support quick filter', () => { it('should support quick filter', () => {
documentListViewService.quickFilter(filterRules) documentListViewService.quickFilter(filterRules)
const req = httpTestingController.expectOne( const req = httpTestingController.expectOne(
@@ -336,7 +364,7 @@ describe('DocumentListViewService', () => {
req = httpTestingController.expectOne( req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true&tags__id__all=9` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true&tags__id__all=9`
) )
documentListViewService.filterRules = [] documentListViewService.setFilterRules([])
req = httpTestingController.expectOne( req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-added&truncate_content=true`
) )
@@ -348,7 +376,7 @@ describe('DocumentListViewService', () => {
}) })
it('should support navigating next / previous', () => { it('should support navigating next / previous', () => {
documentListViewService.filterRules = [] documentListViewService.setFilterRules([])
let req = httpTestingController.expectOne( let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`
) )
@@ -558,7 +586,7 @@ describe('DocumentListViewService', () => {
req.flush(full_results) req.flush(full_results)
expect(documentListViewService.selected.size).toEqual(6) expect(documentListViewService.selected.size).toEqual(6)
documentListViewService.filterRules = filterRules documentListViewService.setFilterRules(filterRules)
httpTestingController.expectOne( httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__all=9` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&tags__id__all=9`
) )
@@ -592,7 +620,7 @@ describe('DocumentListViewService', () => {
documentListViewService.loadSavedView(view2) documentListViewService.loadSavedView(view2)
expect(documentListViewService.sortField).toEqual('score') expect(documentListViewService.sortField).toEqual('score')
documentListViewService.filterRules = [] documentListViewService.setFilterRules([])
expect(documentListViewService.sortField).toEqual('created') expect(documentListViewService.sortField).toEqual('created')
httpTestingController.expectOne( httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true` `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true`

View File

@@ -342,7 +342,7 @@ export class DocumentListViewService {
}) })
} }
set filterRules(filterRules: FilterRule[]) { setFilterRules(filterRules: FilterRule[], resetPage: boolean = false) {
if ( if (
!isFullTextFilterRule(filterRules) && !isFullTextFilterRule(filterRules) &&
this.activeListViewState.sortField == 'score' this.activeListViewState.sortField == 'score'
@@ -350,6 +350,9 @@ export class DocumentListViewService {
this.activeListViewState.sortField = 'created' this.activeListViewState.sortField = 'created'
} }
this.activeListViewState.filterRules = filterRules this.activeListViewState.filterRules = filterRules
if (resetPage) {
this.activeListViewState.currentPage = 1
}
this.reload() this.reload()
this.reduceSelectionToFilter() this.reduceSelectionToFilter()
this.saveDocumentListView() this.saveDocumentListView()
@@ -479,7 +482,7 @@ export class DocumentListViewService {
quickFilter(filterRules: FilterRule[]) { quickFilter(filterRules: FilterRule[]) {
this._activeSavedViewId = null this._activeSavedViewId = null
this.filterRules = filterRules this.setFilterRules(filterRules)
this.router.navigate(['documents']) this.router.navigate(['documents'])
} }

View File

@@ -202,3 +202,43 @@ def audit_log_check(app_configs, **kwargs):
) )
return result return result
@register()
def check_deprecated_db_settings(
app_configs: object,
**kwargs: object,
) -> list[Warning]:
"""Check for deprecated database environment variables.
Detects legacy advanced options that should be migrated to
PAPERLESS_DB_OPTIONS. Returns one Warning per deprecated variable found.
"""
deprecated_vars: dict[str, str] = {
"PAPERLESS_DB_TIMEOUT": "timeout",
"PAPERLESS_DB_POOLSIZE": "pool.min_size / pool.max_size",
"PAPERLESS_DBSSLMODE": "sslmode",
"PAPERLESS_DBSSLROOTCERT": "sslrootcert",
"PAPERLESS_DBSSLCERT": "sslcert",
"PAPERLESS_DBSSLKEY": "sslkey",
}
warnings: list[Warning] = []
for var_name, db_option_key in deprecated_vars.items():
if not os.getenv(var_name):
continue
warnings.append(
Warning(
f"Deprecated environment variable: {var_name}",
hint=(
f"{var_name} is no longer supported and will be removed in v3.2. "
f"Set the equivalent option via PAPERLESS_DB_OPTIONS instead. "
f'Example: PAPERLESS_DB_OPTIONS=\'{{"{db_option_key}": "<value>"}}\'. '
"See https://docs.paperless-ngx.com/migration/ for the full reference."
),
id="paperless.W001",
),
)
return warnings

View File

@@ -17,6 +17,8 @@ from dateparser.languages.loader import LocaleDataLoader
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
from dotenv import load_dotenv from dotenv import load_dotenv
from paperless.settings.custom import parse_db_settings
logger = logging.getLogger("paperless.settings") logger = logging.getLogger("paperless.settings")
# Tap paperless.conf if it's available # Tap paperless.conf if it's available
@@ -722,83 +724,8 @@ EMAIL_CERTIFICATE_FILE = __get_optional_path("PAPERLESS_EMAIL_CERTIFICATE_LOCATI
############################################################################### ###############################################################################
# Database # # Database #
############################################################################### ###############################################################################
def _parse_db_settings() -> dict:
databases = {
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": DATA_DIR / "db.sqlite3",
"OPTIONS": {},
},
}
if os.getenv("PAPERLESS_DBHOST"):
# Have sqlite available as a second option for management commands
# This is important when migrating to/from sqlite
databases["sqlite"] = databases["default"].copy()
databases["default"] = { DATABASES = parse_db_settings(DATA_DIR)
"HOST": os.getenv("PAPERLESS_DBHOST"),
"NAME": os.getenv("PAPERLESS_DBNAME", "paperless"),
"USER": os.getenv("PAPERLESS_DBUSER", "paperless"),
"PASSWORD": os.getenv("PAPERLESS_DBPASS", "paperless"),
"OPTIONS": {},
}
if os.getenv("PAPERLESS_DBPORT"):
databases["default"]["PORT"] = os.getenv("PAPERLESS_DBPORT")
# Leave room for future extensibility
if os.getenv("PAPERLESS_DBENGINE") == "mariadb":
engine = "django.db.backends.mysql"
# Contrary to Postgres, Django does not natively support connection pooling for MariaDB.
# However, since MariaDB uses threads instead of forks, establishing connections is significantly faster
# compared to PostgreSQL, so the lack of pooling is not an issue
options = {
"read_default_file": "/etc/mysql/my.cnf",
"charset": "utf8mb4",
"ssl_mode": os.getenv("PAPERLESS_DBSSLMODE", "PREFERRED"),
"ssl": {
"ca": os.getenv("PAPERLESS_DBSSLROOTCERT", None),
"cert": os.getenv("PAPERLESS_DBSSLCERT", None),
"key": os.getenv("PAPERLESS_DBSSLKEY", None),
},
}
else: # Default to PostgresDB
engine = "django.db.backends.postgresql"
options = {
"sslmode": os.getenv("PAPERLESS_DBSSLMODE", "prefer"),
"sslrootcert": os.getenv("PAPERLESS_DBSSLROOTCERT", None),
"sslcert": os.getenv("PAPERLESS_DBSSLCERT", None),
"sslkey": os.getenv("PAPERLESS_DBSSLKEY", None),
}
if int(os.getenv("PAPERLESS_DB_POOLSIZE", 0)) > 0:
options.update(
{
"pool": {
"min_size": 1,
"max_size": int(os.getenv("PAPERLESS_DB_POOLSIZE")),
},
},
)
databases["default"]["ENGINE"] = engine
databases["default"]["OPTIONS"].update(options)
if os.getenv("PAPERLESS_DB_TIMEOUT") is not None:
if databases["default"]["ENGINE"] == "django.db.backends.sqlite3":
databases["default"]["OPTIONS"].update(
{"timeout": int(os.getenv("PAPERLESS_DB_TIMEOUT"))},
)
else:
databases["default"]["OPTIONS"].update(
{"connect_timeout": int(os.getenv("PAPERLESS_DB_TIMEOUT"))},
)
databases["sqlite"]["OPTIONS"].update(
{"timeout": int(os.getenv("PAPERLESS_DB_TIMEOUT"))},
)
return databases
DATABASES = _parse_db_settings()
if os.getenv("PAPERLESS_DBENGINE") == "mariadb": if os.getenv("PAPERLESS_DBENGINE") == "mariadb":
# Silence Django error on old MariaDB versions. # Silence Django error on old MariaDB versions.

View File

@@ -0,0 +1,128 @@
import os
from pathlib import Path
from typing import Any
from paperless.settings.parsers import get_choice_from_env
from paperless.settings.parsers import get_int_from_env
from paperless.settings.parsers import parse_dict_from_str
def parse_db_settings(data_dir: Path) -> dict[str, dict[str, Any]]:
"""Parse database settings from environment variables.
Core connection variables (no deprecation):
- PAPERLESS_DBENGINE (sqlite/postgresql/mariadb)
- PAPERLESS_DBHOST, PAPERLESS_DBPORT
- PAPERLESS_DBNAME, PAPERLESS_DBUSER, PAPERLESS_DBPASS
Advanced options can be set via:
- Legacy individual env vars (deprecated in v3.0, removed in v3.2)
- PAPERLESS_DB_OPTIONS (recommended v3+ approach)
Args:
data_dir: The data directory path for SQLite database location.
Returns:
A databases dict suitable for Django DATABASES setting.
"""
engine = get_choice_from_env(
"PAPERLESS_DBENGINE",
{"sqlite", "postgresql", "mariadb"},
default="sqlite",
)
db_config: dict[str, Any]
base_options: dict[str, Any]
match engine:
case "sqlite":
db_config = {
"ENGINE": "django.db.backends.sqlite3",
"NAME": str((data_dir / "db.sqlite3").resolve()),
}
base_options = {}
case "postgresql":
db_config = {
"ENGINE": "django.db.backends.postgresql",
"HOST": os.getenv("PAPERLESS_DBHOST"),
"NAME": os.getenv("PAPERLESS_DBNAME", "paperless"),
"USER": os.getenv("PAPERLESS_DBUSER", "paperless"),
"PASSWORD": os.getenv("PAPERLESS_DBPASS", "paperless"),
}
base_options = {
"sslmode": os.getenv("PAPERLESS_DBSSLMODE", "prefer"),
"sslrootcert": os.getenv("PAPERLESS_DBSSLROOTCERT"),
"sslcert": os.getenv("PAPERLESS_DBSSLCERT"),
"sslkey": os.getenv("PAPERLESS_DBSSLKEY"),
}
if (pool_size := get_int_from_env("PAPERLESS_DB_POOLSIZE")) is not None:
base_options["pool"] = {
"min_size": 1,
"max_size": pool_size,
}
case "mariadb":
db_config = {
"ENGINE": "django.db.backends.mysql",
"HOST": os.getenv("PAPERLESS_DBHOST"),
"NAME": os.getenv("PAPERLESS_DBNAME", "paperless"),
"USER": os.getenv("PAPERLESS_DBUSER", "paperless"),
"PASSWORD": os.getenv("PAPERLESS_DBPASS", "paperless"),
}
base_options = {
"read_default_file": "/etc/mysql/my.cnf",
"charset": "utf8mb4",
"collation": "utf8mb4_unicode_ci",
"ssl_mode": os.getenv("PAPERLESS_DBSSLMODE", "PREFERRED"),
"ssl": {
"ca": os.getenv("PAPERLESS_DBSSLROOTCERT"),
"cert": os.getenv("PAPERLESS_DBSSLCERT"),
"key": os.getenv("PAPERLESS_DBSSLKEY"),
},
}
case _: # pragma: no cover
raise NotImplementedError(engine)
# Handle port setting for external databases
if (
engine in ("postgresql", "mariadb")
and (port := get_int_from_env("PAPERLESS_DBPORT")) is not None
):
db_config["PORT"] = port
# Handle timeout setting (common across all engines, different key names)
if (timeout := get_int_from_env("PAPERLESS_DB_TIMEOUT")) is not None:
timeout_key = "timeout" if engine == "sqlite" else "connect_timeout"
base_options[timeout_key] = timeout
# Apply PAPERLESS_DB_OPTIONS overrides
db_config["OPTIONS"] = parse_dict_from_str(
os.getenv("PAPERLESS_DB_OPTIONS"),
defaults=base_options,
separator=";",
type_map={
# SQLite options
"timeout": int,
# Postgres/MariaDB options
"connect_timeout": int,
"pool.min_size": int,
"pool.max_size": int,
},
)
databases = {"default": db_config}
# Add SQLite fallback for PostgreSQL/MariaDB
# TODO: Is this really useful/used?
if engine in ("postgresql", "mariadb"):
databases["sqlite"] = {
"ENGINE": "django.db.backends.sqlite3",
"NAME": str((data_dir / "db.sqlite3").resolve()),
"OPTIONS": {},
}
return databases

View File

@@ -0,0 +1,192 @@
import copy
import os
from collections.abc import Callable
from collections.abc import Mapping
from pathlib import Path
from typing import Any
from typing import TypeVar
from typing import overload
T = TypeVar("T")
def str_to_bool(value: str) -> bool:
"""
Converts a string representation of truth to a boolean value.
Recognizes 'true', '1', 't', 'y', 'yes' as True, and
'false', '0', 'f', 'n', 'no' as False. Case-insensitive.
Args:
value: The string to convert.
Returns:
The boolean representation of the string.
Raises:
ValueError: If the string is not a recognized boolean value.
"""
val_lower = value.strip().lower()
if val_lower in ("true", "1", "t", "y", "yes"):
return True
elif val_lower in ("false", "0", "f", "n", "no"):
return False
raise ValueError(f"Cannot convert '{value}' to a boolean.")
@overload
def get_int_from_env(key: str) -> int | None: ...
@overload
def get_int_from_env(key: str, default: None) -> int | None: ...
@overload
def get_int_from_env(key: str, default: int) -> int: ...
def get_int_from_env(key: str, default: int | None = None) -> int | None:
"""
Return an integer value based on the environment variable.
If default is provided, returns that value when key is missing.
If default is None, returns None when key is missing.
"""
if key not in os.environ:
return default
return int(os.environ[key])
def parse_dict_from_str(
env_str: str | None,
defaults: dict[str, Any] | None = None,
type_map: Mapping[str, Callable[[str], Any]] | None = None,
separator: str = ",",
) -> dict[str, Any]:
"""
Parses a key-value string into a dictionary, applying defaults and casting types.
Supports nested keys via dot-notation, e.g.:
"database.host=localhost,database.port=5432"
Args:
env_str: The string from the environment variable (e.g., "port=9090,debug=true").
defaults: A dictionary of default values (can contain nested dicts).
type_map: A dictionary mapping keys (dot-notation allowed) to a type or a parsing
function (e.g., {'port': int, 'debug': bool, 'database.port': int}).
The special `bool` type triggers custom boolean parsing.
separator: The character used to separate key-value pairs. Defaults to ','.
Returns:
A dictionary with the parsed and correctly-typed settings.
Raises:
ValueError: If a value cannot be cast to its specified type.
"""
def _set_nested(d: dict, keys: list[str], value: Any) -> None:
"""Set a nested value, creating intermediate dicts as needed."""
cur = d
for k in keys[:-1]:
if k not in cur or not isinstance(cur[k], dict):
cur[k] = {}
cur = cur[k]
cur[keys[-1]] = value
def _get_nested(d: dict, keys: list[str]) -> Any:
"""Get nested value or raise KeyError if not present."""
cur = d
for k in keys:
if not isinstance(cur, dict) or k not in cur:
raise KeyError
cur = cur[k]
return cur
def _has_nested(d: dict, keys: list[str]) -> bool:
try:
_get_nested(d, keys)
return True
except KeyError:
return False
settings: dict[str, Any] = copy.deepcopy(defaults) if defaults else {}
_type_map = type_map if type_map else {}
if not env_str:
return settings
# Parse the environment string using the specified separator
pairs = [p.strip() for p in env_str.split(separator) if p.strip()]
for pair in pairs:
if "=" not in pair:
# ignore malformed pairs
continue
key, val = pair.split("=", 1)
key = key.strip()
val = val.strip()
if not key:
continue
parts = key.split(".")
_set_nested(settings, parts, val)
# Apply type casting to the updated settings (supports nested keys in type_map)
for key, caster in _type_map.items():
key_parts = key.split(".")
if _has_nested(settings, key_parts):
raw_val = _get_nested(settings, key_parts)
# Only cast if it's a string (i.e. from env parsing). If defaults already provided
# a different type we leave it as-is.
if isinstance(raw_val, str):
try:
if caster is bool:
parsed = str_to_bool(raw_val)
elif caster is Path:
parsed = Path(raw_val).resolve()
else:
parsed = caster(raw_val)
except (ValueError, TypeError) as e:
caster_name = getattr(caster, "__name__", repr(caster))
raise ValueError(
f"Error casting key '{key}' with value '{raw_val}' "
f"to type '{caster_name}'",
) from e
_set_nested(settings, key_parts, parsed)
return settings
def get_choice_from_env(
env_key: str,
choices: set[str],
default: str | None = None,
) -> str:
"""
Gets and validates an environment variable against a set of allowed choices.
Args:
env_key: The environment variable key to validate
choices: Set of valid choices for the environment variable
default: Optional default value if environment variable is not set
Returns:
The validated environment variable value
Raises:
ValueError: If the environment variable value is not in choices
or if no default is provided and env var is missing
"""
value = os.environ.get(env_key, default)
if value is None:
raise ValueError(
f"Environment variable '{env_key}' is required but not set.",
)
if value not in choices:
raise ValueError(
f"Environment variable '{env_key}' has invalid value '{value}'. "
f"Valid choices are: {', '.join(sorted(choices))}",
)
return value

View File

@@ -2,13 +2,17 @@ import os
from pathlib import Path from pathlib import Path
from unittest import mock from unittest import mock
import pytest
from django.core.checks import Warning
from django.test import TestCase from django.test import TestCase
from django.test import override_settings from django.test import override_settings
from pytest_mock import MockerFixture
from documents.tests.utils import DirectoriesMixin from documents.tests.utils import DirectoriesMixin
from documents.tests.utils import FileSystemAssertsMixin from documents.tests.utils import FileSystemAssertsMixin
from paperless.checks import audit_log_check from paperless.checks import audit_log_check
from paperless.checks import binaries_check from paperless.checks import binaries_check
from paperless.checks import check_deprecated_db_settings
from paperless.checks import debug_mode_check from paperless.checks import debug_mode_check
from paperless.checks import paths_check from paperless.checks import paths_check
from paperless.checks import settings_values_check from paperless.checks import settings_values_check
@@ -237,3 +241,157 @@ class TestAuditLogChecks(TestCase):
("auditlog table was found but audit log is disabled."), ("auditlog table was found but audit log is disabled."),
msg.msg, msg.msg,
) )
DEPRECATED_VARS: dict[str, str] = {
"PAPERLESS_DB_TIMEOUT": "timeout",
"PAPERLESS_DB_POOLSIZE": "pool.min_size / pool.max_size",
"PAPERLESS_DBSSLMODE": "sslmode",
"PAPERLESS_DBSSLROOTCERT": "sslrootcert",
"PAPERLESS_DBSSLCERT": "sslcert",
"PAPERLESS_DBSSLKEY": "sslkey",
}
class TestDeprecatedDbSettings:
"""Test suite for the check_deprecated_db_settings system check."""
def test_no_deprecated_vars_returns_empty(
self,
mocker: MockerFixture,
) -> None:
"""No warnings when none of the deprecated vars are present."""
# clear=True ensures vars from the outer test environment do not leak in
mocker.patch.dict(os.environ, {}, clear=True)
result = check_deprecated_db_settings(None)
assert result == []
@pytest.mark.parametrize(
("env_var", "db_option_key"),
[
("PAPERLESS_DB_TIMEOUT", "timeout"),
("PAPERLESS_DB_POOLSIZE", "pool.min_size / pool.max_size"),
("PAPERLESS_DBSSLMODE", "sslmode"),
("PAPERLESS_DBSSLROOTCERT", "sslrootcert"),
("PAPERLESS_DBSSLCERT", "sslcert"),
("PAPERLESS_DBSSLKEY", "sslkey"),
],
ids=[
"db-timeout",
"db-poolsize",
"ssl-mode",
"ssl-rootcert",
"ssl-cert",
"ssl-key",
],
)
def test_single_deprecated_var_produces_one_warning(
self,
mocker: MockerFixture,
env_var: str,
db_option_key: str,
) -> None:
"""Each deprecated var in isolation produces exactly one warning."""
mocker.patch.dict(os.environ, {env_var: "some_value"}, clear=True)
result = check_deprecated_db_settings(None)
assert len(result) == 1
warning = result[0]
assert isinstance(warning, Warning)
assert warning.id == "paperless.W001"
assert env_var in warning.hint
assert db_option_key in warning.hint
def test_multiple_deprecated_vars_produce_one_warning_each(
self,
mocker: MockerFixture,
) -> None:
"""Each deprecated var present in the environment gets its own warning."""
set_vars = {
"PAPERLESS_DB_TIMEOUT": "30",
"PAPERLESS_DB_POOLSIZE": "10",
"PAPERLESS_DBSSLMODE": "require",
}
mocker.patch.dict(os.environ, set_vars, clear=True)
result = check_deprecated_db_settings(None)
assert len(result) == len(set_vars)
assert all(isinstance(w, Warning) for w in result)
assert all(w.id == "paperless.W001" for w in result)
all_hints = " ".join(w.hint for w in result)
for var_name in set_vars:
assert var_name in all_hints
def test_all_deprecated_vars_produces_one_warning_each(
self,
mocker: MockerFixture,
) -> None:
"""All deprecated vars set simultaneously produces one warning per var."""
all_vars = {var: "some_value" for var in DEPRECATED_VARS}
mocker.patch.dict(os.environ, all_vars, clear=True)
result = check_deprecated_db_settings(None)
assert len(result) == len(DEPRECATED_VARS)
assert all(isinstance(w, Warning) for w in result)
assert all(w.id == "paperless.W001" for w in result)
def test_unset_vars_not_mentioned_in_warnings(
self,
mocker: MockerFixture,
) -> None:
"""Vars absent from the environment do not appear in any warning."""
mocker.patch.dict(
os.environ,
{"PAPERLESS_DB_TIMEOUT": "30"},
clear=True,
)
result = check_deprecated_db_settings(None)
assert len(result) == 1
assert "PAPERLESS_DB_TIMEOUT" in result[0].hint
unset_vars = [v for v in DEPRECATED_VARS if v != "PAPERLESS_DB_TIMEOUT"]
for var_name in unset_vars:
assert var_name not in result[0].hint
def test_empty_string_var_not_treated_as_set(
self,
mocker: MockerFixture,
) -> None:
"""A var set to an empty string is not flagged as a deprecated setting."""
mocker.patch.dict(
os.environ,
{"PAPERLESS_DB_TIMEOUT": ""},
clear=True,
)
result = check_deprecated_db_settings(None)
assert result == []
def test_warning_mentions_migration_target(
self,
mocker: MockerFixture,
) -> None:
"""Each warning hints at PAPERLESS_DB_OPTIONS as the migration target."""
mocker.patch.dict(
os.environ,
{"PAPERLESS_DBSSLMODE": "require"},
clear=True,
)
result = check_deprecated_db_settings(None)
assert len(result) == 1
assert "PAPERLESS_DB_OPTIONS" in result[0].hint
def test_warning_message_identifies_var(
self,
mocker: MockerFixture,
) -> None:
"""The warning message (not just the hint) identifies the offending var."""
mocker.patch.dict(
os.environ,
{"PAPERLESS_DBSSLCERT": "/path/to/cert.pem"},
clear=True,
)
result = check_deprecated_db_settings(None)
assert len(result) == 1
assert "PAPERLESS_DBSSLCERT" in result[0].msg

View File

@@ -1,19 +1,21 @@
import datetime import datetime
import os import os
from pathlib import Path
from unittest import TestCase from unittest import TestCase
from unittest import mock from unittest import mock
import pytest import pytest
from celery.schedules import crontab from celery.schedules import crontab
from pytest_mock import MockerFixture
from paperless.settings import _parse_base_paths from paperless.settings import _parse_base_paths
from paperless.settings import _parse_beat_schedule from paperless.settings import _parse_beat_schedule
from paperless.settings import _parse_dateparser_languages from paperless.settings import _parse_dateparser_languages
from paperless.settings import _parse_db_settings
from paperless.settings import _parse_ignore_dates from paperless.settings import _parse_ignore_dates
from paperless.settings import _parse_paperless_url from paperless.settings import _parse_paperless_url
from paperless.settings import _parse_redis_url from paperless.settings import _parse_redis_url
from paperless.settings import default_threads_per_worker from paperless.settings import default_threads_per_worker
from paperless.settings.custom import parse_db_settings
class TestIgnoreDateParsing(TestCase): class TestIgnoreDateParsing(TestCase):
@@ -378,62 +380,302 @@ class TestCeleryScheduleParsing(TestCase):
) )
class TestDBSettings(TestCase): class TestParseDbSettings:
def test_db_timeout_with_sqlite(self) -> None: """Test suite for parse_db_settings function."""
"""
GIVEN: @pytest.mark.parametrize(
- PAPERLESS_DB_TIMEOUT is set ("env_vars", "expected_database_settings"),
WHEN: [
- Settings are parsed pytest.param(
THEN: {},
- PAPERLESS_DB_TIMEOUT set for sqlite
"""
with mock.patch.dict(
os.environ,
{ {
"PAPERLESS_DB_TIMEOUT": "10", "default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": None, # Will be replaced with tmp_path
"OPTIONS": {},
}, },
},
id="default-sqlite",
),
pytest.param(
{
"PAPERLESS_DBENGINE": "sqlite",
"PAPERLESS_DB_OPTIONS": "timeout=30",
},
{
"default": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": None, # Will be replaced with tmp_path
"OPTIONS": {
"timeout": 30,
},
},
},
id="sqlite-with-timeout-override",
),
pytest.param(
{
"PAPERLESS_DBENGINE": "postgresql",
"PAPERLESS_DBHOST": "localhost",
},
{
"default": {
"ENGINE": "django.db.backends.postgresql",
"HOST": "localhost",
"NAME": "paperless",
"USER": "paperless",
"PASSWORD": "paperless",
"OPTIONS": {
"sslmode": "prefer",
"sslrootcert": None,
"sslcert": None,
"sslkey": None,
},
},
"sqlite": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": None, # Will be replaced with tmp_path
"OPTIONS": {},
},
},
id="postgresql-defaults",
),
pytest.param(
{
"PAPERLESS_DBENGINE": "postgresql",
"PAPERLESS_DBHOST": "paperless-db-host",
"PAPERLESS_DBPORT": "1111",
"PAPERLESS_DBNAME": "customdb",
"PAPERLESS_DBUSER": "customuser",
"PAPERLESS_DBPASS": "custompass",
"PAPERLESS_DB_OPTIONS": "pool.max_size=50;pool.min_size=2;sslmode=require",
},
{
"default": {
"ENGINE": "django.db.backends.postgresql",
"HOST": "paperless-db-host",
"PORT": 1111,
"NAME": "customdb",
"USER": "customuser",
"PASSWORD": "custompass",
"OPTIONS": {
"sslmode": "require",
"sslrootcert": None,
"sslcert": None,
"sslkey": None,
"pool": {
"min_size": 2,
"max_size": 50,
},
},
},
"sqlite": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": None, # Will be replaced with tmp_path
"OPTIONS": {},
},
},
id="postgresql-overrides",
),
pytest.param(
{
"PAPERLESS_DBENGINE": "postgresql",
"PAPERLESS_DBHOST": "pghost",
"PAPERLESS_DB_POOLSIZE": "10",
},
{
"default": {
"ENGINE": "django.db.backends.postgresql",
"HOST": "pghost",
"NAME": "paperless",
"USER": "paperless",
"PASSWORD": "paperless",
"OPTIONS": {
"sslmode": "prefer",
"sslrootcert": None,
"sslcert": None,
"sslkey": None,
"pool": {
"min_size": 1,
"max_size": 10,
},
},
},
"sqlite": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": None, # Will be replaced with tmp_path
"OPTIONS": {},
},
},
id="postgresql-legacy-poolsize",
),
pytest.param(
{
"PAPERLESS_DBENGINE": "postgresql",
"PAPERLESS_DBHOST": "pghost",
"PAPERLESS_DBSSLMODE": "require",
"PAPERLESS_DBSSLROOTCERT": "/certs/ca.crt",
"PAPERLESS_DB_TIMEOUT": "30",
},
{
"default": {
"ENGINE": "django.db.backends.postgresql",
"HOST": "pghost",
"NAME": "paperless",
"USER": "paperless",
"PASSWORD": "paperless",
"OPTIONS": {
"sslmode": "require",
"sslrootcert": "/certs/ca.crt",
"sslcert": None,
"sslkey": None,
"connect_timeout": 30,
},
},
"sqlite": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": None, # Will be replaced with tmp_path
"OPTIONS": {},
},
},
id="postgresql-legacy-ssl-and-timeout",
),
pytest.param(
{
"PAPERLESS_DBENGINE": "mariadb",
"PAPERLESS_DBHOST": "localhost",
},
{
"default": {
"ENGINE": "django.db.backends.mysql",
"HOST": "localhost",
"NAME": "paperless",
"USER": "paperless",
"PASSWORD": "paperless",
"OPTIONS": {
"read_default_file": "/etc/mysql/my.cnf",
"charset": "utf8mb4",
"collation": "utf8mb4_unicode_ci",
"ssl_mode": "PREFERRED",
"ssl": {
"ca": None,
"cert": None,
"key": None,
},
},
},
"sqlite": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": None, # Will be replaced with tmp_path
"OPTIONS": {},
},
},
id="mariadb-defaults",
),
pytest.param(
{
"PAPERLESS_DBENGINE": "mariadb",
"PAPERLESS_DBHOST": "paperless-mariadb-host",
"PAPERLESS_DBPORT": "5555",
"PAPERLESS_DBUSER": "my-cool-user",
"PAPERLESS_DBPASS": "my-secure-password",
"PAPERLESS_DB_OPTIONS": "ssl.ca=/path/to/ca.pem;ssl_mode=REQUIRED",
},
{
"default": {
"ENGINE": "django.db.backends.mysql",
"HOST": "paperless-mariadb-host",
"PORT": 5555,
"NAME": "paperless",
"USER": "my-cool-user",
"PASSWORD": "my-secure-password",
"OPTIONS": {
"read_default_file": "/etc/mysql/my.cnf",
"charset": "utf8mb4",
"collation": "utf8mb4_unicode_ci",
"ssl_mode": "REQUIRED",
"ssl": {
"ca": "/path/to/ca.pem",
"cert": None,
"key": None,
},
},
},
"sqlite": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": None, # Will be replaced with tmp_path
"OPTIONS": {},
},
},
id="mariadb-overrides",
),
pytest.param(
{
"PAPERLESS_DBENGINE": "mariadb",
"PAPERLESS_DBHOST": "mariahost",
"PAPERLESS_DBSSLMODE": "REQUIRED",
"PAPERLESS_DBSSLROOTCERT": "/certs/ca.pem",
"PAPERLESS_DBSSLCERT": "/certs/client.pem",
"PAPERLESS_DBSSLKEY": "/certs/client.key",
"PAPERLESS_DB_TIMEOUT": "25",
},
{
"default": {
"ENGINE": "django.db.backends.mysql",
"HOST": "mariahost",
"NAME": "paperless",
"USER": "paperless",
"PASSWORD": "paperless",
"OPTIONS": {
"read_default_file": "/etc/mysql/my.cnf",
"charset": "utf8mb4",
"collation": "utf8mb4_unicode_ci",
"ssl_mode": "REQUIRED",
"ssl": {
"ca": "/certs/ca.pem",
"cert": "/certs/client.pem",
"key": "/certs/client.key",
},
"connect_timeout": 25,
},
},
"sqlite": {
"ENGINE": "django.db.backends.sqlite3",
"NAME": None, # Will be replaced with tmp_path
"OPTIONS": {},
},
},
id="mariadb-legacy-ssl-and-timeout",
),
],
)
def test_parse_db_settings(
self,
tmp_path: Path,
mocker: MockerFixture,
env_vars: dict[str, str],
expected_database_settings: dict[str, dict],
) -> None:
"""Test various database configurations with defaults and overrides."""
# Clear environment and set test vars
mocker.patch.dict(os.environ, env_vars, clear=True)
# Update expected paths with actual tmp_path
if (
"default" in expected_database_settings
and expected_database_settings["default"]["NAME"] is None
): ):
databases = _parse_db_settings() expected_database_settings["default"]["NAME"] = str(
tmp_path / "db.sqlite3",
self.assertDictEqual( )
{ if "sqlite" in expected_database_settings:
"timeout": 10.0, expected_database_settings["sqlite"]["NAME"] = str(
}, tmp_path / "db.sqlite3",
databases["default"]["OPTIONS"],
) )
def test_db_timeout_with_not_sqlite(self) -> None: settings = parse_db_settings(tmp_path)
"""
GIVEN:
- PAPERLESS_DB_TIMEOUT is set but db is not sqlite
WHEN:
- Settings are parsed
THEN:
- PAPERLESS_DB_TIMEOUT set correctly in non-sqlite db & for fallback sqlite db
"""
with mock.patch.dict(
os.environ,
{
"PAPERLESS_DBHOST": "127.0.0.1",
"PAPERLESS_DB_TIMEOUT": "10",
},
):
databases = _parse_db_settings()
self.assertDictEqual( assert settings == expected_database_settings
databases["default"]["OPTIONS"],
databases["default"]["OPTIONS"]
| {
"connect_timeout": 10.0,
},
)
self.assertDictEqual(
{
"timeout": 10.0,
},
databases["sqlite"]["OPTIONS"],
)
class TestPaperlessURLSettings(TestCase): class TestPaperlessURLSettings(TestCase):

View File

@@ -16,6 +16,7 @@ repo_name = "paperless-ngx/paperless-ngx"
nav = [ nav = [
"index.md", "index.md",
"setup.md", "setup.md",
"migration.md",
"usage.md", "usage.md",
"configuration.md", "configuration.md",
"administration.md", "administration.md",