Finshes the writeup and migration guide

docs/configuration.md

@@ -51,137 +51,163 @@ matcher.
 ### Database
 
 By default, Paperless uses **SQLite** with a database stored at `data/db.sqlite3`.
-To switch to **PostgreSQL** or **MariaDB**, set [`PAPERLESS_DBHOST`](#PAPERLESS_DBHOST) and optionally configure other
+For multi-user or higher-throughput deployments, **PostgreSQL** (recommended) or
-database-related environment variables.
+**MariaDB** can be used instead by setting [`PAPERLESS_DBENGINE`](#PAPERLESS_DBENGINE)
+and the relevant connection variables.
+
+#### [`PAPERLESS_DBENGINE=<engine>`](#PAPERLESS_DBENGINE) {#PAPERLESS_DBENGINE}
+
+: Specifies the database engine to use. Accepted values are `sqlite`, `postgresql`,
+and `mariadb`.
+
+    Defaults to `sqlite` if not set.
+
+    PostgreSQL and MariaDB both require [`PAPERLESS_DBHOST`](#PAPERLESS_DBHOST) to be
+    set. SQLite does not use any other connection variables; the database file is always
+    located at `<PAPERLESS_DATA_DIR>/db.sqlite3`.
+
+    !!! warning
+        Using MariaDB comes with some caveats.
+        See [MySQL Caveats](advanced_usage.md#mysql-caveats).
 
 #### [`PAPERLESS_DBHOST=<hostname>`](#PAPERLESS_DBHOST) {#PAPERLESS_DBHOST}
 
-: If unset, Paperless uses **SQLite** by default.
+: Hostname of the PostgreSQL or MariaDB database server. Required when
-
+`PAPERLESS_DBENGINE` is `postgresql` or `mariadb`.
-    Set `PAPERLESS_DBHOST` to switch to PostgreSQL or MariaDB instead.
-
-#### [`PAPERLESS_DBENGINE=<engine_name>`](#PAPERLESS_DBENGINE) {#PAPERLESS_DBENGINE}
-
-: Optional. Specifies the database engine to use when connecting to a remote database.
-Available options are `postgresql` and `mariadb`.
-
-    Defaults to `postgresql` if `PAPERLESS_DBHOST` is set.
-
-    !!! warning
-
-        Using MariaDB comes with some caveats. See [MySQL Caveats](advanced_usage.md#mysql-caveats).
 
 #### [`PAPERLESS_DBPORT=<port>`](#PAPERLESS_DBPORT) {#PAPERLESS_DBPORT}
 
 : Port to use when connecting to PostgreSQL or MariaDB.
 
-    Default is `5432` for PostgreSQL and `3306` for MariaDB.
+    Defaults to `5432` for PostgreSQL and `3306` for MariaDB.
 
 #### [`PAPERLESS_DBNAME=<name>`](#PAPERLESS_DBNAME) {#PAPERLESS_DBNAME}
 
-: Name of the database to connect to when using PostgreSQL or MariaDB.
+: Name of the PostgreSQL or MariaDB database to connect to.
 
-    Defaults to "paperless".
+    Defaults to `paperless`.
 
-#### [`PAPERLESS_DBUSER=<name>`](#PAPERLESS_DBUSER) {#PAPERLESS_DBUSER}
+#### [`PAPERLESS_DBUSER=<user>`](#PAPERLESS_DBUSER) {#PAPERLESS_DBUSER}
 
 : Username for authenticating with the PostgreSQL or MariaDB database.
 
-    Defaults to "paperless".
+    Defaults to `paperless`.
 
 #### [`PAPERLESS_DBPASS=<password>`](#PAPERLESS_DBPASS) {#PAPERLESS_DBPASS}
 
 : Password for the PostgreSQL or MariaDB database user.
 
-    Defaults to "paperless".
+    Defaults to `paperless`.
 
-#### [`PAPERLESS_DBSSLMODE=<mode>`](#PAPERLESS_DBSSLMODE) {#PAPERLESS_DBSSLMODE}
+#### [`PAPERLESS_DB_OPTIONS=<options>`](#PAPERLESS_DB_OPTIONS) {#PAPERLESS_DB_OPTIONS}
 
-: SSL mode to use when connecting to PostgreSQL or MariaDB.
+: Advanced database connection options as a semicolon-delimited key-value string.
+Keys and values are separated by `=`. Dot-notation produces nested option
+dictionaries; for example, `pool.max_size=20` sets
+`OPTIONS["pool"]["max_size"] = 20`.
 
-    See [the official documentation about
+    Options specified here are merged over the engine defaults. Unrecognised keys
-    sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html).
+    are passed through to the underlying database driver without validation, so a
+    typo will be silently ignored rather than producing an error.
 
-    See [the official documentation about
+    Refer to your database driver's documentation for the full set of accepted keys:
-    sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-mode).
 
-    *Note*: SSL mode values differ between PostgreSQL and MariaDB.
+    - PostgreSQL: [libpq connection parameters](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS)
+    - MariaDB: [MariaDB Connector/Python](https://mariadb.com/kb/en/mariadb-connector-python/)
+    - SQLite: [SQLite PRAGMA statements](https://www.sqlite.org/pragma.html)
 
-    Default is `prefer` for PostgreSQL and `PREFERRED` for MariaDB.
+    **Examples:**
 
-#### [`PAPERLESS_DBSSLROOTCERT=<ca-path>`](#PAPERLESS_DBSSLROOTCERT) {#PAPERLESS_DBSSLROOTCERT}
+```bash
+    # PostgreSQL: require SSL, set a custom CA certificate, and limit the pool size
+    PAPERLESS_DB_OPTIONS="sslmode=require;sslrootcert=/certs/ca.pem;pool.max_size=5"
 
-: Path to the SSL root certificate used to verify the database server.
+    # MariaDB: require SSL with a custom CA certificate
+    PAPERLESS_DB_OPTIONS="ssl_mode=REQUIRED;ssl.ca=/certs/ca.pem"
 
-    See [the official documentation about
+    # PostgreSQL: set a connection timeout
-    sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html).
+    PAPERLESS_DB_OPTIONS="connect_timeout=10"
-    Changes the location of `root.crt`.
+```
 
-    See [the official documentation about
+    !!! note "PostgreSQL connection pooling"
-    sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-ca).
+        Pool size is controlled via `pool.min_size` and `pool.max_size`. When
+        configuring pooling, ensure your PostgreSQL `max_connections` is large enough
+        to handle all pool connections across all workers:
+        `(web_workers + celery_workers) * pool.max_size + safety_margin`.
 
-    Defaults to unset, using the standard location in the home directory.
+#### ~~[`PAPERLESS_DBSSLMODE`](#PAPERLESS_DBSSLMODE)~~ {#PAPERLESS_DBSSLMODE}
 
-#### [`PAPERLESS_DBSSLCERT=<client-cert-path>`](#PAPERLESS_DBSSLCERT) {#PAPERLESS_DBSSLCERT}
+!!! failure "Removed in v3"
+Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
 
-: Path to the client SSL certificate used when connecting securely.
+```bash
+    # PostgreSQL
+    PAPERLESS_DB_OPTIONS="sslmode=require"
 
-    See [the official documentation about
+    # MariaDB
-    sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html).
+    PAPERLESS_DB_OPTIONS="ssl_mode=REQUIRED"
+```
 
-    See [the official documentation about
+#### ~~[`PAPERLESS_DBSSLROOTCERT`](#PAPERLESS_DBSSLROOTCERT)~~ {#PAPERLESS_DBSSLROOTCERT}
-    sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-cert).
 
-    Changes the location of `postgresql.crt`.
+!!! failure "Removed in v3"
+Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
 
-    Defaults to unset, using the standard location in the home directory.
+```bash
+    # PostgreSQL
+    PAPERLESS_DB_OPTIONS="sslrootcert=/path/to/ca.pem"
 
-#### [`PAPERLESS_DBSSLKEY=<client-cert-key>`](#PAPERLESS_DBSSLKEY) {#PAPERLESS_DBSSLKEY}
+    # MariaDB
+    PAPERLESS_DB_OPTIONS="ssl.ca=/path/to/ca.pem"
+```
 
-: Path to the client SSL private key used when connecting securely.
+#### ~~[`PAPERLESS_DBSSLCERT`](#PAPERLESS_DBSSLCERT)~~ {#PAPERLESS_DBSSLCERT}
 
-    See [the official documentation about
+!!! failure "Removed in v3"
-    sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html).
+Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
 
-    See [the official documentation about
+```bash
-    sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-key).
+    # PostgreSQL
+    PAPERLESS_DB_OPTIONS="sslcert=/path/to/client.crt"
 
-    Changes the location of `postgresql.key`.
+    # MariaDB
+    PAPERLESS_DB_OPTIONS="ssl.cert=/path/to/client.crt"
+```
 
-    Defaults to unset, using the standard location in the home directory.
+#### ~~[`PAPERLESS_DBSSLKEY`](#PAPERLESS_DBSSLKEY)~~ {#PAPERLESS_DBSSLKEY}
 
-#### [`PAPERLESS_DB_TIMEOUT=<int>`](#PAPERLESS_DB_TIMEOUT) {#PAPERLESS_DB_TIMEOUT}
+!!! failure "Removed in v3"
+Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
 
-: Sets how long a database connection should wait before timing out.
+```bash
+    # PostgreSQL
+    PAPERLESS_DB_OPTIONS="sslkey=/path/to/client.key"
 
-    For SQLite, this sets how long to wait if the database is locked.
+    # MariaDB
-    For PostgreSQL or MariaDB, this sets the connection timeout.
+    PAPERLESS_DB_OPTIONS="ssl.key=/path/to/client.key"
+```
 
-    Defaults to unset, which uses Django’s built-in defaults.
+#### ~~[`PAPERLESS_DB_TIMEOUT`](#PAPERLESS_DB_TIMEOUT)~~ {#PAPERLESS_DB_TIMEOUT}
 
-#### [`PAPERLESS_DB_POOLSIZE=<int>`](#PAPERLESS_DB_POOLSIZE) {#PAPERLESS_DB_POOLSIZE}
+!!! failure "Removed in v3"
+Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
 
-: Defines the maximum number of database connections to keep in the pool.
+```bash
+    # SQLite
+    PAPERLESS_DB_OPTIONS="timeout=30"
 
-    Only applies to PostgreSQL. This setting is ignored for other database engines.
+    # PostgreSQL or MariaDB
+    PAPERLESS_DB_OPTIONS="connect_timeout=30"
+```
 
-    The value must be greater than or equal to 1 to be used.
+#### ~~[`PAPERLESS_DB_POOLSIZE`](#PAPERLESS_DB_POOLSIZE)~~ {#PAPERLESS_DB_POOLSIZE}
-    Defaults to unset, which disables connection pooling.
 
-    !!! note
+!!! failure "Removed in v3"
+Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
 
-        A pool of 8-10 connections per worker is typically sufficient.
+```bash
-        If you encounter error messages such as `couldn't get a connection`
+    PAPERLESS_DB_OPTIONS="pool.max_size=10"
-        or database connection timeouts, you probably need to increase the pool size.
+```
-
-    !!! warning
-        Make sure your PostgreSQL `max_connections` setting is large enough to handle the connection pools:
-        `(NB_PAPERLESS_WORKERS + NB_CELERY_WORKERS) × POOL_SIZE + SAFETY_MARGIN`. For example, with
-        4 Paperless workers and 2 Celery workers, and a pool size of 8:``(4 + 2) × 8 + 10 = 58`,
-        so `max_connections = 60` (or even more) is appropriate.
-
-        This assumes only Paperless-ngx connects to your PostgreSQL instance. If you have other applications,
-        you should increase `max_connections` accordingly.
 
 #### [`PAPERLESS_DB_READ_CACHE_ENABLED=<bool>`](#PAPERLESS_DB_READ_CACHE_ENABLED) {#PAPERLESS_DB_READ_CACHE_ENABLED}
 

docs/migration.md

@@ -48,3 +48,58 @@ The `CONSUMER_BARCODE_SCANNER` setting has been removed. zxing-cpp is now the on
     reliability.
 -   The `libzbar0` / `libzbar-dev` system packages are no longer required and can be removed from any custom Docker
     images or host installations.
+
+## Database Engine
+
+`PAPERLESS_DBENGINE` is now required to use PostgreSQL or MariaDB. Previously, the
+engine was inferred from the presence of `PAPERLESS_DBHOST`, with `PAPERLESS_DBENGINE`
+only needed to select MariaDB over PostgreSQL.
+
+SQLite users require no changes, though they may explicitly set their engine if desired.
+
+#### Action Required
+
+PostgreSQL and MariaDB users must add `PAPERLESS_DBENGINE` to their environment:
+
+```yaml
+# v2 (PostgreSQL inferred from PAPERLESS_DBHOST)
+PAPERLESS_DBHOST: postgres
+
+# v3 (engine must be explicit)
+PAPERLESS_DBENGINE: postgresql
+PAPERLESS_DBHOST: postgres
+```
+
+See [`PAPERLESS_DBENGINE`](configuration.md#PAPERLESS_DBENGINE) for accepted values.
+
+## Database Advanced Options
+
+The individual SSL, timeout, and pooling variables have been removed in favour of a
+single [`PAPERLESS_DB_OPTIONS`](configuration.md#PAPERLESS_DB_OPTIONS) string. This
+consolidates a growing set of engine-specific variables into one place, and allows
+any option supported by the underlying database driver to be set without requiring a
+dedicated environment variable for each.
+
+The removed variables and their replacements are:
+
+| Removed Variable          | Replacement in `PAPERLESS_DB_OPTIONS`                                        |
+| ------------------------- | ---------------------------------------------------------------------------- |
+| `PAPERLESS_DBSSLMODE`     | `sslmode=<value>` (PostgreSQL) or `ssl_mode=<value>` (MariaDB)               |
+| `PAPERLESS_DBSSLROOTCERT` | `sslrootcert=<path>` (PostgreSQL) or `ssl.ca=<path>` (MariaDB)               |
+| `PAPERLESS_DBSSLCERT`     | `sslcert=<path>` (PostgreSQL) or `ssl.cert=<path>` (MariaDB)                 |
+| `PAPERLESS_DBSSLKEY`      | `sslkey=<path>` (PostgreSQL) or `ssl.key=<path>` (MariaDB)                   |
+| `PAPERLESS_DB_POOLSIZE`   | `pool.max_size=<value>` (PostgreSQL only)                                    |
+| `PAPERLESS_DB_TIMEOUT`    | `timeout=<value>` (SQLite) or `connect_timeout=<value>` (PostgreSQL/MariaDB) |
+
+The deprecated variables will continue to function for now but will be removed in a
+future release. A deprecation warning is logged at startup for each deprecated variable
+that is still set.
+
+#### Action Required
+
+Users with any of the deprecated variables set should migrate to `PAPERLESS_DB_OPTIONS`.
+Multiple options are combined in a single value:
+
+```bash
+PAPERLESS_DB_OPTIONS="sslmode=require;sslrootcert=/certs/ca.pem;pool.max_size=10"
+```

src/paperless/settings/custom.py

@@ -103,6 +103,7 @@ def parse_db_settings(data_dir: Path) -> dict[str, dict[str, Any]]:
     db_config["OPTIONS"] = parse_dict_from_str(
         os.getenv("PAPERLESS_DB_OPTIONS"),
         defaults=base_options,
+        separator=";",
         type_map={
             # SQLite options
             "timeout": int,

src/paperless/tests/test_settings.py

@@ -448,7 +448,7 @@ class TestParseDbSettings:
                     "PAPERLESS_DBNAME": "customdb",
                     "PAPERLESS_DBUSER": "customuser",
                     "PAPERLESS_DBPASS": "custompass",
-                    "PAPERLESS_DB_OPTIONS": "pool.max_size=50,pool.min_size=2,sslmode=require",
+                    "PAPERLESS_DB_OPTIONS": "pool.max_size=50;pool.min_size=2;sslmode=require",
                 },
                 {
                     "default": {
@@ -579,7 +579,7 @@ class TestParseDbSettings:
                     "PAPERLESS_DBPORT": "5555",
                     "PAPERLESS_DBUSER": "my-cool-user",
                     "PAPERLESS_DBPASS": "my-secure-password",
-                    "PAPERLESS_DB_OPTIONS": "ssl.ca=/path/to/ca.pem,ssl_mode=REQUIRED",
+                    "PAPERLESS_DB_OPTIONS": "ssl.ca=/path/to/ca.pem;ssl_mode=REQUIRED",
                 },
                 {
                     "default": {

zensical.toml

@@ -16,6 +16,7 @@ repo_name = "paperless-ngx/paperless-ngx"
 nav = [
   "index.md",
   "setup.md",
+	"migration.md",
   "usage.md",
   "configuration.md",
   "administration.md",