From 304cfc42a92e726b99ce02cf42e35cddfd4063a9 Mon Sep 17 00:00:00 2001 From: Michael Shamoon <4887959+shamoon@users.noreply.github.com> Date: Tue, 29 Nov 2022 22:29:55 -0800 Subject: [PATCH] redirect all RTD pages --- docs/_static/css/custom.css | 8 + docs/_templates/layout.html | 25 + docs/administration.rst | 528 +------- docs/advanced_usage.rst | 446 +------ docs/api.rst | 299 +---- docs/changelog.md | 2445 ----------------------------------- docs/changelog.rst | 11 + docs/configuration.rst | 925 +------------ docs/extending.rst | 429 +----- docs/faq.rst | 113 +- docs/index.rst | 80 +- docs/scanners.rst | 8 +- docs/screenshots.rst | 57 +- docs/setup.rst | 890 +------------ docs/troubleshooting.rst | 326 +---- docs/usage_overview.rst | 418 +----- 16 files changed, 107 insertions(+), 6901 deletions(-) delete mode 100644 docs/changelog.md create mode 100644 docs/changelog.rst diff --git a/docs/_static/css/custom.css b/docs/_static/css/custom.css index c0b3ed83a..1a2734ac0 100644 --- a/docs/_static/css/custom.css +++ b/docs/_static/css/custom.css @@ -595,3 +595,11 @@ html.writer-html5 .rst-content dl.footnote code { .wy-nav-content-wrap { z-index: 20; } + +.rst-content .toctree-wrapper { + display: none; +} + +.redirect-notice { + font-size: 2.5rem; +} diff --git a/docs/_templates/layout.html b/docs/_templates/layout.html index cf648b606..c7c9f37db 100644 --- a/docs/_templates/layout.html +++ b/docs/_templates/layout.html @@ -8,6 +8,31 @@ document.documentElement.classList.toggle("dark-mode", darkModeState); document.documentElement.classList.toggle("light-mode", !darkModeState); + + const RTD_TO_MKD = { + "index.html": "", + "setup.html": "setup", + "usage_overview.html": "usage", + "advanced_usage.html": "advanced_usage", + "administration.html": "administration", + "configuration.html": "configuration", + "api.html": "api", + "faq.html": "faq", + "troubleshooting.html": "troubleshooting", + "extending.html": "development", + "scanners.html": "", + "screenshots.html": "", + "changelog.html": "changelog", + } + + const path = RTD_TO_MKD[window.location.pathname.substring(window.location.pathname.lastIndexOf("/") + 1)] + "/"; + const hash = window.location.hash; + const redirectURL = new URL(path + hash, "https://paperless-ngx.com/"); + console.log(`Redirecting to ${redirectURL} in 3 seconds...`); + + setTimeout(() => { + window.location.replace(redirectURL); + }, 3000); {{ super() }} {% endblock %} diff --git a/docs/administration.rst b/docs/administration.rst index 2984057ed..73c44cca9 100644 --- a/docs/administration.rst +++ b/docs/administration.rst @@ -1,531 +1,11 @@ +.. _administration: ************** Administration ************** -.. _administration-backup: +.. cssclass:: redirect-notice -Making backups -############## + The Paperless-ngx documentation has permanently moved. -Multiple options exist for making backups of your paperless instance, -depending on how you installed paperless. - -Before making backups, make sure that paperless is not running. - -Options available to any installation of paperless: - -* Use the :ref:`document exporter `. - The document exporter exports all your documents, thumbnails and - metadata to a specific folder. You may import your documents into a - fresh instance of paperless again or store your documents in another - DMS with this export. -* The document exporter is also able to update an already existing export. - Therefore, incremental backups with ``rsync`` are entirely possible. - -.. caution:: - - You cannot import the export generated with one version of paperless in a - different version of paperless. The export contains an exact image of the - database, and migrations may change the database layout. - -Options available to docker installations: - -* Backup the docker volumes. These usually reside within - ``/var/lib/docker/volumes`` on the host and you need to be root in order - to access them. - - Paperless uses 4 volumes: - - * ``paperless_media``: This is where your documents are stored. - * ``paperless_data``: This is where auxillary data is stored. This - folder also contains the SQLite database, if you use it. - * ``paperless_pgdata``: Exists only if you use PostgreSQL and contains - the database. - * ``paperless_dbdata``: Exists only if you use MariaDB and contains - the database. - -Options available to bare-metal and non-docker installations: - -* Backup the entire paperless folder. This ensures that if your paperless instance - crashes at some point or your disk fails, you can simply copy the folder back - into place and it works. - - When using PostgreSQL or MariaDB, you'll also have to backup the database. - -.. _migrating-restoring: - -Restoring -========= - -.. _administration-updating: - -Updating Paperless -################## - -Docker Route -============ - -If a new release of paperless-ngx is available, upgrading depends on how you -installed paperless-ngx in the first place. The releases are available at the -`release page `_. - -First of all, ensure that paperless is stopped. - -.. code:: shell-session - - $ cd /path/to/paperless - $ docker-compose down - -After that, :ref:`make a backup `. - -A. If you pull the image from the docker hub, all you need to do is: - - .. code:: shell-session - - $ docker-compose pull - $ docker-compose up - - The docker-compose files refer to the ``latest`` version, which is always the latest - stable release. - -B. If you built the image yourself, do the following: - - .. code:: shell-session - - $ git pull - $ docker-compose build - $ docker-compose up - -Running ``docker-compose up`` will also apply any new database migrations. -If you see everything working, press CTRL+C once to gracefully stop paperless. -Then you can start paperless-ngx with ``-d`` to have it run in the background. - - .. note:: - - In version 0.9.14, the update process was changed. In 0.9.13 and earlier, the - docker-compose files specified exact versions and pull won't automatically - update to newer versions. In order to enable updates as described above, either - get the new ``docker-compose.yml`` file from `here `_ - or edit the ``docker-compose.yml`` file, find the line that says - - .. code:: - - image: ghcr.io/paperless-ngx/paperless-ngx:0.9.x - - and replace the version with ``latest``: - - .. code:: - - image: ghcr.io/paperless-ngx/paperless-ngx:latest - - .. note:: - In version 1.7.1 and onwards, the Docker image can now be pinned to a release series. - This is often combined with automatic updaters such as Watchtower to allow safer - unattended upgrading to new bugfix releases only. It is still recommended to always - review release notes before upgrading. To pin your install to a release series, edit - the ``docker-compose.yml`` find the line that says - - .. code:: - - image: ghcr.io/paperless-ngx/paperless-ngx:latest - - and replace the version with the series you want to track, for example: - - .. code:: - - image: ghcr.io/paperless-ngx/paperless-ngx:1.7 - -Bare Metal Route -================ - -After grabbing the new release and unpacking the contents, do the following: - -1. Update dependencies. New paperless version may require additional - dependencies. The dependencies required are listed in the section about - :ref:`bare metal installations `. - -2. Update python requirements. Keep in mind to activate your virtual environment - before that, if you use one. - - .. code:: shell-session - - $ pip install -r requirements.txt - -3. Migrate the database. - - .. code:: shell-session - - $ cd src - $ python3 manage.py migrate - - This might not actually do anything. Not every new paperless version comes with new - database migrations. - -Downgrading Paperless -##################### - -Downgrades are possible. However, some updates also contain database migrations (these change the layout of the database and may move data). -In order to move back from a version that applied database migrations, you'll have to revert the database migration *before* downgrading, -and then downgrade paperless. - -This table lists the compatible versions for each database migration number. - -+------------------+-----------------+ -| Migration number | Version range | -+------------------+-----------------+ -| 1011 | 1.0.0 | -+------------------+-----------------+ -| 1012 | 1.1.0 - 1.2.1 | -+------------------+-----------------+ -| 1014 | 1.3.0 - 1.3.1 | -+------------------+-----------------+ -| 1016 | 1.3.2 - current | -+------------------+-----------------+ - -Execute the following management command to migrate your database: - -.. code:: shell-session - - $ python3 manage.py migrate documents - -.. note:: - - Some migrations cannot be undone. The command will issue errors if that happens. - -.. _utilities-management-commands: - -Management utilities -#################### - -Paperless comes with some management commands that perform various maintenance -tasks on your paperless instance. You can invoke these commands in the following way: - -With docker-compose, while paperless is running: - -.. code:: shell-session - - $ cd /path/to/paperless - $ docker-compose exec webserver - -With docker, while paperless is running: - -.. code:: shell-session - - $ docker exec -it - -Bare metal: - -.. code:: shell-session - - $ cd /path/to/paperless/src - $ python3 manage.py - -All commands have built-in help, which can be accessed by executing them with -the argument ``--help``. - -.. _utilities-exporter: - -Document exporter -================= - -The document exporter exports all your data from paperless into a folder for -backup or migration to another DMS. - -If you use the document exporter within a cronjob to backup your data you might use the ``-T`` flag behind exec to suppress "The input device is not a TTY" errors. For example: ``docker-compose exec -T webserver document_exporter ../export`` - -.. code:: - - document_exporter target [-c] [-f] [-d] - - optional arguments: - -c, --compare-checksums - -f, --use-filename-format - -d, --delete - -``target`` is a folder to which the data gets written. This includes documents, -thumbnails and a ``manifest.json`` file. The manifest contains all metadata from -the database (correspondents, tags, etc). - -When you use the provided docker compose script, specify ``../export`` as the -target. This path inside the container is automatically mounted on your host on -the folder ``export``. - -If the target directory already exists and contains files, paperless will assume -that the contents of the export directory are a previous export and will attempt -to update the previous export. Paperless will only export changed and added files. -Paperless determines whether a file has changed by inspecting the file attributes -"date/time modified" and "size". If that does not work out for you, specify -``--compare-checksums`` and paperless will attempt to compare file checksums instead. -This is slower. - -Paperless will not remove any existing files in the export directory. If you want -paperless to also remove files that do not belong to the current export such as files -from deleted documents, specify ``--delete``. Be careful when pointing paperless to -a directory that already contains other files. - -The filenames generated by this command follow the format -``[date created] [correspondent] [title].[extension]``. -If you want paperless to use ``PAPERLESS_FILENAME_FORMAT`` for exported filenames -instead, specify ``--use-filename-format``. - - -.. _utilities-importer: - -Document importer -================= - -The document importer takes the export produced by the `Document exporter`_ and -imports it into paperless. - -The importer works just like the exporter. You point it at a directory, and -the script does the rest of the work: - -.. code:: - - document_importer source - -When you use the provided docker compose script, put the export inside the -``export`` folder in your paperless source directory. Specify ``../export`` -as the ``source``. - -.. note:: - - Importing from a previous version of Paperless may work, but for best results - it is suggested to match the versions. - -.. _utilities-retagger: - -Document retagger -================= - -Say you've imported a few hundred documents and now want to introduce -a tag or set up a new correspondent, and apply its matching to all of -the currently-imported docs. This problem is common enough that -there are tools for it. - -.. code:: - - document_retagger [-h] [-c] [-T] [-t] [-i] [--use-first] [-f] - - optional arguments: - -c, --correspondent - -T, --tags - -t, --document_type - -s, --storage_path - -i, --inbox-only - --use-first - -f, --overwrite - -Run this after changing or adding matching rules. It'll loop over all -of the documents in your database and attempt to match documents -according to the new rules. - -Specify any combination of ``-c``, ``-T``, ``-t`` and ``-s`` to have the -retagger perform matching of the specified metadata type. If you don't -specify any of these options, the document retagger won't do anything. - -Specify ``-i`` to have the document retagger work on documents tagged -with inbox tags only. This is useful when you don't want to mess with -your already processed documents. - -When multiple document types or correspondents match a single document, -the retagger won't assign these to the document. Specify ``--use-first`` -to override this behavior and just use the first correspondent or type -it finds. This option does not apply to tags, since any amount of tags -can be applied to a document. - -Finally, ``-f`` specifies that you wish to overwrite already assigned -correspondents, types and/or tags. The default behavior is to not -assign correspondents and types to documents that have this data already -assigned. ``-f`` works differently for tags: By default, only additional tags get -added to documents, no tags will be removed. With ``-f``, tags that don't -match a document anymore get removed as well. - - -Managing the Automatic matching algorithm -========================================= - -The *Auto* matching algorithm requires a trained neural network to work. -This network needs to be updated whenever somethings in your data -changes. The docker image takes care of that automatically with the task -scheduler. You can manually renew the classifier by invoking the following -management command: - -.. code:: - - document_create_classifier - -This command takes no arguments. - -.. _`administration-index`: - -Managing the document search index -================================== - -The document search index is responsible for delivering search results for the -website. The document index is automatically updated whenever documents get -added to, changed, or removed from paperless. However, if the search yields -non-existing documents or won't find anything, you may need to recreate the -index manually. - -.. code:: - - document_index {reindex,optimize} - -Specify ``reindex`` to have the index created from scratch. This may take some -time. - -Specify ``optimize`` to optimize the index. This updates certain aspects of -the index and usually makes queries faster and also ensures that the -autocompletion works properly. This command is regularly invoked by the task -scheduler. - -.. _utilities-renamer: - -Managing filenames -================== - -If you use paperless' feature to -:ref:`assign custom filenames to your documents `, -you can use this command to move all your files after changing -the naming scheme. - -.. warning:: - - Since this command moves your documents, it is advised to do - a backup beforehand. The renaming logic is robust and will never overwrite - or delete a file, but you can't ever be careful enough. - -.. code:: - - document_renamer - -The command takes no arguments and processes all your documents at once. - -Learn how to use :ref:`Management Utilities`. - - -.. _utilities-sanity-checker: - -Sanity checker -============== - -Paperless has a built-in sanity checker that inspects your document collection for issues. - -The issues detected by the sanity checker are as follows: - -* Missing original files. -* Missing archive files. -* Inaccessible original files due to improper permissions. -* Inaccessible archive files due to improper permissions. -* Corrupted original documents by comparing their checksum against what is stored in the database. -* Corrupted archive documents by comparing their checksum against what is stored in the database. -* Missing thumbnails. -* Inaccessible thumbnails due to improper permissions. -* Documents without any content (warning). -* Orphaned files in the media directory (warning). These are files that are not referenced by any document im paperless. - - -.. code:: - - document_sanity_checker - -The command takes no arguments. Depending on the size of your document archive, this may take some time. - - -Fetching e-mail -=============== - -Paperless automatically fetches your e-mail every 10 minutes by default. If -you want to invoke the email consumer manually, call the following management -command: - -.. code:: - - mail_fetcher - -The command takes no arguments and processes all your mail accounts and rules. - - .. note:: - - As of October 2022 Microsoft no longer supports IMAP authentication for Exchange - servers, thus Exchange is no longer supported until a solution is implemented in - the Python IMAP library used by Paperless. See `learn.microsoft.com`_ - -.. _learn.microsoft.com: https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-basic-authentication-exchange-online - -.. _utilities-archiver: - -Creating archived documents -=========================== - -Paperless stores archived PDF/A documents alongside your original documents. -These archived documents will also contain selectable text for image-only -originals. -These documents are derived from the originals, which are always stored -unmodified. If coming from an earlier version of paperless, your documents -won't have archived versions. - -This command creates PDF/A documents for your documents. - -.. code:: - - document_archiver --overwrite --document - -This command will only attempt to create archived documents when no archived -document exists yet, unless ``--overwrite`` is specified. If ``--document `` -is specified, the archiver will only process that document. - -.. note:: - - This command essentially performs OCR on all your documents again, - according to your settings. If you run this with ``PAPERLESS_OCR_MODE=redo``, - it will potentially run for a very long time. You can cancel the command - at any time, since this command will skip already archived versions the next time - it is run. - -.. note:: - - Some documents will cause errors and cannot be converted into PDF/A documents, - such as encrypted PDF documents. The archiver will skip over these documents - each time it sees them. - -.. _utilities-encyption: - -Managing encryption -=================== - -Documents can be stored in Paperless using GnuPG encryption. - -.. danger:: - - Encryption is deprecated since paperless-ngx 0.9 and doesn't really provide any - additional security, since you have to store the passphrase in a configuration - file on the same system as the encrypted documents for paperless to work. - Furthermore, the entire text content of the documents is stored plain in the - database, even if your documents are encrypted. Filenames are not encrypted as - well. - - Also, the web server provides transparent access to your encrypted documents. - - Consider running paperless on an encrypted filesystem instead, which will then - at least provide security against physical hardware theft. - - -Enabling encryption -------------------- - -Enabling encryption is no longer supported. - - -Disabling encryption --------------------- - -Basic usage to disable encryption of your document store: - -(Note: If ``PAPERLESS_PASSPHRASE`` isn't set already, you need to specify it here) - -.. code:: - - decrypt_documents [--passphrase SECR3TP4SSPHRA$E] + You will be redirected shortly... diff --git a/docs/advanced_usage.rst b/docs/advanced_usage.rst index 1fe3e3685..6496bfa13 100644 --- a/docs/advanced_usage.rst +++ b/docs/advanced_usage.rst @@ -1,447 +1,11 @@ +.. _advanced_usage: + *************** Advanced topics *************** -Paperless offers a couple features that automate certain tasks and make your life -easier. +.. cssclass:: redirect-notice -.. _advanced-matching: + The Paperless-ngx documentation has permanently moved. -Matching tags, correspondents, document types, and storage paths -################################################################ - -Paperless will compare the matching algorithms defined by every tag, correspondent, -document type, and storage path in your database to see if they apply to the text -in a document. In other words, if you define a tag called ``Home Utility`` -that had a ``match`` property of ``bc hydro`` and a ``matching_algorithm`` of -``literal``, Paperless will automatically tag your newly-consumed document with -your ``Home Utility`` tag so long as the text ``bc hydro`` appears in the body -of the document somewhere. - -The matching logic is quite powerful. It supports searching the text of your -document with different algorithms, and as such, some experimentation may be -necessary to get things right. - -In order to have a tag, correspondent, document type, or storage path assigned -automatically to newly consumed documents, assign a match and matching algorithm -using the web interface. These settings define when to assign tags, correspondents, -document types, and storage paths to documents. - -The following algorithms are available: - -* **Any:** Looks for any occurrence of any word provided in match in the PDF. - If you define the match as ``Bank1 Bank2``, it will match documents containing - either of these terms. -* **All:** Requires that every word provided appears in the PDF, albeit not in the - order provided. -* **Literal:** Matches only if the match appears exactly as provided (i.e. preserve ordering) in the PDF. -* **Regular expression:** Parses the match as a regular expression and tries to - find a match within the document. -* **Fuzzy match:** I don't know. Look at the source. -* **Auto:** Tries to automatically match new documents. This does not require you - to set a match. See the notes below. - -When using the *any* or *all* matching algorithms, you can search for terms -that consist of multiple words by enclosing them in double quotes. For example, -defining a match text of ``"Bank of America" BofA`` using the *any* algorithm, -will match documents that contain either "Bank of America" or "BofA", but will -not match documents containing "Bank of South America". - -Then just save your tag, correspondent, document type, or storage path and run -another document through the consumer. Once complete, you should see the -newly-created document, automatically tagged with the appropriate data. - - -.. _advanced-automatic_matching: - -Automatic matching -================== - -Paperless-ngx comes with a new matching algorithm called *Auto*. This matching -algorithm tries to assign tags, correspondents, document types, and storage paths -to your documents based on how you have already assigned these on existing documents. -It uses a neural network under the hood. - -If, for example, all your bank statements of your account 123 at the Bank of -America are tagged with the tag "bofa_123" and the matching algorithm of this -tag is set to *Auto*, this neural network will examine your documents and -automatically learn when to assign this tag. - -Paperless tries to hide much of the involved complexity with this approach. -However, there are a couple caveats you need to keep in mind when using this -feature: - -* Changes to your documents are not immediately reflected by the matching - algorithm. The neural network needs to be *trained* on your documents after - changes. Paperless periodically (default: once each hour) checks for changes - and does this automatically for you. -* The Auto matching algorithm only takes documents into account which are NOT - placed in your inbox (i.e. have any inbox tags assigned to them). This ensures - that the neural network only learns from documents which you have correctly - tagged before. -* The matching algorithm can only work if there is a correlation between the - tag, correspondent, document type, or storage path and the document itself. - Your bank statements usually contain your bank account number and the name - of the bank, so this works reasonably well, However, tags such as "TODO" - cannot be automatically assigned. -* The matching algorithm needs a reasonable number of documents to identify when - to assign tags, correspondents, storage paths, and types. If one out of a - thousand documents has the correspondent "Very obscure web shop I bought - something five years ago", it will probably not assign this correspondent - automatically if you buy something from them again. The more documents, the better. -* Paperless also needs a reasonable amount of negative examples to decide when - not to assign a certain tag, correspondent, document type, or storage path. This will - usually be the case as you start filling up paperless with documents. - Example: If all your documents are either from "Webshop" and "Bank", paperless - will assign one of these correspondents to ANY new document, if both are set - to automatic matching. - -Hooking into the consumption process -#################################### - -Sometimes you may want to do something arbitrary whenever a document is -consumed. Rather than try to predict what you may want to do, Paperless lets -you execute scripts of your own choosing just before or after a document is -consumed using a couple simple hooks. - -Just write a script, put it somewhere that Paperless can read & execute, and -then put the path to that script in ``paperless.conf`` or ``docker-compose.env`` with the variable name -of either ``PAPERLESS_PRE_CONSUME_SCRIPT`` or -``PAPERLESS_POST_CONSUME_SCRIPT``. - -.. important:: - - These scripts are executed in a **blocking** process, which means that if - a script takes a long time to run, it can significantly slow down your - document consumption flow. If you want things to run asynchronously, - you'll have to fork the process in your script and exit. - - -Pre-consumption script -====================== - -Executed after the consumer sees a new document in the consumption folder, but -before any processing of the document is performed. This script can access the -following relevant environment variables set: - -* ``DOCUMENT_SOURCE_PATH`` - -A simple but common example for this would be creating a simple script like -this: - -``/usr/local/bin/ocr-pdf`` - -.. code:: bash - - #!/usr/bin/env bash - pdf2pdfocr.py -i ${DOCUMENT_SOURCE_PATH} - -``/etc/paperless.conf`` - -.. code:: bash - - ... - PAPERLESS_PRE_CONSUME_SCRIPT="/usr/local/bin/ocr-pdf" - ... - -This will pass the path to the document about to be consumed to ``/usr/local/bin/ocr-pdf``, -which will in turn call `pdf2pdfocr.py`_ on your document, which will then -overwrite the file with an OCR'd version of the file and exit. At which point, -the consumption process will begin with the newly modified file. - -The script's stdout and stderr will be logged line by line to the webserver log, along -with the exit code of the script. - -.. _pdf2pdfocr.py: https://github.com/LeoFCardoso/pdf2pdfocr - -.. _advanced-post_consume_script: - -Post-consumption script -======================= - -Executed after the consumer has successfully processed a document and has moved it -into paperless. It receives the following environment variables: - -* ``DOCUMENT_ID`` -* ``DOCUMENT_FILE_NAME`` -* ``DOCUMENT_CREATED`` -* ``DOCUMENT_MODIFIED`` -* ``DOCUMENT_ADDED`` -* ``DOCUMENT_SOURCE_PATH`` -* ``DOCUMENT_ARCHIVE_PATH`` -* ``DOCUMENT_THUMBNAIL_PATH`` -* ``DOCUMENT_DOWNLOAD_URL`` -* ``DOCUMENT_THUMBNAIL_URL`` -* ``DOCUMENT_CORRESPONDENT`` -* ``DOCUMENT_TAGS`` -* ``DOCUMENT_ORIGINAL_FILENAME`` - -The script can be in any language, but for a simple shell script -example, you can take a look at `post-consumption-example.sh`_ in this project. - -The post consumption script cannot cancel the consumption process. - -The script's stdout and stderr will be logged line by line to the webserver log, along -with the exit code of the script. - - -Docker ------- -Assumed you have ``/home/foo/paperless-ngx/scripts/post-consumption-example.sh``. - -You can pass that script into the consumer container via a host mount in your ``docker-compose.yml``. - -.. code:: bash - - ... - consumer: - ... - volumes: - ... - - /home/paperless-ngx/scripts:/path/in/container/scripts/ - ... - -Example (docker-compose.yml): ``- /home/foo/paperless-ngx/scripts:/usr/src/paperless/scripts`` - -which in turn requires the variable ``PAPERLESS_POST_CONSUME_SCRIPT`` in ``docker-compose.env`` to point to ``/path/in/container/scripts/post-consumption-example.sh``. - -Example (docker-compose.env): ``PAPERLESS_POST_CONSUME_SCRIPT=/usr/src/paperless/scripts/post-consumption-example.sh`` - -Troubleshooting: - -- Monitor the docker-compose log ``cd ~/paperless-ngx; docker-compose logs -f`` -- Check your script's permission e.g. in case of permission error ``sudo chmod 755 post-consumption-example.sh`` -- Pipe your scripts's output to a log file e.g. ``echo "${DOCUMENT_ID}" | tee --append /usr/src/paperless/scripts/post-consumption-example.log`` - -.. _post-consumption-example.sh: https://github.com/paperless-ngx/paperless-ngx/blob/main/scripts/post-consumption-example.sh - -.. _advanced-file_name_handling: - -File name handling -################## - -By default, paperless stores your documents in the media directory and renames them -using the identifier which it has assigned to each document. You will end up getting -files like ``0000123.pdf`` in your media directory. This isn't necessarily a bad -thing, because you normally don't have to access these files manually. However, if -you wish to name your files differently, you can do that by adjusting the -``PAPERLESS_FILENAME_FORMAT`` configuration option. Paperless adds the correct -file extension e.g. ``.pdf``, ``.jpg`` automatically. - -This variable allows you to configure the filename (folders are allowed) using -placeholders. For example, configuring this to - -.. code:: bash - - PAPERLESS_FILENAME_FORMAT={created_year}/{correspondent}/{title} - -will create a directory structure as follows: - -.. code:: - - 2019/ - My bank/ - Statement January.pdf - Statement February.pdf - 2020/ - My bank/ - Statement January.pdf - Letter.pdf - Letter_01.pdf - Shoe store/ - My new shoes.pdf - -.. danger:: - - Do not manually move your files in the media folder. Paperless remembers the - last filename a document was stored as. If you do rename a file, paperless will - report your files as missing and won't be able to find them. - -Paperless provides the following placeholders within filenames: - -* ``{asn}``: The archive serial number of the document, or "none". -* ``{correspondent}``: The name of the correspondent, or "none". -* ``{document_type}``: The name of the document type, or "none". -* ``{tag_list}``: A comma separated list of all tags assigned to the document. -* ``{title}``: The title of the document. -* ``{created}``: The full date (ISO format) the document was created. -* ``{created_year}``: Year created only, formatted as the year with century. -* ``{created_year_short}``: Year created only, formatted as the year without century, zero padded. -* ``{created_month}``: Month created only (number 01-12). -* ``{created_month_name}``: Month created name, as per locale -* ``{created_month_name_short}``: Month created abbreviated name, as per locale -* ``{created_day}``: Day created only (number 01-31). -* ``{added}``: The full date (ISO format) the document was added to paperless. -* ``{added_year}``: Year added only. -* ``{added_year_short}``: Year added only, formatted as the year without century, zero padded. -* ``{added_month}``: Month added only (number 01-12). -* ``{added_month_name}``: Month added name, as per locale -* ``{added_month_name_short}``: Month added abbreviated name, as per locale -* ``{added_day}``: Day added only (number 01-31). - - -Paperless will try to conserve the information from your database as much as possible. -However, some characters that you can use in document titles and correspondent names (such -as ``: \ /`` and a couple more) are not allowed in filenames and will be replaced with dashes. - -If paperless detects that two documents share the same filename, paperless will automatically -append ``_01``, ``_02``, etc to the filename. This happens if all the placeholders in a filename -evaluate to the same value. - -.. hint:: - You can affect how empty placeholders are treated by changing the following setting to - `true`. - - .. code:: - - PAPERLESS_FILENAME_FORMAT_REMOVE_NONE=True - - Doing this results in all empty placeholders resolving to "" instead of "none" as stated above. - Spaces before empty placeholders are removed as well, empty directories are omitted. - -.. hint:: - - Paperless checks the filename of a document whenever it is saved. Therefore, - you need to update the filenames of your documents and move them after altering - this setting by invoking the :ref:`document renamer `. - -.. warning:: - - Make absolutely sure you get the spelling of the placeholders right, or else - paperless will use the default naming scheme instead. - -.. caution:: - - As of now, you could totally tell paperless to store your files anywhere outside - the media directory by setting - - .. code:: - - PAPERLESS_FILENAME_FORMAT=../../my/custom/location/{title} - - However, keep in mind that inside docker, if files get stored outside of the - predefined volumes, they will be lost after a restart of paperless. - - -Storage paths -############# - -One of the best things in Paperless is that you can not only access the documents via the -web interface, but also via the file system. - -When as single storage layout is not sufficient for your use case, storage paths come to -the rescue. Storage paths allow you to configure more precisely where each document is stored -in the file system. - -- Each storage path is a `PAPERLESS_FILENAME_FORMAT` and follows the rules described above -- Each document is assigned a storage path using the matching algorithms described above, but - can be overwritten at any time - -For example, you could define the following two storage paths: - -1. Normal communications are put into a folder structure sorted by `year/correspondent` -2. Communications with insurance companies are stored in a flat structure with longer file names, - but containing the full date of the correspondence. - -.. code:: - - By Year = {created_year}/{correspondent}/{title} - Insurances = Insurances/{correspondent}/{created_year}-{created_month}-{created_day} {title} - - -If you then map these storage paths to the documents, you might get the following result. -For simplicity, `By Year` defines the same structure as in the previous example above. - -.. code:: text - - 2019/ # By Year - My bank/ - Statement January.pdf - Statement February.pdf - - Insurances/ # Insurances - Healthcare 123/ - 2022-01-01 Statement January.pdf - 2022-02-02 Letter.pdf - 2022-02-03 Letter.pdf - Dental 456/ - 2021-12-01 New Conditions.pdf - - -.. hint:: - - Defining a storage path is optional. If no storage path is defined for a document, the global - `PAPERLESS_FILENAME_FORMAT` is applied. - -.. caution:: - - If you adjust the format of an existing storage path, old documents don't get relocated automatically. - You need to run the :ref:`document renamer ` to adjust their pathes. - -.. _advanced-celery-monitoring: - -Celery Monitoring -################# - -The monitoring tool `Flower `_ can be used to view more -detailed information about the health of the celery workers used for asynchronous tasks. This includes details -on currently running, queued and completed tasks, timing and more. Flower can also be used with Prometheus, as it -exports metrics. For details on its capabilities, refer to the Flower documentation. - -To configure Flower further, create a `flowerconfig.py` and place it into the `src/paperless` directory. For -a Docker installation, you can use volumes to accomplish this: - -.. code:: yaml - - services: - # ... - webserver: - # ... - volumes: - - /path/to/my/flowerconfig.py:/usr/src/paperless/src/paperless/flowerconfig.py:ro - -Custom Container Initialization -############################### - -The Docker image includes the ability to run custom user scripts during startup. This could be -utilized for installing additional tools or Python packages, for example. - -To utilize this, mount a folder containing your scripts to the custom initialization directory, `/custom-cont-init.d` -and place scripts you wish to run inside. For security, the folder and its contents must be owned by `root`. -Additionally, scripts must only be writable by `root`. - -Your scripts will be run directly before the webserver completes startup. Scripts will be run by the `root` user. -This is an advanced functionality with which you could break functionality or lose data. - -For example, using Docker Compose: - - -.. code:: yaml - - services: - # ... - webserver: - # ... - volumes: - - /path/to/my/scripts:/custom-cont-init.d:ro - -.. _advanced-mysql-caveats: - -MySQL Caveats -############# - -Case Sensitivity -================ - -The database interface does not provide a method to configure a MySQL database to -be case sensitive. This would prevent a user from creating a tag ``Name`` and ``NAME`` -as they are considered the same. - -Per Django documentation, to enable this requires manual intervention. To enable -case sensetive tables, you can execute the following command against each table: - -``ALTER TABLE CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;`` - -You can also set the default for new tables (this does NOT affect existing tables) with: - -``ALTER DATABASE CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;`` + You will be redirected shortly... diff --git a/docs/api.rst b/docs/api.rst index 1e70b19fa..e3621dc32 100644 --- a/docs/api.rst +++ b/docs/api.rst @@ -1,303 +1,12 @@ +.. _api: ************ The REST API ************ -Paperless makes use of the `Django REST Framework`_ standard API interface. -It provides a browsable API for most of its endpoints, which you can inspect -at ``http://:/api/``. This also documents most of the -available filters and ordering fields. +.. cssclass:: redirect-notice -.. _Django REST Framework: http://django-rest-framework.org/ + The Paperless-ngx documentation has permanently moved. -The API provides 5 main endpoints: - -* ``/api/documents/``: Full CRUD support, except POSTing new documents. See below. -* ``/api/correspondents/``: Full CRUD support. -* ``/api/document_types/``: Full CRUD support. -* ``/api/logs/``: Read-Only. -* ``/api/tags/``: Full CRUD support. - -All of these endpoints except for the logging endpoint -allow you to fetch, edit and delete individual objects -by appending their primary key to the path, for example ``/api/documents/454/``. - -The objects served by the document endpoint contain the following fields: - -* ``id``: ID of the document. Read-only. -* ``title``: Title of the document. -* ``content``: Plain text content of the document. -* ``tags``: List of IDs of tags assigned to this document, or empty list. -* ``document_type``: Document type of this document, or null. -* ``correspondent``: Correspondent of this document or null. -* ``created``: The date time at which this document was created. -* ``created_date``: The date (YYYY-MM-DD) at which this document was created. Optional. If also passed with created, this is ignored. -* ``modified``: The date at which this document was last edited in paperless. Read-only. -* ``added``: The date at which this document was added to paperless. Read-only. -* ``archive_serial_number``: The identifier of this document in a physical document archive. -* ``original_file_name``: Verbose filename of the original document. Read-only. -* ``archived_file_name``: Verbose filename of the archived document. Read-only. Null if no archived document is available. - - -Downloading documents -##################### - -In addition to that, the document endpoint offers these additional actions on -individual documents: - -* ``/api/documents//download/``: Download the document. -* ``/api/documents//preview/``: Display the document inline, - without downloading it. -* ``/api/documents//thumb/``: Download the PNG thumbnail of a document. - -Paperless generates archived PDF/A documents from consumed files and stores both -the original files as well as the archived files. By default, the endpoints -for previews and downloads serve the archived file, if it is available. -Otherwise, the original file is served. -Some document cannot be archived. - -The endpoints correctly serve the response header fields ``Content-Disposition`` -and ``Content-Type`` to indicate the filename for download and the type of content of -the document. - -In order to download or preview the original document when an archived document is available, -supply the query parameter ``original=true``. - -.. hint:: - - Paperless used to provide these functionality at ``/fetch//preview``, - ``/fetch//thumb`` and ``/fetch//doc``. Redirects to the new URLs - are in place. However, if you use these old URLs to access documents, you - should update your app or script to use the new URLs. - - -Getting document metadata -######################### - -The api also has an endpoint to retrieve read-only metadata about specific documents. this -information is not served along with the document objects, since it requires reading -files and would therefore slow down document lists considerably. - -Access the metadata of a document with an ID ``id`` at ``/api/documents//metadata/``. - -The endpoint reports the following data: - -* ``original_checksum``: MD5 checksum of the original document. -* ``original_size``: Size of the original document, in bytes. -* ``original_mime_type``: Mime type of the original document. -* ``media_filename``: Current filename of the document, under which it is stored inside the media directory. -* ``has_archive_version``: True, if this document is archived, false otherwise. -* ``original_metadata``: A list of metadata associated with the original document. See below. -* ``archive_checksum``: MD5 checksum of the archived document, or null. -* ``archive_size``: Size of the archived document in bytes, or null. -* ``archive_metadata``: Metadata associated with the archived document, or null. See below. - -File metadata is reported as a list of objects in the following form: - -.. code:: json - - [ - { - "namespace": "http://ns.adobe.com/pdf/1.3/", - "prefix": "pdf", - "key": "Producer", - "value": "SparklePDF, Fancy edition" - }, - ] - -``namespace`` and ``prefix`` can be null. The actual metadata reported depends on the file type and the metadata -available in that specific document. Paperless only reports PDF metadata at this point. - -Authorization -############# - -The REST api provides three different forms of authentication. - -1. Basic authentication - - Authorize by providing a HTTP header in the form - - .. code:: - - Authorization: Basic - - where ``credentials`` is a base64-encoded string of ``:`` - -2. Session authentication - - When you're logged into paperless in your browser, you're automatically - logged into the API as well and don't need to provide any authorization - headers. - -3. Token authentication - - Paperless also offers an endpoint to acquire authentication tokens. - - POST a username and password as a form or json string to ``/api/token/`` - and paperless will respond with a token, if the login data is correct. - This token can be used to authenticate other requests with the - following HTTP header: - - .. code:: - - Authorization: Token - - Tokens can be managed and revoked in the paperless admin. - -Searching for documents -####################### - -Full text searching is available on the ``/api/documents/`` endpoint. Two specific -query parameters cause the API to return full text search results: - -* ``/api/documents/?query=your%20search%20query``: Search for a document using a full text query. - For details on the syntax, see :ref:`basic-usage_searching`. - -* ``/api/documents/?more_like=1234``: Search for documents similar to the document with id 1234. - -Pagination works exactly the same as it does for normal requests on this endpoint. - -Certain limitations apply to full text queries: - -* Results are always sorted by search score. The results matching the query best will show up first. - -* Only a small subset of filtering parameters are supported. - -Furthermore, each returned document has an additional ``__search_hit__`` attribute with various information -about the search results: - -.. code:: - - { - "count": 31, - "next": "http://localhost:8000/api/documents/?page=2&query=test", - "previous": null, - "results": [ - - ... - - { - "id": 123, - "title": "title", - "content": "content", - - ... - - "__search_hit__": { - "score": 0.343, - "highlights": "text Test text", - "rank": 23 - } - }, - - ... - - ] - } - -* ``score`` is an indication how well this document matches the query relative to the other search results. -* ``highlights`` is an excerpt from the document content and highlights the search terms with ```` tags as shown above. -* ``rank`` is the index of the search results. The first result will have rank 0. - -``/api/search/autocomplete/`` -============================= - -Get auto completions for a partial search term. - -Query parameters: - -* ``term``: The incomplete term. -* ``limit``: Amount of results. Defaults to 10. - -Results returned by the endpoint are ordered by importance of the term in the -document index. The first result is the term that has the highest Tf/Idf score -in the index. - -.. code:: json - - [ - "term1", - "term3", - "term6", - "term4" - ] - - -.. _api-file_uploads: - -POSTing documents -################# - -The API provides a special endpoint for file uploads: - -``/api/documents/post_document/`` - -POST a multipart form to this endpoint, where the form field ``document`` contains -the document that you want to upload to paperless. The filename is sanitized and -then used to store the document in a temporary directory, and the consumer will -be instructed to consume the document from there. - -The endpoint supports the following optional form fields: - -* ``title``: Specify a title that the consumer should use for the document. -* ``created``: Specify a DateTime where the document was created (e.g. "2016-04-19" or "2016-04-19 06:15:00+02:00"). -* ``correspondent``: Specify the ID of a correspondent that the consumer should use for the document. -* ``document_type``: Similar to correspondent. -* ``tags``: Similar to correspondent. Specify this multiple times to have multiple tags added - to the document. - - -The endpoint will immediately return "OK" if the document consumption process -was started successfully. No additional status information about the consumption -process itself is available, since that happens in a different process. - - -.. _api-versioning: - -API Versioning -############## - -The REST API is versioned since Paperless-ngx 1.3.0. - -* Versioning ensures that changes to the API don't break older clients. -* Clients specify the specific version of the API they wish to use with every request and Paperless will handle the request using the specified API version. -* Even if the underlying data model changes, older API versions will always serve compatible data. -* If no version is specified, Paperless will serve version 1 to ensure compatibility with older clients that do not request a specific API version. - -API versions are specified by submitting an additional HTTP ``Accept`` header with every request: - -.. code:: - - Accept: application/json; version=6 - -If an invalid version is specified, Paperless 1.3.0 will respond with "406 Not Acceptable" and an error message in the body. -Earlier versions of Paperless will serve API version 1 regardless of whether a version is specified via the ``Accept`` header. - -If a client wishes to verify whether it is compatible with any given server, the following procedure should be performed: - -1. Perform an *authenticated* request against any API endpoint. If the server is on version 1.3.0 or newer, the server will - add two custom headers to the response: - - .. code:: - - X-Api-Version: 2 - X-Version: 1.3.0 - -2. Determine whether the client is compatible with this server based on the presence/absence of these headers and their values if present. - - -API Changelog -============= - -Version 1 ---------- - -Initial API version. - -Version 2 ---------- - -* Added field ``Tag.color``. This read/write string field contains a hex color such as ``#a6cee3``. -* Added read-only field ``Tag.text_color``. This field contains the text color to use for a specific tag, which is either black or white depending on the brightness of ``Tag.color``. -* Removed field ``Tag.colour``. + You will be redirected shortly... diff --git a/docs/changelog.md b/docs/changelog.md deleted file mode 100644 index 2a037aebe..000000000 --- a/docs/changelog.md +++ /dev/null @@ -1,2445 +0,0 @@ -# Changelog - -## paperless-ngx 1.10.0 - -### Features - -- Feature: Capture stdout \& stderr of the pre/post consume scripts [@stumpylog](https://github.com/stumpylog) ([#1967](https://github.com/paperless-ngx/paperless-ngx/pull/1967)) -- Feature: Allow running custom container initialization scripts [@stumpylog](https://github.com/stumpylog) ([#1838](https://github.com/paperless-ngx/paperless-ngx/pull/1838)) -- Feature: Add more file name formatting options [@stumpylog](https://github.com/stumpylog) ([#1906](https://github.com/paperless-ngx/paperless-ngx/pull/1906)) -- Feature: 1.9.2 UI tweaks [@shamoon](https://github.com/shamoon) ([#1886](https://github.com/paperless-ngx/paperless-ngx/pull/1886)) -- Feature: Optional celery monitoring with Flower [@stumpylog](https://github.com/stumpylog) ([#1810](https://github.com/paperless-ngx/paperless-ngx/pull/1810)) -- Feature: Save pending tasks for frontend [@stumpylog](https://github.com/stumpylog) ([#1816](https://github.com/paperless-ngx/paperless-ngx/pull/1816)) -- Feature: Improved processing for automatic matching [@stumpylog](https://github.com/stumpylog) ([#1609](https://github.com/paperless-ngx/paperless-ngx/pull/1609)) -- Feature: Transition to celery for background tasks [@stumpylog](https://github.com/stumpylog) ([#1648](https://github.com/paperless-ngx/paperless-ngx/pull/1648)) -- Feature: UI Welcome Tour [@shamoon](https://github.com/shamoon) ([#1644](https://github.com/paperless-ngx/paperless-ngx/pull/1644)) -- Feature: slim sidebar [@shamoon](https://github.com/shamoon) ([#1641](https://github.com/paperless-ngx/paperless-ngx/pull/1641)) -- change default matching algo to auto and move to constant [@NiFNi](https://github.com/NiFNi) ([#1754](https://github.com/paperless-ngx/paperless-ngx/pull/1754)) -- Feature: Enable end to end Tika testing in CI [@stumpylog](https://github.com/stumpylog) ([#1757](https://github.com/paperless-ngx/paperless-ngx/pull/1757)) -- Feature: frontend update checking settings [@shamoon](https://github.com/shamoon) ([#1692](https://github.com/paperless-ngx/paperless-ngx/pull/1692)) -- Feature: Upgrade to qpdf 11, pikepdf 6 \& ocrmypdf 14 [@stumpylog](https://github.com/stumpylog) ([#1642](https://github.com/paperless-ngx/paperless-ngx/pull/1642)) - -### Bug Fixes - -- Bugfix: Fix created_date being a string [@stumpylog](https://github.com/stumpylog) ([#2023](https://github.com/paperless-ngx/paperless-ngx/pull/2023)) -- Bugfix: Fixes an issue with mixed text and images when redoing OCR [@stumpylog](https://github.com/stumpylog) ([#2017](https://github.com/paperless-ngx/paperless-ngx/pull/2017)) -- Bugfix: Always re-try barcodes with pdf2image [@stumpylog](https://github.com/stumpylog) ([#1953](https://github.com/paperless-ngx/paperless-ngx/pull/1953)) -- Fix: using `CONSUMER_SUBDIRS_AS_TAGS` causes failure with Celery in `dev` [@shamoon](https://github.com/shamoon) ([#1942](https://github.com/paperless-ngx/paperless-ngx/pull/1942)) -- Fix mail consumption broken in `dev` after move to celery [@shamoon](https://github.com/shamoon) ([#1934](https://github.com/paperless-ngx/paperless-ngx/pull/1934)) -- Bugfix: Prevent file handling from running with stale data [@stumpylog](https://github.com/stumpylog) ([#1905](https://github.com/paperless-ngx/paperless-ngx/pull/1905)) -- Chore: Reduce nuisance CI test failures [@stumpylog](https://github.com/stumpylog) ([#1922](https://github.com/paperless-ngx/paperless-ngx/pull/1922)) -- Bugfix: Unintentional deletion of feature tagged Docker images [@stumpylog](https://github.com/stumpylog) ([#1896](https://github.com/paperless-ngx/paperless-ngx/pull/1896)) -- Fix: independent control of saved views [@shamoon](https://github.com/shamoon) ([#1868](https://github.com/paperless-ngx/paperless-ngx/pull/1868)) -- Fix: frontend relative date searches [@shamoon](https://github.com/shamoon) ([#1865](https://github.com/paperless-ngx/paperless-ngx/pull/1865)) -- Chore: Fixes pipenv issues [@stumpylog](https://github.com/stumpylog) ([#1873](https://github.com/paperless-ngx/paperless-ngx/pull/1873)) -- Bugfix: Handle password protected PDFs during barcode detection [@stumpylog](https://github.com/stumpylog) ([#1858](https://github.com/paperless-ngx/paperless-ngx/pull/1858)) -- Fix: Allows configuring barcodes with pdf2image instead of pikepdf [@stumpylog](https://github.com/stumpylog) ([#1857](https://github.com/paperless-ngx/paperless-ngx/pull/1857)) -- Bugfix: Reverts the change around skip_noarchive [@stumpylog](https://github.com/stumpylog) ([#1829](https://github.com/paperless-ngx/paperless-ngx/pull/1829)) -- Fix: missing loadViewConfig breaks loading saved view [@shamoon](https://github.com/shamoon) ([#1792](https://github.com/paperless-ngx/paperless-ngx/pull/1792)) -- Bugfix: Fallback to pdf2image if pikepdf fails [@stumpylog](https://github.com/stumpylog) ([#1745](https://github.com/paperless-ngx/paperless-ngx/pull/1745)) -- Fix: creating new storage path on document edit fails to update menu [@shamoon](https://github.com/shamoon) ([#1777](https://github.com/paperless-ngx/paperless-ngx/pull/1777)) -- Bugfix: Files containing barcodes uploaded via web are not consumed after splitting [@stumpylog](https://github.com/stumpylog) ([#1762](https://github.com/paperless-ngx/paperless-ngx/pull/1762)) -- Bugfix: Fix email labeling for non-Gmail servers [@stumpylog](https://github.com/stumpylog) ([#1755](https://github.com/paperless-ngx/paperless-ngx/pull/1755)) -- Fix: allow preview for .csv files [@shamoon](https://github.com/shamoon) ([#1744](https://github.com/paperless-ngx/paperless-ngx/pull/1744)) -- Bugfix: csv recognition by consumer [@bin101](https://github.com/bin101) ([#1726](https://github.com/paperless-ngx/paperless-ngx/pull/1726)) -- Bugfix: Include document title when a duplicate is detected [@stumpylog](https://github.com/stumpylog) ([#1696](https://github.com/paperless-ngx/paperless-ngx/pull/1696)) -- Bugfix: Set MySql charset [@stumpylog](https://github.com/stumpylog) ([#1687](https://github.com/paperless-ngx/paperless-ngx/pull/1687)) -- Mariadb compose files should use `PAPERLESS_DBPASS` [@shamoon](https://github.com/shamoon) ([#1683](https://github.com/paperless-ngx/paperless-ngx/pull/1683)) - -### Documentation - -- Documentation: Update MariaDB docs to note some potential issues [@stumpylog](https://github.com/stumpylog) ([#2016](https://github.com/paperless-ngx/paperless-ngx/pull/2016)) -- Documentation: Add note re MS exchange servers [@shamoon](https://github.com/shamoon) ([#1780](https://github.com/paperless-ngx/paperless-ngx/pull/1780)) -- Chore: Updates Gotenberg versions [@stumpylog](https://github.com/stumpylog) ([#1768](https://github.com/paperless-ngx/paperless-ngx/pull/1768)) -- Documentation: Tweak LinuxServer [@stumpylog](https://github.com/stumpylog) ([#1761](https://github.com/paperless-ngx/paperless-ngx/pull/1761)) -- Documentation: Adds troubleshooting note about Kubernetes and ports [@stumpylog](https://github.com/stumpylog) ([#1731](https://github.com/paperless-ngx/paperless-ngx/pull/1731)) -- Documentation: LinuxServer.io Migration [@stumpylog](https://github.com/stumpylog) ([#1733](https://github.com/paperless-ngx/paperless-ngx/pull/1733)) -- [Documentation] Add v1.9.2 changelog [@github-actions](https://github.com/github-actions) ([#1671](https://github.com/paperless-ngx/paperless-ngx/pull/1671)) - -### Maintenance - -- Bump tj-actions/changed-files from 32 to 34 [@dependabot](https://github.com/dependabot) ([#1915](https://github.com/paperless-ngx/paperless-ngx/pull/1915)) -- Chore: Fix `dev` trying to build Pillow or lxml [@stumpylog](https://github.com/stumpylog) ([#1909](https://github.com/paperless-ngx/paperless-ngx/pull/1909)) -- Chore: Fixes pipenv issues [@stumpylog](https://github.com/stumpylog) ([#1873](https://github.com/paperless-ngx/paperless-ngx/pull/1873)) -- Chore: Simplified registry cleanup [@stumpylog](https://github.com/stumpylog) ([#1812](https://github.com/paperless-ngx/paperless-ngx/pull/1812)) -- Chore: Fixing deprecated workflow commands [@stumpylog](https://github.com/stumpylog) ([#1786](https://github.com/paperless-ngx/paperless-ngx/pull/1786)) -- Chore: Python library update + test fixes [@stumpylog](https://github.com/stumpylog) ([#1773](https://github.com/paperless-ngx/paperless-ngx/pull/1773)) -- Chore: Updates Gotenberg versions [@stumpylog](https://github.com/stumpylog) ([#1768](https://github.com/paperless-ngx/paperless-ngx/pull/1768)) -- Bump leonsteinhaeuser/project-beta-automations from 1.3.0 to 2.0.1 [@dependabot](https://github.com/dependabot) ([#1703](https://github.com/paperless-ngx/paperless-ngx/pull/1703)) -- Bump tj-actions/changed-files from 29.0.2 to 31.0.2 [@dependabot](https://github.com/dependabot) ([#1702](https://github.com/paperless-ngx/paperless-ngx/pull/1702)) -- Bump actions/checkout from 2 to 3 [@dependabot](https://github.com/dependabot) ([#1704](https://github.com/paperless-ngx/paperless-ngx/pull/1704)) -- Bump actions/setup-python from 3 to 4 [@dependabot](https://github.com/dependabot) ([#1705](https://github.com/paperless-ngx/paperless-ngx/pull/1705)) - -### Dependencies - -
-31 changes - -- Bugfix: Downgrade cryptography for armv7 compatibility [@stumpylog](https://github.com/stumpylog) ([#1954](https://github.com/paperless-ngx/paperless-ngx/pull/1954)) -- Chore: Bulk library updates + loosen restrictions [@stumpylog](https://github.com/stumpylog) ([#1949](https://github.com/paperless-ngx/paperless-ngx/pull/1949)) -- Bump tj-actions/changed-files from 32 to 34 [@dependabot](https://github.com/dependabot) ([#1915](https://github.com/paperless-ngx/paperless-ngx/pull/1915)) -- Bump scikit-learn from 1.1.2 to 1.1.3 [@dependabot](https://github.com/dependabot) ([#1903](https://github.com/paperless-ngx/paperless-ngx/pull/1903)) -- Bump angular packages as bundle [@dependabot](https://github.com/dependabot) ([#1910](https://github.com/paperless-ngx/paperless-ngx/pull/1910)) -- Bump ngx-ui-tour-ng-bootstrap from 11.0.0 to 11.1.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1911](https://github.com/paperless-ngx/paperless-ngx/pull/1911)) -- Bump jest-environment-jsdom from 29.1.2 to 29.2.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1914](https://github.com/paperless-ngx/paperless-ngx/pull/1914)) -- Bump pillow from 9.2.0 to 9.3.0 [@dependabot](https://github.com/dependabot) ([#1904](https://github.com/paperless-ngx/paperless-ngx/pull/1904)) -- Bump pytest from 7.1.3 to 7.2.0 [@dependabot](https://github.com/dependabot) ([#1902](https://github.com/paperless-ngx/paperless-ngx/pull/1902)) -- Bump tox from 3.26.0 to 3.27.0 [@dependabot](https://github.com/dependabot) ([#1901](https://github.com/paperless-ngx/paperless-ngx/pull/1901)) -- Bump zipp from 3.9.0 to 3.10.0 [@dependabot](https://github.com/dependabot) ([#1860](https://github.com/paperless-ngx/paperless-ngx/pull/1860)) -- Bump pytest-env from 0.6.2 to 0.8.1 [@dependabot](https://github.com/dependabot) ([#1859](https://github.com/paperless-ngx/paperless-ngx/pull/1859)) -- Bump sphinx from 5.2.3 to 5.3.0 [@dependabot](https://github.com/dependabot) ([#1817](https://github.com/paperless-ngx/paperless-ngx/pull/1817)) -- Chore: downgrade channels-redis [@stumpylog](https://github.com/stumpylog) ([#1802](https://github.com/paperless-ngx/paperless-ngx/pull/1802)) -- Chore: Update to qpdf 11.1.1 and update backend libraries [@stumpylog](https://github.com/stumpylog) ([#1749](https://github.com/paperless-ngx/paperless-ngx/pull/1749)) -- Bump myst-parser from 0.18.0 to 0.18.1 [@dependabot](https://github.com/dependabot) ([#1738](https://github.com/paperless-ngx/paperless-ngx/pull/1738)) -- Bump leonsteinhaeuser/project-beta-automations from 1.3.0 to 2.0.1 [@dependabot](https://github.com/dependabot) ([#1703](https://github.com/paperless-ngx/paperless-ngx/pull/1703)) -- Bump tj-actions/changed-files from 29.0.2 to 31.0.2 [@dependabot](https://github.com/dependabot) ([#1702](https://github.com/paperless-ngx/paperless-ngx/pull/1702)) -- Bump actions/checkout from 2 to 3 [@dependabot](https://github.com/dependabot) ([#1704](https://github.com/paperless-ngx/paperless-ngx/pull/1704)) -- Bump actions/setup-python from 3 to 4 [@dependabot](https://github.com/dependabot) ([#1705](https://github.com/paperless-ngx/paperless-ngx/pull/1705)) -- Bump rxjs from 7.5.6 to 7.5.7 in /src-ui [@dependabot](https://github.com/dependabot) ([#1720](https://github.com/paperless-ngx/paperless-ngx/pull/1720)) -- Bump uuid from 8.3.2 to 9.0.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1716](https://github.com/paperless-ngx/paperless-ngx/pull/1716)) -- Bump ng2-pdf-viewer from 9.1.0 to 9.1.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1717](https://github.com/paperless-ngx/paperless-ngx/pull/1717)) -- Bump ngx-color from 8.0.2 to 8.0.3 in /src-ui [@dependabot](https://github.com/dependabot) ([#1715](https://github.com/paperless-ngx/paperless-ngx/pull/1715)) -- Bump concurrently from 7.3.0 to 7.4.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1719](https://github.com/paperless-ngx/paperless-ngx/pull/1719)) -- Bump [@types/node from 18.7.14 to 18.7.23 in /src-ui @dependabot](https://github.com/types/node from 18.7.14 to 18.7.23 in /src-ui @dependabot) ([#1718](https://github.com/paperless-ngx/paperless-ngx/pull/1718)) -- Bump jest-environment-jsdom from 29.0.1 to 29.1.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1714](https://github.com/paperless-ngx/paperless-ngx/pull/1714)) -- Bump [@angular/cli @angular/core @dependabot](https://github.com/angular/cli @angular/core @dependabot) ([#1708](https://github.com/paperless-ngx/paperless-ngx/pull/1708)) -- Bump cypress from 10.7.0 to 10.9.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1707](https://github.com/paperless-ngx/paperless-ngx/pull/1707)) -- Bump bootstrap from 5.2.0 to 5.2.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1710](https://github.com/paperless-ngx/paperless-ngx/pull/1710)) -- Bump typescript from 4.7.4 to 4.8.4 in /src-ui [@dependabot](https://github.com/dependabot) ([#1706](https://github.com/paperless-ngx/paperless-ngx/pull/1706)) -
- -### All App Changes - -- Add info that re-do OCR doesnt automatically refresh content [@shamoon](https://github.com/shamoon) ([#2025](https://github.com/paperless-ngx/paperless-ngx/pull/2025)) -- Bugfix: Fix created_date being a string [@stumpylog](https://github.com/stumpylog) ([#2023](https://github.com/paperless-ngx/paperless-ngx/pull/2023)) -- Bugfix: Fixes an issue with mixed text and images when redoing OCR [@stumpylog](https://github.com/stumpylog) ([#2017](https://github.com/paperless-ngx/paperless-ngx/pull/2017)) -- Bugfix: Don't allow exceptions during date parsing to fail consume [@stumpylog](https://github.com/stumpylog) ([#1998](https://github.com/paperless-ngx/paperless-ngx/pull/1998)) -- Feature: Capture stdout \& stderr of the pre/post consume scripts [@stumpylog](https://github.com/stumpylog) ([#1967](https://github.com/paperless-ngx/paperless-ngx/pull/1967)) -- Bugfix: Always re-try barcodes with pdf2image [@stumpylog](https://github.com/stumpylog) ([#1953](https://github.com/paperless-ngx/paperless-ngx/pull/1953)) -- Fix: using `CONSUMER_SUBDIRS_AS_TAGS` causes failure with Celery in `dev` [@shamoon](https://github.com/shamoon) ([#1942](https://github.com/paperless-ngx/paperless-ngx/pull/1942)) -- Fix mail consumption broken in `dev` after move to celery [@shamoon](https://github.com/shamoon) ([#1934](https://github.com/paperless-ngx/paperless-ngx/pull/1934)) -- Bugfix: Prevent file handling from running with stale data [@stumpylog](https://github.com/stumpylog) ([#1905](https://github.com/paperless-ngx/paperless-ngx/pull/1905)) -- Chore: Reduce nuisance CI test failures [@stumpylog](https://github.com/stumpylog) ([#1922](https://github.com/paperless-ngx/paperless-ngx/pull/1922)) -- Bump scikit-learn from 1.1.2 to 1.1.3 [@dependabot](https://github.com/dependabot) ([#1903](https://github.com/paperless-ngx/paperless-ngx/pull/1903)) -- Bump angular packages as bundle [@dependabot](https://github.com/dependabot) ([#1910](https://github.com/paperless-ngx/paperless-ngx/pull/1910)) -- Bump ngx-ui-tour-ng-bootstrap from 11.0.0 to 11.1.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1911](https://github.com/paperless-ngx/paperless-ngx/pull/1911)) -- Bump jest-environment-jsdom from 29.1.2 to 29.2.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1914](https://github.com/paperless-ngx/paperless-ngx/pull/1914)) -- Feature: Add more file name formatting options [@stumpylog](https://github.com/stumpylog) ([#1906](https://github.com/paperless-ngx/paperless-ngx/pull/1906)) -- Bump pillow from 9.2.0 to 9.3.0 [@dependabot](https://github.com/dependabot) ([#1904](https://github.com/paperless-ngx/paperless-ngx/pull/1904)) -- Bump pytest from 7.1.3 to 7.2.0 [@dependabot](https://github.com/dependabot) ([#1902](https://github.com/paperless-ngx/paperless-ngx/pull/1902)) -- Bump tox from 3.26.0 to 3.27.0 [@dependabot](https://github.com/dependabot) ([#1901](https://github.com/paperless-ngx/paperless-ngx/pull/1901)) -- directly use rapidfuzz [@maxbachmann](https://github.com/maxbachmann) ([#1899](https://github.com/paperless-ngx/paperless-ngx/pull/1899)) -- Feature: 1.9.2 UI tweaks [@shamoon](https://github.com/shamoon) ([#1886](https://github.com/paperless-ngx/paperless-ngx/pull/1886)) -- Bump zipp from 3.9.0 to 3.10.0 [@dependabot](https://github.com/dependabot) ([#1860](https://github.com/paperless-ngx/paperless-ngx/pull/1860)) -- Fix: independent control of saved views [@shamoon](https://github.com/shamoon) ([#1868](https://github.com/paperless-ngx/paperless-ngx/pull/1868)) -- Fix: frontend relative date searches [@shamoon](https://github.com/shamoon) ([#1865](https://github.com/paperless-ngx/paperless-ngx/pull/1865)) -- Django error W003 - MariaDB may not allow unique CharFields to have a max_length > 255. [@Sblop](https://github.com/Sblop) ([#1881](https://github.com/paperless-ngx/paperless-ngx/pull/1881)) -- Bump pytest-env from 0.6.2 to 0.8.1 [@dependabot](https://github.com/dependabot) ([#1859](https://github.com/paperless-ngx/paperless-ngx/pull/1859)) -- Fix: Allows configuring barcodes with pdf2image instead of pikepdf [@stumpylog](https://github.com/stumpylog) ([#1857](https://github.com/paperless-ngx/paperless-ngx/pull/1857)) -- Feature: Save pending tasks for frontend [@stumpylog](https://github.com/stumpylog) ([#1816](https://github.com/paperless-ngx/paperless-ngx/pull/1816)) -- Bugfix: Reverts the change around skip_noarchive [@stumpylog](https://github.com/stumpylog) ([#1829](https://github.com/paperless-ngx/paperless-ngx/pull/1829)) -- Bump sphinx from 5.2.3 to 5.3.0 [@dependabot](https://github.com/dependabot) ([#1817](https://github.com/paperless-ngx/paperless-ngx/pull/1817)) -- Fix: missing loadViewConfig breaks loading saved view [@shamoon](https://github.com/shamoon) ([#1792](https://github.com/paperless-ngx/paperless-ngx/pull/1792)) -- Bugfix: Fallback to pdf2image if pikepdf fails [@stumpylog](https://github.com/stumpylog) ([#1745](https://github.com/paperless-ngx/paperless-ngx/pull/1745)) -- Fix: creating new storage path on document edit fails to update menu [@shamoon](https://github.com/shamoon) ([#1777](https://github.com/paperless-ngx/paperless-ngx/pull/1777)) -- Chore: Python library update + test fixes [@stumpylog](https://github.com/stumpylog) ([#1773](https://github.com/paperless-ngx/paperless-ngx/pull/1773)) -- Feature: Improved processing for automatic matching [@stumpylog](https://github.com/stumpylog) ([#1609](https://github.com/paperless-ngx/paperless-ngx/pull/1609)) -- Feature: Transition to celery for background tasks [@stumpylog](https://github.com/stumpylog) ([#1648](https://github.com/paperless-ngx/paperless-ngx/pull/1648)) -- Feature: UI Welcome Tour [@shamoon](https://github.com/shamoon) ([#1644](https://github.com/paperless-ngx/paperless-ngx/pull/1644)) -- Feature: slim sidebar [@shamoon](https://github.com/shamoon) ([#1641](https://github.com/paperless-ngx/paperless-ngx/pull/1641)) -- Bugfix: Files containing barcodes uploaded via web are not consumed after splitting [@stumpylog](https://github.com/stumpylog) ([#1762](https://github.com/paperless-ngx/paperless-ngx/pull/1762)) -- change default matching algo to auto and move to constant [@NiFNi](https://github.com/NiFNi) ([#1754](https://github.com/paperless-ngx/paperless-ngx/pull/1754)) -- Bugfix: Fix email labeling for non-Gmail servers [@stumpylog](https://github.com/stumpylog) ([#1755](https://github.com/paperless-ngx/paperless-ngx/pull/1755)) -- Feature: frontend update checking settings [@shamoon](https://github.com/shamoon) ([#1692](https://github.com/paperless-ngx/paperless-ngx/pull/1692)) -- Fix: allow preview for .csv files [@shamoon](https://github.com/shamoon) ([#1744](https://github.com/paperless-ngx/paperless-ngx/pull/1744)) -- Bump myst-parser from 0.18.0 to 0.18.1 [@dependabot](https://github.com/dependabot) ([#1738](https://github.com/paperless-ngx/paperless-ngx/pull/1738)) -- Bugfix: csv recognition by consumer [@bin101](https://github.com/bin101) ([#1726](https://github.com/paperless-ngx/paperless-ngx/pull/1726)) -- Bugfix: Include document title when a duplicate is detected [@stumpylog](https://github.com/stumpylog) ([#1696](https://github.com/paperless-ngx/paperless-ngx/pull/1696)) -- Bump rxjs from 7.5.6 to 7.5.7 in /src-ui [@dependabot](https://github.com/dependabot) ([#1720](https://github.com/paperless-ngx/paperless-ngx/pull/1720)) -- Bump uuid from 8.3.2 to 9.0.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1716](https://github.com/paperless-ngx/paperless-ngx/pull/1716)) -- Bump ng2-pdf-viewer from 9.1.0 to 9.1.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1717](https://github.com/paperless-ngx/paperless-ngx/pull/1717)) -- Bump ngx-color from 8.0.2 to 8.0.3 in /src-ui [@dependabot](https://github.com/dependabot) ([#1715](https://github.com/paperless-ngx/paperless-ngx/pull/1715)) -- Bump concurrently from 7.3.0 to 7.4.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1719](https://github.com/paperless-ngx/paperless-ngx/pull/1719)) -- Bump [@types/node from 18.7.14 to 18.7.23 in /src-ui @dependabot](https://github.com/types/node from 18.7.14 to 18.7.23 in /src-ui @dependabot) ([#1718](https://github.com/paperless-ngx/paperless-ngx/pull/1718)) -- Bump jest-environment-jsdom from 29.0.1 to 29.1.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1714](https://github.com/paperless-ngx/paperless-ngx/pull/1714)) -- Bump [@angular/cli @angular/core @dependabot](https://github.com/angular/cli @angular/core @dependabot) ([#1708](https://github.com/paperless-ngx/paperless-ngx/pull/1708)) -- Bump cypress from 10.7.0 to 10.9.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1707](https://github.com/paperless-ngx/paperless-ngx/pull/1707)) -- Bump bootstrap from 5.2.0 to 5.2.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1710](https://github.com/paperless-ngx/paperless-ngx/pull/1710)) -- Bump typescript from 4.7.4 to 4.8.4 in /src-ui [@dependabot](https://github.com/dependabot) ([#1706](https://github.com/paperless-ngx/paperless-ngx/pull/1706)) -- Bugfix: Set MySql charset [@stumpylog](https://github.com/stumpylog) ([#1687](https://github.com/paperless-ngx/paperless-ngx/pull/1687)) - -## paperless-ngx 1.9.2 - -### Bug Fixes - -- Bugfix: Allow PAPERLESS_OCR_CLEAN=none [@shamoon](https://github.com/shamoon) ([#1670](https://github.com/paperless-ngx/paperless-ngx/pull/1670)) - -### All App Changes - -- Chore: Bumps version numbers to 1.9.2 [@stumpylog](https://github.com/stumpylog) ([#1666](https://github.com/paperless-ngx/paperless-ngx/pull/1666)) - -## paperless-ngx 1.9.1 - -### Notes - -- Version 1.9.1 incorrectly displays the version string as 1.9.0 - -### Bug Fixes - -- Bugfix: Fixes missing OCR mode skip_noarchive [@stumpylog](https://github.com/stumpylog) ([#1645](https://github.com/paperless-ngx/paperless-ngx/pull/1645)) -- Fix reset button padding on small screens [@shamoon](https://github.com/shamoon) ([#1646](https://github.com/paperless-ngx/paperless-ngx/pull/1646)) - -### Documentation - -- Improve docs re [@janis-ax](https://github.com/janis-ax) ([#1625](https://github.com/paperless-ngx/paperless-ngx/pull/1625)) -- [Documentation] Add v1.9.0 changelog [@github-actions](https://github.com/github-actions) ([#1639](https://github.com/paperless-ngx/paperless-ngx/pull/1639)) - -### All App Changes - -- Bugfix: Fixes missing OCR mode skip_noarchive [@stumpylog](https://github.com/stumpylog) ([#1645](https://github.com/paperless-ngx/paperless-ngx/pull/1645)) -- Fix reset button padding on small screens [@shamoon](https://github.com/shamoon) ([#1646](https://github.com/paperless-ngx/paperless-ngx/pull/1646)) - -## paperless-ngx 1.9.0 - -### Features - -- Feature: Faster, less memory barcode handling [@stumpylog](https://github.com/stumpylog) ([#1594](https://github.com/paperless-ngx/paperless-ngx/pull/1594)) -- Feature: Display django-q process names [@stumpylog](https://github.com/stumpylog) ([#1567](https://github.com/paperless-ngx/paperless-ngx/pull/1567)) -- Feature: Add MariaDB support [@bckelly1](https://github.com/bckelly1) ([#543](https://github.com/paperless-ngx/paperless-ngx/pull/543)) -- Feature: Simplify IMAP login for UTF-8 [@stumpylog](https://github.com/stumpylog) ([#1492](https://github.com/paperless-ngx/paperless-ngx/pull/1492)) -- Feature: Even better re-do of OCR [@stumpylog](https://github.com/stumpylog) ([#1451](https://github.com/paperless-ngx/paperless-ngx/pull/1451)) -- Feature: document comments [@tim-vogel](https://github.com/tim-vogel) ([#1375](https://github.com/paperless-ngx/paperless-ngx/pull/1375)) -- Adding date suggestions to the documents details view [@Eckii24](https://github.com/Eckii24) ([#1367](https://github.com/paperless-ngx/paperless-ngx/pull/1367)) -- Feature: Event driven consumer [@stumpylog](https://github.com/stumpylog) ([#1421](https://github.com/paperless-ngx/paperless-ngx/pull/1421)) -- Feature: Adds storage paths to re-tagger command [@stumpylog](https://github.com/stumpylog) ([#1446](https://github.com/paperless-ngx/paperless-ngx/pull/1446)) -- Feature: Preserve original filename in metadata [@GwynHannay](https://github.com/GwynHannay) ([#1440](https://github.com/paperless-ngx/paperless-ngx/pull/1440)) -- Handle tags for gmail email accounts [@sisao](https://github.com/sisao) ([#1433](https://github.com/paperless-ngx/paperless-ngx/pull/1433)) -- Update redis image [@tribut](https://github.com/tribut) ([#1436](https://github.com/paperless-ngx/paperless-ngx/pull/1436)) -- PAPERLESS_REDIS may be set via docker secrets [@DennisGaida](https://github.com/DennisGaida) ([#1405](https://github.com/paperless-ngx/paperless-ngx/pull/1405)) - -### Bug Fixes - -- paperless_cmd.sh: use exec to run supervisord [@lemmi](https://github.com/lemmi) ([#1617](https://github.com/paperless-ngx/paperless-ngx/pull/1617)) -- Fix: Double barcode separation creates empty file [@stumpylog](https://github.com/stumpylog) ([#1596](https://github.com/paperless-ngx/paperless-ngx/pull/1596)) -- Fix: Resolve issue with slow classifier [@stumpylog](https://github.com/stumpylog) ([#1576](https://github.com/paperless-ngx/paperless-ngx/pull/1576)) -- Fix document comments not updating on document navigation [@shamoon](https://github.com/shamoon) ([#1566](https://github.com/paperless-ngx/paperless-ngx/pull/1566)) -- Fix: Include storage paths in document exporter [@shamoon](https://github.com/shamoon) ([#1557](https://github.com/paperless-ngx/paperless-ngx/pull/1557)) -- Chore: Cleanup and validate settings [@stumpylog](https://github.com/stumpylog) ([#1551](https://github.com/paperless-ngx/paperless-ngx/pull/1551)) -- Bugfix: Better gunicorn settings for workers [@stumpylog](https://github.com/stumpylog) ([#1500](https://github.com/paperless-ngx/paperless-ngx/pull/1500)) -- Fix actions button in tasks table [@shamoon](https://github.com/shamoon) ([#1488](https://github.com/paperless-ngx/paperless-ngx/pull/1488)) -- Fix: Add missing filter rule types to SavedViewFilterRule model \& fix migrations [@shamoon](https://github.com/shamoon) ([#1463](https://github.com/paperless-ngx/paperless-ngx/pull/1463)) -- Fix paperless.conf.example typo [@qcasey](https://github.com/qcasey) ([#1460](https://github.com/paperless-ngx/paperless-ngx/pull/1460)) -- Bugfix: Fixes the creation of an archive file, even if noarchive was specified [@stumpylog](https://github.com/stumpylog) ([#1442](https://github.com/paperless-ngx/paperless-ngx/pull/1442)) -- Fix: created_date should not be required [@shamoon](https://github.com/shamoon) ([#1412](https://github.com/paperless-ngx/paperless-ngx/pull/1412)) -- Fix: dev backend testing [@stumpylog](https://github.com/stumpylog) ([#1420](https://github.com/paperless-ngx/paperless-ngx/pull/1420)) -- Bugfix: Catch all exceptions during the task signals [@stumpylog](https://github.com/stumpylog) ([#1387](https://github.com/paperless-ngx/paperless-ngx/pull/1387)) -- Fix: saved view page parameter [@shamoon](https://github.com/shamoon) ([#1376](https://github.com/paperless-ngx/paperless-ngx/pull/1376)) -- Fix: Correct browser unsaved changes warning [@shamoon](https://github.com/shamoon) ([#1369](https://github.com/paperless-ngx/paperless-ngx/pull/1369)) -- Fix: correct date pasting with other formats [@shamoon](https://github.com/shamoon) ([#1370](https://github.com/paperless-ngx/paperless-ngx/pull/1370)) -- Bugfix: Allow webserver bind address to be configured [@stumpylog](https://github.com/stumpylog) ([#1358](https://github.com/paperless-ngx/paperless-ngx/pull/1358)) -- Bugfix: Chain exceptions during exception handling [@stumpylog](https://github.com/stumpylog) ([#1354](https://github.com/paperless-ngx/paperless-ngx/pull/1354)) -- Fix: missing tooltip translation \& filter editor wrapping [@shamoon](https://github.com/shamoon) ([#1305](https://github.com/paperless-ngx/paperless-ngx/pull/1305)) -- Bugfix: Interaction between barcode and directories as tags [@stumpylog](https://github.com/stumpylog) ([#1303](https://github.com/paperless-ngx/paperless-ngx/pull/1303)) - -### Documentation - -- [Beta] Paperless-ngx v1.9.0 Release Candidate [@stumpylog](https://github.com/stumpylog) ([#1560](https://github.com/paperless-ngx/paperless-ngx/pull/1560)) -- docs/configuration: Fix binary variable defaults [@erikarvstedt](https://github.com/erikarvstedt) ([#1528](https://github.com/paperless-ngx/paperless-ngx/pull/1528)) -- Info about installing on subpath [@viktor-c](https://github.com/viktor-c) ([#1350](https://github.com/paperless-ngx/paperless-ngx/pull/1350)) -- Docs: move scanner \& software recs to GH wiki [@shamoon](https://github.com/shamoon) ([#1482](https://github.com/paperless-ngx/paperless-ngx/pull/1482)) -- Docs: Update mobile scanner section [@tooomm](https://github.com/tooomm) ([#1467](https://github.com/paperless-ngx/paperless-ngx/pull/1467)) -- Adding date suggestions to the documents details view [@Eckii24](https://github.com/Eckii24) ([#1367](https://github.com/paperless-ngx/paperless-ngx/pull/1367)) -- docs: scanners: add Brother ads4700w [@ocelotsloth](https://github.com/ocelotsloth) ([#1450](https://github.com/paperless-ngx/paperless-ngx/pull/1450)) -- Feature: Adds storage paths to re-tagger command [@stumpylog](https://github.com/stumpylog) ([#1446](https://github.com/paperless-ngx/paperless-ngx/pull/1446)) -- Changes to Redis documentation [@Zerteax](https://github.com/Zerteax) ([#1441](https://github.com/paperless-ngx/paperless-ngx/pull/1441)) -- Update scanners.rst [@glassbox-sco](https://github.com/glassbox-sco) ([#1430](https://github.com/paperless-ngx/paperless-ngx/pull/1430)) -- Update scanners.rst [@derlucas](https://github.com/derlucas) ([#1415](https://github.com/paperless-ngx/paperless-ngx/pull/1415)) -- Bugfix: Allow webserver bind address to be configured [@stumpylog](https://github.com/stumpylog) ([#1358](https://github.com/paperless-ngx/paperless-ngx/pull/1358)) -- docs: fix small typo [@tooomm](https://github.com/tooomm) ([#1352](https://github.com/paperless-ngx/paperless-ngx/pull/1352)) -- [Documentation] Add v1.8.0 changelog [@github-actions](https://github.com/github-actions) ([#1298](https://github.com/paperless-ngx/paperless-ngx/pull/1298)) - -### Maintenance - -- [Beta] Paperless-ngx v1.9.0 Release Candidate [@stumpylog](https://github.com/stumpylog) ([#1560](https://github.com/paperless-ngx/paperless-ngx/pull/1560)) -- paperless_cmd.sh: use exec to run supervisord [@lemmi](https://github.com/lemmi) ([#1617](https://github.com/paperless-ngx/paperless-ngx/pull/1617)) -- Chore: Extended container image cleanup [@stumpylog](https://github.com/stumpylog) ([#1556](https://github.com/paperless-ngx/paperless-ngx/pull/1556)) -- Chore: Smaller library images [@stumpylog](https://github.com/stumpylog) ([#1546](https://github.com/paperless-ngx/paperless-ngx/pull/1546)) -- Bump tj-actions/changed-files from 24 to 29.0.2 [@dependabot](https://github.com/dependabot) ([#1493](https://github.com/paperless-ngx/paperless-ngx/pull/1493)) -- Bugfix: Better gunicorn settings for workers [@stumpylog](https://github.com/stumpylog) ([#1500](https://github.com/paperless-ngx/paperless-ngx/pull/1500)) -- [CI] Fix release drafter issues [@qcasey](https://github.com/qcasey) ([#1301](https://github.com/paperless-ngx/paperless-ngx/pull/1301)) -- Fix: dev backend testing [@stumpylog](https://github.com/stumpylog) ([#1420](https://github.com/paperless-ngx/paperless-ngx/pull/1420)) -- Chore: Exclude dependabot PRs from Project, set status to Needs Review [@qcasey](https://github.com/qcasey) ([#1397](https://github.com/paperless-ngx/paperless-ngx/pull/1397)) -- Chore: Add to label PRs based on and title [@qcasey](https://github.com/qcasey) ([#1396](https://github.com/paperless-ngx/paperless-ngx/pull/1396)) -- Chore: use pre-commit in the Ci workflow [@stumpylog](https://github.com/stumpylog) ([#1362](https://github.com/paperless-ngx/paperless-ngx/pull/1362)) -- Chore: Fixes permissions for image tag cleanup [@stumpylog](https://github.com/stumpylog) ([#1315](https://github.com/paperless-ngx/paperless-ngx/pull/1315)) -- Bump leonsteinhaeuser/project-beta-automations from 1.2.1 to 1.3.0 [@dependabot](https://github.com/dependabot) ([#1328](https://github.com/paperless-ngx/paperless-ngx/pull/1328)) -- Bump tj-actions/changed-files from 23.1 to 24 [@dependabot](https://github.com/dependabot) ([#1329](https://github.com/paperless-ngx/paperless-ngx/pull/1329)) -- Feature: Remove requirements.txt and use pipenv everywhere [@stumpylog](https://github.com/stumpylog) ([#1316](https://github.com/paperless-ngx/paperless-ngx/pull/1316)) - -### Dependencies - -
-34 changes - -- Bump pikepdf from 5.5.0 to 5.6.1 [@dependabot](https://github.com/dependabot) ([#1537](https://github.com/paperless-ngx/paperless-ngx/pull/1537)) -- Bump black from 22.6.0 to 22.8.0 [@dependabot](https://github.com/dependabot) ([#1539](https://github.com/paperless-ngx/paperless-ngx/pull/1539)) -- Bump tqdm from 4.64.0 to 4.64.1 [@dependabot](https://github.com/dependabot) ([#1540](https://github.com/paperless-ngx/paperless-ngx/pull/1540)) -- Bump pytest from 7.1.2 to 7.1.3 [@dependabot](https://github.com/dependabot) ([#1538](https://github.com/paperless-ngx/paperless-ngx/pull/1538)) -- Bump tj-actions/changed-files from 24 to 29.0.2 [@dependabot](https://github.com/dependabot) ([#1493](https://github.com/paperless-ngx/paperless-ngx/pull/1493)) -- Bump angular packages, jest-preset-angular in src-ui [@dependabot](https://github.com/dependabot) ([#1502](https://github.com/paperless-ngx/paperless-ngx/pull/1502)) -- Bump jest-environment-jsdom from 28.1.3 to 29.0.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1507](https://github.com/paperless-ngx/paperless-ngx/pull/1507)) -- Bump [@types/node from 18.6.3 to 18.7.14 in /src-ui @dependabot](https://github.com/types/node from 18.6.3 to 18.7.14 in /src-ui @dependabot) ([#1506](https://github.com/paperless-ngx/paperless-ngx/pull/1506)) -- Bump [@angular-builders/jest from 14.0.0 to 14.0.1 in /src-ui @dependabot](https://github.com/angular-builders/jest from 14.0.0 to 14.0.1 in /src-ui @dependabot) ([#1505](https://github.com/paperless-ngx/paperless-ngx/pull/1505)) -- Bump zone.js from 0.11.7 to 0.11.8 in /src-ui [@dependabot](https://github.com/dependabot) ([#1504](https://github.com/paperless-ngx/paperless-ngx/pull/1504)) -- Bump ngx-color from 8.0.1 to 8.0.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1494](https://github.com/paperless-ngx/paperless-ngx/pull/1494)) -- Bump cypress from 10.3.1 to 10.7.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1496](https://github.com/paperless-ngx/paperless-ngx/pull/1496)) -- Bump [@cypress/schematic from 2.0.0 to 2.1.1 in /src-ui @dependabot](https://github.com/cypress/schematic from 2.0.0 to 2.1.1 in /src-ui @dependabot) ([#1495](https://github.com/paperless-ngx/paperless-ngx/pull/1495)) -- Bump [@popperjs/core from 2.11.5 to 2.11.6 in /src-ui @dependabot](https://github.com/popperjs/core from 2.11.5 to 2.11.6 in /src-ui @dependabot) ([#1498](https://github.com/paperless-ngx/paperless-ngx/pull/1498)) -- Bump sphinx from 5.0.2 to 5.1.1 [@dependabot](https://github.com/dependabot) ([#1297](https://github.com/paperless-ngx/paperless-ngx/pull/1297)) -- Chore: Bump Python dependencies [@stumpylog](https://github.com/stumpylog) ([#1445](https://github.com/paperless-ngx/paperless-ngx/pull/1445)) -- Chore: Update Python deps [@stumpylog](https://github.com/stumpylog) ([#1391](https://github.com/paperless-ngx/paperless-ngx/pull/1391)) -- Bump watchfiles from 0.15.0 to 0.16.1 [@dependabot](https://github.com/dependabot) ([#1285](https://github.com/paperless-ngx/paperless-ngx/pull/1285)) -- Bump leonsteinhaeuser/project-beta-automations from 1.2.1 to 1.3.0 [@dependabot](https://github.com/dependabot) ([#1328](https://github.com/paperless-ngx/paperless-ngx/pull/1328)) -- Bump tj-actions/changed-files from 23.1 to 24 [@dependabot](https://github.com/dependabot) ([#1329](https://github.com/paperless-ngx/paperless-ngx/pull/1329)) -- Bump cypress from 10.3.0 to 10.3.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1342](https://github.com/paperless-ngx/paperless-ngx/pull/1342)) -- Bump ngx-color from 7.3.3 to 8.0.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1343](https://github.com/paperless-ngx/paperless-ngx/pull/1343)) -- Bump [@angular/cli from 14.0.4 to 14.1.0 in /src-ui @dependabot](https://github.com/angular/cli from 14.0.4 to 14.1.0 in /src-ui @dependabot) ([#1330](https://github.com/paperless-ngx/paperless-ngx/pull/1330)) -- Bump [@types/node from 18.0.0 to 18.6.3 in /src-ui @dependabot](https://github.com/types/node from 18.0.0 to 18.6.3 in /src-ui @dependabot) ([#1341](https://github.com/paperless-ngx/paperless-ngx/pull/1341)) -- Bump jest-preset-angular from 12.1.0 to 12.2.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1340](https://github.com/paperless-ngx/paperless-ngx/pull/1340)) -- Bump concurrently from 7.2.2 to 7.3.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1326](https://github.com/paperless-ngx/paperless-ngx/pull/1326)) -- Bump ng2-pdf-viewer from 9.0.0 to 9.1.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1337](https://github.com/paperless-ngx/paperless-ngx/pull/1337)) -- Bump jest-environment-jsdom from 28.1.2 to 28.1.3 in /src-ui [@dependabot](https://github.com/dependabot) ([#1336](https://github.com/paperless-ngx/paperless-ngx/pull/1336)) -- Bump ngx-file-drop from 13.0.0 to 14.0.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1331](https://github.com/paperless-ngx/paperless-ngx/pull/1331)) -- Bump jest and [@types/jest in /src-ui @dependabot](https://github.com/types/jest in /src-ui @dependabot) ([#1333](https://github.com/paperless-ngx/paperless-ngx/pull/1333)) -- Bump bootstrap from 5.1.3 to 5.2.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1327](https://github.com/paperless-ngx/paperless-ngx/pull/1327)) -- Bump typescript from 4.6.4 to 4.7.4 in /src-ui [@dependabot](https://github.com/dependabot) ([#1324](https://github.com/paperless-ngx/paperless-ngx/pull/1324)) -- Bump ts-node from 10.8.1 to 10.9.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1325](https://github.com/paperless-ngx/paperless-ngx/pull/1325)) -- Bump rxjs from 7.5.5 to 7.5.6 in /src-ui [@dependabot](https://github.com/dependabot) ([#1323](https://github.com/paperless-ngx/paperless-ngx/pull/1323)) -
- -### All App Changes - -- [Beta] Paperless-ngx v1.9.0 Release Candidate [@stumpylog](https://github.com/stumpylog) ([#1560](https://github.com/paperless-ngx/paperless-ngx/pull/1560)) -- Feature: Faster, less memory barcode handling [@stumpylog](https://github.com/stumpylog) ([#1594](https://github.com/paperless-ngx/paperless-ngx/pull/1594)) -- Fix: Consume directory permissions were not updated [@stumpylog](https://github.com/stumpylog) ([#1605](https://github.com/paperless-ngx/paperless-ngx/pull/1605)) -- Fix: Double barcode separation creates empty file [@stumpylog](https://github.com/stumpylog) ([#1596](https://github.com/paperless-ngx/paperless-ngx/pull/1596)) -- Fix: Parsing Tika documents fails with AttributeError [@stumpylog](https://github.com/stumpylog) ([#1591](https://github.com/paperless-ngx/paperless-ngx/pull/1591)) -- Fix: Resolve issue with slow classifier [@stumpylog](https://github.com/stumpylog) ([#1576](https://github.com/paperless-ngx/paperless-ngx/pull/1576)) -- Feature: Display django-q process names [@stumpylog](https://github.com/stumpylog) ([#1567](https://github.com/paperless-ngx/paperless-ngx/pull/1567)) -- Fix document comments not updating on document navigation [@shamoon](https://github.com/shamoon) ([#1566](https://github.com/paperless-ngx/paperless-ngx/pull/1566)) -- Feature: Add MariaDB support [@bckelly1](https://github.com/bckelly1) ([#543](https://github.com/paperless-ngx/paperless-ngx/pull/543)) -- Fix: Include storage paths in document exporter [@shamoon](https://github.com/shamoon) ([#1557](https://github.com/paperless-ngx/paperless-ngx/pull/1557)) -- Chore: Cleanup and validate settings [@stumpylog](https://github.com/stumpylog) ([#1551](https://github.com/paperless-ngx/paperless-ngx/pull/1551)) -- Bump pikepdf from 5.5.0 to 5.6.1 [@dependabot](https://github.com/dependabot) ([#1537](https://github.com/paperless-ngx/paperless-ngx/pull/1537)) -- Bump black from 22.6.0 to 22.8.0 [@dependabot](https://github.com/dependabot) ([#1539](https://github.com/paperless-ngx/paperless-ngx/pull/1539)) -- Bump tqdm from 4.64.0 to 4.64.1 [@dependabot](https://github.com/dependabot) ([#1540](https://github.com/paperless-ngx/paperless-ngx/pull/1540)) -- Bump pytest from 7.1.2 to 7.1.3 [@dependabot](https://github.com/dependabot) ([#1538](https://github.com/paperless-ngx/paperless-ngx/pull/1538)) -- Bump angular packages, jest-preset-angular in src-ui [@dependabot](https://github.com/dependabot) ([#1502](https://github.com/paperless-ngx/paperless-ngx/pull/1502)) -- Bump jest-environment-jsdom from 28.1.3 to 29.0.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1507](https://github.com/paperless-ngx/paperless-ngx/pull/1507)) -- Bump [@types/node from 18.6.3 to 18.7.14 in /src-ui @dependabot](https://github.com/types/node from 18.6.3 to 18.7.14 in /src-ui @dependabot) ([#1506](https://github.com/paperless-ngx/paperless-ngx/pull/1506)) -- Bump [@angular-builders/jest from 14.0.0 to 14.0.1 in /src-ui @dependabot](https://github.com/angular-builders/jest from 14.0.0 to 14.0.1 in /src-ui @dependabot) ([#1505](https://github.com/paperless-ngx/paperless-ngx/pull/1505)) -- Bump zone.js from 0.11.7 to 0.11.8 in /src-ui [@dependabot](https://github.com/dependabot) ([#1504](https://github.com/paperless-ngx/paperless-ngx/pull/1504)) -- Bump ngx-color from 8.0.1 to 8.0.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1494](https://github.com/paperless-ngx/paperless-ngx/pull/1494)) -- Bump cypress from 10.3.1 to 10.7.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1496](https://github.com/paperless-ngx/paperless-ngx/pull/1496)) -- Bump [@cypress/schematic from 2.0.0 to 2.1.1 in /src-ui @dependabot](https://github.com/cypress/schematic from 2.0.0 to 2.1.1 in /src-ui @dependabot) ([#1495](https://github.com/paperless-ngx/paperless-ngx/pull/1495)) -- Bump [@popperjs/core from 2.11.5 to 2.11.6 in /src-ui @dependabot](https://github.com/popperjs/core from 2.11.5 to 2.11.6 in /src-ui @dependabot) ([#1498](https://github.com/paperless-ngx/paperless-ngx/pull/1498)) -- Feature: Simplify IMAP login for UTF-8 [@stumpylog](https://github.com/stumpylog) ([#1492](https://github.com/paperless-ngx/paperless-ngx/pull/1492)) -- Fix actions button in tasks table [@shamoon](https://github.com/shamoon) ([#1488](https://github.com/paperless-ngx/paperless-ngx/pull/1488)) -- Fix: Add missing filter rule types to SavedViewFilterRule model \& fix migrations [@shamoon](https://github.com/shamoon) ([#1463](https://github.com/paperless-ngx/paperless-ngx/pull/1463)) -- Feature: Even better re-do of OCR [@stumpylog](https://github.com/stumpylog) ([#1451](https://github.com/paperless-ngx/paperless-ngx/pull/1451)) -- Feature: document comments [@tim-vogel](https://github.com/tim-vogel) ([#1375](https://github.com/paperless-ngx/paperless-ngx/pull/1375)) -- Adding date suggestions to the documents details view [@Eckii24](https://github.com/Eckii24) ([#1367](https://github.com/paperless-ngx/paperless-ngx/pull/1367)) -- Bump sphinx from 5.0.2 to 5.1.1 [@dependabot](https://github.com/dependabot) ([#1297](https://github.com/paperless-ngx/paperless-ngx/pull/1297)) -- Feature: Event driven consumer [@stumpylog](https://github.com/stumpylog) ([#1421](https://github.com/paperless-ngx/paperless-ngx/pull/1421)) -- Bugfix: Fixes the creation of an archive file, even if noarchive was specified [@stumpylog](https://github.com/stumpylog) ([#1442](https://github.com/paperless-ngx/paperless-ngx/pull/1442)) -- Feature: Adds storage paths to re-tagger command [@stumpylog](https://github.com/stumpylog) ([#1446](https://github.com/paperless-ngx/paperless-ngx/pull/1446)) -- Feature: Preserve original filename in metadata [@GwynHannay](https://github.com/GwynHannay) ([#1440](https://github.com/paperless-ngx/paperless-ngx/pull/1440)) -- Handle tags for gmail email accounts [@sisao](https://github.com/sisao) ([#1433](https://github.com/paperless-ngx/paperless-ngx/pull/1433)) -- Fix: should not be required [@shamoon](https://github.com/shamoon) ([#1412](https://github.com/paperless-ngx/paperless-ngx/pull/1412)) -- Bugfix: Catch all exceptions during the task signals [@stumpylog](https://github.com/stumpylog) ([#1387](https://github.com/paperless-ngx/paperless-ngx/pull/1387)) -- Fix: saved view page parameter [@shamoon](https://github.com/shamoon) ([#1376](https://github.com/paperless-ngx/paperless-ngx/pull/1376)) -- Fix: Correct browser unsaved changes warning [@shamoon](https://github.com/shamoon) ([#1369](https://github.com/paperless-ngx/paperless-ngx/pull/1369)) -- Fix: correct date pasting with other formats [@shamoon](https://github.com/shamoon) ([#1370](https://github.com/paperless-ngx/paperless-ngx/pull/1370)) -- Chore: use pre-commit in the Ci workflow [@stumpylog](https://github.com/stumpylog) ([#1362](https://github.com/paperless-ngx/paperless-ngx/pull/1362)) -- Bugfix: Chain exceptions during exception handling [@stumpylog](https://github.com/stumpylog) ([#1354](https://github.com/paperless-ngx/paperless-ngx/pull/1354)) -- Bump watchfiles from 0.15.0 to 0.16.1 [@dependabot](https://github.com/dependabot) ([#1285](https://github.com/paperless-ngx/paperless-ngx/pull/1285)) -- Bump cypress from 10.3.0 to 10.3.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1342](https://github.com/paperless-ngx/paperless-ngx/pull/1342)) -- Bump ngx-color from 7.3.3 to 8.0.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1343](https://github.com/paperless-ngx/paperless-ngx/pull/1343)) -- Bump [@angular/cli from 14.0.4 to 14.1.0 in /src-ui @dependabot](https://github.com/angular/cli from 14.0.4 to 14.1.0 in /src-ui @dependabot) ([#1330](https://github.com/paperless-ngx/paperless-ngx/pull/1330)) -- Bump [@types/node from 18.0.0 to 18.6.3 in /src-ui @dependabot](https://github.com/types/node from 18.0.0 to 18.6.3 in /src-ui @dependabot) ([#1341](https://github.com/paperless-ngx/paperless-ngx/pull/1341)) -- Bump jest-preset-angular from 12.1.0 to 12.2.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1340](https://github.com/paperless-ngx/paperless-ngx/pull/1340)) -- Bump concurrently from 7.2.2 to 7.3.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1326](https://github.com/paperless-ngx/paperless-ngx/pull/1326)) -- Bump ng2-pdf-viewer from 9.0.0 to 9.1.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1337](https://github.com/paperless-ngx/paperless-ngx/pull/1337)) -- Bump jest-environment-jsdom from 28.1.2 to 28.1.3 in /src-ui [@dependabot](https://github.com/dependabot) ([#1336](https://github.com/paperless-ngx/paperless-ngx/pull/1336)) -- Bump ngx-file-drop from 13.0.0 to 14.0.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1331](https://github.com/paperless-ngx/paperless-ngx/pull/1331)) -- Bump jest and [@types/jest in /src-ui @dependabot](https://github.com/types/jest in /src-ui @dependabot) ([#1333](https://github.com/paperless-ngx/paperless-ngx/pull/1333)) -- Bump bootstrap from 5.1.3 to 5.2.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1327](https://github.com/paperless-ngx/paperless-ngx/pull/1327)) -- Bump typescript from 4.6.4 to 4.7.4 in /src-ui [@dependabot](https://github.com/dependabot) ([#1324](https://github.com/paperless-ngx/paperless-ngx/pull/1324)) -- Bump ts-node from 10.8.1 to 10.9.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1325](https://github.com/paperless-ngx/paperless-ngx/pull/1325)) -- Bump rxjs from 7.5.5 to 7.5.6 in /src-ui [@dependabot](https://github.com/dependabot) ([#1323](https://github.com/paperless-ngx/paperless-ngx/pull/1323)) -- Fix: missing tooltip translation \& filter editor wrapping [@shamoon](https://github.com/shamoon) ([#1305](https://github.com/paperless-ngx/paperless-ngx/pull/1305)) -- Feature: Remove requirements.txt and use pipenv everywhere [@stumpylog](https://github.com/stumpylog) ([#1316](https://github.com/paperless-ngx/paperless-ngx/pull/1316)) -- Bugfix: Interaction between barcode and directories as tags [@stumpylog](https://github.com/stumpylog) ([#1303](https://github.com/paperless-ngx/paperless-ngx/pull/1303)) - -## paperless-ngx 1.8.0 - -### Features - -- Feature use env vars in pre post scripts [@ziprandom](https://github.com/ziprandom) ([#1154](https://github.com/paperless-ngx/paperless-ngx/pull/1154)) -- frontend task queue [@shamoon](https://github.com/shamoon) ([#1020](https://github.com/paperless-ngx/paperless-ngx/pull/1020)) -- Fearless scikit-learn updates [@stumpylog](https://github.com/stumpylog) ([#1082](https://github.com/paperless-ngx/paperless-ngx/pull/1082)) -- Adds support for Docker secrets [@stumpylog](https://github.com/stumpylog) ([#1034](https://github.com/paperless-ngx/paperless-ngx/pull/1034)) -- make frontend timezone un-aware [@shamoon](https://github.com/shamoon) ([#957](https://github.com/paperless-ngx/paperless-ngx/pull/957)) -- Change document thumbnails to WebP [@stumpylog](https://github.com/stumpylog) ([#1127](https://github.com/paperless-ngx/paperless-ngx/pull/1127)) -- Fork django-q to update dependencies [@stumpylog](https://github.com/stumpylog) ([#1014](https://github.com/paperless-ngx/paperless-ngx/pull/1014)) -- Fix: Rework query params logic [@shamoon](https://github.com/shamoon) ([#1000](https://github.com/paperless-ngx/paperless-ngx/pull/1000)) -- Enhancement: show note on language change and offer reload [@shamoon](https://github.com/shamoon) ([#1030](https://github.com/paperless-ngx/paperless-ngx/pull/1030)) -- Include error information when Redis connection fails [@stumpylog](https://github.com/stumpylog) ([#1016](https://github.com/paperless-ngx/paperless-ngx/pull/1016)) -- frontend settings saved to database [@shamoon](https://github.com/shamoon) ([#919](https://github.com/paperless-ngx/paperless-ngx/pull/919)) -- Add "Created" as additional (optional) parameter for post_documents [@eingemaischt](https://github.com/eingemaischt) ([#965](https://github.com/paperless-ngx/paperless-ngx/pull/965)) -- Convert Changelog to markdown, auto-commit future changelogs [@qcasey](https://github.com/qcasey) ([#935](https://github.com/paperless-ngx/paperless-ngx/pull/935)) -- allow all ASN filtering functions [@shamoon](https://github.com/shamoon) ([#920](https://github.com/paperless-ngx/paperless-ngx/pull/920)) -- gunicorn: Allow IPv6 sockets [@vlcty](https://github.com/vlcty) ([#924](https://github.com/paperless-ngx/paperless-ngx/pull/924)) -- initial app loading indicators [@shamoon](https://github.com/shamoon) ([#899](https://github.com/paperless-ngx/paperless-ngx/pull/899)) - -### Bug Fixes - -- Fix: dropdown selected items not visible again [@shamoon](https://github.com/shamoon) ([#1261](https://github.com/paperless-ngx/paperless-ngx/pull/1261)) -- [CI] Fix automatic changelog generation on release [@qcasey](https://github.com/qcasey) ([#1249](https://github.com/paperless-ngx/paperless-ngx/pull/1249)) -- Fix: Prevent duplicate api calls on text filtering [@shamoon](https://github.com/shamoon) ([#1133](https://github.com/paperless-ngx/paperless-ngx/pull/1133)) -- make frontend timezone un-aware [@shamoon](https://github.com/shamoon) ([#957](https://github.com/paperless-ngx/paperless-ngx/pull/957)) -- Feature / fix quick toggleable filters [@shamoon](https://github.com/shamoon) ([#1122](https://github.com/paperless-ngx/paperless-ngx/pull/1122)) -- Chore: Manually downgrade reportlab (and update everything else) [@stumpylog](https://github.com/stumpylog) ([#1116](https://github.com/paperless-ngx/paperless-ngx/pull/1116)) -- Bugfix: Don't assume default Docker folders [@stumpylog](https://github.com/stumpylog) ([#1088](https://github.com/paperless-ngx/paperless-ngx/pull/1088)) -- Bugfix: Better sanity check messages [@stumpylog](https://github.com/stumpylog) ([#1049](https://github.com/paperless-ngx/paperless-ngx/pull/1049)) -- Fix vertical margins between pages of pdf viewer [@shamoon](https://github.com/shamoon) ([#1081](https://github.com/paperless-ngx/paperless-ngx/pull/1081)) -- Bugfix: Pass debug setting on to django-q [@stumpylog](https://github.com/stumpylog) ([#1058](https://github.com/paperless-ngx/paperless-ngx/pull/1058)) -- Bugfix: Don't assume the document has a title set [@stumpylog](https://github.com/stumpylog) ([#1057](https://github.com/paperless-ngx/paperless-ngx/pull/1057)) -- Bugfix: Corrects the setting of max pixel size for OCR [@stumpylog](https://github.com/stumpylog) ([#1008](https://github.com/paperless-ngx/paperless-ngx/pull/1008)) -- better date pasting [@shamoon](https://github.com/shamoon) ([#1007](https://github.com/paperless-ngx/paperless-ngx/pull/1007)) -- Enhancement: Alphabetize tags by default [@shamoon](https://github.com/shamoon) ([#1017](https://github.com/paperless-ngx/paperless-ngx/pull/1017)) -- Fix: Rework query params logic [@shamoon](https://github.com/shamoon) ([#1000](https://github.com/paperless-ngx/paperless-ngx/pull/1000)) -- Fix: add translation for some un-translated tooltips [@shamoon](https://github.com/shamoon) ([#995](https://github.com/paperless-ngx/paperless-ngx/pull/995)) -- Change npm --no-optional to --omit=optional [@shamoon](https://github.com/shamoon) ([#986](https://github.com/paperless-ngx/paperless-ngx/pull/986)) -- Add `myst-parser` to fix readthedocs [@qcasey](https://github.com/qcasey) ([#982](https://github.com/paperless-ngx/paperless-ngx/pull/982)) -- Fix: Title is changed after switching doc quickly [@shamoon](https://github.com/shamoon) ([#979](https://github.com/paperless-ngx/paperless-ngx/pull/979)) -- Fix: warn when closing a document with unsaved changes due to max open docs [@shamoon](https://github.com/shamoon) ([#956](https://github.com/paperless-ngx/paperless-ngx/pull/956)) -- Bugfix: Adds configurable intoify debounce time [@stumpylog](https://github.com/stumpylog) ([#953](https://github.com/paperless-ngx/paperless-ngx/pull/953)) -- Bugfix: Fixes document filename date off by 1 issue [@stumpylog](https://github.com/stumpylog) ([#942](https://github.com/paperless-ngx/paperless-ngx/pull/942)) -- fixes #949: change to MIME detection for files [@gador](https://github.com/gador) ([#962](https://github.com/paperless-ngx/paperless-ngx/pull/962)) -- docs: fix some typos [@Berjou](https://github.com/Berjou) ([#948](https://github.com/paperless-ngx/paperless-ngx/pull/948)) -- [Docs] Fix 2 small typos [@tooomm](https://github.com/tooomm) ([#946](https://github.com/paperless-ngx/paperless-ngx/pull/946)) -- [Readme] Fix typo [@tooomm](https://github.com/tooomm) ([#941](https://github.com/paperless-ngx/paperless-ngx/pull/941)) -- Fix: management pages plurals incorrect in other languages [@shamoon](https://github.com/shamoon) ([#939](https://github.com/paperless-ngx/paperless-ngx/pull/939)) -- Fix: v1.7.1 frontend visual fixes [@shamoon](https://github.com/shamoon) ([#933](https://github.com/paperless-ngx/paperless-ngx/pull/933)) -- Fix: unassigned query params ignored [@shamoon](https://github.com/shamoon) ([#930](https://github.com/paperless-ngx/paperless-ngx/pull/930)) -- Fix: allow commas in non-multi rules query params [@shamoon](https://github.com/shamoon) ([#923](https://github.com/paperless-ngx/paperless-ngx/pull/923)) -- Fix: Include version in export for better error messages [@stumpylog](https://github.com/stumpylog) ([#883](https://github.com/paperless-ngx/paperless-ngx/pull/883)) -- Bugfix: Superuser Management Won't Reset Password [@stumpylog](https://github.com/stumpylog) ([#903](https://github.com/paperless-ngx/paperless-ngx/pull/903)) -- Fix Ignore Date Parsing [@stumpylog](https://github.com/stumpylog) ([#721](https://github.com/paperless-ngx/paperless-ngx/pull/721)) - -### Documentation - -- Feature use env vars in pre post scripts [@ziprandom](https://github.com/ziprandom) ([#1154](https://github.com/paperless-ngx/paperless-ngx/pull/1154)) -- Add `myst-parser` to fix readthedocs [@qcasey](https://github.com/qcasey) ([#982](https://github.com/paperless-ngx/paperless-ngx/pull/982)) -- Add "Created" as additional (optional) parameter for post_documents [@eingemaischt](https://github.com/eingemaischt) ([#965](https://github.com/paperless-ngx/paperless-ngx/pull/965)) -- Bugfix: Adds configurable intoify debounce time [@stumpylog](https://github.com/stumpylog) ([#953](https://github.com/paperless-ngx/paperless-ngx/pull/953)) -- docs: fix some typos [@Berjou](https://github.com/Berjou) ([#948](https://github.com/paperless-ngx/paperless-ngx/pull/948)) -- [Docs] Fix 2 small typos [@tooomm](https://github.com/tooomm) ([#946](https://github.com/paperless-ngx/paperless-ngx/pull/946)) -- Convert Changelog to markdown, auto-commit future changelogs [@qcasey](https://github.com/qcasey) ([#935](https://github.com/paperless-ngx/paperless-ngx/pull/935)) -- [Readme] Fix typo [@tooomm](https://github.com/tooomm) ([#941](https://github.com/paperless-ngx/paperless-ngx/pull/941)) - -### Maintenance - -- Adds support for Docker secrets [@stumpylog](https://github.com/stumpylog) ([#1034](https://github.com/paperless-ngx/paperless-ngx/pull/1034)) -- Bugfix: Don't assume default Docker folders [@stumpylog](https://github.com/stumpylog) ([#1088](https://github.com/paperless-ngx/paperless-ngx/pull/1088)) -- Include error information when Redis connection fails [@stumpylog](https://github.com/stumpylog) ([#1016](https://github.com/paperless-ngx/paperless-ngx/pull/1016)) -- Fix: add translation for some un-translated tooltips [@shamoon](https://github.com/shamoon) ([#995](https://github.com/paperless-ngx/paperless-ngx/pull/995)) -- gunicorn: Allow IPv6 sockets [@vlcty](https://github.com/vlcty) ([#924](https://github.com/paperless-ngx/paperless-ngx/pull/924)) - -### Dependencies - -
-34 changes - -- Fearless scikit-learn updates [@stumpylog](https://github.com/stumpylog) ([#1082](https://github.com/paperless-ngx/paperless-ngx/pull/1082)) -- Bump pillow from 9.1.1 to 9.2.0 [@dependabot](https://github.com/dependabot) ([#1193](https://github.com/paperless-ngx/paperless-ngx/pull/1193)) -- Bump watchdog from 2.1.8 to 2.1.9 [@dependabot](https://github.com/dependabot) ([#1132](https://github.com/paperless-ngx/paperless-ngx/pull/1132)) -- Bump scikit-learn from 1.0.2 to 1.1.1 [@dependabot](https://github.com/dependabot) ([#992](https://github.com/paperless-ngx/paperless-ngx/pull/992)) -- Bump setuptools from 62.3.3 to 62.6.0 [@dependabot](https://github.com/dependabot) ([#1150](https://github.com/paperless-ngx/paperless-ngx/pull/1150)) -- Bump django-filter from 21.1 to 22.1 [@dependabot](https://github.com/dependabot) ([#1191](https://github.com/paperless-ngx/paperless-ngx/pull/1191)) -- Bump actions/setup-python from 3 to 4 [@dependabot](https://github.com/dependabot) ([#1176](https://github.com/paperless-ngx/paperless-ngx/pull/1176)) -- Bump sphinx from 4.5.0 to 5.0.2 [@dependabot](https://github.com/dependabot) ([#1151](https://github.com/paperless-ngx/paperless-ngx/pull/1151)) -- Bump docker/metadata-action from 3 to 4 [@dependabot](https://github.com/dependabot) ([#1178](https://github.com/paperless-ngx/paperless-ngx/pull/1178)) -- Bump tj-actions/changed-files from 22.1 to 23.1 [@dependabot](https://github.com/dependabot) ([#1179](https://github.com/paperless-ngx/paperless-ngx/pull/1179)) -- Bump @angular/cli from 13.3.7 to 14.0.4 in /src-ui [@dependabot](https://github.com/dependabot) ([#1177](https://github.com/paperless-ngx/paperless-ngx/pull/1177)) -- Bump cypress from 10.0.1 to 10.3.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1187](https://github.com/paperless-ngx/paperless-ngx/pull/1187)) -- Bump zone.js from 0.11.5 to 0.11.6 in /src-ui [@dependabot](https://github.com/dependabot) ([#1185](https://github.com/paperless-ngx/paperless-ngx/pull/1185)) -- Bump ts-node from 10.8.0 to 10.8.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1184](https://github.com/paperless-ngx/paperless-ngx/pull/1184)) -- Bump jest-environment-jsdom from 28.1.0 to 28.1.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1175](https://github.com/paperless-ngx/paperless-ngx/pull/1175)) -- Bump @types/node from 17.0.38 to 18.0.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1183](https://github.com/paperless-ngx/paperless-ngx/pull/1183)) -- Bump concurrently from 7.2.1 to 7.2.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1181](https://github.com/paperless-ngx/paperless-ngx/pull/1181)) -- Bump jest-preset-angular from 12.0.1 to 12.1.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1182](https://github.com/paperless-ngx/paperless-ngx/pull/1182)) -- Bump jest and @types/jest in /src-ui [@dependabot](https://github.com/dependabot) ([#1180](https://github.com/paperless-ngx/paperless-ngx/pull/1180)) -- Bump whitenoise from 6.1.0 to 6.2.0 [@dependabot](https://github.com/dependabot) ([#1103](https://github.com/paperless-ngx/paperless-ngx/pull/1103)) -- Bump cypress from 9.6.1 to 10.0.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1083](https://github.com/paperless-ngx/paperless-ngx/pull/1083)) -- Bump docker/setup-qemu-action from 1 to 2 [@dependabot](https://github.com/dependabot) ([#1065](https://github.com/paperless-ngx/paperless-ngx/pull/1065)) -- Bump docker/setup-buildx-action from 1 to 2 [@dependabot](https://github.com/dependabot) ([#1064](https://github.com/paperless-ngx/paperless-ngx/pull/1064)) -- Bump docker/build-push-action from 2 to 3 [@dependabot](https://github.com/dependabot) ([#1063](https://github.com/paperless-ngx/paperless-ngx/pull/1063)) -- Bump @cypress/schematic from 1.7.0 to 2.0.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1075](https://github.com/paperless-ngx/paperless-ngx/pull/1075)) -- Bump tj-actions/changed-files from 19 to 22.1 [@dependabot](https://github.com/dependabot) ([#1062](https://github.com/paperless-ngx/paperless-ngx/pull/1062)) -- Bump concurrently from 7.1.0 to 7.2.1 in /src-ui [@dependabot](https://github.com/dependabot) ([#1073](https://github.com/paperless-ngx/paperless-ngx/pull/1073)) -- Bump @types/jest from 27.4.1 to 27.5.2 in /src-ui [@dependabot](https://github.com/dependabot) ([#1074](https://github.com/paperless-ngx/paperless-ngx/pull/1074)) -- Bump ts-node from 10.7.0 to 10.8.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1070](https://github.com/paperless-ngx/paperless-ngx/pull/1070)) -- Bump jest from 28.0.3 to 28.1.0 in /src-ui [@dependabot](https://github.com/dependabot) ([#1071](https://github.com/paperless-ngx/paperless-ngx/pull/1071)) -- Chore: npm package updates 22-06-01 [@shamoon](https://github.com/shamoon) ([#1069](https://github.com/paperless-ngx/paperless-ngx/pull/1069)) -- Bump docker/login-action from 1 to 2 [@dependabot](https://github.com/dependabot) ([#1061](https://github.com/paperless-ngx/paperless-ngx/pull/1061)) -- Chore: Manually update dependencies [@stumpylog](https://github.com/stumpylog) ([#1013](https://github.com/paperless-ngx/paperless-ngx/pull/1013)) -- Chore: Manually update all Python dependencies [@stumpylog](https://github.com/stumpylog) ([#973](https://github.com/paperless-ngx/paperless-ngx/pull/973)) -
- -## paperless-ngx 1.7.1 - -### Features - -- (chore) Runs pyupgrade to Python 3.8+ [\@stumpylog](https://github.com/stumpylog) ([\#890](https://github.com/paperless-ngx/paperless-ngx/pull/890)) -- Dockerfile Organization \& Enhancements [\@stumpylog](https://github.com/stumpylog) ([\#888](https://github.com/paperless-ngx/paperless-ngx/pull/888)) -- mobile friendlier manage pages [\@shamoon](https://github.com/shamoon) ([\#873](https://github.com/paperless-ngx/paperless-ngx/pull/873)) -- Use semver for release process [\@stumpylog](https://github.com/stumpylog) ([\#851](https://github.com/paperless-ngx/paperless-ngx/pull/851)) -- Enable Docker Hub push [\@stumpylog](https://github.com/stumpylog) ([\#828](https://github.com/paperless-ngx/paperless-ngx/pull/828)) -- Feature barcode tiff support [\@gador](https://github.com/gador) ([\#766](https://github.com/paperless-ngx/paperless-ngx/pull/766)) -- Updates GHA workflow to rebuild intermediate images on changes [\@stumpylog](https://github.com/stumpylog) ([\#820](https://github.com/paperless-ngx/paperless-ngx/pull/820)) -- Adds simple Python to wait for Redis broker to be ready [\@stumpylog](https://github.com/stumpylog) ([\#788](https://github.com/paperless-ngx/paperless-ngx/pull/788)) -- Update GHA workflow to build all Docker images [\@stumpylog](https://github.com/stumpylog) ([\#761](https://github.com/paperless-ngx/paperless-ngx/pull/761)) - -### Bug Fixes - -- Feature / fix saved view \& sort field query params [\@shamoon](https://github.com/shamoon) ([\#881](https://github.com/paperless-ngx/paperless-ngx/pull/881)) -- Mobile friendlier manage pages [\@shamoon](https://github.com/shamoon) ([\#873](https://github.com/paperless-ngx/paperless-ngx/pull/873)) -- Add timeout to healthcheck [\@shamoon](https://github.com/shamoon) ([\#880](https://github.com/paperless-ngx/paperless-ngx/pull/880)) -- Always accept yyyy-mm-dd date inputs [\@shamoon](https://github.com/shamoon) ([\#864](https://github.com/paperless-ngx/paperless-ngx/pull/864)) -- Fix local Docker image building [\@stumpylog](https://github.com/stumpylog) ([\#849](https://github.com/paperless-ngx/paperless-ngx/pull/849)) -- Fix: show errors on invalid date input [\@shamoon](https://github.com/shamoon) ([\#862](https://github.com/paperless-ngx/paperless-ngx/pull/862)) -- Fix: Older dates do not display on frontend [\@shamoon](https://github.com/shamoon) ([\#852](https://github.com/paperless-ngx/paperless-ngx/pull/852)) -- Fixes IMAP UTF8 Authenication [\@stumpylog](https://github.com/stumpylog) ([\#725](https://github.com/paperless-ngx/paperless-ngx/pull/725)) -- Fix password field remains visible [\@shamoon](https://github.com/shamoon) ([\#840](https://github.com/paperless-ngx/paperless-ngx/pull/840)) -- Fixes Pillow build for armv7 [\@stumpylog](https://github.com/stumpylog) ([\#815](https://github.com/paperless-ngx/paperless-ngx/pull/815)) -- Update frontend localization source file [\@shamoon](https://github.com/shamoon) ([\#814](https://github.com/paperless-ngx/paperless-ngx/pull/814)) -- Fix install script extra OCR languages format [\@stumpylog](https://github.com/stumpylog) ([\#777](https://github.com/paperless-ngx/paperless-ngx/pull/777)) - -### Documentation - -- Use semver for release process [\@stumpylog](https://github.com/stumpylog) ([\#851](https://github.com/paperless-ngx/paperless-ngx/pull/851)) -- Deployment: Consolidate tika compose files [\@qcasey](https://github.com/qcasey) ([\#866](https://github.com/paperless-ngx/paperless-ngx/pull/866)) -- Fix local Docker image building [\@stumpylog](https://github.com/stumpylog) ([\#849](https://github.com/paperless-ngx/paperless-ngx/pull/849)) - -### Maintenance - -- Dockerfile Organization \& Enhancements [\@stumpylog](https://github.com/stumpylog) ([\#888](https://github.com/paperless-ngx/paperless-ngx/pull/888)) -- Add timeout to healthcheck [\@shamoon](https://github.com/shamoon) ([\#880](https://github.com/paperless-ngx/paperless-ngx/pull/880)) -- Use semver for release process [\@stumpylog](https://github.com/stumpylog) ([\#851](https://github.com/paperless-ngx/paperless-ngx/pull/851)) -- Deployment: Consolidate tika compose files [\@qcasey](https://github.com/qcasey) ([\#866](https://github.com/paperless-ngx/paperless-ngx/pull/866)) -- Fixes Pillow build for armv7 [\@stumpylog](https://github.com/stumpylog) ([\#815](https://github.com/paperless-ngx/paperless-ngx/pull/815)) -- Update frontend localization source file [\@shamoon](https://github.com/shamoon) ([\#814](https://github.com/paperless-ngx/paperless-ngx/pull/814)) -- Fix install script extra OCR languages format [\@stumpylog](https://github.com/stumpylog) ([\#777](https://github.com/paperless-ngx/paperless-ngx/pull/777)) -- Adds simple Python to wait for Redis broker to be ready [\@stumpylog](https://github.com/stumpylog) ([\#788](https://github.com/paperless-ngx/paperless-ngx/pull/788)) - -### Dependencies - -
-15 changes - -- Bump tj-actions/changed-files from 18.7 to 19 @dependabot ([\#830](https://github.com/paperless-ngx/paperless-ngx/pull/830)) -- Bump asgiref from 3.5.0 to 3.5.1 @dependabot ([\#867](https://github.com/paperless-ngx/paperless-ngx/pull/867)) -- Bump jest from 27.5.1 to 28.0.3 in /src-ui @dependabot ([\#860](https://github.com/paperless-ngx/paperless-ngx/pull/860)) -- Bump @ng-bootstrap/ng-bootstrap from 12.1.0 to 12.1.1 in /src-ui @dependabot ([\#861](https://github.com/paperless-ngx/paperless-ngx/pull/861)) -- Bump @types/node from 17.0.27 to 17.0.29 in /src-ui @dependabot ([\#833](https://github.com/paperless-ngx/paperless-ngx/pull/833)) -- Bump @ng-bootstrap/ng-bootstrap from 12.0.2 to 12.1.0 in /src-ui @dependabot ([\#834](https://github.com/paperless-ngx/paperless-ngx/pull/834)) -- Bump pytest from 7.1.1 to 7.1.2 @dependabot ([\#806](https://github.com/paperless-ngx/paperless-ngx/pull/806)) -- Bump github/codeql-action from 1 to 2 @dependabot ([\#792](https://github.com/paperless-ngx/paperless-ngx/pull/792)) -- Bump imap-tools from 0.53.0 to 0.54.0 @dependabot ([\#758](https://github.com/paperless-ngx/paperless-ngx/pull/758)) -- Bump ocrmypdf from 13.4.2 to 13.4.3 @dependabot ([\#757](https://github.com/paperless-ngx/paperless-ngx/pull/757)) -- Bump importlib-resources from 5.6.0 to 5.7.1 @dependabot ([\#756](https://github.com/paperless-ngx/paperless-ngx/pull/756)) -- Bump tox from 3.24.5 to 3.25.0 @dependabot ([\#692](https://github.com/paperless-ngx/paperless-ngx/pull/692)) -- Bump cypress from 9.5.3 to 9.6.0 in /src-ui @dependabot ([\#800](https://github.com/paperless-ngx/paperless-ngx/pull/800)) -- Bump angular \& tools to 13.3.4 or 13.3.3 [\@shamoon](https://github.com/shamoon) ([\#799](https://github.com/paperless-ngx/paperless-ngx/pull/799)) -- Bump concurrently from 7.0.0 to 7.1.0 in /src-ui @dependabot ([\#797](https://github.com/paperless-ngx/paperless-ngx/pull/797)) -
- -## paperless-ngx 1.7.0 - -### Breaking Changes - -- `PAPERLESS_URL` is now required when using a reverse proxy. See - [\#674](https://github.com/paperless-ngx/paperless-ngx/pull/674). - -### Features - -- Allow setting more than one tag in mail rules - [\@jonasc](https://github.com/jonasc) ([\#270](https://github.com/paperless-ngx/paperless-ngx/pull/270)) -- Global drag\'n\'drop [\@shamoon](https://github.com/shamoon) - ([\#283](https://github.com/paperless-ngx/paperless-ngx/pull/283)) -- Fix: download buttons should disable while waiting - [\@shamoon](https://github.com/shamoon) ([\#630](https://github.com/paperless-ngx/paperless-ngx/pull/630)) -- Update checker [\@shamoon](https://github.com/shamoon) ([\#591](https://github.com/paperless-ngx/paperless-ngx/pull/591)) -- Show prompt on password-protected pdfs - [\@shamoon](https://github.com/shamoon) ([\#564](https://github.com/paperless-ngx/paperless-ngx/pull/564)) -- Filtering query params aka browser navigation for filtering - [\@shamoon](https://github.com/shamoon) ([\#540](https://github.com/paperless-ngx/paperless-ngx/pull/540)) -- Clickable tags in dashboard widgets - [\@shamoon](https://github.com/shamoon) ([\#515](https://github.com/paperless-ngx/paperless-ngx/pull/515)) -- Add bottom pagination [\@shamoon](https://github.com/shamoon) - ([\#372](https://github.com/paperless-ngx/paperless-ngx/pull/372)) -- Feature barcode splitter [\@gador](https://github.com/gador) - ([\#532](https://github.com/paperless-ngx/paperless-ngx/pull/532)) -- App loading screen [\@shamoon](https://github.com/shamoon) ([\#298](https://github.com/paperless-ngx/paperless-ngx/pull/298)) -- Use progress bar for delayed buttons - [\@shamoon](https://github.com/shamoon) ([\#415](https://github.com/paperless-ngx/paperless-ngx/pull/415)) -- Add minimum length for documents text filter - [\@shamoon](https://github.com/shamoon) ([\#401](https://github.com/paperless-ngx/paperless-ngx/pull/401)) -- Added nav buttons in the document detail view - [\@GruberViktor](https://github.com/gruberviktor) ([\#273](https://github.com/paperless-ngx/paperless-ngx/pull/273)) -- Improve date keyboard input [\@shamoon](https://github.com/shamoon) - ([\#253](https://github.com/paperless-ngx/paperless-ngx/pull/253)) -- Color theming [\@shamoon](https://github.com/shamoon) ([\#243](https://github.com/paperless-ngx/paperless-ngx/pull/243)) -- Parse dates when entered without separators - [\@GruberViktor](https://github.com/gruberviktor) ([\#250](https://github.com/paperless-ngx/paperless-ngx/pull/250)) - -### Bug Fixes - -- Add \"localhost\" to ALLOWED_HOSTS - [\@gador](https://github.com/gador) ([\#700](https://github.com/paperless-ngx/paperless-ngx/pull/700)) -- Fix: scanners table [\@qcasey](https://github.com/qcasey) ([\#690](https://github.com/paperless-ngx/paperless-ngx/pull/690)) -- Adds wait for file before consuming - [\@stumpylog](https://github.com/stumpylog) ([\#483](https://github.com/paperless-ngx/paperless-ngx/pull/483)) -- Fix: frontend document editing erases time data - [\@shamoon](https://github.com/shamoon) ([\#654](https://github.com/paperless-ngx/paperless-ngx/pull/654)) -- Increase length of SavedViewFilterRule - [\@stumpylog](https://github.com/stumpylog) ([\#612](https://github.com/paperless-ngx/paperless-ngx/pull/612)) -- Fixes attachment filename matching during mail fetching - [\@stumpylog](https://github.com/stumpylog) ([\#680](https://github.com/paperless-ngx/paperless-ngx/pull/680)) -- Add `PAPERLESS_URL` env variable & CSRF var - [\@shamoon](https://github.com/shamoon) ([\#674](https://github.com/paperless-ngx/paperless-ngx/discussions/674)) -- Fix: download buttons should disable while waiting - [\@shamoon](https://github.com/shamoon) ([\#630](https://github.com/paperless-ngx/paperless-ngx/pull/630)) -- Fixes downloaded filename, add more consumer ignore settings - [\@stumpylog](https://github.com/stumpylog) ([\#599](https://github.com/paperless-ngx/paperless-ngx/pull/599)) -- FIX BUG: case-sensitive matching was not possible - [\@danielBreitlauch](https://github.com/danielbreitlauch) ([\#594](https://github.com/paperless-ngx/paperless-ngx/pull/594)) -- Uses shutil.move instead of rename - [\@gador](https://github.com/gador) ([\#617](https://github.com/paperless-ngx/paperless-ngx/pull/617)) -- Fix npm deps 01.02.22 2 [\@shamoon](https://github.com/shamoon) - ([\#610](https://github.com/paperless-ngx/paperless-ngx/discussions/610)) -- Fix npm dependencies 01.02.22 - [\@shamoon](https://github.com/shamoon) ([\#600](https://github.com/paperless-ngx/paperless-ngx/pull/600)) -- Fix issue 416: implement `PAPERLESS_OCR_MAX_IMAGE_PIXELS` - [\@hacker-h](https://github.com/hacker-h) ([\#441](https://github.com/paperless-ngx/paperless-ngx/pull/441)) -- Fix: exclude cypress from build in Dockerfile - [\@FrankStrieter](https://github.com/FrankStrieter) ([\#526](https://github.com/paperless-ngx/paperless-ngx/pull/526)) -- Corrections to pass pre-commit hooks - [\@schnuffle](https://github.com/schnuffle) ([\#454](https://github.com/paperless-ngx/paperless-ngx/pull/454)) -- Fix 311 unable to click checkboxes in document list - [\@shamoon](https://github.com/shamoon) ([\#313](https://github.com/paperless-ngx/paperless-ngx/pull/313)) -- Fix imap tools bug [\@stumpylog](https://github.com/stumpylog) - ([\#393](https://github.com/paperless-ngx/paperless-ngx/pull/393)) -- Fix filterable dropdown buttons arent translated - [\@shamoon](https://github.com/shamoon) ([\#366](https://github.com/paperless-ngx/paperless-ngx/pull/366)) -- Fix 224: \"Auto-detected date is day before receipt date\" - [\@a17t](https://github.com/a17t) ([\#246](https://github.com/paperless-ngx/paperless-ngx/pull/246)) -- Fix minor sphinx errors [\@shamoon](https://github.com/shamoon) - ([\#322](https://github.com/paperless-ngx/paperless-ngx/pull/322)) -- Fix page links hidden [\@shamoon](https://github.com/shamoon) - ([\#314](https://github.com/paperless-ngx/paperless-ngx/pull/314)) -- Fix: Include excluded items in dropdown count - [\@shamoon](https://github.com/shamoon) ([\#263](https://github.com/paperless-ngx/paperless-ngx/pull/263)) - -### Translation - -- [\@miku323](https://github.com/miku323) contributed to Slovenian - translation -- [\@FaintGhost](https://github.com/FaintGhost) contributed to Chinese - Simplified translation -- [\@DarkoBG79](https://github.com/DarkoBG79) contributed to Serbian - translation -- [Kemal Secer](https://crowdin.com/profile/kemal.secer) contributed - to Turkish translation -- [\@Prominence](https://github.com/Prominence) contributed to - Belarusian translation - -### Documentation - -- Fix: scanners table [\@qcasey](https://github.com/qcasey) ([\#690](https://github.com/paperless-ngx/paperless-ngx/pull/690)) -- Add `PAPERLESS_URL` env variable & CSRF var - [\@shamoon](https://github.com/shamoon) ([\#674](https://github.com/paperless-ngx/paperless-ngx/pull/674)) -- Fixes downloaded filename, add more consumer ignore settings - [\@stumpylog](https://github.com/stumpylog) ([\#599](https://github.com/paperless-ngx/paperless-ngx/pull/599)) -- Fix issue 416: implement `PAPERLESS_OCR_MAX_IMAGE_PIXELS` - [\@hacker-h](https://github.com/hacker-h) ([\#441](https://github.com/paperless-ngx/paperless-ngx/pull/441)) -- Fix minor sphinx errors [\@shamoon](https://github.com/shamoon) - ([\#322](https://github.com/paperless-ngx/paperless-ngx/pull/322)) - -### Maintenance - -- Add `PAPERLESS_URL` env variable & CSRF var - [\@shamoon](https://github.com/shamoon) ([\#674](https://github.com/paperless-ngx/paperless-ngx/pull/674)) -- Chore: Implement release-drafter action for Changelogs - [\@qcasey](https://github.com/qcasey) ([\#669](https://github.com/paperless-ngx/paperless-ngx/pull/669)) -- Chore: Add CODEOWNERS [\@qcasey](https://github.com/qcasey) ([\#667](https://github.com/paperless-ngx/paperless-ngx/pull/667)) -- Support docker-compose v2 in install - [\@stumpylog](https://github.com/stumpylog) ([\#611](https://github.com/paperless-ngx/paperless-ngx/pull/611)) -- Add Belarusian localization [\@shamoon](https://github.com/shamoon) - ([\#588](https://github.com/paperless-ngx/paperless-ngx/pull/588)) -- Add Turkish localization [\@shamoon](https://github.com/shamoon) - ([\#536](https://github.com/paperless-ngx/paperless-ngx/pull/536)) -- Add Serbian localization [\@shamoon](https://github.com/shamoon) - ([\#504](https://github.com/paperless-ngx/paperless-ngx/pull/504)) -- Create PULL_REQUEST_TEMPLATE.md - [\@shamoon](https://github.com/shamoon) ([\#304](https://github.com/paperless-ngx/paperless-ngx/pull/304)) -- Add Chinese localization [\@shamoon](https://github.com/shamoon) - ([\#247](https://github.com/paperless-ngx/paperless-ngx/pull/247)) -- Add Slovenian language for frontend - [\@shamoon](https://github.com/shamoon) ([\#315](https://github.com/paperless-ngx/paperless-ngx/pull/315)) - -## paperless-ngx 1.6.0 - -This is the first release of the revived paperless-ngx project 🎉. Thank -you to everyone on the paperless-ngx team for your initiative and -excellent teamwork! - -Version 1.6.0 merges several pending PRs from jonaswinkler\'s repo and -includes new feature updates and bug fixes. Major backend and UI changes -include: - -- Updated docs, scripts, CI, and containers to paperless-ngx. -- Updated Python and Angular dependencies. -- Dropped support for Python 3.7. -- Dropped support for Ansible playbooks (thanks - [\@slankes](https://github.com/slankes) [\#109](https://github.com/paperless-ngx/paperless-ngx/pull/109)). If someone would - like to continue supporting them, please see our [ansible - repo](https://github.com/paperless-ngx/paperless-ngx-ansible). -- Python code is now required to use Black formatting (thanks - [\@kpj](https://github.com/kpj) [\#168](https://github.com/paperless-ngx/paperless-ngx/pull/168)). -- [\@tribut](https://github.com/tribut) added support for a custom SSO - logout redirect ([jonaswinkler\#1258](https://github.com/jonaswinkler/paperless-ng/pull/1258)). See - `PAPERLESS_LOGOUT_REDIRECT_URL`. -- [\@shamoon](https://github.com/shamoon) added a loading indicator - when document list is reloading ([jonaswinkler\#1297](https://github.com/jonaswinkler/paperless-ng/pull/1297)). -- [\@shamoon](https://github.com/shamoon) improved the PDF viewer on - mobile ([\#2](https://github.com/paperless-ngx/paperless-ngx/pull/2)). -- [\@shamoon](https://github.com/shamoon) added \'any\' / \'all\' and - \'not\' filtering with tags ([\#10](https://github.com/paperless-ngx/paperless-ngx/pull/10)). -- [\@shamoon](https://github.com/shamoon) added warnings for unsaved - changes, with smart edit buttons ([\#13](https://github.com/paperless-ngx/paperless-ngx/pull/13)). -- [\@benjaminfrank](https://github.com/benjaminfrank) enabled a - non-root access to port 80 via systemd ([\#18](https://github.com/paperless-ngx/paperless-ngx/pull/18)). -- [\@tribut](https://github.com/tribut) added simple \"delete to - trash\" functionality ([\#24](https://github.com/paperless-ngx/paperless-ngx/pull/24)). See `PAPERLESS_TRASH_DIR`. -- [\@amenk](https://github.com/amenk) fixed the search box overlay - menu on mobile ([\#32](https://github.com/paperless-ngx/paperless-ngx/pull/32)). -- [\@dblitt](https://github.com/dblitt) updated the login form to not - auto-capitalize usernames ([\#36](https://github.com/paperless-ngx/paperless-ngx/pull/36)). -- [\@evilsidekick293](https://github.com/evilsidekick293) made the - worker timeout configurable ([\#37](https://github.com/paperless-ngx/paperless-ngx/pull/37)). See `PAPERLESS_WORKER_TIMEOUT`. -- [\@Nicarim](https://github.com/Nicarim) fixed downloads of UTF-8 - formatted documents in Firefox ([\#56](https://github.com/paperless-ngx/paperless-ngx/pull/56)). -- [\@mweimerskirch](https://github.com/mweimerskirch) sorted the - language dropdown by locale ([\#78](https://github.com/paperless-ngx/paperless-ngx/issues/78)). -- [\@mweimerskirch](https://github.com/mweimerskirch) enabled the - Czech ([\#83](https://github.com/paperless-ngx/paperless-ngx/pull/83)) and Danish ([\#84](https://github.com/paperless-ngx/paperless-ngx/pull/84)) translations. -- [\@cschmatzler](https://github.com/cschmatzler) enabled specifying - the webserver port ([\#124](https://github.com/paperless-ngx/paperless-ngx/pull/124)). See `PAPERLESS_PORT`. -- [\@muellermartin](https://github.com/muellermartin) fixed an error - when uploading transparent PNGs ([\#133](https://github.com/paperless-ngx/paperless-ngx/pull/133)). -- [\@shamoon](https://github.com/shamoon) created a slick new logo - ([\#165](https://github.com/paperless-ngx/paperless-ngx/pull/165)). -- [\@tim-vogel](https://github.com/tim-vogel) fixed exports missing - groups ([\#193](https://github.com/paperless-ngx/paperless-ngx/pull/193)). - -Known issues: - -- 1.6.0 included a malformed package-lock.json, as a result users who - want to build the docker image themselves need to change line 6 of - the `Dockerfile` to - `RUN npm update npm -g && npm install --legacy-peer-deps`. - -Thank you to the following people for their documentation updates, -fixes, and comprehensive testing: - -[\@m0veax](https://github.com/m0veax), -[\@a17t](https://github.com/a17t), -[\@fignew](https://github.com/fignew), -[\@muued](https://github.com/muued), -[\@bauerj](https://github.com/bauerj), -[\@isigmund](https://github.com/isigmund), -[\@denilsonsa](https://github.com/denilsonsa), -[\@mweimerskirch](https://github.com/mweimerskirch), -[\@alexander-bauer](https://github.com/alexander-bauer), -[\@apeltzer](https://github.com/apeltzer), -[\@tribut](https://github.com/tribut), -[\@yschroeder](https://github.com/yschroeder), -[\@gador](https://github.com/gador), -[\@sAksham-Ar](https://github.com/sAksham-Ar), -[\@sbrunner](https://github.com/sbrunner), -[\@philpagel](https://github.com/philpagel), -[\@davemachado](https://github.com/davemachado), -[\@2600box](https://github.com/2600box), -[\@qcasey](https://github.com/qcasey), -[\@Nicarim](https://github.com/Nicarim), -[\@kpj](https://github.com/kpj), [\@filcuk](https://github.com/filcuk), -[\@Timoms](https://github.com/Timoms), -[\@mattlamb99](https://github.com/mattlamb99), -[\@padraigkitterick](https://github.com/padraigkitterick), -[\@ajkavanagh](https://github.com/ajkavanagh), -[\@Tooa](https://github.com/Tooa), -[\@Unkn0wnCat](https://github.com/Unkn0wnCat), -[\@pewter77](https://github.com/pewter77), -[\@stumpylog](https://github.com/stumpylog), -[\@Toxix](https://github.com/Toxix), -[\@azapater](https://github.com/azapater), -[\@jschpp](https://github.com/jschpp) - -Another big thanks to the people who have contributed translations: - -- Michel Weimerskirch (michel_weimerskirch) suggested 31 translations - into French and Luxembourgish. -- jo.vandeginste suggested 21 translations into Dutch. -- Lars Sørensen (Lrss) suggested 486 translations into Danish. -- Alex (Sky-Dragon) voted for 46 translations in German. -- Yannic Schröder (yschroeder) suggested 14 translations into German. -- David Morais Ferreira (DavidMoraisFerreira) voted for 10 - translations in Portuguese and Luxembourgish. -- David Morais Ferreira (DavidMoraisFerreira) suggested 88 - translations into French, German, Portuguese, Portuguese, Brazilian - and Luxembourgish. -- 汪泠沣 (wlfcss) suggested 13 translations into Chinese Traditional. -- Lars Sørensen (Lrss) suggested 167 translations into Danish. -- Philmo67 suggested 11 translations into French. - -## Paperless-ng - -### paperless-ng 1.5.0 - -Support for Python 3.6 was dropped. - -- Updated python dependencies. -- Base image of the docker image changed from Debian Buster to Debian - Bullseye due to its recent release. -- The docker image now uses python 3.9. -- Added the Luxembourgish locale. Thanks for translating! -- [Daniel Albers](https://github.com/AlD) added support for making the - files and folders ignored by the paperless consume folder scanner - configurable. See `PAPERLESS_CONSUMER_IGNORE_PATTERNS`. - -### paperless-ng 1.4.5 - -This is a maintenance release. - -- Updated Python and Angular dependencies. -- Changed the algorithm that changes permissions during startup. This - is still fast, and will hopefully cause less issues. -- Fixed an issue that would sometimes cause paperless to write an - incomplete classification model file to disk. -- Fixed an issue with the OCRmyPDF parser that would always try to - extract text with PDFminer even from non-PDF files. - -### paperless-ng 1.4.4 - -- Drastically decreased the startup time of the docker container. The - startup script adjusts file permissions of all data only if changes - are required. -- Paperless mail: Added ability to specify the character set for each - server. -- Document consumption: Ignore Mac OS specific files such as - `.DS_STORE` and `._XXXXX.pdf`. -- Fixed an issue with the automatic matching algorithm that prevents - paperless from consuming new files. -- Updated translations. - -### paperless-ng 1.4.3 - -- Additions and changes - - Added Swedish locale. - - [Stéphane Brunner](https://github.com/sbrunner) added an option - to disable the progress bars of all management commands. - - [Jo Vandeginste](https://github.com/jovandeginste) added support - for RTF documents to the Apache TIKA parser. - - [Michael Shamoon](https://github.com/shamoon) added dark mode - for the login and logout pages. - - [Alexander Menk](https://github.com/amenk) added additional - stylesheets for printing. You can now print any page of - paperless and the print result will hide the page header, - sidebar, and action buttons. - - Added support for sorting when using full text search. -- Fixes - - [puuu](https://github.com/puuu) fixed - `PAPERLESS_FORCE_SCRIPT_NAME`. You can now host paperless on sub - paths such as `https://localhost:8000/paperless/`. - - Fixed an issue with the document consumer crashing on certain - documents due to issues with pdfminer.six. This library is used - for PDF text extraction. - -### paperless-ng 1.4.2 - -- Fixed an issue with `sudo` that caused paperless to not start on - many Raspberry Pi devices. Thank you - [WhiteHatTux](https://github.com/WhiteHatTux)! - -### paperless-ng 1.4.1 - -- Added Polish locale. -- Changed some parts of the Dockerfile to hopefully restore - functionality on certain ARM devices. -- Updated python dependencies. -- [Michael Shamoon](https://github.com/shamoon) added a sticky filter - / bulk edit bar. -- [sbrl](https://github.com/sbrl) changed the docker-entrypoint.sh - script to increase compatibility with NFS shares. -- [Chris Nagy](https://github.com/what-name) added support for - creating a super user by passing `PAPERLESS_ADMIN_USER` and - `PAPERLESS_ADMIN_PASSWORD` as environment variables to the docker - container. - -### paperless-ng 1.4.0 - -- Docker images now use tesseract 4.1.1, which should fix a series of - issues with OCR. -- The full text search now displays results using the default document - list. This enables selection, filtering and bulk edit on search - results. -- Changes - - Firefox only: Highlight search query in PDF previews. - - New URL pattern for accessing documents by ASN directly - (\/asn/123) - - Added logging when executing pre\* and post-consume scripts. - - Better error logging during document consumption. - - Updated python dependencies. - - Automatically inserts typed text when opening \"Create new\" - dialogs on the document details page. -- Fixes - - Fixed an issue with null characters in the document content. - -::: {.note} -::: {.title} -Note -::: - -The changed to the full text searching require you to reindex your -documents. _The docker image does this automatically, you don\'t need to -do anything._ To do this, execute the `document_index reindex` -management command (see `administration-index`{.interpreted-text -role="ref"}). -::: - -### paperless-ng 1.3.2 - -- Added translation into Portuguese. -- Changes - - The exporter now exports user accounts, mail accounts, mail - rules and saved views as well. -- Fixes - - Minor layout issues with document cards and the log viewer. - - Fixed an issue with any/all/exact matching when characters used - in regular expressions were used for the match. - -### paperless-ng 1.3.1 - -- Added translation into Spanish and Russian. -- Other changes - - ISO-8601 date format will now always show years with 4 digits. - - Added the ability to search for a document with a specific ASN. - - The document cards now display ASN, types and dates in a more - organized way. - - Added document previews when hovering over the preview button. -- Fixes - - The startup check for write permissions now works properly on - NFS shares. - - Fixed an issue with the search results score indicator. - - Paperless was unable to generate thumbnails for encrypted PDF - files and failed. Paperless will now generate a default - thumbnail for these files. - - Fixed `AUTO_LOGIN_USERNAME`: Unable to perform POST/PUT/DELETE - requests and unable to receive WebSocket messages. - -### paperless-ng 1.3.0 - -This release contains new database migrations. - -- Changes - - The REST API is versioned from this point onwards. This will - allow me to make changes without breaking existing clients. See - the documentation about `api-versioning`{.interpreted-text - role="ref"} for details. - - Added a color picker for tag colors. - - Added the ability to use the filter for searching the document - content as well. - - Added translations into Italian and Romanian. Thank you! - - Close individual documents from the sidebar. Thanks to [Michael - Shamoon](https://github.com/shamoon). - - [BolkoSchreiber](https://github.com/BolkoSchreiber) added an - option to disable/enable thumbnail inversion in dark mode. - - [Simon Taddiken](https://github.com/skuzzle) added the ability - to customize the header used for remote user authentication with - SSO applications. -- Bug fixes - - Fixed an issue with the auto matching algorithm when more than - 256 tags were used. - -### paperless-ng 1.2.1 - -- [Rodrigo Avelino](https://github.com/rodavelino) translated - Paperless into Portuguese (Brazil)! -- The date input fields now respect the currently selected date - format. -- Added a fancy icon when adding paperless to the home screen on iOS - devices. Thanks to [Joel Nordell](https://github.com/joelnordell). -- When using regular expression matching, the regular expression is - now validated before saving the tag/correspondent/type. -- Regression fix: Dates on the front end did not respect date locale - settings in some cases. - -### paperless-ng 1.2.0 - -- Changes to the OCRmyPDF integration - - Added support for deskewing and automatic rotation of - incorrectly rotated pages. This is enabled by default, see - `configuration-ocr`{.interpreted-text role="ref"}. - - Better support for encrypted files. - - Better support for various other PDF files: Paperless will now - attempt to force OCR with safe options when OCR fails with the - configured options. - - Added an explicit option to skip cleaning with `unpaper`. -- Download multiple selected documents as a zip archive. -- The document list now remembers the current page. -- Improved responsiveness when switching between saved views and the - document list. -- Increased the default wait time when observing files in the - consumption folder with polling from 1 to 5 seconds. This will - decrease the likelihood of paperless consuming partially written - files. -- Fixed a crash of the document archiver management command when - trying to process documents with unknown mime types. -- Paperless no longer depends on `libpoppler-cpp-dev`. - -### paperless-ng 1.1.4 - -- Added English (GB) locale. -- Added ISO-8601 date display option. - -### paperless-ng 1.1.3 - -- Added a docker-specific configuration option to adjust the number of - worker processes of the web server. See - `configuration-docker`{.interpreted-text role="ref"}. -- Some more memory usage optimizations. -- Don\'t show inbox statistics if no inbox tag is defined. - -### paperless-ng 1.1.2 - -- Always show top left corner of thumbnails, even for extra wide - documents. -- Added a management command for executing the sanity checker - directly. See `utilities-sanity-checker`{.interpreted-text - role="ref"}. -- The weekly sanity check now reports messages in the log files. -- Fixed an issue with the metadata tab not reporting anything in case - of missing files. -- Reverted a change from 1.1.0 that caused huge memory usage due to - redis caching. -- Some memory usage optimizations. - -### paperless-ng 1.1.1 - -This release contains new database migrations. - -- Fixed a bug in the sanity checker that would cause it to display \"x - not in list\" errors instead of actual issues. -- Fixed a bug with filename generation for archive filenames that - would cause the archive files of two documents to overlap. - - This happened when `PAPERLESS_FILENAME_FORMAT` is used and the - filenames of two or more documents are the same, except for the - file extension. - - Paperless will now store the archive filename in the database as - well instead of deriving it from the original filename, and use - the same logic for detecting and avoiding filename clashes - that\'s also used for original filenames. - - The migrations will repair any missing archive files. If you\'re - using tika, ensure that tika is running while performing the - migration. Docker-compose will take care of that. -- Fixed a bug with thumbnail regeneration when TIKA integration was - used. -- Added ASN as a placeholder field to the filename format. -- The docker image now comes with built-in shortcuts for most - management commands. These are now the recommended way to execute - management commands, since these also ensure that they\'re always - executed as the paperless user and you\'re less likely to run into - permission issues. See - `utilities-management-commands`{.interpreted-text role="ref"}. - -### paperless-ng 1.1.0 - -- Document processing status - - - Paperless now shows the status of processing documents on the - dashboard in real time. - - Status notifications when - - New documents are detected in the consumption folder, in - mails, uploaded on the front end, or added with one of the - mobile apps. - - Documents are successfully added to paperless. - - Document consumption failed (with error messages) - - Configuration options to enable/disable individual - notifications. - -- Live updates to document lists and saved views when new documents - are added. - - ::: {.hint} - ::: {.title} - Hint - ::: - - For status notifications and live updates to work, paperless now - requires an [ASGI](https://asgi.readthedocs.io/en/latest/)-enabled - web server. The docker images uses `gunicorn` and an ASGI-enabled - worker called [uvicorn](http://www.uvicorn.org/), and there is no - need to configure anything. - - For bare metal installations, changes are required for the - notifications to work. Adapt the service - `paperless-webserver.service` to use the supplied `gunicorn.conf.py` - configuration file and adapt the reference to the ASGI application - as follows: - - ``` - ExecStart=/opt/paperless/.local/bin/gunicorn -c /opt/paperless/gunicorn.conf.py paperless.asgi:application - ``` - - Paperless will continue to work with WSGI, but you will not get any - status notifications. - - Apache `mod_wsgi` users, see - `this note `{.interpreted-text role="ref"}. - ::: - -- Paperless now offers suggestions for tags, correspondents and types - on the document detail page. - -- Added an interactive easy install script that automatically - downloads, configures and starts paperless with docker. - -- Official support for Python 3.9. - -- Other changes and fixes - - - Adjusted the default parallelization settings to run more than - one task in parallel on systems with 4 or less cores. This - addresses issues with paperless not consuming any new files when - other tasks are running. - - Fixed a rare race condition that would cause paperless to - process incompletely written files when using the upload on the - dashboard. - - The document classifier no longer issues warnings and errors - when auto matching is not used at all. - - Better icon for document previews. - - Better info section in the side bar. - - Paperless no longer logs to the database. Instead, logs are - written to rotating log files. This solves many \"database is - locked\" issues on Raspberry Pi, especially when SQLite is used. - - By default, log files are written to `PAPERLESS_DATA_DIR/log/`. - Logging settings can be adjusted with `PAPERLESS_LOGGING_DIR`, - `PAPERLESS_LOGROTATE_MAX_SIZE` and - `PAPERLESS_LOGROTATE_MAX_BACKUPS`. - -### paperless-ng 1.0.0 - -Nothing special about this release, but since there are relatively few -bug reports coming in, I think that this is reasonably stable. - -- Document export - - The document exporter has been rewritten to support updating an - already existing export in place. This enables incremental - backups with `rsync`. - - The document exporter supports naming exported files according - to `PAPERLESS_FILENAME_FORMAT`. - - The document exporter locks the media directory and the database - during execution to ensure that the resulting export is - consistent. - - See the - `updated documentation `{.interpreted-text - role="ref"} for more details. -- Other changes and additions - - Added a language selector to the settings. - - Added date format options to the settings. - - Range selection with shift clicking is now possible in the - document list. - - Filtering correspondent, type and tag management pages by name. - - Focus \"Name\" field in dialogs by default. - -### paperless-ng 0.9.14 - -Starting with this version, releases are getting built automatically. -This release also comes with changes on how to install and update -paperless. - -- Paperless now uses GitHub Actions to make releases and build docker - images. - - Docker images are available for amd64, armhf, and aarch64. - - When you pull an image from Docker Hub, Docker will - automatically select the correct image for you. -- Changes to docker installations and updates - - The `-dockerfiles.tar.xz` release archive is gone. Instead, - simply grab the docker files from `/docker/compose` in the - repository if you wish to install paperless by pulling from the - hub. - - The docker compose files in `/docker/compose` were changed to - always use the `latest` version automatically. In order to do - further updates, simply do a `docker-compose pull`. The - documentation has been updated. - - The docker compose files were changed to restart paperless on - system boot only if it was running before shutdown. - - Documentation of the docker-compose files about what they do. -- Changes to bare metal installations and updates - - The release archive is built exactly like before. However, the - release now comes with already compiled translation messages and - collected static files. Therefore, the update steps - `compilemessages` and `collectstatic` are now obsolete. -- Other changes - - A new configuration option `PAPERLESS_IGNORE_DATES` was added by - [jayme-github](http://github.com/jayme-github). This can be used - to instruct paperless to ignore certain dates (such as your date - of birth) when guessing the date from the document content. This - was actually introduced in 0.9.12, I just forgot to mention it - in the changelog. - - The filter drop downs now display selected entries on top of all - other entries. - - The PostgreSQL client now supports setting an explicit `sslmode` - to force encryption of the connection to PostgreSQL. - - The docker images now come with `jbig2enc`, which is a lossless - image encoder for PDF documents and decreases the size of - certain PDF/A documents. - - When using any of the manual matching algorithms, paperless now - logs messages about when and why these matching algorithms - matched. - - The default settings for parallelization in paperless were - adjusted to always leave one CPU core free. - - Added an option to the frontend to choose which method to use - for displaying PDF documents. -- Fixes - - An issue with the tika parser not picking up files from the - consumption directory was fixed. - - A couple changes to the dark mode and fixes to several other - layout issues. - - An issue with the drop downs for correspondents, tags and types - not properly supporting filtering with special characters was - fixed. - - Fixed an issue with filenames of downloaded files: Dates where - off by one day due to timezone issues. - - Searching will continue to work even when the index returns - non-existing documents. This resulted in \"Document does not - exist\" errors before. Instead, a warning is logged, indicating - the issue. - - An issue with the consumer crashing when invalid regular - expression were used was fixed. - -### paperless-ng 0.9.13 - -- Fixed an issue with Paperless not starting due to the new Tika - integration when `USERMAP_UID` and `USERMAP_GID` was used in the - `docker-compose.env` file. - -### paperless-ng 0.9.12 - -- Paperless localization - - Thanks to the combined efforts of many users, Paperless is now - available in English, Dutch, French and German. -- Thanks to [Jo Vandeginste](https://github.com/jovandeginste), - Paperless has optional support for Office documents such as .docx, - .doc, .odt and more. - - See the `configuration`{.interpreted-text - role="ref"} on how to enable this feature. This feature requires - two additional services (one for parsing Office documents and - metadata extraction and another for converting Office documents - to PDF), and is therefore not enabled on default installations. - - As with all other documents, paperless converts Office documents - to PDF and stores both the original as well as the archived PDF. -- Dark mode - - Thanks to [Michael Shamoon](https://github.com/shamoon), - paperless now has a dark mode. Configuration is available in the - settings. -- Other changes and additions - - The PDF viewer now uses a local copy of some dependencies - instead of fetching them from the internet. Thanks to - [slorenz](https://github.com/sisao). - - Revamped search bar styling thanks to [Michael - Shamoon](https://github.com/shamoon). - - Sorting in the document list by clicking on table headers. - - A button was added to the document detail page that assigns a - new ASN to a document. - - Form field validation: When providing invalid input in a form - (such as a duplicate ASN or no name), paperless now has visual - indicators and clearer error messages about what\'s wrong. - - Paperless disables buttons with network actions (such as save - and delete) when a network action is active. This indicates that - something is happening and prevents double clicking. - - When using \"Save & next\", the title field is focussed - automatically to better support keyboard editing. - - E-Mail: Added filter rule parameters to allow inline attachments - (watch out for mails with inlined images!) and attachment - filename filters with wildcards. - - Support for remote user authentication thanks to [Michael - Shamoon](https://github.com/shamoon). This is useful for hiding - Paperless behind single sign on applications such as - [authelia](https://www.authelia.com/). - - \"Clear filters\" has been renamed to \"Reset filters\" and now - correctly restores the default filters on saved views. Thanks to - [Michael Shamoon](https://github.com/shamoon) -- Fixes - - Paperless was unable to save views when \"Not assigned\" was - chosen in one of the filter dropdowns. - - Clearer error messages when pre and post consumption scripts do - not exist. - - The post consumption script is executed later in the consumption - process. Before the change, an ID was passed to the script - referring to a document that did not yet exist in the database. - -### paperless-ng 0.9.11 - -- Fixed an issue with the docker image not starting at all due to a - configuration change of the web server. - -### paperless-ng 0.9.10 - -- Bulk editing - - Thanks to [Michael Shamoon](https://github.com/shamoon), we\'ve - got a new interface for the bulk editor. - - There are some configuration options in the settings to alter - the behavior. -- Other changes and additions - - Thanks to [zjean](https://github.com/zjean), paperless now - publishes a webmanifest, which is useful for adding the - application to home screens on mobile devices. - - The Paperless-ng logo now navigates to the dashboard. - - Filter for documents that don\'t have any correspondents, types - or tags assigned. - - Tags, types and correspondents are now sorted case insensitive. - - Lots of preparation work for localization support. -- Fixes - - Added missing dependencies for Raspberry Pi builds. - - Fixed an issue with plain text file consumption: Thumbnail - generation failed due to missing fonts. - - An issue with the search index reporting missing documents after - bulk deletes was fixed. - - Issue with the tag selector not clearing input correctly. - - The consumer used to stop working when encountering an - incomplete classifier model file. - -::: {.note} -::: {.title} -Note -::: - -The bulk delete operations did not update the search index. Therefore, -documents that you deleted remained in the index and caused the search -to return messages about missing documents when searching. Further bulk -operations will properly update the index. - -However, this change is not retroactive: If you used the delete method -of the bulk editor, you need to reindex your search index by -`running the management command document_index with the argument reindex `{.interpreted-text -role="ref"}. -::: - -### paperless-ng 0.9.9 - -Christmas release! - -- Bulk editing - - Paperless now supports bulk editing. - - The following operations are available: Add and remove - correspondents, tags, document types from selected documents, as - well as mass-deleting documents. - - We\'ve got a more fancy UI in the works that makes these - features more accessible, but that\'s not quite ready yet. -- Searching - - Paperless now supports searching for similar documents (\"More - like this\") both from the document detail page as well as from - individual search results. - - A search score indicates how well a document matches the search - query, or how similar a document is to a given reference - document. -- Other additions and changes - - Clarification in the UI that the fields \"Match\" and \"Is - insensitive\" are not relevant for the Auto matching algorithm. - - New select interface for tags, types and correspondents allows - filtering. This also improves tag selection. Thanks again to - [Michael Shamoon](https://github.com/shamoon)! - - Page navigation controls for the document viewer, thanks to - [Michael Shamoon](https://github.com/shamoon). - - Layout changes to the small cards document list. - - The dashboard now displays the username (or full name if - specified in the admin) on the dashboard. -- Fixes - - An error that caused the document importer to crash was fixed. - - An issue with changes not being possible when - `PAPERLESS_COOKIE_PREFIX` is used was fixed. - - The date selection filters now allow manual entry of dates. -- Feature Removal - - Most of the guesswork features have been removed. Paperless no - longer tries to extract correspondents and tags from file names. - -### paperless-ng 0.9.8 - -This release addresses two severe issues with the previous release. - -- The delete buttons for document types, correspondents and tags were - not working. -- The document section in the admin was causing internal server errors - (500). - -### paperless-ng 0.9.7 - -- Front end - - Thanks to the hard work of [Michael - Shamoon](https://github.com/shamoon), paperless now comes with a - much more streamlined UI for filtering documents. - - [Michael Shamoon](https://github.com/shamoon) replaced the - document preview with another component. This should fix - compatibility with Safari browsers. - - Added buttons to the management pages to quickly show all - documents with one specific tag, correspondent, or title. - - Paperless now stores your saved views on the server and - associates them with your user account. This means that you can - access your views on multiple devices and have separate views - for different users. You will have to recreate your views. - - The GitHub and documentation links now open in new tabs/windows. - Thanks to [rYR79435](https://github.com/rYR79435). - - Paperless now generates default saved view names when saving - views with certain filter rules. - - Added a small version indicator to the front end. -- Other additions and changes - - The new filename format field `{tag_list}` inserts a list of - tags into the filename, separated by comma. - - The `document_retagger` no longer removes inbox tags or tags - without matching rules. - - The new configuration option `PAPERLESS_COOKIE_PREFIX` allows - you to run multiple instances of paperless on different ports. - This option enables you to be logged in into multiple instances - by specifying different cookie names for each instance. -- Fixes - - Sometimes paperless would assign dates in the future to newly - consumed documents. - - The filename format fields `{created_month}` and `{created_day}` - now use a leading zero for single digit values. - - The filename format field `{tags}` can no longer be used without - arguments. - - Paperless was not able to consume many images (especially images - from mobile scanners) due to missing DPI information. Paperless - now assumes A4 paper size for PDF generation if no DPI - information is present. - - Documents with empty titles could not be opened from the table - view due to the link being empty. - - Fixed an issue with filenames containing special characters such - as `:` not being accepted for upload. - - Fixed issues with thumbnail generation for plain text files. - -### paperless-ng 0.9.6 - -This release focusses primarily on many small issues with the UI. - -- Front end - - Paperless now has proper window titles. - - Fixed an issue with the small cards when more than 7 tags were - used. - - Navigation of the \"Show all\" links adjusted. They navigate to - the saved view now, if available in the sidebar. - - Some indication on the document lists that a filter is active - was added. - - There\'s a new filter to filter for documents that do _not_ have - a certain tag. - - The file upload box now shows upload progress. - - The document edit page was reorganized. - - The document edit page shows various information about a - document. - - An issue with the height of the preview was fixed. - - Table issues with too long document titles fixed. -- API - - The API now serves file names with documents. - - The API now serves various metadata about documents. - - API documentation updated. -- Other - - Fixed an issue with the docker image when a non-standard - PostgreSQL port was used. - - The docker image was trying check for installed languages before - actually installing them. - - `FILENAME_FORMAT` placeholder for document types. - - The filename formatter is now less restrictive with file names - and tries to conserve the original correspondents, types and - titles as much as possible. - - The filename formatter does not include the document ID in - filenames anymore. It will rather append `_01`, `_02`, etc when - it detects duplicate filenames. - -::: {.note} -::: {.title} -Note -::: - -The changes to the filename format will apply to newly added documents -and changed documents. If you want all files to reflect these changes, -execute the `document_renamer` management command. -::: - -### paperless-ng 0.9.5 - -This release concludes the big changes I wanted to get rolled into -paperless. The next releases before 1.0 will focus on fixing issues, -primarily. - -- OCR - - Paperless now uses - [OCRmyPDF](https://github.com/jbarlow83/OCRmyPDF) to perform OCR - on documents. It still uses tesseract under the hood, but the - PDF parser of Paperless has changed considerably and will behave - different for some douments. - - OCRmyPDF creates archived PDF/A documents with embedded text - that can be selected in the front end. - - Paperless stores archived versions of documents alongside with - the originals. The originals can be accessed on the document - edit page. If available, a dropdown menu will appear next to the - download button. - - Many of the configuration options regarding OCR have changed. - See `configuration-ocr`{.interpreted-text role="ref"} for - details. - - Paperless no longer guesses the language of your documents. It - always uses the language that you specified with - `PAPERLESS_OCR_LANGUAGE`. Be sure to set this to the language - the majority of your documents are in. Multiple languages can be - specified, but that requires more CPU time. - - The management command - `document_archiver `{.interpreted-text - role="ref"} can be used to create archived versions for already - existing documents. -- Tags from consumption folder. - - Thanks to [jayme-github](http://github.com/jayme-github), - paperless now consumes files from sub folders in the consumption - folder and is able to assign tags based on the sub folders a - document was found in. This can be configured with - `PAPERLESS_CONSUMER_RECURSIVE` and - `PAPERLESS_CONSUMER_SUBDIRS_AS_TAGS`. -- API - - The API now offers token authentication. - - The endpoint for uploading documents now supports specifying - custom titles, correspondents, tags and types. This can be used - by clients to override the default behavior of paperless. See - `api-file_uploads`{.interpreted-text role="ref"}. - - The document endpoint of API now serves documents in this form: - - correspondents, document types and tags are referenced by - their ID in the fields `correspondent`, `document_type` and - `tags`. The `*_id` versions are gone. These fields are - read/write. - - paperless does not serve nested tags, correspondents or - types anymore. -- Front end - - Paperless does some basic caching of correspondents, tags and - types and will only request them from the server when necessary - or when entirely reloading the page. - - Document list fetching is about 10%-30% faster now, especially - when lots of tags/correspondents are present. - - Some minor improvements to the front end, such as document count - in the document list, better highlighting of the current page, - and improvements to the filter behavior. -- Fixes: - - A bug with the generation of filenames for files with - unsupported types caused the exporter and document saving to - crash. - - Mail handling no longer exits entirely when encountering errors. - It will skip the account/rule/message on which the error - occured. - - Assigning correspondents from mail sender names failed for very - long names. Paperless no longer assigns correspondents in these - cases. - -### paperless-ng 0.9.4 - -- Searching: - - Paperless now supports searching by tags, types and dates and - correspondents. In order to have this applied to your existing - documents, you need to perform a `document_index reindex` - management command (see `administration-index`{.interpreted-text - role="ref"}) that adds the data to the search index. You only - need to do this once, since the schema of the search index - changed. Paperless keeps the index updated after that whenever - something changes. - - Paperless now has spelling corrections (\"Did you mean\") for - miss-typed queries. - - The documentation contains - `information about the query syntax `{.interpreted-text - role="ref"}. -- Front end: - - Clickable tags, correspondents and types allow quick filtering - for related documents. - - Saved views are now editable. - - Preview documents directly in the browser. - - Navigation from the dashboard to saved views. -- Fixes: - - A severe error when trying to use post consume scripts. - - An error in the consumer that cause invalid messages of missing - files to show up in the log. -- The documentation now contains information about bare metal installs - and a section about how to setup the development environment. - -### paperless-ng 0.9.3 - -- Setting `PAPERLESS_AUTO_LOGIN_USERNAME` replaces - `PAPERLESS_DISABLE_LOGIN`. You have to specify your username. -- Added a simple sanity checker that checks your documents for missing - or orphaned files, files with wrong checksums, inaccessible files, - and documents with empty content. -- It is no longer possible to encrypt your documents. For the time - being, paperless will continue to operate with already encrypted - documents. -- Fixes: - - Paperless now uses inotify again, since the watchdog was causing - issues which I was not aware of. - - Issue with the automatic classifier not working with only one - tag. - - A couple issues with the search index being opened to eagerly. -- Added lots of tests for various parts of the application. - -### paperless-ng 0.9.2 - -- Major changes to the front end (colors, logo, shadows, layout of the - cards, better mobile support) -- Paperless now uses mime types and libmagic detection to determine if - a file type is supported and which parser to use. Removes all file - type checks that where present in MANY different places in - paperless. -- Mail consumer now correctly consumes documents even when their - content type was not set correctly. (i.e. PDF documents with content - type `application/octet-stream`) -- Basic sorting of mail rules added -- Much better admin for mail rule editing. -- Docker entrypoint script awaits the database server if it is - configured. -- Disabled editing of logs. -- New setting `PAPERLESS_OCR_PAGES` limits the tesseract parser to the - first n pages of scanned documents. -- Fixed a bug where tasks with too long task names would not show up - in the admin. - -### paperless-ng 0.9.1 - -- Moved documentation of the settings to the actual documentation. -- Updated release script to force the user to choose between SQLite - and PostgreSQL. This avoids confusion when upgrading from paperless. - -### paperless-ng 0.9.0 - -- **Deprecated:** GnuPG. - `See this note on the state of GnuPG in paperless-ng. `{.interpreted-text - role="ref"} This features will most likely be removed in future - versions. -- **Added:** New frontend. Features: - - Single page application: It\'s much more responsive than the - django admin pages. - - Dashboard. Shows recently scanned documents, or todo notes, or - other documents at wish. Allows uploading of documents. Shows - basic statistics. - - Better document list with multiple display options. - - Full text search with result highlighting, auto completion and - scoring based on the query. It uses a document search index in - the background. - - Saveable filters. - - Better log viewer. -- **Added:** Document types. Assign these to documents just as - correspondents. They may be used in the future to perform automatic - operations on documents depending on the type. -- **Added:** Inbox tags. Define an inbox tag and it will automatically - be assigned to any new document scanned into the system. -- **Added:** Automatic matching. A new matching algorithm that - automatically assigns tags, document types and correspondents to - your documents. It uses a neural network trained on your data. -- **Added:** Archive serial numbers. Assign these to quickly find - documents stored in physical binders. -- **Added:** Enabled the internal user management of django. This - isn\'t really a multi user solution, however, it allows more than - one user to access the website and set some basic permissions / - renew passwords. -- **Modified \[breaking\]:** All new mail consumer with customizable - filters, actions and multiple account support. Replaces the old mail - consumer. The new mail consumer needs different configuration but - can be configured to act exactly like the old consumer. -- **Modified:** Changes to the consumer: - - Now uses the excellent watchdog library that should make sure - files are discovered no matter what the platform is. - - The consumer now uses a task scheduler to run consumption - processes in parallel. This means that consuming many documents - should be much faster on systems with many cores. - - Concurrency is controlled with the new settings - `PAPERLESS_TASK_WORKERS` and `PAPERLESS_THREADS_PER_WORKER`. See - TODO for details on concurrency. - - The consumer no longer blocks the database for extended periods - of time. - - An issue with tesseract running multiple threads per page and - slowing down the consumer was fixed. -- **Modified \[breaking\]:** REST Api changes: - - New filters added, other filters removed (case sensitive - filters, slug filters) - - Endpoints for thumbnails, previews and downloads replace the old - `/fetch/` urls. Redirects are in place. - - Endpoint for document uploads replaces the old `/push` url. - Redirects are in place. - - Foreign key relationships are now served as IDs, not as urls. -- **Modified \[breaking\]:** PostgreSQL: - - If `PAPERLESS_DBHOST` is specified in the settings, paperless - uses PostgreSQL instead of SQLite. Username, database and - password all default to `paperless` if not specified. -- **Modified \[breaking\]:** document_retagger management command - rework. See `utilities-retagger`{.interpreted-text role="ref"} for - details. Replaces `document_correspondents` management command. -- **Removed \[breaking\]:** Reminders. -- **Removed:** All customizations made to the django admin pages. -- **Removed \[breaking\]:** The docker image no longer supports SSL. - If you want to expose paperless to the internet, hide paperless - behind a proxy server that handles SSL requests. -- **Internal changes:** Mostly code cleanup, including: - - Rework of the code of the tesseract parser. This is now a lot - cleaner. - - Rework of the filename handling code. It was a mess. - - Fixed some issues with the document exporter not exporting all - documents when encountering duplicate filenames. - - Added a task scheduler that takes care of checking mail, - training the classifier, maintaining the document search index - and consuming documents. - - Updated dependencies. Now uses Pipenv all around. - - Updated Dockerfile and docker-compose. Now uses `supervisord` to - run everything paperless-related in a single container. -- **Settings:** - - `PAPERLESS_FORGIVING_OCR` is now default and gone. Reason: Even - if `langdetect` fails to detect a language, tesseract still does - a very good job at ocr\'ing a document with the default - language. Certain language specifics such as umlauts may not get - picked up properly. - - `PAPERLESS_DEBUG` defaults to `false`. - - The presence of `PAPERLESS_DBHOST` now determines whether to use - PostgreSQL or SQLite. - - `PAPERLESS_OCR_THREADS` is gone and replaced with - `PAPERLESS_TASK_WORKERS` and `PAPERLESS_THREADS_PER_WORKER`. - Refer to the config example for details. - - `PAPERLESS_OPTIMIZE_THUMBNAILS` allows you to disable or enable - thumbnail optimization. This is useful on less powerful devices. -- Many more small changes here and there. The usual stuff. - -## Paperless - -### 2.7.0 - -- [syntonym](https://github.com/syntonym) submitted a pull request to - catch IMAP connection errors - [\#475](https://github.com/the-paperless-project/paperless/pull/475). -- [Stéphane Brunner](https://github.com/sbrunner) added `psycopg2` to - the Pipfile - [\#489](https://github.com/the-paperless-project/paperless/pull/489). - He also fixed a syntax error in `docker-compose.yml.example` - [\#488](https://github.com/the-paperless-project/paperless/pull/488) - and added [DjangoQL](https://github.com/ivelum/djangoql), which - allows a litany of handy search functionality - [\#492](https://github.com/the-paperless-project/paperless/pull/492). -- [CkuT](https://github.com/CkuT) and - [JOKer](https://github.com/MasterofJOKers) hacked out a simple, but - super-helpful optimisation to how the thumbnails are served up, - improving performance considerably - [\#481](https://github.com/the-paperless-project/paperless/pull/481). -- [tsia](https://github.com/tsia) added a few fields to the tags REST - API. - [\#483](https://github.com/the-paperless-project/paperless/pull/483). -- [Brian Cribbs](https://github.com/cribbstechnolog) improved the - documentation to help people using Paperless over NFS - [\#484](https://github.com/the-paperless-project/paperless/pull/484). -- [Brendan M. Sleight](https://github.com/bmsleight) updated the - documentation to include a note for setting the `DEBUG` value. The - `paperless.conf.example` file was also updated to mirror the project - defaults. - -### 2.6.1 - -- We now have a logo, complete with a favicon :-) -- Removed some problematic tests. -- Fix the docker-compose example config to include a shared consume - volume so that using the push API will work for users of the Docker - install. Thanks to [Colin Frei](https://github.com/colinfrei) for - fixing this in - [\#466](https://github.com/the-paperless-project/paperless/pull/466). -- [khrise](https://github.com/khrise) submitted a pull request to - include the `added` property to the REST API - [\#471](https://github.com/the-paperless-project/paperless/pull/471). - -### 2.6.0 - -- Allow an infinite number of logs to be deleted. Thanks to - [Ulli](https://github.com/Ulli2k) for noting the problem in - [\#433](https://github.com/the-paperless-project/paperless/issues/433). -- Fix the `RecentCorrespondentsFilter` correspondents filter that was - added in 2.4 to play nice with the defaults. Thanks to - [tsia](https://github.com/tsia) and - [Sblop](https://github.com/Sblop) who pointed this out. - [\#423](https://github.com/the-paperless-project/paperless/issues/423). -- Updated dependencies to include (among other things) a security - patch to requests. -- Fix text in sample data for tests so that the language guesser stops - thinking that everything is in Catalan because we had _Lorem ipsum_ - in there. -- Tweaked the gunicorn sample command to use filesystem paths instead - of Python paths. - [\#441](https://github.com/the-paperless-project/paperless/pull/441) -- Added pretty colour boxes next to the hex values in the Tags - section, thanks to a pull request from [Joshua - Taillon](https://github.com/jat255) - [\#442](https://github.com/the-paperless-project/paperless/pull/442). -- Added a `.editorconfig` file to better specify coding style. -- [Joshua Taillon](https://github.com/jat255) also added some logic to - tie Paperless\' date guessing logic into how it parses file names on - import. - [\#440](https://github.com/the-paperless-project/paperless/pull/440) - -### 2.5.0 - -- **New dependency**: Paperless now optimises thumbnail generation - with [optipng](http://optipng.sourceforge.net/), so you\'ll need to - install that somewhere in your PATH or declare its location in - `PAPERLESS_OPTIPNG_BINARY`. The Docker image has already been - updated on the Docker Hub, so you just need to pull the latest one - from there if you\'re a Docker user. -- \"Login free\" instances of Paperless were breaking whenever you - tried to edit objects in the admin: adding/deleting tags or - correspondents, or even fixing spelling. This was due to the \"user - hack\" we were applying to sessions that weren\'t using a login, as - that hack user didn\'t have a valid id. The fix was to attribute the - first user id in the system to this hack user. - [\#394](https://github.com/the-paperless-project/paperless/issues/394) -- A problem in how we handle slug values on Tags and Correspondents - required a few changes to how we handle this field - [\#393](https://github.com/the-paperless-project/paperless/issues/393): - 1. Slugs are no longer editable. They\'re derived from the name of - the tag or correspondent at save time, so if you wanna change - the slug, you have to change the name, and even then you\'re - restricted to the rules of the `slugify()` function. The slug - value is still visible in the admin though. - 2. I\'ve added a migration to go over all existing tags & - correspondents and rewrite the `.slug` values to ones conforming - to the `slugify()` rules. - 3. The consumption process now uses the same rules as `.save()` in - determining a slug and using that to check for an existing - tag/correspondent. -- An annoying bug in the date capture code was causing some bogus - dates to be attached to documents, which in turn busted the UI. - Thanks to [Andrew Peng](https://github.com/pengc99) for reporting - this. - [\#414](https://github.com/the-paperless-project/paperless/issues/414). -- A bug in the Dockerfile meant that Tesseract language files weren\'t - being installed correctly. [euri10](https://github.com/euri10) was - quick to provide a fix: - [\#406](https://github.com/the-paperless-project/paperless/issues/406), - [\#413](https://github.com/the-paperless-project/paperless/pull/413). -- Document consumption is now wrapped in a transaction as per an old - ticket - [\#262](https://github.com/the-paperless-project/paperless/issues/262). -- The `get_date()` functionality of the parsers has been consolidated - onto the `DocumentParser` class since much of that code was - redundant anyway. - -### 2.4.0 - -- A new set of actions are now available thanks to - [jonaswinkler](https://github.com/jonaswinkler)\'s very first pull - request! You can now do nifty things like tag documents in bulk, or - set correspondents in bulk. - [\#405](https://github.com/the-paperless-project/paperless/pull/405) -- The import/export system is now a little smarter. By default, - documents are tagged as `unencrypted`, since exports are by their - nature unencrypted. It\'s now in the import step that we decide the - storage type. This allows you to export from an encrypted system and - import into an unencrypted one, or vice-versa. -- The migration history has been slightly modified to accommodate - PostgreSQL users. Additionally, you can now tell paperless to use - PostgreSQL simply by declaring `PAPERLESS_DBUSER` in your - environment. This will attempt to connect to your Postgres database - without a password unless you also set `PAPERLESS_DBPASS`. -- A bug was found in the REST API filter system that was the result of - an update of django-filter some time ago. This has now been patched - in - [\#412](https://github.com/the-paperless-project/paperless/issues/412). - Thanks to [thepill](https://github.com/thepill) for spotting it! - -### 2.3.0 - -- Support for consuming plain text & markdown documents was added by - [Joshua Taillon](https://github.com/jat255)! This was a - long-requested feature, and it\'s addition is likely to be greatly - appreciated by the community: - [\#395](https://github.com/the-paperless-project/paperless/pull/395) - Thanks also to [David Martin](https://github.com/ddddavidmartin) for - his assistance on the issue. -- [dubit0](https://github.com/dubit0) found & fixed a bug that - prevented management commands from running before we had an - operational database: - [\#396](https://github.com/the-paperless-project/paperless/pull/396) -- Joshua also added a simple update to the thumbnail generation - process to improve performance: - [\#399](https://github.com/the-paperless-project/paperless/pull/399) -- As his last bit of effort on this release, Joshua also added some - code to allow you to view the documents inline rather than download - them as an attachment. - [\#400](https://github.com/the-paperless-project/paperless/pull/400) -- Finally, [ahyear](https://github.com/ahyear) found a slip in the - Docker documentation and patched it. - [\#401](https://github.com/the-paperless-project/paperless/pull/401) - -### 2.2.1 - -- [Kyle Lucy](https://github.com/kmlucy) reported a bug quickly after - the release of 2.2.0 where we broke the `DISABLE_LOGIN` feature: - [\#392](https://github.com/the-paperless-project/paperless/issues/392). - -### 2.2.0 - -- Thanks to [dadosch](https://github.com/dadosch), [Wolfgang - Mader](https://github.com/wmader), and [Tim - Brooks](https://github.com/brookst) this is the first version of - Paperless that supports Django 2.0! As a result of their hard work, - you can now also run Paperless on Python 3.7 as well: - [\#386](https://github.com/the-paperless-project/paperless/issues/386) - & - [\#390](https://github.com/the-paperless-project/paperless/pull/390). -- [Stéphane Brunner](https://github.com/sbrunner) added a few lines of - code that made tagging interface a lot easier on those of us with - lots of different tags: - [\#391](https://github.com/the-paperless-project/paperless/pull/391). -- [Kilian Koeltzsch](https://github.com/kiliankoe) noticed a bug in - how we capture & automatically create tags, so that\'s fixed now - too: - [\#384](https://github.com/the-paperless-project/paperless/issues/384). -- [erikarvstedt](https://github.com/erikarvstedt) tweaked the - behaviour of the test suite to be better behaved for packaging - environments: - [\#383](https://github.com/the-paperless-project/paperless/pull/383). -- [Lukasz Soluch](https://github.com/LukaszSolo) added CORS support to - make building a new Javascript-based front-end cleaner & easier: - [\#387](https://github.com/the-paperless-project/paperless/pull/387). - -### 2.1.0 - -- [Enno Lohmeier](https://github.com/elohmeier) added three simple - features that make Paperless a lot more user (and developer) - friendly: - 1. There\'s a new search box on the front page: - [\#374](https://github.com/the-paperless-project/paperless/pull/374). - 2. The correspondents & tags pages now have a column showing the - number of relevant documents: - [\#375](https://github.com/the-paperless-project/paperless/pull/375). - 3. The Dockerfile has been tweaked to build faster for those of us - who are doing active development on Paperless using the Docker - environment: - [\#376](https://github.com/the-paperless-project/paperless/pull/376). -- You now also have the ability to customise the interface to your - heart\'s content by creating a file called `overrides.css` and/or - `overrides.js` in the root of your media directory. Thanks to [Mark - McFate](https://github.com/SummittDweller) for this idea: - [\#371](https://github.com/the-paperless-project/paperless/issues/371) - -### 2.0.0 - -This is a big release as we\'ve changed a core-functionality of -Paperless: we no longer encrypt files with GPG by default. - -The reasons for this are many, but it boils down to that the encryption -wasn\'t really all that useful, as files on-disk were still accessible -so long as you had the key, and the key was most typically stored in the -config file. In other words, your files are only as safe as the -`paperless` user is. In addition to that, _the contents of the documents -were never encrypted_, so important numbers etc. were always accessible -simply by querying the database. Still, it was better than nothing, but -the consensus from users appears to be that it was more an annoyance -than anything else, so this feature is now turned off unless you -explicitly set a passphrase in your config file. - -### Migrating from 1.x - -Encryption isn\'t gone, it\'s just off for new users. So long as you -have `PAPERLESS_PASSPHRASE` set in your config or your environment, -Paperless should continue to operate as it always has. If however, you -want to drop encryption too, you only need to do two things: - -1. Run - `./manage.py migrate && ./manage.py change_storage_type gpg unencrypted`. - This will go through your entire database and Decrypt All The - Things. -2. Remove `PAPERLESS_PASSPHRASE` from your `paperless.conf` file, or - simply stop declaring it in your environment. - -Special thanks to [erikarvstedt](https://github.com/erikarvstedt), -[matthewmoto](https://github.com/matthewmoto), and -[mcronce](https://github.com/mcronce) who did the bulk of the work on -this big change. - -### 1.4.0 - -- [Quentin Dawans](https://github.com/ovv) has refactored the document - consumer to allow for some command-line options. Notably, you can - now direct it to consume from a particular `--directory`, limit the - `--loop-time`, set the time between mail server checks with - `--mail-delta` or just run it as a one-off with `--one-shot`. See - [\#305](https://github.com/the-paperless-project/paperless/issues/305) - & - [\#313](https://github.com/the-paperless-project/paperless/pull/313) - for more information. -- Refactor the use of travis/tox/pytest/coverage into two files: - `.travis.yml` and `setup.cfg`. -- Start generating requirements.txt from a Pipfile. I\'ll probably - switch over to just using pipenv in the future. -- All for a alternative FreeBSD-friendly location for - `paperless.conf`. Thanks to [Martin - Arendtsen](https://github.com/Arendtsen) who provided this - ([\#322](https://github.com/the-paperless-project/paperless/pull/322)). -- Document consumption events are now logged in the Django admin - events log. Thanks to [CkuT](https://github.com/CkuT) for doing the - legwork on this one and to [Quentin Dawans](https://github.com/ovv) - & [David Martin](https://github.com/ddddavidmartin) for helping to - coordinate & work out how the feature would be developed. -- [erikarvstedt](https://github.com/erikarvstedt) contributed a pull - request - ([\#328](https://github.com/the-paperless-project/paperless/pull/328)) - to add `--noreload` to the default server start process. This helps - reduce the load imposed by the running webservice. -- Through some discussion on - [\#253](https://github.com/the-paperless-project/paperless/issues/253) - and - [\#323](https://github.com/the-paperless-project/paperless/issues/323), - we\'ve removed a few of the hardcoded URL values to make it easier - for people to host Paperless on a subdirectory. Thanks to [Quentin - Dawans](https://github.com/ovv) and [Kyle - Lucy](https://github.com/kmlucy) for helping to work this out. -- The clickable area for documents on the listing page has been - increased to a more predictable space thanks to a glorious hack from - [erikarvstedt](https://github.com/erikarvstedt) in - [\#344](https://github.com/the-paperless-project/paperless/pull/344). -- [Strubbl](https://github.com/strubbl) noticed an annoying bug in the - bash script wrapping the Docker entrypoint and fixed it with some - very creating Bash skills: - [\#352](https://github.com/the-paperless-project/paperless/pull/352). -- You can now use the search field to find documents by tag thanks to - [thinkjk](https://github.com/thinkjk)\'s _first ever issue_: - [\#354](https://github.com/the-paperless-project/paperless/issues/354). -- Inotify is now being used to detect additions to the consume - directory thanks to some excellent work from - [erikarvstedt](https://github.com/erikarvstedt) on - [\#351](https://github.com/the-paperless-project/paperless/pull/351) - -### 1.3.0 - -- You can now run Paperless without a login, though you\'ll still have - to create at least one user. This is thanks to a pull-request from - [matthewmoto](https://github.com/matthewmoto): - [\#295](https://github.com/the-paperless-project/paperless/pull/295). - Note that logins are still required by default, and that you need to - disable them by setting `PAPERLESS_DISABLE_LOGIN="true"` in your - environment or in `/etc/paperless.conf`. -- Fix for - [\#303](https://github.com/the-paperless-project/paperless/issues/303) - where sketchily-formatted documents could cause the consumer to - break and insert half-records into the database breaking all sorts - of things. We now capture the return codes of both `convert` and - `unpaper` and fail-out nicely. -- Fix for additional date types thanks to input from - [Isaac](https://github.com/isaacsando) and code from - [BastianPoe](https://github.com/BastianPoe) - ([\#301](https://github.com/the-paperless-project/paperless/issues/301)). -- Fix for running migrations in the Docker container - ([\#299](https://github.com/the-paperless-project/paperless/issues/299)). - Thanks to [Georgi Todorov](https://github.com/TeraHz) for the fix - ([\#300](https://github.com/the-paperless-project/paperless/pull/300)) - and to [Pit](https://github.com/pitkley) for the review. -- Fix for Docker cases where the issuing user is not UID 1000. This - was a collaborative fix between [Jeffrey - Portman](https://github.com/ChromoX) and - [Pit](https://github.com/pitkley) in - [\#311](https://github.com/the-paperless-project/paperless/pull/311) - and - [\#312](https://github.com/the-paperless-project/paperless/pull/312) - to fix - [\#306](https://github.com/the-paperless-project/paperless/issues/306). -- Patch the historical migrations to support MySQL\'s um, - _interesting_ way of handing indexes - ([\#308](https://github.com/the-paperless-project/paperless/issues/308)). - Thanks to [Simon Taddiken](https://github.com/skuzzle) for reporting - the problem and helping me find where to fix it. - -### 1.2.0 - -- New Docker image, now based on Alpine, thanks to the efforts of - [addadi](https://github.com/addadi) and - [Pit](https://github.com/pitkley). This new image is dramatically - smaller than the Debian-based one, and it also has [a new home on - Docker Hub](https://hub.docker.com/r/danielquinn/paperless/). A - proper thank-you to [Pit](https://github.com/pitkley) for hosting - the image on his Docker account all this time, but after some - discussion, we decided the image needed a more _official-looking_ - home. -- [BastianPoe](https://github.com/BastianPoe) has added the - long-awaited feature to automatically skip the OCR step when the PDF - already contains text. This can be overridden by setting - `PAPERLESS_OCR_ALWAYS=YES` either in your `paperless.conf` or in the - environment. Note that this also means that Paperless now requires - `libpoppler-cpp-dev` to be installed. **Important**: You\'ll need to - run `pip install -r requirements.txt` after the usual `git pull` to - properly update. -- [BastianPoe](https://github.com/BastianPoe) has also contributed a - monumental amount of work - ([\#291](https://github.com/the-paperless-project/paperless/pull/291)) - to solving - [\#158](https://github.com/the-paperless-project/paperless/issues/158): - setting the document creation date based on finding a date in the - document text. - -### 1.1.0 - -- Fix for - [\#283](https://github.com/the-paperless-project/paperless/issues/283), - a redirect bug which broke interactions with paperless-desktop. - Thanks to [chris-aeviator](https://github.com/chris-aeviator) for - reporting it. -- Addition of an optional new financial year filter, courtesy of - [David Martin](https://github.com/ddddavidmartin) - [\#256](https://github.com/the-paperless-project/paperless/pull/256) -- Fixed a typo in how thumbnails were named in exports - [\#285](https://github.com/the-paperless-project/paperless/pull/285), - courtesy of [Dan Panzarella](https://github.com/pzl) - -### 1.0.0 - -- Upgrade to Django 1.11. **You\'ll need to run \`\`pip install -r - requirements.txt\`\` after the usual \`\`git pull\`\` to properly - update**. -- Replace the templatetag-based hack we had for document listing in - favour of a slightly less ugly solution in the form of another - template tag with less copypasta. -- Support for multi-word-matches for auto-tagging thanks to an - excellent patch from [ishirav](https://github.com/ishirav) - [\#277](https://github.com/the-paperless-project/paperless/pull/277). -- Fixed a CSS bug reported by [Stefan Hagen](https://github.com/xkpd3) - that caused an overlapping of the text and checkboxes under some - resolutions - [\#272](https://github.com/the-paperless-project/paperless/issues/272). -- Patched the Docker config to force the serving of static files. - Credit for this one goes to [dev-rke](https://github.com/dev-rke) - via - [\#248](https://github.com/the-paperless-project/paperless/issues/248). -- Fix file permissions during Docker start up thanks to - [Pit](https://github.com/pitkley) on - [\#268](https://github.com/the-paperless-project/paperless/pull/268). -- Date fields in the admin are now expressed as HTML5 date fields - thanks to [Lukas Winkler](https://github.com/Findus23)\'s issue - [\#278](https://github.com/the-paperless-project/paperless/issues/248) - -### 0.8.0 - -- Paperless can now run in a subdirectory on a host (`/paperless`), - rather than always running in the root (`/`) thanks to - [maphy-psd](https://github.com/maphy-psd)\'s work on - [\#255](https://github.com/the-paperless-project/paperless/pull/255). - -### 0.7.0 - -- **Potentially breaking change**: As per - [\#235](https://github.com/the-paperless-project/paperless/issues/235), - Paperless will no longer automatically delete documents attached to - correspondents when those correspondents are themselves deleted. - This was Django\'s default behaviour, but didn\'t make much sense in - Paperless\' case. Thanks to [Thomas - Brueggemann](https://github.com/thomasbrueggemann) and [David - Martin](https://github.com/ddddavidmartin) for their input on this - one. -- Fix for - [\#232](https://github.com/the-paperless-project/paperless/issues/232) - wherein Paperless wasn\'t recognising `.tif` files properly. Thanks - to [ayounggun](https://github.com/ayounggun) for reporting this one - and to [Kusti Skytén](https://github.com/kskyten) for posting the - correct solution in the Github issue. - -### 0.6.0 - -- Abandon the shared-secret trick we were using for the POST API in - favour of BasicAuth or Django session. -- Fix the POST API so it actually works. - [\#236](https://github.com/the-paperless-project/paperless/issues/236) -- **Breaking change**: We\'ve dropped the use of - `PAPERLESS_SHARED_SECRET` as it was being used both for the API (now - replaced with a normal auth) and form email polling. Now that we\'re - only using it for email, this variable has been renamed to - `PAPERLESS_EMAIL_SECRET`. The old value will still work for a while, - but you should change your config if you\'ve been using the email - polling feature. Thanks to [Joshua - Gilman](https://github.com/jmgilman) for all the help with this - feature. - -### 0.5.0 - -- Support for fuzzy matching in the auto-tagger & auto-correspondent - systems thanks to [Jake Gysland](https://github.com/jgysland)\'s - patch - [\#220](https://github.com/the-paperless-project/paperless/pull/220). -- Modified the Dockerfile to prepare an export directory - ([\#212](https://github.com/the-paperless-project/paperless/pull/212)). - Thanks to combined efforts from [Pit](https://github.com/pitkley) - and [Strubbl](https://github.com/strubbl) in working out the kinks - on this one. -- Updated the import/export scripts to include support for thumbnails. - Big thanks to [CkuT](https://github.com/CkuT) for finding this - shortcoming and doing the work to get it fixed in - [\#224](https://github.com/the-paperless-project/paperless/pull/224). -- All of the following changes are thanks to [David - Martin](https://github.com/ddddavidmartin): \* Bumped the dependency on pyocr to 0.4.7 so new users can make use - of Tesseract 4 if they so prefer - ([\#226](https://github.com/the-paperless-project/paperless/pull/226)). - - Fixed a number of issues with the automated mail handler - ([\#227](https://github.com/the-paperless-project/paperless/pull/227), - [\#228](https://github.com/the-paperless-project/paperless/pull/228)) - - Amended the documentation for better handling of systemd service - files - ([\#229](https://github.com/the-paperless-project/paperless/pull/229)) - - Amended the Django Admin configuration to have nice headers - ([\#230](https://github.com/the-paperless-project/paperless/pull/230)) - -### 0.4.1 - -- Fix for - [\#206](https://github.com/the-paperless-project/paperless/issues/206) - wherein the pluggable parser didn\'t recognise files with all-caps - suffixes like `.PDF` - -### 0.4.0 - -- Introducing reminders. See - [\#199](https://github.com/the-paperless-project/paperless/issues/199) - for more information, but the short explanation is that you can now - attach simple notes & times to documents which are made available - via the API. Currently, the default API (basically just the Django - admin) doesn\'t really make use of this, but [Thomas - Brueggemann](https://github.com/thomasbrueggemann) over at - [Paperless - Desktop](https://github.com/thomasbrueggemann/paperless-desktop) has - said that he would like to make use of this feature in his project. - -### 0.3.6 - -- Fix for - [\#200](https://github.com/the-paperless-project/paperless/issues/200) - (!!) where the API wasn\'t configured to allow updating the - correspondent or the tags for a document. -- The `content` field is now optional, to allow for the edge case of a - purely graphical document. -- You can no longer add documents via the admin. This never worked in - the first place, so all I\'ve done here is remove the link to the - broken form. -- The consumer code has been heavily refactored to support a pluggable - interface. Install a paperless consumer via pip and tell paperless - about it with an environment variable, and you\'re good to go. - Proper documentation is on its way. - -### 0.3.5 - -- A serious facelift for the documents listing page wherein we drop - the tabular layout in favour of a tiled interface. -- Users can now configure the number of items per page. -- Fix for - [\#171](https://github.com/the-paperless-project/paperless/issues/171): - Allow users to specify their own `SECRET_KEY` value. -- Moved the dotenv loading to the top of settings.py -- Fix for - [\#112](https://github.com/the-paperless-project/paperless/issues/112): - Added checks for binaries required for document consumption. - -### 0.3.4 - -- Removal of django-suit due to a licensing conflict I bumped into in - 0.3.3. Note that you _can_ use Django Suit with Paperless, but only - in a non-profit situation as their free license prohibits for-profit - use. As a result, I can\'t bundle Suit with Paperless without - conflicting with the GPL. Further development will be done against - the stock Django admin. -- I shrunk the thumbnails a little \'cause they were too big for me, - even on my high-DPI monitor. -- BasicAuth support for document and thumbnail downloads, as well as - the Push API thanks to \@thomasbrueggemann. See - [\#179](https://github.com/the-paperless-project/paperless/pull/179). - -### 0.3.3 - -- Thumbnails in the UI and a Django-suit -based face-lift courtesy of - \@ekw! -- Timezone, items per page, and default language are now all - configurable, also thanks to \@ekw. - -### 0.3.2 - -- Fix for - [\#172](https://github.com/the-paperless-project/paperless/issues/172): - defaulting ALLOWED_HOSTS to `["*"]` and allowing the user to set - her own value via `PAPERLESS_ALLOWED_HOSTS` should the need arise. - -### 0.3.1 - -- Added a default value for `CONVERT_BINARY` - -### 0.3.0 - -- Updated to using django-filter 1.x -- Added some system checks so new users aren\'t confused by - misconfigurations. -- Consumer loop time is now configurable for systems with slow writes. - Just set `PAPERLESS_CONSUMER_LOOP_TIME` to a number of seconds. The - default is 10. -- As per - [\#44](https://github.com/the-paperless-project/paperless/issues/44), - we\'ve removed support for `PAPERLESS_CONVERT`, `PAPERLESS_CONSUME`, - and `PAPERLESS_SECRET`. Please use `PAPERLESS_CONVERT_BINARY`, - `PAPERLESS_CONSUMPTION_DIR`, and `PAPERLESS_SHARED_SECRET` - respectively instead. - -### 0.2.0 - -- [\#150](https://github.com/the-paperless-project/paperless/pull/150): - The media root is now a variable you can set in `paperless.conf`. -- [\#148](https://github.com/the-paperless-project/paperless/pull/148): - The database location (sqlite) is now a variable you can set in - `paperless.conf`. -- [\#146](https://github.com/the-paperless-project/paperless/issues/146): - Fixed a bug that allowed unauthorised access to the `/fetch` URL. -- [\#131](https://github.com/the-paperless-project/paperless/issues/131): - Document files are now automatically removed from disk when they\'re - deleted in Paperless. -- [\#121](https://github.com/the-paperless-project/paperless/issues/121): - Fixed a bug where Paperless wasn\'t setting document creation time - based on the file naming scheme. -- [\#81](https://github.com/the-paperless-project/paperless/issues/81): - Added a hook to run an arbitrary script after every document is - consumed. -- [\#98](https://github.com/the-paperless-project/paperless/issues/98): - Added optional environment variables for ImageMagick so that it - doesn\'t explode when handling Very Large Documents or when it\'s - just running on a low-memory system. Thanks to [Florian - Harr](https://github.com/evils) for his help on this one. -- [\#89](https://github.com/the-paperless-project/paperless/issues/89) - Ported the auto-tagging code to correspondents as well. Thanks to - [Justin Snyman](https://github.com/stringlytyped) for the pointers - in the issue queue. -- Added support for guessing the date from the file name along with - the correspondent, title, and tags. Thanks to [Tikitu de - Jager](https://github.com/tikitu) for his pull request that I took - forever to merge and to [Pit](https://github.com/pitkley) for his - efforts on the regex front. -- [\#94](https://github.com/the-paperless-project/paperless/issues/94): - Restored support for changing the created date in the UI. Thanks to - [Martin Honermeyer](https://github.com/djmaze) and [Tim - White](https://github.com/timwhite) for working with me on this. - -### 0.1.1 - -- Potentially **Breaking Change**: All references to \"sender\" in the - code have been renamed to \"correspondent\" to better reflect the - nature of the property (one could quite reasonably scan a document - before sending it to someone.) -- [\#67](https://github.com/the-paperless-project/paperless/issues/67): - Rewrote the document exporter and added a new importer that allows - for full metadata retention without depending on the file name and - modification time. A big thanks to [Tikitu de - Jager](https://github.com/tikitu), - [Pit](https://github.com/pitkley), [Florian - Jung](https://github.com/the01), and [Christopher - Luu](https://github.com/nuudles) for their code snippets and - contributing conversation that lead to this change. -- [\#20](https://github.com/the-paperless-project/paperless/issues/20): - Added _unpaper_ support to help in cleaning up the scanned image - before it\'s OCR\'d. Thanks to [Pit](https://github.com/pitkley) for - this one. -- [\#71](https://github.com/the-paperless-project/paperless/issues/71) - Added (encrypted) thumbnails in anticipation of a proper UI. -- [\#68](https://github.com/the-paperless-project/paperless/issues/68): - Added support for using a proper config file at - `/etc/paperless.conf` and modified the systemd unit files to use it. -- Refactored the Vagrant installation process to use environment - variables rather than asking the user to modify `settings.py`. -- [\#44](https://github.com/the-paperless-project/paperless/issues/44): - Harmonise environment variable names with constant names. -- [\#60](https://github.com/the-paperless-project/paperless/issues/60): - Setup logging to actually use the Python native logging framework. -- [\#53](https://github.com/the-paperless-project/paperless/issues/53): - Fixed an annoying bug that caused `.jpeg` and `.JPG` images to be - imported but made unavailable. - -### 0.1.0 - -- Docker support! Big thanks to [Wayne - Werner](https://github.com/waynew), [Brian - Conn](https://github.com/TheConnMan), and [Tikitu de - Jager](https://github.com/tikitu) for this one, and especially to - [Pit](https://github.com/pitkley) who spearheadded this effort. -- A simple REST API is in place, but it should be considered unstable. -- Cleaned up the consumer to use temporary directories instead of a - single scratch space. (Thanks [Pit](https://github.com/pitkley)) -- Improved the efficiency of the consumer by parsing pages more - intelligently and introducing a threaded OCR process (thanks again - [Pit](https://github.com/pitkley)). -- [\#45](https://github.com/the-paperless-project/paperless/issues/45): - Cleaned up the logic for tag matching. Reported by - [darkmatter](https://github.com/darkmatter). -- [\#47](https://github.com/the-paperless-project/paperless/issues/47): - Auto-rotate landscape documents. Reported by - [Paul](https://github.com/polo2ro) and fixed by - [Pit](https://github.com/pitkley). -- [\#48](https://github.com/the-paperless-project/paperless/issues/48): - Matching algorithms should do so on a word boundary - ([darkmatter](https://github.com/darkmatter)) -- [\#54](https://github.com/the-paperless-project/paperless/issues/54): - Documented the re-tagger ([zedster](https://github.com/zedster)) -- [\#57](https://github.com/the-paperless-project/paperless/issues/57): - Make sure file is preserved on import failure - ([darkmatter](https://github.com/darkmatter)) -- Added tox with pep8 checking - -### 0.0.6 - -- Added support for parallel OCR (significant work from - [Pit](https://github.com/pitkley)) -- Sped up the language detection (significant work from - [Pit](https://github.com/pitkley)) -- Added simple logging - -### 0.0.5 - -- Added support for image files as documents (png, jpg, gif, tiff) -- Added a crude means of HTTP POST for document imports -- Added IMAP mail support -- Added a re-tagging utility -- Documentation for the above as well as data migration - -### 0.0.4 - -- Added automated tagging basted on keyword matching -- Cleaned up the document listing page -- Removed `User` and `Group` from the admin -- Added `pytz` to the list of requirements - -### 0.0.3 - -- Added basic tagging - -### 0.0.2 - -- Added language detection -- Added datestamps to `document_exporter`. -- Changed `settings.TESSERACT_LANGUAGE` to `settings.OCR_LANGUAGE`. - -### 0.0.1 - -- Initial release diff --git a/docs/changelog.rst b/docs/changelog.rst new file mode 100644 index 000000000..879188340 --- /dev/null +++ b/docs/changelog.rst @@ -0,0 +1,11 @@ +.. _changelog: + +********* +Changelog +********* + +.. cssclass:: redirect-notice + + The Paperless-ngx documentation has permanently moved. + + You will be redirected shortly... diff --git a/docs/configuration.rst b/docs/configuration.rst index 1684cc63e..d8aa7c857 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -4,928 +4,9 @@ Configuration ************* -Paperless provides a wide range of customizations. -Depending on how you run paperless, these settings have to be defined in different -places. -* If you run paperless on docker, ``paperless.conf`` is not used. Rather, configure - paperless by copying necessary options to ``docker-compose.env``. -* If you are running paperless on anything else, paperless will search for the - configuration file in these locations and use the first one it finds: +.. cssclass:: redirect-notice - .. code:: + The Paperless-ngx documentation has permanently moved. - /path/to/paperless/paperless.conf - /etc/paperless.conf - /usr/local/etc/paperless.conf - - -Required services -################# - -PAPERLESS_REDIS= - This is required for processing scheduled tasks such as email fetching, index - optimization and for training the automatic document matcher. - - * If your Redis server needs login credentials PAPERLESS_REDIS = ``redis://:@:`` - - * With the requirepass option PAPERLESS_REDIS = ``redis://:@:`` - - `More information on securing your Redis Instance `_. - - Defaults to redis://localhost:6379. - -PAPERLESS_DBENGINE= - Optional, gives the ability to choose Postgres or MariaDB for database engine. - Available options are `postgresql` and `mariadb`. - - Default is `postgresql`. - - .. warning:: - - Using MariaDB comes with some caveats. See :ref:`advanced-mysql-caveats` for details. - - -PAPERLESS_DBHOST= - By default, sqlite is used as the database backend. This can be changed here. - - Set PAPERLESS_DBHOST and another database will be used instead of sqlite. - -PAPERLESS_DBPORT= - Adjust port if necessary. - - Default is 5432. - -PAPERLESS_DBNAME= - Database name in PostgreSQL or MariaDB. - - Defaults to "paperless". - -PAPERLESS_DBUSER= - Database user in PostgreSQL or MariaDB. - - Defaults to "paperless". - -PAPERLESS_DBPASS= - Database password for PostgreSQL or MariaDB. - - Defaults to "paperless". - -PAPERLESS_DBSSLMODE= - SSL mode to use when connecting to PostgreSQL. - - See `the official documentation about sslmode `_. - - Default is ``prefer``. - -PAPERLESS_DB_TIMEOUT= - Amount of time for a database connection to wait for the database to unlock. - Mostly applicable for an sqlite based installation, consider changing to postgresql - if you need to increase this. - - Defaults to unset, keeping the Django defaults. - -Paths and folders -################# - -PAPERLESS_CONSUMPTION_DIR= - This where your documents should go to be consumed. Make sure that it exists - and that the user running the paperless service can read/write its contents - before you start Paperless. - - Don't change this when using docker, as it only changes the path within the - container. Change the local consumption directory in the docker-compose.yml - file instead. - - Defaults to "../consume/", relative to the "src" directory. - -PAPERLESS_DATA_DIR= - This is where paperless stores all its data (search index, SQLite database, - classification model, etc). - - Defaults to "../data/", relative to the "src" directory. - -PAPERLESS_TRASH_DIR= - Instead of removing deleted documents, they are moved to this directory. - - This must be writeable by the user running paperless. When running inside - docker, ensure that this path is within a permanent volume (such as - "../media/trash") so it won't get lost on upgrades. - - Defaults to empty (i.e. really delete documents). - -PAPERLESS_MEDIA_ROOT= - This is where your documents and thumbnails are stored. - - You can set this and PAPERLESS_DATA_DIR to the same folder to have paperless - store all its data within the same volume. - - Defaults to "../media/", relative to the "src" directory. - -PAPERLESS_STATICDIR= - Override the default STATIC_ROOT here. This is where all static files - created using "collectstatic" manager command are stored. - - Unless you're doing something fancy, there is no need to override this. - - Defaults to "../static/", relative to the "src" directory. - -PAPERLESS_FILENAME_FORMAT= - Changes the filenames paperless uses to store documents in the media directory. - See :ref:`advanced-file_name_handling` for details. - - Default is none, which disables this feature. - -PAPERLESS_FILENAME_FORMAT_REMOVE_NONE= - Tells paperless to replace placeholders in `PAPERLESS_FILENAME_FORMAT` that would resolve - to 'none' to be omitted from the resulting filename. This also holds true for directory - names. - See :ref:`advanced-file_name_handling` for details. - - Defaults to `false` which disables this feature. - -PAPERLESS_LOGGING_DIR= - This is where paperless will store log files. - - Defaults to "``PAPERLESS_DATA_DIR``/log/". - - -Logging -####### - -PAPERLESS_LOGROTATE_MAX_SIZE= - Maximum file size for log files before they are rotated, in bytes. - - Defaults to 1 MiB. - -PAPERLESS_LOGROTATE_MAX_BACKUPS= - Number of rotated log files to keep. - - Defaults to 20. - -.. _hosting-and-security: - -Hosting & Security -################## - -PAPERLESS_SECRET_KEY= - Paperless uses this to make session tokens. If you expose paperless on the - internet, you need to change this, since the default secret is well known. - - Use any sequence of characters. The more, the better. You don't need to - remember this. Just face-roll your keyboard. - - Default is listed in the file ``src/paperless/settings.py``. - -PAPERLESS_URL= - This setting can be used to set the three options below (ALLOWED_HOSTS, - CORS_ALLOWED_HOSTS and CSRF_TRUSTED_ORIGINS). If the other options are - set the values will be combined with this one. Do not include a trailing - slash. E.g. https://paperless.domain.com - - Defaults to empty string, leaving the other settings unaffected. - -PAPERLESS_CSRF_TRUSTED_ORIGINS= - A list of trusted origins for unsafe requests (e.g. POST). As of Django 4.0 - this is required to access the Django admin via the web. - See https://docs.djangoproject.com/en/4.0/ref/settings/#csrf-trusted-origins - - Can also be set using PAPERLESS_URL (see above). - - Defaults to empty string, which does not add any origins to the trusted list. - -PAPERLESS_ALLOWED_HOSTS= - If you're planning on putting Paperless on the open internet, then you - really should set this value to the domain name you're using. Failing to do - so leaves you open to HTTP host header attacks: - https://docs.djangoproject.com/en/3.1/topics/security/#host-header-validation - - Just remember that this is a comma-separated list, so "example.com" is fine, - as is "example.com,www.example.com", but NOT " example.com" or "example.com," - - Can also be set using PAPERLESS_URL (see above). - - If manually set, please remember to include "localhost". Otherwise docker - healthcheck will fail. - - Defaults to "*", which is all hosts. - -PAPERLESS_CORS_ALLOWED_HOSTS= - You need to add your servers to the list of allowed hosts that can do CORS - calls. Set this to your public domain name. - - Can also be set using PAPERLESS_URL (see above). - - Defaults to "http://localhost:8000". - -PAPERLESS_FORCE_SCRIPT_NAME= - To host paperless under a subpath url like example.com/paperless you set - this value to /paperless. No trailing slash! - - Defaults to none, which hosts paperless at "/". - -PAPERLESS_STATIC_URL= - Override the STATIC_URL here. Unless you're hosting Paperless off a - subdomain like /paperless/, you probably don't need to change this. - If you do change it, be sure to include the trailing slash. - - Defaults to "/static/". - - .. note:: - - When hosting paperless behind a reverse proxy like Traefik or Nginx at a subpath e.g. - example.com/paperlessngx you will also need to set ``PAPERLESS_FORCE_SCRIPT_NAME`` - (see above). - -PAPERLESS_AUTO_LOGIN_USERNAME= - Specify a username here so that paperless will automatically perform login - with the selected user. - - .. danger:: - - Do not use this when exposing paperless on the internet. There are no - checks in place that would prevent you from doing this. - - Defaults to none, which disables this feature. - -PAPERLESS_ADMIN_USER= - If this environment variable is specified, Paperless automatically creates - a superuser with the provided username at start. This is useful in cases - where you can not run the `createsuperuser` command separately, such as Kubernetes - or AWS ECS. - - Requires `PAPERLESS_ADMIN_PASSWORD` to be set. - - .. note:: - - This will not change an existing [super]user's password, nor will - it recreate a user that already exists. You can leave this throughout - the lifecycle of the containers. - -PAPERLESS_ADMIN_MAIL= - (Optional) Specify superuser email address. Only used when - `PAPERLESS_ADMIN_USER` is set. - - Defaults to ``root@localhost``. - -PAPERLESS_ADMIN_PASSWORD= - Only used when `PAPERLESS_ADMIN_USER` is set. - This will be the password of the automatically created superuser. - - -PAPERLESS_COOKIE_PREFIX= - Specify a prefix that is added to the cookies used by paperless to identify - the currently logged in user. This is useful for when you're running two - instances of paperless on the same host. - - After changing this, you will have to login again. - - Defaults to ``""``, which does not alter the cookie names. - -PAPERLESS_ENABLE_HTTP_REMOTE_USER= - Allows authentication via HTTP_REMOTE_USER which is used by some SSO - applications. - - .. warning:: - - This will allow authentication by simply adding a ``Remote-User: `` header - to a request. Use with care! You especially *must* ensure that any such header is not - passed from your proxy server to paperless. - - If you're exposing paperless to the internet directly, do not use this. - - Also see the warning `in the official documentation `. - - Defaults to `false` which disables this feature. - -PAPERLESS_HTTP_REMOTE_USER_HEADER_NAME= - If `PAPERLESS_ENABLE_HTTP_REMOTE_USER` is enabled, this property allows to - customize the name of the HTTP header from which the authenticated username - is extracted. Values are in terms of - [HttpRequest.META](https://docs.djangoproject.com/en/3.1/ref/request-response/#django.http.HttpRequest.META). - Thus, the configured value must start with `HTTP_` followed by the - normalized actual header name. - - Defaults to `HTTP_REMOTE_USER`. - -PAPERLESS_LOGOUT_REDIRECT_URL= - URL to redirect the user to after a logout. This can be used together with - `PAPERLESS_ENABLE_HTTP_REMOTE_USER` to redirect the user back to the SSO - application's logout page. - - Defaults to None, which disables this feature. - -.. _configuration-ocr: - -OCR settings -############ - -Paperless uses `OCRmyPDF `_ for -performing OCR on documents and images. Paperless uses sensible defaults for -most settings, but all of them can be configured to your needs. - -PAPERLESS_OCR_LANGUAGE= - Customize the language that paperless will attempt to use when - parsing documents. - - It should be a 3-letter language code consistent with ISO - 639: https://www.loc.gov/standards/iso639-2/php/code_list.php - - Set this to the language most of your documents are written in. - - This can be a combination of multiple languages such as ``deu+eng``, - in which case tesseract will use whatever language matches best. - Keep in mind that tesseract uses much more cpu time with multiple - languages enabled. - - Defaults to "eng". - - Note: If your language contains a '-' such as chi-sim, you must use chi_sim - -PAPERLESS_OCR_MODE= - Tell paperless when and how to perform ocr on your documents. Four modes - are available: - - * ``skip``: Paperless skips all pages and will perform ocr only on pages - where no text is present. This is the safest option. - * ``skip_noarchive``: In addition to skip, paperless won't create an - archived version of your documents when it finds any text in them. - This is useful if you don't want to have two almost-identical versions - of your digital documents in the media folder. This is the fastest option. - * ``redo``: Paperless will OCR all pages of your documents and attempt to - replace any existing text layers with new text. This will be useful for - documents from scanners that already performed OCR with insufficient - results. It will also perform OCR on purely digital documents. - - This option may fail on some documents that have features that cannot - be removed, such as forms. In this case, the text from the document is - used instead. - * ``force``: Paperless rasterizes your documents, converting any text - into images and puts the OCRed text on top. This works for all documents, - however, the resulting document may be significantly larger and text - won't appear as sharp when zoomed in. - - The default is ``skip``, which only performs OCR when necessary and always - creates archived documents. - - Read more about this in the `OCRmyPDF documentation `_. - -PAPERLESS_OCR_CLEAN= - Tells paperless to use ``unpaper`` to clean any input document before - sending it to tesseract. This uses more resources, but generally results - in better OCR results. The following modes are available: - - * ``clean``: Apply unpaper. - * ``clean-final``: Apply unpaper, and use the cleaned images to build the - output file instead of the original images. - * ``none``: Do not apply unpaper. - - Defaults to ``clean``. - - .. note:: - - ``clean-final`` is incompatible with ocr mode ``redo``. When both - ``clean-final`` and the ocr mode ``redo`` is configured, ``clean`` - is used instead. - -PAPERLESS_OCR_DESKEW= - Tells paperless to correct skewing (slight rotation of input images mainly - due to improper scanning) - - Defaults to ``true``, which enables this feature. - - .. note:: - - Deskewing is incompatible with ocr mode ``redo``. Deskewing will get - disabled automatically if ``redo`` is used as the ocr mode. - -PAPERLESS_OCR_ROTATE_PAGES= - Tells paperless to correct page rotation (90°, 180° and 270° rotation). - - If you notice that paperless is not rotating incorrectly rotated - pages (or vice versa), try adjusting the threshold up or down (see below). - - Defaults to ``true``, which enables this feature. - - -PAPERLESS_OCR_ROTATE_PAGES_THRESHOLD= - Adjust the threshold for automatic page rotation by ``PAPERLESS_OCR_ROTATE_PAGES``. - This is an arbitrary value reported by tesseract. "15" is a very conservative value, - whereas "2" is a very aggressive option and will often result in correctly rotated pages - being rotated as well. - - Defaults to "12". - -PAPERLESS_OCR_OUTPUT_TYPE= - Specify the the type of PDF documents that paperless should produce. - - * ``pdf``: Modify the PDF document as little as possible. - * ``pdfa``: Convert PDF documents into PDF/A-2b documents, which is a - subset of the entire PDF specification and meant for storing - documents long term. - * ``pdfa-1``, ``pdfa-2``, ``pdfa-3`` to specify the exact version of - PDF/A you wish to use. - - If not specified, ``pdfa`` is used. Remember that paperless also keeps - the original input file as well as the archived version. - - -PAPERLESS_OCR_PAGES= - Tells paperless to use only the specified amount of pages for OCR. Documents - with less than the specified amount of pages get OCR'ed completely. - - Specifying 1 here will only use the first page. - - When combined with ``PAPERLESS_OCR_MODE=redo`` or ``PAPERLESS_OCR_MODE=force``, - paperless will not modify any text it finds on excluded pages and copy it - verbatim. - - Defaults to 0, which disables this feature and always uses all pages. - -PAPERLESS_OCR_IMAGE_DPI= - Paperless will OCR any images you put into the system and convert them - into PDF documents. This is useful if your scanner produces images. - In order to do so, paperless needs to know the DPI of the image. - Most images from scanners will have this information embedded and - paperless will detect and use that information. In case this fails, it - uses this value as a fallback. - - Set this to the DPI your scanner produces images at. - - Default is none, which will automatically calculate image DPI so that - the produced PDF documents are A4 sized. - -PAPERLESS_OCR_MAX_IMAGE_PIXELS= - Paperless will raise a warning when OCRing images which are over this limit and - will not OCR images which are more than twice this limit. Note this does not - prevent the document from being consumed, but could result in missing text content. - - If unset, will default to the value determined by - `Pillow `_. - - .. note:: - - Increasing this limit could cause Paperless to consume additional resources - when consuming a file. Be sure you have sufficient system resources. - - .. caution:: - - The limit is intended to prevent malicious files from consuming system resources - and causing crashes and other errors. Only increase this value if you are certain - your documents are not malicious and you need the text which was not OCRed - -PAPERLESS_OCR_USER_ARGS= - OCRmyPDF offers many more options. Use this parameter to specify any - additional arguments you wish to pass to OCRmyPDF. Since Paperless uses - the API of OCRmyPDF, you have to specify these in a format that can be - passed to the API. See `the API reference of OCRmyPDF `_ - for valid parameters. All command line options are supported, but they - use underscores instead of dashes. - - .. caution:: - - Paperless has been tested to work with the OCR options provided - above. There are many options that are incompatible with each other, - so specifying invalid options may prevent paperless from consuming - any documents. - - Specify arguments as a JSON dictionary. Keep note of lower case booleans - and double quoted parameter names and strings. Examples: - - .. code:: json - - {"deskew": true, "optimize": 3, "unpaper_args": "--pre-rotate 90"} - -.. _configuration-tika: - -Tika settings -############# - -Paperless can make use of `Tika `_ and -`Gotenberg `_ for parsing and -converting "Office" documents (such as ".doc", ".xlsx" and ".odt"). If you -wish to use this, you must provide a Tika server and a Gotenberg server, -configure their endpoints, and enable the feature. - -PAPERLESS_TIKA_ENABLED= - Enable (or disable) the Tika parser. - - Defaults to false. - -PAPERLESS_TIKA_ENDPOINT= - Set the endpoint URL were Paperless can reach your Tika server. - - Defaults to "http://localhost:9998". - -PAPERLESS_TIKA_GOTENBERG_ENDPOINT= - Set the endpoint URL were Paperless can reach your Gotenberg server. - - Defaults to "http://localhost:3000". - -If you run paperless on docker, you can add those services to the docker-compose -file (see the provided ``docker-compose.sqlite-tika.yml`` file for reference). The changes -requires are as follows: - -.. code:: yaml - - services: - # ... - - webserver: - # ... - - environment: - # ... - - PAPERLESS_TIKA_ENABLED: 1 - PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000 - PAPERLESS_TIKA_ENDPOINT: http://tika:9998 - - # ... - - gotenberg: - image: gotenberg/gotenberg:7.6 - restart: unless-stopped - command: - - "gotenberg" - - "--chromium-disable-routes=true" - - tika: - image: ghcr.io/paperless-ngx/tika:latest - restart: unless-stopped - -Add the configuration variables to the environment of the webserver (alternatively -put the configuration in the ``docker-compose.env`` file) and add the additional -services below the webserver service. Watch out for indentation. - -Make sure to use the correct format `PAPERLESS_TIKA_ENABLED = 1` so python_dotenv can parse the statement correctly. - -Software tweaks -############### - -PAPERLESS_TASK_WORKERS= - Paperless does multiple things in the background: Maintain the search index, - maintain the automatic matching algorithm, check emails, consume documents, - etc. This variable specifies how many things it will do in parallel. - - Defaults to 1 - - -PAPERLESS_THREADS_PER_WORKER= - Furthermore, paperless uses multiple threads when consuming documents to - speed up OCR. This variable specifies how many pages paperless will process - in parallel on a single document. - - .. caution:: - - Ensure that the product - - PAPERLESS_TASK_WORKERS * PAPERLESS_THREADS_PER_WORKER - - does not exceed your CPU core count or else paperless will be extremely slow. - If you want paperless to process many documents in parallel, choose a high - worker count. If you want paperless to process very large documents faster, - use a higher thread per worker count. - - The default is a balance between the two, according to your CPU core count, - with a slight favor towards threads per worker: - - +----------------+---------+---------+ - | CPU core count | Workers | Threads | - +----------------+---------+---------+ - | 1 | 1 | 1 | - +----------------+---------+---------+ - | 2 | 2 | 1 | - +----------------+---------+---------+ - | 4 | 2 | 2 | - +----------------+---------+---------+ - | 6 | 2 | 3 | - +----------------+---------+---------+ - | 8 | 2 | 4 | - +----------------+---------+---------+ - | 12 | 3 | 4 | - +----------------+---------+---------+ - | 16 | 4 | 4 | - +----------------+---------+---------+ - - If you only specify PAPERLESS_TASK_WORKERS, paperless will adjust - PAPERLESS_THREADS_PER_WORKER automatically. - - -PAPERLESS_WORKER_TIMEOUT= - Machines with few cores or weak ones might not be able to finish OCR on - large documents within the default 1800 seconds. So extending this timeout - may prove to be useful on weak hardware setups. - -PAPERLESS_WORKER_RETRY= - If PAPERLESS_WORKER_TIMEOUT has been configured, the retry time for a task can - also be configured. By default, this value will be set to 10s more than the - worker timeout. This value should never be set less than the worker timeout. - -PAPERLESS_TIME_ZONE= - Set the time zone here. - See https://docs.djangoproject.com/en/3.1/ref/settings/#std:setting-TIME_ZONE - for details on how to set it. - - Defaults to UTC. - - -.. _configuration-polling: - -PAPERLESS_CONSUMER_POLLING= - If paperless won't find documents added to your consume folder, it might - not be able to automatically detect filesystem changes. In that case, - specify a polling interval in seconds here, which will then cause paperless - to periodically check your consumption directory for changes. This will also - disable listening for file system changes with ``inotify``. - - Defaults to 0, which disables polling and uses filesystem notifications. - -PAPERLESS_CONSUMER_POLLING_RETRY_COUNT= - If consumer polling is enabled, sets the number of times paperless will check for a - file to remain unmodified. - - Defaults to 5. - -PAPERLESS_CONSUMER_POLLING_DELAY= - If consumer polling is enabled, sets the delay in seconds between each check (above) paperless - will do while waiting for a file to remain unmodified. - - Defaults to 5. - -.. _configuration-inotify: - -PAPERLESS_CONSUMER_INOTIFY_DELAY= - Sets the time in seconds the consumer will wait for additional events - from inotify before the consumer will consider a file ready and begin consumption. - Certain scanners or network setups may generate multiple events for a single file, - leading to multiple consumers working on the same file. Configure this to - prevent that. - - Defaults to 0.5 seconds. - -PAPERLESS_CONSUMER_DELETE_DUPLICATES= - When the consumer detects a duplicate document, it will not touch the - original document. This default behavior can be changed here. - - Defaults to false. - - -PAPERLESS_CONSUMER_RECURSIVE= - Enable recursive watching of the consumption directory. Paperless will - then pickup files from files in subdirectories within your consumption - directory as well. - - Defaults to false. - - -PAPERLESS_CONSUMER_SUBDIRS_AS_TAGS= - Set the names of subdirectories as tags for consumed files. - E.g. /foo/bar/file.pdf will add the tags "foo" and "bar" to - the consumed file. Paperless will create any tags that don't exist yet. - - This is useful for sorting documents with certain tags such as ``car`` or - ``todo`` prior to consumption. These folders won't be deleted. - - PAPERLESS_CONSUMER_RECURSIVE must be enabled for this to work. - - Defaults to false. - -PAPERLESS_CONSUMER_ENABLE_BARCODES= - Enables the scanning and page separation based on detected barcodes. - This allows for scanning and adding multiple documents per uploaded - file, which are separated by one or multiple barcode pages. - - For ease of use, it is suggested to use a standardized separation page, - e.g. `here `_. - - If no barcodes are detected in the uploaded file, no page separation - will happen. - - The original document will be removed and the separated pages will be - saved as pdf. - - Defaults to false. - - -PAPERLESS_CONSUMER_BARCODE_TIFF_SUPPORT= - Whether TIFF image files should be scanned for barcodes. - This will automatically convert any TIFF image(s) to pdfs for later - processing. - This only has an effect, if PAPERLESS_CONSUMER_ENABLE_BARCODES has been - enabled. - - Defaults to false. - -PAPERLESS_CONSUMER_BARCODE_STRING=PATCHT - Defines the string to be detected as a separator barcode. - If paperless is used with the PATCH-T separator pages, users - shouldn't change this. - - Defaults to "PATCHT" - -PAPERLESS_CONVERT_MEMORY_LIMIT= - On smaller systems, or even in the case of Very Large Documents, the consumer - may explode, complaining about how it's "unable to extend pixel cache". In - such cases, try setting this to a reasonably low value, like 32. The - default is to use whatever is necessary to do everything without writing to - disk, and units are in megabytes. - - For more information on how to use this value, you should search - the web for "MAGICK_MEMORY_LIMIT". - - Defaults to 0, which disables the limit. - -PAPERLESS_CONVERT_TMPDIR= - Similar to the memory limit, if you've got a small system and your OS mounts - /tmp as tmpfs, you should set this to a path that's on a physical disk, like - /home/your_user/tmp or something. ImageMagick will use this as scratch space - when crunching through very large documents. - - For more information on how to use this value, you should search - the web for "MAGICK_TMPDIR". - - Default is none, which disables the temporary directory. - -PAPERLESS_POST_CONSUME_SCRIPT= - After a document is consumed, Paperless can trigger an arbitrary script if - you like. This script will be passed a number of arguments for you to work - with. For more information, take a look at :ref:`advanced-post_consume_script`. - - The default is blank, which means nothing will be executed. - -PAPERLESS_FILENAME_DATE_ORDER= - Paperless will check the document text for document date information. - Use this setting to enable checking the document filename for date - information. The date order can be set to any option as specified in - https://dateparser.readthedocs.io/en/latest/settings.html#date-order. - The filename will be checked first, and if nothing is found, the document - text will be checked as normal. - - A date in a filename must have some separators (`.`, `-`, `/`, etc) - for it to be parsed. - - Defaults to none, which disables this feature. - -PAPERLESS_NUMBER_OF_SUGGESTED_DATES= - Paperless searches an entire document for dates. The first date found will - be used as the initial value for the created date. When this variable is - greater than 0 (or left to it's default value), paperless will also suggest - other dates found in the document, up to a maximum of this setting. Note that - duplicates will be removed, which can result in fewer dates displayed in the - frontend than this setting value. - - The task to find all dates can be time-consuming and increases with a higher - (maximum) number of suggested dates and slower hardware. - - Defaults to 3. Set to 0 to disable this feature. - -PAPERLESS_THUMBNAIL_FONT_NAME= - Paperless creates thumbnails for plain text files by rendering the content - of the file on an image and uses a predefined font for that. This - font can be changed here. - - Note that this won't have any effect on already generated thumbnails. - - Defaults to ``/usr/share/fonts/liberation/LiberationSerif-Regular.ttf``. - -PAPERLESS_IGNORE_DATES= - Paperless parses a documents creation date from filename and file content. - You may specify a comma separated list of dates that should be ignored during - this process. This is useful for special dates (like date of birth) that appear - in documents regularly but are very unlikely to be the documents creation date. - - The date is parsed using the order specified in PAPERLESS_DATE_ORDER - - Defaults to an empty string to not ignore any dates. - -PAPERLESS_DATE_ORDER= - Paperless will try to determine the document creation date from its contents. - Specify the date format Paperless should expect to see within your documents. - - This option defaults to DMY which translates to day first, month second, and year - last order. Characters D, M, or Y can be shuffled to meet the required order. - -PAPERLESS_CONSUMER_IGNORE_PATTERNS= - By default, paperless ignores certain files and folders in the consumption - directory, such as system files created by the Mac OS. - - This can be adjusted by configuring a custom json array with patterns to exclude. - - Defaults to ``[".DS_STORE/*", "._*", ".stfolder/*", ".stversions/*", ".localized/*", "desktop.ini"]``. - -Binaries -######## - -There are a few external software packages that Paperless expects to find on -your system when it starts up. Unless you've done something creative with -their installation, you probably won't need to edit any of these. However, -if you've installed these programs somewhere where simply typing the name of -the program doesn't automatically execute it (ie. the program isn't in your -$PATH), then you'll need to specify the literal path for that program. - -PAPERLESS_CONVERT_BINARY= - Defaults to "convert". - -PAPERLESS_GS_BINARY= - Defaults to "gs". - - -.. _configuration-docker: - -Docker-specific options -####################### - -These options don't have any effect in ``paperless.conf``. These options adjust -the behavior of the docker container. Configure these in `docker-compose.env`. - -PAPERLESS_WEBSERVER_WORKERS= - The number of worker processes the webserver should spawn. More worker processes - usually result in the front end to load data much quicker. However, each worker process - also loads the entire application into memory separately, so increasing this value - will increase RAM usage. - - Defaults to 1. - -PAPERLESS_BIND_ADDR= - The IP address the webserver will listen on inside the container. There are - special setups where you may need to configure this value to restrict the - Ip address or interface the webserver listens on. - - Defaults to [::], meaning all interfaces, including IPv6. - -PAPERLESS_PORT= - The port number the webserver will listen on inside the container. There are - special setups where you may need this to avoid collisions with other - services (like using podman with multiple containers in one pod). - - Don't change this when using Docker. To change the port the webserver is - reachable outside of the container, instead refer to the "ports" key in - ``docker-compose.yml``. - - Defaults to 8000. - -USERMAP_UID= - The ID of the paperless user in the container. Set this to your actual user ID on the - host system, which you can get by executing - - .. code:: shell-session - - $ id -u - - Paperless will change ownership on its folders to this user, so you need to get this right - in order to be able to write to the consumption directory. - - Defaults to 1000. - -USERMAP_GID= - The ID of the paperless Group in the container. Set this to your actual group ID on the - host system, which you can get by executing - - .. code:: shell-session - - $ id -g - - Paperless will change ownership on its folders to this group, so you need to get this right - in order to be able to write to the consumption directory. - - Defaults to 1000. - -PAPERLESS_OCR_LANGUAGES= - Additional OCR languages to install. By default, paperless comes with - English, German, Italian, Spanish and French. If your language is not in this list, install - additional languages with this configuration option: - - .. code:: bash - - PAPERLESS_OCR_LANGUAGES=tur ces - - To actually use these languages, also set the default OCR language of paperless: - - .. code:: bash - - PAPERLESS_OCR_LANGUAGE=tur - - Defaults to none, which does not install any additional languages. - -PAPERLESS_ENABLE_FLOWER= - If this environment variable is defined, the Celery monitoring tool - `Flower `_ will - be started by the container. - - You can read more about this in the :ref:`advanced setup ` - documentation. - - -.. _configuration-update-checking: - -Update Checking -############### - -PAPERLESS_ENABLE_UPDATE_CHECK= - - .. note:: - - This setting was deprecated in favor of a frontend setting after v1.9.2. A one-time - migration is performed for users who have this setting set. This setting is always - ignored if the corresponding frontend setting has been set. + You will be redirected shortly... diff --git a/docs/extending.rst b/docs/extending.rst index e8126fd4d..ad303b517 100644 --- a/docs/extending.rst +++ b/docs/extending.rst @@ -1,431 +1,12 @@ .. _extending: +************************* Paperless-ngx Development -######################### +************************* -This section describes the steps you need to take to start development on paperless-ngx. -Check out the source from github. The repository is organized in the following way: +.. cssclass:: redirect-notice -* ``main`` always represents the latest release and will only see changes - when a new release is made. -* ``dev`` contains the code that will be in the next release. -* ``feature-X`` contain bigger changes that will be in some release, but not - necessarily the next one. + The Paperless-ngx documentation has permanently moved. -When making functional changes to paperless, *always* make your changes on the ``dev`` branch. - -Apart from that, the folder structure is as follows: - -* ``docs/`` - Documentation. -* ``src-ui/`` - Code of the front end. -* ``src/`` - Code of the back end. -* ``scripts/`` - Various scripts that help with different parts of development. -* ``docker/`` - Files required to build the docker image. - -Contributing to Paperless -========================= - -Maybe you've been using Paperless for a while and want to add a feature or two, -or maybe you've come across a bug that you have some ideas how to solve. The -beauty of open source software is that you can see what's wrong and help to get -it fixed for everyone! - -Before contributing please review our `code of conduct`_ and other important -information in the `contributing guidelines`_. - -.. _code-formatting-with-pre-commit-hooks: - -Code formatting with pre-commit Hooks -===================================== - -To ensure a consistent style and formatting across the project source, the project -utilizes a Git `pre-commit` hook to perform some formatting and linting before a -commit is allowed. That way, everyone uses the same style and some common issues -can be caught early on. See below for installation instructions. - -Once installed, hooks will run when you commit. If the formatting isn't quite right -or a linter catches something, the commit will be rejected. You'll need to look at the -output and fix the issue. Some hooks, such as the Python formatting tool `black`, -will format failing files, so all you need to do is `git add` those files again and -retry your commit. - -Initial setup and first start -============================= - -After you forked and cloned the code from github you need to perform a first-time setup. -To do the setup you need to perform the steps from the following chapters in a certain order: - -1. Install prerequisites + pipenv as mentioned in :ref:`Bare metal route ` -2. Copy ``paperless.conf.example`` to ``paperless.conf`` and enable debug mode. -3. Install the Angular CLI interface: - - .. code:: shell-session - - $ npm install -g @angular/cli - -4. Install pre-commit - - .. code:: shell-session - - pre-commit install - -5. Create ``consume`` and ``media`` folders in the cloned root folder. - - .. code:: shell-session - - mkdir -p consume media - -6. You can now either ... - - * install redis or - * use the included scripts/start-services.sh to use docker to fire up a redis instance (and some other services such as tika, gotenberg and a database server) or - * spin up a bare redis container - - .. code:: shell-session - - docker run -d -p 6379:6379 --restart unless-stopped redis:latest - -7. Install the python dependencies by performing in the src/ directory. - - .. code:: shell-session - - pipenv install --dev - - * Make sure you're using python 3.9.x or lower. Otherwise you might get issues with building dependencies. You can use `pyenv `_ to install a specific python version. - -8. Generate the static UI so you can perform a login to get session that is required for frontend development (this needs to be done one time only). From src-ui directory: - - .. code:: shell-session - - npm install . - ./node_modules/.bin/ng build --configuration production - -9. Apply migrations and create a superuser for your dev instance: - - .. code:: shell-session - - python3 manage.py migrate - python3 manage.py createsuperuser - -10. Now spin up the dev backend. Depending on which part of paperless you're developing for, you need to have some or all of them running. - - .. code:: shell-session - - python3 manage.py runserver & python3 manage.py document_consumer & celery --app paperless worker - -11. Login with the superuser credentials provided in step 8 at ``http://localhost:8000`` to create a session that enables you to use the backend. - -Backend development environment is now ready, to start Frontend development go to ``/src-ui`` and run ``ng serve``. From there you can use ``http://localhost:4200`` for a preview. - -Back end development -==================== - -The backend is a django application. PyCharm works well for development, but you can use whatever -you want. - -Configure the IDE to use the src/ folder as the base source folder. Configure the following -launch configurations in your IDE: - -* python3 manage.py runserver -* celery --app paperless worker -* python3 manage.py document_consumer - -To start them all: - -.. code:: shell-session - - python3 manage.py runserver & python3 manage.py document_consumer & celery --app paperless worker - -Testing and code style: - -* Run ``pytest`` in the src/ directory to execute all tests. This also generates a HTML coverage - report. When runnings test, paperless.conf is loaded as well. However: the tests rely on the default - configuration. This is not ideal. But for now, make sure no settings except for DEBUG are overridden when testing. -* Coding style is enforced by the Git pre-commit hooks. These will ensure your code is formatted and do some - linting when you do a `git commit`. -* You can also run ``black`` manually to format your code - - .. note:: - - The line length rule E501 is generally useful for getting multiple source files - next to each other on the screen. However, in some cases, its just not possible - to make some lines fit, especially complicated IF cases. Append ``# NOQA: E501`` - to disable this check for certain lines. - -Front end development -===================== - -The front end is built using Angular. In order to get started, you need ``npm``. -Install the Angular CLI interface with - -.. code:: shell-session - - $ npm install -g @angular/cli - -and make sure that it's on your path. Next, in the src-ui/ directory, install the -required dependencies of the project. - -.. code:: shell-session - - $ npm install - -You can launch a development server by running - -.. code:: shell-session - - $ ng serve - -This will automatically update whenever you save. However, in-place compilation might fail -on syntax errors, in which case you need to restart it. - -By default, the development server is available on ``http://localhost:4200/`` and is configured -to access the API at ``http://localhost:8000/api/``, which is the default of the backend. -If you enabled DEBUG on the back end, several security overrides for allowed hosts, CORS and -X-Frame-Options are in place so that the front end behaves exactly as in production. This also -relies on you being logged into the back end. Without a valid session, The front end will simply -not work. - -Testing and code style: - -* The frontend code (.ts, .html, .scss) use ``prettier`` for code formatting via the Git - ``pre-commit`` hooks which run automatically on commit. See - :ref:`above ` for installation. You can also run this - via cli with a command such as - - .. code:: shell-session - - $ git ls-files -- '*.ts' | xargs pre-commit run prettier --files - -* Frontend testing uses jest and cypress. There is currently a need for significantly more - frontend tests. Unit tests and e2e tests, respectively, can be run non-interactively with: - - .. code:: shell-session - - $ ng test - $ npm run e2e:ci - - Cypress also includes a UI which can be run from within the ``src-ui`` directory with - - .. code:: shell-session - - $ ./node_modules/.bin/cypress open - -In order to build the front end and serve it as part of django, execute - -.. code:: shell-session - - $ ng build --prod - -This will build the front end and put it in a location from which the Django server will serve -it as static content. This way, you can verify that authentication is working. - - -Localization -============ - -Paperless is available in many different languages. Since paperless consists both of a django -application and an Angular front end, both these parts have to be translated separately. - -Front end localization ----------------------- - -* The Angular front end does localization according to the `Angular documentation `_. -* The source language of the project is "en_US". -* The source strings end up in the file "src-ui/messages.xlf". -* The translated strings need to be placed in the "src-ui/src/locale/" folder. -* In order to extract added or changed strings from the source files, call ``ng xi18n --ivy``. - -Adding new languages requires adding the translated files in the "src-ui/src/locale/" folder and adjusting a couple files. - -1. Adjust "src-ui/angular.json": - - .. code:: json - - "i18n": { - "sourceLocale": "en-US", - "locales": { - "de": "src/locale/messages.de.xlf", - "nl-NL": "src/locale/messages.nl_NL.xlf", - "fr": "src/locale/messages.fr.xlf", - "en-GB": "src/locale/messages.en_GB.xlf", - "pt-BR": "src/locale/messages.pt_BR.xlf", - "language-code": "language-file" - } - } - -2. Add the language to the available options in "src-ui/src/app/services/settings.service.ts": - - .. code:: typescript - - getLanguageOptions(): LanguageOption[] { - return [ - {code: "en-us", name: $localize`English (US)`, englishName: "English (US)", dateInputFormat: "mm/dd/yyyy"}, - {code: "en-gb", name: $localize`English (GB)`, englishName: "English (GB)", dateInputFormat: "dd/mm/yyyy"}, - {code: "de", name: $localize`German`, englishName: "German", dateInputFormat: "dd.mm.yyyy"}, - {code: "nl", name: $localize`Dutch`, englishName: "Dutch", dateInputFormat: "dd-mm-yyyy"}, - {code: "fr", name: $localize`French`, englishName: "French", dateInputFormat: "dd/mm/yyyy"}, - {code: "pt-br", name: $localize`Portuguese (Brazil)`, englishName: "Portuguese (Brazil)", dateInputFormat: "dd/mm/yyyy"} - // Add your new language here - ] - } - - ``dateInputFormat`` is a special string that defines the behavior of the date input fields and absolutely needs to contain "dd", "mm" and "yyyy". - -3. Import and register the Angular data for this locale in "src-ui/src/app/app.module.ts": - - .. code:: typescript - - import localeDe from '@angular/common/locales/de'; - registerLocaleData(localeDe) - -Back end localization ---------------------- - -A majority of the strings that appear in the back end appear only when the admin is used. However, -some of these are still shown on the front end (such as error messages). - -* The django application does localization according to the `django documentation `_. -* The source language of the project is "en_US". -* Localization files end up in the folder "src/locale/". -* In order to extract strings from the application, call ``python3 manage.py makemessages -l en_US``. This is important after making changes to translatable strings. -* The message files need to be compiled for them to show up in the application. Call ``python3 manage.py compilemessages`` to do this. The generated files don't get - committed into git, since these are derived artifacts. The build pipeline takes care of executing this command. - -Adding new languages requires adding the translated files in the "src/locale/" folder and adjusting the file "src/paperless/settings.py" to include the new language: - -.. code:: python - - LANGUAGES = [ - ("en-us", _("English (US)")), - ("en-gb", _("English (GB)")), - ("de", _("German")), - ("nl-nl", _("Dutch")), - ("fr", _("French")), - ("pt-br", _("Portuguese (Brazil)")), - # Add language here. - ] - - -Building the documentation -========================== - -The documentation is built using sphinx. I've configured ReadTheDocs to automatically build -the documentation when changes are pushed. If you want to build the documentation locally, -this is how you do it: - -1. Install python dependencies. - - .. code:: shell-session - - $ cd /path/to/paperless - $ pipenv install --dev - -2. Build the documentation - - .. code:: shell-session - - $ cd /path/to/paperless/docs - $ pipenv run make clean html - -This will build the HTML documentation, and put the resulting files in the ``_build/html`` -directory. - -Building the Docker image -========================= - -The docker image is primarily built by the GitHub actions workflow, but it can be -faster when developing to build and tag an image locally. - -To provide the build arguments automatically, build the image using the helper -script ``build-docker-image.sh``. - -Building the docker image from source: - - .. code:: shell-session - - ./build-docker-image.sh Dockerfile -t - -Extending Paperless -=================== - -Paperless does not have any fancy plugin systems and will probably never have. However, -some parts of the application have been designed to allow easy integration of additional -features without any modification to the base code. - -Making custom parsers ---------------------- - -Paperless uses parsers to add documents to paperless. A parser is responsible for: - -* Retrieve the content from the original -* Create a thumbnail -* Optional: Retrieve a created date from the original -* Optional: Create an archived document from the original - -Custom parsers can be added to paperless to support more file types. In order to do that, -you need to write the parser itself and announce its existence to paperless. - -The parser itself must extend ``documents.parsers.DocumentParser`` and must implement the -methods ``parse`` and ``get_thumbnail``. You can provide your own implementation to -``get_date`` if you don't want to rely on paperless' default date guessing mechanisms. - -.. code:: python - - class MyCustomParser(DocumentParser): - - def parse(self, document_path, mime_type): - # This method does not return anything. Rather, you should assign - # whatever you got from the document to the following fields: - - # The content of the document. - self.text = "content" - - # Optional: path to a PDF document that you created from the original. - self.archive_path = os.path.join(self.tempdir, "archived.pdf") - - # Optional: "created" date of the document. - self.date = get_created_from_metadata(document_path) - - def get_thumbnail(self, document_path, mime_type): - # This should return the path to a thumbnail you created for this - # document. - return os.path.join(self.tempdir, "thumb.png") - -If you encounter any issues during parsing, raise a ``documents.parsers.ParseError``. - -The ``self.tempdir`` directory is a temporary directory that is guaranteed to be empty -and removed after consumption finished. You can use that directory to store any -intermediate files and also use it to store the thumbnail / archived document. - -After that, you need to announce your parser to paperless. You need to connect a -handler to the ``document_consumer_declaration`` signal. Have a look in the file -``src/paperless_tesseract/apps.py`` on how that's done. The handler is a method -that returns information about your parser: - -.. code:: python - - def myparser_consumer_declaration(sender, **kwargs): - return { - "parser": MyCustomParser, - "weight": 0, - "mime_types": { - "application/pdf": ".pdf", - "image/jpeg": ".jpg", - } - } - -* ``parser`` is a reference to a class that extends ``DocumentParser``. - -* ``weight`` is used whenever two or more parsers are able to parse a file: The parser with - the higher weight wins. This can be used to override the parsers provided by - paperless. - -* ``mime_types`` is a dictionary. The keys are the mime types your parser supports and the value - is the default file extension that paperless should use when storing files and serving them for - download. We could guess that from the file extensions, but some mime types have many extensions - associated with them and the python methods responsible for guessing the extension do not always - return the same value. - -.. _code of conduct: https://github.com/paperless-ngx/paperless-ngx/blob/main/CODE_OF_CONDUCT.md -.. _contributing guidelines: https://github.com/paperless-ngx/paperless-ngx/blob/main/CONTRIBUTING.md + You will be redirected shortly... diff --git a/docs/faq.rst b/docs/faq.rst index 1b2892fc3..868961417 100644 --- a/docs/faq.rst +++ b/docs/faq.rst @@ -1,117 +1,12 @@ +.. _faq: ************************** Frequently asked questions ************************** -**Q:** *What's the general plan for Paperless-ngx?* -**A:** While Paperless-ngx is already considered largely "feature-complete" it is a community-driven -project and development will be guided in this way. New features can be submitted via -GitHub discussions and "up-voted" by the community but this is not a guarantee the feature -will be implemented. This project will always be open to collaboration in the form of PRs, -ideas etc. +.. cssclass:: redirect-notice -**Q:** *I'm using docker. Where are my documents?* + The Paperless-ngx documentation has permanently moved. -**A:** Your documents are stored inside the docker volume ``paperless_media``. -Docker manages this volume automatically for you. It is a persistent storage -and will persist as long as you don't explicitly delete it. The actual location -depends on your host operating system. On Linux, chances are high that this location -is - -.. code:: - - /var/lib/docker/volumes/paperless_media/_data - -.. caution:: - - Do not mess with this folder. Don't change permissions and don't move - files around manually. This folder is meant to be entirely managed by docker - and paperless. - -**Q:** *Let's say I want to switch tools in a year. Can I easily move to other systems?* - -**A:** Your documents are stored as plain files inside the media folder. You can always drag those files -out of that folder to use them elsewhere. Here are a couple notes about that. - -* Paperless-ngx never modifies your original documents. It keeps checksums of all documents and uses a - scheduled sanity checker to check that they remain the same. -* By default, paperless uses the internal ID of each document as its filename. This might not be very - convenient for export. However, you can adjust the way files are stored in paperless by - :ref:`configuring the filename format `. -* :ref:`The exporter ` is another easy way to get your files out of paperless with reasonable file names. - -**Q:** *What file types does paperless-ngx support?* - -**A:** Currently, the following files are supported: - -* PDF documents, PNG images, JPEG images, TIFF images and GIF images are processed with OCR and converted into PDF documents. -* Plain text documents are supported as well and are added verbatim - to paperless. -* With the optional Tika integration enabled (see :ref:`Configuration `), Paperless also supports various - Office documents (.docx, .doc, odt, .ppt, .pptx, .odp, .xls, .xlsx, .ods). - -Paperless-ngx determines the type of a file by inspecting its content. The -file extensions do not matter. - -**Q:** *Will paperless-ngx run on Raspberry Pi?* - -**A:** The short answer is yes. I've tested it on a Raspberry Pi 3 B. -The long answer is that certain parts of -Paperless will run very slow, such as the OCR. On Raspberry Pi, -try to OCR documents before feeding them into paperless so that paperless can -reuse the text. The web interface is a lot snappier, since it runs -in your browser and paperless has to do much less work to serve the data. - -.. note:: - - You can adjust some of the settings so that paperless uses less processing - power. See :ref:`setup-less_powerful_devices` for details. - - -**Q:** *How do I install paperless-ngx on Raspberry Pi?* - -**A:** Docker images are available for arm and arm64 hardware, so just follow -the docker-compose instructions. Apart from more required disk space compared to -a bare metal installation, docker comes with close to zero overhead, even on -Raspberry Pi. - -If you decide to got with the bare metal route, be aware that some of the -python requirements do not have precompiled packages for ARM / ARM64. Installation -of these will require additional development libraries and compilation will take -a long time. - -**Q:** *How do I run this on Unraid?* - -**A:** Paperless-ngx is available as `community app `_ -in Unraid. `Uli Fahrer `_ created a container template for that. - -**Q:** *How do I run this on my toaster?* - -**A:** I honestly don't know! As for all other devices that might be able -to run paperless, you're a bit on your own. If you can't run the docker image, -the documentation has instructions for bare metal installs. I'm running -paperless on an i3 processor from 2015 or so. This is also what I use to test -new releases with. Apart from that, I also have a Raspberry Pi, which I -occasionally build the image on and see if it works. - -**Q:** *How do I proxy this with NGINX?* - -**A:** See :ref:`here `. - -.. _faq-mod_wsgi: - -**Q:** *How do I get WebSocket support with Apache mod_wsgi*? - -**A:** ``mod_wsgi`` by itself does not support ASGI. Paperless will continue -to work with WSGI, but certain features such as status notifications about -document consumption won't be available. - -If you want to continue using ``mod_wsgi``, you will have to run an ASGI-enabled -web server as well that processes WebSocket connections, and configure Apache to -redirect WebSocket connections to this server. Multiple options for ASGI servers -exist: - -* ``gunicorn`` with ``uvicorn`` as the worker implementation (the default of paperless) -* ``daphne`` as a standalone server, which is the reference implementation for ASGI. -* ``uvicorn`` as a standalone server + You will be redirected shortly... diff --git a/docs/index.rst b/docs/index.rst index 735804560..a7d78cd0f 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,74 +2,24 @@ Paperless ********* -Paperless is a simple Django application running in two parts: -a *Consumer* (the thing that does the indexing) and -the *Web server* (the part that lets you search & -download already-indexed documents). If you want to learn more about its -functions keep on reading after the installation section. +.. cssclass:: redirect-notice -Why This Exists -=============== + The Paperless-ngx documentation has permanently moved. -Paper is a nightmare. Environmental issues aside, there's no excuse for it in -the 21st century. It takes up space, collects dust, doesn't support any form -of a search feature, indexing is tedious, it's heavy and prone to damage & -loss. - -I wrote this to make "going paperless" easier. I do not have to worry about -finding stuff again. I feed documents right from the post box into the scanner -and then shred them. Perhaps you might find it useful too. - - -Paperless-ngx -============= - -Paperless-ngx is a document management system that transforms your physical -documents into a searchable online archive so you can keep, well, *less paper*. - -Paperless-ngx forked from paperless-ng to continue the great work and -distribute responsibility of supporting and advancing the project among a team -of people. - -NG stands for both Angular (the framework used for the -Frontend) and next-gen. Publishing this project under a different name also -avoids confusion between paperless and paperless-ngx. - -If you want to learn about what's different in paperless-ngx from Paperless, check out these -resources in the documentation: - -* :ref:`Some screenshots ` of the new UI are available. -* Read :ref:`this section ` if you want to - learn about how paperless automates all tagging using machine learning. -* Paperless now comes with a :ref:`proper email consumer ` - that's fully tested and production ready. -* Paperless creates searchable PDF/A documents from whatever you put into - the consumption directory. This means that you can select text in - image-only documents coming from your scanner. -* See :ref:`this note ` about GnuPG encryption in - paperless-ngx. -* Paperless is now integrated with a - :ref:`task processing queue ` that tells you - at a glance when and why something is not working. -* The :doc:`changelog ` contains a detailed list of all changes - in paperless-ngx. - -Contents -======== + You will be redirected shortly... .. toctree:: - :maxdepth: 1 - setup - usage_overview - advanced_usage - administration - configuration - api - faq - troubleshooting - extending - scanners - screenshots - changelog + screenshots + scanners + administration + advanced_usage + usage_overview + setup + troubleshooting + changelog + configuration + extending + api + faq diff --git a/docs/scanners.rst b/docs/scanners.rst index 68b0b335f..f975c47a9 100644 --- a/docs/scanners.rst +++ b/docs/scanners.rst @@ -1,8 +1,12 @@ - .. _scanners: ******************* Scanners & Software ******************* -Paperless-ngx is compatible with many different scanners and scanning tools. A user-maintained list of scanners and other software is available on `the wiki `_. + +.. cssclass:: redirect-notice + + The Paperless-ngx documentation has permanently moved. + + You will be redirected shortly... diff --git a/docs/screenshots.rst b/docs/screenshots.rst index 1815575f1..18b40d1ee 100644 --- a/docs/screenshots.rst +++ b/docs/screenshots.rst @@ -4,60 +4,9 @@ Screenshots *********** -This is what Paperless-ngx looks like. -The dashboard shows customizable views on your document and allows document uploads: +.. cssclass:: redirect-notice -.. image:: _static/screenshots/dashboard.png - :target: _static/screenshots/dashboard.png + The Paperless-ngx documentation has permanently moved. -The document list provides three different styles to scroll through your documents: - -.. image:: _static/screenshots/documents-table.png - :target: _static/screenshots/documents-table.png -.. image:: _static/screenshots/documents-smallcards.png - :target: _static/screenshots/documents-smallcards.png -.. image:: _static/screenshots/documents-largecards.png - :target: _static/screenshots/documents-largecards.png - -Paperless-ngx also supports "dark mode": - -.. image:: _static/screenshots/documents-smallcards-dark.png - :target: _static/screenshots/documents-smallcards-dark.png - -Extensive filtering mechanisms: - -.. image:: _static/screenshots/documents-filter.png - :target: _static/screenshots/documents-filter.png - -Bulk editing of document tags, correspondents, etc.: - -.. image:: _static/screenshots/bulk-edit.png - :target: _static/screenshots/bulk-edit.png - -Side-by-side editing of documents: - -.. image:: _static/screenshots/editing.png - :target: _static/screenshots/editing.png - -Tag editing. This looks about the same for correspondents and document types. - -.. image:: _static/screenshots/new-tag.png - :target: _static/screenshots/new-tag.png - -Searching provides auto complete and highlights the results. - -.. image:: _static/screenshots/search-preview.png - :target: _static/screenshots/search-preview.png -.. image:: _static/screenshots/search-results.png - :target: _static/screenshots/search-results.png - -Fancy mail filters! - -.. image:: _static/screenshots/mail-rules-edited.png - :target: _static/screenshots/mail-rules-edited.png - -Mobile devices are supported. - -.. image:: _static/screenshots/mobile.png - :target: _static/screenshots/mobile.png + You will be redirected shortly... diff --git a/docs/setup.rst b/docs/setup.rst index db96c6b58..e0540eed8 100644 --- a/docs/setup.rst +++ b/docs/setup.rst @@ -1,894 +1,12 @@ +.. _setup: ***** Setup ***** -Overview of Paperless-ngx -######################### -Compared to paperless, paperless-ngx works a little different under the hood and has -more moving parts that work together. While this increases the complexity of -the system, it also brings many benefits. +.. cssclass:: redirect-notice -Paperless consists of the following components: + The Paperless-ngx documentation has permanently moved. -* **The webserver:** This is pretty much the same as in paperless. It serves - the administration pages, the API, and the new frontend. This is the main - tool you'll be using to interact with paperless. You may start the webserver - with - - .. code:: shell-session - - $ cd /path/to/paperless/src/ - $ gunicorn -c ../gunicorn.conf.py paperless.wsgi - - or by any other means such as Apache ``mod_wsgi``. - -* **The consumer:** This is what watches your consumption folder for documents. - However, the consumer itself does not really consume your documents. - Now it notifies a task processor that a new file is ready for consumption. - I suppose it should be named differently. - This was also used to check your emails, but that's now done elsewhere as well. - - Start the consumer with the management command ``document_consumer``: - - .. code:: shell-session - - $ cd /path/to/paperless/src/ - $ python3 manage.py document_consumer - - .. _setup-task_processor: - -* **The task processor:** Paperless relies on `Celery - Distributed Task Queue `_ - for doing most of the heavy lifting. This is a task queue that accepts tasks from - multiple sources and processes these in parallel. It also comes with a scheduler that executes - certain commands periodically. - - This task processor is responsible for: - - * Consuming documents. When the consumer finds new documents, it notifies the task processor to - start a consumption task. - * The task processor also performs the consumption of any documents you upload through - the web interface. - * Consuming emails. It periodically checks your configured accounts for new emails and - notifies the task processor to consume the attachment of an email. - * Maintaining the search index and the automatic matching algorithm. These are things that paperless - needs to do from time to time in order to operate properly. - - This allows paperless to process multiple documents from your consumption folder in parallel! On - a modern multi core system, this makes the consumption process with full OCR blazingly fast. - - The task processor comes with a built-in admin interface that you can use to check whenever any of the - tasks fail and inspect the errors (i.e., wrong email credentials, errors during consuming a specific - file, etc). - -* A `redis `_ message broker: This is a really lightweight service that is responsible - for getting the tasks from the webserver and the consumer to the task scheduler. These run in a different - process (maybe even on different machines!), and therefore, this is necessary. - -* Optional: A database server. Paperless supports PostgreSQL, MariaDB and SQLite for storing its data. - - -Installation -############ - -You can go multiple routes to setup and run Paperless: - -* :ref:`Use the easy install docker script ` -* :ref:`Pull the image from Docker Hub ` -* :ref:`Build the Docker image yourself ` -* :ref:`Install Paperless directly on your system manually (bare metal) ` - -The Docker routes are quick & easy. These are the recommended routes. This configures all the stuff -from the above automatically so that it just works and uses sensible defaults for all configuration options. -Here you find a cheat-sheet for docker beginners: `CLI Basics `_ - -The bare metal route is complicated to setup but makes it easier -should you want to contribute some code back. You need to configure and -run the above mentioned components yourself. - -.. _CLI Basics: https://www.sehn.tech/refs/devops-with-docker/ - -.. _setup-docker_script: - -Install Paperless from Docker Hub using the installation script -=============================================================== - -Paperless provides an interactive installation script. This script will ask you -for a couple configuration options, download and create the necessary configuration files, pull the docker image, start paperless and create your user account. This script essentially -performs all the steps described in :ref:`setup-docker_hub` automatically. - -1. Make sure that docker and docker-compose are installed. -2. Download and run the installation script: - - .. code:: shell-session - - $ bash -c "$(curl -L https://raw.githubusercontent.com/paperless-ngx/paperless-ngx/main/install-paperless-ngx.sh)" - -.. _setup-docker_hub: - -Install Paperless from Docker Hub -================================= - -1. Login with your user and create a folder in your home-directory `mkdir -v ~/paperless-ngx` to have a place for your configuration files and consumption directory. - -2. Go to the `/docker/compose directory on the project page `_ - and download one of the `docker-compose.*.yml` files, depending on which database backend you - want to use. Rename this file to `docker-compose.yml`. - If you want to enable optional support for Office documents, download a file with `-tika` in the file name. - Download the ``docker-compose.env`` file and the ``.env`` file as well and store them - in the same directory. - - .. hint:: - - For new installations, it is recommended to use PostgreSQL as the database - backend. - -3. Install `Docker`_ and `docker-compose`_. - - .. caution:: - - If you want to use the included ``docker-compose.*.yml`` file, you - need to have at least Docker version **17.09.0** and docker-compose - version **1.17.0**. - To check do: `docker-compose -v` or `docker -v` - - See the `Docker installation guide`_ on how to install the current - version of Docker for your operating system or Linux distribution of - choice. To get the latest version of docker-compose, follow the - `docker-compose installation guide`_ if your package repository doesn't - include it. - - .. _Docker installation guide: https://docs.docker.com/engine/installation/ - .. _docker-compose installation guide: https://docs.docker.com/compose/install/ - -4. Modify ``docker-compose.yml`` to your preferences. You may want to change the path - to the consumption directory. Find the line that specifies where - to mount the consumption directory: - - .. code:: - - - ./consume:/usr/src/paperless/consume - - Replace the part BEFORE the colon with a local directory of your choice: - - .. code:: - - - /home/jonaswinkler/paperless-inbox:/usr/src/paperless/consume - - Don't change the part after the colon or paperless wont find your documents. - - You may also need to change the default port that the webserver will use - from the default (8000): - - .. code:: - - ports: - - 8000:8000 - - Replace the part BEFORE the colon with a port of your choice: - - .. code:: - - ports: - - 8010:8000 - - Don't change the part after the colon or edit other lines that refer to - port 8000. Modifying the part before the colon will map requests on another - port to the webserver running on the default port. - - **Rootless** - - If you want to run Paperless as a rootless container, you will need to do the - following in your ``docker-compose.yml``: - - - set the ``user`` running the container to map to the ``paperless`` user in the - container. - This value (``user_id`` below), should be the same id that ``USERMAP_UID`` and - ``USERMAP_GID`` are set to in the next step. - See ``USERMAP_UID`` and ``USERMAP_GID`` :ref:`here `. - - Your entry for Paperless should contain something like: - - .. code:: - - webserver: - image: ghcr.io/paperless-ngx/paperless-ngx:latest - user: - -5. Modify ``docker-compose.env``, following the comments in the file. The - most important change is to set ``USERMAP_UID`` and ``USERMAP_GID`` - to the uid and gid of your user on the host system. Use ``id -u`` and - ``id -g`` to get these. - - This ensures that - both the docker container and you on the host machine have write access - to the consumption directory. If your UID and GID on the host system is - 1000 (the default for the first normal user on most systems), it will - work out of the box without any modifications. `id "username"` to check. - - .. note:: - - You can copy any setting from the file ``paperless.conf.example`` and paste it here. - Have a look at :ref:`configuration` to see what's available. - - .. note:: - - You can utilize Docker secrets for some configuration settings by - appending `_FILE` to some configuration values. This is supported currently - only by: - - * PAPERLESS_DBUSER - * PAPERLESS_DBPASS - * PAPERLESS_SECRET_KEY - * PAPERLESS_AUTO_LOGIN_USERNAME - * PAPERLESS_ADMIN_USER - * PAPERLESS_ADMIN_MAIL - * PAPERLESS_ADMIN_PASSWORD - - .. caution:: - - Some file systems such as NFS network shares don't support file system - notifications with ``inotify``. When storing the consumption directory - on such a file system, paperless will not pick up new files - with the default configuration. You will need to use ``PAPERLESS_CONSUMER_POLLING``, - which will disable inotify. See :ref:`here `. - -6. Run ``docker-compose pull``, followed by ``docker-compose up -d``. - This will pull the image, create and start the necessary containers. - -7. To be able to login, you will need a super user. To create it, execute the - following command: - - .. code-block:: shell-session - - $ docker-compose run --rm webserver createsuperuser - - This will prompt you to set a username, an optional e-mail address and - finally a password (at least 8 characters). - -8. The default ``docker-compose.yml`` exports the webserver on your local port - 8000. If you did not change this, you should now be able to visit your - Paperless instance at ``http://127.0.0.1:8000`` or your servers IP-Address:8000. - Use the login credentials you have created with the previous step. - -.. _Docker: https://www.docker.com/ -.. _docker-compose: https://docs.docker.com/compose/install/ - -.. _setup-docker_build: - -Build the Docker image yourself -=============================== - -1. Clone the entire repository of paperless: - - .. code:: shell-session - - git clone https://github.com/paperless-ngx/paperless-ngx - - The master branch always reflects the latest stable version. - -2. Copy one of the ``docker/compose/docker-compose.*.yml`` to ``docker-compose.yml`` in the root folder, - depending on which database backend you want to use. Copy - ``docker-compose.env`` into the project root as well. - -3. In the ``docker-compose.yml`` file, find the line that instructs docker-compose to pull the paperless image from Docker Hub: - - .. code:: yaml - - webserver: - image: ghcr.io/paperless-ngx/paperless-ngx:latest - - and replace it with a line that instructs docker-compose to build the image from the current working directory instead: - - .. code:: yaml - - webserver: - build: - context: . - args: - QPDF_VERSION: x.y.x - PIKEPDF_VERSION: x.y.z - PSYCOPG2_VERSION: x.y.z - JBIG2ENC_VERSION: 0.29 - - .. note:: - - You should match the build argument versions to the version for the release you have - checked out. These are pre-built images with certain, more updated software. - If you want to build these images your self, that is possible, but beyond - the scope of these steps. - -4. Follow steps 3 to 8 of :ref:`setup-docker_hub`. When asked to run - ``docker-compose pull`` to pull the image, do - - .. code:: shell-session - - $ docker-compose build - - instead to build the image. - -.. _setup-bare_metal: - -Bare Metal Route -================ - -Paperless runs on linux only. The following procedure has been tested on a minimal -installation of Debian/Buster, which is the current stable release at the time of -writing. Windows is not and will never be supported. - -1. Install dependencies. Paperless requires the following packages. - - * ``python3`` 3.8, 3.9 - * ``python3-pip`` - * ``python3-dev`` - - * ``default-libmysqlclient-dev`` for MariaDB - * ``fonts-liberation`` for generating thumbnails for plain text files - * ``imagemagick`` >= 6 for PDF conversion - * ``gnupg`` for handling encrypted documents - * ``libpq-dev`` for PostgreSQL - * ``libmagic-dev`` for mime type detection - * ``mariadb-client`` for MariaDB compile time - * ``mime-support`` for mime type detection - * ``libzbar0`` for barcode detection - * ``poppler-utils`` for barcode detection - - Use this list for your preferred package management: - - .. code:: - - python3 python3-pip python3-dev imagemagick fonts-liberation gnupg libpq-dev default-libmysqlclient-dev libmagic-dev mime-support libzbar0 poppler-utils - - These dependencies are required for OCRmyPDF, which is used for text recognition. - - * ``unpaper`` - * ``ghostscript`` - * ``icc-profiles-free`` - * ``qpdf`` - * ``liblept5`` - * ``libxml2`` - * ``pngquant`` (suggested for certain PDF image optimizations) - * ``zlib1g`` - * ``tesseract-ocr`` >= 4.0.0 for OCR - * ``tesseract-ocr`` language packs (``tesseract-ocr-eng``, ``tesseract-ocr-deu``, etc) - - Use this list for your preferred package management: - - .. code:: - - unpaper ghostscript icc-profiles-free qpdf liblept5 libxml2 pngquant zlib1g tesseract-ocr - - On Raspberry Pi, these libraries are required as well: - - * ``libatlas-base-dev`` - * ``libxslt1-dev`` - - You will also need ``build-essential``, ``python3-setuptools`` and ``python3-wheel`` - for installing some of the python dependencies. - -2. Install ``redis`` >= 6.0 and configure it to start automatically. - -3. Optional. Install ``postgresql`` and configure a database, user and password for paperless. If you do not wish - to use PostgreSQL, MariaDB and SQLite are available as well. - - .. note:: - - On bare-metal installations using SQLite, ensure the - `JSON1 extension `_ is enabled. This is - usually the case, but not always. - -4. Get the release archive from ``_. - If you clone the git repo as it is, you also have to compile the front end by yourself. - Extract the archive to a place from where you wish to execute it, such as ``/opt/paperless``. - -5. Configure paperless. See :ref:`configuration` for details. Edit the included ``paperless.conf`` and adjust the - settings to your needs. Required settings for getting paperless running are: - - * ``PAPERLESS_REDIS`` should point to your redis server, such as redis://localhost:6379. - * ``PAPERLESS_DBENGINE`` optional, and should be one of `postgres, mariadb, or sqlite` - * ``PAPERLESS_DBHOST`` should be the hostname on which your PostgreSQL server is running. Do not configure this - to use SQLite instead. Also configure port, database name, user and password as necessary. - * ``PAPERLESS_CONSUMPTION_DIR`` should point to a folder which paperless should watch for documents. You might - want to have this somewhere else. Likewise, ``PAPERLESS_DATA_DIR`` and ``PAPERLESS_MEDIA_ROOT`` define where - paperless stores its data. If you like, you can point both to the same directory. - * ``PAPERLESS_SECRET_KEY`` should be a random sequence of characters. It's used for authentication. Failure - to do so allows third parties to forge authentication credentials. - * ``PAPERLESS_URL`` if you are behind a reverse proxy. This should point to your domain. Please see - :ref:`configuration` for more information. - - Many more adjustments can be made to paperless, especially the OCR part. The following options are recommended - for everyone: - - * Set ``PAPERLESS_OCR_LANGUAGE`` to the language most of your documents are written in. - * Set ``PAPERLESS_TIME_ZONE`` to your local time zone. - -6. Create a system user under which you wish to run paperless. - - .. code:: shell-session - - adduser paperless --system --home /opt/paperless --group - -7. Ensure that these directories exist - and that the paperless user has write permissions to the following directories: - - * ``/opt/paperless/media`` - * ``/opt/paperless/data`` - * ``/opt/paperless/consume`` - - Adjust as necessary if you configured different folders. - -8. Install python requirements from the ``requirements.txt`` file. - It is up to you if you wish to use a virtual environment or not. First you should update your pip, so it gets the actual packages. - - .. code:: shell-session - - sudo -Hu paperless pip3 install --upgrade pip - - .. code:: shell-session - - sudo -Hu paperless pip3 install -r requirements.txt - - This will install all python dependencies in the home directory of - the new paperless user. - -9. Go to ``/opt/paperless/src``, and execute the following commands: - - .. code:: bash - - # This creates the database schema. - sudo -Hu paperless python3 manage.py migrate - - # This creates your first paperless user - sudo -Hu paperless python3 manage.py createsuperuser - -10. Optional: Test that paperless is working by executing - - .. code:: bash - - # This collects static files from paperless and django. - sudo -Hu paperless python3 manage.py runserver - - and pointing your browser to http://localhost:8000/. - - .. warning:: - - This is a development server which should not be used in - production. It is not audited for security and performance - is inferior to production ready web servers. - - .. hint:: - - This will not start the consumer. Paperless does this in a - separate process. - -11. Setup systemd services to run paperless automatically. You may - use the service definition files included in the ``scripts`` folder - as a starting point. - - Paperless needs the ``webserver`` script to run the webserver, the - ``consumer`` script to watch the input folder, ``taskqueue`` for the background workers - used to handle things like document consumption and the ``scheduler`` script to run tasks such as - email checking at certain times . - - The ``socket`` script enables ``gunicorn`` to run on port 80 without - root privileges. For this you need to uncomment the ``Require=paperless-webserver.socket`` - in the ``webserver`` script and configure ``gunicorn`` to listen on port 80 (see ``paperless/gunicorn.conf.py``). - - You may need to adjust the path to the ``gunicorn`` executable. This - will be installed as part of the python dependencies, and is either located - in the ``bin`` folder of your virtual environment, or in ``~/.local/bin/`` if - no virtual environment is used. - - These services rely on redis and optionally the database server, but - don't need to be started in any particular order. The example files - depend on redis being started. If you use a database server, you should - add additional dependencies. - - .. caution:: - - The included scripts run a ``gunicorn`` standalone server, - which is fine for running paperless. It does support SSL, - however, the documentation of GUnicorn states that you should - use a proxy server in front of gunicorn instead. - - For instructions on how to use nginx for that, - :ref:`see the instructions below `. - -12. Optional: Install a samba server and make the consumption folder - available as a network share. - -13. Configure ImageMagick to allow processing of PDF documents. Most distributions have - this disabled by default, since PDF documents can contain malware. If - you don't do this, paperless will fall back to ghostscript for certain steps - such as thumbnail generation. - - Edit ``/etc/ImageMagick-6/policy.xml`` and adjust - - .. code:: - - - - to - - .. code:: - - - -14. Optional: Install the `jbig2enc `_ - encoder. This will reduce the size of generated PDF documents. You'll most likely need - to compile this by yourself, because this software has been patented until around 2017 and - binary packages are not available for most distributions. - -15. Optional: If using the NLTK machine learning processing (see ``PAPERLESS_ENABLE_NLTK`` in - :ref:`configuration` for details), download the NLTK data for the Snowball Stemmer, Stopwords - and Punkt tokenizer to your ``PAPERLESS_DATA_DIR/nltk``. Refer to - the `NLTK instructions `_ for details on how to - download the data. - - -Migrating to Paperless-ngx -########################## - -Migration is possible both from Paperless-ng or directly from the 'original' Paperless. - -Migrating from Paperless-ng -=========================== - -Paperless-ngx is meant to be a drop-in replacement for Paperless-ng and thus upgrading should be -trivial for most users, especially when using docker. However, as with any major change, it is -recommended to take a full backup first. Once you are ready, simply change the docker image to -point to the new source. E.g. if using Docker Compose, edit ``docker-compose.yml`` and change: - -.. code:: - - image: jonaswinkler/paperless-ng:latest - -to - -.. code:: - - image: ghcr.io/paperless-ngx/paperless-ngx:latest - -and then run ``docker-compose up -d`` which will pull the new image recreate the container. -That's it! - -Users who installed with the bare-metal route should also update their Git clone to point to -``https://github.com/paperless-ngx/paperless-ngx``, e.g. using the command -``git remote set-url origin https://github.com/paperless-ngx/paperless-ngx`` and then pull the -lastest version. - -Migrating from Paperless -======================== - -At its core, paperless-ngx is still paperless and fully compatible. However, some -things have changed under the hood, so you need to adapt your setup depending on -how you installed paperless. - -This setup describes how to update an existing paperless Docker installation. -The important things to keep in mind are as follows: - -* Read the :doc:`changelog ` and take note of breaking changes. -* You should decide if you want to stick with SQLite or want to migrate your database - to PostgreSQL. See :ref:`setup-sqlite_to_psql` for details on how to move your data from - SQLite to PostgreSQL. Both work fine with paperless. However, if you already have a - database server running for other services, you might as well use it for paperless as well. -* The task scheduler of paperless, which is used to execute periodic tasks - such as email checking and maintenance, requires a `redis`_ message broker - instance. The docker-compose route takes care of that. -* The layout of the folder structure for your documents and data remains the - same, so you can just plug your old docker volumes into paperless-ngx and - expect it to find everything where it should be. - -Migration to paperless-ngx is then performed in a few simple steps: - -1. Stop paperless. - - .. code:: bash - - $ cd /path/to/current/paperless - $ docker-compose down - -2. Do a backup for two purposes: If something goes wrong, you still have your - data. Second, if you don't like paperless-ngx, you can switch back to - paperless. - -3. Download the latest release of paperless-ngx. You can either go with the - docker-compose files from `here `__ - or clone the repository to build the image yourself (see :ref:`above `). - You can either replace your current paperless folder or put paperless-ngx - in a different location. - - .. caution:: - - Paperless-ngx includes a ``.env`` file. This will set the - project name for docker compose to ``paperless``, which will also define the name - of the volumes by paperless-ngx. However, if you experience that paperless-ngx - is not using your old paperless volumes, verify the names of your volumes with - - .. code:: shell-session - - $ docker volume ls | grep _data - - and adjust the project name in the ``.env`` file so that it matches the name - of the volumes before the ``_data`` part. - - -4. Download the ``docker-compose.sqlite.yml`` file to ``docker-compose.yml``. - If you want to switch to PostgreSQL, do that after you migrated your existing - SQLite database. - -5. Adjust ``docker-compose.yml`` and ``docker-compose.env`` to your needs. - See :ref:`setup-docker_hub` for details on which edits are advised. - -6. :ref:`Update paperless. ` - -7. In order to find your existing documents with the new search feature, you need - to invoke a one-time operation that will create the search index: - - .. code:: shell-session - - $ docker-compose run --rm webserver document_index reindex - - This will migrate your database and create the search index. After that, - paperless will take care of maintaining the index by itself. - -8. Start paperless-ngx. - - .. code:: bash - - $ docker-compose up -d - - This will run paperless in the background and automatically start it on system boot. - -9. Paperless installed a permanent redirect to ``admin/`` in your browser. This - redirect is still in place and prevents access to the new UI. Clear your - browsing cache in order to fix this. - -10. Optionally, follow the instructions below to migrate your existing data to PostgreSQL. - - -Migrating from LinuxServer.io Docker Image -========================================== - -As with any upgrades and large changes, it is highly recommended to create a backup before -starting. This assumes the image was running using Docker Compose, but the instructions -are translatable to Docker commands as well. - -1. Stop and remove the paperless container -2. If using an external database, stop the container -3. Update Redis configuration - - a) If ``REDIS_URL`` is already set, change it to ``PAPERLESS_REDIS`` and continue - to step 4. - b) Otherwise, in the ``docker-compose.yml`` add a new service for Redis, - following `the example compose files `_ - c) Set the environment variable ``PAPERLESS_REDIS`` so it points to the new Redis container - -4. Update user mapping - - a) If set, change the environment variable ``PUID`` to ``USERMAP_UID`` - b) If set, change the environment variable ``PGID`` to ``USERMAP_GID`` - -5. Update configuration paths - - a) Set the environment variable ``PAPERLESS_DATA_DIR`` - to ``/config`` - -6. Update media paths - - a) Set the environment variable ``PAPERLESS_MEDIA_ROOT`` - to ``/data/media`` - -7. Update timezone - - a) Set the environment variable ``PAPERLESS_TIME_ZONE`` - to the same value as ``TZ`` - -8. Modify the ``image:`` to point to ``ghcr.io/paperless-ngx/paperless-ngx:latest`` or - a specific version if preferred. - -9. Start the containers as before, using ``docker-compose``. - -.. _setup-sqlite_to_psql: - -Moving data from SQLite to PostgreSQL or MySQL/MariaDB -====================================================== - -Moving your data from SQLite to PostgreSQL or MySQL/MariaDB is done via executing a series of django -management commands as below. The commands below use PostgreSQL, but are applicable to MySQL/MariaDB -with the - -.. caution:: - - Make sure that your SQLite database is migrated to the latest version. - Starting paperless will make sure that this is the case. If your try to - load data from an old database schema in SQLite into a newer database - schema in PostgreSQL, you will run into trouble. - -.. warning:: - - On some database fields, PostgreSQL enforces predefined limits on maximum - length, whereas SQLite does not. The fields in question are the title of documents - (128 characters), names of document types, tags and correspondents (128 characters), - and filenames (1024 characters). If you have data in these fields that surpasses these - limits, migration to PostgreSQL is not possible and will fail with an error. - -.. warning:: - - MySQL is case insensitive by default, treating values like "Name" and "NAME" as identical. - See :ref:`advanced-mysql-caveats` for details. - - -1. Stop paperless, if it is running. -2. Tell paperless to use PostgreSQL: - - a) With docker, copy the provided ``docker-compose.postgres.yml`` file to - ``docker-compose.yml``. Remember to adjust the consumption directory, - if necessary. - b) Without docker, configure the database in your ``paperless.conf`` file. - See :ref:`configuration` for details. - -3. Open a shell and initialize the database: - - a) With docker, run the following command to open a shell within the paperless - container: - - .. code:: shell-session - - $ cd /path/to/paperless - $ docker-compose run --rm webserver /bin/bash - - This will launch the container and initialize the PostgreSQL database. - - b) Without docker, remember to activate any virtual environment, switch to - the ``src`` directory and create the database schema: - - .. code:: shell-session - - $ cd /path/to/paperless/src - $ python3 manage.py migrate - - This will not copy any data yet. - -4. Dump your data from SQLite: - - .. code:: shell-session - - $ python3 manage.py dumpdata --database=sqlite --exclude=contenttypes --exclude=auth.Permission > data.json - -5. Load your data into PostgreSQL: - - .. code:: shell-session - - $ python3 manage.py loaddata data.json - -6. If operating inside Docker, you may exit the shell now. - - .. code:: shell-session - - $ exit - -7. Start paperless. - - -Moving back to Paperless -======================== - -Lets say you migrated to Paperless-ngx and used it for a while, but decided that -you don't like it and want to move back (If you do, send me a mail about what -part you didn't like!), you can totally do that with a few simple steps. - -Paperless-ngx modified the database schema slightly, however, these changes can -be reverted while keeping your current data, so that your current data will -be compatible with original Paperless. - -Execute this: - -.. code:: shell-session - - $ cd /path/to/paperless - $ docker-compose run --rm webserver migrate documents 0023 - -Or without docker: - -.. code:: shell-session - - $ cd /path/to/paperless/src - $ python3 manage.py migrate documents 0023 - -After that, you need to clear your cookies (Paperless-ngx comes with updated -dependencies that do cookie-processing differently) and probably your cache -as well. - -.. _setup-less_powerful_devices: - - -Considerations for less powerful devices -######################################## - -Paperless runs on Raspberry Pi. However, some things are rather slow on the Pi and -configuring some options in paperless can help improve performance immensely: - -* Stick with SQLite to save some resources. -* Consider setting ``PAPERLESS_OCR_PAGES`` to 1, so that paperless will only OCR - the first page of your documents. In most cases, this page contains enough - information to be able to find it. -* ``PAPERLESS_TASK_WORKERS`` and ``PAPERLESS_THREADS_PER_WORKER`` are configured - to use all cores. The Raspberry Pi models 3 and up have 4 cores, meaning that - paperless will use 2 workers and 2 threads per worker. This may result in - sluggish response times during consumption, so you might want to lower these - settings (example: 2 workers and 1 thread to always have some computing power - left for other tasks). -* Keep ``PAPERLESS_OCR_MODE`` at its default value ``skip`` and consider OCR'ing - your documents before feeding them into paperless. Some scanners are able to - do this! You might want to even specify ``skip_noarchive`` to skip archive - file generation for already ocr'ed documents entirely. -* If you want to perform OCR on the device, consider using ``PAPERLESS_OCR_CLEAN=none``. - This will speed up OCR times and use less memory at the expense of slightly worse - OCR results. -* If using docker, consider setting ``PAPERLESS_WEBSERVER_WORKERS`` to - 1. This will save some memory. -* Consider setting ``PAPERLESS_ENABLE_NLTK`` to false, to disable the more - advanced language processing, which can take more memory and processing time. - -For details, refer to :ref:`configuration`. - -.. note:: - - Updating the :ref:`automatic matching algorithm ` - takes quite a bit of time. However, the update mechanism checks if your - data has changed before doing the heavy lifting. If you experience the - algorithm taking too much cpu time, consider changing the schedule in the - admin interface to daily. You can also manually invoke the task - by changing the date and time of the next run to today/now. - - The actual matching of the algorithm is fast and works on Raspberry Pi as - well as on any other device. - -.. _redis: https://redis.io/ - - -.. _setup-nginx: - -Using nginx as a reverse proxy -############################## - -If you want to expose paperless to the internet, you should hide it behind a -reverse proxy with SSL enabled. - -In addition to the usual configuration for SSL, -the following configuration is required for paperless to operate: - -.. code:: nginx - - http { - - # Adjust as required. This is the maximum size for file uploads. - # The default value 1M might be a little too small. - client_max_body_size 10M; - - server { - - location / { - - # Adjust host and port as required. - proxy_pass http://localhost:8000/; - - # These configuration options are required for WebSockets to work. - proxy_http_version 1.1; - proxy_set_header Upgrade $http_upgrade; - proxy_set_header Connection "upgrade"; - - proxy_redirect off; - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Host $server_name; - } - } - } - -The ``PAPERLESS_URL`` configuration variable is also required when using a reverse proxy. Please refer to the :ref:`hosting-and-security` docs. - -Also read `this `__, towards the end of the section. + You will be redirected shortly... diff --git a/docs/troubleshooting.rst b/docs/troubleshooting.rst index 0ef573805..2e038ad54 100644 --- a/docs/troubleshooting.rst +++ b/docs/troubleshooting.rst @@ -1,328 +1,12 @@ +.. _troubleshooting: + *************** Troubleshooting *************** -No files are added by the consumer -################################## -Check for the following issues: +.. cssclass:: redirect-notice -* Ensure that the directory you're putting your documents in is the folder - paperless is watching. With docker, this setting is performed in the - ``docker-compose.yml`` file. Without docker, look at the ``CONSUMPTION_DIR`` - setting. Don't adjust this setting if you're using docker. -* Ensure that redis is up and running. Paperless does its task processing - asynchronously, and for documents to arrive at the task processor, it needs - redis to run. -* Ensure that the task processor is running. Docker does this automatically. - Manually invoke the task processor by executing + The Paperless-ngx documentation has permanently moved. - .. code:: shell-session - - $ celery --app paperless worker - -* Look at the output of paperless and inspect it for any errors. -* Go to the admin interface, and check if there are failed tasks. If so, the - tasks will contain an error message. - -Consumer warns ``OCR for XX failed`` -#################################### - -If you find the OCR accuracy to be too low, and/or the document consumer warns -that ``OCR for XX failed, but we're going to stick with what we've got since -FORGIVING_OCR is enabled``, then you might need to install the -`Tesseract language files `_ -marching your document's languages. - -As an example, if you are running Paperless-ngx from any Ubuntu or Debian -box, and your documents are written in Spanish you may need to run:: - - apt-get install -y tesseract-ocr-spa - -Consumer fails to pickup any new files -###################################### - -If you notice that the consumer will only pickup files in the consumption -directory at startup, but won't find any other files added later, you will need to -enable filesystem polling with the configuration option -``PAPERLESS_CONSUMER_POLLING``, see :ref:`here `. - -This will disable listening to filesystem changes with inotify and paperless will -manually check the consumption directory for changes instead. - - -Paperless always redirects to /admin -#################################### - -You probably had the old paperless installed at some point. Paperless installed -a permanent redirect to /admin in your browser, and you need to clear your -browsing data / cache to fix that. - - -Operation not permitted -####################### - -You might see errors such as: - -.. code:: shell-session - - chown: changing ownership of '../export': Operation not permitted - -The container tries to set file ownership on the listed directories. This is -required so that the user running paperless inside docker has write permissions -to these folders. This happens when pointing these directories to NFS shares, -for example. - -Ensure that ``chown`` is possible on these directories. - - -Classifier error: No training data available -############################################ - -This indicates that the Auto matching algorithm found no documents to learn from. -This may have two reasons: - -* You don't use the Auto matching algorithm: The error can be safely ignored in this case. -* You are using the Auto matching algorithm: The classifier explicitly excludes documents - with Inbox tags. Verify that there are documents in your archive without inbox tags. - The algorithm will only learn from documents not in your inbox. - - -UserWarning in sklearn on every single document -############################################### - -You may encounter warnings like this: - -.. code:: - - /usr/local/lib/python3.7/site-packages/sklearn/base.py:315: - UserWarning: Trying to unpickle estimator CountVectorizer from version 0.23.2 when using version 0.24.0. - This might lead to breaking code or invalid results. Use at your own risk. - -This happens when certain dependencies of paperless that are responsible for the auto matching algorithm are -updated. After updating these, your current training data *might* not be compatible anymore. This can be ignored -in most cases. This warning will disappear automatically when paperless updates the training data. - -If you want to get rid of the warning or actually experience issues with automatic matching, delete -the file ``classification_model.pickle`` in the data directory and let paperless recreate it. - - -504 Server Error: Gateway Timeout when adding Office documents -############################################################## - -You may experience these errors when using the optional TIKA integration: - -.. code:: - - requests.exceptions.HTTPError: 504 Server Error: Gateway Timeout for url: http://gotenberg:3000/forms/libreoffice/convert - -Gotenberg is a server that converts Office documents into PDF documents and has a default timeout of 30 seconds. -When conversion takes longer, Gotenberg raises this error. - -You can increase the timeout by configuring a command flag for Gotenberg (see also `here `__). -If using docker-compose, this is achieved by the following configuration change in the ``docker-compose.yml`` file: - -.. code:: yaml - - gotenberg: - image: gotenberg/gotenberg:7.6 - restart: unless-stopped - command: - - "gotenberg" - - "--chromium-disable-routes=true" - - "--api-timeout=60" - -Permission denied errors in the consumption directory -##################################################### - -You might encounter errors such as: - -.. code:: shell-session - - The following error occured while consuming document.pdf: [Errno 13] Permission denied: '/usr/src/paperless/src/../consume/document.pdf' - -This happens when paperless does not have permission to delete files inside the consumption directory. -Ensure that ``USERMAP_UID`` and ``USERMAP_GID`` are set to the user id and group id you use on the host operating system, if these are -different from ``1000``. See :ref:`setup-docker_hub`. - -Also ensure that you are able to read and write to the consumption directory on the host. - - -OSError: [Errno 19] No such device when consuming files -####################################################### - -If you experience errors such as: - -.. code:: shell-session - - File "/usr/local/lib/python3.7/site-packages/whoosh/codec/base.py", line 570, in open_compound_file - return CompoundStorage(dbfile, use_mmap=storage.supports_mmap) - File "/usr/local/lib/python3.7/site-packages/whoosh/filedb/compound.py", line 75, in __init__ - self._source = mmap.mmap(fileno, 0, access=mmap.ACCESS_READ) - OSError: [Errno 19] No such device - - During handling of the above exception, another exception occurred: - - Traceback (most recent call last): - File "/usr/local/lib/python3.7/site-packages/django_q/cluster.py", line 436, in worker - res = f(*task["args"], **task["kwargs"]) - File "/usr/src/paperless/src/documents/tasks.py", line 73, in consume_file - override_tag_ids=override_tag_ids) - File "/usr/src/paperless/src/documents/consumer.py", line 271, in try_consume_file - raise ConsumerError(e) - -Paperless uses a search index to provide better and faster full text searching. This search index is stored inside -the ``data`` folder. The search index uses memory-mapped files (mmap). The above error indicates that paperless -was unable to create and open these files. - -This happens when you're trying to store the data directory on certain file systems (mostly network shares) -that don't support memory-mapped files. - - -Web-UI stuck at "Loading..." -############################ - -This might have multiple reasons. - - -1. If you built the docker image yourself or deployed using the bare metal route, - make sure that there are files in ``/static/frontend//``. - If there are no files, make sure that you executed ``collectstatic`` successfully, either - manually or as part of the docker image build. - - If the front end is still missing, make sure that the front end is compiled (files present in - ``src/documents/static/frontend``). If it is not, you need to compile the front end yourself - or download the release archive instead of cloning the repository. - -2. Check the output of the web server. You might see errors like this: - - - .. code:: - - [2021-01-25 10:08:04 +0000] [40] [ERROR] Socket error processing request. - Traceback (most recent call last): - File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 134, in handle - self.handle_request(listener, req, client, addr) - File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 190, in handle_request - util.reraise(*sys.exc_info()) - File "/usr/local/lib/python3.7/site-packages/gunicorn/util.py", line 625, in reraise - raise value - File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 178, in handle_request - resp.write_file(respiter) - File "/usr/local/lib/python3.7/site-packages/gunicorn/http/wsgi.py", line 396, in write_file - if not self.sendfile(respiter): - File "/usr/local/lib/python3.7/site-packages/gunicorn/http/wsgi.py", line 386, in sendfile - sent += os.sendfile(sockno, fileno, offset + sent, count) - OSError: [Errno 22] Invalid argument - - To fix this issue, add - - .. code:: - - SENDFILE=0 - - to your `docker-compose.env` file. - -Error while reading metadata -############################ - -You might find messages like these in your log files: - -.. code:: - - [WARNING] [paperless.parsing.tesseract] Error while reading metadata - -This indicates that paperless failed to read PDF metadata from one of your documents. This happens when you -open the affected documents in paperless for editing. Paperless will continue to work, and will simply not -show the invalid metadata. - -Consumer fails with a FileNotFoundError -####################################### - -You might find messages like these in your log files: - -.. code:: - - [ERROR] [paperless.consumer] Error while consuming document SCN_0001.pdf: FileNotFoundError: [Errno 2] No such file or directory: '/tmp/ocrmypdf.io.yhk3zbv0/origin.pdf' - Traceback (most recent call last): - File "/app/paperless/src/paperless_tesseract/parsers.py", line 261, in parse - ocrmypdf.ocr(**args) - File "/usr/local/lib/python3.8/dist-packages/ocrmypdf/api.py", line 337, in ocr - return run_pipeline(options=options, plugin_manager=plugin_manager, api=True) - File "/usr/local/lib/python3.8/dist-packages/ocrmypdf/_sync.py", line 385, in run_pipeline - exec_concurrent(context, executor) - File "/usr/local/lib/python3.8/dist-packages/ocrmypdf/_sync.py", line 302, in exec_concurrent - pdf = post_process(pdf, context, executor) - File "/usr/local/lib/python3.8/dist-packages/ocrmypdf/_sync.py", line 235, in post_process - pdf_out = metadata_fixup(pdf_out, context) - File "/usr/local/lib/python3.8/dist-packages/ocrmypdf/_pipeline.py", line 798, in metadata_fixup - with pikepdf.open(context.origin) as original, pikepdf.open(working_file) as pdf: - File "/usr/local/lib/python3.8/dist-packages/pikepdf/_methods.py", line 923, in open - pdf = Pdf._open( - FileNotFoundError: [Errno 2] No such file or directory: '/tmp/ocrmypdf.io.yhk3zbv0/origin.pdf' - -This probably indicates paperless tried to consume the same file twice. This can happen for a number of reasons, -depending on how documents are placed into the consume folder. If paperless is using inotify (the default) to -check for documents, try adjusting the :ref:`inotify configuration `. If polling is enabled, -try adjusting the :ref:`polling configuration `. - -Consumer fails waiting for file to remain unmodified. -##################################################### - -You might find messages like these in your log files: - -.. code:: - - [ERROR] [paperless.management.consumer] Timeout while waiting on file /usr/src/paperless/src/../consume/SCN_0001.pdf to remain unmodified. - -This indicates paperless timed out while waiting for the file to be completely written to the consume folder. -Adjusting :ref:`polling configuration ` values should resolve the issue. - -.. note:: - - The user will need to manually move the file out of the consume folder and - back in, for the initial failing file to be consumed. - -Consumer fails reporting "OS reports file as busy still". -######################################################### - -You might find messages like these in your log files: - -.. code:: - - [WARNING] [paperless.management.consumer] Not consuming file /usr/src/paperless/src/../consume/SCN_0001.pdf: OS reports file as busy still - -This indicates paperless was unable to open the file, as the OS reported the file as still being in use. To prevent a -crash, paperless did not try to consume the file. If paperless is using inotify (the default) to -check for documents, try adjusting the :ref:`inotify configuration `. If polling is enabled, -try adjusting the :ref:`polling configuration `. - -.. note:: - - The user will need to manually move the file out of the consume folder and - back in, for the initial failing file to be consumed. - -Log reports "Creating PaperlessTask failed". -######################################################### - -You might find messages like these in your log files: - -.. code:: - - [ERROR] [paperless.management.consumer] Creating PaperlessTask failed: db locked - -You are likely using an sqlite based installation, with an increased number of workers and are running into sqlite's concurrency limitations. -Uploading or consuming multiple files at once results in many workers attempting to access the database simultaneously. - -Consider changing to the PostgreSQL database if you will be processing many documents at once often. Otherwise, -try tweaking the ``PAPERLESS_DB_TIMEOUT`` setting to allow more time for the database to unlock. This may have -minor performance implications. - - -gunicorn fails to start with "is not a valid port number" -######################################################### - -You are likely running using Kubernetes, which automatically creates an environment variable named `${serviceName}_PORT`. -This is the same environment variable which is used by Paperless to optionally change the port gunicorn listens on. - -To fix this, set `PAPERLESS_PORT` again to your desired port, or the default of 8000. + You will be redirected shortly... diff --git a/docs/usage_overview.rst b/docs/usage_overview.rst index a321afb93..67d5709bd 100644 --- a/docs/usage_overview.rst +++ b/docs/usage_overview.rst @@ -1,420 +1,12 @@ +.. _usage_overview: + ************** Usage Overview ************** -Paperless is an application that manages your personal documents. With -the help of a document scanner (see :ref:`scanners`), paperless transforms -your wieldy physical document binders into a searchable archive and -provides many utilities for finding and managing your documents. +.. cssclass:: redirect-notice -Terms and definitions -##################### + The Paperless-ngx documentation has permanently moved. -Paperless essentially consists of two different parts for managing your -documents: - -* The *consumer* watches a specified folder and adds all documents in that - folder to paperless. -* The *web server* provides a UI that you use to manage and search for your - scanned documents. - -Each document has a couple of fields that you can assign to them: - -* A *Document* is a piece of paper that sometimes contains valuable - information. -* The *correspondent* of a document is the person, institution or company that - a document either originates from, or is sent to. -* A *tag* is a label that you can assign to documents. Think of labels as more - powerful folders: Multiple documents can be grouped together with a single - tag, however, a single document can also have multiple tags. This is not - possible with folders. The reason folders are not implemented in paperless - is simply that tags are much more versatile than folders. -* A *document type* is used to demarcate the type of a document such as letter, - bank statement, invoice, contract, etc. It is used to identify what a document - is about. -* The *date added* of a document is the date the document was scanned into - paperless. You cannot and should not change this date. -* The *date created* of a document is the date the document was initially issued. - This can be the date you bought a product, the date you signed a contract, or - the date a letter was sent to you. -* The *archive serial number* (short: ASN) of a document is the identifier of - the document in your physical document binders. See - :ref:`usage-recommended_workflow` below. -* The *content* of a document is the text that was OCR'ed from the document. - This text is fed into the search engine and is used for matching tags, - correspondents and document types. - - -Frontend overview -################# - -.. warning:: - - TBD. Add some fancy screenshots! - -Adding documents to paperless -############################# - -Once you've got Paperless setup, you need to start feeding documents into it. -When adding documents to paperless, it will perform the following operations on -your documents: - -1. OCR the document, if it has no text. Digital documents usually have text, - and this step will be skipped for those documents. -2. Paperless will create an archivable PDF/A document from your document. - If this document is coming from your scanner, it will have embedded selectable text. -3. Paperless performs automatic matching of tags, correspondents and types on the - document before storing it in the database. - -.. hint:: - - This process can be configured to fit your needs. If you don't want paperless - to create archived versions for digital documents, you can configure that by - configuring ``PAPERLESS_OCR_MODE=skip_noarchive``. Please read the - :ref:`relevant section in the documentation `. - -.. note:: - - No matter which options you choose, Paperless will always store the original - document that it found in the consumption directory or in the mail and - will never overwrite that document. Archived versions are stored alongside the - original versions. - - -The consumption directory -========================= - -The primary method of getting documents into your database is by putting them in -the consumption directory. The consumer runs in an infinite loop, looking for new -additions to this directory. When it finds them, the consumer goes about the process -of parsing them with the OCR, indexing what it finds, and storing it in the media directory. - -Getting stuff into this directory is up to you. If you're running Paperless -on your local computer, you might just want to drag and drop files there, but if -you're running this on a server and want your scanner to automatically push -files to this directory, you'll need to setup some sort of service to accept the -files from the scanner. Typically, you're looking at an FTP server like -`Proftpd`_ or a Windows folder share with `Samba`_. - -.. _Proftpd: http://www.proftpd.org/ -.. _Samba: http://www.samba.org/ - -.. TODO: hyperref to configuration of the location of this magic folder. - -Web UI Upload -============= - -The dashboard has a file drop field to upload documents to paperless. Simply drag a file -onto this field or select a file with the file dialog. Multiple files are supported. - -You can also upload documents on any other page of the web UI by dragging-and-dropping -files into your browser window. - -.. _usage-mobile_upload: - -Mobile upload -============= - -The mobile app over at ``_ allows Android users -to share any documents with paperless. This can be combined with any of the mobile -scanning apps out there, such as Office Lens. - -Furthermore, there is the `Paperless App `_ as well, -which not only has document upload, but also document browsing and download features. - -.. _usage-email: - -IMAP (Email) -============ - -You can tell paperless-ngx to consume documents from your email accounts. -This is a very flexible and powerful feature, if you regularly received documents -via mail that you need to archive. The mail consumer can be configured by using the -admin interface in the following manner: - -1. Define e-mail accounts. -2. Define mail rules for your account. - -These rules perform the following: - -1. Connect to the mail server. -2. Fetch all matching mails (as defined by folder, maximum age and the filters) -3. Check if there are any consumable attachments. -4. If so, instruct paperless to consume the attachments and optionally - use the metadata provided in the rule for the new document. -5. If documents were consumed from a mail, the rule action is performed - on that mail. - -Paperless will completely ignore mails that do not match your filters. It will also -only perform the action on mails that it has consumed documents from. - -The actions all ensure that the same mail is not consumed twice by different means. -These are as follows: - -* **Delete:** Immediately deletes mail that paperless has consumed documents from. - Use with caution. -* **Mark as read:** Mark consumed mail as read. Paperless will not consume documents - from already read mails. If you read a mail before paperless sees it, it will be - ignored. -* **Flag:** Sets the 'important' flag on mails with consumed documents. Paperless - will not consume flagged mails. -* **Move to folder:** Moves consumed mails out of the way so that paperless wont - consume them again. -* **Add custom Tag:** Adds a custom tag to mails with consumed documents (the IMAP - standard calls these "keywords"). Paperless will not consume mails already tagged. - Not all mail servers support this feature! - -.. caution:: - - The mail consumer will perform these actions on all mails it has consumed - documents from. Keep in mind that the actual consumption process may fail - for some reason, leaving you with missing documents in paperless. - -.. note:: - - With the correct set of rules, you can completely automate your email documents. - Create rules for every correspondent you receive digital documents from and - paperless will read them automatically. The default action "mark as read" is - pretty tame and will not cause any damage or data loss whatsoever. - - You can also setup a special folder in your mail account for paperless and use - your favorite mail client to move to be consumed mails into that folder - automatically or manually and tell paperless to move them to yet another folder - after consumption. It's up to you. - -.. note:: - - When defining a mail rule with a folder, you may need to try different characters to - define how the sub-folders are separated. Common values include ".", "/" or "|", but - this varies by the mail server. Check the documentation for your mail server. In the - event of an error fetching mail from a certain folder, check the Paperless logs. When - a folder is not located, Paperless will attempt to list all folders found in the account - to the Paperless logs. - -.. note:: - - Paperless will process the rules in the order defined in the admin page. - - You can define catch-all rules and have them executed last to consume - any documents not matched by previous rules. Such a rule may assign an "Unknown - mail document" tag to consumed documents so you can inspect them further. - -Paperless is set up to check your mails every 10 minutes. This can be configured on the -'Scheduled tasks' page in the admin. - - -REST API -======== - -You can also submit a document using the REST API, see :ref:`api-file_uploads` for details. - -.. _basic-searching: - - -Best practices -############## - -Paperless offers a couple tools that help you organize your document collection. However, -it is up to you to use them in a way that helps you organize documents and find specific -documents when you need them. This section offers a couple ideas for managing your collection. - -Document types allow you to classify documents according to what they are. You can define -types such as "Receipt", "Invoice", or "Contract". If you used to collect all your receipts -in a single binder, you can recreate that system in paperless by defining a document type, -assigning documents to that type and then filtering by that type to only see all receipts. - -Not all documents need document types. Sometimes its hard to determine what the type of a -document is or it is hard to justify creating a document type that you only need once or twice. -This is okay. As long as the types you define help you organize your collection in the way -you want, paperless is doing its job. - -Tags can be used in many different ways. Think of tags are more versatile folders or binders. -If you have a binder for documents related to university / your car or health care, you can -create these binders in paperless by creating tags and assigning them to relevant documents. -Just as with documents, you can filter the document list by tags and only see documents of -a certain topic. - -With physical documents, you'll often need to decide which folder the document belongs to. -The advantage of tags over folders and binders is that a single document can have multiple -tags. A physical document cannot magically appear in two different folders, but with tags, -this is entirely possible. - -.. hint:: - - This can be used in many different ways. One example: Imagine you're working on a particular - task, such as signing up for university. Usually you'll need to collect a bunch of different - documents that are already sorted into various folders. With the tag system of paperless, - you can create a new group of documents that are relevant to this task without destroying - the already existing organization. When you're done with the task, you could delete the - tag again, which would be equal to sorting documents back into the folder they belong into. - Or keep the tag, up to you. - -All of the logic above applies to correspondents as well. Attach them to documents if you -feel that they help you organize your collection. - -When you've started organizing your documents, create a couple saved views for document collections -you regularly access. This is equal to having labeled physical binders on your desk, except -that these saved views are dynamic and simply update themselves as you add documents to the system. - -Here are a couple examples of tags and types that you could use in your collection. - -* An ``inbox`` tag for newly added documents that you haven't manually edited yet. -* A tag ``car`` for everything car related (repairs, registration, insurance, etc) -* A tag ``todo`` for documents that you still need to do something with, such as reply, or - perform some task online. -* A tag ``bank account x`` for all bank statement related to that account. -* A tag ``mail`` for anything that you added to paperless via its mail processing capabilities. -* A tag ``missing_metadata`` when you still need to add some metadata to a document, but can't - or don't want to do this right now. - -.. _basic-usage_searching: - -Searching -######### - -Paperless offers an extensive searching mechanism that is designed to allow you to quickly -find a document you're looking for (for example, that thing that just broke and you bought -a couple months ago, that contract you signed 8 years ago). - -When you search paperless for a document, it tries to match this query against your documents. -Paperless will look for matching documents by inspecting their content, title, correspondent, -type and tags. Paperless returns a scored list of results, so that documents matching your query -better will appear further up in the search results. - -By default, paperless returns only documents which contain all words typed in the search bar. -However, paperless also offers advanced search syntax if you want to drill down the results -further. - -Matching documents with logical expressions: - -.. code:: - - shopname AND (product1 OR product2) - -Matching specific tags, correspondents or types: - -.. code:: - - type:invoice tag:unpaid - correspondent:university certificate - -Matching dates: - -.. code:: - - created:[2005 to 2009] - added:yesterday - modified:today - -Matching inexact words: - -.. code:: - - produ*name - -.. note:: - - Inexact terms are hard for search indexes. These queries might take a while to execute. That's why paperless offers - auto complete and query correction. - -All of these constructs can be combined as you see fit. -If you want to learn more about the query language used by paperless, paperless uses Whoosh's default query language. -Head over to `Whoosh query language `_. -For details on what date parsing utilities are available, see -`Date parsing `_. - - -.. _usage-recommended_workflow: - -The recommended workflow -######################## - -Once you have familiarized yourself with paperless and are ready to use it -for all your documents, the recommended workflow for managing your documents -is as follows. This workflow also takes into account that some documents -have to be kept in physical form, but still ensures that you get all the -advantages for these documents as well. - -The following diagram shows how easy it is to manage your documents. - -.. image:: _static/recommended_workflow.png - -Preparations in paperless -========================= - -* Create an inbox tag that gets assigned to all new documents. -* Create a TODO tag. - -Processing of the physical documents -==================================== - -Keep a physical inbox. Whenever you receive a document that you need to -archive, put it into your inbox. Regularly, do the following for all documents -in your inbox: - -1. For each document, decide if you need to keep the document in physical - form. This applies to certain important documents, such as contracts and - certificates. -2. If you need to keep the document, write a running number on the document - before scanning, starting at one and counting upwards. This is the archive - serial number, or ASN in short. -3. Scan the document. -4. If the document has an ASN assigned, store it in a *single* binder, sorted - by ASN. Don't order this binder in any other way. -5. If the document has no ASN, throw it away. Yay! - -Over time, you will notice that your physical binder will fill up. If it is -full, label the binder with the range of ASNs in this binder (i.e., "Documents -1 to 343"), store the binder in your cellar or elsewhere, and start a new -binder. - -The idea behind this process is that you will never have to use the physical -binders to find a document. If you need a specific physical document, you -may find this document by: - -1. Searching in paperless for the document. -2. Identify the ASN of the document, since it appears on the scan. -3. Grab the relevant document binder and get the document. This is easy since - they are sorted by ASN. - -Processing of documents in paperless -==================================== - -Once you have scanned in a document, proceed in paperless as follows. - -1. If the document has an ASN, assign the ASN to the document. -2. Assign a correspondent to the document (i.e., your employer, bank, etc) - This isn't strictly necessary but helps in finding a document when you need - it. -3. Assign a document type (i.e., invoice, bank statement, etc) to the document - This isn't strictly necessary but helps in finding a document when you need - it. -4. Assign a proper title to the document (the name of an item you bought, the - subject of the letter, etc) -5. Check that the date of the document is correct. Paperless tries to read - the date from the content of the document, but this fails sometimes if the - OCR is bad or multiple dates appear on the document. -6. Remove inbox tags from the documents. - -.. hint:: - - You can setup manual matching rules for your correspondents and tags and - paperless will assign them automatically. After consuming a couple documents, - you can even ask paperless to *learn* when to assign tags and correspondents - by itself. For details on this feature, see :ref:`advanced-matching`. - -Task management -=============== - -Some documents require attention and require you to act on the document. You -may take two different approaches to handle these documents based on how -regularly you intend to scan documents and use paperless. - -* If you scan and process your documents in paperless regularly, assign a - TODO tag to all scanned documents that you need to process. Create a saved - view on the dashboard that shows all documents with this tag. -* If you do not scan documents regularly and use paperless solely for archiving, - create a physical todo box next to your physical inbox and put documents you - need to process in the TODO box. When you performed the task associated with - the document, move it to the inbox. + You will be redirected shortly...