From 48b042bfb0af66ef495c9af095d5cdb8697e04db Mon Sep 17 00:00:00 2001 From: Jonas Winkler Date: Mon, 1 Feb 2021 20:14:29 +0100 Subject: [PATCH 1/4] troubleshooting for #216 --- docs/troubleshooting.rst | 39 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/docs/troubleshooting.rst b/docs/troubleshooting.rst index f55d57af5..ae7fe14c7 100644 --- a/docs/troubleshooting.rst +++ b/docs/troubleshooting.rst @@ -60,7 +60,8 @@ required so that the user running paperless inside docker has write permissions to these folders. This happens when pointing these directories to NFS shares, for example. -Ensure that `chown` is possible on these directories. +Ensure that ``chown`` is possible on these directories. + Classifier error: No training data available ############################################ @@ -73,6 +74,7 @@ This may have two reasons: with Inbox tags. Verify that there are documents in your archive without inbox tags. The algorithm will only learn from documents not in your inbox. + UserWarning in sklearn on every single document ############################################### @@ -91,6 +93,7 @@ in most cases. This warning will disappear automatically when paperless updates If you want to get rid of the warning or actually experience issues with automatic matching, delete the file ``classification_model.pickle`` in the data directory and let paperless recreate it. + Permission denied errors in the consumption directory ##################################################### @@ -106,6 +109,38 @@ different from ``1000``. See :ref:`setup-docker_hub`. Also ensure that you are able to read and write to the consumption directory on the host. + +OSError: [Errno 19] No such device when consuming files +####################################################### + +If you experience errors such as: + +.. code:: shell-session + + File "/usr/local/lib/python3.7/site-packages/whoosh/codec/base.py", line 570, in open_compound_file + return CompoundStorage(dbfile, use_mmap=storage.supports_mmap) + File "/usr/local/lib/python3.7/site-packages/whoosh/filedb/compound.py", line 75, in __init__ + self._source = mmap.mmap(fileno, 0, access=mmap.ACCESS_READ) + OSError: [Errno 19] No such device + + During handling of the above exception, another exception occurred: + + Traceback (most recent call last): + File "/usr/local/lib/python3.7/site-packages/django_q/cluster.py", line 436, in worker + res = f(*task["args"], **task["kwargs"]) + File "/usr/src/paperless/src/documents/tasks.py", line 73, in consume_file + override_tag_ids=override_tag_ids) + File "/usr/src/paperless/src/documents/consumer.py", line 271, in try_consume_file + raise ConsumerError(e) + +Paperless uses a search index to provide better and faster full text searching. This search index is stored inside +the ``data`` folder. The search index uses memory-mapped files (mmap). The above error indicates that paperless +was unable to create and open these files. + +This happens when you're trying to store the data directory on certain file systems (mostly network shares) +that don't support memory-mapped files. + + Web-UI stuck at "Loading..." ############################ @@ -148,4 +183,4 @@ This might have multiple reasons. SENDFILE=0 - to your `docker-compose.env` file. \ No newline at end of file + to your `docker-compose.env` file. From 0a28ccf62fac76d1fef0e18434d81ce1c3597ca2 Mon Sep 17 00:00:00 2001 From: Jonas Winkler Date: Mon, 1 Feb 2021 21:43:50 +0100 Subject: [PATCH 2/4] adjustments to setup --- docs/setup.rst | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/docs/setup.rst b/docs/setup.rst index afb3784d6..a86c4ce27 100644 --- a/docs/setup.rst +++ b/docs/setup.rst @@ -151,13 +151,16 @@ Install Paperless from Docker Hub .. code:: - /home/jonaswinkler/paperless-inbox:/usr/src/paperless/consume - + Don't change the part after the colon or paperless wont find your documents. 5. Modify ``docker-compose.env``, following the comments in the file. The most important change is to set ``USERMAP_UID`` and ``USERMAP_GID`` - to the uid and gid of your user on the host system. This ensures that + to the uid and gid of your user on the host system. Use ``id -u`` and + ``id -g`` to get these. + + This ensures that both the docker container and you on the host machine have write access to the consumption directory. If your UID and GID on the host system is 1000 (the default for the first normal user on most systems), it will @@ -175,15 +178,11 @@ Install Paperless from Docker Hub on such a file system, paperless will not pick up new files with the default configuration. You will need to use ``PAPERLESS_CONSUMER_POLLING``, which will disable inotify. See :ref:`here `. - -6. Now head over to: https://hub.docker.com/r/jonaswinkler/paperless-ng and choose your preferred - image and copy the link. To download this image do a `docker pull` followed by the link. Do this within the directory with the .yml files. - Depending on your network connection and CPU this will take a while. You have time to get a beverage. -7. Run ``docker-compose up -d``. This will create and start the necessary - containers, but your are not done yet! +6. Run ``docker-compose up -d``. This will pull the image, create and start the necessary + containers. -8. To be able to login, you will need a super user. To create it, execute the +7. To be able to login, you will need a super user. To create it, execute the following command: .. code-block:: shell-session @@ -193,8 +192,8 @@ Install Paperless from Docker Hub This will prompt you to set a username, an optional e-mail address and finally a password (at least 8 characters). -9. The default ``docker-compose.yml`` exports the webserver on your local port - 8000. If you haven't adapted this, you should now be able to visit your +8. The default ``docker-compose.yml`` exports the webserver on your local port + 8000. If you did not change this, you should now be able to visit your Paperless instance at ``http://127.0.0.1:8000`` or your servers IP-Address:8000. Use the login credentials you have created with the previous step. From 2efb99192a67fb28269ec513ef1f2f3acddb511c Mon Sep 17 00:00:00 2001 From: Jonas Winkler Date: Mon, 1 Feb 2021 21:45:45 +0100 Subject: [PATCH 3/4] add pull to docker setup --- docs/setup.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/setup.rst b/docs/setup.rst index a86c4ce27..5c3c9c361 100644 --- a/docs/setup.rst +++ b/docs/setup.rst @@ -179,8 +179,8 @@ Install Paperless from Docker Hub with the default configuration. You will need to use ``PAPERLESS_CONSUMER_POLLING``, which will disable inotify. See :ref:`here `. -6. Run ``docker-compose up -d``. This will pull the image, create and start the necessary - containers. +6. Run ``docker-compose pull``, followed by ``docker-compose up -d``. + This will pull the image, create and start the necessary containers. 7. To be able to login, you will need a super user. To create it, execute the following command: From 32c9369ba18e1f109c3b6707a49dd459767925e7 Mon Sep 17 00:00:00 2001 From: Fabian Koller Date: Wed, 3 Feb 2021 07:53:49 +0100 Subject: [PATCH 4/4] Apply consistent name structure to vars --- ansible/README.md | 2 +- ansible/defaults/main.yml | 16 ++++++++-------- ansible/tasks/main.yml | 16 ++++++++-------- docs/setup.rst | 20 ++++++++++---------- 4 files changed, 27 insertions(+), 27 deletions(-) diff --git a/ansible/README.md b/ansible/README.md index 6cd9e9f9b..0940cf072 100644 --- a/ansible/README.md +++ b/ansible/README.md @@ -107,7 +107,7 @@ Example Playbook paperlessng_db_type: postgresql paperlessng_db_pass: PLEASEPROVIDEASTRONGPASSWORDHERE - paperless_secret_key: AGAINPLEASECHANGETHISNOW + paperlessng_secret_key: AGAINPLEASECHANGETHISNOW paperlessng_ocr_languages: - eng diff --git a/ansible/defaults/main.yml b/ansible/defaults/main.yml index aaeffa507..f1cbb8f88 100644 --- a/ansible/defaults/main.yml +++ b/ansible/defaults/main.yml @@ -23,14 +23,14 @@ paperlessng_filename_format: paperlessng_virtualenv: "{{ paperlessng_directory }}/.venv" # Hosting & Security -paperless_secret_key: PLEASECHANGETHISFORTHELOVEOFGOD -paperless_allowed_hosts: "*" -paperless_cors_allowed_hosts: http://localhost:8000 -paperless_force_script_name: -paperless_static_url: /static/ -paperless_auto_login_username: -paperless_cookie_prefix: "" -paperless_enable_http_remote_user: False +paperlessng_secret_key: PLEASECHANGETHISFORTHELOVEOFGOD +paperlessng_allowed_hosts: "*" +paperlessng_cors_allowed_hosts: http://localhost:8000 +paperlessng_force_script_name: +paperlessng_static_url: /static/ +paperlessng_auto_login_username: +paperlessng_cookie_prefix: "" +paperlessng_enable_http_remote_user: False # OCR settings paperlessng_ocr_languages: diff --git a/ansible/tasks/main.yml b/ansible/tasks/main.yml index dba409549..ed05cd05e 100644 --- a/ansible/tasks/main.yml +++ b/ansible/tasks/main.yml @@ -286,21 +286,21 @@ line: "PAPERLESS_FILENAME_FORMAT={{ paperlessng_filename_format }}" # Hosting & Security - regexp: PAPERLESS_SECRET_KEY - line: "PAPERLESS_SECRET_KEY={{ paperless_secret_key }}" + line: "PAPERLESS_SECRET_KEY={{ paperlessng_secret_key }}" - regexp: PAPERLESS_ALLOWED_HOSTS - line: "PAPERLESS_ALLOWED_HOSTS={{ paperless_allowed_hosts }}" + line: "PAPERLESS_ALLOWED_HOSTS={{ paperlessng_allowed_hosts }}" - regexp: PAPERLESS_CORS_ALLOWED_HOSTS - line: "PAPERLESS_CORS_ALLOWED_HOSTS={{ paperless_cors_allowed_hosts }}" + line: "PAPERLESS_CORS_ALLOWED_HOSTS={{ paperlessng_cors_allowed_hosts }}" - regexp: PAPERLESS_FORCE_SCRIPT_NAME - line: "PAPERLESS_FORCE_SCRIPT_NAME={{ paperless_force_script_name }}" + line: "PAPERLESS_FORCE_SCRIPT_NAME={{ paperlessng_force_script_name }}" - regexp: PAPERLESS_STATIC_URL - line: "PAPERLESS_STATIC_URL={{ paperless_static_url }}" + line: "PAPERLESS_STATIC_URL={{ paperlessng_static_url }}" - regexp: PAPERLESS_AUTO_LOGIN_USERNAME - line: "PAPERLESS_AUTO_LOGIN_USERNAME={{ paperless_auto_login_username }}" + line: "PAPERLESS_AUTO_LOGIN_USERNAME={{ paperlessng_auto_login_username }}" - regexp: PAPERLESS_COOKIE_PREFIX - line: "PAPERLESS_COOKIE_PREFIX={{ paperless_cookie_prefix }}" + line: "PAPERLESS_COOKIE_PREFIX={{ paperlessng_cookie_prefix }}" - regexp: PAPERLESS_ENABLE_HTTP_REMOTE_USER - line: "PAPERLESS_ENABLE_HTTP_REMOTE_USER={{ paperless_enable_http_remote_user }}" + line: "PAPERLESS_ENABLE_HTTP_REMOTE_USER={{ paperlessng_enable_http_remote_user }}" # OCR settings - regexp: PAPERLESS_OCR_LANGUAGE line: "PAPERLESS_OCR_LANGUAGE={{ paperlessng_ocr_languages | join('+') }}" diff --git a/docs/setup.rst b/docs/setup.rst index 5c3c9c361..e040db790 100644 --- a/docs/setup.rst +++ b/docs/setup.rst @@ -88,7 +88,7 @@ You can go multiple routes to setup and run Paperless: The Docker routes are quick & easy. These are the recommended routes. This configures all the stuff from the above automatically so that it just works and uses sensible defaults for all configuration options. -Here you find a cheat-sheet for docker beginners: `CLI Basics `_ +Here you find a cheat-sheet for docker beginners: `CLI Basics `_ The bare metal route is complicated to setup but makes it easier should you want to contribute some code back. You need to configure and @@ -126,7 +126,7 @@ Install Paperless from Docker Hub If you want to use the included ``docker-compose.*.yml`` file, you need to have at least Docker version **17.09.0** and docker-compose - version **1.17.0**. + version **1.17.0**. To check do: `docker-compose -v` or `docker -v` See the `Docker installation guide`_ on how to install the current @@ -151,7 +151,7 @@ Install Paperless from Docker Hub .. code:: - /home/jonaswinkler/paperless-inbox:/usr/src/paperless/consume - + Don't change the part after the colon or paperless wont find your documents. @@ -159,7 +159,7 @@ Install Paperless from Docker Hub most important change is to set ``USERMAP_UID`` and ``USERMAP_GID`` to the uid and gid of your user on the host system. Use ``id -u`` and ``id -g`` to get these. - + This ensures that both the docker container and you on the host machine have write access to the consumption directory. If your UID and GID on the host system is @@ -194,7 +194,7 @@ Install Paperless from Docker Hub 8. The default ``docker-compose.yml`` exports the webserver on your local port 8000. If you did not change this, you should now be able to visit your - Paperless instance at ``http://127.0.0.1:8000`` or your servers IP-Address:8000. + Paperless instance at ``http://127.0.0.1:8000`` or your servers IP-Address:8000. Use the login credentials you have created with the previous step. .. _Docker: https://www.docker.com/ @@ -223,7 +223,7 @@ Build the docker image yourself webserver: image: jonaswinkler/paperless-ng:latest - + and replace it with a line that instructs docker-compose to build the image from the current working directory instead: .. code:: yaml @@ -426,7 +426,7 @@ Install Paperless using ansible This role currently only supports Debian 10 Buster and Ubuntu 20.04 Focal or later as target hosts. -1. Install ansible 2.7+ on the management node. +1. Install ansible 2.7+ on the management node. This may be the target host paperless-ng is being installed on or any remote host which can access the target host. For further details, check the ansible `inventory `_ documentation. @@ -461,7 +461,7 @@ Install Paperless using ansible .. code:: sh cd paperless-ng - git checkout ng-0.9.14 + git checkout ng-1.0.0 3. Create an ansible ``playbook.yml`` in the paperless-ng root directory: @@ -511,7 +511,7 @@ Install Paperless using ansible .. code:: yaml - paperless_secret_key: PleaseGenerateAStrongKeyForThis + paperlessng_secret_key: PleaseGenerateAStrongKeyForThis paperlessng_superuser_name: YourUserName paperlessng_superuser_email: name@domain.tld @@ -584,7 +584,7 @@ Migration to paperless-ng is then performed in a few simple steps: .. code:: shell-session $ docker volume ls | grep _data - + and adjust the project name in the ``.env`` file so that it matches the name of the volumes before the ``_data`` part.