From 38ba1d1a52280073e50a7161bfce44cde34e052a Mon Sep 17 00:00:00 2001 From: Trenton Holmes Date: Wed, 28 Sep 2022 19:37:19 -0700 Subject: [PATCH] Cleans up and documnets installation on bare metal, includes systemd service file for the celery and celery beat --- docs/setup.rst | 33 +++++++++++++++++----------- scripts/paperless-scheduler.service | 4 ++-- scripts/paperless-task-queue.service | 12 ++++++++++ 3 files changed, 34 insertions(+), 15 deletions(-) create mode 100644 scripts/paperless-task-queue.service diff --git a/docs/setup.rst b/docs/setup.rst index ca07c1032..f2970fd9b 100644 --- a/docs/setup.rst +++ b/docs/setup.rst @@ -39,7 +39,7 @@ Paperless consists of the following components: .. _setup-task_processor: -* **The task processor:** Paperless relies on `Django Q `_ +* **The task processor:** Paperless relies on `Celery - Distributed Task Queue `_ for doing most of the heavy lifting. This is a task queue that accepts tasks from multiple sources and processes these in parallel. It also comes with a scheduler that executes certain commands periodically. @@ -62,13 +62,6 @@ Paperless consists of the following components: tasks fail and inspect the errors (i.e., wrong email credentials, errors during consuming a specific file, etc). - You may start the task processor by executing: - - .. code:: shell-session - - $ cd /path/to/paperless/src/ - $ python3 manage.py qcluster - * A `redis `_ message broker: This is a really lightweight service that is responsible for getting the tasks from the webserver and the consumer to the task scheduler. These run in a different process (maybe even on different machines!), and therefore, this is necessary. @@ -291,7 +284,20 @@ Build the Docker image yourself .. code:: yaml webserver: - build: . + build: + context: . + args: + QPDF_VERSION: x.y.x + PIKEPDF_VERSION: x.y.z + PSYCOPG2_VERSION: x.y.z + JBIG2ENC_VERSION: 0.29 + + .. note:: + + You should match the build argument versions to the version for the release you have + checked out. These are pre-built images with certain, more updated software. + If you want to build these images your self, that is possible, but beyond + the scope of these steps. 4. Follow steps 3 to 8 of :ref:`setup-docker_hub`. When asked to run ``docker-compose pull`` to pull the image, do @@ -332,7 +338,7 @@ writing. Windows is not and will never be supported. .. code:: - python3 python3-pip python3-dev imagemagick fonts-liberation gnupg libpq-dev libmagic-dev mime-support libzbar0 poppler-utils + python3 python3-pip python3-dev imagemagick fonts-liberation gnupg libpq-dev default-libmysqlclient-dev libmagic-dev mime-support libzbar0 poppler-utils These dependencies are required for OCRmyPDF, which is used for text recognition. @@ -361,7 +367,7 @@ writing. Windows is not and will never be supported. You will also need ``build-essential``, ``python3-setuptools`` and ``python3-wheel`` for installing some of the python dependencies. -2. Install ``redis`` >= 5.0 and configure it to start automatically. +2. Install ``redis`` >= 6.0 and configure it to start automatically. 3. Optional. Install ``postgresql`` and configure a database, user and password for paperless. If you do not wish to use PostgreSQL, MariaDB and SQLite are available as well. @@ -461,8 +467,9 @@ writing. Windows is not and will never be supported. as a starting point. Paperless needs the ``webserver`` script to run the webserver, the - ``consumer`` script to watch the input folder, and the ``scheduler`` - script to run tasks such as email checking and document consumption. + ``consumer`` script to watch the input folder, ``taskqueue`` for the background workers + used to handle things like document consumption and the ``scheduler`` script to run tasks such as + email checking at certain times . The ``socket`` script enables ``gunicorn`` to run on port 80 without root privileges. For this you need to uncomment the ``Require=paperless-webserver.socket`` diff --git a/scripts/paperless-scheduler.service b/scripts/paperless-scheduler.service index b1c82a38e..b15c53e79 100644 --- a/scripts/paperless-scheduler.service +++ b/scripts/paperless-scheduler.service @@ -1,12 +1,12 @@ [Unit] -Description=Paperless scheduler +Description=Paperless Celery Beat Requires=redis.service [Service] User=paperless Group=paperless WorkingDirectory=/opt/paperless/src -ExecStart=python3 manage.py qcluster +ExecStart=celery --app paperless beat --loglevel INFO [Install] WantedBy=multi-user.target diff --git a/scripts/paperless-task-queue.service b/scripts/paperless-task-queue.service new file mode 100644 index 000000000..5fade360a --- /dev/null +++ b/scripts/paperless-task-queue.service @@ -0,0 +1,12 @@ +[Unit] +Description=Paperless Celery Workers +Requires=redis.service + +[Service] +User=paperless +Group=paperless +WorkingDirectory=/opt/paperless/src +ExecStart=celery --app paperless worker --loglevel INFO + +[Install] +WantedBy=multi-user.target