paradizelost/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2025-04-02 13:45:10 -05:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into dev

Browse Source
This commit is contained in:
jonaswinkler 2021-01-18 15:44:26 +01:00
commit 8427d58337
6 changed files with 331 additions and 70 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

108
ansible/README.md
View File

@ -1,38 +1,114 @@
Role Name
=========
Ansible Role: paperless-ng
==========================
A brief description of the role goes here.
Installs and configures paperless-ng EDMS on Debian/Ubuntu systems.
Requirements
------------
Any pre-requisites that may not be covered by Ansible itself or the role should be mentioned here. For instance, if the role uses the EC2 module, it may be a good idea to mention in this section that the boto package is required.
No special system requirements. Ansible 2.7 or newer is required.
Note that this role requires root access, so either run it in a playbook with a global `become: yes`, or invoke the role in your playbook like:
- hosts: all
roles:
- role: ansible
become: yes
Role Variables
--------------
A description of the settable variables for this role should go here, including any variables that are in defaults/main.yml, vars/main.yml, and any variables that can/should be set via parameters to the role. Any variables that are read from other roles and/or the global scope (ie. hostvars, group vars, etc.) should be mentioned here as well.
Most configuration variables from paperless-ng itself are available and accept their respective arguments.
Every `PAPERLESS_*` configuration varaible is lowercased and instead prefixed with `paperlessng_*` in `defaults/main.yml`.
For a full listing including explainations and allowed values, see the current [documentation](https://paperless-ng.readthedocs.io/en/ng-0.9.14/configuration.html).
Additional variables available in this role are listed below, along with default values:
paperlessng_version: 0.9.14
The [release](https://github.com/jonaswinkler/paperless-ng/releases) archive version of paperless-ng to install.
paperlessng_redis_host: localhost
paperlessng_redis_port: 6379
Seperate configuration values that combine into `PAPERLESS_REDIS`.
paperlessng_db_type: sqlite
Database to use. Default is file-based SQLite.
paperlessng_db_host: localhost
paperlessng_db_port: 5432
paperlessng_db_name: paperlessng
paperlessng_db_user: paperlessng
paperlessng_db_pass: paperlessng
paperlessng_db_sslmode: prefer
Database configuration (only applicable if `paperlessng_db_type == 'postgresql'`).
paperlessng_directory: /opt/paperless-ng
Root directory paperless-ng is installed into.
paperlessng_virtualenv: "{{ paperlessng_directory }}/.venv"
Directory used for the virtual environment for paperless-ng.
paperlessng_ocr_languages:
- eng
List of OCR languages to install and configure (`apt search tesseract-ocr-*`).
paperlessng_use_jbig2enc: True
Whether to install and use [jbig2enc](https://github.com/agl/jbig2enc) for OCRmyPDF.
paperlessng_big2enc_lossy: False
Whether to use jbig2enc's lossy compression mode.
paperlessng_superuser_name: paperlessng
paperlessng_superuser_email: paperlessng@example.com
paperlessng_superuser_password: paperlessng
Credentials of the initial superuser in paperless-ng.
paperlessng_system_user: paperlessng
paperlessng_system_group: paperlessng
System user and group to run the paperless-ng services as (will be created if required).
paperlessng_listen_address: 127.0.0.1
paperlessng_listen_port: 8000
Address and port for the paperless-ng service to listen on.
Dependencies
------------
A list of other roles hosted on Galaxy should go here, plus any details in regards to parameters that may need to be set for other roles, or variables that are used from other roles.
No ansible dependencies.
Example Playbook
----------------
`playbook.yml`:
Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too:
- hosts: servers
- hosts: all
become: yes
vars_files:
- vars/main.yml
roles:
- { role: username.rolename, x: 42 }
- ansible
License
-------
`vars/main.yml`:
BSD
paperlessng_media_root: /mnt/media/smbshare
Author Information
------------------
paperlessng_db_type: postgresql
paperlessng_db_pass: PLEASEPROVIDEASTRONGPASSWORDHERE
An optional section for the role authors to include contact information, or a website (HTML is not allowed).
paperless_secret_key: AGAINPLEASECHANGETHISNOW
paperlessng_ocr_languages:
- eng
- deu

71
ansible/defaults/main.yml
View File

@ -1,5 +1,19 @@
---
paperlessng_version: 0.9.13
paperlessng_version: 0.9.14
# Required services
paperlessng_redis_host: localhost
paperlessng_redis_port: 6379
paperlessng_db_type: sqlite # or postgresql
# Below entries only apply for paperlessng_db_type=='postgresql'
paperlessng_db_host: localhost
paperlessng_db_port: 5432
paperlessng_db_name: paperlessng
paperlessng_db_user: paperlessng
paperlessng_db_pass: paperlessng
paperlessng_db_sslmode: prefer
# Paths and folders
paperlessng_directory: /opt/paperless-ng
paperlessng_consumption_dir: "{{ paperlessng_directory }}/consumption"
paperlessng_data_dir: "{{ paperlessng_directory }}/data"
@ -8,36 +22,57 @@ paperlessng_static_dir: "{{ paperlessng_directory }}/static"
paperlessng_filename_format:
paperlessng_virtualenv: "{{ paperlessng_directory }}/.venv"
# Hosting & Security
paperless_secret_key: PLEASECHANGETHISFORTHELOVEOFGOD
paperless_allowed_hosts: "*"
paperless_cors_allowed_hosts: http://localhost:8000
paperless_force_script_name:
paperless_static_url: /static/
paperless_auto_login_username:
paperless_cookie_prefix: ""
paperless_enable_http_remote_user: False
# OCR settings
paperlessng_ocr_languages:
- eng
paperlessng_time_zone: Europe/Berlin
paperlessng_ocr_mode: skip
paperlessng_ocr_output_type: pdfa
paperlessng_ocr_pages: 0
paperlessng_ocr_image_dpi:
# see https://ocrmypdf.readthedocs.io/en/latest/api.html#ocrmypdf.ocr
paperlessng_ocrmypdf_args:
#- "deskew": true # https://github.com/jonaswinkler/paperless-ng/issues/231
paperlessng_ocr_user_args:
#- "deskew": True # https://github.com/jonaswinkler/paperless-ng/issues/231
- "optimize": 1
paperlessng_use_jbig2enc: true
paperlessng_big2enc_lossy: false
paperlessng_tika_enabled: false
paperlessng_use_jbig2enc: True
paperlessng_big2enc_lossy: False
# Tika settings
paperlessng_tika_enabled: False
paperlessng_tika_endpoint: http://localhost:9998
paperlessng_tika_gotenberg_endpoint: http://localhost:3000
# Software tweaks
paperlessng_time_zone: Europe/Berlin
paperlessng_consumer_polling: 0
paperlessng_consumer_delete_duplicates: False
paperlessng_consumer_recursive: False
paperlessng_consumer_subdirs_as_tags: False
paperlessng_optimize_thumbnails: True
paperlessng_post_consume_script:
paperlessng_filename_date_order:
paperlessng_filename_parse_transforms:
paperlessng_thumbnail_font_name: /usr/share/fonts/liberation/LiberationSerif-Regular.ttf
paperlessng_ignore_dates: ""
# Superuser settings
paperlessng_superuser_name: paperlessng
paperlessng_superuser_email: paperlessng@example.com
paperlessng_superuser_password: paperlessng
# System user settings
paperlessng_system_user: paperlessng
paperlessng_system_group: paperlessng
# Webserver settings
paperlessng_listen_address: 127.0.0.1
paperlessng_listen_port: 8000
paperlessng_redis_host: localhost
paperlessng_redis_port: 6379
paperlessng_db_type: sqlite # or postgresql
# Below entries only apply for paperlessng_db_type=='postgresql'
paperlessng_db_host: localhost
paperlessng_db_port: 5432
paperlessng_db_name: paperlessng
paperlessng_db_user: paperlessng
paperlessng_db_pass: paperlessng

95
ansible/tasks/main.yml
View File

@ -9,26 +9,26 @@
update_cache: yes
pkg:
# paperless-ng
- python3-dev
- python3-pip
- gettext
- python3-dev
- fonts-liberation
- imagemagick
- unpaper
- ghostscript
- optipng
- tesseract-ocr
- gnupg
- libpoppler-cpp-dev
- libmagic-dev
- libpq-dev
- libmagic-dev
- mime-support
# OCRmyPDF
- unpaper
- ghostscript
- icc-profiles-free
- qpdf
- liblept5
- libxml2
- pngquant
- zlib1g
- tesseract-ocr
# dev
- sudo
- build-essential
@ -154,7 +154,7 @@
group: "{{ paperlessng_system_group }}"
mode: "750"
with_items:
- "{{ paperlessng_directory }}" # ansible `copy:` does not set correct permissions on `dest:` for recursive copies
- "{{ paperlessng_directory }}"
- "{{ paperlessng_consumption_dir }}"
- "{{ paperlessng_data_dir }}"
- "{{ paperlessng_media_root }}"
@ -171,8 +171,10 @@
regexp: "^#?{{ item.regexp }}="
line: "{{ item.line }}"
with_items:
# Required services
- regexp: PAPERLESS_REDIS
line: "PAPERLESS_REDIS=redis://{{ paperlessng_redis_host }}:{{ paperlessng_redis_port }}"
# Paths and folders
- regexp: PAPERLESS_CONSUMPTION_DIR
line: "PAPERLESS_CONSUMPTION_DIR={{ paperlessng_consumption_dir }}"
- regexp: PAPERLESS_DATA_DIR
@ -183,26 +185,65 @@
line: "PAPERLESS_STATICDIR={{ paperlessng_static_dir }}"
- regexp: PAPERLESS_FILENAME_FORMAT
line: "PAPERLESS_FILENAME_FORMAT={{ paperlessng_filename_format }}"
# Hosting & Security
- regexp: PAPERLESS_SECRET_KEY
line: "PAPERLESS_SECRET_KEY={{ paperless_secret_key }}"
- regexp: PAPERLESS_ALLOWED_HOSTS
line: "PAPERLESS_ALLOWED_HOSTS={{ paperless_allowed_hosts }}"
- regexp: PAPERLESS_CORS_ALLOWED_HOSTS
line: "PAPERLESS_CORS_ALLOWED_HOSTS={{ paperless_cors_allowed_hosts }}"
- regexp: PAPERLESS_FORCE_SCRIPT_NAME
line: "PAPERLESS_FORCE_SCRIPT_NAME={{ paperless_force_script_name }}"
- regexp: PAPERLESS_STATIC_URL
line: "PAPERLESS_STATIC_URL={{ paperless_static_url }}"
- regexp: PAPERLESS_AUTO_LOGIN_USERNAME
line: "PAPERLESS_AUTO_LOGIN_USERNAME={{ paperless_auto_login_username }}"
- regexp: PAPERLESS_COOKIE_PREFIX
line: "PAPERLESS_COOKIE_PREFIX={{ paperless_cookie_prefix }}"
- regexp: PAPERLESS_ENABLE_HTTP_REMOTE_USER
line: "PAPERLESS_ENABLE_HTTP_REMOTE_USER={{ paperless_enable_http_remote_user }}"
# OCR settings
- regexp: PAPERLESS_OCR_LANGUAGE
line: "PAPERLESS_OCR_LANGUAGE={{ paperlessng_ocr_languages | join('+') }}"
- regexp: PAPERLESS_OCR_MODE
line: "PAPERLESS_OCR_MODE={{ paperlessng_ocr_mode }}"
- regexp: PAPERLESS_OCR_OUTPUT_TYPE
line: "PAPERLESS_OCR_OUTPUT_TYPE={{ paperlessng_ocr_output_type }}"
- regexp: PAPERLESS_OCR_PAGES
line: "PAPERLESS_OCR_PAGES={{ paperlessng_ocr_pages }}"
- regexp: PAPERLESS_OCR_IMAGE_DPI
line: "PAPERLESS_OCR_IMAGE_DPI={{ paperlessng_ocr_image_dpi }}"
- regexp: PAPERLESS_OCR_USER_ARGS
line: "PAPERLESS_OCR_USER_ARGS={{ paperlessng_ocrmypdf_args | combine({'jbig2_lossy': true} if paperlessng_big2enc_lossy else {}) | to_json }}"
- regexp: PAPERLESS_TIME_ZONE
line: "PAPERLESS_TIME_ZONE={{ paperlessng_time_zone }}"
line: "PAPERLESS_OCR_USER_ARGS={{ paperlessng_ocr_user_args | combine({'jbig2_lossy': true} if paperlessng_big2enc_lossy else {}) | to_json }}"
# Tika settings
- regexp: PAPERLESS_TIKA_ENABLED
line: "PAPERLESS_TIKA_ENABLED={{ paperlessng_tika_enabled }}"
no_log: yes
- name: configure paperless-ng [tika]
lineinfile:
path: "{{ paperlessng_directory }}/paperless.conf.template"
regexp: "^#?{{ item.regexp }}="
line: "'{{ item.line }}' if paperlessng_tika_enabled else '#{{ item.line }}'"
with_items:
- regexp: PAPERLESS_TIKA_ENDPOINT
line: "PAPERLESS_TIKA_ENDPOINT={{ paperlessng_tika_endpoint }}"
- regexp: PAPERLESS_TIKA_GOTENBERG_ENDPOINT
line: "PAPERLESS_TIKA_GOTENBERG_ENDPOINT={{ paperlessng_tika_endpoint }}"
# Software tweaks
- regexp: PAPERLESS_TIME_ZONE
line: "PAPERLESS_TIME_ZONE={{ paperlessng_time_zone }}"
- regexp: PAPERLESS_CONSUMER_POLLING
line: "PAPERLESS_CONSUMER_POLLING={{ paperlessng_consumer_polling }}"
- regexp: PAPERLESS_CONSUMER_DELETE_DUPLICATES
line: "PAPERLESS_CONSUMER_DELETE_DUPLICATES={{ paperlessng_consumer_delete_duplicates }}"
- regexp: PAPERLESS_CONSUMER_RECURSIVE
line: "PAPERLESS_CONSUMER_RECURSIVE={{ paperlessng_consumer_recursive }}"
- regexp: PAPERLESS_CONSUMER_SUBDIRS_AS_TAGS
line: "PAPERLESS_CONSUMER_SUBDIRS_AS_TAGS={{ paperlessng_consumer_subdirs_as_tags }}"
- regexp: PAPERLESS_OPTIMIZE_THUMBNAILS
line: "PAPERLESS_OPTIMIZE_THUMBNAILS={{ paperlessng_optimize_thumbnails }}"
- regexp: PAPERLESS_POST_CONSUME_SCRIPT
line: "PAPERLESS_POST_CONSUME_SCRIPT={{ paperlessng_post_consume_script }}"
- regexp: PAPERLESS_FILENAME_DATE_ORDER
line: "PAPERLESS_FILENAME_DATE_ORDER={{ paperlessng_filename_date_order }}"
- regexp: PAPERLESS_THUMBNAIL_FONT_NAME
line: "PAPERLESS_THUMBNAIL_FONT_NAME={{ paperlessng_thumbnail_font_name }}"
- regexp: PAPERLESS_IGNORE_DATES
line: "PAPERLESS_IGNORE_DATES={{ paperlessng_ignore_dates }}"
no_log: yes
- name: configure paperless-ng database [sqlite]
lineinfile:
@ -228,6 +269,8 @@
line: "PAPERLESS_DBUSER={{ paperlessng_db_user }}"
- regexp: PAPERLESS_DBPASS
line: "PAPERLESS_DBPASS={{ paperlessng_db_pass }}"
- regexp: PAPERLESS_DBSSLMODE
line: "PAPERLESS_DBSSLMODE={{ paperlessng_db_sslmode }}"
when: paperlessng_db_type == 'postgresql'
no_log: yes
@ -258,15 +301,7 @@
extra_args: --upgrade
when: paperlessng_current_version.stdout != paperlessng_version | string
- name: collect static files
become: yes
become_user: "{{ paperlessng_system_user }}"
command: "{{ paperlessng_virtualenv }}/bin/python3 {{ paperlessng_directory }}/src/manage.py collectstatic --no-input"
when: paperlessng_current_version.stdout != paperlessng_version | string
register: static_files
changed_when: static_files.stdout is not match("0 static files copied .*")
- name: create database schema
- name: migrate database schema
become: yes
become_user: "{{ paperlessng_system_user }}"
command: "{{ paperlessng_virtualenv }}/bin/python3 {{ paperlessng_directory }}/src/manage.py migrate"
@ -274,12 +309,6 @@
register: database_schema
changed_when: '"No migrations to apply." not in database_schema.stdout'
- name: compile translations
become: yes
become_user: "{{ paperlessng_system_user }}"
command: "{{ paperlessng_virtualenv }}/bin/python3 {{ paperlessng_directory }}/src/manage.py compilemessages"
when: paperlessng_current_version.stdout != paperlessng_version | string
- name: configure paperless superuser
become: yes
become_user: "{{ paperlessng_system_user }}"

2
docs/configuration.rst
View File

@ -267,7 +267,7 @@ PAPERLESS_OCR_IMAGE_DPI=<num>
present in an image.
PAPERLESS_OCR_USER_ARG=<json>
PAPERLESS_OCR_USER_ARGS=<json>
OCRmyPDF offers many more options. Use this parameter to specify any
additional arguments you wish to pass to OCRmyPDF. Since Paperless uses
the API of OCRmyPDF, you have to specify these in a format that can be

123
docs/setup.rst
View File

@ -83,7 +83,8 @@ You can go multiple routes with setting up and running Paperless:
* :ref:`Pull the image from Docker Hub <setup-docker_hub>`
* :ref:`Build the Docker image yourself <setup-docker_build>`
* :ref:`Install Paperless directly on your system (bare metal) <setup-bare_metal>`
* :ref:`Install Paperless directly on your system manually (bare metal) <setup-bare_metal>`
* :ref:`Use ansible to install Paperless on your system automatically (bare metal) <setup-ansible>`
The Docker routes are quick & easy. These are the recommended routes. This configures all the stuff
from above automatically so that it just works and uses sensible defaults for all configuration options.
@ -92,6 +93,11 @@ The bare metal route is more complicated to setup but makes it easier
should you want to contribute some code back. You need to configure and
run the above mentioned components yourself.
The ansible route cobines benefits from both options:
the setup process is fully automated, reproducible and idempotent,
it includes the same sensible defaults,
and it simultaneously provides the flexibility of a bare metal installation.
.. _setup-docker_hub:
Install Paperless from Docker Hub
@ -392,6 +398,121 @@ writing. Windows is not and will never be supported.
to compile this by yourself, because this software has been patented until around 2017 and
binary packages are not available for most distributions.
.. _setup-ansible:
Install Paperless using ansible
===============================
.. note::
This role currently only supports Debian 10 Buster and Ubuntu 20.04 Focal or later as target hosts.
1. Install ansible 2.7+ on the management node.
This may be the target host paperless-ng is being installed on or any remote host which can access the target host.
For further details, check the ansible `inventory <https://docs.ansible.com/ansible/latest/user_guide/intro_inventory.html>`_ documentation.
On Debian and Ubuntu, the official repositories should provide a suitable version:
.. code:: bash
apt install ansible
ansible --version
Alternatively, you can install the most recent ansible release using PyPI:
.. code:: bash
python3 -m pip install ansible
ansible --version
Make sure your taget hosts are accessible:
.. code:: sh
ansible -m ping YourAnsibleTargetHostGoesHere
2. Clone the repository of paperless-ng:
.. code:: sh
git clone https://github.com/jonaswinkler/paperless-ng
Checkout the latest release tag:
.. code:: sh
cd paperless-ng
git checkout ng-0.9.14
3. Create an ansible ``playbook.yml`` in the paperless-ng root directory:
.. code:: yaml
- hosts: YourAnsibleTargetHostGoesHere
become: yes
vars_files:
- ansible/vars.yml
roles:
- ansible
Optional: If you also want to use PostgreSQL on the target system, install and add (for example) the `geerlingguy.postgresql <https://github.com/geerlingguy/ansible-role-postgresql>`_ role:
.. code:: sh
ansible-galaxy install geerlingguy.postgresql
.. code:: yaml
- hosts: YourAnsibleTargetHostGoesHere
become: yes
vars_files:
- ansible/vars.yml
roles:
- geerlingguy.postgresql
- ansible
Optional: If you also want to use a reverse proxy on the target system, install and add (for example) the `geerlingguy.nginx <https://github.com/geerlingguy/ansible-role-nginx>`_ role:
.. code:: sh
ansible-galaxy install geerlingguy.nginx
.. code:: yaml
- hosts: YourAnsibleTargetHostGoesHere
become: yes
vars_files:
- ansible/vars.yml
roles:
- geerlingguy.postgresql
- ansible
- geerlingguy.nginx
4. Create ``ansible/vars.yml`` to configure your ansible deployment:
.. code:: yaml
paperless_secret_key: PleaseGenerateAStrongKeyForThis
paperlessng_superuser_name: YourUserName
paperlessng_superuser_email: name@domain.tld
paperlessng_superuser_password: YourDesiredPasswordUsedForFirstLogin
paperlessng_ocr_languages:
- eng
- deu
For all of the available options, please check ``ansible/README.md`` and :ref:`configuration`.
Optional configurations for the above-mentioned PostgreSQL and nginx roles would also go here.
5. Run the ansible playbook from the management node:
.. code:: sh
ansible-playbook playbook.yml
When this step completes successfully, paperless-ng will be available on the target host at ``http://127.0.0.1:8000`` (or the address you configured).
Migration to paperless-ng
#########################

2
paperless.conf.example
View File

@ -27,7 +27,7 @@
#PAPERLESS_SECRET_KEY=change-me
#PAPERLESS_ALLOWED_HOSTS=example.com,www.example.com
#PAPERLESS_CORS_ALLOWED_HOSTS=localhost:8080,example.com,localhost:8000
#PAPERLESS_CORS_ALLOWED_HOSTS=http://example.com,http://localhost:8000
#PAPERLESS_FORCE_SCRIPT_NAME=
#PAPERLESS_STATIC_URL=/static/
#PAPERLESS_AUTO_LOGIN_USERNAME=