This file, like `docker-compose.yml`, should be edited by the user. To
avoid merge conflicts when pulling updates, the edited version should
not be committed to the repository.
The export workflow reusing the `/consume` volume is complex and error-
prone, and not at all necessary if the `docker-compose.yml` file has a
volume for `/export` from the beginning.
This commit adds a `Dockerfile` to the root of the project, accompanied
by a `docker-compose.yml.example` for simplified deployment. The
`Dockerfile` is agnostic to whether it will be the webserver, the
consumer, or if it is run for a one-off command (i.e. creation of a
superuser, migration of the database, document export, ...).
The containers entrypoint is the `scripts/docker-entrypoint.sh` script.
This script verifies that the required permissions are set, remaps the
default users and/or groups id if required and installs additional
languages if the user wishes to.
After initialization, it analyzes the command the user supplied:
- If the command starts with a slash, it is expected that the user
wants to execute a binary file and the command will be executed
without further intervention. (Using `exec` to effectively replace
the started shell-script and not have any reaping-issues.)
- If the command does not start with a slash, the command will be
passed directly to the `manage.py` script without further
modification. (Again using `exec`.)
The default command is set to `--help`.
If the user wants to execute a command that is not meant for `manage.py`
but doesn't start with a slash, the Docker `--entrypoint` parameter can
be used to circumvent the mechanics of `docker-entrypoint.sh`.
Further information can be found in `docs/setup.rst` and in
`docs/migrating.rst`.
For additional convenience, a `Dockerfile` has been added to the `docs/`
directory which allows for easy building and serving of the
documentation. This is documented in `docs/requirements.rst`.
Creating the scratch-files in `_get_grayscale` using a random integer is
for one inherently unsafe and can cause a collision. On the other hand,
it should be unnecessary given that the files will be cleaned up after
the OCR run.
Since we don't know if OCR runs might be parallel in the future, this
commit implements thread-safe and deterministic directory-creation.
Additionally it fixes the call to `_cleanup` by `consume`. In the
current implementation `_cleanup` will not be called if the last
consumed document failed with an `OCRError`, this commit fixes this.
To detect the language currently the entire document gets processed. If
a different language has been detected than the default one, the entire
document will be processed again for the new language.
This PR analyzes the middle page for its language and either processes
the remaining pages with the default language if it didn't differ, or
processes all pages for the new guessed language.
The amount of processed pages comes down from the worst case `2n` to
worst case `n+1`.
At the moment, every page in a PDF will be processed one by one using
tesseract. Since the processing of a single page is independent from every
other page, one can make use of multi-core machines.
This PR introduces a multiprocessing pool to process multiple pages
simultaneously. The amount of threads to use can be specified in the
environment variable `PAPERLESS_OCR_THREADS`. This will default to the
number of cores/hyperthreads Python detects for your system.
