Feature: Allow setting backend configuration settings via the UI (#5126)

* Saving some start on this

* At least partially working for the tesseract parser

* Problems with migration testing need to figure out

* Work around that error

* Fixes max m_pixels

* Moving the settings to main paperless application

* Starting some consumer options

* More fixes and work

* Fixes these last tests

* Fix max_length on OcrSettings.mode field

* Fix all fields on Common & Ocr settings serializers

* Umbrellla config view

* Revert "Umbrellla config view"

This reverts commit fbaf9f4be30f89afeb509099180158a3406416a5.

* Updates to use a single configuration object for all settings

* Squashed commit of the following:

commit 8a0a49dd57
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 23:02:47 2023 -0800

    Fix formatting

commit 66b2d90c50
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 22:36:35 2023 -0800

    Refactor frontend data models

commit 5723bd8dd8
Author: Adam Bogdał <adam@bogdal.pl>
Date:   Wed Dec 20 01:17:43 2023 +0100

    Fix: speed up admin panel for installs with a large number of documents (#5052)

commit 9b08ce1761
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:18:51 2023 -0800

    Update PULL_REQUEST_TEMPLATE.md

commit a6248bec2d
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 15:02:05 2023 -0800

    Chore: Update Angular to v17 (#4980)

commit b1f6f52486
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:53:56 2023 -0800

    Fix: Dont allow null custom_fields property via API (#5063)

commit 638d9970fd
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 13:43:50 2023 -0800

    Enhancement: symmetric document links (#4907)

commit 5e8de4c1da
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Dec 19 12:45:04 2023 -0800

    Enhancement: shared icon & shared by me filter (#4859)

commit 088bad9030
Author: Trenton H <797416+stumpylog@users.noreply.github.com>
Date:   Tue Dec 19 12:04:03 2023 -0800

    Bulk updates all the backend libraries (#5061)

* Saving some work on frontend config

* Very basic but dynamically-generated config form

* Saving work on slightly less ugly frontend config

* JSON validation for user_args field

* Fully dynamic config form

* Adds in some additional validators for a nicer error message

* Cleaning up the testing and coverage more

* Reverts unintentional change

* Adds documentation about the settings and the precedence

* Couple more commenting and style fixes

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>

src/paperless/config.py

@@ -0,0 +1,88 @@
+import dataclasses
+import json
+from typing import Optional
+
+from django.conf import settings
+
+from paperless.models import ApplicationConfiguration
+
+
+@dataclasses.dataclass
+class OutputTypeConfig:
+    """
+    Almost all parsers care about the chosen PDF output format
+    """
+
+    output_type: str = dataclasses.field(init=False)
+
+    @staticmethod
+    def _get_config_instance() -> ApplicationConfiguration:
+        app_config = ApplicationConfiguration.objects.all().first()
+        # Workaround for a test where the migration hasn't run to create the single model
+        if app_config is None:
+            ApplicationConfiguration.objects.create()
+            app_config = ApplicationConfiguration.objects.all().first()
+        return app_config
+
+    def __post_init__(self) -> None:
+        app_config = self._get_config_instance()
+
+        self.output_type = app_config.output_type or settings.OCR_OUTPUT_TYPE
+
+
+@dataclasses.dataclass
+class OcrConfig(OutputTypeConfig):
+    """
+    Specific settings for the Tesseract based parser.  Options generally
+    correspond almost directly to the OCRMyPDF options
+    """
+
+    pages: Optional[int] = dataclasses.field(init=False)
+    language: str = dataclasses.field(init=False)
+    mode: str = dataclasses.field(init=False)
+    skip_archive_file: str = dataclasses.field(init=False)
+    image_dpi: Optional[int] = dataclasses.field(init=False)
+    clean: str = dataclasses.field(init=False)
+    deskew: bool = dataclasses.field(init=False)
+    rotate: bool = dataclasses.field(init=False)
+    rotate_threshold: float = dataclasses.field(init=False)
+    max_image_pixel: Optional[float] = dataclasses.field(init=False)
+    color_conversion_strategy: str = dataclasses.field(init=False)
+    user_args: Optional[dict[str, str]] = dataclasses.field(init=False)
+
+    def __post_init__(self) -> None:
+        super().__post_init__()
+
+        app_config = self._get_config_instance()
+
+        self.pages = app_config.pages or settings.OCR_PAGES
+        self.language = app_config.language or settings.OCR_LANGUAGE
+        self.mode = app_config.mode or settings.OCR_MODE
+        self.skip_archive_file = (
+            app_config.skip_archive_file or settings.OCR_SKIP_ARCHIVE_FILE
+        )
+        self.image_dpi = app_config.image_dpi or settings.OCR_IMAGE_DPI
+        self.clean = app_config.unpaper_clean or settings.OCR_CLEAN
+        self.deskew = app_config.deskew or settings.OCR_DESKEW
+        self.rotate = app_config.rotate_pages or settings.OCR_ROTATE_PAGES
+        self.rotate_threshold = (
+            app_config.rotate_pages_threshold or settings.OCR_ROTATE_PAGES_THRESHOLD
+        )
+        self.max_image_pixel = (
+            app_config.max_image_pixels or settings.OCR_MAX_IMAGE_PIXELS
+        )
+        self.color_conversion_strategy = (
+            app_config.color_conversion_strategy
+            or settings.OCR_COLOR_CONVERSION_STRATEGY
+        )
+
+        user_args = None
+        if app_config.user_args:
+            user_args = app_config.user_args
+        elif settings.OCR_USER_ARGS is not None:  # pragma: no cover
+            try:
+                user_args = json.loads(settings.OCR_USER_ARGS)
+            except json.JSONDecodeError:
+                user_args = {}
+
+        self.user_args = user_args