Bump the frontend-jest-dependencies group in /src-ui with 4 updates (#4112)

* Bump the frontend-jest-dependencies group in /src-ui with 4 updates

Bumps the frontend-jest-dependencies group in /src-ui with 4 updates: [jest](https://github.com/jestjs/jest/tree/HEAD/packages/jest), [@types/jest](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/jest), [jest-environment-jsdom](https://github.com/jestjs/jest/tree/HEAD/packages/jest-environment-jsdom) and [jest-websocket-mock](https://github.com/romgain/jest-websocket-mock).


Updates `jest` from 29.6.2 to 29.6.4
- [Release notes](https://github.com/jestjs/jest/releases)
- [Changelog](https://github.com/jestjs/jest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/jestjs/jest/commits/v29.6.4/packages/jest)

Updates `@types/jest` from 29.5.3 to 29.5.4
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/jest)

Updates `jest-environment-jsdom` from 29.6.2 to 29.6.4
- [Release notes](https://github.com/jestjs/jest/releases)
- [Changelog](https://github.com/jestjs/jest/blob/main/CHANGELOG.md)
- [Commits](https://github.com/jestjs/jest/commits/v29.6.4/packages/jest-environment-jsdom)

Updates `jest-websocket-mock` from 2.4.0 to 2.4.1
- [Release notes](https://github.com/romgain/jest-websocket-mock/releases)
- [Commits](https://github.com/romgain/jest-websocket-mock/compare/v2.4.0...v2.4.1)

---
updated-dependencies:
- dependency-name: jest
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-jest-dependencies
- dependency-name: "@types/jest"
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-jest-dependencies
- dependency-name: jest-environment-jsdom
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-jest-dependencies
- dependency-name: jest-websocket-mock
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: frontend-jest-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update toasts.component.spec.ts

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>

.github/dependabot.yml

@@ -27,7 +27,7 @@ updates:
       frontend-jest-dependencies:
         patterns:
           - "@types/jest"
-          - "jest"
+          - "jest*"
       frontend-eslint-dependencies:
         patterns:
           - "@typescript-eslint*"

docs/changelog.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## paperless-ngx 1.17.3
+
+### Bug Fixes
+
+- Fix: When PDF/A rendering fails, add a consideration for the user to add args to override [@stumpylog](https://github.com/stumpylog) ([#4083](https://github.com/paperless-ngx/paperless-ngx/pull/4083))
+
+### Dependencies
+
+- Chore: update frontend PDF viewer (including pdf-js) [@shamoon](https://github.com/shamoon) ([#4065](https://github.com/paperless-ngx/paperless-ngx/pull/4065))
+
+### Maintenance
+
+- Dev: Upload code coverage in the same job [@stumpylog](https://github.com/stumpylog) ([#4084](https://github.com/paperless-ngx/paperless-ngx/pull/4084))
+
+### All App Changes
+
+<details>
+<summary>3 changes</summary>
+
+- Fix: When PDF/A rendering fails, add a consideration for the user to add args to override [@stumpylog](https://github.com/stumpylog) ([#4083](https://github.com/paperless-ngx/paperless-ngx/pull/4083))
+- Chore: update frontend PDF viewer (including pdf-js) [@shamoon](https://github.com/shamoon) ([#4065](https://github.com/paperless-ngx/paperless-ngx/pull/4065))
+- Chore: Prepare for Python 3.11 support [@stumpylog](https://github.com/stumpylog) ([#4066](https://github.com/paperless-ngx/paperless-ngx/pull/4066))
+</details>
+
 ## paperless-ngx 1.17.2
 
 ### Features

docs/troubleshooting.md

@@ -345,3 +345,11 @@ STATEMENT:  INSERT INTO "documents_tag" ("owner_id", "name", "match", "matching_
 
 This can happen during heavy consumption when using polling. Paperless will handle it correctly and the file
 will still be consumed
+
+## Consumption fails with "Ghostscript PDF/A rendering failed"
+
+Newer versions of OCRmyPDF will fail if it encounters errors during processing.
+This is intentional as the output archive file may differ in unexpected or undesired
+ways from the original. As the logs indicate, if you encounter this error you can set
+`PAPERLESS_OCR_USER_ARGS: '{"continue_on_soft_render_error": true}'` to try to 'force'
+processing documents with this issue.

src-ui/package.json

@@ -10,15 +10,15 @@
   },
   "private": true,
   "dependencies": {
-    "@angular/common": "~16.1.7",
+    "@angular/common": "~16.2.3",
-    "@angular/compiler": "~16.1.7",
+    "@angular/compiler": "~16.2.3",
-    "@angular/core": "~16.1.7",
+    "@angular/core": "~16.2.3",
-    "@angular/forms": "~16.1.7",
+    "@angular/forms": "~16.2.3",
-    "@angular/localize": "~16.1.7",
+    "@angular/localize": "~16.2.3",
-    "@angular/platform-browser": "~16.1.7",
+    "@angular/platform-browser": "~16.2.3",
-    "@angular/platform-browser-dynamic": "~16.1.7",
+    "@angular/platform-browser-dynamic": "~16.2.3",
-    "@angular/router": "~16.1.7",
+    "@angular/router": "~16.2.3",
-    "@ng-bootstrap/ng-bootstrap": "^15.1.0",
+    "@ng-bootstrap/ng-bootstrap": "^15.1.1",
     "@ng-select/ng-select": "^11.1.1",
     "@ngneat/dirty-check-forms": "^3.0.3",
     "@popperjs/core": "^2.11.8",
@@ -27,35 +27,35 @@
     "mime-names": "^1.0.0",
     "ng2-pdf-viewer": "^10.0.0",
     "ngx-color": "^9.0.0",
-    "ngx-cookie-service": "^16.0.0",
+    "ngx-cookie-service": "^16.0.1",
     "ngx-file-drop": "^16.0.0",
     "ngx-ui-tour-ng-bootstrap": "^13.0.3",
     "rxjs": "^7.8.1",
-    "tslib": "^2.6.1",
+    "tslib": "^2.6.2",
     "uuid": "^9.0.0",
     "zone.js": "^0.13.0"
   },
   "devDependencies": {
-    "@angular-builders/jest": "16.0.0",
+    "@angular-builders/jest": "16.0.1",
-    "@angular-devkit/build-angular": "~16.1.6",
+    "@angular-devkit/build-angular": "~16.2.1",
-    "@angular-eslint/builder": "16.1.0",
+    "@angular-eslint/builder": "16.1.1",
-    "@angular-eslint/eslint-plugin": "16.1.0",
+    "@angular-eslint/eslint-plugin": "16.1.1",
-    "@angular-eslint/eslint-plugin-template": "16.1.0",
+    "@angular-eslint/eslint-plugin-template": "16.1.1",
-    "@angular-eslint/schematics": "16.1.0",
+    "@angular-eslint/schematics": "16.1.1",
-    "@angular-eslint/template-parser": "16.1.0",
+    "@angular-eslint/template-parser": "16.1.1",
-    "@angular/cli": "~16.1.6",
+    "@angular/cli": "~16.2.1",
-    "@angular/compiler-cli": "~16.1.3",
+    "@angular/compiler-cli": "~16.2.3",
     "@playwright/test": "^1.36.2",
-    "@types/jest": "^29.5.3",
+    "@types/jest": "^29.5.4",
-    "@types/node": "^20.4.5",
+    "@types/node": "^20.5.8",
-    "@typescript-eslint/eslint-plugin": "^6.2.1",
+    "@typescript-eslint/eslint-plugin": "^6.5.0",
-    "@typescript-eslint/parser": "^6.2.1",
+    "@typescript-eslint/parser": "^6.5.0",
-    "concurrently": "^8.1.0",
+    "concurrently": "^8.2.1",
-    "eslint": "^8.46.0",
+    "eslint": "^8.48.0",
-    "jest": "29.6.2",
+    "jest": "29.6.4",
-    "jest-environment-jsdom": "^29.6.2",
+    "jest-environment-jsdom": "^29.6.4",
     "jest-preset-angular": "^13.1.1",
-    "jest-websocket-mock": "^2.4.0",
+    "jest-websocket-mock": "^2.4.1",
     "ts-node": "~10.9.1",
     "typescript": "^5.1.6",
     "wait-on": "^7.0.1"

src-ui/src/app/components/common/toasts/toasts.component.spec.ts

@@ -52,7 +52,7 @@ describe('ToastsComponent', () => {
   })
 
   it('should call getToasts and return toasts', fakeAsync(() => {
-    const spy = jest.spyOn(toastService, 'getToasts').mockReset()
+    const spy = jest.spyOn(toastService, 'getToasts')
 
     component.ngOnInit()
     fixture.detectChanges()

src-ui/src/app/components/manage/tasks/tasks.component.html

@@ -53,7 +53,7 @@
             <label class="form-check-label" for="task{{task.id}}"></label>
           </div>
         </th>
-        <td class="overflow-auto">{{ task.task_file_name }}</td>
+        <td class="overflow-auto name-col">{{ task.task_file_name }}</td>
         <td class="d-none d-lg-table-cell">{{ task.date_created | customDate:'short' }}</td>
         <td class="d-none d-lg-table-cell" *ngIf="activeTab !== 'started' && activeTab !== 'queued'">
           <div *ngIf="task.result?.length > 50" class="result" (click)="expandTask(task); $event.stopPropagation();"

src-ui/src/app/components/manage/tasks/tasks.component.scss

@@ -20,3 +20,9 @@ pre {
     width: 0.8rem;
     height: 0.8rem;
 }
+
+@media (max-width: 575.98px) {
+    .name-col {
+        max-width: 150px;
+    }
+}

src-ui/src/environments/environment.prod.ts

@@ -5,7 +5,7 @@ export const environment = {
   apiBaseUrl: document.baseURI + 'api/',
   apiVersion: '3',
   appTitle: 'Paperless-ngx',
-  version: '1.17.3',
+  version: '1.17.4-dev',
   webSocketHost: window.location.host,
   webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
   webSocketBaseUrl: base_url.pathname + 'ws/',

src-ui/src/locale/messages.el_GR.xlf

@@ -4962,7 +4962,7 @@
           <context context-type="sourcefile">src/app/components/manage/settings/settings.component.ts</context>
           <context context-type="linenumber">278</context>
         </context-group>
-        <target state="needs-translation">Error retrieving groups</target>
+        <target state="translated">Σφάλμα κατά την ανάκτηση ομάδων</target>
       </trans-unit>
       <trans-unit id="1235706724900303689" datatype="html">
         <source>Error retrieving users</source>
@@ -4970,7 +4970,7 @@
           <context context-type="sourcefile">src/app/components/manage/settings/settings.component.ts</context>
           <context context-type="linenumber">287</context>
         </context-group>
-        <target state="needs-translation">Error retrieving users</target>
+        <target state="translated">Σφάλμα στην ανάκτηση χρηστών</target>
       </trans-unit>
       <trans-unit id="5241231471117657636" datatype="html">
         <source>Error retrieving mail rules</source>
@@ -4978,7 +4978,7 @@
           <context context-type="sourcefile">src/app/components/manage/settings/settings.component.ts</context>
           <context context-type="linenumber">314</context>
         </context-group>
-        <target state="needs-translation">Error retrieving mail rules</target>
+        <target state="translated">Σφάλμα κατά την ανάκτηση κανόνων αλληλογραφίας</target>
       </trans-unit>
       <trans-unit id="3178554336792037159" datatype="html">
         <source>Error retrieving mail accounts</source>
@@ -4986,7 +4986,7 @@
           <context context-type="sourcefile">src/app/components/manage/settings/settings.component.ts</context>
           <context context-type="linenumber">323</context>
         </context-group>
-        <target state="needs-translation">Error retrieving mail accounts</target>
+        <target state="translated">Σφάλμα στην ανάκτηση λογαριασμών αλληλογραφίας</target>
       </trans-unit>
       <trans-unit id="5610279464668232148" datatype="html">
         <source>Saved view &quot;<x id="PH" equiv-text="savedView.name"/>&quot; deleted.</source>

src/documents/consumer.py

@@ -450,11 +450,18 @@ class Consumer(LoggingMixin):
             archive_path = document_parser.get_archive_path()
 
         except ParseError as e:
+            self._fail(
+                str(e),
+                f"Error occurred while consuming document {self.filename}: {e}",
+                exc_info=True,
+                exception=e,
+            )
+        except Exception as e:
             document_parser.cleanup()
             tempdir.cleanup()
             self._fail(
                 str(e),
-                f"Error while consuming document {self.filename}: {e}",
+                f"Unexpected error while consuming document {self.filename}: {e}",
                 exc_info=True,
                 exception=e,
             )
@@ -544,8 +551,8 @@ class Consumer(LoggingMixin):
         except Exception as e:
             self._fail(
                 str(e),
-                f"The following error occurred while consuming "
+                f"The following error occurred while storing document "
-                f"{self.filename}: {e}",
+                f"{self.filename} after parsing: {e}",
                 exc_info=True,
                 exception=e,
             )

src/documents/tests/test_consumer.py

@@ -211,6 +211,18 @@ class FaultyParser(DocumentParser):
         raise ParseError("Does not compute.")
 
 
+class FaultyGenericExceptionParser(DocumentParser):
+    def __init__(self, logging_group, scratch_dir):
+        super().__init__(logging_group)
+        _, self.fake_thumb = tempfile.mkstemp(suffix=".webp", dir=scratch_dir)
+
+    def get_thumbnail(self, document_path, mime_type, file_name=None):
+        return self.fake_thumb
+
+    def parse(self, document_path, mime_type, file_name=None):
+        raise Exception("Generic exception.")
+
+
 def fake_magic_from_file(file, mime=False):
     if mime:
         if os.path.splitext(file)[1] == ".pdf":
@@ -260,6 +272,13 @@ class TestConsumer(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
     def make_faulty_parser(self, logging_group, progress_callback=None):
         return FaultyParser(logging_group, self.dirs.scratch_dir)
 
+    def make_faulty_generic_exception_parser(
+        self,
+        logging_group,
+        progress_callback=None,
+    ):
+        return FaultyGenericExceptionParser(logging_group, self.dirs.scratch_dir)
+
     def setUp(self):
         super().setUp()
 
@@ -496,7 +515,29 @@ class TestConsumer(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
 
         self.assertRaisesMessage(
             ConsumerError,
-            "sample.pdf: Error while consuming document sample.pdf: Does not compute.",
+            "sample.pdf: Error occurred while consuming document sample.pdf: Does not compute.",
+            self.consumer.try_consume_file,
+            self.get_test_file(),
+        )
+
+        self._assert_first_last_send_progress(last_status="FAILED")
+
+    @mock.patch("documents.parsers.document_consumer_declaration.send")
+    def testGenericParserException(self, m):
+        m.return_value = [
+            (
+                None,
+                {
+                    "parser": self.make_faulty_generic_exception_parser,
+                    "mime_types": {"application/pdf": ".pdf"},
+                    "weight": 0,
+                },
+            ),
+        ]
+
+        self.assertRaisesMessage(
+            ConsumerError,
+            "sample.pdf: Unexpected error while consuming document sample.pdf: Generic exception.",
             self.consumer.try_consume_file,
             self.get_test_file(),
         )
@@ -510,7 +551,7 @@ class TestConsumer(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
 
         self.assertRaisesMessage(
             ConsumerError,
-            "sample.pdf: The following error occurred while consuming sample.pdf: NO.",
+            "sample.pdf: The following error occurred while storing document sample.pdf after parsing: NO.",
             self.consumer.try_consume_file,
             filename,
         )

src/paperless/version.py

@@ -1,7 +1,7 @@
 from typing import Final
 from typing import Tuple
 
-__version__: Final[Tuple[int, int, int]] = (1, 17, 3)
+__version__: Final[Tuple[int, int, int]] = (1, 17, 4)
 # Version string like X.Y.Z
 __full_version_str__: Final[str] = ".".join(map(str, __version__))
 # Version string like X.Y

src/paperless_tesseract/parsers.py

@@ -340,7 +340,10 @@ class RasterisedDocumentParser(DocumentParser):
                     "Ghostscript PDF/A rendering failed, consider setting "
                     "PAPERLESS_OCR_USER_ARGS: '{\"continue_on_soft_render_error\": true}'",  # noqa: E501
                 )
-            raise e
+
+            raise ParseError(
+                f"SubprocessOutputError: {e!s}. See logs for more information.",
+            ) from e
         except (NoTextFoundException, InputFileError) as e:
             self.log.warning(
                 f"Encountered an error while running OCR: {e!s}. "

src/paperless_tesseract/tests/test_parser.py

@@ -8,6 +8,7 @@ from unittest import mock
 
 from django.test import TestCase
 from django.test import override_settings
+from ocrmypdf import SubprocessOutputError
 
 from documents.parsers import ParseError
 from documents.parsers import run_convert
@@ -827,6 +828,18 @@ class TestParser(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
         # Copied from the PDF to here.  Don't even look at it
         self.assertIn("ةﯾﻠﺧﺎدﻻ ةرازو", parser.get_text())
 
+    @mock.patch("ocrmypdf.ocr")
+    def test_gs_rendering_error(self, m):
+        m.side_effect = SubprocessOutputError("Ghostscript PDF/A rendering failed")
+        parser = RasterisedDocumentParser(None)
+
+        self.assertRaises(
+            ParseError,
+            parser.parse,
+            os.path.join(self.SAMPLE_FILES, "simple-digital.pdf"),
+            "application/pdf",
+        )
+
 
 class TestParserFileTypes(DirectoriesMixin, FileSystemAssertsMixin, TestCase):
     SAMPLE_FILES = os.path.join(os.path.dirname(__file__), "samples")