Commit Graph

266 Commits

Author SHA1 Message Date
Trenton Holmes
e34b8c4b1f Replace usages of os.rename with shutil.move to properly handle cases where the source and dest arent't on the same filesystem 2023-05-01 07:28:52 -07:00
Trenton H
aabcc9a1c4 Upgrades black to v23, upgrades ruff 2023-04-26 09:35:27 -07:00
Trenton H
30655f1b73 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Trenton H
98811df439 Make the importer a little more robust against some types of errors 2023-04-26 07:08:50 -07:00
Trenton H
d2c02b9102 Configures ruff as the one stop linter and resolves warnings it raised 2023-04-01 17:03:52 -07:00
Trenton H
36a6df0bae Creates a data model for the document consumption, allowing stronger typing of arguments and setting of some information about the file only once 2023-04-01 11:05:34 -07:00
Trenton H
8d0e65d369 Limit the number of threads waiting for files to be ready during polling 2023-03-21 07:46:57 -07:00
shamoon
5108bbc192 Rename comments --> notes 2023-03-18 13:59:17 -07:00
Trenton H
67015a8eef Be sure the scratch directory exists before using it as temporary directory 2023-03-01 07:13:31 -08:00
Trenton H
a9c77216b2 Updates ignore path filtering so files in a folder in an ignored folder will be ignored correctly 2023-02-16 09:05:11 -08:00
Trenton H
342713d6d2 Moves the mktime call into the if block where it is used, preventing exceptions during rare cases 2023-02-02 07:25:32 -08:00
Matthieu Helleboid
23157b8050 replace --use-filename-prefix with --use-folder-prefix 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
b9807c96ae sort exporter option by alphabetical order 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
ec02009c91 use pathlib glob 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
420ec2187d add document comments to dedicated manifest file when using 'split-manifest' 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
343bd7df36 Fix split_manifest default value 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
7f287bb311 fix json serialization bug after migration after to Pathlib) 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
8d34fbc756 fix bug on administration exporter when using -d, --delete option 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
b947f63fd9 add split-manifest option to administration exporter 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
44d8bd3151 add prefix option to administration exporter 2023-01-24 11:06:49 -08:00
Matthieu Helleboid
42d4550d9c add no-archive and no-thumbnail options to administration exporter and importer 2023-01-24 11:06:49 -08:00
Trenton H
873522fd82 Updates the exporter to use pathlib and add a few more tests for coverage 2023-01-14 06:33:12 -08:00
Trenton Holmes
a79be46b81 Adds option to allow a user to export directory to a zipfile 2022-12-04 16:38:25 -08:00
Michael Shamoon
1268daf6fe Celery JSON serializable object for override_tag_ids 2022-11-07 07:41:23 -08:00
Trenton Holmes
09287701ae Transitions the backend to celery and celery beat 2022-09-26 11:25:34 -07:00
Michael Shamoon
24da6ce84e Include storage path in exporter + tests 2022-09-11 07:39:35 -07:00
Quinn Casey
75991f4268 Merge pull request #1451 from paperless-ngx/feature-better-redo-ocr
Feature: Even better re-do of OCR
2022-08-25 17:01:54 -07:00
shamoon
4f6ad5369b Merge pull request #1375 from tim-vogel/add_comments
Feature: document comments
2022-08-25 11:48:31 -07:00
shamoon
08e143547b Merge pull request #1421 from paperless-ngx/feature-consumer-event-driven
Feature: Event driven consumer
2022-08-25 08:16:47 -07:00
Michael Shamoon
680055474e Basic verification of Comment export & exporter comment tuple fix
From oprhaned commits b1855a4b7a
53f21574fd

Co-Authored-By: Trenton Holmes <797416+stumpylog@users.noreply.github.com>
2022-08-24 14:24:10 -07:00
Michael Shamoon
b0ae307add python code style 2022-08-23 19:20:08 -07:00
tim-vogel
278e9c12e1 add comment function 2022-08-23 19:19:21 -07:00
Trenton Holmes
059242b7dc Implements a better re-do of OCR by making the document archiver function common. Actually creates updated file now 2022-08-21 18:20:59 -07:00
Trenton Holmes
b6060ff990 Adds the storage paths to the re-tagger command 2022-08-21 18:01:19 -07:00
Trenton Holmes
f06cb40141 Corrects the timeout's interaction with files waiting to be consumed 2022-08-14 17:47:59 -07:00
Trenton Holmes
9e091e333d Attempts to make production consumer more event driven while still allowing unit testing 2022-08-14 17:47:59 -07:00
Michael Shamoon
9d8ac76264 Squashed commit of the following:
commit a4709b1175
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Jul 28 15:36:13 2022 -0700

    Update stale.yml

    [skip ci]

commit 3a031084f3
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Jul 28 15:24:23 2022 -0700

    Update changelog.md

commit 0c517e5351
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Jul 28 15:18:49 2022 -0700

    v1.8.0 version strings

commit 5fe435048b
Merge: 278cedf3 a722bfd0
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Thu Jul 28 15:17:30 2022 -0700

    Merge pull request #1240 from paperless-ngx/beta

    [Beta] Paperless-ngx v1.8.0 Release Candidate 1

commit a722bfd099
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Thu Jul 28 07:46:12 2022 -0700

    New Crowdin updates (#1291)

    * New translations django.po (French)
    [ci skip]

    * New translations messages.xlf (French)
    [ci skip]

    * New translations django.po (French)
    [ci skip]

    * New translations messages.xlf (French)
    [ci skip]

    * New translations messages.xlf (Turkish)
    [ci skip]

    * New translations django.po (Turkish)
    [ci skip]

commit f3d99a5fdb
Merge: ca334770 79de0989
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Jul 26 11:21:42 2022 -0700

    Merge pull request #1277 from paperless-ngx/fix/redo-ocr-button-on-edit

    Fix/feature: add redo ocr button to document edit view

commit 79de0989d5
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Jul 26 09:54:05 2022 -0700

    fix button icon spacing on mobile

commit ca334770b7
Author: Paperless-ngx Translation Bot [bot] <99855517+paperless-l10n@users.noreply.github.com>
Date:   Tue Jul 26 09:45:21 2022 -0700

    New Crowdin updates (#1242)

    * New translations messages.xlf (Turkish)
    [ci skip]

    * New translations messages.xlf (German)
    [ci skip]

    * New translations django.po (German)
    [ci skip]

    * New translations messages.xlf (Italian)
    [ci skip]

    * New translations messages.xlf (Italian)
    [ci skip]

    * New translations messages.xlf (Finnish)
    [ci skip]

    * New translations messages.xlf (Finnish)
    [ci skip]

commit 1071357505
Merge: f32dfe02 ef790ca6
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Jul 26 09:44:42 2022 -0700

    Merge pull request #1268 from paperless-ngx/bugfix-db-locked

    Bugfix: Adds configuration for database timeout, fixing database locked error

commit f32dfe0278
Merge: 611707a3 4e78ca5d
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Jul 25 12:59:31 2022 -0700

    Merge pull request #1261 from paperless-ngx/fix/b1.8.0-ng-select-dropdowns

    Fix: dropdown selected items not visible again

commit 278cedf3d0
Merge: b141671d ecc4553e
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Jul 25 09:25:52 2022 -0700

    Merge pull request #1272 from paperless-ngx/fix-1263

    Documentation: fix occasional code block color legibility

commit 45a6b5a436
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Jul 24 20:15:26 2022 -0700

    Add redo OCR button to document edit

commit 611707a3d1
Merge: 2d88638d b4d20d9b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Jul 24 19:59:30 2022 -0700

    Merge pull request #1276 from paperless-ngx/bugfix-webp-import

    Bugfix: Document import doesn't convert thumbnails to WebP

commit b4d20d9b9a
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Sun Jul 24 10:22:53 2022 -0700

    Fixes document import copying PNG files to .webp extensions without actual conversion

commit ecc4553e67
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Jul 22 15:10:33 2022 -0700

    fix occasional code block color legibility

commit ef790ca6f4
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Fri Jul 22 11:08:52 2022 -0700

    Fixes the copy and paste of the log line

commit 2d88638da7
Merge: 0e2e5f34 91ba0bd0
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Jul 22 10:45:53 2022 -0700

    Merge pull request #1269 from paperless-ngx/beta-deps-final

    Chore: Locks dependencies to the final versions for the beta

commit 91ba0bd0af
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Fri Jul 22 08:53:02 2022 -0700

    Locks dependencies to the final versions for the beta

commit 0e2e5f3413
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Tue Jul 19 13:57:00 2022 -0700

    Creates utiliy to ensure all paths in settings are normalized and absolute

commit 7a99dcf693
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Thu Jul 21 08:02:11 2022 -0700

    Adds configuration for database timeout, documentation and troubleshotting suggestion

commit 4e78ca5d82
Author: Michael Shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Jul 20 11:15:35 2022 -0700

    remove merge error ng-select css

commit 83de38e56f
Merge: f4be2e4f b1b6d50a
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Wed Jul 20 08:26:12 2022 -0700

    Merge pull request #1247 from paperless-ngx/bugfix-pikepdf-ocrmypdf-warnings

    Bugfix: Adds pngquant and jbig2dec to Docker image

commit f4be2e4fe7
Merge: 4444925d 16b0f7f9
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Tue Jul 19 21:03:16 2022 -0700

    Merge pull request #1259 from paperless-ngx/chore-add-ci-hadolint

    Chore: Add Hadolint job to CI

commit 16b0f7f9ee
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Tue Jul 19 14:18:47 2022 -0700

    Removes a Dockerfile I can't find referenced anywhere

commit 27721aef71
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Tue Jul 19 14:01:47 2022 -0700

    Fixes and updates the Hadolint action version

commit 329a317fdf
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Tue Jul 19 13:54:33 2022 -0700

    Configure Hadolint in a single location for both hooks and CI

commit daad634894
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Fri Jul 15 13:45:23 2022 -0700

    Adds a CI job for hadolint over all the Dockerfiles, fixes the minor thing it complained about

commit 4444925dea
Merge: 4c697ab5 9c1ae96d
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Mon Jul 18 15:55:29 2022 -0700

    Merge pull request #1249 from paperless-ngx/fix-generated-changelog

    [CI] Fix automatic changelog generation on release

commit 9c1ae96d33
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Mon Jul 18 09:48:03 2022 -0700

    Create PR for changelog instead of direct commit

commit b1b6d50af6
Author: Trenton Holmes <holmes.trenton@gmail.com>
Date:   Mon Jul 18 09:46:31 2022 -0700

    Adds a couple packages to the Docker image for ocrmypdf and pikepdf

commit 4c697ab50e
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Sun Jul 17 15:23:28 2022 -0700

    Bump version to beta

commit b141671d90
Merge: 48dfbbeb 2ab2d912
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Sun Jul 17 13:18:57 2022 -0700

    Merge pull request #1237 from tooomm/patch-1

    chore: Run stale bot only on certain labels

commit 2ab2d9127d
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Fri Jul 15 20:19:28 2022 -0700

    Use cant-reproduce for stale

commit 278453451e
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Fri Jul 15 21:18:38 2022 +0200

    only run on certain labels

commit 48dfbbebc6
Merge: 8efb97ef e568b300
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sun Jul 3 07:42:20 2022 -0700

    Merge pull request #1110 from paperless-ngx/update-issue-form

commit 8efb97ef4e
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Jul 2 19:06:32 2022 -0700

    Update stale.yml

    [ci skip]

commit d8cda7fc1b
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Sat Jul 2 17:51:39 2022 -0700

    Use any-of-labels for stalebot

    [ci skip]

commit 68f0cf419b
Merge: 666b9385 26b12512
Author: Felix E <felix@eckhofer.com>
Date:   Mon Jun 20 14:25:59 2022 +0200

    Merge pull request #1148 from pReya/patch-1

    fix: update scanner capability

commit 26b12512b1
Author: Moritz Stückler <moritz.stueckler@gmail.com>
Date:   Mon Jun 20 12:06:54 2022 +0200

    fix: update scanner capability

    The Brother ADS-A1700W does indeed support SFTP. I've just bought it, and set it up like this.

commit e568b3000e
Author: shamoon <4887959+shamoon@users.noreply.github.com>
Date:   Tue Jun 7 15:28:49 2022 -0700

    Add lsio to issue form

commit 666b938550
Merge: de5eaf1c 163231d3
Author: Quinn Casey <quinn@quinncasey.com>
Date:   Thu May 19 17:23:23 2022 -0700

    Merge pull request #990 from tooomm/patch-2

    Docs: Fix headings and add links to PRs in changelog

commit 163231d307
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Thu May 19 23:12:40 2022 +0200

    Link issues, capitalization and minor fixes

commit e530750fc6
Author: tooomm <tooomm@users.noreply.github.com>
Date:   Thu May 19 22:05:43 2022 +0200

    update heading levels for v1.7.0
2022-07-28 15:36:24 -07:00
Trenton Holmes
5d2062a076 Includes the actual OSError string in the log, instead of assuming it's a busy file 2022-07-11 11:57:02 -07:00
shamoon
720e89ec9a Merge branch 'dev' into feature-redo-ocr 2022-07-02 08:41:18 -07:00
Michael Shamoon
225dd4a065 Extract redo ocr to task 2022-07-02 08:39:09 -07:00
Trenton Holmes
11cb7a29d8 Updates following testing of command 2022-07-02 08:39:09 -07:00
Trenton Holmes
44cb70625f Includes the progress bar 2022-07-02 08:39:09 -07:00
Trenton Holmes
823e8e73e1 Saves work on a new management comment to re-ocr a file 2022-07-02 08:39:09 -07:00
Trenton Holmes
39ee284fcd Adds a few more test cases for coverage and adds no coverage to some others 2022-07-02 16:19:22 +02:00
Trenton Holmes
ee4e3cebe2 Removes last vestiges of PNG from the tests, code, docs and samples 2022-06-11 14:20:50 -07:00
Trenton Holmes
762d28cc47 Converts the conversion into a database migration 2022-06-11 13:04:21 -07:00
Trenton Holmes
ef6ebf9888 Entirely removes the optipng, updates ghostscript fall back to also use WebP. Updates the conversion to use a multiprocessing pool 2022-06-11 08:38:49 -07:00
Trenton Holmes
ec4630a846 Fixes the re-generation of thumbnails making webp file, but named as PNG files 2022-06-10 13:17:41 -07:00
Trenton Holmes
ce2958531a Corrects the functionality of the webp conversion script 2022-06-10 08:56:25 -07:00
Trenton Holmes
799d80272e Adds untested manual conversion command 2022-06-10 07:17:15 -07:00