diff --git a/docs/changelog.rst b/docs/changelog.rst index c4443504f..d6ad73ce2 100644 --- a/docs/changelog.rst +++ b/docs/changelog.rst @@ -5,6 +5,16 @@ Changelog ********* +paperless-ng 0.9.4 +################## + +* Front end: Clickable tags, correspondents and types allow quick filtering for related documents. +* Front end: Saved views are now editable. +* Front end: Preview documents directly in the browser. +* Fixes: + * A severe error when trying to use post consume scripts. +* The documentation now contains information about bare metal installs. + paperless-ng 0.9.3 ################## @@ -20,7 +30,7 @@ paperless-ng 0.9.3 aware of. * Issue with the automatic classifier not working with only one tag. * A couple issues with the search index being opened to eagerly. - + * Added lots of tests for various parts of the application. paperless-ng 0.9.2 diff --git a/docs/setup.rst b/docs/setup.rst index 4e2826dd6..9853ce2f1 100644 --- a/docs/setup.rst +++ b/docs/setup.rst @@ -208,9 +208,147 @@ Docker Route Bare Metal Route ================ -.. warning:: +Paperless runs on linux only. The following procedure has been tested on a minimal +installation of Debian/Buster, which is the current stable release at the time of +writing. Windows is not and will never be supported. - TBD. User docker for now. +1. Install dependencies. Paperless requires the following packages. + + * ``python3`` 3.6, 3.7, 3.8 (3.9 is untested). + * ``python3-pip``, optionally ``pipenv`` for package installation + * ``python3-dev`` + + * ``imagemagick`` >= 6 for PDF conversion + * ``unpaper`` for cleaning documents before OCR + * ``ghostscript`` + * ``optipng`` for optimising thumbnails + * ``tesseract-ocr`` >= 4.0.0 for OCR + * ``tesseract-ocr`` language packs (``tesseract-ocr-eng``, ``tesseract-ocr-deu``, etc) + * ``gnupg`` for handling encrypted documents + * ``libpoppler-cpp-dev`` for PDF to text conversion + * ``libmagic-dev`` for mime type detection + * ``libpq-dev`` for PostgreSQL + + You will also need ``build-essential``, ``python3-setuptools`` and ``python3-wheel`` + for installing some of the python dependencies. You can remove that + again after installation. + +2. Install ``redis`` >= 5.0 and configure it to start automatically. + +3. Optional. Install ``postgresql`` and configure a database, user and password for paperless. If you do not wish + to use PostgreSQL, SQLite is avialable as well. + +4. Get the release archive. If you pull the git repo as it is, you also have to compile the front end by yourself. + Extract the frontend to a place from where you wish to execute it, such as ``/opt/paperless``. + +5. Configure paperless. See :ref:`configuration` for details. Edit the included ``paperless.conf`` and adjust the + settings to your needs. Required settings for getting paperless running are: + + * ``PAPERLESS_REDIS`` should point to your redis server, such as redis://localhost:6379. + * ``PAPERLESS_DBHOST`` should be the hostname on which your PostgreSQL server is running. Do not configure this + to use SQLite instead. Also configure port, database name, user and password as necessary. + * ``PAPERLESS_CONSUMPTION_DIR`` should point to a folder which paperless should watch for documents. You might + want to have this somewhere else. Likewise, ``PAPERLESS_DATA_DIR`` and ``PAPERLESS_MEDIA_ROOT`` define where + paperless stores its data. If you like, you can point both to the same directory. + * ``PAPERLESS_SECRET_KEY`` should be a random sequence of characters. It's used for authentication. Failure + to do so allows third parties to forge authentication credentials. + + Many more adjustments can be made to paperless, especially the OCR part. The following options are recommended + for everyone: + + * Set ``PAPERLESS_OCR_LANGUAGE`` to the language most of your documents are written in. + * Set ``PAPERLESS_TIME_ZONE`` to your local time zone. + +6. Setup permissions. Create a system users under which you wish to run paperless. Ensure that these directories exist + and that the user has write permissions to the following directories + + * ``/opt/paperless/media`` + * ``/opt/paperless/data`` + * ``/opt/paperless/consume`` + + Adjust as necessary if you configured different folders. + +7. Install python requirements. Paperless comes with both Pipfiles for ``pipenv`` as well as with a ``requirements.txt``. + Both will install exactly the same requirements. It is up to you if you wish to use a virtual environment or not. + +8. Go to ``/opt/paperless/src``, and execute the following commands: + + .. code:: bash + + # This collects static files from paperless and django. + python3 manage.py collectstatic --clear --no-input + + # This creates the database schema. + python3 manage.py migrate + + # This creates your first paperless user + python3 manage.py createsuperuser + +9. Optional: Test that paperless is working by executing + + .. code:: bash + + # This collects static files from paperless and django. + python3 manage.py runserver + + and pointing your browser to http://localhost:8000/. + + .. warning:: + + This is a development server which should not be used in + production. + + .. hint:: + + This will not start the consumer. Paperless does this in a + separate process. + +10. Setup systemd services to run paperless automatically. You may + use the service definition files included in the ``scripts`` folder + as a starting point. + + Paperless needs the ``webserver`` script to run the webserver, the + ``consumer`` script to watch the input folder, and the ``scheduler`` + script to run tasks such as email checking and document consumption. + + These services rely on redis and optionally the database server, but + don't need to be started in any particular order. The example files + depend on redis being started. If you use a database server, you should + add additinal dependencies. + + .. hint:: + + You may optionally set up your preferred web server to serve + paperless as a wsgi application directly instead of running the + ``webserver`` service. The module containing the wsgi application + is named ``paperless.wsgi``. + + .. caution:: + + The included scripts run a ``gunicorn`` standalone server, + which is fine for running paperless. It does support SSL, + however, the documentation of GUnicorn states that you should + use a proxy server in front of gunicorn instead. + +11. Optional: Install a samba server and make the consumption folder + available as a network share. + +12. Configure ImageMagick to allow processing of PDF documents. Most distributions have + this disabled by default, since PDF documents can contain malware. If + you don't do this, paperless will fall back to ghostscript for certain steps + such as thumbnail generation. + + Edit ``/etc/ImageMagick-6/policy.xml`` and adjust + + .. code:: + + + + to + + .. code:: + + Migration to paperless-ng ######################### diff --git a/scripts/make-release.sh b/scripts/make-release.sh index 3c3892aed..0a7bc7a9b 100755 --- a/scripts/make-release.sh +++ b/scripts/make-release.sh @@ -42,6 +42,7 @@ fi mkdir "$PAPERLESS_DIST" mkdir "$PAPERLESS_DIST_APP" mkdir "$PAPERLESS_DIST_APP/docker" +mkdir "$PAPERLESS_DIST_APP/scripts" mkdir "$PAPERLESS_DIST_DOCKERFILES" # setup dependencies. @@ -104,6 +105,11 @@ cp "$PAPERLESS_ROOT/docker/gunicorn.conf.py" "$PAPERLESS_DIST_APP/docker/" cp "$PAPERLESS_ROOT/docker/imagemagick-policy.xml" "$PAPERLESS_DIST_APP/docker/" cp "$PAPERLESS_ROOT/docker/supervisord.conf" "$PAPERLESS_DIST_APP/docker/" +# auxiliary files for bare metal installs +cp "$PAPERLESS_ROOT/scripts/paperless-webserver.service" "$PAPERLESS_DIST_APP/scripts/" +cp "$PAPERLESS_ROOT/scripts/paperless-consumer.service" "$PAPERLESS_DIST_APP/scripts/" +cp "$PAPERLESS_ROOT/scripts/paperless-scheduler.service" "$PAPERLESS_DIST_APP/scripts/" + # try to make the docker build. cd "$PAPERLESS_DIST_APP" diff --git a/scripts/paperless-consumer.service b/scripts/paperless-consumer.service index 79a27d3ce..04512b719 100644 --- a/scripts/paperless-consumer.service +++ b/scripts/paperless-consumer.service @@ -1,10 +1,12 @@ [Unit] Description=Paperless consumer +Requires=redis.service [Service] User=paperless Group=paperless -ExecStart=/home/paperless/project/virtualenv/bin/python /home/paperless/project/src/manage.py document_consumer +WorkingDirectory=/opt/paperless/src +ExecStart=python3 manage.py document_consumer [Install] WantedBy=multi-user.target diff --git a/scripts/paperless-scheduler.service b/scripts/paperless-scheduler.service new file mode 100644 index 000000000..c565d8af4 --- /dev/null +++ b/scripts/paperless-scheduler.service @@ -0,0 +1,12 @@ +[Unit] +Description=Paperless consumer +Requires=redis.service + +[Service] +User=paperless +Group=paperless +WorkingDirectory=/opt/paperless/src +ExecStart=python3 manage.py qcluster + +[Install] +WantedBy=multi-user.target diff --git a/scripts/paperless-webserver.service b/scripts/paperless-webserver.service index 8ff746be2..a43110be2 100644 --- a/scripts/paperless-webserver.service +++ b/scripts/paperless-webserver.service @@ -2,11 +2,13 @@ Description=Paperless webserver After=network.target Wants=network.target +Requires=redis.service [Service] User=paperless Group=paperless -ExecStart=/home/paperless/project/virtualenv/bin/gunicorn --pythonpath=/home/paperless/project/src paperless.wsgi -w 2 +WorkingDirectory=/opt/paperless/src +ExecStart=/opt/paperless/.local/bin/gunicorn paperless.wsgi -w 2 -b 0.0.0.0:8000 [Install] WantedBy=multi-user.target diff --git a/src-ui/src/app/components/common/tag/tag.component.html b/src-ui/src/app/components/common/tag/tag.component.html index a83c53c77..8b9632a65 100644 --- a/src-ui/src/app/components/common/tag/tag.component.html +++ b/src-ui/src/app/components/common/tag/tag.component.html @@ -1,2 +1,2 @@ {{tag.name}} -{{tag.name}} \ No newline at end of file +{{tag.name}} \ No newline at end of file diff --git a/src-ui/src/app/components/common/tag/tag.component.ts b/src-ui/src/app/components/common/tag/tag.component.ts index ec59e86f3..c032c51db 100644 --- a/src-ui/src/app/components/common/tag/tag.component.ts +++ b/src-ui/src/app/components/common/tag/tag.component.ts @@ -14,10 +14,10 @@ export class TagComponent implements OnInit { tag: PaperlessTag @Input() - clickable: boolean = false + linkTitle: string = "" - @Output() - click = new EventEmitter() + @Input() + clickable: boolean = false ngOnInit(): void { } diff --git a/src-ui/src/app/components/document-detail/document-detail.component.ts b/src-ui/src/app/components/document-detail/document-detail.component.ts index 21ed6a4a5..7c396692e 100644 --- a/src-ui/src/app/components/document-detail/document-detail.component.ts +++ b/src-ui/src/app/components/document-detail/document-detail.component.ts @@ -1,4 +1,3 @@ -import { DatePipe, formatDate } from '@angular/common'; import { Component, OnInit } from '@angular/core'; import { FormControl, FormGroup } from '@angular/forms'; import { ActivatedRoute, Router } from '@angular/router'; @@ -7,17 +6,14 @@ import { PaperlessCorrespondent } from 'src/app/data/paperless-correspondent'; import { PaperlessDocument } from 'src/app/data/paperless-document'; import { PaperlessDocumentMetadata } from 'src/app/data/paperless-document-metadata'; import { PaperlessDocumentType } from 'src/app/data/paperless-document-type'; -import { TAG_COLOURS, PaperlessTag } from 'src/app/data/paperless-tag'; import { DocumentListViewService } from 'src/app/services/document-list-view.service'; import { OpenDocumentsService } from 'src/app/services/open-documents.service'; import { CorrespondentService } from 'src/app/services/rest/correspondent.service'; import { DocumentTypeService } from 'src/app/services/rest/document-type.service'; import { DocumentService } from 'src/app/services/rest/document.service'; -import { TagService } from 'src/app/services/rest/tag.service'; import { DeleteDialogComponent } from '../common/delete-dialog/delete-dialog.component'; import { CorrespondentEditDialogComponent } from '../manage/correspondent-list/correspondent-edit-dialog/correspondent-edit-dialog.component'; import { DocumentTypeEditDialogComponent } from '../manage/document-type-list/document-type-edit-dialog/document-type-edit-dialog.component'; -import { TagEditDialogComponent } from '../manage/tag-list/tag-edit-dialog/tag-edit-dialog.component'; @Component({ selector: 'app-document-detail', @@ -140,8 +136,8 @@ export class DocumentDetailComponent implements OnInit { close() { this.openDocumentService.closeDocument(this.document) - if (this.documentListViewService.viewId) { - this.router.navigate(['view', this.documentListViewService.viewId]) + if (this.documentListViewService.savedViewId) { + this.router.navigate(['view', this.documentListViewService.savedViewId]) } else { this.router.navigate(['documents']) } diff --git a/src-ui/src/app/components/document-list/document-card-large/document-card-large.component.html b/src-ui/src/app/components/document-list/document-card-large/document-card-large.component.html index 305cb37d2..63a8bf710 100644 --- a/src-ui/src/app/components/document-list/document-card-large/document-card-large.component.html +++ b/src-ui/src/app/components/document-list/document-card-large/document-card-large.component.html @@ -7,7 +7,12 @@
-
{{document.correspondent ? document.correspondent.name + ': ' : ''}}{{document.title}}
+
+ + {{document.correspondent.name}}: + + {{document.title}} +
#{{document.archive_serial_number}}

@@ -24,6 +29,13 @@ Edit + + + + + + View + diff --git a/src-ui/src/app/components/document-list/document-card-large/document-card-large.component.ts b/src-ui/src/app/components/document-list/document-card-large/document-card-large.component.ts index 1c1e110c0..4a44909ec 100644 --- a/src-ui/src/app/components/document-list/document-card-large/document-card-large.component.ts +++ b/src-ui/src/app/components/document-list/document-card-large/document-card-large.component.ts @@ -1,6 +1,7 @@ -import { Component, Input, OnInit } from '@angular/core'; +import { Component, EventEmitter, Input, OnInit, Output } from '@angular/core'; import { DomSanitizer } from '@angular/platform-browser'; import { PaperlessDocument } from 'src/app/data/paperless-document'; +import { PaperlessTag } from 'src/app/data/paperless-tag'; import { DocumentService } from 'src/app/services/rest/document.service'; @Component({ @@ -18,6 +19,12 @@ export class DocumentCardLargeComponent implements OnInit { @Input() details: any + @Output() + clickTag = new EventEmitter() + + @Output() + clickCorrespondent = new EventEmitter() + ngOnInit(): void { } @@ -41,4 +48,8 @@ export class DocumentCardLargeComponent implements OnInit { getDownloadUrl() { return this.documentService.getDownloadUrl(this.document.id) } + + getPreviewUrl() { + return this.documentService.getPreviewUrl(this.document.id) + } } diff --git a/src-ui/src/app/components/document-list/document-card-small/document-card-small.component.html b/src-ui/src/app/components/document-list/document-card-small/document-card-small.component.html index 022b92a4d..4da5cdf9b 100644 --- a/src-ui/src/app/components/document-list/document-card-small/document-card-small.component.html +++ b/src-ui/src/app/components/document-list/document-card-small/document-card-small.component.html @@ -2,26 +2,34 @@

- +
-

- {{document.correspondent? document.correspondent.name + ': ' : ''}} {{document.title}} + + {{document.correspondent.name}}: + + {{document.title}}

- + - + + + + + + + diff --git a/src-ui/src/app/components/document-list/document-card-small/document-card-small.component.ts b/src-ui/src/app/components/document-list/document-card-small/document-card-small.component.ts index 1ca9847b9..2c0ca8dfb 100644 --- a/src-ui/src/app/components/document-list/document-card-small/document-card-small.component.ts +++ b/src-ui/src/app/components/document-list/document-card-small/document-card-small.component.ts @@ -1,5 +1,6 @@ -import { Component, Input, OnInit } from '@angular/core'; +import { Component, EventEmitter, Input, OnInit, Output } from '@angular/core'; import { PaperlessDocument } from 'src/app/data/paperless-document'; +import { PaperlessTag } from 'src/app/data/paperless-tag'; import { DocumentService } from 'src/app/services/rest/document.service'; @Component({ @@ -14,6 +15,12 @@ export class DocumentCardSmallComponent implements OnInit { @Input() document: PaperlessDocument + @Output() + clickTag = new EventEmitter() + + @Output() + clickCorrespondent = new EventEmitter() + ngOnInit(): void { } @@ -24,4 +31,8 @@ export class DocumentCardSmallComponent implements OnInit { getDownloadUrl() { return this.documentService.getDownloadUrl(this.document.id) } + + getPreviewUrl() { + return this.documentService.getPreviewUrl(this.document.id) + } } diff --git a/src-ui/src/app/components/document-list/document-list.component.html b/src-ui/src/app/components/document-list/document-list.component.html index 7c0abebc8..cc682b8e3 100644 --- a/src-ui/src/app/components/document-list/document-list.component.html +++ b/src-ui/src/app/components/document-list/document-list.component.html @@ -21,13 +21,12 @@
-
+
- +
-
+
- -
- + + +
+ + + +
@@ -72,16 +75,16 @@
- +
- +
- +
@@ -91,20 +94,37 @@ - - - - - - - + + + + + + +
ASN CorrespondentAdded
{{d.archive_serial_number}}{{d.correspondent ? d.correspondent.name : ''}}{{d.title}}{{d.document_type ? d.document_type.name : ''}}{{d.created | date}}{{d.added | date}}
+ {{d.archive_serial_number}} + + + {{d.correspondent.name}} + + + {{d.title}} + + + + {{d.document_type.name}} + + + {{d.created | date}} + + {{d.added | date}} +
- +
-

No results

\ No newline at end of file +

No results

diff --git a/src-ui/src/app/components/document-list/document-list.component.ts b/src-ui/src/app/components/document-list/document-list.component.ts index 2b155c34f..08f94c2ee 100644 --- a/src-ui/src/app/components/document-list/document-list.component.ts +++ b/src-ui/src/app/components/document-list/document-list.component.ts @@ -1,11 +1,16 @@ import { Component, OnInit } from '@angular/core'; -import { ActivatedRoute, Router } from '@angular/router'; +import { ActivatedRoute } from '@angular/router'; import { NgbModal } from '@ng-bootstrap/ng-bootstrap'; import { cloneFilterRules, FilterRule } from 'src/app/data/filter-rule'; +import { FILTER_CORRESPONDENT, FILTER_DOCUMENT_TYPE, FILTER_HAS_TAG, FILTER_RULE_TYPES } from 'src/app/data/filter-rule-type'; +import { PaperlessCorrespondent } from 'src/app/data/paperless-correspondent'; +import { PaperlessDocumentType } from 'src/app/data/paperless-document-type'; +import { PaperlessTag } from 'src/app/data/paperless-tag'; import { SavedViewConfig } from 'src/app/data/saved-view-config'; import { DocumentListViewService } from 'src/app/services/document-list-view.service'; import { DOCUMENT_SORT_FIELDS } from 'src/app/services/rest/document.service'; import { SavedViewConfigService } from 'src/app/services/saved-view-config.service'; +import { Toast, ToastService } from 'src/app/services/toast.service'; import { SaveViewConfigDialogComponent } from './save-view-config-dialog/save-view-config-dialog.component'; @Component({ @@ -16,9 +21,10 @@ import { SaveViewConfigDialogComponent } from './save-view-config-dialog/save-vi export class DocumentListComponent implements OnInit { constructor( - public docs: DocumentListViewService, + public list: DocumentListViewService, public savedViewConfigService: SavedViewConfigService, public route: ActivatedRoute, + private toastService: ToastService, public modalService: NgbModal) { } displayMode = 'smallCards' // largeCards, smallCards, details @@ -27,17 +33,13 @@ export class DocumentListComponent implements OnInit { showFilter = false getTitle() { - return this.docs.viewConfigOverride ? this.docs.viewConfigOverride.title : "Documents" + return this.list.savedViewTitle || "Documents" } getSortFields() { return DOCUMENT_SORT_FIELDS } - setSort(field: string) { - this.docs.sortField = field - } - saveDisplayMode() { localStorage.setItem('document-list:displayMode', this.displayMode) } @@ -48,41 +50,74 @@ export class DocumentListComponent implements OnInit { } this.route.paramMap.subscribe(params => { if (params.has('id')) { - this.docs.viewConfigOverride = this.savedViewConfigService.getConfig(params.get('id')) + this.list.savedView = this.savedViewConfigService.getConfig(params.get('id')) } else { - this.filterRules = this.docs.filterRules - this.showFilter = this.filterRules.length > 0 - this.docs.viewConfigOverride = null + this.list.savedView = null } - this.reload() + this.filterRules = this.list.filterRules + //this.showFilter = this.filterRules.length > 0 + // prevents temporarily visible results from previous views + this.list.documents = [] + this.list.reload() }) } - reload() { - this.docs.reload() - } - applyFilterRules() { - this.docs.filterRules = this.filterRules + this.list.filterRules = this.filterRules } loadViewConfig(config: SavedViewConfig) { this.filterRules = cloneFilterRules(config.filterRules) - this.docs.loadViewConfig(config) + this.list.load(config) } saveViewConfig() { + this.savedViewConfigService.updateConfig(this.list.savedView) + this.toastService.showToast(Toast.make("Information", `View "${this.list.savedView.title}" saved successfully.`)) + } + + saveViewConfigAs() { let modal = this.modalService.open(SaveViewConfigDialogComponent, {backdrop: 'static'}) modal.componentInstance.saveClicked.subscribe(formValue => { - this.savedViewConfigService.saveConfig({ + this.savedViewConfigService.newConfig({ title: formValue.title, showInDashboard: formValue.showInDashboard, showInSideBar: formValue.showInSideBar, - filterRules: this.docs.filterRules, - sortDirection: this.docs.sortDirection, - sortField: this.docs.sortField + filterRules: this.list.filterRules, + sortDirection: this.list.sortDirection, + sortField: this.list.sortField }) modal.close() }) } + + filterByTag(t: PaperlessTag) { + if (this.filterRules.find(rule => rule.type.id == FILTER_HAS_TAG && rule.value == t.id)) { + return + } + + this.filterRules.push({type: FILTER_RULE_TYPES.find(t => t.id == FILTER_HAS_TAG), value: t.id}) + this.applyFilterRules() + } + + filterByCorrespondent(c: PaperlessCorrespondent) { + let existing_rule = this.filterRules.find(rule => rule.type.id == FILTER_CORRESPONDENT) + if (existing_rule) { + existing_rule.value = c.id + } else { + this.filterRules.push({type: FILTER_RULE_TYPES.find(t => t.id == FILTER_CORRESPONDENT), value: c.id}) + } + this.applyFilterRules() + } + + filterByDocumentType(dt: PaperlessDocumentType) { + let existing_rule = this.filterRules.find(rule => rule.type.id == FILTER_DOCUMENT_TYPE) + if (existing_rule) { + existing_rule.value = dt.id + } else { + this.filterRules.push({type: FILTER_RULE_TYPES.find(t => t.id == FILTER_DOCUMENT_TYPE), value: dt.id}) + } + this.applyFilterRules() + } + } diff --git a/src-ui/src/app/components/search/search.component.scss b/src-ui/src/app/components/search/search.component.scss index 5be54746a..40ca79a61 100644 --- a/src-ui/src/app/components/search/search.component.scss +++ b/src-ui/src/app/components/search/search.component.scss @@ -11,5 +11,5 @@ } .result-content-searching { - opacity: 0.2; + opacity: 0.3; } \ No newline at end of file diff --git a/src-ui/src/app/data/filter-rule-type.ts b/src-ui/src/app/data/filter-rule-type.ts index e5de30271..e1db34298 100644 --- a/src-ui/src/app/data/filter-rule-type.ts +++ b/src-ui/src/app/data/filter-rule-type.ts @@ -1,31 +1,51 @@ +export const FILTER_TITLE = 0 +export const FILTER_CONTENT = 1 +export const FILTER_ASN = 2 +export const FILTER_CORRESPONDENT = 3 +export const FILTER_DOCUMENT_TYPE = 4 +export const FILTER_IS_IN_INBOX = 5 +export const FILTER_HAS_TAG = 6 +export const FILTER_HAS_ANY_TAG = 7 +export const FILTER_CREATED_BEFORE = 8 +export const FILTER_CREATED_AFTER = 9 +export const FILTER_CREATED_YEAR = 10 +export const FILTER_CREATED_MONTH = 11 +export const FILTER_CREATED_DAY = 12 +export const FILTER_ADDED_BEFORE = 13 +export const FILTER_ADDED_AFTER = 14 +export const FILTER_MODIFIED_BEFORE = 15 +export const FILTER_MODIFIED_AFTER = 16 + export const FILTER_RULE_TYPES: FilterRuleType[] = [ - {name: "Title contains", filtervar: "title__icontains", datatype: "string", multi: false}, - {name: "Content contains", filtervar: "content__icontains", datatype: "string", multi: false}, + + {id: FILTER_TITLE, name: "Title contains", filtervar: "title__icontains", datatype: "string", multi: false}, + {id: FILTER_CONTENT, name: "Content contains", filtervar: "content__icontains", datatype: "string", multi: false}, - {name: "ASN is", filtervar: "archive_serial_number", datatype: "number", multi: false}, + {id: FILTER_ASN, name: "ASN is", filtervar: "archive_serial_number", datatype: "number", multi: false}, - {name: "Correspondent is", filtervar: "correspondent__id", datatype: "correspondent", multi: false}, - {name: "Document type is", filtervar: "document_type__id", datatype: "document_type", multi: false}, + {id: FILTER_CORRESPONDENT, name: "Correspondent is", filtervar: "correspondent__id", datatype: "correspondent", multi: false}, + {id: FILTER_DOCUMENT_TYPE, name: "Document type is", filtervar: "document_type__id", datatype: "document_type", multi: false}, - {name: "Is in Inbox", filtervar: "is_in_inbox", datatype: "boolean", multi: false}, - {name: "Has tag", filtervar: "tags__id__all", datatype: "tag", multi: true}, - {name: "Has any tag", filtervar: "is_tagged", datatype: "boolean", multi: false}, + {id: FILTER_IS_IN_INBOX, name: "Is in Inbox", filtervar: "is_in_inbox", datatype: "boolean", multi: false}, + {id: FILTER_HAS_TAG, name: "Has tag", filtervar: "tags__id__all", datatype: "tag", multi: true}, + {id: FILTER_HAS_ANY_TAG, name: "Has any tag", filtervar: "is_tagged", datatype: "boolean", multi: false}, - {name: "Created before", filtervar: "created__date__lt", datatype: "date", multi: false}, - {name: "Created after", filtervar: "created__date__gt", datatype: "date", multi: false}, + {id: FILTER_CREATED_BEFORE, name: "Created before", filtervar: "created__date__lt", datatype: "date", multi: false}, + {id: FILTER_CREATED_AFTER, name: "Created after", filtervar: "created__date__gt", datatype: "date", multi: false}, - {name: "Year created is", filtervar: "created__year", datatype: "number", multi: false}, - {name: "Month created is", filtervar: "created__month", datatype: "number", multi: false}, - {name: "Day created is", filtervar: "created__day", datatype: "number", multi: false}, + {id: FILTER_CREATED_YEAR, name: "Year created is", filtervar: "created__year", datatype: "number", multi: false}, + {id: FILTER_CREATED_MONTH, name: "Month created is", filtervar: "created__month", datatype: "number", multi: false}, + {id: FILTER_CREATED_DAY, name: "Day created is", filtervar: "created__day", datatype: "number", multi: false}, - {name: "Added before", filtervar: "added__date__lt", datatype: "date", multi: false}, - {name: "Added after", filtervar: "added__date__gt", datatype: "date", multi: false}, + {id: FILTER_ADDED_BEFORE, name: "Added before", filtervar: "added__date__lt", datatype: "date", multi: false}, + {id: FILTER_ADDED_AFTER, name: "Added after", filtervar: "added__date__gt", datatype: "date", multi: false}, - {name: "Modified before", filtervar: "modified__date__lt", datatype: "date", multi: false}, - {name: "Modified after", filtervar: "modified__date__gt", datatype: "date", multi: false}, + {id: FILTER_MODIFIED_BEFORE, name: "Modified before", filtervar: "modified__date__lt", datatype: "date", multi: false}, + {id: FILTER_MODIFIED_AFTER, name: "Modified after", filtervar: "modified__date__gt", datatype: "date", multi: false}, ] export interface FilterRuleType { + id: number name: string filtervar: string datatype: string //number, string, boolean, date diff --git a/src-ui/src/app/services/document-list-view.service.ts b/src-ui/src/app/services/document-list-view.service.ts index 39a8661b9..9c7e244f4 100644 --- a/src-ui/src/app/services/document-list-view.service.ts +++ b/src-ui/src/app/services/document-list-view.service.ts @@ -7,6 +7,12 @@ import { DOCUMENT_LIST_SERVICE, GENERAL_SETTINGS } from '../data/storage-keys'; import { DocumentService } from './rest/document.service'; +/** + * This service manages the document list which is displayed using the document list view. + * + * This service also serves saved views by transparently switching between the document list + * and saved views on request. See below. + */ @Injectable({ providedIn: 'root' }) @@ -14,80 +20,127 @@ export class DocumentListViewService { static DEFAULT_SORT_FIELD = 'created' + isReloading: boolean = false documents: PaperlessDocument[] = [] currentPage = 1 currentPageSize: number = +localStorage.getItem(GENERAL_SETTINGS.DOCUMENT_LIST_SIZE) || GENERAL_SETTINGS.DOCUMENT_LIST_SIZE_DEFAULT collectionSize: number - private currentViewConfig: SavedViewConfig - //TODO: make private - viewConfigOverride: SavedViewConfig + /** + * This is the current config for the document list. The service will always remember the last settings used for the document list. + */ + private _documentListViewConfig: SavedViewConfig + /** + * Optionally, this is the currently selected saved view, which might be null. + */ + private _savedViewConfig: SavedViewConfig - get viewId() { - return this.viewConfigOverride?.id + get savedView() { + return this._savedViewConfig + } + + set savedView(value) { + if (value) { + //this is here so that we don't modify value, which might be the actual instance of the saved view. + this._savedViewConfig = Object.assign({}, value) + } else { + this._savedViewConfig = null + } + } + + get savedViewId() { + return this.savedView?.id + } + + get savedViewTitle() { + return this.savedView?.title + } + + get documentListView() { + return this._documentListViewConfig + } + + set documentListView(value) { + if (value) { + this._documentListViewConfig = Object.assign({}, value) + this.saveDocumentListView() + } + } + + /** + * This is what switches between the saved views and the document list view. Everything on the document list uses + * this property to determine the settings for the currently displayed document list. + */ + get view() { + return this.savedView || this.documentListView + } + + load(config: SavedViewConfig) { + this.view.filterRules = cloneFilterRules(config.filterRules) + this.view.sortDirection = config.sortDirection + this.view.sortField = config.sortField + this.reload() } reload(onFinish?) { - let viewConfig = this.viewConfigOverride || this.currentViewConfig - + this.isReloading = true this.documentService.list( this.currentPage, this.currentPageSize, - viewConfig.sortField, - viewConfig.sortDirection, - viewConfig.filterRules).subscribe( + this.view.sortField, + this.view.sortDirection, + this.view.filterRules).subscribe( result => { this.collectionSize = result.count this.documents = result.results if (onFinish) { onFinish() } + this.isReloading = false }, error => { if (error.error['detail'] == 'Invalid page.') { this.currentPage = 1 this.reload() } + this.isReloading = false }) } set filterRules(filterRules: FilterRule[]) { - this.currentViewConfig.filterRules = cloneFilterRules(filterRules) - this.saveCurrentViewConfig() + //we're going to clone the filterRules object, since we don't + //want changes in the filter editor to propagate into here right away. + this.view.filterRules = cloneFilterRules(filterRules) this.reload() + this.saveDocumentListView() } get filterRules(): FilterRule[] { - return cloneFilterRules(this.currentViewConfig.filterRules) + return cloneFilterRules(this.view.filterRules) } set sortField(field: string) { - this.currentViewConfig.sortField = field - this.saveCurrentViewConfig() + this.view.sortField = field + this.saveDocumentListView() this.reload() } get sortField(): string { - return this.currentViewConfig.sortField + return this.view.sortField } set sortDirection(direction: string) { - this.currentViewConfig.sortDirection = direction - this.saveCurrentViewConfig() + this.view.sortDirection = direction + this.saveDocumentListView() this.reload() } get sortDirection(): string { - return this.currentViewConfig.sortDirection + return this.view.sortDirection } - loadViewConfig(config: SavedViewConfig) { - Object.assign(this.currentViewConfig, config) - this.reload() - } - - private saveCurrentViewConfig() { - sessionStorage.setItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG, JSON.stringify(this.currentViewConfig)) + private saveDocumentListView() { + sessionStorage.setItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG, JSON.stringify(this.documentListView)) } getLastPage(): number { @@ -134,21 +187,21 @@ export class DocumentListViewService { } constructor(private documentService: DocumentService) { - let currentViewConfigJson = sessionStorage.getItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG) - if (currentViewConfigJson) { + let documentListViewConfigJson = sessionStorage.getItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG) + if (documentListViewConfigJson) { try { - this.currentViewConfig = JSON.parse(currentViewConfigJson) + this.documentListView = JSON.parse(documentListViewConfigJson) } catch (e) { sessionStorage.removeItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG) - this.currentViewConfig = null + this.documentListView = null } } - if (!this.currentViewConfig) { - this.currentViewConfig = { + if (!this.documentListView) { + this.documentListView = { filterRules: [], sortDirection: 'des', sortField: 'created' } - } + } } } diff --git a/src-ui/src/app/services/saved-view-config.service.ts b/src-ui/src/app/services/saved-view-config.service.ts index d69791209..41c28216b 100644 --- a/src-ui/src/app/services/saved-view-config.service.ts +++ b/src-ui/src/app/services/saved-view-config.service.ts @@ -36,13 +36,21 @@ export class SavedViewConfigService { return this.configs.find(sf => sf.id == id) } - saveConfig(config: SavedViewConfig) { + newConfig(config: SavedViewConfig) { config.id = uuidv4() this.configs.push(config) this.save() } + updateConfig(config: SavedViewConfig) { + let savedConfig = this.configs.find(c => c.id == config.id) + if (savedConfig) { + Object.assign(savedConfig, config) + this.save() + } + } + private save() { localStorage.setItem('saved-view-config-service:savedConfigs', JSON.stringify(this.configs)) } diff --git a/src/documents/signals/handlers.py b/src/documents/signals/handlers.py index 9672b884b..e2d055fef 100755 --- a/src/documents/signals/handlers.py +++ b/src/documents/signals/handlers.py @@ -9,6 +9,7 @@ from django.contrib.contenttypes.models import ContentType from django.db import models, DatabaseError from django.dispatch import receiver from django.utils import timezone +from rest_framework.reverse import reverse from .. import index, matching from ..file_handling import delete_empty_directories, generate_filename, \ @@ -157,10 +158,10 @@ def run_post_consume_script(sender, document, **kwargs): settings.POST_CONSUME_SCRIPT, str(document.pk), document.file_name, - document.source_path, - document.thumbnail_path, - None, - None, + os.path.normpath(document.source_path), + os.path.normpath(document.thumbnail_path), + reverse("document-download", kwargs={"pk": document.pk}), + reverse("document-thumb", kwargs={"pk": document.pk}), str(document.correspondent), str(",".join(document.tags.all().values_list("slug", flat=True))) )).wait() diff --git a/src/documents/tests/test_post_consume_handlers.py b/src/documents/tests/test_post_consume_handlers.py new file mode 100644 index 000000000..aa712832a --- /dev/null +++ b/src/documents/tests/test_post_consume_handlers.py @@ -0,0 +1,57 @@ +from unittest import mock + +from django.test import TestCase, override_settings + +from documents.models import Document, Tag, Correspondent +from documents.signals.handlers import run_post_consume_script + + +class PostConsumeTestCase(TestCase): + + @mock.patch("documents.signals.handlers.Popen") + @override_settings(POST_CONSUME_SCRIPT=None) + def test_no_post_consume_script(self, m): + doc = Document.objects.create(title="Test", mime_type="application/pdf") + tag1 = Tag.objects.create(name="a") + tag2 = Tag.objects.create(name="b") + doc.tags.add(tag1) + doc.tags.add(tag2) + + run_post_consume_script(None, doc) + + m.assert_not_called() + + @mock.patch("documents.signals.handlers.Popen") + @override_settings(POST_CONSUME_SCRIPT="script") + def test_post_consume_script_simple(self, m): + doc = Document.objects.create(title="Test", mime_type="application/pdf") + + run_post_consume_script(None, doc) + + m.assert_called_once() + + @mock.patch("documents.signals.handlers.Popen") + @override_settings(POST_CONSUME_SCRIPT="script") + def test_post_consume_script_simple(self, m): + c = Correspondent.objects.create(name="my_bank") + doc = Document.objects.create(title="Test", mime_type="application/pdf", correspondent=c) + tag1 = Tag.objects.create(name="a") + tag2 = Tag.objects.create(name="b") + doc.tags.add(tag1) + doc.tags.add(tag2) + + run_post_consume_script(None, doc) + + m.assert_called_once() + + args, kwargs = m.call_args + + command = args[0] + + self.assertEqual(command[0], "script") + self.assertEqual(command[1], str(doc.pk)) + self.assertEqual(command[5], f"/api/documents/{doc.pk}/download/") + self.assertEqual(command[6], f"/api/documents/{doc.pk}/thumb/") + self.assertEqual(command[7], "my_bank") + # TODO: tags are unordered by default. + self.assertEqual(command[8], "a,b") diff --git a/src/paperless/db.py b/src/paperless/db.py index 92275808b..5f476b5e7 100644 --- a/src/paperless/db.py +++ b/src/paperless/db.py @@ -17,16 +17,3 @@ class GnuPG: passphrase = settings.PASSPHRASE return cls.gpg.decrypt_file(file_handle, passphrase=passphrase).data - - @classmethod - def encrypted(cls, file_handle, passphrase=None): - - if not passphrase: - passphrase = settings.PASSPHRASE - - return cls.gpg.encrypt_file( - file_handle, - recipients=None, - passphrase=passphrase, - symmetric=True - ).data