paradizelost/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2025-04-25 10:49:30 -05:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'dev' into feature-ocrmypdf

Browse Source
This commit is contained in:
jonaswinkler 2020-11-29 01:35:37 +01:00
commit fdaf419a7e
22 changed files with 524 additions and 135 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
docs/changelog.rst
View File

@ -5,6 +5,16 @@
Changelog
*********
paperless-ng 0.9.4
##################
* Front end: Clickable tags, correspondents and types allow quick filtering for related documents.
* Front end: Saved views are now editable.
* Front end: Preview documents directly in the browser.
* Fixes:
* A severe error when trying to use post consume scripts.
* The documentation now contains information about bare metal installs.
paperless-ng 0.9.3
##################
@ -20,7 +30,7 @@ paperless-ng 0.9.3
aware of.
* Issue with the automatic classifier not working with only one tag.
* A couple issues with the search index being opened to eagerly.
* Added lots of tests for various parts of the application.
paperless-ng 0.9.2

142
docs/setup.rst
View File

@ -208,9 +208,147 @@ Docker Route
Bare Metal Route
================
.. warning::
Paperless runs on linux only. The following procedure has been tested on a minimal
installation of Debian/Buster, which is the current stable release at the time of
writing. Windows is not and will never be supported.
TBD. User docker for now.
1. Install dependencies. Paperless requires the following packages.
* ``python3`` 3.6, 3.7, 3.8 (3.9 is untested).
* ``python3-pip``, optionally ``pipenv`` for package installation
* ``python3-dev``
* ``imagemagick`` >= 6 for PDF conversion
* ``unpaper`` for cleaning documents before OCR
* ``ghostscript``
* ``optipng`` for optimising thumbnails
* ``tesseract-ocr`` >= 4.0.0 for OCR
* ``tesseract-ocr`` language packs (``tesseract-ocr-eng``, ``tesseract-ocr-deu``, etc)
* ``gnupg`` for handling encrypted documents
* ``libpoppler-cpp-dev`` for PDF to text conversion
* ``libmagic-dev`` for mime type detection
* ``libpq-dev`` for PostgreSQL
You will also need ``build-essential``, ``python3-setuptools`` and ``python3-wheel``
for installing some of the python dependencies. You can remove that
again after installation.
2. Install ``redis`` >= 5.0 and configure it to start automatically.
3. Optional. Install ``postgresql`` and configure a database, user and password for paperless. If you do not wish
to use PostgreSQL, SQLite is avialable as well.
4. Get the release archive. If you pull the git repo as it is, you also have to compile the front end by yourself.
Extract the frontend to a place from where you wish to execute it, such as ``/opt/paperless``.
5. Configure paperless. See :ref:`configuration` for details. Edit the included ``paperless.conf`` and adjust the
settings to your needs. Required settings for getting paperless running are:
* ``PAPERLESS_REDIS`` should point to your redis server, such as redis://localhost:6379.
* ``PAPERLESS_DBHOST`` should be the hostname on which your PostgreSQL server is running. Do not configure this
to use SQLite instead. Also configure port, database name, user and password as necessary.
* ``PAPERLESS_CONSUMPTION_DIR`` should point to a folder which paperless should watch for documents. You might
want to have this somewhere else. Likewise, ``PAPERLESS_DATA_DIR`` and ``PAPERLESS_MEDIA_ROOT`` define where
paperless stores its data. If you like, you can point both to the same directory.
* ``PAPERLESS_SECRET_KEY`` should be a random sequence of characters. It's used for authentication. Failure
to do so allows third parties to forge authentication credentials.
Many more adjustments can be made to paperless, especially the OCR part. The following options are recommended
for everyone:
* Set ``PAPERLESS_OCR_LANGUAGE`` to the language most of your documents are written in.
* Set ``PAPERLESS_TIME_ZONE`` to your local time zone.
6. Setup permissions. Create a system users under which you wish to run paperless. Ensure that these directories exist
and that the user has write permissions to the following directories
* ``/opt/paperless/media``
* ``/opt/paperless/data``
* ``/opt/paperless/consume``
Adjust as necessary if you configured different folders.
7. Install python requirements. Paperless comes with both Pipfiles for ``pipenv`` as well as with a ``requirements.txt``.
Both will install exactly the same requirements. It is up to you if you wish to use a virtual environment or not.
8. Go to ``/opt/paperless/src``, and execute the following commands:
.. code:: bash
# This collects static files from paperless and django.
python3 manage.py collectstatic --clear --no-input
# This creates the database schema.
python3 manage.py migrate
# This creates your first paperless user
python3 manage.py createsuperuser
9. Optional: Test that paperless is working by executing
.. code:: bash
# This collects static files from paperless and django.
python3 manage.py runserver
and pointing your browser to http://localhost:8000/.
.. warning::
This is a development server which should not be used in
production.
.. hint::
This will not start the consumer. Paperless does this in a
separate process.
10. Setup systemd services to run paperless automatically. You may
use the service definition files included in the ``scripts`` folder
as a starting point.
Paperless needs the ``webserver`` script to run the webserver, the
``consumer`` script to watch the input folder, and the ``scheduler``
script to run tasks such as email checking and document consumption.
These services rely on redis and optionally the database server, but
don't need to be started in any particular order. The example files
depend on redis being started. If you use a database server, you should
add additinal dependencies.
.. hint::
You may optionally set up your preferred web server to serve
paperless as a wsgi application directly instead of running the
``webserver`` service. The module containing the wsgi application
is named ``paperless.wsgi``.
.. caution::
The included scripts run a ``gunicorn`` standalone server,
which is fine for running paperless. It does support SSL,
however, the documentation of GUnicorn states that you should
use a proxy server in front of gunicorn instead.
11. Optional: Install a samba server and make the consumption folder
available as a network share.
12. Configure ImageMagick to allow processing of PDF documents. Most distributions have
this disabled by default, since PDF documents can contain malware. If
you don't do this, paperless will fall back to ghostscript for certain steps
such as thumbnail generation.
Edit ``/etc/ImageMagick-6/policy.xml`` and adjust
.. code::
<policy domain="coder" rights="none" pattern="PDF" />
to
.. code::
<policy domain="coder" rights="read|write" pattern="PDF" />
Migration to paperless-ng
#########################

6
scripts/make-release.sh
View File

@ -42,6 +42,7 @@ fi
mkdir "$PAPERLESS_DIST"
mkdir "$PAPERLESS_DIST_APP"
mkdir "$PAPERLESS_DIST_APP/docker"
mkdir "$PAPERLESS_DIST_APP/scripts"
mkdir "$PAPERLESS_DIST_DOCKERFILES"
# setup dependencies.
@ -104,6 +105,11 @@ cp "$PAPERLESS_ROOT/docker/gunicorn.conf.py" "$PAPERLESS_DIST_APP/docker/"
cp "$PAPERLESS_ROOT/docker/imagemagick-policy.xml" "$PAPERLESS_DIST_APP/docker/"
cp "$PAPERLESS_ROOT/docker/supervisord.conf" "$PAPERLESS_DIST_APP/docker/"
# auxiliary files for bare metal installs
cp "$PAPERLESS_ROOT/scripts/paperless-webserver.service" "$PAPERLESS_DIST_APP/scripts/"
cp "$PAPERLESS_ROOT/scripts/paperless-consumer.service" "$PAPERLESS_DIST_APP/scripts/"
cp "$PAPERLESS_ROOT/scripts/paperless-scheduler.service" "$PAPERLESS_DIST_APP/scripts/"
# try to make the docker build.
cd "$PAPERLESS_DIST_APP"

4
scripts/paperless-consumer.service
View File

@ -1,10 +1,12 @@
[Unit]
Description=Paperless consumer
Requires=redis.service
[Service]
User=paperless
Group=paperless
ExecStart=/home/paperless/project/virtualenv/bin/python /home/paperless/project/src/manage.py document_consumer
WorkingDirectory=/opt/paperless/src
ExecStart=python3 manage.py document_consumer
[Install]
WantedBy=multi-user.target

12
scripts/paperless-scheduler.service Normal file
View File

@ -0,0 +1,12 @@
[Unit]
Description=Paperless consumer
Requires=redis.service
[Service]
User=paperless
Group=paperless
WorkingDirectory=/opt/paperless/src
ExecStart=python3 manage.py qcluster
[Install]
WantedBy=multi-user.target

4
scripts/paperless-webserver.service
View File

@ -2,11 +2,13 @@
Description=Paperless webserver
After=network.target
Wants=network.target
Requires=redis.service
[Service]
User=paperless
Group=paperless
ExecStart=/home/paperless/project/virtualenv/bin/gunicorn --pythonpath=/home/paperless/project/src paperless.wsgi -w 2
WorkingDirectory=/opt/paperless/src
ExecStart=/opt/paperless/.local/bin/gunicorn paperless.wsgi -w 2 -b 0.0.0.0:8000
[Install]
WantedBy=multi-user.target

2
src-ui/src/app/components/common/tag/tag.component.html
View File

@ -1,2 +1,2 @@
<span *ngIf="!clickable" class="badge" [style.background]="getColour().value" [style.color]="getColour().textColor">{{tag.name}}</span>
<a [routerLink]="" *ngIf="clickable" class="badge" [style.background]="getColour().value" [style.color]="getColour().textColor">{{tag.name}}</a>
<a [routerLink]="" [title]="linkTitle" *ngIf="clickable" class="badge" [style.background]="getColour().value" [style.color]="getColour().textColor">{{tag.name}}</a>

6
src-ui/src/app/components/common/tag/tag.component.ts
View File

@ -14,10 +14,10 @@ export class TagComponent implements OnInit {
tag: PaperlessTag
@Input()
clickable: boolean = false
linkTitle: string = ""
@Output()
click = new EventEmitter()
@Input()
clickable: boolean = false
ngOnInit(): void {
}

8
src-ui/src/app/components/document-detail/document-detail.component.ts
View File

@ -1,4 +1,3 @@
import { DatePipe, formatDate } from '@angular/common';
import { Component, OnInit } from '@angular/core';
import { FormControl, FormGroup } from '@angular/forms';
import { ActivatedRoute, Router } from '@angular/router';
@ -7,17 +6,14 @@ import { PaperlessCorrespondent } from 'src/app/data/paperless-correspondent';
import { PaperlessDocument } from 'src/app/data/paperless-document';
import { PaperlessDocumentMetadata } from 'src/app/data/paperless-document-metadata';
import { PaperlessDocumentType } from 'src/app/data/paperless-document-type';
import { TAG_COLOURS, PaperlessTag } from 'src/app/data/paperless-tag';
import { DocumentListViewService } from 'src/app/services/document-list-view.service';
import { OpenDocumentsService } from 'src/app/services/open-documents.service';
import { CorrespondentService } from 'src/app/services/rest/correspondent.service';
import { DocumentTypeService } from 'src/app/services/rest/document-type.service';
import { DocumentService } from 'src/app/services/rest/document.service';
import { TagService } from 'src/app/services/rest/tag.service';
import { DeleteDialogComponent } from '../common/delete-dialog/delete-dialog.component';
import { CorrespondentEditDialogComponent } from '../manage/correspondent-list/correspondent-edit-dialog/correspondent-edit-dialog.component';
import { DocumentTypeEditDialogComponent } from '../manage/document-type-list/document-type-edit-dialog/document-type-edit-dialog.component';
import { TagEditDialogComponent } from '../manage/tag-list/tag-edit-dialog/tag-edit-dialog.component';
@Component({
selector: 'app-document-detail',
@ -140,8 +136,8 @@ export class DocumentDetailComponent implements OnInit {
close() {
this.openDocumentService.closeDocument(this.document)
if (this.documentListViewService.viewId) {
this.router.navigate(['view', this.documentListViewService.viewId])
if (this.documentListViewService.savedViewId) {
this.router.navigate(['view', this.documentListViewService.savedViewId])
} else {
this.router.navigate(['documents'])
}

14
src-ui/src/app/components/document-list/document-card-large/document-card-large.component.html
View File

@ -7,7 +7,12 @@
<div class="card-body">
<div class="d-flex justify-content-between align-items-center">
<h5 class="card-title">{{document.correspondent ? document.correspondent.name + ': ' : ''}}{{document.title}}<app-tag [tag]="t" *ngFor="let t of document.tags" class="ml-1"></app-tag></h5>
<h5 class="card-title">
<ng-container *ngIf="document.correspondent">
<a [routerLink]="" title="Filter by correspondent" (click)="clickCorrespondent.emit(document.correspondent)" class="font-weight-bold">{{document.correspondent.name}}</a>:
</ng-container>
{{document.title}}<app-tag [tag]="t" linkTitle="Filter by tag" *ngFor="let t of document.tags" class="ml-1" (click)="clickTag.emit(t)" [clickable]="true"></app-tag>
</h5>
<h5 class="card-title" *ngIf="document.archive_serial_number">#{{document.archive_serial_number}}</h5>
</div>
<p class="card-text">
@ -24,6 +29,13 @@
</svg>
Edit
</a>
<a type="button" class="btn btn-sm btn-outline-secondary" [href]="getPreviewUrl()">
<svg width="1em" height="1em" viewBox="0 0 16 16" class="bi bi-search" fill="currentColor" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" d="M10.442 10.442a1 1 0 0 1 1.415 0l3.85 3.85a1 1 0 0 1-1.414 1.415l-3.85-3.85a1 1 0 0 1 0-1.415z"/>
<path fill-rule="evenodd" d="M6.5 12a5.5 5.5 0 1 0 0-11 5.5 5.5 0 0 0 0 11zM13 6.5a6.5 6.5 0 1 1-13 0 6.5 6.5 0 0 1 13 0z"/>
</svg>
View
</a>
<a type="button" class="btn btn-sm btn-outline-secondary" [href]="getDownloadUrl()">
<svg width="1em" height="1em" viewBox="0 0 16 16" class="bi bi-download" fill="currentColor" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" d="M.5 9.9a.5.5 0 0 1 .5.5v2.5a1 1 0 0 0 1 1h12a1 1 0 0 0 1-1v-2.5a.5.5 0 0 1 1 0v2.5a2 2 0 0 1-2 2H2a2 2 0 0 1-2-2v-2.5a.5.5 0 0 1 .5-.5z"/>

13
src-ui/src/app/components/document-list/document-card-large/document-card-large.component.ts
View File

@ -1,6 +1,7 @@
import { Component, Input, OnInit } from '@angular/core';
import { Component, EventEmitter, Input, OnInit, Output } from '@angular/core';
import { DomSanitizer } from '@angular/platform-browser';
import { PaperlessDocument } from 'src/app/data/paperless-document';
import { PaperlessTag } from 'src/app/data/paperless-tag';
import { DocumentService } from 'src/app/services/rest/document.service';
@Component({
@ -18,6 +19,12 @@ export class DocumentCardLargeComponent implements OnInit {
@Input()
details: any
@Output()
clickTag = new EventEmitter<PaperlessTag>()
@Output()
clickCorrespondent = new EventEmitter<PaperlessDocument>()
ngOnInit(): void {
}
@ -41,4 +48,8 @@ export class DocumentCardLargeComponent implements OnInit {
getDownloadUrl() {
return this.documentService.getDownloadUrl(this.document.id)
}
getPreviewUrl() {
return this.documentService.getPreviewUrl(this.document.id)
}
}

18
src-ui/src/app/components/document-list/document-card-small/document-card-small.component.html
View File

@ -2,26 +2,34 @@
<div class="card h-100 shadow-sm">
<div class=" border-bottom doc-img pr-1" [ngStyle]="{'background-image': 'url(' + getThumbUrl() + ')'}">
<div class="row" *ngFor="let t of document.tags">
<app-tag [tag]="t" class="col text-right"></app-tag>
<app-tag style="font-size: large;" [tag]="t" class="col text-right" (click)="clickTag.emit(t)" [clickable]="true" linkTitle="Filter by tag"></app-tag>
</div>
</div>
<div class="card-body p-2">
<p class="card-text">
<span class="font-weight-bold">{{document.correspondent? document.correspondent.name + ': ' : ''}}</span> {{document.title}}
<ng-container *ngIf="document.correspondent">
<a [routerLink]="" title="Filter by correspondent" (click)="clickCorrespondent.emit(document.correspondent)" class="font-weight-bold">{{document.correspondent.name}}</a>:
</ng-container>
{{document.title}}
</p>
</div>
<div class="card-footer">
<div class="d-flex justify-content-between align-items-center ml-n2">
<div class="btn-group">
<a routerLink="/documents/{{document.id}}" class="btn btn-sm btn-outline-secondary">
<a routerLink="/documents/{{document.id}}" class="btn btn-sm btn-outline-secondary" title="Edit">
<svg width="1em" height="1em" viewBox="0 0 16 16" class="bi bi-pencil" fill="currentColor" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" d="M12.146.146a.5.5 0 0 1 .708 0l3 3a.5.5 0 0 1 0 .708l-10 10a.5.5 0 0 1-.168.11l-5 2a.5.5 0 0 1-.65-.65l2-5a.5.5 0 0 1 .11-.168l10-10zM11.207 2.5L13.5 4.793 14.793 3.5 12.5 1.207 11.207 2.5zm1.586 3L10.5 3.207 4 9.707V10h.5a.5.5 0 0 1 .5.5v.5h.5a.5.5 0 0 1 .5.5v.5h.293l6.5-6.5zm-9.761 5.175l-.106.106-1.528 3.821 3.821-1.528.106-.106A.5.5 0 0 1 5 12.5V12h-.5a.5.5 0 0 1-.5-.5V11h-.5a.5.5 0 0 1-.468-.325z"/>
</svg>
</a>
<a [href]="getDownloadUrl()" class="btn btn-sm btn-outline-secondary">
<a [href]="getPreviewUrl()" class="btn btn-sm btn-outline-secondary" title="View in browser">
<svg width="1em" height="1em" viewBox="0 0 16 16" class="bi bi-search" fill="currentColor" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" d="M10.442 10.442a1 1 0 0 1 1.415 0l3.85 3.85a1 1 0 0 1-1.414 1.415l-3.85-3.85a1 1 0 0 1 0-1.415z"/>
<path fill-rule="evenodd" d="M6.5 12a5.5 5.5 0 1 0 0-11 5.5 5.5 0 0 0 0 11zM13 6.5a6.5 6.5 0 1 1-13 0 6.5 6.5 0 0 1 13 0z"/>
</svg>
</a>
<a [href]="getDownloadUrl()" class="btn btn-sm btn-outline-secondary" title="Download">
<svg width="1em" height="1em" viewBox="0 0 16 16" class="bi bi-download" fill="currentColor" xmlns="http://www.w3.org/2000/svg">
<path fill-rule="evenodd" d="M.5 9.9a.5.5 0 0 1 .5.5v2.5a1 1 0 0 0 1 1h12a1 1 0 0 0 1-1v-2.5a.5.5 0 0 1 1 0v2.5a2 2 0 0 1-2 2H2a2 2 0 0 1-2-2v-2.5a.5.5 0 0 1 .5-.5z"/>
<path fill-rule="evenodd" d="M7.646 11.854a.5.5 0 0 0 .708 0l3-3a.5.5 0 0 0-.708-.708L8.5 10.293V1.5a.5.5 0 0 0-1 0v8.793L5.354 8.146a.5.5 0 1 0-.708.708l3 3z"/>

13
src-ui/src/app/components/document-list/document-card-small/document-card-small.component.ts
View File

@ -1,5 +1,6 @@
import { Component, Input, OnInit } from '@angular/core';
import { Component, EventEmitter, Input, OnInit, Output } from '@angular/core';
import { PaperlessDocument } from 'src/app/data/paperless-document';
import { PaperlessTag } from 'src/app/data/paperless-tag';
import { DocumentService } from 'src/app/services/rest/document.service';
@Component({
@ -14,6 +15,12 @@ export class DocumentCardSmallComponent implements OnInit {
@Input()
document: PaperlessDocument
@Output()
clickTag = new EventEmitter<PaperlessTag>()
@Output()
clickCorrespondent = new EventEmitter<PaperlessDocument>()
ngOnInit(): void {
}
@ -24,4 +31,8 @@ export class DocumentCardSmallComponent implements OnInit {
getDownloadUrl() {
return this.documentService.getDownloadUrl(this.document.id)
}
getPreviewUrl() {
return this.documentService.getPreviewUrl(this.document.id)
}
}

62
src-ui/src/app/components/document-list/document-list.component.html
View File

@ -21,13 +21,12 @@
</svg>
</label>
</div>
<div class="btn-group btn-group-toggle ml-2" ngbRadioGroup [(ngModel)]="docs.sortDirection"
*ngIf="!docs.viewId">
<div class="btn-group btn-group-toggle ml-2" ngbRadioGroup [(ngModel)]="list.sortDirection">
<div ngbDropdown class="btn-group">
<button class="btn btn-outline-primary btn-sm" id="dropdownBasic1" ngbDropdownToggle>Sort by</button>
<div ngbDropdownMenu aria-labelledby="dropdownBasic1">
<button *ngFor="let f of getSortFields()" ngbDropdownItem (click)="setSort(f.field)"
[class.active]="docs.sortField == f.field">{{f.name}}</button>
<button *ngFor="let f of getSortFields()" ngbDropdownItem (click)="list.sortField = f.field"
[class.active]="list.sortField == f.field">{{f.name}}</button>
</div>
</div>
<label ngbButtonLabel class="btn-outline-primary btn-sm">
@ -43,7 +42,7 @@
</svg>
</label>
</div>
<div class="btn-group ml-2" *ngIf="!docs.viewId">
<div class="btn-group ml-2">
<button type="button" class="btn btn-sm btn-outline-primary" (click)="showFilter=!showFilter">
<svg class="toolbaricon" fill="currentColor">
@ -55,9 +54,13 @@
<div class="btn-group" ngbDropdown role="group">
<button class="btn btn-sm btn-outline-primary dropdown-toggle-split" ngbDropdownToggle></button>
<div class="dropdown-menu" ngbDropdownMenu>
<button ngbDropdownItem *ngFor="let config of savedViewConfigService.getConfigs()" (click)="loadViewConfig(config)">{{config.title}}</button>
<div class="dropdown-divider" *ngIf="savedViewConfigService.getConfigs().length > 0"></div>
<button ngbDropdownItem (click)="saveViewConfig()">Save current view</button>
<ng-container *ngIf="!list.savedViewId" >
<button ngbDropdownItem *ngFor="let config of savedViewConfigService.getConfigs()" (click)="loadViewConfig(config)">{{config.title}}</button>
<div class="dropdown-divider" *ngIf="savedViewConfigService.getConfigs().length > 0"></div>
</ng-container>
<button ngbDropdownItem (click)="saveViewConfig()" *ngIf="list.savedViewId">Save "{{list.savedViewTitle}}"</button>
<button ngbDropdownItem (click)="saveViewConfigAs()">Save as...</button>
</div>
</div>
@ -72,16 +75,16 @@
</div>
<div class="row m-0 justify-content-end">
<ngb-pagination [pageSize]="docs.currentPageSize" [collectionSize]="docs.collectionSize" [(page)]="docs.currentPage" [maxSize]="5"
[rotate]="true" (pageChange)="reload()" aria-label="Default pagination"></ngb-pagination>
<ngb-pagination [pageSize]="list.currentPageSize" [collectionSize]="list.collectionSize" [(page)]="list.currentPage" [maxSize]="5"
[rotate]="true" (pageChange)="list.reload()" aria-label="Default pagination"></ngb-pagination>
</div>
<div *ngIf="displayMode == 'largeCards'">
<app-document-card-large *ngFor="let d of docs.documents" [document]="d" [details]="d.content">
<app-document-card-large *ngFor="let d of list.documents" [document]="d" [details]="d.content" (clickTag)="filterByTag($event)" (clickCorrespondent)="filterByCorrespondent($event)">
</app-document-card-large>
</div>
<table class="table table-hover table-sm border shadow" *ngIf="displayMode == 'details'">
<table class="table table-sm border shadow" *ngIf="displayMode == 'details'">
<thead>
<th class="d-none d-lg-table-cell">ASN</th>
<th class="d-none d-md-table-cell">Correspondent</th>
@ -91,20 +94,37 @@
<th class="d-none d-xl-table-cell">Added</th>
</thead>
<tbody>
<tr *ngFor="let d of docs.documents" routerLink="/documents/{{d.id}}">
<td class="d-none d-lg-table-cell">{{d.archive_serial_number}}</td>
<td class="d-none d-md-table-cell">{{d.correspondent ? d.correspondent.name : ''}}</td>
<td>{{d.title}}<app-tag [tag]="t" *ngFor="let t of d.tags" class="ml-1"></app-tag></td>
<td class="d-none d-xl-table-cell">{{d.document_type ? d.document_type.name : ''}}</td>
<td>{{d.created | date}}</td>
<td class="d-none d-xl-table-cell">{{d.added | date}}</td>
<tr *ngFor="let d of list.documents">
<td class="d-none d-lg-table-cell">
{{d.archive_serial_number}}
</td>
<td class="d-none d-md-table-cell">
<ng-container *ngIf="d.correspondent">
<a [routerLink]="" (click)="filterByCorrespondent(d.correspondent)" title="Filter by correspondent">{{d.correspondent.name}}</a>
</ng-container>
</td>
<td>
<a routerLink="/documents/{{d.id}}" title="Edit document">{{d.title}}</a>
<app-tag [tag]="t" *ngFor="let t of d.tags" class="ml-1" clickable="true" linkTitle="Filter by tag" (click)="filterByTag(t)"></app-tag>
</td>
<td class="d-none d-xl-table-cell">
<ng-container *ngIf="d.document_type">
<a [routerLink]="" (click)="filterByDocumentType(d.document_type)" title="Filter by document type">{{d.document_type.name}}</a>
</ng-container>
</td>
<td>
{{d.created | date}}
</td>
<td class="d-none d-xl-table-cell">
{{d.added | date}}
</td>
</tr>
</tbody>
</table>
<div class=" m-n2 row" *ngIf="displayMode == 'smallCards'">
<app-document-card-small [document]="d" *ngFor="let d of docs.documents"></app-document-card-small>
<app-document-card-small [document]="d" *ngFor="let d of list.documents" (clickTag)="filterByTag($event)" (clickCorrespondent)="filterByCorrespondent($event)"></app-document-card-small>
</div>
<p *ngIf="docs.documents.length == 0" class="mx-auto">No results</p>
<p *ngIf="list.documents.length == 0" class="mx-auto">No results</p>

79
src-ui/src/app/components/document-list/document-list.component.ts
View File

@ -1,11 +1,16 @@
import { Component, OnInit } from '@angular/core';
import { ActivatedRoute, Router } from '@angular/router';
import { ActivatedRoute } from '@angular/router';
import { NgbModal } from '@ng-bootstrap/ng-bootstrap';
import { cloneFilterRules, FilterRule } from 'src/app/data/filter-rule';
import { FILTER_CORRESPONDENT, FILTER_DOCUMENT_TYPE, FILTER_HAS_TAG, FILTER_RULE_TYPES } from 'src/app/data/filter-rule-type';
import { PaperlessCorrespondent } from 'src/app/data/paperless-correspondent';
import { PaperlessDocumentType } from 'src/app/data/paperless-document-type';
import { PaperlessTag } from 'src/app/data/paperless-tag';
import { SavedViewConfig } from 'src/app/data/saved-view-config';
import { DocumentListViewService } from 'src/app/services/document-list-view.service';
import { DOCUMENT_SORT_FIELDS } from 'src/app/services/rest/document.service';
import { SavedViewConfigService } from 'src/app/services/saved-view-config.service';
import { Toast, ToastService } from 'src/app/services/toast.service';
import { SaveViewConfigDialogComponent } from './save-view-config-dialog/save-view-config-dialog.component';
@Component({
@ -16,9 +21,10 @@ import { SaveViewConfigDialogComponent } from './save-view-config-dialog/save-vi
export class DocumentListComponent implements OnInit {
constructor(
public docs: DocumentListViewService,
public list: DocumentListViewService,
public savedViewConfigService: SavedViewConfigService,
public route: ActivatedRoute,
private toastService: ToastService,
public modalService: NgbModal) { }
displayMode = 'smallCards' // largeCards, smallCards, details
@ -27,17 +33,13 @@ export class DocumentListComponent implements OnInit {
showFilter = false
getTitle() {
return this.docs.viewConfigOverride ? this.docs.viewConfigOverride.title : "Documents"
return this.list.savedViewTitle || "Documents"
}
getSortFields() {
return DOCUMENT_SORT_FIELDS
}
setSort(field: string) {
this.docs.sortField = field
}
saveDisplayMode() {
localStorage.setItem('document-list:displayMode', this.displayMode)
}
@ -48,41 +50,74 @@ export class DocumentListComponent implements OnInit {
}
this.route.paramMap.subscribe(params => {
if (params.has('id')) {
this.docs.viewConfigOverride = this.savedViewConfigService.getConfig(params.get('id'))
this.list.savedView = this.savedViewConfigService.getConfig(params.get('id'))
} else {
this.filterRules = this.docs.filterRules
this.showFilter = this.filterRules.length > 0
this.docs.viewConfigOverride = null
this.list.savedView = null
}
this.reload()
this.filterRules = this.list.filterRules
//this.showFilter = this.filterRules.length > 0
// prevents temporarily visible results from previous views
this.list.documents = []
this.list.reload()
})
}
reload() {
this.docs.reload()
}
applyFilterRules() {
this.docs.filterRules = this.filterRules
this.list.filterRules = this.filterRules
}
loadViewConfig(config: SavedViewConfig) {
this.filterRules = cloneFilterRules(config.filterRules)
this.docs.loadViewConfig(config)
this.list.load(config)
}
saveViewConfig() {
this.savedViewConfigService.updateConfig(this.list.savedView)
this.toastService.showToast(Toast.make("Information", `View "${this.list.savedView.title}" saved successfully.`))
}
saveViewConfigAs() {
let modal = this.modalService.open(SaveViewConfigDialogComponent, {backdrop: 'static'})
modal.componentInstance.saveClicked.subscribe(formValue => {
this.savedViewConfigService.saveConfig({
this.savedViewConfigService.newConfig({
title: formValue.title,
showInDashboard: formValue.showInDashboard,
showInSideBar: formValue.showInSideBar,
filterRules: this.docs.filterRules,
sortDirection: this.docs.sortDirection,
sortField: this.docs.sortField
filterRules: this.list.filterRules,
sortDirection: this.list.sortDirection,
sortField: this.list.sortField
})
modal.close()
})
}
filterByTag(t: PaperlessTag) {
if (this.filterRules.find(rule => rule.type.id == FILTER_HAS_TAG && rule.value == t.id)) {
return
}
this.filterRules.push({type: FILTER_RULE_TYPES.find(t => t.id == FILTER_HAS_TAG), value: t.id})
this.applyFilterRules()
}
filterByCorrespondent(c: PaperlessCorrespondent) {
let existing_rule = this.filterRules.find(rule => rule.type.id == FILTER_CORRESPONDENT)
if (existing_rule) {
existing_rule.value = c.id
} else {
this.filterRules.push({type: FILTER_RULE_TYPES.find(t => t.id == FILTER_CORRESPONDENT), value: c.id})
}
this.applyFilterRules()
}
filterByDocumentType(dt: PaperlessDocumentType) {
let existing_rule = this.filterRules.find(rule => rule.type.id == FILTER_DOCUMENT_TYPE)
if (existing_rule) {
existing_rule.value = dt.id
} else {
this.filterRules.push({type: FILTER_RULE_TYPES.find(t => t.id == FILTER_DOCUMENT_TYPE), value: dt.id})
}
this.applyFilterRules()
}
}

2
src-ui/src/app/components/search/search.component.scss
View File

@ -11,5 +11,5 @@
}
.result-content-searching {
opacity: 0.2;
opacity: 0.3;
}

54
src-ui/src/app/data/filter-rule-type.ts
View File

@ -1,31 +1,51 @@
export const FILTER_TITLE = 0
export const FILTER_CONTENT = 1
export const FILTER_ASN = 2
export const FILTER_CORRESPONDENT = 3
export const FILTER_DOCUMENT_TYPE = 4
export const FILTER_IS_IN_INBOX = 5
export const FILTER_HAS_TAG = 6
export const FILTER_HAS_ANY_TAG = 7
export const FILTER_CREATED_BEFORE = 8
export const FILTER_CREATED_AFTER = 9
export const FILTER_CREATED_YEAR = 10
export const FILTER_CREATED_MONTH = 11
export const FILTER_CREATED_DAY = 12
export const FILTER_ADDED_BEFORE = 13
export const FILTER_ADDED_AFTER = 14
export const FILTER_MODIFIED_BEFORE = 15
export const FILTER_MODIFIED_AFTER = 16
export const FILTER_RULE_TYPES: FilterRuleType[] = [
{name: "Title contains", filtervar: "title__icontains", datatype: "string", multi: false},
{name: "Content contains", filtervar: "content__icontains", datatype: "string", multi: false},
{id: FILTER_TITLE, name: "Title contains", filtervar: "title__icontains", datatype: "string", multi: false},
{id: FILTER_CONTENT, name: "Content contains", filtervar: "content__icontains", datatype: "string", multi: false},
{name: "ASN is", filtervar: "archive_serial_number", datatype: "number", multi: false},
{id: FILTER_ASN, name: "ASN is", filtervar: "archive_serial_number", datatype: "number", multi: false},
{name: "Correspondent is", filtervar: "correspondent__id", datatype: "correspondent", multi: false},
{name: "Document type is", filtervar: "document_type__id", datatype: "document_type", multi: false},
{id: FILTER_CORRESPONDENT, name: "Correspondent is", filtervar: "correspondent__id", datatype: "correspondent", multi: false},
{id: FILTER_DOCUMENT_TYPE, name: "Document type is", filtervar: "document_type__id", datatype: "document_type", multi: false},
{name: "Is in Inbox", filtervar: "is_in_inbox", datatype: "boolean", multi: false},
{name: "Has tag", filtervar: "tags__id__all", datatype: "tag", multi: true},
{name: "Has any tag", filtervar: "is_tagged", datatype: "boolean", multi: false},
{id: FILTER_IS_IN_INBOX, name: "Is in Inbox", filtervar: "is_in_inbox", datatype: "boolean", multi: false},
{id: FILTER_HAS_TAG, name: "Has tag", filtervar: "tags__id__all", datatype: "tag", multi: true},
{id: FILTER_HAS_ANY_TAG, name: "Has any tag", filtervar: "is_tagged", datatype: "boolean", multi: false},
{name: "Created before", filtervar: "created__date__lt", datatype: "date", multi: false},
{name: "Created after", filtervar: "created__date__gt", datatype: "date", multi: false},
{id: FILTER_CREATED_BEFORE, name: "Created before", filtervar: "created__date__lt", datatype: "date", multi: false},
{id: FILTER_CREATED_AFTER, name: "Created after", filtervar: "created__date__gt", datatype: "date", multi: false},
{name: "Year created is", filtervar: "created__year", datatype: "number", multi: false},
{name: "Month created is", filtervar: "created__month", datatype: "number", multi: false},
{name: "Day created is", filtervar: "created__day", datatype: "number", multi: false},
{id: FILTER_CREATED_YEAR, name: "Year created is", filtervar: "created__year", datatype: "number", multi: false},
{id: FILTER_CREATED_MONTH, name: "Month created is", filtervar: "created__month", datatype: "number", multi: false},
{id: FILTER_CREATED_DAY, name: "Day created is", filtervar: "created__day", datatype: "number", multi: false},
{name: "Added before", filtervar: "added__date__lt", datatype: "date", multi: false},
{name: "Added after", filtervar: "added__date__gt", datatype: "date", multi: false},
{id: FILTER_ADDED_BEFORE, name: "Added before", filtervar: "added__date__lt", datatype: "date", multi: false},
{id: FILTER_ADDED_AFTER, name: "Added after", filtervar: "added__date__gt", datatype: "date", multi: false},
{name: "Modified before", filtervar: "modified__date__lt", datatype: "date", multi: false},
{name: "Modified after", filtervar: "modified__date__gt", datatype: "date", multi: false},
{id: FILTER_MODIFIED_BEFORE, name: "Modified before", filtervar: "modified__date__lt", datatype: "date", multi: false},
{id: FILTER_MODIFIED_AFTER, name: "Modified after", filtervar: "modified__date__gt", datatype: "date", multi: false},
]
export interface FilterRuleType {
id: number
name: string
filtervar: string
datatype: string //number, string, boolean, date

119
src-ui/src/app/services/document-list-view.service.ts
View File

@ -7,6 +7,12 @@ import { DOCUMENT_LIST_SERVICE, GENERAL_SETTINGS } from '../data/storage-keys';
import { DocumentService } from './rest/document.service';
/**
* This service manages the document list which is displayed using the document list view.
*
* This service also serves saved views by transparently switching between the document list
* and saved views on request. See below.
*/
@Injectable({
providedIn: 'root'
})
@ -14,80 +20,127 @@ export class DocumentListViewService {
static DEFAULT_SORT_FIELD = 'created'
isReloading: boolean = false
documents: PaperlessDocument[] = []
currentPage = 1
currentPageSize: number = +localStorage.getItem(GENERAL_SETTINGS.DOCUMENT_LIST_SIZE) || GENERAL_SETTINGS.DOCUMENT_LIST_SIZE_DEFAULT
collectionSize: number
private currentViewConfig: SavedViewConfig
//TODO: make private
viewConfigOverride: SavedViewConfig
/**
* This is the current config for the document list. The service will always remember the last settings used for the document list.
*/
private _documentListViewConfig: SavedViewConfig
/**
* Optionally, this is the currently selected saved view, which might be null.
*/
private _savedViewConfig: SavedViewConfig
get viewId() {
return this.viewConfigOverride?.id
get savedView() {
return this._savedViewConfig
}
set savedView(value) {
if (value) {
//this is here so that we don't modify value, which might be the actual instance of the saved view.
this._savedViewConfig = Object.assign({}, value)
} else {
this._savedViewConfig = null
}
}
get savedViewId() {
return this.savedView?.id
}
get savedViewTitle() {
return this.savedView?.title
}
get documentListView() {
return this._documentListViewConfig
}
set documentListView(value) {
if (value) {
this._documentListViewConfig = Object.assign({}, value)
this.saveDocumentListView()
}
}
/**
* This is what switches between the saved views and the document list view. Everything on the document list uses
* this property to determine the settings for the currently displayed document list.
*/
get view() {
return this.savedView || this.documentListView
}
load(config: SavedViewConfig) {
this.view.filterRules = cloneFilterRules(config.filterRules)
this.view.sortDirection = config.sortDirection
this.view.sortField = config.sortField
this.reload()
}
reload(onFinish?) {
let viewConfig = this.viewConfigOverride || this.currentViewConfig
this.isReloading = true
this.documentService.list(
this.currentPage,
this.currentPageSize,
viewConfig.sortField,
viewConfig.sortDirection,
viewConfig.filterRules).subscribe(
this.view.sortField,
this.view.sortDirection,
this.view.filterRules).subscribe(
result => {
this.collectionSize = result.count
this.documents = result.results
if (onFinish) {
onFinish()
}
this.isReloading = false
},
error => {
if (error.error['detail'] == 'Invalid page.') {
this.currentPage = 1
this.reload()
}
this.isReloading = false
})
}
set filterRules(filterRules: FilterRule[]) {
this.currentViewConfig.filterRules = cloneFilterRules(filterRules)
this.saveCurrentViewConfig()
//we're going to clone the filterRules object, since we don't
//want changes in the filter editor to propagate into here right away.
this.view.filterRules = cloneFilterRules(filterRules)
this.reload()
this.saveDocumentListView()
}
get filterRules(): FilterRule[] {
return cloneFilterRules(this.currentViewConfig.filterRules)
return cloneFilterRules(this.view.filterRules)
}
set sortField(field: string) {
this.currentViewConfig.sortField = field
this.saveCurrentViewConfig()
this.view.sortField = field
this.saveDocumentListView()
this.reload()
}
get sortField(): string {
return this.currentViewConfig.sortField
return this.view.sortField
}
set sortDirection(direction: string) {
this.currentViewConfig.sortDirection = direction
this.saveCurrentViewConfig()
this.view.sortDirection = direction
this.saveDocumentListView()
this.reload()
}
get sortDirection(): string {
return this.currentViewConfig.sortDirection
return this.view.sortDirection
}
loadViewConfig(config: SavedViewConfig) {
Object.assign(this.currentViewConfig, config)
this.reload()
}
private saveCurrentViewConfig() {
sessionStorage.setItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG, JSON.stringify(this.currentViewConfig))
private saveDocumentListView() {
sessionStorage.setItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG, JSON.stringify(this.documentListView))
}
getLastPage(): number {
@ -134,21 +187,21 @@ export class DocumentListViewService {
}
constructor(private documentService: DocumentService) {
let currentViewConfigJson = sessionStorage.getItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG)
if (currentViewConfigJson) {
let documentListViewConfigJson = sessionStorage.getItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG)
if (documentListViewConfigJson) {
try {
this.currentViewConfig = JSON.parse(currentViewConfigJson)
this.documentListView = JSON.parse(documentListViewConfigJson)
} catch (e) {
sessionStorage.removeItem(DOCUMENT_LIST_SERVICE.CURRENT_VIEW_CONFIG)
this.currentViewConfig = null
this.documentListView = null
}
}
if (!this.currentViewConfig) {
this.currentViewConfig = {
if (!this.documentListView) {
this.documentListView = {
filterRules: [],
sortDirection: 'des',
sortField: 'created'
}
}
}
}
}

10
src-ui/src/app/services/saved-view-config.service.ts
View File

@ -36,13 +36,21 @@ export class SavedViewConfigService {
return this.configs.find(sf => sf.id == id)
}
saveConfig(config: SavedViewConfig) {
newConfig(config: SavedViewConfig) {
config.id = uuidv4()
this.configs.push(config)
this.save()
}
updateConfig(config: SavedViewConfig) {
let savedConfig = this.configs.find(c => c.id == config.id)
if (savedConfig) {
Object.assign(savedConfig, config)
this.save()
}
}
private save() {
localStorage.setItem('saved-view-config-service:savedConfigs', JSON.stringify(this.configs))
}

9
src/documents/signals/handlers.py
View File

@ -9,6 +9,7 @@ from django.contrib.contenttypes.models import ContentType
from django.db import models, DatabaseError
from django.dispatch import receiver
from django.utils import timezone
from rest_framework.reverse import reverse
from .. import index, matching
from ..file_handling import delete_empty_directories, generate_filename, \
@ -157,10 +158,10 @@ def run_post_consume_script(sender, document, **kwargs):
settings.POST_CONSUME_SCRIPT,
str(document.pk),
document.file_name,
document.source_path,
document.thumbnail_path,
None,
None,
os.path.normpath(document.source_path),
os.path.normpath(document.thumbnail_path),
reverse("document-download", kwargs={"pk": document.pk}),
reverse("document-thumb", kwargs={"pk": document.pk}),
str(document.correspondent),
str(",".join(document.tags.all().values_list("slug", flat=True)))
)).wait()

57
src/documents/tests/test_post_consume_handlers.py Normal file
View File

@ -0,0 +1,57 @@
from unittest import mock
from django.test import TestCase, override_settings
from documents.models import Document, Tag, Correspondent
from documents.signals.handlers import run_post_consume_script
class PostConsumeTestCase(TestCase):
@mock.patch("documents.signals.handlers.Popen")
@override_settings(POST_CONSUME_SCRIPT=None)
def test_no_post_consume_script(self, m):
doc = Document.objects.create(title="Test", mime_type="application/pdf")
tag1 = Tag.objects.create(name="a")
tag2 = Tag.objects.create(name="b")
doc.tags.add(tag1)
doc.tags.add(tag2)
run_post_consume_script(None, doc)
m.assert_not_called()
@mock.patch("documents.signals.handlers.Popen")
@override_settings(POST_CONSUME_SCRIPT="script")
def test_post_consume_script_simple(self, m):
doc = Document.objects.create(title="Test", mime_type="application/pdf")
run_post_consume_script(None, doc)
m.assert_called_once()
@mock.patch("documents.signals.handlers.Popen")
@override_settings(POST_CONSUME_SCRIPT="script")
def test_post_consume_script_simple(self, m):
c = Correspondent.objects.create(name="my_bank")
doc = Document.objects.create(title="Test", mime_type="application/pdf", correspondent=c)
tag1 = Tag.objects.create(name="a")
tag2 = Tag.objects.create(name="b")
doc.tags.add(tag1)
doc.tags.add(tag2)
run_post_consume_script(None, doc)
m.assert_called_once()
args, kwargs = m.call_args
command = args[0]
self.assertEqual(command[0], "script")
self.assertEqual(command[1], str(doc.pk))
self.assertEqual(command[5], f"/api/documents/{doc.pk}/download/")
self.assertEqual(command[6], f"/api/documents/{doc.pk}/thumb/")
self.assertEqual(command[7], "my_bank")
# TODO: tags are unordered by default.
self.assertEqual(command[8], "a,b")

13
src/paperless/db.py
View File

@ -17,16 +17,3 @@ class GnuPG:
passphrase = settings.PASSPHRASE
return cls.gpg.decrypt_file(file_handle, passphrase=passphrase).data
@classmethod
def encrypted(cls, file_handle, passphrase=None):
if not passphrase:
passphrase = settings.PASSPHRASE
return cls.gpg.encrypt_file(
file_handle,
recipients=None,
passphrase=passphrase,
symmetric=True
).data