543 Commits

Author SHA1 Message Date
Erik Arvstedt
88a05947f7 Update Consumer class documentation 2018-06-17 20:17:40 +01:00
Daniel Quinn
5f3212b6a3 Put imports at the top 2018-06-17 20:14:46 +01:00
Erik Arvstedt
0790e77df6 Improve password check
1. Fail when the db contains encrypted docs and no password is set.
   Previously, this case wasn't detected.

2. Exit with an error instead of showing warnings.
   This ensures that we never store docs with different encryption passwords.
2018-06-17 20:07:32 +01:00
Daniel Quinn
e7e69d3f6f Remove emoji from storage-type changer 2018-06-17 17:23:50 +01:00
Daniel Quinn
d566a014d2 Drop lines thanks to @erikarvstedt's eagle-eye 2018-06-17 17:10:45 +01:00
Daniel Quinn
044d707c40 Update import & export to handle encryption toggle 2018-06-17 17:06:22 +01:00
Daniel Quinn
0b32e3d861 Fix migrations 2018-06-17 16:47:38 +01:00
Daniel Quinn
e7fefc40fe Merge branch 'master' into mcronce-disable_encryption 2018-06-17 16:32:51 +01:00
Erik Arvstedt
d0a5d750db Fix unclickable checkbox in documents view
1. Clicks to the document selection checkbox were captured by the onclick
   handler of the document item header. This is now fixed.

2. Reexpose the doc title link to mouse events by putting it on top of
   the header link layer.
2018-06-01 14:07:34 +02:00
Daniel Quinn
c8829186d2 Merge branch 'document_field_added' of git://github.com/erikarvstedt/paperless into erikarvstedt-document_field_added 2018-06-01 07:51:44 +01:00
Erik Arvstedt
67b4cfaa83 Fix incompatibility with Python versions < 3.6
Direct index access to a match was only added in 3.6.

Fixes #359
2018-06-01 00:45:59 +02:00
Erik Arvstedt
33e3277d2a Add field 'added' to documents
This field indicates when the document was added to the database
2018-05-31 10:17:03 +02:00
Daniel Quinn
d1b6e9329f It's exist_ok=, not exists_ok= -- my bad. 2018-05-28 13:08:00 +01:00
Daniel Quinn
26a471f5e0 Merge branch 'inotify' of git://github.com/erikarvstedt/paperless into erikarvstedt-inotify 2018-05-28 13:03:06 +01:00
Daniel Quinn
5d6e30bf47 Don't check changed passphrase if no passphrase set 2018-05-28 12:58:28 +01:00
Daniel Quinn
c8e711322e Remove old Python2.7-style code 2018-05-28 12:58:28 +01:00
Daniel Quinn
4576541c28 Add script to (de|en)crypt all documents 2018-05-28 12:58:28 +01:00
Daniel Quinn
a9382ffd1a Drop STORAGE_TYPE in favour of just using PAPERLESS_PASSPHRASE 2018-05-28 12:58:28 +01:00
Daniel Quinn
c18901df5a Change default storage_type to unencrypted 2018-05-28 12:58:28 +01:00
Daniel Quinn
afe3bfe23a Don't run document checks if table doesn't exist yet 2018-05-28 12:58:28 +01:00
Daniel Quinn
f5a198d96e Fix migration conflict 2018-05-28 12:58:28 +01:00
Daniel Quinn
3ec1dd656e Add check for changed password
These tests are incomplete, but I have no idea how to write the other
half.
2018-05-28 12:58:28 +01:00
Daniel Quinn
e9fd22059b Add "fat finger" check to password status 2018-05-28 12:58:28 +01:00
Daniel Quinn
27b541c330 Remove checks we weren't using 2018-05-28 12:58:28 +01:00
Daniel Quinn
6b123daf28 Only require a passphrase if STORAGE_TYPE is not "unencrypted" 2018-05-28 12:58:28 +01:00
Daniel Quinn
92d9506a2e Make the consumer aware of the different storage types 2018-05-28 12:58:28 +01:00
Daniel Quinn
3db9544c7f Move the encrypt/decrypt decision out of db and into the view 2018-05-28 12:58:28 +01:00
Daniel Quinn
e3eb95eb47 Attach storage_type to Documents 2018-05-28 12:58:28 +01:00
Daniel Quinn
2309076038 Set STORAGE_TYPE instead of ENABLE_ENCRYPTION boolean
This allows for future decisions around the types of encryption used (if any).  Ideally, I want to replace GPG one day with something elegant out of the cryptography module.
2018-05-28 12:58:28 +01:00
Daniel Quinn
4925ecd1da Try to be more pep8 in the settings file 2018-05-28 12:58:28 +01:00
Mike Cronce
24e56c0899 src/paperless/db.py: If encryption is disabled, just directly read the file contents 2018-05-28 12:58:28 +01:00
Mike Cronce
d2d9804d01 src/manage.py: Added check to see whether or not encryption is enabled before prompting for passphrase if it's empty 2018-05-28 12:58:28 +01:00
Mike Cronce
8df12400b3 src/paperless/settings.py: Added DISABLE_ENCRYPTION environment variable 2018-05-28 12:58:28 +01:00
Daniel Quinn
a808d5b483 Fix redirect for subpaths (hopefully) 2018-05-28 12:56:20 +01:00
Daniel Quinn
dac59af437 Code clean up 2018-05-28 12:56:06 +01:00
Erik Arvstedt
901a810e97 fixup: require usage of PAPERLESS_EMAIL_SECRET 2018-05-21 12:11:56 +02:00
Erik Arvstedt
d132e2b9f5 fixup: remove helper fn 'make_dirs' 2018-05-21 00:45:00 +02:00
Erik Arvstedt
1eeae1df00 fixup: break up complex if condition 2018-05-21 00:44:58 +02:00
Daniel Quinn
3ac2e32bcf Allow the searching of documents by tag #354 2018-05-20 17:28:00 +01:00
Daniel Quinn
999475e4c6 Add hack to allow for logentries to show for all users. 2018-05-20 16:29:00 +01:00
Erik Arvstedt
3db175dfe2 Add inotify support 2018-05-11 14:14:50 +02:00
Erik Arvstedt
b74b47423d Consumer loop: make sleep duration dynamic
Make the sleep duration dynamic to account for the time spent in
loop_step.
This improves responsiveness when repeatedly consuming newly
arriving docs.

Use float epoch seconds (time.time()) as the time type for
MailFetcher.last_checked to allow for natural time arithmetic.
2018-05-11 14:14:50 +02:00
Erik Arvstedt
aac17670de Refactor: renamings, extract fn 'loop'
Renamings:
loop -> loop_step
delta -> next_mail_time (this variable names a point in time, not a duration)

Extracting the 'loop' fn is a preparation for later commits where a
second type of loop is added.
2018-05-11 14:14:25 +02:00
Erik Arvstedt
8b37af994a Consider mtime of ignored files, garbage-collect ignore list
1. Store the mtime of ignored files so that we can reconsider them if
they have changed.

2. Regularly reset the ignore list to files that still exist in the
consumption dir. Previously, the list could grow indefinitely.
2018-05-11 14:05:30 +02:00
Erik Arvstedt
cc22204e5a Simplify ignoring docs 2018-05-11 14:05:29 +02:00
Erik Arvstedt
f56ec70aad Ensure docs have been unmodified for some time before consuming
Previously, the second mtime check for new files usually happened right
after the first one, which could have caused consumption of docs that
were still being modified.

We're now waiting for at least FILES_MIN_UNMODIFIED_DURATION (0.5s).

This also cleans up the logic by eliminating the consumer.stats attribute
and the weird double call to consumer.run().

Additionally, this a fixes memory leak in consumer.stats where paths could be
added but never removed if the corresponding files disappeared from
the consumer dir before being considered ready.
2018-05-11 14:05:29 +02:00
Erik Arvstedt
0db6ed225b Refactor: extract fn try_consume_file
The main purpose of this change is to make the following commits more
readable.
2018-05-11 14:05:28 +02:00
Erik Arvstedt
312a6a91b5 Use os.scandir instead of os.listdir
It's simpler and better suited for use cases introduced in later commits.
2018-05-11 14:05:25 +02:00
Erik Arvstedt
2c64e70754 Consume documents in order of increasing mtime
This increases overall usability, especially for multi-page scans.
Previously, the consumption order was undefined (see os.listdir())
2018-05-11 14:04:37 +02:00
Erik Arvstedt
9320230100 Refactor: extract fn 'make_dirs' 2018-05-11 14:04:36 +02:00