paradizelost/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2025-04-02 13:45:10 -05:00
Code Issues Packages Projects Releases Wiki Activity

Feature: OAuth2 Gmail and Outlook email support (#7866)

Browse Source
This commit is contained in:
shamoon 2024-10-10 13:57:32 -07:00 committed by GitHub
parent dcc8d4046a
commit 2353f7c2db
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
30 changed files with 1110 additions and 121 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
Pipfile
View File

@ -30,8 +30,10 @@ filelock = "*"
flower = "*"
gotenberg-client = "*"
gunicorn = "*"
httpx-oauth = "*"
imap-tools = "*"
inotifyrecursive = "~=0.3"
jinja2 = "~=3.1"
langdetect = "*"
mysqlclient = "*"
nltk = "*"
@ -57,7 +59,6 @@ watchdog = "~=4.0"
whitenoise = "~=6.7"
whoosh = "~=2.7"
zxing-cpp = {version = "*", platform_machine = "== 'x86_64'"}
jinja2 = "~=3.1"
[dev-packages]
# Linting

13
Pipfile.lock generated
View File

@ -1,7 +1,7 @@
{
"_meta": {
"hash": {
"sha256": "1e113d0879e4e0bc3c384115057647ac8d9be05252dd7c708a1fc873f294ef28"
"sha256": "584249cbeaf29659c975000b5e02b12e45d768d795e4a8ac36118e73bd7c0b8a"
},
"pipfile-spec": 6,
"requires": {},
@ -799,9 +799,18 @@
"sha256:7bb2708e112d8fdd7829cd4243970f0c223274051cb35ee80c03301ee29a3df0",
"sha256:f7c2be1d2f3c3c3160d441802406b206c2b76f5947b11115e6df10c6c65e66c2"
],
"markers": "python_version >= '3.9'",
"markers": "python_version >= '3.8'",
"version": "==0.27.2"
},
"httpx-oauth": {
"hashes": [
"sha256:4094cf0938fc7252b5f5dfd62cd1ab5aee2fcb6734e621942ee17d1af4806b74",
"sha256:89b45f250e93e42bbe9631adf349cab0e3d3ced958c07e06651735198d1bdf00"
],
"index": "pypi",
"markers": "python_version >= '3.8'",
"version": "==0.15.1"
},
"humanize": {
"hashes": [
"sha256:06b6eb0293e4b85e8d385397c5868926820db32b9b654b932f57fa41c23c9978",

48
docs/configuration.md
View File

@ -1164,12 +1164,6 @@ within your documents.
Defaults to false.
#### [`PAPERLESS_EMAIL_GNUPG_HOME=<str>`](#PAPERLESS_EMAIL_GNUPG_HOME) {#PAPERLESS_EMAIL_GNUPG_HOME}
: Optional, sets the `GNUPG_HOME` path to use with GPG decryptor for encrypted emails. See [GPG Decryptor](advanced_usage.md#gpg-decryptor) for more information. If not set, defaults to the default `GNUPG_HOME` path.
Defaults to <not set>.
### Polling {#polling}
#### [`PAPERLESS_CONSUMER_POLLING=<num>`](#PAPERLESS_CONSUMER_POLLING) {#PAPERLESS_CONSUMER_POLLING}
@ -1213,6 +1207,48 @@ consumers working on the same file. Configure this to prevent that.
Defaults to 0.5 seconds.
## Incoming Mail {#incoming_mail}
### Email OAuth {#email_oauth}
#### [`PAPERLESS_OAUTH_CALLBACK_BASE_URL=<str>`](#PAPERLESS_OAUTH_CALLBACK_BASE_URL) {#PAPERLESS_OAUTH_CALLBACK_BASE_URL}
: The base URL for the OAuth callback. This is used to construct the full URL for the OAuth callback. This should be the URL that the Paperless instance is accessible at. If not set, defaults to the `PAPERLESS_URL` setting. At least one of these settings must be set to enable OAuth Email setup.
Defaults to none (thus will use [PAPERLESS_URL](#PAPERLESS_URL)).
#### [`PAPERLESS_GMAIL_OAUTH_CLIENT_ID=<str>`](#PAPERLESS_GMAIL_OAUTH_CLIENT_ID) {#PAPERLESS_GMAIL_OAUTH_CLIENT_ID}
: The OAuth client ID for Gmail. This is required for Gmail OAuth Email setup. See [OAuth Email Setup](usage.md#oauth-email-setup) for more information.
Defaults to none.
#### [`PAPERLESS_GMAIL_OAUTH_CLIENT_SECRET=<str>`](#PAPERLESS_GMAIL_OAUTH_CLIENT_SECRET) {#PAPERLESS_GMAIL_OAUTH_CLIENT_SECRET}
: The OAuth client secret for Gmail. This is required for Gmail OAuth Email setup. See [OAuth Email Setup](usage.md#oauth-email-setup) for more information.
Defaults to none.
#### [`PAPERLESS_OUTLOOK_OAUTH_CLIENT_ID=<str>`](#PAPERLESS_OUTLOOK_OAUTH_CLIENT_ID) {#PAPERLESS_OUTLOOK_OAUTH_CLIENT_ID}
: The OAuth client ID for Outlook. This is required for Outlook OAuth Email setup. See [OAuth Email Setup](usage.md#oauth-email-setup) for more information.
Defaults to none.
#### [`PAPERLESS_OUTLOOK_OAUTH_CLIENT_SECRET=<str>`](#PAPERLESS_OUTLOOK_OAUTH_CLIENT_SECRET) {#PAPERLESS_OUTLOOK_OAUTH_CLIENT_SECRET}
: The OAuth client secret for Outlook. This is required for Outlook OAuth Email setup. See [OAuth Email Setup](usage.md#oauth-email-setup) for more information.
Defaults to none.
### Encrypted Emails {#encrypted_emails}
#### [`PAPERLESS_EMAIL_GNUPG_HOME=<str>`](#PAPERLESS_EMAIL_GNUPG_HOME) {#PAPERLESS_EMAIL_GNUPG_HOME}
: Optional, sets the `GNUPG_HOME` path to use with GPG decryptor for encrypted emails. See [GPG Decryptor](advanced_usage.md#gpg-decryptor) for more information. If not set, defaults to the default `GNUPG_HOME` path.
Defaults to <not set>.
## Barcodes {#barcodes}
#### [`PAPERLESS_CONSUMER_ENABLE_BARCODES=<bool>`](#PAPERLESS_CONSUMER_ENABLE_BARCODES) {#PAPERLESS_CONSUMER_ENABLE_BARCODES}

10
docs/usage.md
View File

@ -112,7 +112,7 @@ process.
Please see [the wiki](https://github.com/paperless-ngx/paperless-ngx/wiki/Related-Projects) for a user-maintained list of related projects and
software (e.g. for mobile devices) that is compatible with Paperless-ngx.
### IMAP (Email) {#usage-email}
### Email {#usage-email}
You can tell paperless-ngx to consume documents from your email
accounts. This is a very flexible and powerful feature, if you regularly
@ -200,6 +200,14 @@ different means. These are as follows:
Paperless is set up to check your mails every 10 minutes. This can be
configured via [`PAPERLESS_EMAIL_TASK_CRON`](configuration.md#PAPERLESS_EMAIL_TASK_CRON)
#### OAuth Email Setup
Paperless-ngx supports OAuth2 authentication for Gmail and Outlook email accounts. To set up an email account with OAuth2, you will need to create a 'developer' app with the respective provider and obtain the client ID and client secret and set the appropriate [configuration variables](configuration.md#email_oauth). You will also need to set either [`PAPERLESS_OAUTH_CALLBACK_BASE_URL`](configuration.md#PAPERLESS_OAUTH_CALLBACK_BASE_URL) or [`PAPERLESS_URL`](configuration.md#PAPERLESS_URL) to the correct value for the OAuth2 flow to work correctly.
Specific instructions for setting up the required 'developer' app with Google or Microsoft are beyond the scope of this documentation, but you can find user-maintained instructions in [the wiki](https://github.com/paperless-ngx/paperless-ngx/wiki/Email-OAuth-App-Setup) or by searching the web.
Once setup, navigating to the email settings page in Paperless-ngx will allow you to add an email account for Gmail or Outlook using OAuth2. After authenticating, you will be presented with the newly-created account where you will need to enter and save your email address. After this, the account will work as any other email account in Paperless-ngx and refreshing tokens will be handled automatically.
### REST API
You can also submit a document using the REST API, see [POSTing documents](api.md#file-uploads)

134
src-ui/messages.xlf
View File

@ -734,7 +734,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">146</context>
<context context-type="linenumber">164</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/management-list/management-list.component.html</context>
@ -1100,19 +1100,19 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">41</context>
<context context-type="linenumber">59</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">51</context>
<context context-type="linenumber">69</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">108</context>
<context context-type="linenumber">126</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">120</context>
<context context-type="linenumber">138</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/management-list/management-list.component.html</context>
@ -1410,11 +1410,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">23</context>
<context context-type="linenumber">33</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">82</context>
<context context-type="linenumber">100</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/management-list/management-list.component.html</context>
@ -1509,19 +1509,19 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">42</context>
<context context-type="linenumber">60</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">54</context>
<context context-type="linenumber">72</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">109</context>
<context context-type="linenumber">127</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">123</context>
<context context-type="linenumber">141</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/management-list/management-list.component.html</context>
@ -1907,11 +1907,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">20</context>
<context context-type="linenumber">30</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">78</context>
<context context-type="linenumber">96</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/management-list/management-list.component.html</context>
@ -2235,11 +2235,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">114</context>
<context context-type="linenumber">160</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">194</context>
<context context-type="linenumber">240</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/management-list/management-list.component.ts</context>
@ -2380,7 +2380,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">22</context>
<context context-type="linenumber">32</context>
</context-group>
</trans-unit>
<trans-unit id="5944812089887969249" datatype="html">
@ -2438,19 +2438,19 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">40</context>
<context context-type="linenumber">58</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">48</context>
<context context-type="linenumber">66</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">107</context>
<context context-type="linenumber">125</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">117</context>
<context context-type="linenumber">135</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/management-list/management-list.component.html</context>
@ -2590,11 +2590,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">116</context>
<context context-type="linenumber">162</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">196</context>
<context context-type="linenumber">242</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/management-list/management-list.component.ts</context>
@ -3786,7 +3786,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">80</context>
<context context-type="linenumber">98</context>
</context-group>
</trans-unit>
<trans-unit id="220550782947016929" datatype="html">
@ -3808,7 +3808,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">96</context>
<context context-type="linenumber">114</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/workflows/workflows.component.html</context>
@ -5163,11 +5163,11 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">110</context>
<context context-type="linenumber">128</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">128</context>
<context context-type="linenumber">146</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/workflows/workflows.component.html</context>
@ -5480,7 +5480,7 @@
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">81</context>
<context context-type="linenumber">99</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/workflows/workflows.component.html</context>
@ -7607,46 +7607,60 @@
<context context-type="linenumber">14</context>
</context-group>
</trans-unit>
<trans-unit id="5088684330574277786" datatype="html">
<source>Connect Gmail Account</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">18</context>
</context-group>
</trans-unit>
<trans-unit id="6630732552154686829" datatype="html">
<source>Connect Outlook Account</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">23</context>
</context-group>
</trans-unit>
<trans-unit id="2188854519574316630" datatype="html">
<source>Server</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">21</context>
<context context-type="linenumber">31</context>
</context-group>
</trans-unit>
<trans-unit id="6235247415162820954" datatype="html">
<source>No mail accounts defined.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">62</context>
<context context-type="linenumber">80</context>
</context-group>
</trans-unit>
<trans-unit id="5364020217520256833" datatype="html">
<source>Mail rules</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">70</context>
<context context-type="linenumber">88</context>
</context-group>
</trans-unit>
<trans-unit id="1372022816709469401" datatype="html">
<source>Add Rule</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">72</context>
<context context-type="linenumber">90</context>
</context-group>
</trans-unit>
<trans-unit id="2535466903620876415" datatype="html">
<source>Sort Order</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">79</context>
<context context-type="linenumber">97</context>
</context-group>
</trans-unit>
<trans-unit id="5769292297914455214" datatype="html">
<source>Disabled</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">96</context>
<context context-type="linenumber">114</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/workflows/workflows.component.html</context>
@ -7657,140 +7671,154 @@
<source>No mail rules defined.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.html</context>
<context context-type="linenumber">137</context>
<context context-type="linenumber">155</context>
</context-group>
</trans-unit>
<trans-unit id="3178554336792037159" datatype="html">
<source>Error retrieving mail accounts</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">56</context>
<context context-type="linenumber">81</context>
</context-group>
</trans-unit>
<trans-unit id="5241231471117657636" datatype="html">
<source>Error retrieving mail rules</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">70</context>
<context context-type="linenumber">95</context>
</context-group>
</trans-unit>
<trans-unit id="763945516325093575" datatype="html">
<source>OAuth2 authentication success</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">103</context>
</context-group>
</trans-unit>
<trans-unit id="9022978370268070156" datatype="html">
<source>OAuth2 authentication failed, see logs for details</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">114</context>
</context-group>
</trans-unit>
<trans-unit id="6327501535846658797" datatype="html">
<source>Saved account &quot;<x id="PH" equiv-text="newMailAccount.name"/>&quot;.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">92</context>
<context context-type="linenumber">138</context>
</context-group>
</trans-unit>
<trans-unit id="8067594003836508139" datatype="html">
<source>Error saving account.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">104</context>
<context context-type="linenumber">150</context>
</context-group>
</trans-unit>
<trans-unit id="5641934153807844674" datatype="html">
<source>Confirm delete mail account</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">112</context>
<context context-type="linenumber">158</context>
</context-group>
</trans-unit>
<trans-unit id="7176985344323395435" datatype="html">
<source>This operation will permanently delete this mail account.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">113</context>
<context context-type="linenumber">159</context>
</context-group>
</trans-unit>
<trans-unit id="4233826387148482123" datatype="html">
<source>Deleted mail account</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">122</context>
<context context-type="linenumber">168</context>
</context-group>
</trans-unit>
<trans-unit id="6202503362522392111" datatype="html">
<source>Error deleting mail account.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">132</context>
<context context-type="linenumber">178</context>
</context-group>
</trans-unit>
<trans-unit id="123368655395433699" datatype="html">
<source>Saved rule &quot;<x id="PH" equiv-text="newMailRule.name"/>&quot;.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">151</context>
<context context-type="linenumber">197</context>
</context-group>
</trans-unit>
<trans-unit id="8951124554918814321" datatype="html">
<source>Error saving rule.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">162</context>
<context context-type="linenumber">208</context>
</context-group>
</trans-unit>
<trans-unit id="3574401690710711341" datatype="html">
<source>Rule &quot;<x id="PH" equiv-text="rule.name"/>&quot; enabled.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">178</context>
<context context-type="linenumber">224</context>
</context-group>
</trans-unit>
<trans-unit id="7171685227222299542" datatype="html">
<source>Rule &quot;<x id="PH" equiv-text="rule.name"/>&quot; disabled.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">179</context>
<context context-type="linenumber">225</context>
</context-group>
</trans-unit>
<trans-unit id="684458488797860482" datatype="html">
<source>Error toggling rule.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">183</context>
<context context-type="linenumber">229</context>
</context-group>
</trans-unit>
<trans-unit id="3896080636020672118" datatype="html">
<source>Confirm delete mail rule</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">192</context>
<context context-type="linenumber">238</context>
</context-group>
</trans-unit>
<trans-unit id="2250372580580310337" datatype="html">
<source>This operation will permanently delete this mail rule.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">193</context>
<context context-type="linenumber">239</context>
</context-group>
</trans-unit>
<trans-unit id="9077981247971516916" datatype="html">
<source>Deleted mail rule</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">202</context>
<context context-type="linenumber">248</context>
</context-group>
</trans-unit>
<trans-unit id="2033194641751367552" datatype="html">
<source>Error deleting mail rule.</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">211</context>
<context context-type="linenumber">257</context>
</context-group>
</trans-unit>
<trans-unit id="3061362835271417984" datatype="html">
<source>Permissions updated</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">233</context>
<context context-type="linenumber">279</context>
</context-group>
</trans-unit>
<trans-unit id="4639647950943944112" datatype="html">
<source>Error updating permissions</source>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/mail/mail.component.ts</context>
<context context-type="linenumber">238</context>
<context context-type="linenumber">284</context>
</context-group>
<context-group purpose="location">
<context context-type="sourcefile">src/app/components/manage/management-list/management-list.component.ts</context>

6
src-ui/src/app/app.module.ts
View File

@ -175,6 +175,7 @@ import {
download,
envelope,
envelopeAt,
envelopeAtFill,
exclamationCircleFill,
exclamationTriangle,
exclamationTriangleFill,
@ -191,6 +192,7 @@ import {
folderFill,
funnel,
gear,
google,
grid,
gripVertical,
hash,
@ -201,6 +203,7 @@ import {
link,
listTask,
listUl,
microsoft,
nodePlus,
pencil,
people,
@ -279,6 +282,7 @@ const icons = {
download,
envelope,
envelopeAt,
envelopeAtFill,
exclamationCircleFill,
exclamationTriangle,
exclamationTriangleFill,
@ -295,6 +299,7 @@ const icons = {
folderFill,
funnel,
gear,
google,
grid,
gripVertical,
hash,
@ -305,6 +310,7 @@ const icons = {
link,
listTask,
listUl,
microsoft,
nodePlus,
pencil,
people,

3
src-ui/src/app/components/common/edit-dialog/mail-account-edit-dialog/mail-account-edit-dialog.component.spec.ts
View File

@ -11,7 +11,7 @@ import {
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { NgbActiveModal, NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgSelectModule } from '@ng-select/ng-select'
import { IMAPSecurity } from 'src/app/data/mail-account'
import { IMAPSecurity, MailAccountType } from 'src/app/data/mail-account'
import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { SettingsService } from 'src/app/services/settings.service'
@ -82,6 +82,7 @@ describe('MailAccountEditDialogComponent', () => {
imap_port: 443,
imap_security: IMAPSecurity.SSL,
is_token: false,
account_type: MailAccountType.IMAP,
}
// success

20
src-ui/src/app/components/manage/mail/mail.component.html
View File

@ -13,6 +13,16 @@
<button type="button" class="btn btn-sm btn-outline-primary ms-4" (click)="editMailAccount()" *pngxIfPermissions="{ action: PermissionAction.Add, type: PermissionType.MailAccount }">
<i-bs name="plus-circle"></i-bs>&nbsp;<ng-container i18n>Add Account</ng-container>
</button>
@if (gmailOAuthUrl) {
<a class="btn btn-sm btn-outline-secondary ms-2" [href]="gmailOAuthUrl" *pngxIfPermissions="{ action: PermissionAction.Add, type: PermissionType.MailAccount }">
<i-bs name="google"></i-bs>&nbsp;<ng-container i18n>Connect Gmail Account</ng-container>
</a>
}
@if (outlookOAuthUrl) {
<a class="btn btn-sm btn-outline-secondary ms-2" [href]="outlookOAuthUrl" *pngxIfPermissions="{ action: PermissionAction.Add, type: PermissionType.MailAccount }">
<i-bs name="microsoft"></i-bs>&nbsp;<ng-container i18n>Connect Outlook Account</ng-container>
</a>
}
</h4>
<ul class="list-group">
<li class="list-group-item">
@ -27,7 +37,15 @@
@for (account of mailAccounts; track account) {
<li class="list-group-item">
<div class="row">
<div class="col d-flex align-items-center"><button class="btn btn-link p-0 text-start" type="button" (click)="editMailAccount(account)" [disabled]="!permissionsService.currentUserCan(PermissionAction.Change, PermissionType.MailAccount)">{{account.name}}</button></div>
<div class="col d-flex align-items-center">
<button class="btn btn-link p-0 text-start" type="button" (click)="editMailAccount(account)" [disabled]="!permissionsService.currentUserCan(PermissionAction.Change, PermissionType.MailAccount)">
{{account.name}}@switch (account.account_type) {
@case (MailAccountType.IMAP) {<i-bs name="envelope-at-fill" class="ms-2"></i-bs>}
@case (MailAccountType.Gmail_OAuth) {<i-bs name="google" class="ms-2"></i-bs>}
@case (MailAccountType.Outlook_OAuth) {<i-bs name="microsoft" class="ms-2"></i-bs>}
}
</button>
</div>
<div class="col d-flex align-items-center">{{account.imap_server}}</div>
<div class="col d-flex align-items-center d-none d-sm-block">{{account.username}}</div>
<div class="col">

46
src-ui/src/app/components/manage/mail/mail.component.spec.ts
View File

@ -13,7 +13,7 @@ import {
import { NgSelectModule } from '@ng-select/ng-select'
import { of, throwError } from 'rxjs'
import { routes } from 'src/app/app-routing.module'
import { MailAccount } from 'src/app/data/mail-account'
import { MailAccount, MailAccountType } from 'src/app/data/mail-account'
import { MailRule } from 'src/app/data/mail-rule'
import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
@ -44,10 +44,13 @@ import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { SwitchComponent } from '../../common/input/switch/switch.component'
import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { By } from '@angular/platform-browser'
import { ActivatedRoute, convertToParamMap } from '@angular/router'
import { SettingsService } from 'src/app/services/settings.service'
const mailAccounts = [
{ id: 1, name: 'account1' },
{ id: 2, name: 'account2' },
{ id: 1, name: 'account1', account_type: MailAccountType.IMAP },
{ id: 2, name: 'account2', account_type: MailAccountType.IMAP },
{ id: 3, name: 'account3', accout_type: MailAccountType.Gmail_OAuth },
]
const mailRules = [
{ id: 1, name: 'rule1', owner: 1, account: 1, enabled: true },
@ -62,6 +65,8 @@ describe('MailComponent', () => {
let modalService: NgbModal
let toastService: ToastService
let permissionsService: PermissionsService
let activatedRoute: ActivatedRoute
let settingsService: SettingsService
beforeEach(() => {
TestBed.configureTestingModule({
@ -110,6 +115,9 @@ describe('MailComponent', () => {
modalService = TestBed.inject(NgbModal)
toastService = TestBed.inject(ToastService)
permissionsService = TestBed.inject(PermissionsService)
activatedRoute = TestBed.inject(ActivatedRoute)
settingsService = TestBed.inject(SettingsService)
settingsService.currentUser = { id: 1 }
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest
.spyOn(permissionsService, 'currentUserHasObjectPermissions')
@ -348,4 +356,36 @@ describe('MailComponent', () => {
expect(patchSpy).toHaveBeenCalled()
expect(toastInfoSpy).toHaveBeenCalled()
})
it('should show success message when oauth account is connected', () => {
const queryParams = { oauth_success: '1' }
jest
.spyOn(activatedRoute, 'queryParamMap', 'get')
.mockReturnValue(of(convertToParamMap(queryParams)))
const toastInfoSpy = jest.spyOn(toastService, 'showInfo')
completeSetup()
expect(toastInfoSpy).toHaveBeenCalled()
})
it('should show error message when oauth account connect fails', () => {
const queryParams = { oauth_success: '0' }
jest
.spyOn(activatedRoute, 'queryParamMap', 'get')
.mockReturnValue(of(convertToParamMap(queryParams)))
const toastErrorSpy = jest.spyOn(toastService, 'showError')
completeSetup()
expect(toastErrorSpy).toHaveBeenCalled()
})
it('should open account edit dialog if oauth account is connected', () => {
const queryParams = { oauth_success: '1', oauth_account: '3' }
jest
.spyOn(activatedRoute, 'queryParamMap', 'get')
.mockReturnValue(of(convertToParamMap(queryParams)))
completeSetup()
component.oAuthAccountId = 3
const editSpy = jest.spyOn(component, 'editMailAccount')
component.ngOnInit()
expect(editSpy).toHaveBeenCalled()
})
})

50
src-ui/src/app/components/manage/mail/mail.component.ts
View File

@ -2,7 +2,7 @@ import { Component, OnInit, OnDestroy } from '@angular/core'
import { NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { Subject, first, takeUntil } from 'rxjs'
import { ObjectWithPermissions } from 'src/app/data/object-with-permissions'
import { MailAccount } from 'src/app/data/mail-account'
import { MailAccount, MailAccountType } from 'src/app/data/mail-account'
import { MailRule } from 'src/app/data/mail-rule'
import {
PermissionsService,
@ -18,6 +18,9 @@ import { MailAccountEditDialogComponent } from '../../common/edit-dialog/mail-ac
import { MailRuleEditDialogComponent } from '../../common/edit-dialog/mail-rule-edit-dialog/mail-rule-edit-dialog.component'
import { PermissionsDialogComponent } from '../../common/permissions-dialog/permissions-dialog.component'
import { ComponentWithPermissions } from '../../with-permissions/with-permissions.component'
import { SettingsService } from 'src/app/services/settings.service'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { ActivatedRoute } from '@angular/router'
@Component({
selector: 'pngx-mail',
@ -28,17 +31,30 @@ export class MailComponent
extends ComponentWithPermissions
implements OnInit, OnDestroy
{
public MailAccountType = MailAccountType
mailAccounts: MailAccount[] = []
mailRules: MailRule[] = []
unsubscribeNotifier: Subject<any> = new Subject()
oAuthAccountId: number
public get gmailOAuthUrl(): string {
return this.settingsService.get(SETTINGS_KEYS.GMAIL_OAUTH_URL)
}
public get outlookOAuthUrl(): string {
return this.settingsService.get(SETTINGS_KEYS.OUTLOOK_OAUTH_URL)
}
constructor(
public mailAccountService: MailAccountService,
public mailRuleService: MailRuleService,
private toastService: ToastService,
private modalService: NgbModal,
public permissionsService: PermissionsService
public permissionsService: PermissionsService,
private settingsService: SettingsService,
private route: ActivatedRoute
) {
super()
}
@ -50,6 +66,15 @@ export class MailComponent
.subscribe({
next: (r) => {
this.mailAccounts = r.results
console.log(this.mailAccounts, this.oAuthAccountId)
if (this.oAuthAccountId) {
this.editMailAccount(
this.mailAccounts.find(
(account) => account.id === this.oAuthAccountId
)
)
}
},
error: (e) => {
this.toastService.showError(
@ -70,6 +95,27 @@ export class MailComponent
this.toastService.showError($localize`Error retrieving mail rules`, e)
},
})
this.route.queryParamMap.subscribe((params) => {
if (params.get('oauth_success')) {
const success = params.get('oauth_success') === '1'
if (success) {
this.toastService.showInfo($localize`OAuth2 authentication success`)
this.oAuthAccountId = parseInt(params.get('account_id'))
if (this.mailAccounts.length > 0) {
this.editMailAccount(
this.mailAccounts.find(
(account) => account.id === this.oAuthAccountId
)
)
}
} else {
this.toastService.showError(
$localize`OAuth2 authentication failed, see logs for details`
)
}
}
})
}
ngOnDestroy() {

10
src-ui/src/app/data/mail-account.ts
View File

@ -6,6 +6,12 @@ export enum IMAPSecurity {
STARTTLS = 3,
}
export enum MailAccountType {
IMAP = 1,
Gmail_OAuth = 2,
Outlook_OAuth = 3,
}
export interface MailAccount extends ObjectWithPermissions {
name: string
@ -22,4 +28,8 @@ export interface MailAccount extends ObjectWithPermissions {
character_set?: string
is_token: boolean
account_type: MailAccountType
expiration?: string // Date
}

12
src-ui/src/app/data/ui-settings.ts
View File

@ -64,6 +64,8 @@ export const SETTINGS_KEYS = {
SEARCH_DB_ONLY: 'general-settings:search:db-only',
SEARCH_FULL_TYPE: 'general-settings:search:more-link',
EMPTY_TRASH_DELAY: 'trash_delay',
GMAIL_OAUTH_URL: 'gmail_oauth_url',
OUTLOOK_OAUTH_URL: 'outlook_oauth_url',
}
export const SETTINGS: UiSetting[] = [
@ -242,4 +244,14 @@ export const SETTINGS: UiSetting[] = [
type: 'number',
default: 30,
},
{
key: SETTINGS_KEYS.GMAIL_OAUTH_URL,
type: 'string',
default: null,
},
{
key: SETTINGS_KEYS.OUTLOOK_OAUTH_URL,
type: 'string',
default: null,
},
]

19
src-ui/src/app/services/rest/mail-account.service.spec.ts
View File

@ -4,7 +4,7 @@ import { TestBed } from '@angular/core/testing'
import { environment } from 'src/environments/environment'
import { commonAbstractPaperlessServiceTests } from './abstract-paperless-service.spec'
import { MailAccountService } from './mail-account.service'
import { IMAPSecurity } from 'src/app/data/mail-account'
import { IMAPSecurity, MailAccountType } from 'src/app/data/mail-account'
let httpTestingController: HttpTestingController
let service: MailAccountService
@ -20,6 +20,7 @@ const mail_accounts = [
username: 'user',
password: 'pass',
is_token: false,
account_type: MailAccountType.IMAP,
},
{
name: 'Mail Account 2',
@ -30,6 +31,7 @@ const mail_accounts = [
username: 'user',
password: 'pass',
is_token: false,
account_type: MailAccountType.IMAP,
},
{
name: 'Mail Account 3',
@ -40,6 +42,7 @@ const mail_accounts = [
username: 'user',
password: 'pass',
is_token: false,
account_type: MailAccountType.IMAP,
},
]
@ -55,20 +58,6 @@ describe(`Additional service tests for MailAccountService`, () => {
expect(req.request.method).toEqual('POST')
})
it('should support patchMany', () => {
subscription = service.patchMany(mail_accounts).subscribe()
mail_accounts.forEach((mail_account) => {
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/${mail_account.id}/`
)
expect(req.request.method).toEqual('PATCH')
req.flush(mail_account)
})
httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/?page=1&page_size=100000`
)
})
it('should support reload', () => {
service['reload']()
const req = httpTestingController.expectOne(

9
src-ui/src/app/services/rest/mail-account.service.ts
View File

@ -1,6 +1,5 @@
import { HttpClient } from '@angular/common/http'
import { Injectable } from '@angular/core'
import { combineLatest, Observable } from 'rxjs'
import { tap } from 'rxjs/operators'
import { MailAccount } from 'src/app/data/mail-account'
import { AbstractPaperlessService } from './abstract-paperless-service'
@ -34,15 +33,11 @@ export class MailAccountService extends AbstractPaperlessService<MailAccount> {
}
update(o: MailAccount) {
// Remove expiration from the object before updating
delete o.expiration
return super.update(o).pipe(tap(() => this.reload()))
}
patchMany(objects: MailAccount[]): Observable<MailAccount[]> {
return combineLatest(objects.map((o) => super.patch(o))).pipe(
tap(() => this.reload())
)
}
delete(o: MailAccount) {
return super.delete(o).pipe(tap(() => this.reload()))
}

15
src-ui/src/app/services/rest/mail-rule.service.spec.ts
View File

@ -76,21 +76,6 @@ const mail_rules = [
commonAbstractPaperlessServiceTests(endpoint, MailRuleService)
describe(`Additional service tests for MailRuleService`, () => {
it('should support patchMany', () => {
subscription = service.patchMany(mail_rules).subscribe()
mail_rules.forEach((mail_rule) => {
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/${mail_rule.id}/`
)
expect(req.request.method).toEqual('PATCH')
req.flush(mail_rule)
})
const reloadReq = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/?page=1&page_size=100000`
)
reloadReq.flush({ results: mail_rules })
})
it('should support reload', () => {
service['reload']()
const req = httpTestingController.expectOne(

6
src-ui/src/app/services/rest/mail-rule.service.ts
View File

@ -37,12 +37,6 @@ export class MailRuleService extends AbstractPaperlessService<MailRule> {
return super.update(o).pipe(tap(() => this.reload()))
}
patchMany(objects: MailRule[]): Observable<MailRule[]> {
return combineLatest(objects.map((o) => super.patch(o))).pipe(
tap(() => this.reload())
)
}
delete(o: MailRule) {
return super.delete(o).pipe(tap(() => this.reload()))
}

5
src-ui/src/styles.scss
View File

@ -699,3 +699,8 @@ canvas.hiddenCanvasElement {
height: 0;
width: 0;
}
// bs icons
i-bs svg {
vertical-align: text-bottom;
}

20
src/documents/tests/test_api_uisettings.py
View File

@ -2,6 +2,7 @@ import json
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import override_settings
from rest_framework import status
from rest_framework.test import APITestCase
@ -113,3 +114,22 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
@override_settings(
OAUTH_CALLBACK_BASE_URL="http://localhost:8000",
GMAIL_OAUTH_CLIENT_ID="abc123",
GMAIL_OAUTH_CLIENT_SECRET="def456",
GMAIL_OAUTH_ENABLED=True,
OUTLOOK_OAUTH_CLIENT_ID="ghi789",
OUTLOOK_OAUTH_CLIENT_SECRET="jkl012",
OUTLOOK_OAUTH_ENABLED=True,
)
def test_settings_includes_oauth_urls_if_enabled(self):
response = self.client.get(self.ENDPOINT, format="json")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertIsNotNone(
response.data["settings"]["gmail_oauth_url"],
)
self.assertIsNotNone(
response.data["settings"]["outlook_oauth_url"],
)

2
src/documents/tests/test_migration_workflows.py
View File

@ -8,7 +8,7 @@ class TestMigrateWorkflow(TestMigrations):
dependencies = (
(
"paperless_mail",
"0026_mailrule_enabled",
"0027_mailaccount_expiration_mailaccount_account_type_and_more",
),
)

10
src/documents/views.py
View File

@ -162,6 +162,7 @@ from paperless.serialisers import UserSerializer
from paperless.views import StandardPagination
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from paperless_mail.oauth import PaperlessMailOAuth2Manager
from paperless_mail.serialisers import MailAccountSerializer
from paperless_mail.serialisers import MailRuleSerializer
@ -1605,6 +1606,15 @@ class UiSettingsView(GenericAPIView):
ui_settings["auditlog_enabled"] = settings.AUDIT_LOG_ENABLED
if settings.GMAIL_OAUTH_ENABLED or settings.OUTLOOK_OAUTH_ENABLED:
manager = PaperlessMailOAuth2Manager()
if settings.GMAIL_OAUTH_ENABLED:
ui_settings["gmail_oauth_url"] = manager.get_gmail_authorization_url()
if settings.OUTLOOK_OAUTH_ENABLED:
ui_settings["outlook_oauth_url"] = (
manager.get_outlook_authorization_url()
)
user_resp = {
"id": user.id,
"username": user.username,

16
src/paperless/settings.py
View File

@ -1195,3 +1195,19 @@ EMAIL_ENABLE_GPG_DECRYPTOR: Final[bool] = __get_boolean(
# Soft Delete #
###############################################################################
EMPTY_TRASH_DELAY = max(__get_int("PAPERLESS_EMPTY_TRASH_DELAY", 30), 1)
###############################################################################
# Oauth Email #
###############################################################################
OAUTH_CALLBACK_BASE_URL = os.getenv("PAPERLESS_OAUTH_CALLBACK_BASE_URL")
GMAIL_OAUTH_CLIENT_ID = os.getenv("PAPERLESS_GMAIL_OAUTH_CLIENT_ID")
GMAIL_OAUTH_CLIENT_SECRET = os.getenv("PAPERLESS_GMAIL_OAUTH_CLIENT_SECRET")
GMAIL_OAUTH_ENABLED = bool(
OAUTH_CALLBACK_BASE_URL and GMAIL_OAUTH_CLIENT_ID and GMAIL_OAUTH_CLIENT_SECRET,
)
OUTLOOK_OAUTH_CLIENT_ID = os.getenv("PAPERLESS_OUTLOOK_OAUTH_CLIENT_ID")
OUTLOOK_OAUTH_CLIENT_SECRET = os.getenv("PAPERLESS_OUTLOOK_OAUTH_CLIENT_SECRET")
OUTLOOK_OAUTH_ENABLED = bool(
OAUTH_CALLBACK_BASE_URL and OUTLOOK_OAUTH_CLIENT_ID and OUTLOOK_OAUTH_CLIENT_SECRET,
)

6
src/paperless/urls.py
View File

@ -55,6 +55,7 @@ from paperless.views import UserViewSet
from paperless_mail.views import MailAccountTestView
from paperless_mail.views import MailAccountViewSet
from paperless_mail.views import MailRuleViewSet
from paperless_mail.views import OauthCallbackView
api_router = DefaultRouter()
api_router.register(r"correspondents", CorrespondentViewSet)
@ -171,6 +172,11 @@ urlpatterns = [
StoragePathTestView.as_view(),
name="storage_paths_test",
),
re_path(
r"^oauth/callback/",
OauthCallbackView.as_view(),
name="oauth_callback",
),
*api_router.urls,
],
),

13
src/paperless_mail/mail.py
View File

@ -18,6 +18,7 @@ from celery import shared_task
from celery.canvas import Signature
from django.conf import settings
from django.db import DatabaseError
from django.utils import timezone
from django.utils.timezone import is_naive
from django.utils.timezone import make_aware
from imap_tools import AND
@ -42,6 +43,7 @@ from documents.tasks import consume_file
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from paperless_mail.models import ProcessedMail
from paperless_mail.oauth import PaperlessMailOAuth2Manager
from paperless_mail.preprocessor import MailMessageDecryptor
from paperless_mail.preprocessor import MailMessagePreprocessor
@ -530,6 +532,17 @@ class MailAccountHandler(LoggingMixin):
account.imap_port,
account.imap_security,
) as M:
if (
account.is_token
and account.expiration is not None
and account.expiration < timezone.now()
):
manager = PaperlessMailOAuth2Manager()
if manager.refresh_account_oauth_token(account):
account.refresh_from_db()
else:
return total_processed_files
supports_gmail_labels = "X-GM-EXT-1" in M.client.capabilities
supports_auth_plain = "AUTH=PLAIN" in M.client.capabilities

43
src/paperless_mail/migrations/0027_mailaccount_expiration_mailaccount_account_type_and_more.py Normal file
View File

@ -0,0 +1,43 @@
# Generated by Django 5.1.1 on 2024-10-05 17:12
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("paperless_mail", "0026_mailrule_enabled"),
]
operations = [
migrations.AddField(
model_name="mailaccount",
name="expiration",
field=models.DateTimeField(
blank=True,
help_text="The expiration date of the refresh token. ",
null=True,
verbose_name="expiration",
),
),
migrations.AddField(
model_name="mailaccount",
name="account_type",
field=models.PositiveIntegerField(
choices=[(1, "IMAP"), (2, "Gmail OAuth"), (3, "Outlook OAuth")],
default=1,
verbose_name="account type",
),
),
migrations.AddField(
model_name="mailaccount",
name="refresh_token",
field=models.CharField(
blank=True,
help_text="The refresh token to use for token authentication e.g. with oauth2.",
max_length=2048,
null=True,
verbose_name="refresh token",
),
),
]

30
src/paperless_mail/models.py
View File

@ -15,6 +15,11 @@ class MailAccount(document_models.ModelWithOwner):
SSL = 2, _("Use SSL")
STARTTLS = 3, _("Use STARTTLS")
class MailAccountType(models.IntegerChoices):
IMAP = 1, _("IMAP")
GMAIL_OAUTH = 2, _("Gmail OAuth")
OUTLOOK_OAUTH = 3, _("Outlook OAuth")
name = models.CharField(_("name"), max_length=256, unique=True)
imap_server = models.CharField(_("IMAP server"), max_length=256)
@ -51,6 +56,31 @@ class MailAccount(document_models.ModelWithOwner):
),
)
account_type = models.PositiveIntegerField(
_("account type"),
choices=MailAccountType.choices,
default=MailAccountType.IMAP,
)
refresh_token = models.CharField(
_("refresh token"),
max_length=2048,
blank=True,
null=True,
help_text=_(
"The refresh token to use for token authentication e.g. with oauth2.",
),
)
expiration = models.DateTimeField(
_("expiration"),
blank=True,
null=True,
help_text=_(
"The expiration date of the refresh token. ",
),
)
def __str__(self):
return self.name

111
src/paperless_mail/oauth.py Normal file
View File

@ -0,0 +1,111 @@
import asyncio
import logging
from datetime import timedelta
from django.conf import settings
from django.utils import timezone
from httpx_oauth.clients.google import GoogleOAuth2
from httpx_oauth.clients.microsoft import MicrosoftGraphOAuth2
from httpx_oauth.oauth2 import OAuth2Token
from httpx_oauth.oauth2 import RefreshTokenError
from paperless_mail.models import MailAccount
class PaperlessMailOAuth2Manager:
def __init__(self):
self._gmail_client = None
self._outlook_client = None
@property
def gmail_client(self) -> GoogleOAuth2:
if self._gmail_client is None:
self._gmail_client = GoogleOAuth2(
settings.GMAIL_OAUTH_CLIENT_ID,
settings.GMAIL_OAUTH_CLIENT_SECRET,
)
return self._gmail_client
@property
def outlook_client(self) -> MicrosoftGraphOAuth2:
if self._outlook_client is None:
self._outlook_client = MicrosoftGraphOAuth2(
settings.OUTLOOK_OAUTH_CLIENT_ID,
settings.OUTLOOK_OAUTH_CLIENT_SECRET,
)
return self._outlook_client
@property
def oauth_callback_url(self) -> str:
return f"{settings.OAUTH_CALLBACK_BASE_URL if settings.OAUTH_CALLBACK_BASE_URL is not None else settings.PAPERLESS_URL}{settings.BASE_URL}api/oauth/callback/"
@property
def oauth_redirect_url(self) -> str:
return f"{'http://localhost:4200/' if settings.DEBUG else settings.BASE_URL}mail" # e.g. "http://localhost:4200/mail" or "/mail"
def get_gmail_authorization_url(self) -> str:
return asyncio.run(
self.gmail_client.get_authorization_url(
redirect_uri=self.oauth_callback_url,
scope=["https://mail.google.com/"],
extras_params={"prompt": "consent", "access_type": "offline"},
),
)
def get_outlook_authorization_url(self) -> str:
return asyncio.run(
self.outlook_client.get_authorization_url(
redirect_uri=self.oauth_callback_url,
scope=[
"offline_access",
"https://outlook.office.com/IMAP.AccessAsUser.All",
],
),
)
def get_gmail_access_token(self, code: str) -> OAuth2Token:
return asyncio.run(
self.gmail_client.get_access_token(
code=code,
redirect_uri=self.oauth_callback_url,
),
)
def get_outlook_access_token(self, code: str) -> OAuth2Token:
return asyncio.run(
self.outlook_client.get_access_token(
code=code,
redirect_uri=self.oauth_callback_url,
),
)
def refresh_account_oauth_token(self, account: MailAccount) -> bool:
"""
Refreshes the oauth token for the given mail account.
"""
logger = logging.getLogger("paperless_mail")
logger.debug(f"Attempting to refresh oauth token for account {account}")
try:
result: OAuth2Token
if account.account_type == MailAccount.MailAccountType.GMAIL_OAUTH:
result = asyncio.run(
self.gmail_client.refresh_token(
refresh_token=account.refresh_token,
),
)
elif account.account_type == MailAccount.MailAccountType.OUTLOOK_OAUTH:
result = asyncio.run(
self.outlook_client.refresh_token(
refresh_token=account.refresh_token,
),
)
account.password = result["access_token"]
account.expiration = timezone.now() + timedelta(
seconds=result["expires_in"],
)
account.save()
logger.debug(f"Successfully refreshed oauth token for account {account}")
return True
except RefreshTokenError as e:
logger.error(f"Failed to refresh oauth token for account {account}: {e}")
return False

2
src/paperless_mail/serialisers.py
View File

@ -39,6 +39,8 @@ class MailAccountSerializer(OwnedObjectSerializer):
"user_can_change",
"permissions",
"set_permissions",
"account_type",
"expiration",
]
def update(self, instance, validated_data):

129
src/paperless_mail/tests/test_mail.py
View File

@ -4,9 +4,11 @@ import random
import uuid
from collections import namedtuple
from contextlib import AbstractContextManager
from datetime import timedelta
from unittest import mock
import pytest
from django.contrib.auth.models import User
from django.core.management import call_command
from django.db import DatabaseError
from django.test import TestCase
@ -19,6 +21,8 @@ from imap_tools import MailboxLoginError
from imap_tools import MailMessage
from imap_tools import MailMessageFlags
from imap_tools import errors
from rest_framework import status
from rest_framework.test import APITestCase
from documents.models import Correspondent
from documents.tests.utils import DirectoriesMixin
@ -1590,3 +1594,128 @@ class TestTasks(TestCase):
tasks.process_mail_accounts()
self.assertEqual(m.call_count, 0)
class TestMailAccountTestView(APITestCase):
def setUp(self):
self.mailMocker = MailMocker()
self.mailMocker.setUp()
self.user = User.objects.create_user(
username="testuser",
password="testpassword",
)
self.client.force_authenticate(user=self.user)
self.url = "/api/mail_accounts/test/"
def test_mail_account_test_view_success(self):
data = {
"imap_server": "imap.example.com",
"imap_port": 993,
"imap_security": MailAccount.ImapSecurity.SSL,
"username": "admin",
"password": "secret",
"account_type": MailAccount.MailAccountType.IMAP,
"is_token": False,
}
response = self.client.post(self.url, data, format="json")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data, {"success": True})
def test_mail_account_test_view_mail_error(self):
data = {
"imap_server": "imap.example.com",
"imap_port": 993,
"imap_security": MailAccount.ImapSecurity.SSL,
"username": "admin",
"password": "wrong",
"account_type": MailAccount.MailAccountType.IMAP,
"is_token": False,
}
response = self.client.post(self.url, data, format="json")
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertEqual(response.content.decode(), "Unable to connect to server")
@mock.patch(
"paperless_mail.oauth.PaperlessMailOAuth2Manager.refresh_account_oauth_token",
)
def test_mail_account_test_view_refresh_token(
self,
mock_refresh_account_oauth_token,
):
"""
GIVEN:
- Mail account with expired token
WHEN:
- Mail account is tested
THEN:
- Should refresh the token
"""
existing_account = MailAccount.objects.create(
imap_server="imap.example.com",
imap_port=993,
imap_security=MailAccount.ImapSecurity.SSL,
username="admin",
password="secret",
account_type=MailAccount.MailAccountType.GMAIL_OAUTH,
refresh_token="oldtoken",
expiration=timezone.now() - timedelta(days=1),
is_token=True,
)
mock_refresh_account_oauth_token.return_value = True
data = {
"id": existing_account.id,
"imap_server": "imap.example.com",
"imap_port": 993,
"imap_security": MailAccount.ImapSecurity.SSL,
"username": "admin",
"password": "****",
"is_token": True,
}
self.client.post(self.url, data, format="json")
self.assertEqual(mock_refresh_account_oauth_token.call_count, 1)
@mock.patch(
"paperless_mail.oauth.PaperlessMailOAuth2Manager.refresh_account_oauth_token",
)
def test_mail_account_test_view_refresh_token_fails(
self,
mock_mock_refresh_account_oauth_token,
):
"""
GIVEN:
- Mail account with expired token
WHEN:
- Mail account is tested
- Token refresh fails
THEN:
- Should log an error
"""
existing_account = MailAccount.objects.create(
imap_server="imap.example.com",
imap_port=993,
imap_security=MailAccount.ImapSecurity.SSL,
username="admin",
password="secret",
account_type=MailAccount.MailAccountType.GMAIL_OAUTH,
refresh_token="oldtoken",
expiration=timezone.now() - timedelta(days=1),
is_token=True,
)
mock_mock_refresh_account_oauth_token.return_value = False
data = {
"id": existing_account.id,
"imap_server": "imap.example.com",
"imap_port": 993,
"imap_security": MailAccount.ImapSecurity.SSL,
"username": "admin",
"password": "****",
"is_token": True,
}
with self.assertLogs("paperless_mail", level="ERROR") as cm:
response = self.client.post(self.url, data, format="json")
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
error_str = cm.output[0]
expected_str = "Unable to refresh oauth token"
self.assertIn(expected_str, error_str)

334
src/paperless_mail/tests/test_mail_oauth.py Normal file
View File

@ -0,0 +1,334 @@
from datetime import timedelta
from unittest import mock
from django.conf import settings
from django.contrib.auth.models import Permission
from django.contrib.auth.models import User
from django.test import TestCase
from django.test import override_settings
from django.utils import timezone
from httpx_oauth.oauth2 import GetAccessTokenError
from httpx_oauth.oauth2 import RefreshTokenError
from rest_framework import status
from paperless_mail.mail import MailAccountHandler
from paperless_mail.models import MailAccount
from paperless_mail.oauth import PaperlessMailOAuth2Manager
class TestMailOAuth(
TestCase,
):
def setUp(self) -> None:
self.user = User.objects.create_user("testuser")
self.user.user_permissions.add(
*Permission.objects.filter(
codename__in=[
"add_mailaccount",
],
),
)
self.user.save()
self.client.force_login(self.user)
self.mail_account_handler = MailAccountHandler()
# Mock settings
settings.OAUTH_CALLBACK_BASE_URL = "http://localhost:8000"
settings.GMAIL_OAUTH_CLIENT_ID = "test_gmail_client_id"
settings.GMAIL_OAUTH_CLIENT_SECRET = "test_gmail_client_secret"
settings.OUTLOOK_OAUTH_CLIENT_ID = "test_outlook_client_id"
settings.OUTLOOK_OAUTH_CLIENT_SECRET = "test_outlook_client_secret"
super().setUp()
def test_generate_paths(self):
"""
GIVEN:
- Mocked settings for OAuth callback and base URLs
WHEN:
- get_oauth_callback_url and get_oauth_redirect_url are called
THEN:
- Correct URLs are generated
"""
# Callback URL
oauth_manager = PaperlessMailOAuth2Manager()
with override_settings(OAUTH_CALLBACK_BASE_URL="http://paperless.example.com"):
self.assertEqual(
oauth_manager.oauth_callback_url,
"http://paperless.example.com/api/oauth/callback/",
)
with override_settings(
OAUTH_CALLBACK_BASE_URL=None,
PAPERLESS_URL="http://paperless.example.com",
):
self.assertEqual(
oauth_manager.oauth_callback_url,
"http://paperless.example.com/api/oauth/callback/",
)
with override_settings(
OAUTH_CALLBACK_BASE_URL=None,
PAPERLESS_URL="http://paperless.example.com",
BASE_URL="/paperless/",
):
self.assertEqual(
oauth_manager.oauth_callback_url,
"http://paperless.example.com/paperless/api/oauth/callback/",
)
# Redirect URL
with override_settings(DEBUG=True):
self.assertEqual(
oauth_manager.oauth_redirect_url,
"http://localhost:4200/mail",
)
with override_settings(DEBUG=False):
self.assertEqual(
oauth_manager.oauth_redirect_url,
"/mail",
)
@mock.patch(
"paperless_mail.oauth.PaperlessMailOAuth2Manager.get_gmail_access_token",
)
@mock.patch(
"paperless_mail.oauth.PaperlessMailOAuth2Manager.get_outlook_access_token",
)
def test_oauth_callback_view_success(
self,
mock_get_outlook_access_token,
mock_get_gmail_access_token,
):
"""
GIVEN:
- Mocked settings for Gmail and Outlook OAuth client IDs and secrets
WHEN:
- OAuth callback is called with a code and scope
- OAuth callback is called with a code and no scope
THEN:
- Gmail mail account is created
- Outlook mail account is created
"""
mock_get_gmail_access_token.return_value = {
"access_token": "test_access_token",
"refresh_token": "test_refresh_token",
"expires_in": 3600,
}
mock_get_outlook_access_token.return_value = {
"access_token": "test_access_token",
"refresh_token": "test_refresh_token",
"expires_in": 3600,
}
# Test Google OAuth callback
response = self.client.get(
"/api/oauth/callback/?code=test_code&scope=https://mail.google.com/",
)
self.assertEqual(response.status_code, status.HTTP_302_FOUND)
self.assertIn("oauth_success=1", response.url)
mock_get_gmail_access_token.assert_called_once()
self.assertTrue(
MailAccount.objects.filter(imap_server="imap.gmail.com").exists(),
)
# Test Outlook OAuth callback
response = self.client.get("/api/oauth/callback/?code=test_code")
self.assertEqual(response.status_code, status.HTTP_302_FOUND)
self.assertIn("oauth_success=1", response.url)
self.assertTrue(
MailAccount.objects.filter(imap_server="outlook.office365.com").exists(),
)
@mock.patch("httpx_oauth.oauth2.BaseOAuth2.get_access_token")
def test_oauth_callback_view_fails(self, mock_get_access_token):
"""
GIVEN:
- Mocked settings for Gmail and Outlook OAuth client IDs and secrets
WHEN:
- OAuth callback is called and get access token returns an error
THEN:
- No mail account is created
- Error is logged
"""
mock_get_access_token.side_effect = GetAccessTokenError("test_error")
with self.assertLogs("paperless_mail", level="ERROR") as cm:
# Test Google OAuth callback
response = self.client.get(
"/api/oauth/callback/?code=test_code&scope=https://mail.google.com/",
)
self.assertEqual(response.status_code, status.HTTP_302_FOUND)
self.assertIn("oauth_success=0", response.url)
self.assertFalse(
MailAccount.objects.filter(imap_server="imap.gmail.com").exists(),
)
# Test Outlook OAuth callback
response = self.client.get("/api/oauth/callback/?code=test_code")
self.assertEqual(response.status_code, status.HTTP_302_FOUND)
self.assertIn("oauth_success=0", response.url)
self.assertFalse(
MailAccount.objects.filter(
imap_server="outlook.office365.com",
).exists(),
)
self.assertIn("Error getting access token: test_error", cm.output[0])
def test_oauth_callback_view_insufficient_permissions(self):
"""
GIVEN:
- Mocked settings for Gmail and Outlook OAuth client IDs and secrets
- User without add_mailaccount permission
WHEN:
- OAuth callback is called
THEN:
- 400 bad request returned, no mail accounts are created
"""
self.user.user_permissions.remove(
*Permission.objects.filter(
codename__in=[
"add_mailaccount",
],
),
)
self.user.save()
response = self.client.get(
"/api/oauth/callback/?code=test_code&scope=https://mail.google.com/",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertFalse(
MailAccount.objects.filter(imap_server="imap.gmail.com").exists(),
)
self.assertFalse(
MailAccount.objects.filter(imap_server="outlook.office365.com").exists(),
)
def test_oauth_callback_view_no_code(self):
"""
GIVEN:
- Mocked settings for Gmail and Outlook OAuth client IDs and secrets
WHEN:
- OAuth callback is called without a code
THEN:
- 400 bad request returned, no mail accounts are created
"""
response = self.client.get(
"/api/oauth/callback/",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertFalse(
MailAccount.objects.filter(imap_server="imap.gmail.com").exists(),
)
self.assertFalse(
MailAccount.objects.filter(imap_server="outlook.office365.com").exists(),
)
@mock.patch("paperless_mail.mail.get_mailbox")
@mock.patch(
"httpx_oauth.oauth2.BaseOAuth2.refresh_token",
)
def test_refresh_token_on_handle_mail_account(
self,
mock_refresh_token,
mock_get_mailbox,
):
"""
GIVEN:
- Mail account with refresh token and expiration
WHEN:
- handle_mail_account is called
THEN:
- Refresh token is called
"""
mock_mailbox = mock.MagicMock()
mock_get_mailbox.return_value.__enter__.return_value = mock_mailbox
mail_account = MailAccount.objects.create(
name="Test Gmail Mail Account",
username="test_username",
imap_security=MailAccount.ImapSecurity.SSL,
imap_port=993,
account_type=MailAccount.MailAccountType.GMAIL_OAUTH,
is_token=True,
refresh_token="test_refresh_token",
expiration=timezone.now() - timedelta(days=1),
)
mock_refresh_token.return_value = {
"access_token": "test_access_token",
"refresh_token": "test_refresh_token",
"expires_in": 3600,
}
self.mail_account_handler.handle_mail_account(mail_account)
mock_refresh_token.assert_called_once()
mock_refresh_token.reset_mock()
mock_refresh_token.return_value = {
"access_token": "test_access_token",
"refresh_token": "test_refresh",
"expires_in": 3600,
}
outlook_mail_account = MailAccount.objects.create(
name="Test Outlook Mail Account",
username="test_username",
imap_security=MailAccount.ImapSecurity.SSL,
imap_port=993,
account_type=MailAccount.MailAccountType.OUTLOOK_OAUTH,
is_token=True,
refresh_token="test_refresh_token",
expiration=timezone.now() - timedelta(days=1),
)
self.mail_account_handler.handle_mail_account(outlook_mail_account)
mock_refresh_token.assert_called_once()
@mock.patch("paperless_mail.mail.get_mailbox")
@mock.patch(
"httpx_oauth.oauth2.BaseOAuth2.refresh_token",
)
def test_refresh_token_on_handle_mail_account_fails(
self,
mock_refresh_token,
mock_get_mailbox,
):
"""
GIVEN:
- Mail account with refresh token and expiration
WHEN:
- handle_mail_account is called
- Refresh token is called but fails
THEN:
- Error is logged
- 0 processed mails is returned
"""
mock_mailbox = mock.MagicMock()
mock_get_mailbox.return_value.__enter__.return_value = mock_mailbox
mail_account = MailAccount.objects.create(
name="Test Gmail Mail Account",
username="test_username",
imap_security=MailAccount.ImapSecurity.SSL,
imap_port=993,
account_type=MailAccount.MailAccountType.GMAIL_OAUTH,
is_token=True,
refresh_token="test_refresh_token",
expiration=timezone.now() - timedelta(days=1),
)
mock_refresh_token.side_effect = RefreshTokenError("test_error")
with self.assertLogs("paperless_mail", level="ERROR") as cm:
# returns 0 processed mails
self.assertEqual(
self.mail_account_handler.handle_mail_account(mail_account),
0,
)
mock_refresh_token.assert_called_once()
self.assertIn(
f"Failed to refresh oauth token for account {mail_account}: test_error",
cm.output[0],
)

106
src/paperless_mail/views.py
View File

@ -1,7 +1,11 @@
import datetime
import logging
from datetime import timedelta
from django.http import HttpResponseBadRequest
from django.http import HttpResponseRedirect
from django.utils import timezone
from httpx_oauth.oauth2 import GetAccessTokenError
from rest_framework.generics import GenericAPIView
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
@ -16,6 +20,7 @@ from paperless_mail.mail import get_mailbox
from paperless_mail.mail import mailbox_login
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from paperless_mail.oauth import PaperlessMailOAuth2Manager
from paperless_mail.serialisers import MailAccountSerializer
from paperless_mail.serialisers import MailRuleSerializer
@ -50,27 +55,114 @@ class MailAccountTestView(GenericAPIView):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
# account exists, use the password from there instead of ***
# account exists, use the password from there instead of *** and refresh_token / expiration
if (
len(serializer.validated_data.get("password").replace("*", "")) == 0
and request.data["id"] is not None
):
serializer.validated_data["password"] = MailAccount.objects.get(
pk=request.data["id"],
).password
existing_account = MailAccount.objects.get(pk=request.data["id"])
serializer.validated_data["password"] = existing_account.password
serializer.validated_data["account_type"] = existing_account.account_type
serializer.validated_data["refresh_token"] = existing_account.refresh_token
serializer.validated_data["expiration"] = existing_account.expiration
account = MailAccount(**serializer.validated_data)
with get_mailbox(
account.imap_server,
account.imap_port,
account.imap_security,
) as M:
try:
if (
account.is_token
and account.expiration is not None
and account.expiration < timezone.now()
):
oauth_manager = PaperlessMailOAuth2Manager()
if oauth_manager.refresh_account_oauth_token(existing_account):
# User is not changing password and token needs to be refreshed
existing_account.refresh_from_db()
account.password = existing_account.password
else:
raise MailError("Unable to refresh oauth token")
mailbox_login(M, account)
return Response({"success": True})
except MailError:
except MailError as e:
logger.error(
f"Mail account {account} test failed",
f"Mail account {account} test failed: {e}",
)
return HttpResponseBadRequest("Unable to connect to server")
class OauthCallbackView(GenericAPIView):
permission_classes = (IsAuthenticated,)
def get(self, request, format=None):
if not (
request.user and request.user.has_perms(["paperless_mail.add_mailaccount"])
):
return HttpResponseBadRequest(
"You do not have permission to add mail accounts",
)
logger = logging.getLogger("paperless_mail")
code = request.query_params.get("code")
# Gmail passes scope as a query param, Outlook does not
scope = request.query_params.get("scope")
if code is None:
logger.error(
f"Invalid oauth callback request, code: {code}, scope: {scope}",
)
return HttpResponseBadRequest("Invalid request, see logs for more detail")
oauth_manager = PaperlessMailOAuth2Manager()
try:
if scope is not None and "google" in scope:
# Google
account_type = MailAccount.MailAccountType.GMAIL_OAUTH
imap_server = "imap.gmail.com"
defaults = {
"name": f"Gmail OAuth {timezone.now()}",
"username": "",
"imap_security": MailAccount.ImapSecurity.SSL,
"imap_port": 993,
"account_type": account_type,
}
result = oauth_manager.get_gmail_access_token(code)
elif scope is None:
# Outlook
account_type = MailAccount.MailAccountType.OUTLOOK_OAUTH
imap_server = "outlook.office365.com"
defaults = {
"name": f"Outlook OAuth {timezone.now()}",
"username": "",
"imap_security": MailAccount.ImapSecurity.SSL,
"imap_port": 993,
"account_type": account_type,
}
result = oauth_manager.get_outlook_access_token(code)
access_token = result["access_token"]
refresh_token = result["refresh_token"]
expires_in = result["expires_in"]
account, _ = MailAccount.objects.update_or_create(
password=access_token,
is_token=True,
imap_server=imap_server,
refresh_token=refresh_token,
expiration=timezone.now() + timedelta(seconds=expires_in),
defaults=defaults,
)
return HttpResponseRedirect(
f"{oauth_manager.oauth_redirect_url}?oauth_success=1&account_id={account.pk}",
)
except GetAccessTokenError as e:
logger.error(f"Error getting access token: {e}")
return HttpResponseRedirect(
f"{oauth_manager.oauth_redirect_url}?oauth_success=0",
)