Updated Backend Ideas List (markdown)

Backend-Ideas-List.md

@@ -20,6 +20,7 @@
 - This would be streamed the document, eventually return the content and an optional archive file
 - Is time consuming, so might need celery/huey/task queue there?  And a database?
 - fastapi could easily set this up, if there is no need for a database.
+- Could use Redis/Valkey streams to manage state and show progress without a database
 
 ### External Machine Learning
 
@@ -59,6 +60,18 @@
 ### Blockers
 
 - Would need to implement Token based authentication
-  - Could track, with some resolution, when a token was last used.  Might be nice to display
+  - Could track, with some resolution, when a token was last used.  Might be nice to display and allow removing old tokens which haven't been used
   - Could implement expiration too
-- Async pagination isn't working quite yet
+- Async pagination isn't working quite yet
+
+## Vector Embeddings
+
+- This would require either a new database or everyone to use the same database (ala Immich with pgvecto.rs)
+- Would enable semantic search, document similarity search
+  - Maybe replacing Whoosh entirely?
+
+## New Sanity Checker
+
+- Sanity checker messages are attached to a document
+- Can be dismissed (but still viewed)
+- Visible in the UI somehow