Bulk document upload and deduplicated file storage

- Bulk Upload page: pick many files or a folder; each file becomes a
  document named after it, and files sharing a name become one multi-page
  document. Files go up as direct uploads with per-file progress.
- Direct uploads require a signed-in user and check type/size up front;
  only fresh, unattached blobs can be attached; abandoned uploads are
  purged nightly.
- DedupDisk storage service: each distinct file is stored once under its
  SHA-256 and blobs hard-link to it; nightly sweep (and storage:dedupe)
  links pre-existing duplicates and drops unused objects.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
rickyandClaude Opus 5.5 committed 2026-11-07 08:20:22 -05:00
1 parent 230a569b77
commit 2bfcb845ca
15 files changed
+735 -5

No files matched your search

+1 -1
View File
@@ -14,7 +14,7 @@ module SimpleAircraftManager
# Please, add to the `ignore` list any other `lib` subdirectories that do
# not contain `.rb` files, or that should not be reloaded or eager loaded.
# Common ones are `templates`, `generators`, or `middleware`, for example.
config.autoload_lib(ignore: %w[assets tasks])
config.autoload_lib(ignore: %w[active_storage assets tasks])
# Configuration for the application, engines, and railties goes here.
#
+1
View File
@@ -8,3 +8,4 @@ pin_all_from "app/javascript/controllers", under: "controllers"
pin "chartkick", to: "chartkick.js"
pin "Chart.bundle", to: "Chart.bundle.js"
pin "next_due"
pin "@rails/activestorage", to: "activestorage.esm.js"
+6
View File
@@ -13,3 +13,9 @@ production:
clear_solid_queue_finished_jobs:
command: "SolidQueue::Job.clear_finished_in_batches(sleep_between_batches: 0.3)"
schedule: every hour at minute 12
purge_abandoned_uploads:
command: "ActiveStorage::Blob.unattached.where(created_at: ..2.days.ago).find_each(&:purge_later)"
schedule: every day at 4am
dedupe_storage:
command: "ActiveStorage::Blob.service.try(:sweep)"
schedule: every day at 4:30am
+10 -1
View File
@@ -1,4 +1,8 @@
Rails.application.routes.draw do
# Signed-in users only, with the upload rules checked before any bytes are sent
# (overrides Active Storage's open endpoint at the same path).
post "rails/active_storage/direct_uploads", to: "direct_uploads#create"
resource :session
resources :passwords, param: :token
resource :setup, only: %i[new create], controller: "setup"
@@ -45,7 +49,12 @@ Rails.application.routes.draw do
end
resources :logbook_imports, path: "logbook/import", only: %i[new create show]
resources :logbook_entries
resources :documents
resources :documents do
collection do
get :bulk, action: :bulk_new, as: :new_bulk
post :bulk, action: :bulk_create
end
end
resources :document_collections, except: %i[index show]
resources :inspections, only: :index
resources :inspection_types, except: %i[index show]
+2 -2
View File
@@ -1,9 +1,9 @@
test:
service: Disk
service: DedupDisk
root: <%= Rails.root.join("tmp/storage") %>
local:
service: Disk
service: DedupDisk
root: <%= Rails.root.join("storage") %>
# Use bin/rails credentials:edit to set the AWS secrets (as aws:access_key_id|secret_access_key)