mirror of
https://github.com/rickbarrette/simple-aircraft-manager-rails.git
synced 2026-11-17 23:50:36 -05:00
Off-site backup to Google Drive
- Admin → Off-site Backup: connect a Google account with your own OAuth client (drive.file scope; refresh token and secret encrypted) - Nightly after the verified database backup: upload the gzipped snapshot (newest N kept, Settings) and every uploaded file not yet copied - Restore test downloads the newest copy, opens and checks it, and checks sampled files against their checksums; failures emailed to admins - About, procedures document, policy completeness and audits use it - offsite:restore_files rake task; restore steps in the admin guide Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
fe3fba1736
commit
7694719582
31 files changed
+853
-20
No files matched your search
@@ -91,6 +91,8 @@ rsync -aH --exclude '*.sqlite3*' storage/ backup/ # uploaded files (-H keeps th
|
|||||||
|
|
||||||
The app also makes a verified copy of the main database every night in `storage/backups` (Admin → About shows the last one), so a backup of the storage volume always contains a consistent database snapshot.
|
The app also makes a verified copy of the main database every night in `storage/backups` (Admin → About shows the last one), so a backup of the storage volume always contains a consistent database snapshot.
|
||||||
|
|
||||||
|
For an off-site copy without extra tools, **Admin → Off-site Backup** connects a Google Drive account: each night the verified database copy and any new uploaded files are copied there, and a restore test checks them. Setup and restoring are in the [administration guide](docs/guide/administration.md#off-site-backup-to-google-drive).
|
||||||
|
|
||||||
Alternatively, stop the container and copy the volume.
|
Alternatively, stop the container and copy the volume.
|
||||||
|
|
||||||
To back up or move a single aircraft with all its files, use **Access & Sharing → Download .sam.zip** (or `bin/rails "sam:export[N12345]"`). Import it from **Fleet → Import** (or `bin/rails "sam:import[file.sam.zip,owner@example.com]"`). The archive format is the original Simple Aircraft Manager's, so archives move between the two apps.
|
To back up or move a single aircraft with all its files, use **Access & Sharing → Download .sam.zip** (or `bin/rails "sam:export[N12345]"`). Import it from **Fleet → Import** (or `bin/rails "sam:import[file.sam.zip,owner@example.com]"`). The archive format is the original Simple Aircraft Manager's, so archives move between the two apps.
|
||||||
@@ -114,6 +116,7 @@ To back up or move a single aircraft with all its files, use **Access & Sharing
|
|||||||
| Documents | Collections, multi-file documents with a page viewer, per-document share visibility, bulk upload, and expiration dates (registration, insurance…). |
|
| Documents | Collections, multi-file documents with a page viewer, per-document share visibility, bulk upload, and expiration dates (registration, insurance…). |
|
||||||
| Electronic signatures | Logbook entries can be signed electronically along the lines of AC 120-78B chapter 2: the signer reviews the entry, picks their certificate, re-enters their password (or signs in again with SSO) and confirms a "Certified by" statement. The signature records who, which certificate, the statement, the time and a SHA-256 fingerprint of the entry, sealed by the server; the signer gets an email. Signed entries are locked: a correction is a new signed entry that supersedes the original. Owners can invite an outside mechanic to sign one entry with their own account. Disabled accounts can't sign; their past signatures stay. |
|
| Electronic signatures | Logbook entries can be signed electronically along the lines of AC 120-78B chapter 2: the signer reviews the entry, picks their certificate, re-enters their password (or signs in again with SSO) and confirms a "Certified by" statement. The signature records who, which certificate, the statement, the time and a SHA-256 fingerprint of the entry, sealed by the server; the signer gets an email. Signed entries are locked: a correction is a new signed entry that supersedes the original. Owners can invite an outside mechanic to sign one entry with their own account. Disabled accounts can't sign; their past signatures stay. |
|
||||||
| Records integrity | Every change to maintenance records is kept (who, when, field by field) in a hash chain per aircraft (Activity → Record history). Nightly: a verified SQLite backup in `storage/backups` (how many are kept: Admin → Settings), then a check of the history chains and every signature; admins are emailed on a problem (Admin → About). Software versions are recorded. A records package (Access & Sharing) bundles every entry as a PDF with signatures, a signature register, the history, the status report and the `.sam.zip` for the FAA or a buyer. Scans of paper logbooks can be marked as legacy records, with an "electronic records since" date per aircraft. |
|
| Records integrity | Every change to maintenance records is kept (who, when, field by field) in a hash chain per aircraft (Activity → Record history). Nightly: a verified SQLite backup in `storage/backups` (how many are kept: Admin → Settings), then a check of the history chains and every signature; admins are emailed on a problem (Admin → About). Software versions are recorded. A records package (Access & Sharing) bundles every entry as a PDF with signatures, a signature register, the history, the status report and the `.sam.zip` for the FAA or a buyer. Scans of paper logbooks can be marked as legacy records, with an "electronic records since" date per aircraft. |
|
||||||
|
| Off-site backup | Nightly copy of the verified database snapshot (newest 30 kept) and every new uploaded file to Google Drive, using your own Google OAuth client and the `drive.file` scope (the app sees only its own files). Restore test from the admin page; failures emailed to admins; `bin/rails offsite:restore_files` brings files back after a restore. |
|
||||||
| Recordkeeping procedures and audits | Admin → Recordkeeping Procedures generates the procedures document AC 120-78B asks for (who is responsible, who may sign, signing, corrections, backups, access, audits, transfer, legacy records) from the system setup plus a few details you fill in; adopt it as numbered, unchangeable revisions, included in the records package. Admin → Records Audits is a periodic QC checklist (app checks plus a restore test, a sample of entries against source documents and a user review); admins get the due date in the weekly reminder. The users' instructions are the guide's [Electronic records and signing](docs/guide/records.md) page. |
|
| Recordkeeping procedures and audits | Admin → Recordkeeping Procedures generates the procedures document AC 120-78B asks for (who is responsible, who may sign, signing, corrections, backups, access, audits, transfer, legacy records) from the system setup plus a few details you fill in; adopt it as numbered, unchangeable revisions, included in the records package. Admin → Records Audits is a periodic QC checklist (app checks plus a restore test, a sample of entries against source documents and a user review); admins get the due date in the weekly reminder. The users' instructions are the guide's [Electronic records and signing](docs/guide/records.md) page. |
|
||||||
| Major repairs and alterations | Form 337, STC, and ICA tracking; an STC's ICA can become a tracked recurring inspection. |
|
| Major repairs and alterations | Form 337, STC, and ICA tracking; an STC's ICA can become a tracked recurring inspection. |
|
||||||
| Weight & balance | History of empty weight and CG (linked to the alteration that changed them) and a loading calculator: takeoff and landing weight and CG against max gross and the CG envelope. |
|
| Weight & balance | History of empty weight and CG (linked to the alteration that changed them) and a loading calculator: takeoff and landing weight and CG against max gross and the CG envelope. |
|
||||||
|
|||||||
@@ -0,0 +1,69 @@
|
|||||||
|
# Admin → Off-site Backup: connect a Google Drive account the nightly
|
||||||
|
# backup is copied to, run a backup or a restore test now.
|
||||||
|
class Admin::OffsiteBackupsController < ApplicationController
|
||||||
|
before_action :require_admin!
|
||||||
|
before_action :set_connection
|
||||||
|
|
||||||
|
def show
|
||||||
|
@runs = OffsiteRun.order(created_at: :desc, id: :desc).limit(15)
|
||||||
|
@files_copied = OffsiteFile.count
|
||||||
|
@files_total = ActiveStorage::Blob.count
|
||||||
|
end
|
||||||
|
|
||||||
|
def update
|
||||||
|
attrs = params.require(:offsite_connection).permit(:client_id, :client_secret)
|
||||||
|
attrs.delete(:client_secret) if attrs[:client_secret].blank?
|
||||||
|
changed_client = attrs[:client_id].present? && attrs[:client_id].strip != @connection.client_id
|
||||||
|
@connection.disconnect! if changed_client && @connection.connected?
|
||||||
|
@connection.update!(attrs.transform_values(&:strip))
|
||||||
|
redirect_to admin_offsite_backup_path, notice: "Saved. Now connect your Google account."
|
||||||
|
end
|
||||||
|
|
||||||
|
# Off to Google to grant access (a link, so no form-action CSP entry is needed).
|
||||||
|
def connect
|
||||||
|
return redirect_to(admin_offsite_backup_path, alert: "Enter the Google OAuth client ID and secret first.") unless @connection.configured?
|
||||||
|
session[:offsite_oauth_state] = state = SecureRandom.urlsafe_base64(24)
|
||||||
|
redirect_to GoogleDriveClient.authorize_url(client_id: @connection.client_id, redirect_uri: @connection.redirect_uri, state: state),
|
||||||
|
allow_other_host: true
|
||||||
|
end
|
||||||
|
|
||||||
|
def callback
|
||||||
|
expected = session.delete(:offsite_oauth_state)
|
||||||
|
unless expected.present? && params[:state].present? && ActiveSupport::SecurityUtils.secure_compare(expected, params[:state].to_s)
|
||||||
|
return redirect_to(admin_offsite_backup_path, alert: "That Google sign-in didn't match a request from here. Try connecting again.")
|
||||||
|
end
|
||||||
|
return redirect_to(admin_offsite_backup_path, alert: "Google Drive wasn't connected (#{params[:error]}).") if params[:error].present? || params[:code].blank?
|
||||||
|
|
||||||
|
token = GoogleDriveClient.exchange_code(client_id: @connection.client_id, client_secret: @connection.client_secret,
|
||||||
|
code: params[:code].to_s, redirect_uri: @connection.redirect_uri)
|
||||||
|
@connection.update!(refresh_token: token, connected_by: current_user, connected_at: Time.current)
|
||||||
|
drive = @connection.client
|
||||||
|
@connection.update!(account_email: drive.account_email)
|
||||||
|
@connection.ensure_folders!(drive)
|
||||||
|
Rails.logger.info("[offsite] #{current_user.email_address} connected Google Drive #{@connection.account_email}")
|
||||||
|
OffsiteBackupJob.perform_later
|
||||||
|
redirect_to admin_offsite_backup_path, notice: "Google Drive connected (#{@connection.account_email}). The first off-site backup has started; it copies every uploaded file, so it can take a while."
|
||||||
|
rescue GoogleDriveClient::Error => e
|
||||||
|
redirect_to admin_offsite_backup_path, alert: "Couldn't connect Google Drive: #{e.message}"
|
||||||
|
end
|
||||||
|
|
||||||
|
def destroy
|
||||||
|
@connection.disconnect!
|
||||||
|
Rails.logger.info("[offsite] #{current_user.email_address} disconnected Google Drive")
|
||||||
|
redirect_to admin_offsite_backup_path, notice: "Google Drive disconnected. The copies already there stay in your Drive."
|
||||||
|
end
|
||||||
|
|
||||||
|
def backup_now
|
||||||
|
OffsiteBackupJob.perform_later
|
||||||
|
redirect_to admin_offsite_backup_path, notice: "Off-site backup started. Refresh in a minute to see the result."
|
||||||
|
end
|
||||||
|
|
||||||
|
def restore_test
|
||||||
|
OffsiteBackupJob.perform_later("restore_test")
|
||||||
|
redirect_to admin_offsite_backup_path, notice: "Restore test started. Refresh in a minute to see the result."
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def set_connection = @connection = OffsiteConnection.current
|
||||||
|
end
|
||||||
@@ -28,7 +28,7 @@ class Admin::ProceduresController < ApplicationController
|
|||||||
|
|
||||||
def adopt
|
def adopt
|
||||||
policy = RecordkeepingPolicy.current
|
policy = RecordkeepingPolicy.current
|
||||||
return redirect_to(admin_procedures_path, alert: "Fill in the operator, responsible person, hosting and off-site backup first.") unless policy.complete?
|
return redirect_to(admin_procedures_path, alert: "Fill in the operator, responsible person, hosting and off-site backup (or connect Google Drive) first.") unless policy.complete?
|
||||||
number = (ProceduresRevision.maximum(:number) || 0) + 1
|
number = (ProceduresRevision.maximum(:number) || 0) + 1
|
||||||
revision = ProceduresRevision.create!(number: number, adopted_by: current_user, adopted_at: Time.current,
|
revision = ProceduresRevision.create!(number: number, adopted_by: current_user, adopted_at: Time.current,
|
||||||
summary: params[:summary].to_s.strip.first(500).presence || "Revision #{number}",
|
summary: params[:summary].to_s.strip.first(500).presence || "Revision #{number}",
|
||||||
|
|||||||
@@ -2,18 +2,18 @@ class Admin::SettingsController < ApplicationController
|
|||||||
before_action :require_admin!
|
before_action :require_admin!
|
||||||
|
|
||||||
def edit
|
def edit
|
||||||
@values = { "max_upload_mb" => Setting.max_upload_mb, "backups_kept" => Setting.integer("backups_kept") }
|
@values = %w[max_upload_mb backups_kept offsite_backups_kept].index_with { Setting.integer(_1) }
|
||||||
end
|
end
|
||||||
|
|
||||||
def update
|
def update
|
||||||
submitted = params.fetch(:settings, {}).permit(:max_upload_mb, :backups_kept).to_h
|
submitted = params.fetch(:settings, {}).permit(:max_upload_mb, :backups_kept, :offsite_backups_kept).to_h
|
||||||
records = Setting.update_integers(submitted)
|
records = Setting.update_integers(submitted)
|
||||||
@errors = records.flat_map { _1.errors.full_messages }
|
@errors = records.flat_map { _1.errors.full_messages }
|
||||||
if @errors.empty?
|
if @errors.empty?
|
||||||
Rails.logger.info("[settings] #{current_user.email_address} changed #{submitted.inspect}")
|
Rails.logger.info("[settings] #{current_user.email_address} changed #{submitted.inspect}")
|
||||||
redirect_to edit_admin_settings_path, notice: "Settings saved."
|
redirect_to edit_admin_settings_path, notice: "Settings saved."
|
||||||
else
|
else
|
||||||
@values = { "max_upload_mb" => Setting.max_upload_mb, "backups_kept" => Setting.integer("backups_kept") }.merge(submitted)
|
@values = %w[max_upload_mb backups_kept offsite_backups_kept].index_with { Setting.integer(_1) }.merge(submitted)
|
||||||
render :edit, status: :unprocessable_entity
|
render :edit, status: :unprocessable_entity
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
# Copies the latest verified database backup and new uploaded files to
|
||||||
|
# Google Drive (queued by the nightly RecordsIntegrityJob, or from Admin →
|
||||||
|
# Off-site Backup). Admins are emailed if it fails.
|
||||||
|
class OffsiteBackupJob < ApplicationJob
|
||||||
|
queue_as :default
|
||||||
|
limits_concurrency to: 1, key: "offsite_backup", duration: 12.hours if respond_to?(:limits_concurrency)
|
||||||
|
|
||||||
|
def perform(kind = "backup")
|
||||||
|
return unless OffsiteConnection.connected?
|
||||||
|
run = kind == "restore_test" ? OffsiteBackup.restore_test! : backup
|
||||||
|
return if run.nil? || run.ok || !MailSettings.configured?
|
||||||
|
User.active.where(admin: true).find_each { AdminMailer.offsite_alert(_1, run).deliver_now }
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def backup
|
||||||
|
latest = BackupRun.where(ok: true).order(:created_at, :id).last
|
||||||
|
latest = DatabaseBackup.run! unless latest && latest.created_at > 1.day.ago && latest.path.present? && File.exist?(latest.path)
|
||||||
|
return OffsiteRun.create!(kind: "backup", ok: false, message: "No verified database backup to copy: #{latest.message}") unless latest.ok
|
||||||
|
OffsiteBackup.run!(latest.path)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -1,4 +1,5 @@
|
|||||||
# Nightly: back up the database, then verify the records. Admins are emailed
|
# Nightly: back up the database, then verify the records, then copy the
|
||||||
|
# backup off-site (Google Drive) if connected. Admins are emailed
|
||||||
# if either fails.
|
# if either fails.
|
||||||
class RecordsIntegrityJob < ApplicationJob
|
class RecordsIntegrityJob < ApplicationJob
|
||||||
queue_as :default
|
queue_as :default
|
||||||
@@ -6,6 +7,7 @@ class RecordsIntegrityJob < ApplicationJob
|
|||||||
def perform
|
def perform
|
||||||
backup = DatabaseBackup.run!
|
backup = DatabaseBackup.run!
|
||||||
check = RecordsIntegrity.check!
|
check = RecordsIntegrity.check!
|
||||||
|
OffsiteBackupJob.perform_later if backup.ok && OffsiteConnection.connected?
|
||||||
return if backup.ok && check.ok
|
return if backup.ok && check.ok
|
||||||
return unless MailSettings.configured?
|
return unless MailSettings.configured?
|
||||||
User.active.where(admin: true).find_each { AdminMailer.records_alert(_1, backup, check).deliver_now }
|
User.active.where(admin: true).find_each { AdminMailer.records_alert(_1, backup, check).deliver_now }
|
||||||
|
|||||||
@@ -5,6 +5,11 @@ class AdminMailer < ApplicationMailer
|
|||||||
mail subject: "Records check needs attention", to: user.email_address
|
mail subject: "Records check needs attention", to: user.email_address
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def offsite_alert(user, run)
|
||||||
|
@run = run
|
||||||
|
mail subject: run.kind == "restore_test" ? "Off-site restore test failed" : "Off-site backup failed", to: user.email_address
|
||||||
|
end
|
||||||
|
|
||||||
def test_email(user)
|
def test_email(user)
|
||||||
@user = user
|
@user = user
|
||||||
mail subject: "Test email from Simple Aircraft Manager", to: user.email_address
|
mail subject: "Test email from Simple Aircraft Manager", to: user.email_address
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
# Off-site backup to Google Drive (AC 120-78B 3.2.7). One row for the
|
||||||
|
# installation: the admin's own Google OAuth client and the refresh token
|
||||||
|
# from connecting an account. The app only has access to files it created
|
||||||
|
# (the drive.file scope).
|
||||||
|
class OffsiteConnection < ApplicationRecord
|
||||||
|
ROOT_FOLDER = "Simple Aircraft Manager backups"
|
||||||
|
|
||||||
|
belongs_to :connected_by, class_name: "User", optional: true
|
||||||
|
encrypts :client_secret, :refresh_token
|
||||||
|
|
||||||
|
# Tests swap in a fake Drive.
|
||||||
|
class_attribute :client_factory, default: ->(conn) { GoogleDriveClient.new(client_id: conn.client_id, client_secret: conn.client_secret, refresh_token: conn.refresh_token) }
|
||||||
|
|
||||||
|
def self.current = first || create!
|
||||||
|
def self.connected? = current.connected?
|
||||||
|
|
||||||
|
def configured? = client_id.present? && client_secret.present?
|
||||||
|
def connected? = configured? && refresh_token.present?
|
||||||
|
def client = client_factory.call(self)
|
||||||
|
|
||||||
|
def redirect_uri = "#{Sso.app_url}/admin/offsite_backup/callback"
|
||||||
|
|
||||||
|
def disconnect!
|
||||||
|
begin
|
||||||
|
GoogleDriveClient.revoke(refresh_token) if refresh_token.present?
|
||||||
|
rescue GoogleDriveClient::Error
|
||||||
|
nil # already revoked or unreachable; forget it anyway
|
||||||
|
end
|
||||||
|
update!(refresh_token: nil, account_email: nil, folder_id: nil, database_folder_id: nil, files_folder_id: nil, connected_at: nil)
|
||||||
|
end
|
||||||
|
|
||||||
|
# Folders in Drive, made on first use (again if someone deleted them).
|
||||||
|
def ensure_folders!(drive)
|
||||||
|
changes = {}
|
||||||
|
root = folder_id if folder_id.present? && drive.folder?(folder_id)
|
||||||
|
root ||= changes[:folder_id] = drive.create_folder("#{ROOT_FOLDER} (#{URI.parse(Sso.app_url).host})")
|
||||||
|
{ database_folder_id: "database", files_folder_id: "files" }.each do |column, name|
|
||||||
|
id = self[column]
|
||||||
|
next if id.present? && !changes.key?(:folder_id) && drive.folder?(id)
|
||||||
|
changes[column] = drive.create_folder(name, parent: root)
|
||||||
|
end
|
||||||
|
update!(changes) if changes.any?
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
class OffsiteFile < ApplicationRecord
|
||||||
|
belongs_to :blob, class_name: "ActiveStorage::Blob"
|
||||||
|
end
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
class OffsiteRun < ApplicationRecord
|
||||||
|
KINDS = %w[backup restore_test].freeze
|
||||||
|
validates :kind, inclusion: { in: KINDS }
|
||||||
|
|
||||||
|
scope :backups, -> { where(kind: "backup") }
|
||||||
|
scope :restore_tests, -> { where(kind: "restore_test") }
|
||||||
|
|
||||||
|
def self.latest_backup = backups.order(:created_at, :id).last
|
||||||
|
def self.latest_restore_test = restore_tests.order(:created_at, :id).last
|
||||||
|
def recent? = created_at > 2.days.ago
|
||||||
|
end
|
||||||
@@ -7,5 +7,8 @@ class RecordkeepingPolicy < ApplicationRecord
|
|||||||
|
|
||||||
def self.current = first || create!
|
def self.current = first || create!
|
||||||
|
|
||||||
def complete? = [ operator_name, responsible_name, hosting_description, offsite_backup_description ].all?(&:present?)
|
def complete? = [ operator_name, responsible_name, hosting_description ].all?(&:present?) && offsite_backup?
|
||||||
|
|
||||||
|
# Off-site backup is described, or done by the app to Google Drive.
|
||||||
|
def offsite_backup? = offsite_backup_description.present? || OffsiteConnection.connected?
|
||||||
end
|
end
|
||||||
@@ -8,6 +8,7 @@ class Setting < ApplicationRecord
|
|||||||
INTEGERS = {
|
INTEGERS = {
|
||||||
"max_upload_mb" => { default: 50, in: 1..1024, label: "Maximum upload size (MB)" },
|
"max_upload_mb" => { default: 50, in: 1..1024, label: "Maximum upload size (MB)" },
|
||||||
"backups_kept" => { default: 14, in: 1..365, label: "Database backups kept" },
|
"backups_kept" => { default: 14, in: 1..365, label: "Database backups kept" },
|
||||||
|
"offsite_backups_kept" => { default: 30, in: 1..365, label: "Database copies kept in Google Drive" },
|
||||||
"logbook_templates_seeded" => { default: 0, in: 0..1, label: "Built-in logbook templates added" } # internal flag
|
"logbook_templates_seeded" => { default: 0, in: 0..1, label: "Built-in logbook templates added" } # internal flag
|
||||||
}.freeze
|
}.freeze
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,150 @@
|
|||||||
|
require "net/http"
|
||||||
|
|
||||||
|
# Minimal Google Drive v3 client for the off-site backup: OAuth (offline
|
||||||
|
# access, drive.file scope), folders, resumable uploads, listing,
|
||||||
|
# downloading and deleting the files this app created.
|
||||||
|
class GoogleDriveClient
|
||||||
|
class Error < StandardError; end
|
||||||
|
class AuthError < Error; end # revoked or wrong credentials: reconnect
|
||||||
|
|
||||||
|
AUTH_URL = "https://accounts.google.com/o/oauth2/v2/auth".freeze
|
||||||
|
TOKEN_URL = "https://oauth2.googleapis.com/token".freeze
|
||||||
|
REVOKE_URL = "https://oauth2.googleapis.com/revoke".freeze
|
||||||
|
API = "https://www.googleapis.com/drive/v3".freeze
|
||||||
|
UPLOAD = "https://www.googleapis.com/upload/drive/v3/files".freeze
|
||||||
|
SCOPE = "https://www.googleapis.com/auth/drive.file".freeze
|
||||||
|
FOLDER = "application/vnd.google-apps.folder".freeze
|
||||||
|
|
||||||
|
def self.authorize_url(client_id:, redirect_uri:, state:)
|
||||||
|
"#{AUTH_URL}?#{URI.encode_www_form(client_id: client_id, redirect_uri: redirect_uri, response_type: 'code', scope: SCOPE,
|
||||||
|
access_type: 'offline', prompt: 'consent', include_granted_scopes: 'true', state: state)}"
|
||||||
|
end
|
||||||
|
|
||||||
|
# The refresh token from the code Google sent back.
|
||||||
|
def self.exchange_code(client_id:, client_secret:, code:, redirect_uri:)
|
||||||
|
token = token_request(client_id: client_id, client_secret: client_secret, code: code, redirect_uri: redirect_uri, grant_type: "authorization_code")
|
||||||
|
token["refresh_token"].presence or raise AuthError, "Google didn't return a refresh token; remove the app's access in your Google account and connect again"
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.revoke(token)
|
||||||
|
res = Net::HTTP.post_form(URI(REVOKE_URL), token: token)
|
||||||
|
raise Error, "revoke failed (#{res.code})" unless res.is_a?(Net::HTTPSuccess)
|
||||||
|
rescue SocketError, SystemCallError, Net::OpenTimeout, Net::ReadTimeout => e
|
||||||
|
raise Error, e.message
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.token_request(**form)
|
||||||
|
res = Net::HTTP.post_form(URI(TOKEN_URL), form)
|
||||||
|
body = JSON.parse(res.body) rescue {}
|
||||||
|
return body if res.is_a?(Net::HTTPSuccess)
|
||||||
|
raise AuthError, "Google refused the credentials (#{body['error_description'] || body['error'] || res.code})" if res.code.to_i.in?([ 400, 401 ])
|
||||||
|
raise Error, "Google token request failed (#{res.code})"
|
||||||
|
rescue SocketError, SystemCallError, Net::OpenTimeout, Net::ReadTimeout => e
|
||||||
|
raise Error, "couldn't reach Google: #{e.message}"
|
||||||
|
end
|
||||||
|
|
||||||
|
def initialize(client_id:, client_secret:, refresh_token:)
|
||||||
|
@client_id = client_id
|
||||||
|
@client_secret = client_secret
|
||||||
|
@refresh_token = refresh_token
|
||||||
|
end
|
||||||
|
|
||||||
|
def account_email = request(:get, "#{API}/about", query: { fields: "user(emailAddress)" }).dig("user", "emailAddress")
|
||||||
|
|
||||||
|
def folder?(id)
|
||||||
|
file = request(:get, "#{API}/files/#{CGI.escape(id)}", query: { fields: "id,mimeType,trashed" })
|
||||||
|
file["mimeType"] == FOLDER && !file["trashed"]
|
||||||
|
rescue Error => e
|
||||||
|
raise if e.is_a?(AuthError)
|
||||||
|
false
|
||||||
|
end
|
||||||
|
|
||||||
|
def create_folder(name, parent: nil)
|
||||||
|
request(:post, "#{API}/files", query: { fields: "id" }, json: { name: name, mimeType: FOLDER, parents: Array(parent) }).fetch("id")
|
||||||
|
end
|
||||||
|
|
||||||
|
# Uploads an IO of known size (resumable session, one PUT); returns the file id.
|
||||||
|
def upload(io, name:, parent:, size:, content_type: "application/octet-stream", properties: {})
|
||||||
|
session = request(:post, UPLOAD, query: { uploadType: "resumable", fields: "id" },
|
||||||
|
json: { name: name, parents: [ parent ], appProperties: properties },
|
||||||
|
headers: { "X-Upload-Content-Type" => content_type, "X-Upload-Content-Length" => size.to_s }, raw: true)
|
||||||
|
location = session["Location"] or raise Error, "Google didn't start the upload"
|
||||||
|
put = Net::HTTP::Put.new(URI(location))
|
||||||
|
put["Content-Length"] = size.to_s
|
||||||
|
put["Content-Type"] = content_type
|
||||||
|
put.body_stream = io
|
||||||
|
JSON.parse(perform(URI(location), put, timeout: 3600).body).fetch("id")
|
||||||
|
end
|
||||||
|
|
||||||
|
# [{ "id", "name", "size", "createdTime", "appProperties" }] in a folder.
|
||||||
|
def list(parent)
|
||||||
|
files = []
|
||||||
|
page = nil
|
||||||
|
loop do
|
||||||
|
q = { q: "'#{parent.gsub("'", "")}' in parents and trashed = false", pageSize: 1000,
|
||||||
|
fields: "nextPageToken,files(id,name,size,createdTime,appProperties)", orderBy: "name" }
|
||||||
|
q[:pageToken] = page if page
|
||||||
|
body = request(:get, "#{API}/files", query: q)
|
||||||
|
files.concat(body.fetch("files", []))
|
||||||
|
page = body["nextPageToken"] or break
|
||||||
|
end
|
||||||
|
files
|
||||||
|
end
|
||||||
|
|
||||||
|
# Streams a file's content into the IO.
|
||||||
|
def download(id, to:)
|
||||||
|
uri = URI("#{API}/files/#{CGI.escape(id)}?alt=media")
|
||||||
|
get = Net::HTTP::Get.new(uri)
|
||||||
|
get["Authorization"] = "Bearer #{access_token}"
|
||||||
|
Net::HTTP.start(uri.host, uri.port, use_ssl: true, open_timeout: 15, read_timeout: 600) do |http|
|
||||||
|
http.request(get) do |res|
|
||||||
|
raise_for(res) unless res.is_a?(Net::HTTPSuccess)
|
||||||
|
res.read_body { to.write(_1) }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
to.rewind if to.respond_to?(:rewind)
|
||||||
|
to
|
||||||
|
rescue SocketError, SystemCallError, Net::OpenTimeout, Net::ReadTimeout => e
|
||||||
|
raise Error, "couldn't reach Google Drive: #{e.message}"
|
||||||
|
end
|
||||||
|
|
||||||
|
def delete(id) = request(:delete, "#{API}/files/#{CGI.escape(id)}")
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def access_token
|
||||||
|
return @access_token if @access_token && @expires_at > Time.current
|
||||||
|
token = self.class.token_request(client_id: @client_id, client_secret: @client_secret, refresh_token: @refresh_token, grant_type: "refresh_token")
|
||||||
|
@expires_at = Time.current + token.fetch("expires_in", 3600).to_i - 60
|
||||||
|
@access_token = token.fetch("access_token")
|
||||||
|
end
|
||||||
|
|
||||||
|
def request(method, url, query: nil, json: nil, headers: {}, raw: false)
|
||||||
|
uri = URI(url)
|
||||||
|
uri.query = URI.encode_www_form(query) if query
|
||||||
|
req = Net::HTTP.const_get(method.to_s.capitalize).new(uri)
|
||||||
|
req["Authorization"] = "Bearer #{access_token}"
|
||||||
|
headers.each { req[_1] = _2 }
|
||||||
|
if json
|
||||||
|
req["Content-Type"] = "application/json; charset=UTF-8"
|
||||||
|
req.body = json.to_json
|
||||||
|
end
|
||||||
|
res = perform(uri, req)
|
||||||
|
return res if raw
|
||||||
|
res.body.present? ? JSON.parse(res.body) : {}
|
||||||
|
end
|
||||||
|
|
||||||
|
def perform(uri, req, timeout: 60)
|
||||||
|
res = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https", open_timeout: 15, read_timeout: timeout, write_timeout: timeout) { _1.request(req) }
|
||||||
|
raise_for(res) unless res.is_a?(Net::HTTPSuccess)
|
||||||
|
res
|
||||||
|
rescue SocketError, SystemCallError, Net::OpenTimeout, Net::ReadTimeout, Net::WriteTimeout => e
|
||||||
|
raise Error, "couldn't reach Google Drive: #{e.message}"
|
||||||
|
end
|
||||||
|
|
||||||
|
def raise_for(res)
|
||||||
|
message = (JSON.parse(res.body.to_s).dig("error", "message") rescue nil) || res.message
|
||||||
|
raise AuthError, "Google Drive refused access (#{message}); connect again" if res.code.to_i == 401
|
||||||
|
raise Error, "Google Drive: #{message} (#{res.code})"
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,112 @@
|
|||||||
|
require "zlib"
|
||||||
|
|
||||||
|
# Off-site copy to Google Drive (AC 120-78B 3.2.7), after the nightly
|
||||||
|
# database backup: the verified database snapshot (gzipped; the newest few
|
||||||
|
# kept) and every uploaded file not yet copied (never deleted off-site,
|
||||||
|
# so a file removed by mistake can still be recovered).
|
||||||
|
#
|
||||||
|
# database/sam-20261108-033000.sqlite3.gz
|
||||||
|
# files/<blob key> (appProperties: filename, checksum)
|
||||||
|
class OffsiteBackup
|
||||||
|
SAMPLE_FILES = 3
|
||||||
|
|
||||||
|
def self.run!(database_path) = new.run!(database_path)
|
||||||
|
def self.restore_test! = new.restore_test!
|
||||||
|
|
||||||
|
def initialize(connection = OffsiteConnection.current)
|
||||||
|
@connection = connection
|
||||||
|
end
|
||||||
|
|
||||||
|
def run!(database_path)
|
||||||
|
raise "Google Drive isn't connected" unless @connection.connected?
|
||||||
|
drive = @connection.client
|
||||||
|
@connection.ensure_folders!(drive)
|
||||||
|
bytes = upload_database(drive, database_path)
|
||||||
|
pruned = prune(drive)
|
||||||
|
uploaded, failed = upload_files(drive)
|
||||||
|
message = "Database copy uploaded (#{ActiveSupport::NumberHelper.number_to_human_size(bytes)}); #{uploaded.size} new #{'file'.pluralize(uploaded.size)} copied"
|
||||||
|
message += "; #{failed.size} couldn't be copied: #{failed.first(3).join('; ')}" if failed.any?
|
||||||
|
OffsiteRun.create!(kind: "backup", ok: failed.empty?, bytes: bytes + uploaded.sum, files_uploaded: uploaded.size, message: message,
|
||||||
|
details: { "account" => @connection.account_email, "pruned" => pruned, "failed" => failed.first(20) })
|
||||||
|
rescue StandardError => e
|
||||||
|
Rails.logger.error("[offsite] #{e.class}: #{e.message}")
|
||||||
|
OffsiteRun.create!(kind: "backup", ok: false, message: "#{e.class.name.demodulize}: #{e.message}".truncate(500))
|
||||||
|
end
|
||||||
|
|
||||||
|
# Downloads the newest database copy from Drive, opens it and checks it,
|
||||||
|
# and downloads a few random files and checks them against their checksums.
|
||||||
|
def restore_test!
|
||||||
|
raise "Google Drive isn't connected" unless @connection.connected?
|
||||||
|
drive = @connection.client
|
||||||
|
latest = drive.list(@connection.database_folder_id).select { _1["name"].end_with?(".sqlite3.gz") }.max_by { _1["name"] } or raise "no database copy in Google Drive yet"
|
||||||
|
db_result = Dir.mktmpdir("sam-restore") { |dir| check_database(drive, latest, dir) }
|
||||||
|
samples = OffsiteFile.includes(:blob).order(Arel.sql("RANDOM()")).limit(SAMPLE_FILES).to_a
|
||||||
|
bad = samples.reject { check_file(drive, _1) }
|
||||||
|
message = "#{latest['name']}: #{db_result}. #{samples.size - bad.size} of #{samples.size} sampled files match their checksums"
|
||||||
|
OffsiteRun.create!(kind: "restore_test", ok: bad.empty?, message: message,
|
||||||
|
details: { "database" => latest["name"], "files_checked" => samples.map { _1.blob.filename.to_s }, "mismatched" => bad.map { _1.blob.filename.to_s } })
|
||||||
|
rescue StandardError => e
|
||||||
|
Rails.logger.error("[offsite restore test] #{e.class}: #{e.message}")
|
||||||
|
OffsiteRun.create!(kind: "restore_test", ok: false, message: "#{e.class.name.demodulize}: #{e.message}".truncate(500))
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def upload_database(drive, path)
|
||||||
|
Tempfile.create([ "sam-offsite", ".gz" ], binmode: true) do |gz|
|
||||||
|
Zlib::GzipWriter.wrap(File.open(gz.path, "wb")) { |w| File.open(path, "rb") { IO.copy_stream(_1, w) } }
|
||||||
|
size = File.size(gz.path)
|
||||||
|
File.open(gz.path, "rb") do |io|
|
||||||
|
drive.upload(io, name: "#{File.basename(path)}.gz", parent: @connection.database_folder_id, size: size, content_type: "application/gzip")
|
||||||
|
end
|
||||||
|
size
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def prune(drive)
|
||||||
|
keep = Setting.integer("offsite_backups_kept")
|
||||||
|
copies = drive.list(@connection.database_folder_id).select { _1["name"].start_with?("sam-") }.sort_by { _1["name"] }.reverse
|
||||||
|
copies.drop(keep).each { drive.delete(_1["id"]) }.map { _1["name"] }
|
||||||
|
end
|
||||||
|
|
||||||
|
# Returns [sizes uploaded, failure messages].
|
||||||
|
def upload_files(drive)
|
||||||
|
uploaded = []
|
||||||
|
failed = []
|
||||||
|
ActiveStorage::Blob.where.not(id: OffsiteFile.select(:blob_id)).find_each do |blob|
|
||||||
|
blob.open do |file|
|
||||||
|
id = drive.upload(file, name: blob.key, parent: @connection.files_folder_id, size: blob.byte_size,
|
||||||
|
content_type: blob.content_type.presence || "application/octet-stream",
|
||||||
|
properties: { filename: blob.filename.to_s.first(100), checksum: blob.checksum.to_s })
|
||||||
|
OffsiteFile.create!(blob: blob, drive_file_id: id, bytes: blob.byte_size)
|
||||||
|
uploaded << blob.byte_size
|
||||||
|
end
|
||||||
|
rescue GoogleDriveClient::AuthError
|
||||||
|
raise
|
||||||
|
rescue StandardError => e
|
||||||
|
failed << "#{blob.filename}: #{e.message.truncate(120)}"
|
||||||
|
end
|
||||||
|
[ uploaded, failed ]
|
||||||
|
end
|
||||||
|
|
||||||
|
def check_database(drive, file, dir)
|
||||||
|
gz = File.join(dir, "db.sqlite3.gz")
|
||||||
|
File.open(gz, "wb") { drive.download(file["id"], to: _1) }
|
||||||
|
path = File.join(dir, "db.sqlite3")
|
||||||
|
Zlib::GzipReader.open(gz) { |r| File.open(path, "wb") { IO.copy_stream(r, _1) } }
|
||||||
|
db = SQLite3::Database.new(path, readonly: true)
|
||||||
|
result = db.get_first_value("PRAGMA integrity_check")
|
||||||
|
raise "integrity check of the downloaded copy failed: #{result}" unless result == "ok"
|
||||||
|
entries, signatures = %w[logbook_entries entry_signatures].map { db.get_first_value("SELECT COUNT(*) FROM #{_1}") }
|
||||||
|
"opened, SQLite integrity check ok, #{entries} logbook entries and #{signatures} signatures (#{LogbookEntry.count} and #{EntrySignature.count} now)"
|
||||||
|
ensure
|
||||||
|
db&.close
|
||||||
|
end
|
||||||
|
|
||||||
|
def check_file(drive, offsite)
|
||||||
|
io = drive.download(offsite.drive_file_id, to: StringIO.new(+"", "wb"))
|
||||||
|
Digest::MD5.base64digest(io.string) == offsite.blob.checksum
|
||||||
|
rescue GoogleDriveClient::Error
|
||||||
|
false
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -28,6 +28,7 @@ class RecordsAudit
|
|||||||
Item.new("backup", "A verified database backup was made in the last 2 days", true,
|
Item.new("backup", "A verified database backup was made in the last 2 days", true,
|
||||||
backup.present? && backup.ok && backup.created_at > 2.days.ago,
|
backup.present? && backup.ok && backup.created_at > 2.days.ago,
|
||||||
backup ? "#{backup.created_at.to_date}: #{backup.message}" : "Never run"),
|
backup ? "#{backup.created_at.to_date}: #{backup.message}" : "Never run"),
|
||||||
|
*offsite_items,
|
||||||
Item.new("procedures", "The procedures document is adopted and its details filled in", true,
|
Item.new("procedures", "The procedures document is adopted and its details filled in", true,
|
||||||
revision.present? && @policy.complete?,
|
revision.present? && @policy.complete?,
|
||||||
revision ? "Revision #{revision.number}, adopted #{revision.adopted_at.to_date}" : "Not adopted yet"),
|
revision ? "Revision #{revision.number}, adopted #{revision.adopted_at.to_date}" : "Not adopted yet"),
|
||||||
@@ -62,8 +63,18 @@ class RecordsAudit
|
|||||||
AuditRun.create!(performed_by: user, performed_at: Time.current, passed: results.all? { _1["passed"] }, results: results, notes: notes.to_s.strip.first(5000))
|
AuditRun.create!(performed_by: user, performed_at: Time.current, passed: results.all? { _1["passed"] }, results: results, notes: notes.to_s.strip.first(5000))
|
||||||
end
|
end
|
||||||
|
|
||||||
|
# The latest automatic restore test (shown next to the manual restore item).
|
||||||
|
def restore_test = OffsiteConnection.connected? ? OffsiteRun.latest_restore_test : nil
|
||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
|
def offsite_items
|
||||||
|
return [] unless OffsiteConnection.connected?
|
||||||
|
run = OffsiteRun.latest_backup
|
||||||
|
[ Item.new("offsite", "The backup was copied off-site (Google Drive) in the last 2 days", true, run.present? && run.ok && run.recent?,
|
||||||
|
run ? "#{run.created_at.to_date}: #{run.message}" : "Never run") ]
|
||||||
|
end
|
||||||
|
|
||||||
def since_label = @since ? @since.to_date.to_s : "the start"
|
def since_label = @since ? @since.to_date.to_s : "the start"
|
||||||
def signatures_since = EntrySignature.where(signed_at: (@since || Time.at(0))..)
|
def signatures_since = EntrySignature.where(signed_at: (@since || Time.at(0))..)
|
||||||
def versions_since = AppRevision.where(first_seen_at: (@since || Time.at(0))..).order(:first_seen_at)
|
def versions_since = AppRevision.where(first_seen_at: (@since || Time.at(0))..).order(:first_seen_at)
|
||||||
|
|||||||
@@ -94,6 +94,7 @@
|
|||||||
[ "Last backup", backup ? "#{l(backup.created_at, format: :short)} · #{backup.ok ? number_to_human_size(backup.bytes) : 'FAILED'}" : "not run yet" ],
|
[ "Last backup", backup ? "#{l(backup.created_at, format: :short)} · #{backup.ok ? number_to_human_size(backup.bytes) : 'FAILED'}" : "not run yet" ],
|
||||||
[ "Backup check", backup&.message ],
|
[ "Backup check", backup&.message ],
|
||||||
[ "Backups kept", "#{Setting.integer('backups_kept')} (in storage/backups)" ],
|
[ "Backups kept", "#{Setting.integer('backups_kept')} (in storage/backups)" ],
|
||||||
|
[ "Off-site backup", OffsiteConnection.connected? ? ((o = OffsiteRun.latest_backup) ? tag.span("#{o.ok ? 'OK' : 'FAILED'} #{o.created_at.to_date}", class: (o.ok && o.recent? ? nil : "badge-orange")) + " · Google Drive (#{OffsiteConnection.current.account_email})" : "Google Drive connected, none yet") : link_to("not set up", admin_offsite_backup_path, class: "badge-orange") ],
|
||||||
[ "Procedures", (r = ProceduresRevision.current) ? "Revision #{r.number}, adopted #{r.adopted_at.to_date}" : tag.span("not adopted", class: "badge-orange") ],
|
[ "Procedures", (r = ProceduresRevision.current) ? "Revision #{r.number}, adopted #{r.adopted_at.to_date}" : tag.span("not adopted", class: "badge-orange") ],
|
||||||
[ "Last audit", AuditRun.latest ? "#{AuditRun.latest.performed_at.to_date} · #{AuditRun.latest.passed ? 'passed' : 'items not passed'}" : "none yet" ],
|
[ "Last audit", AuditRun.latest ? "#{AuditRun.latest.performed_at.to_date} · #{AuditRun.latest.passed ? 'passed' : 'items not passed'}" : "none yet" ],
|
||||||
[ "Next audit due", AuditRun.due_soon? ? tag.span(AuditRun.next_due.to_s, class: "badge-orange") : AuditRun.next_due.to_s ]
|
[ "Next audit due", AuditRun.due_soon? ? tag.span(AuditRun.next_due.to_s, class: "badge-orange") : AuditRun.next_due.to_s ]
|
||||||
|
|||||||
@@ -19,6 +19,12 @@
|
|||||||
<% RecordsAudit::MANUAL.each do |key, title| %>
|
<% RecordsAudit::MANUAL.each do |key, title| %>
|
||||||
<div class="space-y-2 rounded-md p-3 ring-1 ring-slate-200">
|
<div class="space-y-2 rounded-md p-3 ring-1 ring-slate-200">
|
||||||
<label class="flex items-start gap-2 text-sm font-medium"><%= check_box_tag "answers[#{key}][passed]", "1", false, class: "mt-0.5 h-4 w-4 shrink-0 rounded border-slate-300" %><span><%= title %></span></label>
|
<label class="flex items-start gap-2 text-sm font-medium"><%= check_box_tag "answers[#{key}][passed]", "1", false, class: "mt-0.5 h-4 w-4 shrink-0 rounded border-slate-300" %><span><%= title %></span></label>
|
||||||
|
<% if key == "restore_test" && OffsiteConnection.connected? %>
|
||||||
|
<p class="ml-6 text-xs text-slate-600">
|
||||||
|
<% if (rt = @audit.restore_test) %>Last automatic restore test (<%= rt.created_at.to_date %>): <span class="<%= rt.ok ? 'badge-green' : 'badge-red' %>"><%= rt.ok ? "OK" : "Failed" %></span> <%= rt.message %><% else %>No automatic restore test yet.<% end %>
|
||||||
|
<%= link_to "Run one", admin_offsite_backup_path, class: "link", target: "_blank" %> (Admin → Off-site Backup → Test restore), then also open a copy downloaded from Google Drive yourself.
|
||||||
|
</p>
|
||||||
|
<% end %>
|
||||||
<% if key == "sample" %>
|
<% if key == "sample" %>
|
||||||
<ul class="ml-6 list-disc text-xs text-slate-600">
|
<ul class="ml-6 list-disc text-xs text-slate-600">
|
||||||
<% sample.each do |e| %><li><%= hidden_field_tag "sample_ids[]", e.id, id: nil %><%= link_to "#{e.aircraft.tail_number} · #{e.date} · #{e.text.truncate(80)}", aircraft_logbook_entry_path(e.aircraft, e), class: "link", target: "_blank" %><%= " (signed)" if e.signature %></li><% end %>
|
<% sample.each do |e| %><li><%= hidden_field_tag "sample_ids[]", e.id, id: nil %><%= link_to "#{e.aircraft.tail_number} · #{e.date} · #{e.text.truncate(80)}", aircraft_logbook_entry_path(e.aircraft, e), class: "link", target: "_blank" %><%= " (signed)" if e.signature %></li><% end %>
|
||||||
|
|||||||
@@ -0,0 +1,59 @@
|
|||||||
|
<% content_for :title, "Off-site backup" %>
|
||||||
|
<h1 class="mb-1 text-2xl font-bold">Off-site backup</h1>
|
||||||
|
<p class="mb-5 max-w-3xl text-sm text-slate-600">Every night, after the verified database backup, the app copies it to Google Drive with every uploaded file not already there, so the records survive losing this server (AC 120-78B 3.2.7). The newest <%= Setting.integer("offsite_backups_kept") %> database copies are kept (<%= link_to "Settings", edit_admin_settings_path, class: "link" %>); uploaded files are copied once and never deleted there. The app can only see the files it created in your Drive.</p>
|
||||||
|
|
||||||
|
<div class="grid gap-5 lg:grid-cols-2">
|
||||||
|
<section class="card card-body space-y-3">
|
||||||
|
<h2 class="card-title">Google Drive</h2>
|
||||||
|
<% if @connection.connected? %>
|
||||||
|
<p class="text-sm"><span class="badge-green">Connected</span> <strong><%= @connection.account_email %></strong>, by <%= @connection.connected_by&.display_name %> on <%= @connection.connected_at&.to_date %>.
|
||||||
|
Folder: <em><%= OffsiteConnection::ROOT_FOLDER %></em>.</p>
|
||||||
|
<p class="text-sm">Uploaded files copied: <%= @files_copied %> of <%= @files_total %>.</p>
|
||||||
|
<div class="flex flex-wrap gap-2">
|
||||||
|
<%= button_to "Back up now", backup_now_admin_offsite_backup_path, class: "btn-primary" %>
|
||||||
|
<%= button_to "Test restore", restore_test_admin_offsite_backup_path, class: "btn-secondary",
|
||||||
|
title: "Download the newest database copy and a few files from Drive and check them" %>
|
||||||
|
<%= button_to "Disconnect", admin_offsite_backup_path, method: :delete, class: "btn-secondary",
|
||||||
|
form: { data: { turbo_confirm: "Stop copying backups to #{@connection.account_email}? The copies already there stay in that Drive." } } %>
|
||||||
|
</div>
|
||||||
|
<% elsif @connection.configured? %>
|
||||||
|
<p class="text-sm">The OAuth client is saved. Connect the Google account whose Drive will hold the backups.</p>
|
||||||
|
<%= link_to "Connect Google Drive", connect_admin_offsite_backup_path, class: "btn-primary self-start", data: { turbo: false } %>
|
||||||
|
<% else %>
|
||||||
|
<p class="text-sm text-amber-800">Not set up. Follow the steps to create a Google OAuth client, then connect.</p>
|
||||||
|
<% end %>
|
||||||
|
|
||||||
|
<%= form_with model: @connection, url: admin_offsite_backup_path, method: :patch, class: "space-y-3 border-t border-slate-100 pt-3" do |f| %>
|
||||||
|
<%= f.field :client_id, label: "OAuth client ID", placeholder: "1234567890-abc.apps.googleusercontent.com" %>
|
||||||
|
<%= f.field :client_secret, as: :password, label: "OAuth client secret", value: nil,
|
||||||
|
placeholder: (@connection.client_secret.present? ? "saved (leave blank to keep)" : nil), autocomplete: "off" %>
|
||||||
|
<%= f.submit "Save client", class: "btn-secondary" %>
|
||||||
|
<% end %>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="card card-body space-y-2 text-sm">
|
||||||
|
<h2 class="card-title">Setting up Google (once)</h2>
|
||||||
|
<ol class="list-decimal space-y-1.5 pl-5">
|
||||||
|
<li>Open the <a class="link" href="https://console.cloud.google.com/projectcreate" target="_blank" rel="noopener">Google Cloud console</a> signed in as the account that will hold the backups, and create a project (e.g. “Aircraft records backup”).</li>
|
||||||
|
<li><a class="link" href="https://console.cloud.google.com/apis/library/drive.googleapis.com" target="_blank" rel="noopener">Enable the Google Drive API</a> for it.</li>
|
||||||
|
<li>Under <strong>Google Auth Platform → Branding</strong>, give the app a name and your email; under <strong>Audience</strong> choose <strong>External</strong> and then <strong>Publish app</strong> (in “Testing”, Google expires the access after 7 days). Google doesn't need to verify it: the app only asks for access to files it creates.</li>
|
||||||
|
<li>Under <strong>Clients</strong>, create an <strong>OAuth client ID</strong> of type <strong>Web application</strong> with this authorized redirect URI:
|
||||||
|
<code class="mt-1 block break-all rounded bg-slate-100 px-2 py-1"><%= @connection.redirect_uri %></code>
|
||||||
|
(It must be the address you open the app at; set <code>APP_URL</code> if it isn't.)</li>
|
||||||
|
<li>Paste the client ID and secret here, save, and choose <strong>Connect Google Drive</strong>. Google will warn that the app isn't verified: choose <strong>Advanced → Go to …</strong> and allow access.</li>
|
||||||
|
</ol>
|
||||||
|
<p class="text-slate-600">Restoring: see <%= link_to "Help → Administration", help_page_path("administration", anchor: "restoring-from-google-drive"), class: "link" %>.</p>
|
||||||
|
</section>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<section class="card mt-5">
|
||||||
|
<div class="card-header"><h2 class="card-title">Recent runs</h2></div>
|
||||||
|
<% if @runs.any? %>
|
||||||
|
<ul class="divide-y divide-slate-100 text-sm">
|
||||||
|
<% @runs.each do |r| %>
|
||||||
|
<li class="flex items-start gap-3 px-4 py-2"><span class="<%= r.ok ? 'badge-green' : 'badge-red' %>"><%= r.ok ? "OK" : "Failed" %></span>
|
||||||
|
<span><span class="font-medium"><%= r.kind == "restore_test" ? "Restore test" : "Backup" %></span> · <%= l(r.created_at.in_time_zone, format: :short) %><span class="block text-xs text-slate-600"><%= r.message %></span></span></li>
|
||||||
|
<% end %>
|
||||||
|
</ul>
|
||||||
|
<% else %><%= empty_state "None yet." %><% end %>
|
||||||
|
</section>
|
||||||
@@ -34,7 +34,7 @@
|
|||||||
<li><strong>Software:</strong> Simple Aircraft Manager, version <%= info.revision %> (Ruby <%= RUBY_VERSION %>, Rails <%= Rails.version %>, SQLite <%= SQLite3::SQLITE_VERSION %>).</li>
|
<li><strong>Software:</strong> Simple Aircraft Manager, version <%= info.revision %> (Ruby <%= RUBY_VERSION %>, Rails <%= Rails.version %>, SQLite <%= SQLite3::SQLITE_VERSION %>).</li>
|
||||||
<li><strong>Hosting and network:</strong> <%= policy.hosting_description.presence || "[describe the server, location and network]" %> Access is over HTTPS<%= " with single sign-on (#{Sso.label})" if Sso.enabled? %>.</li>
|
<li><strong>Hosting and network:</strong> <%= policy.hosting_description.presence || "[describe the server, location and network]" %> Access is over HTTPS<%= " with single sign-on (#{Sso.label})" if Sso.enabled? %>.</li>
|
||||||
<li><strong>Data:</strong> one SQLite database plus uploaded files (stored once per identical file) on the server's storage volume.</li>
|
<li><strong>Data:</strong> one SQLite database plus uploaded files (stored once per identical file) on the server's storage volume.</li>
|
||||||
<li><strong>Backups:</strong> every night the app makes a verified copy of the database in the storage volume (SQLite integrity check and record counts compared) and keeps the newest <%= Setting.integer("backups_kept") %>. Off-site: <%= policy.offsite_backup_description.presence || "[describe the off-site backup]" %></li>
|
<li><strong>Backups:</strong> every night the app makes a verified copy of the database in the storage volume (SQLite integrity check and record counts compared) and keeps the newest <%= Setting.integer("backups_kept") %>. Off-site: <% if OffsiteConnection.connected? %>after each nightly backup the app copies the verified database snapshot (compressed) to Google Drive (account <%= OffsiteConnection.current.account_email %>, folder “<%= OffsiteConnection::ROOT_FOLDER %>”), keeping the newest <%= Setting.integer("offsite_backups_kept") %>, together with every uploaded file (scans, documents, attachments) not already copied; files are never deleted from the off-site copy. A failed copy is emailed to the administrators. A restore test downloads the newest copy, opens it and checks it, and checks a sample of files against their checksums.<%= " #{policy.offsite_backup_description}" if policy.offsite_backup_description.present? %><% else %><%= policy.offsite_backup_description.presence || "[describe the off-site backup]" %><% end %></li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|
||||||
<h2 class="<%= h2 %>">4. Responsible personnel</h2>
|
<h2 class="<%= h2 %>">4. Responsible personnel</h2>
|
||||||
|
|||||||
@@ -13,7 +13,8 @@
|
|||||||
</div>
|
</div>
|
||||||
<%= f.field :hosting_description, as: :textarea, rows: 2, label: "Hosting and network",
|
<%= f.field :hosting_description, as: :textarea, rows: 2, label: "Hosting and network",
|
||||||
placeholder: "Self-hosted on a server at the owner's home, behind a firewall and an authenticating proxy." %>
|
placeholder: "Self-hosted on a server at the owner's home, behind a firewall and an authenticating proxy." %>
|
||||||
<%= f.field :offsite_backup_description, as: :textarea, rows: 2, label: "Off-site backup",
|
<%= f.field :offsite_backup_description, as: :textarea, rows: 2, label: (OffsiteConnection.connected? ? "Off-site backup (optional: Google Drive is connected)" : "Off-site backup"),
|
||||||
|
hint: (OffsiteConnection.connected? ? "The Google Drive backup is described automatically; add anything else here." : "Or set up the built-in Google Drive backup under Admin → Off-site Backup."),
|
||||||
placeholder: "The storage volume is backed up nightly with restic to an encrypted off-site repository; retention 30 days." %>
|
placeholder: "The storage volume is backed up nightly with restic to an encrypted off-site repository; retention 30 days." %>
|
||||||
<div class="grid gap-4 sm:grid-cols-2">
|
<div class="grid gap-4 sm:grid-cols-2">
|
||||||
<%= f.field :audit_interval_months, as: :number, min: 1, max: 24, label: "Audit every (months)" %>
|
<%= f.field :audit_interval_months, as: :number, min: 1, max: 24, label: "Audit every (months)" %>
|
||||||
@@ -36,7 +37,7 @@
|
|||||||
<%= form_with url: adopt_admin_procedures_path, class: "flex flex-wrap items-end gap-2" do %>
|
<%= form_with url: adopt_admin_procedures_path, class: "flex flex-wrap items-end gap-2" do %>
|
||||||
<div class="min-w-0 flex-1"><%= label_tag :summary, "What changed", class: "label" %><%= text_field_tag :summary, nil, class: "input", placeholder: current ? "e.g. new mechanic added" : "Initial issue" %></div>
|
<div class="min-w-0 flex-1"><%= label_tag :summary, "What changed", class: "label" %><%= text_field_tag :summary, nil, class: "input", placeholder: current ? "e.g. new mechanic added" : "Initial issue" %></div>
|
||||||
<%= submit_tag "Adopt as revision #{(current&.number || 0) + 1}", class: "btn-primary", disabled: !@policy.complete?,
|
<%= submit_tag "Adopt as revision #{(current&.number || 0) + 1}", class: "btn-primary", disabled: !@policy.complete?,
|
||||||
title: (@policy.complete? ? nil : "Fill in the operator, responsible person, hosting and off-site backup first") %>
|
title: (@policy.complete? ? nil : "Fill in the operator, responsible person, hosting and off-site backup (or connect Google Drive) first") %>
|
||||||
<% end %>
|
<% end %>
|
||||||
</section>
|
</section>
|
||||||
<section class="card">
|
<section class="card">
|
||||||
|
|||||||
@@ -28,7 +28,14 @@
|
|||||||
<%= label_tag "settings[backups_kept]", kept[:label], class: "label" %>
|
<%= label_tag "settings[backups_kept]", kept[:label], class: "label" %>
|
||||||
<%= number_field_tag "settings[backups_kept]", @values["backups_kept"], id: "settings_backups_kept", min: kept[:in].min, max: kept[:in].max,
|
<%= number_field_tag "settings[backups_kept]", @values["backups_kept"], id: "settings_backups_kept", min: kept[:in].min, max: kept[:in].max,
|
||||||
step: 1, required: true, class: "input w-32" %>
|
step: 1, required: true, class: "input w-32" %>
|
||||||
<p class="hint">A verified copy of the database is made nightly in storage/backups; this many of the newest are kept. Back up the whole storage volume off the server too.</p>
|
<p class="hint">A verified copy of the database is made nightly in storage/backups; this many of the newest are kept. Back up off the server too: <%= link_to "Off-site Backup", admin_offsite_backup_path, class: "link" %>.</p>
|
||||||
|
</div>
|
||||||
|
<% okept = Setting::INTEGERS["offsite_backups_kept"] %>
|
||||||
|
<div>
|
||||||
|
<%= label_tag "settings[offsite_backups_kept]", okept[:label], class: "label" %>
|
||||||
|
<%= number_field_tag "settings[offsite_backups_kept]", @values["offsite_backups_kept"], id: "settings_offsite_backups_kept", min: okept[:in].min, max: okept[:in].max,
|
||||||
|
step: 1, required: true, class: "input w-32" %>
|
||||||
|
<p class="hint">With off-site backup to Google Drive, this many of the newest nightly database copies are kept there. Uploaded files are copied once and kept.</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="flex gap-2"><%= f.submit "Save Settings", class: "btn-primary" %><%= link_to "Cancel", admin_about_path, class: "btn-secondary" %></div>
|
<div class="flex gap-2"><%= f.submit "Save Settings", class: "btn-primary" %><%= link_to "Cancel", admin_about_path, class: "btn-secondary" %></div>
|
||||||
|
|||||||
@@ -0,0 +1,3 @@
|
|||||||
|
<p><%= @run.kind == "restore_test" ? "The off-site restore test failed." : "Copying the backup to Google Drive failed." %></p>
|
||||||
|
<p><%= @run.message %></p>
|
||||||
|
<p>Check <%= link_to "Admin → Off-site Backup", admin_offsite_backup_url %>. If Google Drive refused access, connect it again there.</p>
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
<%= @run.kind == "restore_test" ? "The off-site restore test failed." : "Copying the backup to Google Drive failed." %>
|
||||||
|
|
||||||
|
<%= @run.message %>
|
||||||
|
|
||||||
|
Check Admin → Off-site Backup: <%= admin_offsite_backup_url %>
|
||||||
|
If Google Drive refused access, connect it again there.
|
||||||
@@ -38,7 +38,7 @@
|
|||||||
<%= link_to "Help", help_path, class: "block px-4 py-2.5 hover:bg-slate-100" %>
|
<%= link_to "Help", help_path, class: "block px-4 py-2.5 hover:bg-slate-100" %>
|
||||||
<% if current_user.admin? %>
|
<% if current_user.admin? %>
|
||||||
<p class="border-t border-slate-100 px-4 pt-2 pb-1 text-xs font-semibold text-slate-500">Admin</p>
|
<p class="border-t border-slate-100 px-4 pt-2 pb-1 text-xs font-semibold text-slate-500">Admin</p>
|
||||||
<% [ [ "Users", admin_users_path ], [ "Component Types", admin_component_types_path ], [ "Airports", admin_airports_path ], [ "Logbook Templates", admin_logbook_templates_path ], [ "Settings", edit_admin_settings_path ], [ "Recordkeeping Procedures", admin_procedures_path ], [ "Records Audits", admin_audits_path ], [ "About", admin_about_path ] ].each do |label, path| %>
|
<% [ [ "Users", admin_users_path ], [ "Component Types", admin_component_types_path ], [ "Airports", admin_airports_path ], [ "Logbook Templates", admin_logbook_templates_path ], [ "Settings", edit_admin_settings_path ], [ "Off-site Backup", admin_offsite_backup_path ], [ "Recordkeeping Procedures", admin_procedures_path ], [ "Records Audits", admin_audits_path ], [ "About", admin_about_path ] ].each do |label, path| %>
|
||||||
<%= link_to label, path, class: "block px-4 py-2.5 hover:bg-slate-100" %>
|
<%= link_to label, path, class: "block px-4 py-2.5 hover:bg-slate-100" %>
|
||||||
<% end %>
|
<% end %>
|
||||||
<% end %>
|
<% end %>
|
||||||
@@ -54,7 +54,7 @@
|
|||||||
Admin <span aria-hidden="true" class="text-xs">▾</span>
|
Admin <span aria-hidden="true" class="text-xs">▾</span>
|
||||||
</summary>
|
</summary>
|
||||||
<div class="absolute right-0 z-30 mt-2 w-48 rounded-md bg-surface py-1 text-slate-700 shadow-lg ring-1 ring-slate-200">
|
<div class="absolute right-0 z-30 mt-2 w-48 rounded-md bg-surface py-1 text-slate-700 shadow-lg ring-1 ring-slate-200">
|
||||||
<% [ [ "Users", admin_users_path ], [ "Component Types", admin_component_types_path ], [ "Airports", admin_airports_path ], [ "Logbook Templates", admin_logbook_templates_path ], [ "Settings", edit_admin_settings_path ], [ "Recordkeeping Procedures", admin_procedures_path ], [ "Records Audits", admin_audits_path ], [ "About", admin_about_path ] ].each do |label, path| %>
|
<% [ [ "Users", admin_users_path ], [ "Component Types", admin_component_types_path ], [ "Airports", admin_airports_path ], [ "Logbook Templates", admin_logbook_templates_path ], [ "Settings", edit_admin_settings_path ], [ "Off-site Backup", admin_offsite_backup_path ], [ "Recordkeeping Procedures", admin_procedures_path ], [ "Records Audits", admin_audits_path ], [ "About", admin_about_path ] ].each do |label, path| %>
|
||||||
<%= link_to label, path, class: [ "block px-4 py-2 text-sm hover:bg-slate-100", ("font-semibold text-sky-700" if current_page?(path)) ] %>
|
<%= link_to label, path, class: [ "block px-4 py-2 text-sm hover:bg-slate-100", ("font-semibold text-sky-700" if current_page?(path)) ] %>
|
||||||
<% end %>
|
<% end %>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -136,6 +136,12 @@ Rails.application.routes.draw do
|
|||||||
get "revisions/:id", action: :revision, as: :revision
|
get "revisions/:id", action: :revision, as: :revision
|
||||||
end
|
end
|
||||||
resources :audits, only: %i[index new create show]
|
resources :audits, only: %i[index new create show]
|
||||||
|
resource :offsite_backup, only: %i[show update destroy] do
|
||||||
|
get :connect
|
||||||
|
get :callback
|
||||||
|
post :backup_now
|
||||||
|
post :restore_test
|
||||||
|
end
|
||||||
resources :logbook_templates, except: :show do
|
resources :logbook_templates, except: :show do
|
||||||
collection { post :restore }
|
collection { post :restore }
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,37 @@
|
|||||||
|
class CreateOffsiteBackups < ActiveRecord::Migration[8.1]
|
||||||
|
def change
|
||||||
|
# One row: the Google Drive account the off-site copies go to.
|
||||||
|
create_table :offsite_connections do |t|
|
||||||
|
t.string :provider, null: false, default: "google_drive"
|
||||||
|
t.string :client_id
|
||||||
|
t.text :client_secret
|
||||||
|
t.text :refresh_token
|
||||||
|
t.string :account_email
|
||||||
|
t.string :folder_id
|
||||||
|
t.string :database_folder_id
|
||||||
|
t.string :files_folder_id
|
||||||
|
t.references :connected_by, foreign_key: { to_table: :users }
|
||||||
|
t.datetime :connected_at
|
||||||
|
t.timestamps
|
||||||
|
end
|
||||||
|
|
||||||
|
create_table :offsite_runs do |t|
|
||||||
|
t.string :kind, null: false # backup | restore_test
|
||||||
|
t.boolean :ok, null: false
|
||||||
|
t.text :message
|
||||||
|
t.bigint :bytes
|
||||||
|
t.integer :files_uploaded, null: false, default: 0
|
||||||
|
t.json :details
|
||||||
|
t.timestamps
|
||||||
|
end
|
||||||
|
add_index :offsite_runs, [ :kind, :created_at ]
|
||||||
|
|
||||||
|
# Uploaded files already copied off-site (by blob), so each goes once.
|
||||||
|
create_table :offsite_files do |t|
|
||||||
|
t.references :blob, null: false, index: { unique: true }
|
||||||
|
t.string :drive_file_id, null: false
|
||||||
|
t.bigint :bytes
|
||||||
|
t.timestamps
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
Generated
+39
-1
@@ -10,7 +10,7 @@
|
|||||||
#
|
#
|
||||||
# It's strongly recommended that you check this file into your version control system.
|
# It's strongly recommended that you check this file into your version control system.
|
||||||
|
|
||||||
ActiveRecord::Schema[8.1].define(version: 2026_11_08_030000) do
|
ActiveRecord::Schema[8.1].define(version: 2026_11_08_040000) do
|
||||||
create_table "active_storage_attachments", force: :cascade do |t|
|
create_table "active_storage_attachments", force: :cascade do |t|
|
||||||
t.string "name", null: false
|
t.string "name", null: false
|
||||||
t.string "record_type", null: false
|
t.string "record_type", null: false
|
||||||
@@ -597,6 +597,43 @@ ActiveRecord::Schema[8.1].define(version: 2026_11_08_030000) do
|
|||||||
t.index ["stc_document_id"], name: "index_major_records_on_stc_document_id"
|
t.index ["stc_document_id"], name: "index_major_records_on_stc_document_id"
|
||||||
end
|
end
|
||||||
|
|
||||||
|
create_table "offsite_connections", force: :cascade do |t|
|
||||||
|
t.string "provider", default: "google_drive", null: false
|
||||||
|
t.string "client_id"
|
||||||
|
t.text "client_secret"
|
||||||
|
t.text "refresh_token"
|
||||||
|
t.string "account_email"
|
||||||
|
t.string "folder_id"
|
||||||
|
t.string "database_folder_id"
|
||||||
|
t.string "files_folder_id"
|
||||||
|
t.integer "connected_by_id"
|
||||||
|
t.datetime "connected_at"
|
||||||
|
t.datetime "created_at", null: false
|
||||||
|
t.datetime "updated_at", null: false
|
||||||
|
t.index ["connected_by_id"], name: "index_offsite_connections_on_connected_by_id"
|
||||||
|
end
|
||||||
|
|
||||||
|
create_table "offsite_files", force: :cascade do |t|
|
||||||
|
t.integer "blob_id", null: false
|
||||||
|
t.string "drive_file_id", null: false
|
||||||
|
t.bigint "bytes"
|
||||||
|
t.datetime "created_at", null: false
|
||||||
|
t.datetime "updated_at", null: false
|
||||||
|
t.index ["blob_id"], name: "index_offsite_files_on_blob_id", unique: true
|
||||||
|
end
|
||||||
|
|
||||||
|
create_table "offsite_runs", force: :cascade do |t|
|
||||||
|
t.string "kind", null: false
|
||||||
|
t.boolean "ok", null: false
|
||||||
|
t.text "message"
|
||||||
|
t.bigint "bytes"
|
||||||
|
t.integer "files_uploaded", default: 0, null: false
|
||||||
|
t.json "details"
|
||||||
|
t.datetime "created_at", null: false
|
||||||
|
t.datetime "updated_at", null: false
|
||||||
|
t.index ["kind", "created_at"], name: "index_offsite_runs_on_kind_and_created_at"
|
||||||
|
end
|
||||||
|
|
||||||
create_table "oil_analyses", force: :cascade do |t|
|
create_table "oil_analyses", force: :cascade do |t|
|
||||||
t.integer "aircraft_id", null: false
|
t.integer "aircraft_id", null: false
|
||||||
t.integer "component_id"
|
t.integer "component_id"
|
||||||
@@ -832,6 +869,7 @@ ActiveRecord::Schema[8.1].define(version: 2026_11_08_030000) do
|
|||||||
add_foreign_key "major_records", "documents", column: "stc_document_id", on_delete: :nullify
|
add_foreign_key "major_records", "documents", column: "stc_document_id", on_delete: :nullify
|
||||||
add_foreign_key "major_records", "inspection_types", column: "ica_inspection_type_id", on_delete: :nullify
|
add_foreign_key "major_records", "inspection_types", column: "ica_inspection_type_id", on_delete: :nullify
|
||||||
add_foreign_key "major_records", "logbook_entries", on_delete: :nullify
|
add_foreign_key "major_records", "logbook_entries", on_delete: :nullify
|
||||||
|
add_foreign_key "offsite_connections", "users", column: "connected_by_id"
|
||||||
add_foreign_key "oil_analyses", "aircraft"
|
add_foreign_key "oil_analyses", "aircraft"
|
||||||
add_foreign_key "oil_analyses", "components"
|
add_foreign_key "oil_analyses", "components"
|
||||||
add_foreign_key "procedures_revisions", "users", column: "adopted_by_id", on_delete: :nullify
|
add_foreign_key "procedures_revisions", "users", column: "adopted_by_id", on_delete: :nullify
|
||||||
|
|||||||
@@ -26,6 +26,6 @@ The same pages are in the app under **Help** in the menu.
|
|||||||
|
|
||||||
## Administrators
|
## Administrators
|
||||||
|
|
||||||
- [Administration](administration.md): users, settings, templates, backups, recordkeeping procedures and audits
|
- [Administration](administration.md): users, settings, templates, backups and off-site backup to Google Drive, recordkeeping procedures and audits
|
||||||
|
|
||||||
Installing and hosting the app is covered in the project's main README.
|
Installing and hosting the app is covered in the project's main README.
|
||||||
@@ -21,14 +21,14 @@ With single sign-on, accounts can be created automatically at first sign-in, and
|
|||||||
## Settings
|
## Settings
|
||||||
|
|
||||||
- **Uploads**: the largest file anyone can upload. A proxy in front of the app may have its own smaller limit.
|
- **Uploads**: the largest file anyone can upload. A proxy in front of the app may have its own smaller limit.
|
||||||
- **Backups**: how many nightly database copies to keep.
|
- **Backups**: how many nightly database copies to keep, on the server and in Google Drive.
|
||||||
|
|
||||||
## About
|
## About
|
||||||
|
|
||||||
**About** shows the running version, features and storage, and has tools:
|
**About** shows the running version, features and storage, and has tools:
|
||||||
|
|
||||||
- **Send test email** and **Send me this week's reminder** check that email works.
|
- **Send test email** and **Send me this week's reminder** check that email works.
|
||||||
- **Records integrity**: the last nightly backup and records check. **Back up and check now** runs both straight away.
|
- **Records integrity**: the last nightly backup, off-site copy and records check. **Back up and check now** runs the backup and check straight away.
|
||||||
- **Records**: the current recordkeeping procedures revision, the last audit and when the next is due.
|
- **Records**: the current recordkeeping procedures revision, the last audit and when the next is due.
|
||||||
- **Software versions**: when each version of the app was first run.
|
- **Software versions**: when each version of the app was first run.
|
||||||
|
|
||||||
@@ -36,13 +36,47 @@ With single sign-on, accounts can be created automatically at first sign-in, and
|
|||||||
|
|
||||||
Every night at 3:30 the app makes a verified copy of the database in `storage/backups`, then checks the record history and every signature. Admins are emailed if anything fails.
|
Every night at 3:30 the app makes a verified copy of the database in `storage/backups`, then checks the record history and every signature. Admins are emailed if anything fails.
|
||||||
|
|
||||||
Those copies sit on the same server. **Back up the whole storage volume somewhere else too** (another machine or a cloud service), and test restoring it now and then. That's part of every records audit.
|
Those copies sit on the same server, so the records also need a copy somewhere else. The app can make that copy to Google Drive itself (below). Otherwise back up the whole storage volume to another machine or a cloud service, and test restoring it now and then. A restore test is part of every records audit.
|
||||||
|
|
||||||
|
## Off-site backup to Google Drive
|
||||||
|
|
||||||
|
**Admin → Off-site Backup** connects a Google account. Every night after the database backup, the app copies to that account's Drive:
|
||||||
|
|
||||||
|
- the verified database copy, compressed (`database/sam-….sqlite3.gz`), keeping the newest 30 (change it in **Settings**);
|
||||||
|
- every uploaded file (scans, documents, photos, track logs) not already copied (`files/`). Files are never deleted there, so one deleted here by mistake can still be recovered.
|
||||||
|
|
||||||
|
The app can only see the files it created in your Drive. Admins are emailed if a copy fails, and the About page shows the last one.
|
||||||
|
|
||||||
|
Google Drive is the only cloud service built in. iCloud Drive can't be used: Apple offers no way for a server to upload to it. To use iCloud anyway, a Mac signed in to iCloud can copy `storage/backups` and the uploaded files into iCloud Drive on a schedule; describe that in the procedures' off-site backup field, since the app can't check it.
|
||||||
|
|
||||||
|
### Setting it up (once)
|
||||||
|
|
||||||
|
You create your own Google "OAuth client" so the app can ask for access. The steps are also on the page:
|
||||||
|
|
||||||
|
1. In the [Google Cloud console](https://console.cloud.google.com/), signed in as the account that will hold the backups, create a project and enable the **Google Drive API**.
|
||||||
|
2. Under **Google Auth Platform**, fill in **Branding** (an app name and your email). Under **Audience**, choose **External**, then **Publish app**. While it's left in "Testing", Google ends the access after 7 days. Google doesn't need to verify it, because the app only asks for access to files it creates.
|
||||||
|
3. Under **Clients**, create an **OAuth client ID** of type **Web application**, with the redirect URI shown on the Off-site Backup page (it ends in `/admin/offsite_backup/callback`, at the address you use for the app).
|
||||||
|
4. Paste the client ID and secret on the page, choose **Save client**, then **Connect Google Drive** and allow access. Google warns that the app isn't verified: choose **Advanced → Go to …**.
|
||||||
|
|
||||||
|
The first backup starts straight away. It copies every uploaded file, so it can take a while.
|
||||||
|
|
||||||
|
**Back up now** runs a backup straight away. **Test restore** downloads the newest database copy from Drive, opens it and checks it, and downloads a few random files and compares them with their checksums. **Disconnect** stops the copies; what's already in Drive stays there.
|
||||||
|
|
||||||
|
### Restoring from Google Drive
|
||||||
|
|
||||||
|
If the server is lost:
|
||||||
|
|
||||||
|
1. Set up the app again (see the main README) with the **same `SECRET_KEY_BASE`**, but don't start it yet.
|
||||||
|
2. From Google Drive, download the newest file in **Simple Aircraft Manager backups → database**, unzip it (`gunzip sam-….sqlite3.gz`), and put it in place of `storage/production.sqlite3` (delete any `production.sqlite3-wal` and `-shm` files next to it).
|
||||||
|
3. Start the app and sign in. The records are back, but uploaded files are still missing.
|
||||||
|
4. Restore the files: `bin/rails offsite:restore_files` downloads them from Drive (the connection is in the restored database). If that doesn't work (for example the `SECRET_KEY_BASE` is lost), download the **files** folder from Drive, unzip it on the server and run `bin/rails "offsite:restore_files[/path/to/files]"`.
|
||||||
|
5. Check the About page (run **Back up and check now**) and open a few entries and documents.
|
||||||
|
|
||||||
## Recordkeeping procedures
|
## Recordkeeping procedures
|
||||||
|
|
||||||
If the app replaces the paper logbooks, AC 120-78B asks for a written procedures document. **Recordkeeping Procedures** generates one from the app's setup:
|
If the app replaces the paper logbooks, AC 120-78B asks for a written procedures document. **Recordkeeping Procedures** generates one from the app's setup:
|
||||||
|
|
||||||
1. Fill in the **details** the app can't know: the owner/operator, the person responsible for the records, how the app is hosted, how it's backed up off-site, and how often to audit and how many entries to sample.
|
1. Fill in the **details** the app can't know: the owner/operator, the person responsible for the records, how the app is hosted, how it's backed up off-site (described automatically when Google Drive is connected), and how often to audit and how many entries to sample.
|
||||||
2. **Preview the draft** and read it through.
|
2. **Preview the draft** and read it through.
|
||||||
3. **Adopt** it as a revision, with a note of what changed. Adopted revisions can't be changed; print one or save it as a PDF with the records.
|
3. **Adopt** it as a revision, with a note of what changed. Adopted revisions can't be changed; print one or save it as a PDF with the records.
|
||||||
|
|
||||||
@@ -52,8 +86,8 @@ The document covers responsible personnel, users and roles, the signing process,
|
|||||||
|
|
||||||
**Records Audits** is the periodic quality check the procedures promise. **Start an audit**:
|
**Records Audits** is the periodic quality check the procedures promise. **Start an audit**:
|
||||||
|
|
||||||
- The app shows its own checks: the latest integrity check and backup, the procedures, signatures, expired certificates, and software changes since the last audit.
|
- The app shows its own checks: the latest integrity check, backup and off-site copy, the procedures, signatures, expired certificates, and software changes since the last audit.
|
||||||
- You confirm the rest: that you restored the off-site backup somewhere separate and opened it; that a random sample of entries matches their work orders or receipts; that the users and roles are right (disable anyone who no longer needs access); and that the paper logbooks are kept.
|
- You confirm the rest: that you restored the off-site backup somewhere separate and opened it (with Google Drive, run **Test restore** and also open a downloaded copy yourself); that a random sample of entries matches their work orders or receipts; that the users and roles are right (disable anyone who no longer needs access); and that the paper logbooks are kept.
|
||||||
- Add a note for each item and overall, then **Record audit**. Audits can't be changed afterwards.
|
- Add a note for each item and overall, then **Record audit**. Audits can't be changed afterwards.
|
||||||
|
|
||||||
The next one is due after the interval set in the procedures. It shows on About, and admins are reminded in the weekly email.
|
The next one is due after the interval set in the procedures. It shows on About, and admins are reminded in the weekly email.
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
namespace :offsite do
|
||||||
|
desc "Restore uploaded files missing from storage, from Google Drive or from a downloaded copy of its files folder: offsite:restore_files[DIR]"
|
||||||
|
task :restore_files, [ :dir ] => :environment do |_, args|
|
||||||
|
dir = args[:dir].presence
|
||||||
|
connection = OffsiteConnection.current
|
||||||
|
abort "Google Drive isn't connected: download the backup's files folder and pass its path, e.g. offsite:restore_files[/path/to/files]" if dir.nil? && !connection.connected?
|
||||||
|
if dir.nil?
|
||||||
|
drive = connection.client
|
||||||
|
drive_ids = drive.list(connection.files_folder_id).to_h { [ _1["name"], _1["id"] ] }
|
||||||
|
end
|
||||||
|
restored = 0
|
||||||
|
missing = []
|
||||||
|
ActiveStorage::Blob.find_each do |blob|
|
||||||
|
next if blob.service.exist?(blob.key)
|
||||||
|
io = if dir
|
||||||
|
path = File.join(dir, blob.key)
|
||||||
|
File.open(path, "rb") if File.file?(path)
|
||||||
|
elsif (id = drive_ids[blob.key])
|
||||||
|
drive.download(id, to: Tempfile.new("sam-restore", binmode: true))
|
||||||
|
end
|
||||||
|
next missing << "#{blob.key} (#{blob.filename})" unless io
|
||||||
|
begin
|
||||||
|
blob.service.upload(blob.key, io, checksum: blob.checksum)
|
||||||
|
restored += 1
|
||||||
|
ensure
|
||||||
|
io.close
|
||||||
|
end
|
||||||
|
end
|
||||||
|
puts "Restored #{restored} #{'file'.pluralize(restored)}."
|
||||||
|
puts "Not found (#{missing.size}):", missing.first(50) if missing.any?
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
require "test_helper"
|
||||||
|
|
||||||
|
# Off-site backup to Google Drive (AC 120-78B 3.2.7).
|
||||||
|
class OffsiteBackupTest < ActionDispatch::IntegrationTest
|
||||||
|
include ActionMailer::TestHelper
|
||||||
|
include ActiveJob::TestHelper
|
||||||
|
|
||||||
|
# An in-memory Drive.
|
||||||
|
class FakeDrive
|
||||||
|
attr_reader :files, :folders
|
||||||
|
|
||||||
|
def initialize = (@files = {}; @folders = {}; @n = 0)
|
||||||
|
def account_email = "backups@example.com"
|
||||||
|
def folder?(id) = @folders.key?(id)
|
||||||
|
def create_folder(name, parent: nil) = "f#{@n += 1}".tap { @folders[_1] = [ name, parent ] }
|
||||||
|
|
||||||
|
def upload(io, name:, parent:, size:, content_type: nil, properties: {})
|
||||||
|
data = io.read
|
||||||
|
raise "size mismatch" unless data.bytesize == size
|
||||||
|
"d#{@n += 1}".tap { @files[_1] = { "name" => name, "parent" => parent, "data" => data, "appProperties" => properties } }
|
||||||
|
end
|
||||||
|
|
||||||
|
def list(parent) = @files.select { _2["parent"] == parent }.map { |id, f| { "id" => id, "name" => f["name"] } }
|
||||||
|
def download(id, to:) = to.tap { _1.write(@files.fetch(id)["data"]); _1.rewind }
|
||||||
|
def delete(id) = @files.delete(id)
|
||||||
|
end
|
||||||
|
|
||||||
|
setup do
|
||||||
|
@admin = create_user("admin@example.com", admin: true)
|
||||||
|
@a = create_aircraft(owner: @admin, tail_number: "N12345")
|
||||||
|
@drive = FakeDrive.new
|
||||||
|
drive = @drive
|
||||||
|
@factory = OffsiteConnection.client_factory
|
||||||
|
OffsiteConnection.client_factory = ->(_) { drive }
|
||||||
|
end
|
||||||
|
|
||||||
|
teardown { OffsiteConnection.client_factory = @factory }
|
||||||
|
|
||||||
|
def connect!
|
||||||
|
OffsiteConnection.current.update!(client_id: "id.apps.googleusercontent.com", client_secret: "secret", refresh_token: "refresh",
|
||||||
|
account_email: "backups@example.com", connected_at: Time.current)
|
||||||
|
end
|
||||||
|
|
||||||
|
def database_snapshot
|
||||||
|
path = Rails.root.join("tmp", "sam-offsite-test-#{SecureRandom.hex(4)}.sqlite3")
|
||||||
|
db = SQLite3::Database.new(path.to_s)
|
||||||
|
db.execute("CREATE TABLE logbook_entries (id integer)")
|
||||||
|
db.execute("CREATE TABLE entry_signatures (id integer)")
|
||||||
|
db.execute("INSERT INTO logbook_entries VALUES (1)")
|
||||||
|
db.close
|
||||||
|
path
|
||||||
|
end
|
||||||
|
|
||||||
|
test "an admin saves the OAuth client and connects through Google" do
|
||||||
|
sign_in @admin
|
||||||
|
get admin_offsite_backup_path
|
||||||
|
assert_response :success
|
||||||
|
assert_includes response.body, "/admin/offsite_backup/callback"
|
||||||
|
patch admin_offsite_backup_path, params: { offsite_connection: { client_id: " id.apps.googleusercontent.com ", client_secret: "secret" } }
|
||||||
|
conn = OffsiteConnection.current
|
||||||
|
assert_equal "id.apps.googleusercontent.com", conn.client_id
|
||||||
|
assert_not_equal "secret", conn.read_attribute_before_type_cast(:client_secret) # encrypted
|
||||||
|
patch admin_offsite_backup_path, params: { offsite_connection: { client_id: "id.apps.googleusercontent.com", client_secret: "" } }
|
||||||
|
assert_equal "secret", conn.reload.client_secret # blank keeps it
|
||||||
|
|
||||||
|
get connect_admin_offsite_backup_path
|
||||||
|
assert_match %r{\Ahttps://accounts\.google\.com/o/oauth2/v2/auth\?}, response.location
|
||||||
|
query = Rack::Utils.parse_query(URI(response.location).query)
|
||||||
|
assert_equal "https://www.googleapis.com/auth/drive.file", query["scope"]
|
||||||
|
assert_equal "offline", query["access_type"]
|
||||||
|
|
||||||
|
get callback_admin_offsite_backup_path, params: { state: "forged", code: "x" }
|
||||||
|
assert_nil conn.reload.refresh_token
|
||||||
|
|
||||||
|
get connect_admin_offsite_backup_path
|
||||||
|
state = Rack::Utils.parse_query(URI(response.location).query)["state"]
|
||||||
|
stub_request(:post, GoogleDriveClient::TOKEN_URL).to_return(status: 200, body: { refresh_token: "refresh-1", access_token: "a", expires_in: 3600 }.to_json)
|
||||||
|
assert_enqueued_with(job: OffsiteBackupJob) do
|
||||||
|
get callback_admin_offsite_backup_path, params: { state: state, code: "auth-code" }
|
||||||
|
end
|
||||||
|
conn.reload
|
||||||
|
assert conn.connected?
|
||||||
|
assert_equal "refresh-1", conn.refresh_token
|
||||||
|
assert_equal "backups@example.com", conn.account_email
|
||||||
|
assert_equal 3, @drive.folders.size
|
||||||
|
assert_equal conn.folder_id, @drive.folders[conn.files_folder_id].last
|
||||||
|
|
||||||
|
stub_request(:post, GoogleDriveClient::REVOKE_URL).to_return(status: 200)
|
||||||
|
delete admin_offsite_backup_path
|
||||||
|
refute conn.reload.connected?
|
||||||
|
end
|
||||||
|
|
||||||
|
test "the backup copies the database and new files once, prunes old copies, and the restore test checks them" do
|
||||||
|
connect!
|
||||||
|
doc = @a.documents.create!(name: "Registration", doc_type: "REGISTRATION")
|
||||||
|
doc.pages.attach(io: StringIO.new("scan one"), filename: "reg.pdf", content_type: "application/pdf")
|
||||||
|
path = database_snapshot
|
||||||
|
with_setting("offsite_backups_kept", 2) do
|
||||||
|
3.times { |i| travel(i.minutes) { OffsiteBackup.run!(path) } }
|
||||||
|
end
|
||||||
|
run = OffsiteRun.latest_backup
|
||||||
|
assert run.ok, run.message
|
||||||
|
conn = OffsiteConnection.current
|
||||||
|
assert_equal 2, @drive.list(conn.database_folder_id).size, OffsiteRun.pluck(:ok, :message, :details).inspect
|
||||||
|
files = @drive.list(conn.files_folder_id)
|
||||||
|
assert_equal [ doc.pages.first.blob.key ], files.map { _1["name"] } # copied once
|
||||||
|
assert_equal "reg.pdf", @drive.files[files.first["id"]]["appProperties"][:filename]
|
||||||
|
assert_equal 1, OffsiteRun.backups.where("files_uploaded > 0").count
|
||||||
|
|
||||||
|
test_run = OffsiteBackup.restore_test!
|
||||||
|
assert test_run.ok, test_run.message
|
||||||
|
assert_includes test_run.message, "integrity check ok"
|
||||||
|
assert_includes test_run.message, "1 of 1 sampled files match"
|
||||||
|
|
||||||
|
@drive.files[files.first["id"]]["data"] = "tampered"
|
||||||
|
refute OffsiteBackup.restore_test!.ok
|
||||||
|
ensure
|
||||||
|
File.delete(path) if path && File.exist?(path)
|
||||||
|
end
|
||||||
|
|
||||||
|
test "failures are recorded and emailed to admins; procedures and audits use the Drive backup" do
|
||||||
|
connect!
|
||||||
|
def @drive.upload(*, **) = raise(GoogleDriveClient::AuthError, "Google Drive refused access; connect again")
|
||||||
|
BackupRun.create!(ok: true, path: database_snapshot.to_s, message: "ok")
|
||||||
|
assert_emails(1) { OffsiteBackupJob.perform_now }
|
||||||
|
refute OffsiteRun.latest_backup.ok
|
||||||
|
assert_includes ActionMailer::Base.deliveries.last.subject, "Off-site backup failed"
|
||||||
|
|
||||||
|
policy = RecordkeepingPolicy.current
|
||||||
|
policy.update!(operator_name: "Jane Pilot", responsible_name: "Jane Pilot", hosting_description: "Home server")
|
||||||
|
assert policy.complete? # Drive stands in for the description
|
||||||
|
item = RecordsAudit.new.auto_items.find { _1.key == "offsite" }
|
||||||
|
refute item.passed
|
||||||
|
sign_in @admin
|
||||||
|
get draft_admin_procedures_path
|
||||||
|
assert_includes response.body, "Google Drive (account backups@example.com"
|
||||||
|
get admin_about_path
|
||||||
|
assert_includes response.body, "Off-site backup"
|
||||||
|
ensure
|
||||||
|
File.delete(BackupRun.last.path) if BackupRun.last&.path && File.exist?(BackupRun.last.path)
|
||||||
|
end
|
||||||
|
|
||||||
|
test "the Drive client uploads with a resumable session and refreshes its token" do
|
||||||
|
stub_request(:post, GoogleDriveClient::TOKEN_URL).with(body: hash_including("grant_type" => "refresh_token"))
|
||||||
|
.to_return(status: 200, body: { access_token: "tok", expires_in: 3600 }.to_json)
|
||||||
|
stub_request(:post, %r{\Ahttps://www\.googleapis\.com/upload/drive/v3/files\?.*uploadType=resumable})
|
||||||
|
.with(headers: { "Authorization" => "Bearer tok", "X-Upload-Content-Length" => "5" })
|
||||||
|
.to_return(status: 200, headers: { "Location" => "https://www.googleapis.com/upload/drive/v3/files?upload_id=abc" })
|
||||||
|
stub_request(:put, "https://www.googleapis.com/upload/drive/v3/files?upload_id=abc").with(body: "hello")
|
||||||
|
.to_return(status: 200, body: { id: "file-1" }.to_json)
|
||||||
|
client = GoogleDriveClient.new(client_id: "id", client_secret: "s", refresh_token: "r")
|
||||||
|
assert_equal "file-1", client.upload(StringIO.new("hello"), name: "x", parent: "p", size: 5)
|
||||||
|
|
||||||
|
stub_request(:get, %r{/drive/v3/about}).to_return(status: 401, body: { error: { message: "Invalid Credentials" } }.to_json)
|
||||||
|
assert_raises(GoogleDriveClient::AuthError) { client.account_email }
|
||||||
|
stub_request(:post, GoogleDriveClient::TOKEN_URL).to_return(status: 400, body: { error: "invalid_grant" }.to_json)
|
||||||
|
assert_raises(GoogleDriveClient::AuthError) { GoogleDriveClient.new(client_id: "id", client_secret: "s", refresh_token: "r").account_email }
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def with_setting(key, value)
|
||||||
|
Setting.create!(key: key, value: value.to_s)
|
||||||
|
yield
|
||||||
|
end
|
||||||
|
end
|
||||||
Reference in new issue
Block a user