From 5bf682502b06789183c44219cd3d144a6acdfdc8 Mon Sep 17 00:00:00 2001 From: Rick Barrette Date: Thu, 5 Nov 2026 19:40:20 -0500 Subject: [PATCH] AirTrail sync and export, GPS tracks, airport database with autocomplete MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AirTrail (https://github.com/johanohly/AirTrail): - File export (Flights → Export for AirTrail) in AirTrail's import format, validated against its schema; same-day repeat legs get leg numbers so AirTrail's de-duplication doesn't merge them. - Automatic sync through AirTrail's REST API (Profile → AirTrail sync): per-user connection with an encrypted API key and scope check; flights are created/updated/deleted in background jobs; a full sync first links flights already in AirTrail (e.g. from a file import) instead of duplicating them; airports are resolved and cached, with manual mapping for unknown codes. - GPS tracks: GPX, KML and CSV track logs are converted to AirTrail's track format for both sync and export; the flight page shows a summary. - Aircraft ICAO type (e.g. C150) for AirTrail's aircraft type. Airport database (OurAirports, the source AirTrail uses): - Managed like AirTrail: downloaded in the background on first start, updated in place from Admin → Airports (adds, changes, removes; ids stable), with admin-managed custom airports that updates never touch. - Each airport carries AirTrail's code, so logged codes translate to AirTrail's exactly (1B9 → K1B9, 4MB → K5M3). - From/To autocomplete on new flights (airports this aircraft has used rank first); From is pre-filled with the last flight's destination; codes are normalized to uppercase. - Flights → Standardize airport codes: rewrite logged codes to the standard code, and fix codes not in the database (typos such as 0R8 → 08R are suggested). Also: admin pages grouped under an Admin dropdown menu. Co-Authored-By: Claude Opus 5.5 --- Gemfile | 2 + Gemfile.lock | 12 ++ app/controllers/admin/airports_controller.rb | 63 +++++++ .../air_trail_connections_controller.rb | 94 ++++++++++ .../air_trail_exports_controller.rb | 28 +++ app/controllers/aircraft_controller.rb | 2 +- app/controllers/airport_codes_controller.rb | 19 ++ app/controllers/airports_controller.rb | 15 ++ app/controllers/flight_logs_controller.rb | 12 +- app/helpers/aircraft_helper.rb | 2 +- .../airport_autocomplete_controller.js | 130 +++++++++++++ .../controllers/dropdown_controller.js | 12 ++ app/jobs/air_trail_backfill_job.rb | 13 ++ app/jobs/air_trail_delete_job.rb | 12 ++ app/jobs/air_trail_push_job.rb | 14 ++ app/jobs/airport_update_job.rb | 23 +++ app/models/air_trail_airport.rb | 5 + app/models/air_trail_connection.rb | 34 ++++ app/models/air_trail_sync.rb | 6 + app/models/aircraft.rb | 2 + app/models/airport.rb | 64 +++++++ app/models/airport_update.rb | 17 ++ app/models/flight_log.rb | 18 ++ app/models/user.rb | 1 + app/services/air_trail_client.rb | 97 ++++++++++ app/services/air_trail_export.rb | 96 ++++++++++ app/services/air_trail_flight_data.rb | 61 ++++++ app/services/air_trail_syncer.rb | 177 ++++++++++++++++++ app/services/airport_code_standardizer.rb | 89 +++++++++ app/services/track_parser.rb | 169 +++++++++++++++++ app/views/admin/airports/_form.html.erb | 16 ++ app/views/admin/airports/edit.html.erb | 2 + app/views/admin/airports/index.html.erb | 87 +++++++++ app/views/admin/airports/new.html.erb | 2 + app/views/air_trail_connections/show.html.erb | 83 ++++++++ app/views/air_trail_exports/new.html.erb | 43 +++++ app/views/aircraft/_form.html.erb | 1 + app/views/airport_codes/show.html.erb | 65 +++++++ app/views/flight_logs/index.html.erb | 2 +- app/views/flight_logs/new.html.erb | 4 +- app/views/flight_logs/show.html.erb | 12 +- app/views/layouts/application.html.erb | 12 +- app/views/profiles/edit.html.erb | 8 + app/views/shared/_airport_field.html.erb | 11 ++ .../initializers/active_record_encryption.rb | 11 ++ config/initializers/airports.rb | 12 ++ config/routes.rb | 12 ++ .../20261106130000_create_air_trail_sync.rb | 46 +++++ db/migrate/20261106140000_create_airports.rb | 32 ++++ ...6150000_add_custom_airports_and_updates.rb | 21 +++ db/schema.rb | 92 ++++++++- lib/airport_data.rb | 95 ++++++++++ lib/tasks/airports.rake | 9 + test/fixtures/files/tracks/flight.csv | 4 + test/fixtures/files/tracks/flight.gpx | 8 + test/fixtures/files/tracks/flight.kml | 7 + test/fixtures/files/tracks/linestring.kml | 4 + test/integration/admin_airports_test.rb | 78 ++++++++ .../integration/air_trail_export_flow_test.rb | 31 +++ test/integration/air_trail_sync_flow_test.rb | 70 +++++++ test/integration/pages_test.rb | 32 ++++ test/models/airport_test.rb | 72 +++++++ test/services/air_trail_export_test.rb | 88 +++++++++ test/services/air_trail_syncer_test.rb | 130 +++++++++++++ .../airport_code_standardizer_test.rb | 52 +++++ test/services/track_parser_test.rb | 52 +++++ test/support/fake_air_trail.rb | 79 ++++++++ test/test_helper.rb | 3 + 68 files changed, 2667 insertions(+), 10 deletions(-) create mode 100644 app/controllers/admin/airports_controller.rb create mode 100644 app/controllers/air_trail_connections_controller.rb create mode 100644 app/controllers/air_trail_exports_controller.rb create mode 100644 app/controllers/airport_codes_controller.rb create mode 100644 app/controllers/airports_controller.rb create mode 100644 app/javascript/controllers/airport_autocomplete_controller.js create mode 100644 app/javascript/controllers/dropdown_controller.js create mode 100644 app/jobs/air_trail_backfill_job.rb create mode 100644 app/jobs/air_trail_delete_job.rb create mode 100644 app/jobs/air_trail_push_job.rb create mode 100644 app/jobs/airport_update_job.rb create mode 100644 app/models/air_trail_airport.rb create mode 100644 app/models/air_trail_connection.rb create mode 100644 app/models/air_trail_sync.rb create mode 100644 app/models/airport.rb create mode 100644 app/models/airport_update.rb create mode 100644 app/services/air_trail_client.rb create mode 100644 app/services/air_trail_export.rb create mode 100644 app/services/air_trail_flight_data.rb create mode 100644 app/services/air_trail_syncer.rb create mode 100644 app/services/airport_code_standardizer.rb create mode 100644 app/services/track_parser.rb create mode 100644 app/views/admin/airports/_form.html.erb create mode 100644 app/views/admin/airports/edit.html.erb create mode 100644 app/views/admin/airports/index.html.erb create mode 100644 app/views/admin/airports/new.html.erb create mode 100644 app/views/air_trail_connections/show.html.erb create mode 100644 app/views/air_trail_exports/new.html.erb create mode 100644 app/views/airport_codes/show.html.erb create mode 100644 app/views/shared/_airport_field.html.erb create mode 100644 config/initializers/active_record_encryption.rb create mode 100644 config/initializers/airports.rb create mode 100644 db/migrate/20261106130000_create_air_trail_sync.rb create mode 100644 db/migrate/20261106140000_create_airports.rb create mode 100644 db/migrate/20261106150000_add_custom_airports_and_updates.rb create mode 100644 lib/airport_data.rb create mode 100644 lib/tasks/airports.rake create mode 100644 test/fixtures/files/tracks/flight.csv create mode 100644 test/fixtures/files/tracks/flight.gpx create mode 100644 test/fixtures/files/tracks/flight.kml create mode 100644 test/fixtures/files/tracks/linestring.kml create mode 100644 test/integration/admin_airports_test.rb create mode 100644 test/integration/air_trail_export_flow_test.rb create mode 100644 test/integration/air_trail_sync_flow_test.rb create mode 100644 test/models/airport_test.rb create mode 100644 test/services/air_trail_export_test.rb create mode 100644 test/services/air_trail_syncer_test.rb create mode 100644 test/services/airport_code_standardizer_test.rb create mode 100644 test/services/track_parser_test.rb create mode 100644 test/support/fake_air_trail.rb diff --git a/Gemfile b/Gemfile index 25436f6..42d21be 100644 --- a/Gemfile +++ b/Gemfile @@ -71,3 +71,5 @@ gem "roo", "~> 2.10" gem "csv", "~> 3.3" gem "caxlsx", "~> 4.5", group: :test + +gem "webmock", "~> 3.34", group: :test diff --git a/Gemfile.lock b/Gemfile.lock index d68a005..b9b43bc 100644 --- a/Gemfile.lock +++ b/Gemfile.lock @@ -107,6 +107,9 @@ GEM chartkick (5.2.1) concurrent-ruby (1.3.8) connection_pool (3.0.2) + crack (1.0.1) + bigdecimal + rexml crass (1.0.8) csv (3.3.7) date (3.6.0) @@ -125,6 +128,7 @@ GEM raabro (~> 1.4) globalid (1.4.0) activesupport (>= 6.1) + hashdiff (1.2.1) htmlentities (4.4.2) i18n (1.15.2) concurrent-ruby (~> 1.0) @@ -377,6 +381,10 @@ GEM actionview (>= 8.0.0) bindex (>= 0.4.0) railties (>= 8.0.0) + webmock (3.34.0) + addressable (>= 2.8.0) + crack (>= 0.3.2) + hashdiff (>= 0.4.0, < 2.0.0) websocket (1.2.11) websocket-driver (0.8.2) base64 @@ -425,6 +433,7 @@ DEPENDENCIES turbo-rails tzinfo-data web-console + webmock (~> 3.34) CHECKSUMS action_text-trix (2.1.19) sha256=7012f59421009cf284aa651294896414d653a61a2417c9b8714c8476d2f74009 @@ -456,6 +465,7 @@ CHECKSUMS chartkick (5.2.1) sha256=2848d7de87189f30f28d077eb0bbdebc8a1f0f6f81de1ded95008fe564369949 concurrent-ruby (1.3.8) sha256=b2f1be836e968ccc78ccfce277ea79c72a88633f22306782c16ff23fb415d1e1 connection_pool (3.0.2) sha256=33fff5ba71a12d2aa26cb72b1db8bba2a1a01823559fb01d29eb74c286e62e0a + crack (1.0.1) sha256=ff4a10390cd31d66440b7524eb1841874db86201d5b70032028553130b6d4c7e crass (1.0.8) sha256=7a0145c7a2c59f8e7923b3a69ce819c444823a00a5127d805298fea13be24efe csv (3.3.7) sha256=15a3f16f6628da8664d455eb634e7f22031bcf53044923caf56975109e71b9f8 date (3.6.0) sha256=18ac945c4e24ebd93534f8eab8c2fd8c5604b3e977105b3798d62a1e8d27b580 @@ -468,6 +478,7 @@ CHECKSUMS et-orbi (1.4.2) sha256=bb555dae668419cb24caa2a293a170e58be6d4df1e017c51f5030bdc133cd20c fugit (1.14.0) sha256=e311a9cc85a54bb67649df45060e721282a1e2c8ebad40e41874a3a7d576f37e globalid (1.4.0) sha256=037f12fbf1d9d7a014d501c2d5c77356fd4ddd96d7a7991d6700bba96706f427 + hashdiff (1.2.1) sha256=9c079dbc513dfc8833ab59c0c2d8f230fa28499cc5efb4b8dd276cf931457cd1 htmlentities (4.4.2) sha256=bbafbdf69f2eca9262be4efef7e43e6a1de54c95eb600f26984f71d2fe96c5c3 i18n (1.15.2) sha256=00f9eb62412fe593b2a65a97daa75300d37abb8f7202ec748e94b6d46a9dd1b5 importmap-rails (2.2.3) sha256=7101be2a4dc97cf1558fb8f573a718404c5f6bcfe94f304bf1f39e444feeb16a @@ -573,6 +584,7 @@ CHECKSUMS uri (1.1.2) sha256=89292e98eabbef6248b0a0ed5e7e398e152f83f17560023a6d94a7f492b169fb useragent (0.16.11) sha256=700e6413ad4bb954bb63547fa098dddf7b0ebe75b40cc6f93b8d54255b173844 web-console (4.3.0) sha256=e13b71301cdfc2093f155b5aa3a622db80b4672d1f2f713119cc7ec7ac6a6da4 + webmock (3.34.0) sha256=72eed29dc8c00148d592acaedaaa6b869e2e397de57dfc8602b84aa834f46b33 websocket (1.2.11) sha256=b7e7a74e2410b5e85c25858b26b3322f29161e300935f70a0e0d3c35e0462737 websocket-driver (0.8.2) sha256=97c556b019bf3410b4961002ac501621e9322d3f8a7bc02161a09301cc4c4146 websocket-extensions (0.1.5) sha256=1c6ba63092cda343eb53fc657110c71c754c56484aad42578495227d717a8241 diff --git a/app/controllers/admin/airports_controller.rb b/app/controllers/admin/airports_controller.rb new file mode 100644 index 0000000..8f86268 --- /dev/null +++ b/app/controllers/admin/airports_controller.rb @@ -0,0 +1,63 @@ +# Admin → Airports: OurAirports updates and custom airports. +class Admin::AirportsController < ApplicationController + before_action :require_admin! + before_action :set_airport, only: %i[edit update destroy] + + def index + @total = Airport.count + @custom = Airport.custom.order(:ident) + @updates = AirportUpdate.recent_first.includes(:requested_by).limit(5) + @running = AirportUpdate.in_progress.exists? + @last_success = AirportUpdate.last_success + @query = params[:q].to_s.strip + @results = @query.present? ? Airport.search(@query, limit: 20) : [] + end + + # Queue a download + sync from OurAirports. + def refresh + if AirportUpdate.in_progress.exists? + redirect_to admin_airports_path, alert: "An update is already running." + else + update = AirportUpdate.create!(source_url: AirportData::SOURCE_URL, requested_by: current_user) + AirportUpdateJob.perform_later(update.id) + redirect_to admin_airports_path, notice: "Airport update started. Refresh this page in a minute or two." + end + end + + def new + @airport = Airport.new(custom: true, kind: "small_airport", ident: params[:ident]) + end + + def create + @airport = Airport.new(airport_params.merge(custom: true)) + if @airport.save + redirect_to admin_airports_path, notice: "Custom airport #{@airport.ident} added." + else + render :new, status: :unprocessable_entity + end + end + + def edit; end + + def update + if @airport.update(airport_params) + redirect_to admin_airports_path, notice: "#{@airport.ident} updated." + else + render :edit, status: :unprocessable_entity + end + end + + def destroy + @airport.destroy! + redirect_to admin_airports_path, notice: "#{@airport.ident} removed." + end + + private + + # Only custom airports can be edited; OurAirports data comes from updates. + def set_airport = @airport = Airport.custom.find(params[:id]) + + def airport_params + params.require(:airport).permit(:ident, :name, :kind, :municipality, :region, :country, :latitude, :longitude, :elevation_ft, :gps_code, :local_code, :iata) + end +end diff --git a/app/controllers/air_trail_connections_controller.rb b/app/controllers/air_trail_connections_controller.rb new file mode 100644 index 0000000..ed89d7f --- /dev/null +++ b/app/controllers/air_trail_connections_controller.rb @@ -0,0 +1,94 @@ +# The signed-in user's AirTrail sync settings (Profile → AirTrail). +class AirTrailConnectionsController < ApplicationController + before_action :set_connection, except: %i[show create] + + def show + @connection = current_user.air_trail_connection || current_user.build_air_trail_connection + return unless @connection.persisted? + @counts = @connection.counts + @failed = @connection.syncs.failed.includes(flight_log: :aircraft).order(updated_at: :desc).limit(20) + @unresolved = @connection.unresolved_airport_codes + @flight_count = FlightLog.where(user: current_user).count + end + + def create + @connection = current_user.build_air_trail_connection(connection_params) + save_and_verify + end + + def update + attrs = connection_params + attrs.delete(:api_key) if attrs[:api_key].blank? # keep the stored key + url_changed = attrs[:base_url].present? && attrs[:base_url].chomp("/") != @connection.base_url + @connection.assign_attributes(attrs) + @connection.remote_user_id = nil if url_changed || attrs.key?(:api_key) # re-identify + save_and_verify + end + + def test + verify + redirect_to air_trail_connection_path + end + + # Link existing AirTrail flights and push everything else, in the background. + def sync + AirTrailBackfillJob.perform_later(@connection.id) + redirect_to air_trail_connection_path, notice: "Sync started. Refresh this page in a minute to see progress." + end + + def map_airport + code = params[:code].to_s.strip.upcase + target = params[:airtrail_code].to_s.strip.upcase + match = target.present? && AirTrailSyncer.new(@connection).map_airport(code, target) + if match + AirTrailBackfillJob.perform_later(@connection.id) # retry flights that needed it + redirect_to air_trail_connection_path, notice: "#{code} mapped to #{match['icao']} (#{match['name']}). Retrying affected flights." + else + redirect_to air_trail_connection_path, alert: "AirTrail has no airport matching “#{target}”." + end + rescue AirTrailClient::Error => e + redirect_to air_trail_connection_path, alert: e.message + end + + # Stops syncing. Flights already in AirTrail are left there. + def destroy + @connection.destroy! + redirect_to air_trail_connection_path, notice: "AirTrail disconnected. Flights already in AirTrail were left as they are." + end + + private + + def set_connection + @connection = current_user.air_trail_connection or redirect_to(air_trail_connection_path) + end + + def connection_params + params.require(:air_trail_connection).permit(:base_url, :api_key, :enabled) + end + + def save_and_verify + if @connection.save + verify + redirect_to air_trail_connection_path + else + render :show, status: :unprocessable_entity + end + end + + def verify + me = @connection.client.me + scopes = Array(me.dig("credential", "scopes")) + missing = me.dig("credential", "kind") == "apiKey" ? AirTrailConnection::REQUIRED_SCOPES - scopes : [] + @connection.update!(remote_user_id: me["id"], remote_username: me["username"], checked_at: Time.current, + status: missing.empty? ? "ok" : "error", + last_error: missing.empty? ? nil : "API key is missing scopes: #{missing.join(', ')}") + if missing.empty? + flash[:notice] = "Connected to AirTrail as #{me['username']}." + else + flash[:alert] = @connection.last_error + end + rescue AirTrailClient::Error => e + @connection.update!(status: "error", last_error: e.message, checked_at: Time.current) + flash[:alert] = e.message + end +end diff --git a/app/controllers/air_trail_exports_controller.rb b/app/controllers/air_trail_exports_controller.rb new file mode 100644 index 0000000..f765de6 --- /dev/null +++ b/app/controllers/air_trail_exports_controller.rb @@ -0,0 +1,28 @@ +# Download the aircraft's flights as an AirTrail import file. +class AirTrailExportsController < ApplicationController + include AircraftScoped + + def new + @export = build_export + end + + def create + @export = build_export + if @export.valid? + send_data @export.to_json, filename: @export.filename, type: "application/json", disposition: "attachment" + else + render :new, status: :unprocessable_entity + end + end + + private + + def build_export + AirTrailExport.new(@aircraft, username: params[:username].presence || default_username, + aircraft_type: params[:aircraft_type], + only_user: params[:scope] == "mine" ? current_user : nil) + end + + # A reasonable guess for the AirTrail username: the part of the email before "@". + def default_username = current_user.email_address.to_s.split("@").first.to_s.gsub(/[^A-Za-z0-9._-]/, "").first(20) +end diff --git a/app/controllers/aircraft_controller.rb b/app/controllers/aircraft_controller.rb index a1dd8fb..0a94489 100644 --- a/app/controllers/aircraft_controller.rb +++ b/app/controllers/aircraft_controller.rb @@ -68,7 +68,7 @@ class AircraftController < ApplicationController params.require(:aircraft).permit( :tail_number, :make, :model, :serial_number, :description, :purchased, :status, :tach_time, :tach_time_offset, :hobbs_time, :hobbs_time_offset, - :airworthiness_enforcement, :picture + :airworthiness_enforcement, :picture, :icao_type ) end end diff --git a/app/controllers/airport_codes_controller.rb b/app/controllers/airport_codes_controller.rb new file mode 100644 index 0000000..1665331 --- /dev/null +++ b/app/controllers/airport_codes_controller.rb @@ -0,0 +1,19 @@ +# Flights → Standardize airport codes (owners). +class AirportCodesController < ApplicationController + include AircraftScoped + before_action :require_owner! + + def show + @standardizer = AirportCodeStandardizer.new(@aircraft) + end + + def update + codes = Array(params[:codes]).map(&:to_s) + standardizer = AirportCodeStandardizer.new(@aircraft) + # Only codes that are actually unknown in this log, and only the ticked ones. + fixes = params.fetch(:fixes, {}).permit(*standardizer.unknown.map(&:code)).to_h + .select { |code, _| Array(params[:fix]).include?(code) } + updated = standardizer.apply!(codes, fixes: fixes) + redirect_to aircraft_flight_logs_path(@aircraft), notice: "Updated #{updated} airport entries across your flights." + end +end diff --git a/app/controllers/airports_controller.rb b/app/controllers/airports_controller.rb new file mode 100644 index 0000000..b3f71ea --- /dev/null +++ b/app/controllers/airports_controller.rb @@ -0,0 +1,15 @@ +class AirportsController < ApplicationController + # GET /airports/search?q=quonset → [{ code, name, place, airtrail_code, kind }] + def search + render json: Airport.search(params[:q], prefer: familiar_codes).map(&:as_suggestion) + end + + private + + # Airports this aircraft has flown from/to rank first. + def familiar_codes + aircraft = params[:aircraft_id].presence && current_user.accessible_aircraft.find_by(id: params[:aircraft_id]) + return [] unless aircraft + aircraft.flight_logs.distinct.pluck(:departure_location, :destination_location).flatten.compact_blank.uniq + end +end diff --git a/app/controllers/flight_logs_controller.rb b/app/controllers/flight_logs_controller.rb index dfd4a3f..238719f 100644 --- a/app/controllers/flight_logs_controller.rb +++ b/app/controllers/flight_logs_controller.rb @@ -9,11 +9,21 @@ class FlightLogsController < ApplicationController def show @flight_log = @aircraft.flight_logs.find(params[:id]) + begin + @track = TrackParser.parse_attachment(@flight_log.track_log) + rescue TrackParser::Error => e + @track_error = "Couldn't read track: #{e.message}" + end + @air_trail_connection = @flight_log.user&.air_trail_connection + @air_trail_sync = @flight_log.air_trail_syncs.first end def new + # Aircraft almost always depart from where they last landed. + last = @aircraft.flight_logs.where.not(destination_location: [ nil, "" ]).recent_first.first @flight_log = @aircraft.flight_logs.new(date: Date.current, tach_out: @aircraft.tach_time, - hobbs_out: @aircraft.hobbs_time) + hobbs_out: @aircraft.hobbs_time, + departure_location: last&.destination_location) end def create diff --git a/app/helpers/aircraft_helper.rb b/app/helpers/aircraft_helper.rb index bc43d6e..9f732e7 100644 --- a/app/helpers/aircraft_helper.rb +++ b/app/helpers/aircraft_helper.rb @@ -5,7 +5,7 @@ module AircraftHelper [ "Overview", aircraft_path(aircraft), %w[aircraft] ], [ "Components", aircraft_components_path(aircraft), %w[components] ], [ "Squawks", aircraft_squawks_path(aircraft), %w[squawks] ], - [ "Flights", aircraft_flight_logs_path(aircraft), %w[flight_logs flight_imports] ], + [ "Flights", aircraft_flight_logs_path(aircraft), %w[flight_logs flight_imports air_trail_exports airport_codes] ], [ "Oil & Fuel", aircraft_consumable_records_path(aircraft), %w[consumable_records] ], [ "Logbook", aircraft_logbook_entries_path(aircraft), %w[logbook_entries] ], [ "Inspections", aircraft_inspections_path(aircraft), %w[inspections inspection_types inspection_records] ], diff --git a/app/javascript/controllers/airport_autocomplete_controller.js b/app/javascript/controllers/airport_autocomplete_controller.js new file mode 100644 index 0000000..0881378 --- /dev/null +++ b/app/javascript/controllers/airport_autocomplete_controller.js @@ -0,0 +1,130 @@ +import { Controller } from "@hotwired/stimulus" + +// Airport code field with suggestions from the airport database. +//
+// +// +//

+//
+export default class extends Controller { + static targets = ["input", "list", "hint"] + static values = { url: String } + + connect() { + this.results = [] + this.active = -1 + this.inputTarget.setAttribute("autocomplete", "off") + this.inputTarget.setAttribute("role", "combobox") + this.inputTarget.setAttribute("aria-autocomplete", "list") + this.inputTarget.setAttribute("aria-expanded", "false") + this.listTarget.id ||= `${this.inputTarget.id}-suggestions` + this.inputTarget.setAttribute("aria-controls", this.listTarget.id) + this.describe() // e.g. From pre-filled from the last flight + } + + disconnect() { clearTimeout(this.timer) } + + search() { + clearTimeout(this.timer) + this.timer = setTimeout(() => this.fetch(), 150) + } + + async fetch() { + const q = this.inputTarget.value.trim() + if (q.length < 2) { this.close(); this.setHint(""); return } + const url = new URL(this.urlValue, window.location.origin) + url.searchParams.set("q", q) + const response = await fetch(url, { headers: { Accept: "application/json" } }) + if (!response.ok || q !== this.inputTarget.value.trim()) return + this.results = await response.json() + this.active = -1 + this.render() + this.updateHint() + } + + render() { + this.listTarget.replaceChildren(...this.results.map((airport, i) => { + const li = document.createElement("li") + li.id = `${this.listTarget.id}-${i}` + li.setAttribute("role", "option") + li.className = "cursor-pointer px-3 py-1.5 text-sm hover:bg-sky-50" + const code = document.createElement("span") + code.className = "font-mono font-semibold" + code.textContent = airport.code + const name = document.createElement("span") + name.className = "ml-2 text-slate-700" + name.textContent = airport.name + const place = document.createElement("span") + place.className = "ml-1 text-xs text-slate-500" + place.textContent = [airport.place, airport.aka?.length ? `also ${airport.aka.join(", ")}` : null, airport.kind].filter(Boolean).join(" · ") + li.append(code, name, place) + li.addEventListener("mousedown", (event) => { event.preventDefault(); this.choose(i) }) + return li + })) + const open = this.results.length > 0 + this.listTarget.hidden = !open + this.inputTarget.setAttribute("aria-expanded", String(open)) + } + + keydown(event) { + if (this.listTarget.hidden) return + if (event.key === "ArrowDown" || event.key === "ArrowUp") { + event.preventDefault() + const step = event.key === "ArrowDown" ? 1 : -1 + this.active = (this.active + step + this.results.length) % this.results.length + this.highlight() + } else if (event.key === "Enter" && this.active >= 0) { + event.preventDefault() + this.choose(this.active) + } else if (event.key === "Escape") { + this.close() + } + } + + highlight() { + Array.from(this.listTarget.children).forEach((li, i) => { + li.classList.toggle("bg-sky-100", i === this.active) + li.setAttribute("aria-selected", String(i === this.active)) + }) + this.inputTarget.setAttribute("aria-activedescendant", this.active >= 0 ? `${this.listTarget.id}-${this.active}` : "") + } + + choose(index) { + const airport = this.results[index] + this.inputTarget.value = airport.code + this.setHint(this.describeAirport(airport)) + this.close() + } + + blur() { setTimeout(() => this.close(), 100) } + + close() { + this.listTarget.hidden = true + this.inputTarget.setAttribute("aria-expanded", "false") + } + + // Show the airport name for a typed code, or note that it isn't in the database. + updateHint() { + const code = this.inputTarget.value.trim().toUpperCase() + const match = this.results.find((a) => a.code === code || (a.aka || []).includes(code)) + if (match) this.setHint(this.describeAirport(match)) + else if (code.length >= 2 && code.length <= 4 && !code.includes(" ")) this.setHint("Not in the airport database", true) + else this.setHint("") + } + + async describe() { + if (!this.inputTarget.value.trim()) return + await this.fetch() + this.close() + } + + describeAirport(a) { + return [a.name, a.place].filter(Boolean).join(", ") + } + + setHint(text, warn = false) { + if (!this.hasHintTarget) return + this.hintTarget.textContent = text + this.hintTarget.classList.toggle("text-amber-700", warn) + } +} diff --git a/app/javascript/controllers/dropdown_controller.js b/app/javascript/controllers/dropdown_controller.js new file mode 100644 index 0000000..d3e7415 --- /dev/null +++ b/app/javascript/controllers/dropdown_controller.js @@ -0,0 +1,12 @@ +import { Controller } from "@hotwired/stimulus" + +// Closes a
dropdown when clicking elsewhere or pressing Escape. +export default class extends Controller { + closeOutside(event) { + if (!this.element.contains(event.target)) this.close() + } + + close() { + this.element.removeAttribute("open") + } +} diff --git a/app/jobs/air_trail_backfill_job.rb b/app/jobs/air_trail_backfill_job.rb new file mode 100644 index 0000000..c0c86a3 --- /dev/null +++ b/app/jobs/air_trail_backfill_job.rb @@ -0,0 +1,13 @@ +# Sync all of a user's flights: link ones already in AirTrail, push the rest. +class AirTrailBackfillJob < ApplicationJob + queue_as :default + limits_concurrency to: 1, key: ->(connection_id) { "air_trail_backfill_#{connection_id}" } if respond_to?(:limits_concurrency) + retry_on AirTrailClient::NetworkError, wait: :polynomially_longer, attempts: 3 + discard_on AirTrailClient::AuthError + + def perform(connection_id) + connection = AirTrailConnection.find_by(id: connection_id) or return + return unless connection.enabled? + AirTrailSyncer.new(connection).backfill + end +end diff --git a/app/jobs/air_trail_delete_job.rb b/app/jobs/air_trail_delete_job.rb new file mode 100644 index 0000000..d0e6d0a --- /dev/null +++ b/app/jobs/air_trail_delete_job.rb @@ -0,0 +1,12 @@ +# Remove a deleted flight from AirTrail. +class AirTrailDeleteJob < ApplicationJob + queue_as :default + retry_on AirTrailClient::NetworkError, wait: :polynomially_longer, attempts: 6 + discard_on AirTrailClient::AuthError + + def perform(connection_id, remote_id) + connection = AirTrailConnection.find_by(id: connection_id) or return + return unless connection.enabled? + AirTrailSyncer.new(connection).delete_remote(remote_id) + end +end diff --git a/app/jobs/air_trail_push_job.rb b/app/jobs/air_trail_push_job.rb new file mode 100644 index 0000000..89f149a --- /dev/null +++ b/app/jobs/air_trail_push_job.rb @@ -0,0 +1,14 @@ +# Create/update one flight in its pilot's AirTrail account. +class AirTrailPushJob < ApplicationJob + queue_as :default + retry_on AirTrailClient::NetworkError, wait: :polynomially_longer, attempts: 6 + discard_on AirTrailClient::AuthError # recorded on the connection; user must fix the key + + def perform(flight_log_id) + flight = FlightLog.includes(:aircraft, user: :air_trail_connection).find_by(id: flight_log_id) or return + connection = flight.user&.air_trail_connection + return unless connection&.enabled? + return unless AirTrailFlightData.exportable?(flight) + AirTrailSyncer.new(connection).push(flight) + end +end diff --git a/app/jobs/airport_update_job.rb b/app/jobs/airport_update_job.rb new file mode 100644 index 0000000..e1b0976 --- /dev/null +++ b/app/jobs/airport_update_job.rb @@ -0,0 +1,23 @@ +# Downloads OurAirports and syncs the airports table (AirportData.refresh!). +class AirportUpdateJob < ApplicationJob + queue_as :default + limits_concurrency to: 1, key: "airport_update", duration: 1.hour if respond_to?(:limits_concurrency) + + def perform(update_id = nil) + update = (update_id && AirportUpdate.find_by(id: update_id)) || AirportUpdate.create!(source_url: AirportData::SOURCE_URL) + update.update!(status: "running", started_at: Time.current, error: nil) + counts = AirportData.refresh!(update.source_url.presence || AirportData::SOURCE_URL) + update.update!(status: "done", finished_at: Time.current, created_count: counts[:created], + updated_count: counts[:updated], removed_count: counts[:removed]) + rescue StandardError => e + update&.update!(status: "failed", finished_at: Time.current, error: "#{e.class}: #{e.message}".truncate(1000)) + Rails.logger.error("[airports] update failed: #{e.class}: #{e.message}") + end + + # Called at boot: populate the table on first start, like AirTrail does. + def self.ensure_populated + return if Airport.where(custom: false).exists? || AirportUpdate.in_progress.exists? + update = AirportUpdate.create!(source_url: AirportData::SOURCE_URL) + perform_later(update.id) + end +end diff --git a/app/models/air_trail_airport.rb b/app/models/air_trail_airport.rb new file mode 100644 index 0000000..c6bba78 --- /dev/null +++ b/app/models/air_trail_airport.rb @@ -0,0 +1,5 @@ +# Cached AirTrail airport id for an airport code (remote_id nil = not found). +class AirTrailAirport < ApplicationRecord + belongs_to :connection, class_name: "AirTrailConnection", foreign_key: :air_trail_connection_id + normalizes :code, with: ->(c) { c.to_s.strip.upcase } +end diff --git a/app/models/air_trail_connection.rb b/app/models/air_trail_connection.rb new file mode 100644 index 0000000..5d2a075 --- /dev/null +++ b/app/models/air_trail_connection.rb @@ -0,0 +1,34 @@ +# A user's AirTrail account (https://github.com/johanohly/AirTrail). Flights +# the user logs are pushed there through AirTrail's REST API. +# +# The API key needs these scopes: profile.read, flight.read.own, +# flight.create.own, flight.update.own, flight.delete.own, +# reference_data.read and tracks.write. +class AirTrailConnection < ApplicationRecord + REQUIRED_SCOPES = %w[profile.read flight.read.own flight.create.own flight.update.own + flight.delete.own reference_data.read tracks.write].freeze + + belongs_to :user + has_many :syncs, class_name: "AirTrailSync", dependent: :delete_all + has_many :airports, class_name: "AirTrailAirport", dependent: :delete_all + + encrypts :api_key + + normalizes :base_url, with: ->(url) { url.strip.chomp("/") } + validates :base_url, presence: true, format: { with: %r{\Ahttps?://[^\s/]+(:\d+)?(/\S*)?\z}i, message: "must be a http(s) URL" } + validates :api_key, presence: true + + scope :active, -> { where(enabled: true) } + + def client = AirTrailClient.new(base_url: base_url, api_key: api_key) + + def ok? = status == "ok" + + def unresolved_airport_codes + airports.where(remote_id: nil).order(:code).pluck(:code) + end + + def counts + syncs.group(:status).count.reverse_merge("synced" => 0, "pending" => 0, "error" => 0) + end +end diff --git a/app/models/air_trail_sync.rb b/app/models/air_trail_sync.rb new file mode 100644 index 0000000..02019f3 --- /dev/null +++ b/app/models/air_trail_sync.rb @@ -0,0 +1,6 @@ +class AirTrailSync < ApplicationRecord + belongs_to :connection, class_name: "AirTrailConnection", foreign_key: :air_trail_connection_id + belongs_to :flight_log + + scope :failed, -> { where(status: "error") } +end diff --git a/app/models/aircraft.rb b/app/models/aircraft.rb index f973053..adba9ff 100644 --- a/app/models/aircraft.rb +++ b/app/models/aircraft.rb @@ -28,6 +28,8 @@ class Aircraft < ApplicationRecord validates :tail_number, presence: true validates :status, inclusion: { in: STATUSES.keys } + validates :icao_type, format: { with: /\A[A-Z0-9]{1,4}\z/, message: "must be an ICAO type designator like C150" }, allow_blank: true + normalizes :icao_type, with: ->(t) { t.strip.upcase.presence } validate :picture_is_image scope :ordered, -> { order(:tail_number) } diff --git a/app/models/airport.rb b/app/models/airport.rb new file mode 100644 index 0000000..68be95e --- /dev/null +++ b/app/models/airport.rb @@ -0,0 +1,64 @@ +# Airport reference data from OurAirports (the source AirTrail uses). +# Loaded by AirportData; refresh with `bin/rails airports:refresh`. +class Airport < ApplicationRecord + KINDS = %w[small_airport medium_airport large_airport seaplane_base heliport balloonport closed].freeze + KIND_RANK = { "large_airport" => 0, "medium_airport" => 1, "small_airport" => 2, "seaplane_base" => 3, + "heliport" => 4, "balloonport" => 5, "closed" => 9 }.freeze + KIND_LABELS = { "seaplane_base" => "seaplane base", "heliport" => "heliport", "balloonport" => "balloonport", "closed" => "closed" }.freeze + + # Custom airports (admin-added) use their identifier as the AirTrail code. + before_validation(if: :custom?) { self.airtrail_code = ident } + normalizes :ident, :gps_code, :local_code, :iata, with: ->(c) { c.strip.upcase.presence } + + validates :ident, presence: true, uniqueness: true, length: { maximum: 10 } + validates :name, presence: true + validates :kind, inclusion: { in: KINDS }, allow_blank: true + validates :latitude, numericality: { in: -90..90 }, allow_nil: true + validates :longitude, numericality: { in: -180..180 }, allow_nil: true + + scope :custom, -> { where(custom: true) } + + # The standard code for logging: AirTrail's (ICAO-style, e.g. K1B9 for FAA + # 1B9), so logged codes, the airport database and AirTrail all agree. + def code = airtrail_code + + # Other codes the airport goes by (FAA/local, GPS, IATA), e.g. ["1B9"]. + def other_codes = [ local_code, gps_code, iata ].compact.uniq - [ code ] + + def place = [ municipality, region.to_s.split("-", 2).last.presence ].compact.join(", ") + + def as_suggestion + { code: code, name: name, place: place, airtrail_code: airtrail_code, kind: KIND_LABELS[kind], aka: other_codes } + end + + # Best match for a code someone logged, whichever code system it's in. + def self.lookup(code) + c = code.to_s.strip.upcase + return nil if c.empty? + candidates = where(ident: c).or(where(gps_code: c)).or(where(local_code: c)).or(where(airtrail_code: c)).or(where(iata: c)).to_a + candidates.min_by do |a| + field = [ a.ident, a.gps_code, a.local_code, a.airtrail_code, a.iata ].index(c) || 9 + [ a.kind == "closed" ? 1 : 0, field, KIND_RANK.fetch(a.kind, 8) ] + end + end + + # The code AirTrail uses for the airport someone logged, or nil if unknown. + def self.airtrail_code_for(code) = lookup(code)&.airtrail_code + + # Autocomplete: exact codes, then code prefixes, then name/town matches. + # `prefer`: codes to rank first (e.g. airports this aircraft has used). + def self.search(query, limit: 10, prefer: []) + q = query.to_s.strip + return [] if q.length < 2 + up = q.upcase + like_prefix = "#{sanitize_sql_like(up)}%" + like_any = "%#{sanitize_sql_like(q)}%" + exact = where(ident: up).or(where(airtrail_code: up)).or(where(gps_code: up)).or(where(local_code: up)).or(where(iata: up)).to_a + prefix = up.match?(/\A[A-Z0-9]{1,4}\z/) ? where("gps_code LIKE :p OR local_code LIKE :p OR ident LIKE :p", p: like_prefix).limit(200).to_a : [] + named = where("name LIKE :q OR municipality LIKE :q", q: like_any).limit(400).to_a + preferred = prefer.to_set { _1.to_s.upcase } + familiar = ->(a) { [ a.code, a.ident, a.gps_code, a.local_code ].any? { preferred.include?(_1) } ? 0 : 1 } + rank = ->(a) { [ familiar.(a), a.kind == "closed" ? 1 : 0, KIND_RANK.fetch(a.kind, 8), a.code.length, a.code ] } + (exact.sort_by(&rank) + prefix.sort_by(&rank) + named.sort_by(&rank)).uniq(&:id).first(limit) + end +end diff --git a/app/models/airport_update.rb b/app/models/airport_update.rb new file mode 100644 index 0000000..46ab2d3 --- /dev/null +++ b/app/models/airport_update.rb @@ -0,0 +1,17 @@ +# One run of the OurAirports update (see AirportData / AirportUpdateJob). +class AirportUpdate < ApplicationRecord + belongs_to :requested_by, class_name: "User", optional: true + + scope :recent_first, -> { order(created_at: :desc) } + + def self.in_progress = where(status: %w[queued running]).where(created_at: 2.hours.ago..) + def self.last_success = where(status: "done").order(finished_at: :desc).first + + def failed? = status == "failed" + def done? = status == "done" + def in_progress? = status.in?(%w[queued running]) + + def summary + "#{created_count} added, #{updated_count} updated, #{removed_count} removed" + end +end diff --git a/app/models/flight_log.rb b/app/models/flight_log.rb index 556a691..05c11e5 100644 --- a/app/models/flight_log.rb +++ b/app/models/flight_log.rb @@ -4,6 +4,9 @@ class FlightLog < ApplicationRecord belongs_to :aircraft belongs_to :user, optional: true has_many :consumable_records, dependent: :nullify + # Capture AirTrail ids before the sync rows are deleted, so the remote flights can be removed too. + before_destroy :remember_air_trail_flights, prepend: true + has_many :air_trail_syncs, dependent: :delete_all has_one_attached :track_log validates :date, presence: true @@ -11,6 +14,9 @@ class FlightLog < ApplicationRecord validates :hobbs_time, numericality: { greater_than_or_equal_to: 0 }, allow_nil: true validates_uploads :track_log before_validation :derive_times + normalizes :departure_location, :destination_location, with: ->(code) { code.strip.upcase.presence } + after_commit :push_to_air_trail, on: %i[create update] + after_commit :delete_from_air_trail, on: :destroy scope :recent_first, -> { order(date: :desc, created_at: :desc) } @@ -20,6 +26,18 @@ class FlightLog < ApplicationRecord private + def push_to_air_trail + AirTrailPushJob.perform_later(id) if user_id && AirTrailConnection.active.exists?(user_id: user_id) + end + + def remember_air_trail_flights + @air_trail_remote = air_trail_syncs.where.not(remote_id: nil).pluck(:air_trail_connection_id, :remote_id) + end + + def delete_from_air_trail + Array(@air_trail_remote).each { |connection_id, remote_id| AirTrailDeleteJob.perform_later(connection_id, remote_id) } + end + # Let the pilot enter out/in readings instead of the delta. def derive_times self.tach_time = tach_in - tach_out if tach_time.blank? && tach_in && tach_out diff --git a/app/models/user.rb b/app/models/user.rb index f9ede5c..7e47d67 100644 --- a/app/models/user.rb +++ b/app/models/user.rb @@ -3,6 +3,7 @@ class User < ApplicationRecord has_many :sessions, dependent: :destroy has_many :aircraft_roles, dependent: :destroy has_many :aircraft, through: :aircraft_roles + has_one :air_trail_connection, dependent: :destroy normalizes :email_address, with: ->(e) { e.strip.downcase } diff --git a/app/services/air_trail_client.rb b/app/services/air_trail_client.rb new file mode 100644 index 0000000..846e528 --- /dev/null +++ b/app/services/air_trail_client.rb @@ -0,0 +1,97 @@ +require "net/http" + +# Minimal client for AirTrail's REST API (/api/v1, Bearer API key). +class AirTrailClient + class Error < StandardError + attr_reader :status, :code + + def initialize(message, status: nil, code: nil) + super(message) + @status = status + @code = code + end + end + class NetworkError < Error; end # retryable: timeouts, connection refused, 5xx + class AuthError < Error; end # 401/403: bad key or missing scope + class NotFound < Error; end + class ValidationError < Error; end # 400/422 + + def initialize(base_url:, api_key:, timeout: 20) + @base = URI.parse(base_url.to_s.chomp("/")) + @api_key = api_key + @timeout = timeout + end + + def me = request(:get, "/api/v1/me").fetch("data") + + def search_airports(query) = request(:get, "/api/v1/airports", query: { query: query }).fetch("data") + def search_aircraft(query) = request(:get, "/api/v1/aircraft", query: { query: query }).fetch("data") + + # Yields every flight the key's user owns (follows cursor pagination). + def each_flight + cursor = nil + loop do + q = { scope: "mine", limit: 100 } + q[:cursor] = cursor if cursor + page = request(:get, "/api/v1/flights", query: q) + page.fetch("data").each { yield _1 } + cursor = page.dig("page", "nextCursor") + break unless cursor + end + end + + def create_flight(body) = request(:post, "/api/v1/flights", body: body).dig("data", "id") + def update_flight(id, body) = request(:put, "/api/v1/flights/#{Integer(id)}", body: body) + + def delete_flight(id) + request(:delete, "/api/v1/flights/#{Integer(id)}") + rescue NotFound + nil # already gone + end + + private + + def request(method, path, query: nil, body: nil) + uri = @base.dup + uri.path = "#{@base.path}#{path}" + uri.query = URI.encode_www_form(query) if query + req = Net::HTTP.const_get(method.to_s.capitalize).new(uri) + req["Authorization"] = "Bearer #{@api_key}" + req["Accept"] = "application/json" + if body + req["Content-Type"] = "application/json" + req.body = JSON.generate(body) + end + + response = Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https", + open_timeout: @timeout, read_timeout: @timeout) { _1.request(req) } + handle(response) + rescue Timeout::Error, SocketError, SystemCallError, OpenSSL::SSL::SSLError, EOFError, IOError => e + raise NetworkError, "Can't reach AirTrail at #{@base.host}: #{e.message}" + end + + def handle(response) + status = response.code.to_i + json = begin + response.body.to_s.empty? ? {} : JSON.parse(response.body) + rescue JSON::ParserError + {} + end + return json if status.between?(200, 299) + + error = json["error"].is_a?(Hash) ? json["error"] : {} + message = [ error["message"] || "HTTP #{status}", details(error["details"]) ].compact.join(": ") + klass = case status + when 401, 403 then AuthError + when 404 then NotFound + when 400, 409, 422 then ValidationError + else NetworkError + end + raise klass.new("AirTrail: #{message}", status: status, code: error["code"]) + end + + def details(list) + return nil unless list.is_a?(Array) && list.any? + list.first(3).map { |d| [ Array(d["path"]).join("."), d["message"] ].compact_blank.join(" ") }.join("; ") + end +end diff --git a/app/services/air_trail_export.rb b/app/services/air_trail_export.rb new file mode 100644 index 0000000..0d00b45 --- /dev/null +++ b/app/services/air_trail_export.rb @@ -0,0 +1,96 @@ +# Builds an AirTrail import file (Settings → Import → AirTrail in AirTrail, +# https://github.com/johanohly/AirTrail) from an aircraft's flight log. +# +# Format follows AirTrail's own backup schema (src/lib/import/airtrail.ts): +# { flights: [...], users: [...] }, with airports referenced by code +# (AirTrail asks the user to map codes it doesn't know) and the pilot as the +# single passenger in the "pilot" seat. AirTrail matches the exported user to +# a local account by username. +# +# AirTrail de-duplicates imports on date + airports + flight number + +# registration + departure/arrival times. Our flights have no times, so +# repeat legs on the same day and route get a stable leg number ("#2", "#3") +# in the flight-number field; re-exporting and re-importing is then safe. +class AirTrailExport + USERNAME_FORMAT = /\A[A-Za-z0-9._-]{3,20}\z/ + TYPE_FORMAT = /\A[A-Z0-9]{1,4}\z/ + EXPORT_USER_ID = "sam-pilot".freeze + + attr_reader :errors + + def initialize(aircraft, username:, aircraft_type: nil, only_user: nil) + @aircraft = aircraft + @username = username.to_s.strip + @aircraft_type = aircraft_type.to_s.strip.upcase.presence || aircraft.icao_type + @only_user = only_user + @errors = [] + end + + def valid? + @errors = [] + @errors << "AirTrail username must be 3–20 letters, numbers, “.”, “_” or “-”" unless @username.match?(USERNAME_FORMAT) + @errors << "Aircraft type must be an ICAO type designator like C150 (up to 4 letters/numbers)" if @aircraft_type && !@aircraft_type.match?(TYPE_FORMAT) + @errors << "No flights to export" if exportable.empty? + @errors.empty? + end + + def flights + @flights ||= begin + scope = @aircraft.flight_logs.includes(track_log_attachment: :blob).order(:date, :created_at, :id) + scope = scope.where(user: @only_user) if @only_user + scope.to_a + end + end + + # Flights need both airports, as codes AirTrail can look up. + def exportable = @exportable ||= flights.select { AirTrailFlightData.exportable?(_1) } + def skipped = flights.size - exportable.size + + def filename = "#{@aircraft.tail_number.parameterize}-airtrail-#{Date.current}.json" + + def to_json(*) = JSON.pretty_generate(as_json) + + def as_json(*) + { + flights: exportable.zip(leg_numbers).map { |f, leg| flight_json(f, leg) }, + users: [ { id: EXPORT_USER_ID, username: @username, displayName: @username } ] + } + end + + private + + def airport(code) = AirTrailFlightData.airport_code(code) + + def leg_numbers + legs = AirTrailFlightData.leg_numbers(exportable) + exportable.map { legs[_1.id] } + end + + def flight_json(f, leg) + json = { + date: f.date.iso8601, + datePrecision: "day", + from: { code: airport(f.departure_location) }, + to: { code: airport(f.destination_location) }, + departure: nil, + arrival: nil, + duration: AirTrailFlightData.duration(f), # seconds + airline: nil, + aircraft: @aircraft_type && { name: aircraft_name, icao: @aircraft_type }, + flightNumber: leg, + aircraftReg: AirTrailFlightData.registration(@aircraft), + note: AirTrailFlightData.note(f), + passengers: [ + { userId: EXPORT_USER_ID, guestName: nil, seat: "pilot", seatNumber: nil, + seatClass: nil, flightReason: nil, customFields: {} } + ] + } + track = AirTrailFlightData.track(f) + json[:track] = track if track + json + end + + def aircraft_name + [ @aircraft.make, @aircraft.model ].compact_blank.join(" ").presence || @aircraft_type + end +end diff --git a/app/services/air_trail_flight_data.rb b/app/services/air_trail_flight_data.rb new file mode 100644 index 0000000..f3e0a3b --- /dev/null +++ b/app/services/air_trail_flight_data.rb @@ -0,0 +1,61 @@ +# Flight details shared by the AirTrail file export and the API sync, so a +# flight looks the same in AirTrail whichever way it got there (and the sync +# can recognize flights that came in through a file import). +module AirTrailFlightData + AIRPORT_FORMAT = /\A[A-Z0-9]{1,4}\z/ + + module_function + + # The code AirTrail uses for a logged airport, or nil if AirTrail can't take + # it. Logged codes are translated through the airport database (the same + # OurAirports data AirTrail uses), e.g. 1B9 → K1B9, 4MB → K5M3. + def airport_code(code) + c = code.to_s.strip.upcase + return nil if c.empty? + translated = Airport.airtrail_code_for(c) || c + translated if translated.match?(AIRPORT_FORMAT) + end + + def exportable?(flight) + airport_code(flight.departure_location) && airport_code(flight.destination_location) + end + + # Seconds, from Hobbs time (or tach); nil for zero-time entries. + def duration(flight) + hours = (flight.hobbs_time || flight.tach_time).to_f + hours.positive? ? (hours * 3600).round : nil + end + + def note(flight) + times = [ "Tach #{flight.tach_time} h", (flight.hobbs_time && "Hobbs #{flight.hobbs_time} h") ].compact.join(" · ") + [ times, flight.route.presence, flight.notes.presence ].compact.join("\n").truncate(1000) + end + + def registration(aircraft) = aircraft.tail_number.to_s.upcase.first(10).presence + + # AirTrail de-duplicates on date + airports + flight number (+ reg, times). + # Repeat legs on the same day and route get "#2", "#3"… in log order. + def leg_numbers(flights) + seen = Hash.new(0) + flights.to_h do |f| + key = [ f.aircraft_id, f.date, airport_code(f.departure_location), airport_code(f.destination_location) ] + seen[key] += 1 + [ f.id, (seen[key] > 1 ? "##{seen[key]}" : nil) ] + end + end + + def leg_number(flight) + same_route = flight.aircraft.flight_logs.where(date: flight.date).order(:created_at, :id).select do |f| + airport_code(f.departure_location) == airport_code(flight.departure_location) && + airport_code(f.destination_location) == airport_code(flight.destination_location) + end + leg_numbers(same_route)[flight.id] + end + + # AirTrail TrackInput from the flight's track log, or nil. + def track(flight) + TrackParser.parse_attachment(flight.track_log)&.as_air_trail_json + rescue TrackParser::Error, ActiveStorage::FileNotFoundError + nil + end +end diff --git a/app/services/air_trail_syncer.rb b/app/services/air_trail_syncer.rb new file mode 100644 index 0000000..f3fade4 --- /dev/null +++ b/app/services/air_trail_syncer.rb @@ -0,0 +1,177 @@ +# Pushes a user's flights to their AirTrail account through the REST API. +# +# - push(flight): create the AirTrail flight, or replace it (PUT) if it +# changed since the last sync. Skips unchanged flights (payload digest). +# - backfill: link flights that already exist in AirTrail (e.g. from an +# earlier file import) instead of duplicating them, then push the rest. +# - Airports are looked up by code once per connection and cached; codes +# AirTrail doesn't know are recorded so the user can map them by hand. +class AirTrailSyncer + class UnresolvedAirport < StandardError; end + + AIRPORT_RECHECK = 1.day + + def initialize(connection, client: connection.client) + @connection = connection + @client = client + @aircraft_ids = {} + end + + # Flights this connection mirrors: the ones its user logged. + def flights = FlightLog.where(user_id: @connection.user_id) + + def ensure_identity! + return if @connection.remote_user_id.present? + me = @client.me + @connection.update!(remote_user_id: me["id"], remote_username: me["username"], status: "ok", + last_error: nil, checked_at: Time.current) + end + + def push(flight, leg: :lookup) + ensure_identity! + sync = @connection.syncs.find_or_initialize_by(flight_log: flight) + body = payload(flight, leg: leg == :lookup ? AirTrailFlightData.leg_number(flight) : leg) + digest = Digest::SHA256.hexdigest(JSON.generate(body)) + return sync if sync.remote_id && sync.status == "synced" && sync.payload_digest == digest + + if sync.remote_id + begin + @client.update_flight(sync.remote_id, body) + rescue AirTrailClient::NotFound # deleted in AirTrail: recreate + sync.remote_id = @client.create_flight(body) + end + else + sync.remote_id = @client.create_flight(body) + end + sync.update!(status: "synced", payload_digest: digest, synced_at: Time.current, last_error: nil) + sync + rescue UnresolvedAirport, AirTrailClient::ValidationError => e + sync&.update!(status: "error", last_error: e.message) + sync + rescue AirTrailClient::AuthError => e + @connection.update!(status: "error", last_error: e.message, checked_at: Time.current) + sync&.update!(status: "error", last_error: e.message) + raise + end + + def delete_remote(remote_id) = @client.delete_flight(remote_id) + + # Returns { linked:, pushed:, failed: }. + def backfill + ensure_identity! + local = flights.includes(:aircraft).order(:date, :created_at, :id).to_a + legs = AirTrailFlightData.leg_numbers(local) + linked = link_existing(local, legs) + pushed = 0 + failed = 0 + local.each do |f| + next unless AirTrailFlightData.exportable?(f) + sync = push(f, leg: legs[f.id]) + sync.status == "synced" ? pushed += 1 : failed += 1 + end + @connection.update!(backfilled_at: Time.current, status: "ok", last_error: nil) + { linked: linked, pushed: pushed, failed: failed } + end + + def payload(flight, leg:) + from = airport_id(flight.departure_location) + to = airport_id(flight.destination_location) + body = { + date: flight.date.iso8601, + datePrecision: "day", + departure: nil, + arrival: nil, + duration: AirTrailFlightData.duration(flight), + flightNumber: leg, + aircraftReg: AirTrailFlightData.registration(flight.aircraft), + note: AirTrailFlightData.note(flight), + fromId: from, + toId: to, + aircraftId: aircraft_id(flight.aircraft.icao_type), + airlineId: nil, + passengers: [ { userId: @connection.remote_user_id, guestName: nil, seat: "pilot", seatNumber: nil, + seatClass: nil, flightReason: nil, customFields: {} } ], + customFields: {} + } + track = AirTrailFlightData.track(flight) + body[:track] = track if track + body + end + + # Look an airport code up in AirTrail (cached per connection). + def airport_id(code) + normalized = AirTrailFlightData.airport_code(code) + raise UnresolvedAirport, "Airport “#{code.presence || '(blank)'}” isn't a code AirTrail can use" unless normalized + + cached = @connection.airports.find_or_initialize_by(code: normalized) + fresh = cached.manual? || cached.remote_id || (cached.looked_up_at && cached.looked_up_at > AIRPORT_RECHECK.ago) + unless fresh + results = @client.search_airports(normalized) + match = results.find { _1["icao"].to_s.casecmp?(normalized) } || results.find { _1["iata"].to_s.casecmp?(normalized) } + cached.update!(remote_id: match&.dig("id"), remote_icao: match&.dig("icao"), looked_up_at: Time.current) + end + cached.remote_id or raise UnresolvedAirport, "AirTrail doesn't know airport #{normalized}. Map it on your AirTrail settings page" + end + + # Map a code to an airport the user names (e.g. its ICAO/GPS code in AirTrail). + def map_airport(code, airtrail_code) + results = @client.search_airports(airtrail_code) + match = results.find { _1["icao"].to_s.casecmp?(airtrail_code) || _1["iata"].to_s.casecmp?(airtrail_code) } || results.first + return nil unless match + @connection.airports.find_or_initialize_by(code: code) + .update!(remote_id: match["id"], remote_icao: match["icao"], manual: true, looked_up_at: Time.current) + match + end + + private + + def aircraft_id(icao_type) + return nil if icao_type.blank? + @aircraft_ids.fetch(icao_type) do + @aircraft_ids[icao_type] = @client.search_aircraft(icao_type).find { _1["icao"].to_s.casecmp?(icao_type) }&.dig("id") + end + end + + # Match local flights to AirTrail flights already there (same signature + # AirTrail de-duplicates on), falling back to date + registration + leg + + # duration when the airports were mapped differently. Learns airport ids + # from matched flights so hand-mapped airports carry over. + def link_existing(local, legs) + already = @connection.syncs.where.not(remote_id: nil).pluck(:flight_log_id).to_set + remote = [] + @client.each_flight { remote << _1 } + taken = @connection.syncs.where.not(remote_id: nil).pluck(:remote_id).to_set + remote.reject! { taken.include?(_1["id"]) } + + exact = remote.group_by { |r| [ r["date"], r.dig("from", "icao").to_s.upcase, r.dig("to", "icao").to_s.upcase, r["flightNumber"].to_s, r["aircraftReg"].to_s.upcase ] } + loose = remote.group_by { |r| [ r["date"], r["aircraftReg"].to_s.upcase, r["flightNumber"].to_s, r["duration"] ] } + used = Set.new + linked = 0 + + local.each do |f| + next if already.include?(f.id) || !AirTrailFlightData.exportable?(f) + reg = AirTrailFlightData.registration(f.aircraft).to_s + leg = legs[f.id].to_s + candidates = exact[[ f.date.iso8601, AirTrailFlightData.airport_code(f.departure_location), + AirTrailFlightData.airport_code(f.destination_location), leg, reg ]] || [] + candidates += loose[[ f.date.iso8601, reg, leg, AirTrailFlightData.duration(f) ]] || [] + match = candidates.find { !used.include?(_1["id"]) } + next unless match + + used << match["id"] + learn_airport(f.departure_location, match["from"]) + learn_airport(f.destination_location, match["to"]) + @connection.syncs.find_or_initialize_by(flight_log: f).update!(remote_id: match["id"], status: "pending") + linked += 1 + end + linked + end + + def learn_airport(code, remote_airport) + normalized = AirTrailFlightData.airport_code(code) + return unless normalized && remote_airport.is_a?(Hash) && remote_airport["id"] + cached = @connection.airports.find_or_initialize_by(code: normalized) + return if cached.manual? || cached.remote_id + cached.update!(remote_id: remote_airport["id"], remote_icao: remote_airport["icao"], looked_up_at: Time.current) + end +end diff --git a/app/services/airport_code_standardizer.rb b/app/services/airport_code_standardizer.rb new file mode 100644 index 0000000..73fdcd8 --- /dev/null +++ b/app/services/airport_code_standardizer.rb @@ -0,0 +1,89 @@ +# Rewrites an aircraft's logged airport codes to the airport database's +# standard code (AirTrail's ICAO-style code, e.g. 1B9 → K1B9). +# +# s = AirportCodeStandardizer.new(aircraft) +# s.changes # => [Change(from: "1B9", to: "K1B9", airport:, flights: 12, suggested: true), ...] +# s.unknown # => [Unknown(code: "0R8", flights: 2, suggestion: <08R Richmond Airport>)] +# s.apply!(%w[1B9 44N], fixes: { "0R8" => "08R" }) # returns entries updated +class AirportCodeStandardizer + Change = Struct.new(:from, :to, :airport, :flights, :suggested, :note, keyword_init: true) + # A code not in the database, with a likely intended airport (e.g. a typo). + Unknown = Struct.new(:code, :flights, :suggestion, keyword_init: true) + + def initialize(aircraft) + @aircraft = aircraft + end + + def changes + analyze[:changes] + end + + def unknown + analyze[:unknown] + end + + # codes: logged codes to standardize; fixes: { unknown code => airport code } + # for codes not in the database (resolved to that airport's standard code). + def apply!(codes, fixes: {}) + chosen = changes.select { codes.include?(_1.from) } + known_unknown = unknown.map(&:code) + fixes.each do |from, target| + from = from.to_s.strip.upcase + airport = Airport.lookup(target) + next unless airport && known_unknown.include?(from) + chosen << Change.new(from: from, to: airport.code, airport: airport) + end + updated = 0 + FlightLog.transaction do + chosen.each do |c| + updated += @aircraft.flight_logs.where(departure_location: c.from).update_all(departure_location: c.to, updated_at: Time.current) + updated += @aircraft.flight_logs.where(destination_location: c.from).update_all(destination_location: c.to, updated_at: Time.current) + end + if chosen.any? + @aircraft.log_event!("flight", "Standardized #{chosen.size} airport codes", + notes: chosen.map { "#{_1.from}→#{_1.to}" }.join(", ")) + end + end + updated + end + + private + + def analyze + @analyze ||= begin + counts = Hash.new(0) + @aircraft.flight_logs.pluck(:departure_location, :destination_location).flatten.each do |code| + counts[code.to_s.strip.upcase] += 1 if code.present? + end + changes = [] + unknown = [] + counts.sort.each do |code, n| + airport = Airport.lookup(code) + if airport.nil? + unknown << Unknown.new(code: code, flights: n, suggestion: suggest(code, counts.keys)) + elsif airport.code != code + renamed = !airport.code.end_with?(code) # e.g. 4MB → K5M3, not just a K prefix + note = if airport.kind == "closed" then "Closed airport. Check it's the right one" + elsif renamed then "Different identifier, not just a prefix. Check it's the right field" + end + changes << Change.new(from: code, to: airport.code, airport: airport, flights: n, + suggested: airport.kind != "closed", note: note) + end + end + { changes: changes, unknown: unknown } + end + end + + # Likely intended airport for an unknown code: the same characters in a + # different order (e.g. 0R8 → 08R), preferring codes this aircraft has used. + def suggest(code, logged_codes) + return nil unless code.length.between?(3, 4) + candidates = code.chars.permutation.map(&:join).uniq - [ code ] + familiar = candidates & logged_codes + (familiar + (candidates - familiar)).each do |candidate| + airport = Airport.lookup(candidate) + return airport if airport && airport.kind != "closed" + end + nil + end +end diff --git a/app/services/track_parser.rb b/app/services/track_parser.rb new file mode 100644 index 0000000..7c92b57 --- /dev/null +++ b/app/services/track_parser.rb @@ -0,0 +1,169 @@ +require "csv" + +# Parses a flight's GPS track log (GPX, KML or CSV) into AirTrail's track +# format: coordinates as [lon, lat, altitude_m] plus aligned epoch-second +# times. Mirrors AirTrail's own parser (src/lib/track/parser.ts): +# GPX: / lat/lon attributes, in meters,