examples/jupyter_demo.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f211c033-f1e2-42d8-b2e8-8e262b4b73ac",
   "metadata": {
    "tags": []
   },
   "source": [
    "# MagNav.jl Demonstration Notebook\n",
    "This notebook provides a demonstration of some of the key functions provided in MagNav.jl: https://github.com/MIT-AI-Accelerator/MagNav.jl\n",
    "\n",
    "Feel free to change any parameters of interest."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59aa9fa0",
   "metadata": {},
   "source": [
    "## Import packages and DataFrames\n",
    "\n",
    "The DataFrames listed below provide useful information about the flight data collected by Sander Geophysics Ltd. (SGL) and magnetic anomaly maps.\n",
    "\n",
    "Dataframe  | Description\n",
    ":--------- | :----------\n",
    "`df_map`   | map files relevant for SGL flights\n",
    "`df_cal`   | SGL calibration flight lines\n",
    "`df_flight`| SGL flight files\n",
    "`df_all`   | all flight lines\n",
    "`df_nav`   | all *navigation-capable* flight lines\n",
    "`df_event` | pilot-recorded in-flight events"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6c7b3f87-825b-4dc1-bc6c-cbddb61eb2f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "cd(@__DIR__)\n",
    "# uncomment line below to use local MagNav.jl (downloaded folder)\n",
    "# using Pkg; Pkg.activate(\"../\"); Pkg.instantiate()\n",
    "using MagNav\n",
    "using CSV, DataFrames\n",
    "using Plots: plot, plot!\n",
    "using Random: seed!\n",
    "using Statistics: mean, median, std\n",
    "seed!(33); # for reproducibility\n",
    "include(\"dataframes_setup.jl\"); # setup DataFrames\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "95ea6673-0257-4404-a4e8-6e56a80bce82",
   "metadata": {},
   "source": [
    "## Flight line selection\n",
    "\n",
    "Select Flight 1006 (see [readme](https://github.com/MIT-AI-Accelerator/MagNav.jl/blob/master/readmes/Flt1006_readme.txt)) and gather the [`XYZ20` data structure](https://mit-ai-accelerator.github.io/MagNav.jl/stable/structs/#MagNav.XYZ20), which contains the GPS-based trajectory [`Traj` data structure](https://mit-ai-accelerator.github.io/MagNav.jl/stable/structs/#MagNav.Traj), inertial navigation system [`INS` data structure](https://mit-ai-accelerator.github.io/MagNav.jl/stable/structs/#MagNav.INS), flight information, magnetometer readings, and auxilliary sensor data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc5a3f2a-cfb6-4f9d-a916-71b0a1266543",
   "metadata": {},
   "outputs": [],
   "source": [
    "flight = :Flt1006 # select flight, full list in df_flight\n",
    "xyz    = get_XYZ(flight,df_flight); # load flight data\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1290da99",
   "metadata": {},
   "source": [
    "The `xyz` flight data struct is of type `MagNav.XYZ20` (for the 2020 SGL flight data collection), which is a subtype of `MagNav.XYZ` (the abstract type for any flight data in MagNav.jl). There are 76 fields, which can be accessed using dot notation. Note that `xyz` holds all the flight data from the HDF5 file, but Boolean indices can be used as a mask to return specific portion(s) of flight data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5791811d",
   "metadata": {},
   "outputs": [],
   "source": [
    "typeof(xyz)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "715fd82b",
   "metadata": {},
   "outputs": [],
   "source": [
    "fieldnames(MagNav.XYZ20)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "388cca61-380e-4db7-8b54-ba1ea5c0474a",
   "metadata": {},
   "source": [
    "Here a map is selected, then `df_nav` is filtered into `df_options` to ensure that the selected flight line(s) both corresponds with the selected flight (`:Flt1006`) and exists on the map (`:Eastern_395`). The full list of SGL flights is in `df_flight`, the full list of maps is in `df_map`, and the full list of navigation-capable flight lines is in `df_nav`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88b58bee-2ca4-4967-a881-67b481efefc3",
   "metadata": {},
   "outputs": [],
   "source": [
    "map_name   = :Eastern_395 # select map, full list in df_map\n",
    "df_options = df_nav[(df_nav.flight   .== flight  ) .&\n",
    "                    (df_nav.map_name .== map_name),:]\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "edaf7a58-9d3a-4f0a-813e-384a201ad0b4",
   "metadata": {},
   "source": [
    "## Get data for selected flight line\n",
    "\n",
    "To test navigation, we use Boolean indices (mask) corresponding to flight line 1006.08 in `df_options`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10aae222-a331-4aea-8940-a2882a582bf8",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "line = 1006.08 # select flight line (row) from df_options\n",
    "ind  = get_ind(xyz,line,df_options); # get Boolean indices\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "741742c1-007c-43cc-abde-1860ce4c5d4c",
   "metadata": {},
   "source": [
    "For the Tolles-Lawson calibration, flight line 1006.04 is selected, which occurred at a higher altitude (see [readme](https://github.com/MIT-AI-Accelerator/MagNav.jl/blob/master/readmes/Flt1006_readme.txt)). This is the first calibration box of this flight line. `TL_ind` holds the Boolean indices (mask) just for this portion of the calibration flight line. The full list of calibration flight line options is in `df_cal`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "87ad787d-8cec-41cd-aecc-8938c40fe139",
   "metadata": {},
   "outputs": [],
   "source": [
    "TL_i   = 6 # select first calibration box of 1006.04\n",
    "TL_ind = get_ind(xyz;tt_lim=[df_cal.t_start[TL_i],df_cal.t_end[TL_i]]);\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d022f7da-dcbe-4daa-9a6a-3f2423eb675d",
   "metadata": {},
   "source": [
    "## Plotting\n",
    "\n",
    "To get an idea of the magnetometer data, we can call some utility functions for plotting.\n",
    "\n",
    "Note that these are filtered using the `ind` Boolean indices corresponding to the held-out flight `line`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26421d0a-b8a3-40d5-83d3-b2e8623aa3df",
   "metadata": {},
   "outputs": [],
   "source": [
    "show_plot = true\n",
    "save_plot = false\n",
    "use_mags  = [:mag_1_uc,:mag_4_uc,:mag_5_uc] # scalar magnetometers to plot\n",
    "\n",
    "p1 = plot_mag(xyz;ind,show_plot,save_plot, # plot scalar magnetometers\n",
    "              use_mags     = use_mags,\n",
    "              detrend_data = true);\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3d3b61c-bb13-4b85-be65-f5ebbd03e3f4",
   "metadata": {},
   "outputs": [],
   "source": [
    "p2 = plot_mag(xyz;ind,show_plot,save_plot, # plot vector magnetometer (fluxgate)\n",
    "              use_mags     = [:flux_d], # try changing to :flux_a, :flux_b, :flux_c\n",
    "              detrend_data = true);\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "727f1fad-6634-4fc1-ba58-00b97ec3a46f",
   "metadata": {},
   "source": [
    "Clearly, the in-cabin scalar and vector magnetometers are noisy compared to the stinger magnetometer (Mag 1).\n",
    "\n",
    "## Create the (linear) Tolles-Lawson model\n",
    "\n",
    "Next, we select scalar and vector magnetometer readings during the calibration flight and generate the coefficients to perform linear Tolles-Lawson compensation. We are choosing in-cabin scalar magnetometer 4 and vector (flux) magnetometer D. Mag 4 is located on the floor in the rear of the cabin, and Flux D is nearby on the starboard side. Mag 4 is particularly challenging since it contains several 100s to 1000 nT excursions in comparison to the tail stinger.\n",
    "\n",
    "### Tolles-Lawson calibration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44198a4c-53c2-47fb-a0da-fbe4498f7eec",
   "metadata": {},
   "outputs": [],
   "source": [
    "λ       = 0.025   # ridge parameter for ridge regression\n",
    "use_vec = :flux_d # selected vector (flux) magnetometer\n",
    "flux    = getfield(xyz,use_vec) # load Flux D data\n",
    "TL_d_4  = create_TL_coef(flux,xyz.mag_4_uc,TL_ind;λ=λ); # create Tolles-Lawson coefficients with Flux D & Mag 4\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e27e818c-a339-4da8-96fa-40374d086924",
   "metadata": {},
   "source": [
    "### Tolles-Lawson compensation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20dae634-db16-4ef4-b006-ee97b614fa98",
   "metadata": {},
   "outputs": [],
   "source": [
    "A = create_TL_A(flux,ind)     # Tolles-Lawson `A` matrix for Flux D\n",
    "mag_1_sgl = xyz.mag_1_c[ind]  # professionally compensated tail stinger, Mag 1\n",
    "mag_4_uc  = xyz.mag_4_uc[ind] # uncompensated Mag 4\n",
    "mag_4_c   = mag_4_uc - detrend(A*TL_d_4;mean_only=true); # compensated Mag 4\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "44b5bbc1-d3c8-4d8c-847a-5f093621c0d3",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Navigation\n",
    "\n",
    "Here, several steps are done to prepare the flight and map data for the navigation filter. The trajectory (`Traj`) and `INS` data structures are pulled out of the `XYZ20` data structure for convenience, and map data is loaded into a map (`MapS`) data structure. The map is then \"upward continued\" (via a Fast Fourier Transform) to the trajectory altitude and outfitted with an interpolation function (`itp_mapS`). Finally, the expected scalar magnetometer reading along the flight path (`map_val`) is computed using the interpolation function, as done in `get_map_val()`, then corrected for diurnal effects and the core magnetic field ([IGRF](https://www.ncei.noaa.gov/products/international-geomagnetic-reference-field))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "90441078-a258-4530-a871-a5edb87487d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "traj = get_traj(xyz,ind) # trajectory (GPS) struct\n",
    "ins  = get_ins( xyz,ind;N_zero_ll=1) # INS struct, \"zero\" lat/lon to match first `traj` data point\n",
    "mapS = get_map(map_name,df_map) # load map data\n",
    "# get map values & map interpolation function\n",
    "(map_val,itp_mapS) = get_map_val(mapS,traj;return_itp=true)\n",
    "map_val += (xyz.diurnal + xyz.igrf)[ind] # add in diurnal & core (IGRF)\n",
    "println(\"Error for scalar Mag 4: \",round(std(map_val-mag_4_c),digits=2),\" nT\")\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c3f1a29e-7cb8-4886-a22d-b8c58fe7335b",
   "metadata": {},
   "source": [
    "### Create the filter model\n",
    "\n",
    "Next, we do a rough analysis to estimate appropriate filter parameters, and then we initialize a filter model. Specifically, the autocorrelation between the measured (`mag_4_c`) and true (`map_val`) magnetic values is evaluated. This provides information for setting the magnitudes of the measurement variance `meas_var` and first-order Gauss-Markov (FOGM) standard deviation (`fogm_sigma`) and time constant (`fogm_tau`). These are all approximate, and may not be consistently appropriate for the entire flight line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c897e5f4-4628-49ac-953d-c4b41b67aa00",
   "metadata": {},
   "outputs": [],
   "source": [
    "(sigma,tau) = get_autocor(mag_4_c - map_val)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69770670-a9e7-4e4a-97c1-4d9172443b88",
   "metadata": {},
   "outputs": [],
   "source": [
    "(P0,Qd,R) = create_model(traj.dt,traj.lat[1];\n",
    "                         init_pos_sigma = 0.1,\n",
    "                         init_alt_sigma = 1.0,\n",
    "                         init_vel_sigma = 1.0,\n",
    "                         meas_var       = 134^2, # increase for bad mag\n",
    "                         fogm_sigma     = 134,\n",
    "                         fogm_tau       = 10.0);\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a8780636-6c8e-4e26-bb23-a809ab8fa10a",
   "metadata": {},
   "source": [
    "### Run the filter\n",
    "\n",
    "Here, we use an extended Kalman filter (`:ekf`) and compute:\n",
    "- `crlb_out`: Cramér–Rao lower bound error\n",
    "- `ins_out`: error when using the INS alone (dead reckoning)\n",
    "- `filt_out`: error when using the INS + magnetic measurements (MagNav)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f07e4f3-e048-469f-9c27-6f4921b7b196",
   "metadata": {},
   "outputs": [],
   "source": [
    "mag_use = mag_4_c # select Mag 4 for navigation\n",
    "mag_use .+= map_val[1] - mag_use[1] # remove initial DC offset\n",
    "(crlb_out,ins_out,filt_out) = run_filt(traj,ins,mag_use,itp_mapS,:ekf;P0,Qd,R,core=true); # run the filter\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4345e92d-3781-4ceb-b4c0-2e248ad35050",
   "metadata": {},
   "source": [
    "### Plot the results\n",
    "\n",
    "Despite the 149 m distance-root-mean-square (DRMS) error, the navigation does not go too far awry for the linear model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5de61de6-478e-458b-828e-a767c9f3bffc",
   "metadata": {},
   "outputs": [],
   "source": [
    "p3 = plot_map(mapS;legend=false); # plot map background\n",
    "plot_filt!(p3,traj,ins,filt_out;show_plot=false); # plot GPS, INS (after zeroing), and filter\n",
    "plot!(p3,legend=:topleft) # move legend as desired\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "52f7a8f4-a441-4b2f-9572-bf884ffc3cca",
   "metadata": {},
   "source": [
    "### Corrupting fields\n",
    "\n",
    "In the below plot, there are some spiky excursions in the TL-compensated magnetometer that do not occur on the map. We can easily compare the two using the `plot_mag_map` convenience function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a75ac132-6469-446d-8061-823b3eefe75b",
   "metadata": {},
   "outputs": [],
   "source": [
    "p4 = plot_mag_map(traj,mag_use,itp_mapS) # plot magnetometer vs map\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b71fe7a4-3f3b-4d9e-a7fe-44e02cdfa864",
   "metadata": {},
   "source": [
    "However, current sensors for the 💡 strobe lights 💡 picked some of these up. The current sensors have severe high-frequency noise, so we first low-pass filter them using two convenience functions, `get_bpf` and `bpf_data`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2ec90a60-b97b-45f7-821d-ac732909ee02",
   "metadata": {},
   "outputs": [],
   "source": [
    "lpf     = get_bpf(;pass1=0.0,pass2=0.2,fs=10.0) # get low-pass filter\n",
    "lpf_sig = -bpf_data(xyz.cur_strb[ind];bpf=lpf)  # apply low-pass filter, sign switched for easier comparison\n",
    "p5      = plot_basic(xyz.traj.tt[ind],lpf_sig;lab=\"filtered current for strobe lights\"); # plot the low-pass filtered strobe light current sensor\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "e9d684ef-6080-444c-98ec-5accb97f446a",
   "metadata": {},
   "source": [
    "## Training a physics-based neural network architecture\n",
    "\n",
    "Now we attempt to improve on the prior results by training a neural network that incorporates the Tolles-Lawson (TL) model together with an artificial neural network (NN). Specifically, the NN is provided with cabin current sensor information that produces an additive correction to the TL model. Both the NN parameters and the TL coefficients are trainable, which we designate as model 2c (model 2b, by comparison, keeps the TL coefficients static).\n",
    "\n",
    "First, we select all available flight data from Flights 1003-1006 (see [readmes](https://github.com/MIT-AI-Accelerator/MagNav.jl/tree/master/readmes)) into `lines_train`, except the held-out flight `line`. The full list of flight lines is in `df_all`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b07fce32-3515-41a7-a8b3-f559f27cdddb",
   "metadata": {},
   "outputs": [],
   "source": [
    "flts = [:Flt1003,:Flt1004,:Flt1005,:Flt1006] # select flights for training\n",
    "df_train = df_all[(df_all.flight .∈ (flts,) ) .& # use all flight data\n",
    "                  (df_all.line   .!= line),:]    # except held-out line\n",
    "lines_train = df_train.line # training lines\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "0ce00e24-884c-42a4-bc25-8fba001b4206",
   "metadata": {},
   "source": [
    "Here, we add current sensors to the usual inputs, making them available to the neural network. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "539e597e-2593-46a2-bbbb-aadce4977358",
   "metadata": {},
   "outputs": [],
   "source": [
    "features = [:mag_4_uc, :lpf_cur_com_1, :lpf_cur_strb, :lpf_cur_outpwr, :lpf_cur_ac_lo];\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "58f24f74-c7c1-4b03-a31b-874e96d41284",
   "metadata": {},
   "source": [
    "The neural network-based compensation parameters (type of `NNCompParams`) are provided to (and returned by) the training function. These take default values unless they are specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5119f435-4caf-4861-9f4d-c683fe2f29ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "comp_params = NNCompParams(features_setup = features,\n",
    "                           model_type     = :m2c,\n",
    "                           y_type         = :d,\n",
    "                           use_mag        = :mag_4_uc,\n",
    "                           use_vec        = :flux_d,\n",
    "                           terms          = [:permanent,:induced,:fdm],\n",
    "                           terms_A        = [:permanent,:induced,:eddy],\n",
    "                           sub_diurnal    = true,\n",
    "                           sub_igrf       = true,\n",
    "                           bpf_mag        = false,\n",
    "                           norm_type_A    = :none,\n",
    "                           norm_type_x    = :standardize,\n",
    "                           norm_type_y    = :standardize,\n",
    "                           TL_coef        = TL_d_4,\n",
    "                           η_adam         = 0.001,\n",
    "                           epoch_adam     = 300,\n",
    "                           epoch_lbfgs    = 0,\n",
    "                           hidden         = [8,4]);\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eaaad14b-1c94-4eda-a2ce-4f1b02f321f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "(comp_params,y_train,y_train_hat,err_train,feats) =\n",
    "    comp_train(comp_params,lines_train,df_all,df_flight,df_map);\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7af45ad2-f450-4932-bf45-45f3406ed153",
   "metadata": {},
   "source": [
    "After 300 epochs, model 2c has 37 nT of error on the training data. We next test the performance on the held-out flight `line` using the `comp_test` convenience function. Note that there is also a `comp_train_test` convenience function that does both."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a84d932-13be-4ee1-9070-a4563a0e2407",
   "metadata": {},
   "outputs": [],
   "source": [
    "(_,y_hat,_) =\n",
    "    comp_test(comp_params,[line],df_all,df_flight,df_map);\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "d98d0e67-7642-4fd1-aeef-3e6f1370dfca",
   "metadata": {},
   "source": [
    "We are now in a position to compare the magnetometer readings to the expected map values. The model 2c results ameliorate the signal excursions that are present in the uncompensated and TL-compensated readings. \n",
    "\n",
    "Note that the `detrend` function helps remove any persistent bias in the signal, which does not affect the navigation error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d256014e-8844-474c-90eb-c7c702db6adf",
   "metadata": {},
   "outputs": [],
   "source": [
    "tt = (xyz.traj.tt[ind] .- xyz.traj.tt[ind][1]) / 60;\n",
    "p6 = plot(xlab=\"time [min]\", ylab=\"magnetic field [nT]\");\n",
    "plot!(p6, tt, detrend(map_val  - (xyz.diurnal + xyz.igrf)[ind], mean_only=true), lab=\"anomaly map\");\n",
    "plot!(p6, tt, detrend(mag_4_uc - (xyz.diurnal + xyz.igrf)[ind], mean_only=true), lab=\"uncompensated\");\n",
    "plot!(p6, tt, detrend(mag_4_c  - (xyz.diurnal + xyz.igrf)[ind], mean_only=true), lab=\"Tolles-Lawson\");\n",
    "plot!(p6, tt, detrend(mag_4_uc - y_hat - (xyz.diurnal + xyz.igrf)[ind], mean_only=true), lab=\"model 2c\")\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "a2a84569-7781-4fe6-a054-3800ff9ac601",
   "metadata": {},
   "source": [
    "Next, we use the NN output to perform the compensation. Since we selected `y_type = :d` in the NN compensation parameters (`NNCompParams`), we treat the output as the platform component that must be subtracted from the total scalar signal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ef977fd3-f61c-44cb-ab3f-692effc906fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "mag_use = mag_4_uc - y_hat # compensate Mag 4 using NN output\n",
    "mag_use .+= map_val[1] - mag_use[1] # remove initial DC offset\n",
    "println(\"TL + NN σ: \",round(std(map_val-mag_use),digits=2))\n",
    "println(\"TL σ:      \",round(std(mag_4_c.+(map_val[1]-mag_4_c[1])-map_val),digits=2))\n",
    "(sigma,tau) = get_autocor(mag_use - map_val)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6b85c5f4-2d51-442f-9df9-a581bf20e81e",
   "metadata": {},
   "source": [
    "Finally, we use these results in an extended Kalman filter (EKF) to check navigation performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f1dbd92-5f0b-4df5-886b-624304c4ceb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "(P0,Qd,R) = create_model(traj.dt,traj.lat[1];\n",
    "                         init_pos_sigma = 0.1,\n",
    "                         init_alt_sigma = 1.0,\n",
    "                         init_vel_sigma = 1.0,\n",
    "                         meas_var       = 56^2, # increase for bad mag\n",
    "                         fogm_sigma     = 56,\n",
    "                         fogm_tau       = 10);\n",
    "\n",
    "(crlb_out,ins_out,filt_out) = run_filt(traj,ins,mag_use,itp_mapS,:ekf;P0,Qd,R,core=true);\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "14e34e8b-a114-4d0f-9fe1-178144aa7eb8",
   "metadata": {},
   "source": [
    "The physics informed neural network outperforms the linear model!\n",
    "\n",
    "The notebook continues below to show a similar comparison using a \"vanilla\" neural network (no linear model). We also provide results for model 2b, which leaves the TL coefficients as constant in the NN learning process. To reproduce the model 2b results, simply replace `model_type = :m2c` with `model_type = :m2b` above and replace the `fogm_sigma` and `fogm_tau` values with the corresponding outputs from `plot_autocor`.\n",
    "\n",
    "Model             | Mag Error [nT] | Nav Error [m]\n",
    "----------------- | -------------- | -------\n",
    "Tolles-Lawson     | 134            | 149 \n",
    "TL + NN, model 2c | 55             | 74 \n",
    "TL + NN, model 2b | 39             | 124 \n",
    "vanilla NN        | 67             | 116 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69ef09d8-fe51-4e6a-8044-63d172d110cd",
   "metadata": {},
   "outputs": [],
   "source": [
    "p7 = plot_map(mapS;legend=false); # plot map background\n",
    "plot_filt!(p7,traj,ins,filt_out;show_plot=false); # plot GPS, INS (after zeroing), and filter\n",
    "plot!(p7,legend=:topleft) # move legend as desired\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "7dc8858b-8b8d-48af-a83f-d0b582f9ed65",
   "metadata": {},
   "source": [
    "## Bonus comparison to vanilla neural network\n",
    "\n",
    "The above approaches used in-cabin vector (Flux D) and scalar (Mag 4) magnetometers, as well as current sensors. Here, we provide the same information to a \"vanilla\" neural network that does not contain embedded Tolles-Lawson information. We have simply added the vector (fluxgate) magnetometer to the feature list and changed to `model type = :m1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c33f103-411d-40bc-9021-d5ab22e364d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "features = [:TL_A_flux_d, :mag_4_uc, :lpf_cur_com_1, :lpf_cur_strb, :lpf_cur_outpwr, :lpf_cur_ac_lo]\n",
    "comp_params_vanilla = NNCompParams(features_setup = features,\n",
    "                                   model_type     = :m1,\n",
    "                                   y_type         = :d,\n",
    "                                   use_mag        = :mag_4_uc,\n",
    "                                   terms          = [:permanent,:induced,:fdm],\n",
    "                                   sub_diurnal    = true,\n",
    "                                   sub_igrf       = true,\n",
    "                                   bpf_mag        = false,\n",
    "                                   norm_type_x    = :standardize,\n",
    "                                   norm_type_y    = :standardize,\n",
    "                                   η_adam         = 0.001,\n",
    "                                   epoch_adam     = 300,\n",
    "                                   epoch_lbfgs    = 0,\n",
    "                                   hidden         = [8,4]);\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf671f0c",
   "metadata": {},
   "outputs": [],
   "source": [
    "(comp_params_vanilla,y_train,y_train_hat,err_train,feats) =\n",
    "    comp_train(comp_params_vanilla,lines_train,df_all,df_flight,df_map);\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "07f1a9d5-702b-4fea-97cc-f5c3573b95ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "(_,y_hat_vanilla,_) =\n",
    "    comp_test(comp_params_vanilla,[line],df_all,df_flight,df_map);\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "227d0f68-49b2-483a-b685-3d8cfa23467f",
   "metadata": {},
   "outputs": [],
   "source": [
    "mag_use = mag_4_uc - y_hat_vanilla # compensate Mag 4 using NN output\n",
    "mag_use .+= map_val[1] - mag_use[1] # remove initial DC offset\n",
    "println(\"NN σ: \",round(std(map_val-mag_use),digits=2))\n",
    "println(\"TL σ: \",round(std(mag_4_c.+(map_val[1]-mag_4_c[1])-map_val),digits=2))\n",
    "(sigma,tau) = get_autocor(mag_use - map_val)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a01959e5-04a0-4879-b69c-e92c91f3b491",
   "metadata": {},
   "outputs": [],
   "source": [
    "tt = (xyz.traj.tt[ind] .- xyz.traj.tt[ind][1]) / 60;\n",
    "p8 = plot(xlab=\"time [min]\", ylab=\"magnetic field [nT]\");\n",
    "plot!(p8, tt, detrend(map_val  - (xyz.diurnal + xyz.igrf)[ind], mean_only=true), lab=\"anomaly map\");\n",
    "plot!(p8, tt, detrend(mag_4_uc - (xyz.diurnal + xyz.igrf)[ind], mean_only=true), lab=\"uncompensated\");\n",
    "plot!(p8, tt, detrend(mag_4_c  - (xyz.diurnal + xyz.igrf)[ind], mean_only=true), lab=\"Tolles-Lawson\");\n",
    "plot!(p8, tt, detrend(mag_4_uc - y_hat_vanilla - (xyz.diurnal + xyz.igrf)[ind], mean_only=true), lab=\"vanilla NN\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2e72221c-d403-4ffd-80cf-1a2af3b35c37",
   "metadata": {},
   "outputs": [],
   "source": [
    "(P0,Qd,R) = create_model(traj.dt,traj.lat[1];\n",
    "                         init_pos_sigma = 0.1,\n",
    "                         init_alt_sigma = 1.0,\n",
    "                         init_vel_sigma = 1.0,\n",
    "                         meas_var       = 56^2, # increase for bad mag\n",
    "                         fogm_sigma     = 56,\n",
    "                         fogm_tau       = 12);\n",
    "\n",
    "(crlb_out,ins_out,filt_out) = run_filt(traj,ins,mag_use,itp_mapS,:ekf;P0,Qd,R,core=true);\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "08a47828-2c38-4ea4-8b27-2cce6e62f2f5",
   "metadata": {},
   "source": [
    "The vanilla neural network *does* do better than Tolles-Lawson alone, but not as well as model 2c.\n",
    "\n",
    "We are continuing to work on ways to do more with less data."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Julia 1.11.4",
   "language": "julia",
   "name": "julia-1.11"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}