Merge branch 'main' into nov_doc_changes

docs/configfiles.rst

@@ -49,11 +49,11 @@ Sorcha's **ppsqldbquery** config file parameter contain the sql query for obtain
 
 From rubin_sim v2.0 simulations onward use the query::
 
-  SELECT observationId, observationStartMJD as observationStartMJD_TAI, filter, seeingFwhmGeom, seeingFwhmEff, fiveSigmaDepth, fieldRA, fieldDec, rotSkyPos FROM observations order by observationId
+  SELECT observationId, observationStartMJD as observationStartMJD_TAI, visitTime, filter, seeingFwhmGeom, seeingFwhmEff, fiveSigmaDepth, fieldRA, fieldDec, rotSkyPos FROM observations order by observationId
 
 For past rubin_sim/OpSim simulations pre-v2.0 use the query::
 
-  SELECT observationId, observationStartMJD as observationStartMJD_TAI, filter, seeingFwhmGeom, seeingFwhmEff, fiveSigmaDepth, fieldRA, fieldDec, rotSkyPos FROM SummaryAllProps order by observationId
+  SELECT observationId, observationStartMJD as observationStartMJD_TAI, visitTime, filter, seeingFwhmGeom, seeingFwhmEff, fiveSigmaDepth, fieldRA, fieldDec, rotSkyPos FROM SummaryAllProps order by observationId
 
 .. _makeConfigPP:
 

docs/ephemerisgen.rst

@@ -3,7 +3,7 @@
 Ephemeris Generator
 ==========================================================
 
-Sorcha's ephemeris generator is powered by `ASSIST  <https://assist.readthedocs.io/en/latest/>`__, a software package for ephemeris-quality integrations of test particles, and the `REBOUND <https://rebound.readthedocs.io/en/latest/>`__ N-body integrator. If the user prefers to use a different generator or provide the ephemeris output from a previous Sorcha run,  they have the ability to point Sorcha to an external file to ingest instead.
+Sorcha's ephemeris generator is powered by `ASSIST  <https://assist.readthedocs.io/en/latest/>`__, a software package for ephemeris-quality integrations of test particles, and the `REBOUND <https://rebound.readthedocs.io/en/latest/>`__ N-body integration package. If the user prefers to use a different generator or provide the ephemeris output from a previous Sorcha run,  they have the ability to point Sorcha to an external file to ingest instead.
 
 .. tip::
   We recommend using Sorcha's ephemeris generator for all your survey simulations. 
@@ -12,12 +12,9 @@ Sorcha's ephemeris generator is powered by `ASSIST  <https://assist.readthedocs.
 How It Works
 --------------------------------------------------------
 
-Because ASSIST uses REBOUND's `IAS15 integrator <https://ui.adsabs.harvard.edu/abs/2015MNRAS.446.1424R/abstract>`_, which has an adaptive time step, Sorcha's ephemeris generator instantiates a REBOUND n-body simulation for each individual massless synethetic object including the effects of the Sun, planets, Moon, and 16 asteroids (see the :ref:`MAP` section). It also includes the J2, J3, and J4 gravitational harmonics of the Earth, the J2 gravitational harmonic of the Sun, and general relativistic correction terms for the Sun, using the Parameterized Post-Newtonian (PPN) formulation. The positions of the massive bodies come from the latest `DE441 <https://iopscience.iop.org/article/10.3847/1538-3881/abd414>`_ ephemeris, provided by NASA's `Navigation and Ancillary Information Facility (NAIF) <https://naif.jpl.nasa.gov/naif/credit.html>`_. We note that the coordinate frame for ASSIST+REBOUND  is the equatorial International Celestial Reference Frame (ICRF). We note that this is barycentric, rather than heliocentric. The ephemeris generator translates the input barycentric or heliocentric orbits into x,y, z and velocities in the ICRF to be read into ASSIST. 
+The Sorcha ephemeris generator determines which objects will appear in or near the camera field of view (FOV) for any given exposure.  It uses spatial indexing to speed up these calculations.  It runs through the survey visits and does on-the-fly checks of where every synthetic object is near the center of each night for which there are visits and organizes those positions using the `HEALPix (Hierarchical Equal Area isoLatitude Pixelation of a sphere) <https://healpix.sourceforge.io/>`_ tesselation of the sky.  Given that information, it then steps through the visits for that night, doing precise calculations for just those objects that are near the camera FOV (field-of-view) of each survey on-sky visit.   Specifically, for each visit, the generator calculates the unit vector from the observatory's location to the RA/Dec location of the field center. Then it finds the  set of HEALPix tiles that are overlapped by the survey vist's camera FOV (nside=64). The ephemeris generator then collects the IDs for the particles in the HEALPix tiles overlapped by the given survey visit FOV.  It then does light time corrected ephemeris calculations for just those, outputting the right ascenion, declination, rates, and relevant distances, and phase angle values for each of the particles. 
 
-
-The ephemeris generator runs through the survey visits and does on-the-fly checks of where every synthetic object is near the center of each night for which there are visits (like planting the pickets (vetical planks of wood) along a picket fence. Given that information, it then steps through the visits for that night, doing precise calculations for just those objects that are near the camera FOV (field-of-view) of each survey on-sky visit.
-
-For each survey visit, the generator calculates the location of the observatory and the  topocentric unit vector to the field RA/Dec. Then it finds the  set of `HEALPix (Hierarchical Equal Area isoLatitude Pixelation of a sphere) <https://healpix.sourceforge.io/>`_ tiles that are overlapped by the survey vist's camera FOV (nside=64). The ephemeris generator then collects the IDs for the particles in the HEALPix tiles overlapped by the given survey visit FOV, and do light time corrected ephemeris calculations for just those, outputting the right ascenion, declination, rates, and relevant distances, and phase angle values for each of the particles. 
+Because ASSIST uses REBOUND's `IAS15 integrator <https://ui.adsabs.harvard.edu/abs/2015MNRAS.446.1424R/abstract>`_, which has an adaptive time step, Sorcha's ephemeris generator instantiates a REBOUND n-body simulation for each individual massless synthetic object including the effects of the Sun, planets, Moon, and 16 asteroids (see the :ref:`MAP` section). It also includes the J2, J3, and J4 gravitational harmonics of the Earth, the J2 gravitational harmonic of the Sun, and general relativistic correction terms for the Sun, using the Parameterized Post-Newtonian (PPN) formulation. The positions of the massive bodies come from the latest `DE441 <https://iopscience.iop.org/article/10.3847/1538-3881/abd414>`_ ephemeris, provided by NASA's `Navigation and Ancillary Information Facility (NAIF) <https://naif.jpl.nasa.gov/naif/credit.html>`_. We note that the coordinate frame for ASSIST+REBOUND  is the equatorial International Celestial Reference Frame (ICRF).  The positions and velocities are barycentric within this frame, rather than heliocentric. The ephemeris generator translates the input barycentric or heliocentric orbits into x,y, z and velocities into the barycentric ICRF to be read into ASSIST. 
 
 .. tip::
   If using Sorcha's internal ephemeris generation mode (which is the default mode), **we recommend calculating/creating your input orbits with epochs close in time to the start of the first survey observation**. This will minimize the REBOUND n-body integrations required to set up the ephemeris generation.
@@ -49,7 +46,7 @@ Here's the list of asteroid pertubers that are included in the ASSIST+REBOUND in
 - **(4) Vesta = A807 FA** 
 
 .. warning::
-  If you simulate the orbits of these select asteroids you will get *POOR results* with the internal Sorcha epehmeris generator because of how the n-body integration is setup. We recommend getting the positions of these asteroids from some other source and inputting them as an external ephemeris file. 
+  If you simulate the orbits of these select asteroids you will get **POOR results** with the internal Sorcha epehmeris generator because of how the n-body integration is setup. We recommend getting the positions of these asteroids from some other source and inputting them as an external ephemeris file. 
 
 Tuning the Ephemeris Generator
 -----------------------------------
@@ -79,5 +76,5 @@ A number of auxillary file available from the `Minor Planet Center <https://www.
 - **obscodes_extended.json** - observatory position information and Minor Planet Center (MPC) observatory codes.
 
 .. tip::
-  See our :ref:`installation_aux` instructions to find out how to download and install these auxillary files automaticlal using our download utility. 
+  See our :ref:`installation_aux` instructions to find out how to download and install these auxillary files automatically using our download utility. 
 

docs/filters.rst

@@ -48,8 +48,7 @@ To include this filter, the following options should be set in the configuration
 
 .. image:: images/fading_function.png
   :width: 400
-  :alt: Graph showing the fading function. Detection probability is plotted against magnitude - limiting magnitude,
-  showing three smoothed step-functions centred on 0.0 on the x axis for three different widths.
+  :alt: Graph showing the fading function. Detection probability is plotted against magnitude - limiting magnitude, showing three smoothed step-functions centred on 0.0 on the x axis for three different widths.
   :align: center
 
 
@@ -87,8 +86,7 @@ gaps between the chips. The LSST detector shape is included in the repo and desc
 
 .. image:: images/Footprint.png
   :width: 600
-  :alt: Plot of the LSST camera footprint where x and y are x and y distance from the pupil in degrees. The footprint also shows
-  two overplotted circle radii of 1.75deg (corresponding to a 75% fill factor) and 2.06deg.
+  :alt: Plot of the LSST camera footprint where x and y are x and y distance from the pupil in degrees. The footprint also shows two overplotted circle radii of 1.75deg (corresponding to a 75% fill factor) and 2.06deg.
 
 However, the full camera footprint is most relevant for slow-moving objects, where an 
 object may move only a small amount per night and could thus in a subsequent observation fall into a chip gap. This is less concerning for 
@@ -208,4 +206,4 @@ The defaults given below are those used by SSP and are explained in the comments
     # complete track/detection.
     SSP_track_window = 15
 
- 
+ 

docs/support.rst

@@ -31,7 +31,7 @@ We are very happy to receivet feedback on the online documentation through the `
 You will need to install the development version of Sorcha from a clone of the Sorcha repository. See the our  :ref:`dev_mode` instructions for further details. 
 
 
-Once you are properly set up with a clone of the repository, you can move to the docs directory (cd sorcha/docs/), edit the *.rst files, and run::
+If you move to the docs directory (cd sorcha/docs/), edit the .rst files, and run::
 
    make html
 

src/sorcha/modules/PPConfigParser.py

@@ -779,7 +779,7 @@ def PPPrintConfigsToLog(configs, cmd_args):
             pplogger.info(
                 "The code will approximate chip gaps using filling factor: " + str(configs["fill_factor"])
             )
-        elif configs["circleRadius"]:
+        elif configs["circle_radius"]:
             pplogger.info(
                 "A circular footprint will be applied with radius: " + str(configs["circle_radius"])
             )

src/sorcha/modules/PPOutput.py

@@ -3,6 +3,10 @@
 import sqlite3
 import logging
 
+# this is for suppressing a warning in PyTables when writing to HDF5
+import warnings
+from tables import NaturalNameWarning
+
 
 def PPOutWriteCSV(padain, outf):
     """
@@ -43,6 +47,14 @@ def PPOutWriteHDF5(pp_results, outf, keyin):
 
     """
 
+    # pytables doesn't like the Pandas extension dtype StringDtype
+    # converting the ObjID to 'str' type fixes this
+    pp_results = pp_results.astype({"ObjID": str})
+
+    # this suppresses a warning when ObjIDs begin with a number
+    # as long as the user isn't going to use PyTables to access the data this doesn't matter
+    warnings.filterwarnings("ignore", category=NaturalNameWarning)
+
     of = pp_results.to_hdf(outf, mode="a", format="table", append=True, key=keyin)
 
     return of