Skip to content

Reprocessing 2018 Structure Review

Max Burnette edited this page Nov 1, 2018 · 5 revisions

Review of current data structure on disk (i.e. Globus)

/ua-mac 
  /raw_data 
    /3DScannerRawDataTopTmp    DELETE (redundant to /scanner3DTop, only metadata files)
    /3DTest                    DELETE (minimal 2016 data, DELETE)
    /EnvironmentLogger
    /FLIR_TG167_handheld       UNKNOWN (data from June 2017)
    /SWIR
    /TestIn                    DELETE (a couple empty 2016 folders)
    /UAV
    /VNIR
    /co2Sensor
    /flirIrCamera
    /irrigation
    /lightning
    /ndviSensor
    /phenotractor
    /priSensor
    /ps2Top
    /scanner3DLowerOnEastSide
    /scanner3DLowerOnWestSide
    /scanner3DTop             KEEP (this is raw PNG data directly from MAC)
    /stereoEast
    /stereoTop
    /stereoWest
    /weather
  /Level_1
    /canopy_height            MOVE (Level_3 /laser3d_canopy_height)
    /envlog_netcdf            
    /fullfield                MOVE (Level_2, move rgb_canopycover CSVs into Level_3 and split out FF types)
    /ir_geotiff
    /ir_meanTemp              MOVE (Level_3)
    /laser3d_heightmap        MOVE (Level_2)
    /laser3d_mergedlas
    /ps2_fluorescence         UPDATE EXTRACTOR AND MOVE (Level_2)
    /ps2_png
    /rgb_geotiff
    /scanner3DTop             KEEP (this is Level_1 PLY data directly from MAC, change to laser3d_ply)
    /scanner3DTop_mergedlas   DELETE (deprecated format from old extractor, replaced by /laser3d_las)
    /soil_removal             MERGE (Level_2, merge with vnir/swir_soil_masks SWIR and VNIR subdirectories)
    /texture_analysis         MOVE (Level_3 /rbg_textureanalysis, evaluate extractor but needs work still. 
                                 filter out - https://github.com/terraref/reference-data/issues/172)
    /UAV                      KEEP & CAPITALIZE (rededge and sequoia subdirectories)
    /VNIR_netcdf
    /SWIR_soil_masks          CREATE (Level_2)
    /VNIR_soil_masks          MOVE (Level_2 already has some)
  /Level_2
    /VNIR_soil_masks
    /rgb_soil_masks           Zongyang working on this.

https://github.com/terraref/reference-data/issues/235

Extractors in development & GitHub links: https://docs.google.com/spreadsheets/d/1VnjcP1OAxY274tEo7yVAQGmr0tuf8uMXreMhP36PBeQ/edit#gid=0