PoroTomo

PoroTomo.jl is a Julia API for downloading data from the PoroTomo open dataset on AWS. PoroTomo.jl is intended for use on AWS EC2 instances for maximum file transfer speed but will work (slowly) on local compute.

Features:

• Streaming of DAS files using the SeisIO.Nodal module
• Downloading of DAS files to disk

Installation

julia> using Pkg; Pkg.add(PackageSpec(url="https://github.com/tclements/PoroTomo.jl", rev="main"))

Quickstart

The PoroTomo Dataset is organized by SEG-Y/DAS{orientation}/YYYYMMDD/filename where orientation is either H (horizontal) or V (vertical) and YYYYMMDD range between 20160308 and 20160326.

First we'll query the available files for a specific date-range:

julia> using Dates, PoroTomo

julia> startdate = DateTime(2016,3,12,22)

julia> enddate = DateTime(2016,3,13,2,45)

julia> paths = get_porotomo_files(startdate,enddate=enddate)
570-element Array{String,1}:
 "SEG-Y/DASH/20160312/PoroTomo_iDAS16043_160312220018.sgy"
 "SEG-Y/DASH/20160312/PoroTomo_iDAS16043_160312220048.sgy"
 "SEG-Y/DASH/20160312/PoroTomo_iDAS16043_160312220118.sgy"
 ⋮
 "SEG-Y/DASH/20160313/PoroTomo_iDAS16043_160313024348.sgy"
 "SEG-Y/DASH/20160313/PoroTomo_iDAS16043_160313024418.sgy"
 "SEG-Y/DASH/20160313/PoroTomo_iDAS16043_160313024448.sgy"

paths are the locations of files in the PoroTomo bucket in S3. To stream one of these files to a SeisIO.NodalData, use the porotomo_stream function:

julia> N = porotomo_stream(paths[1])
NodalData with 8721 channels (2 shown)
COORDS: X = 0.0, Y = 0.0, Z = 0.0
 INFO: 32 entries
   ID: N0.00001..OG0                      N0.00002..OG0                      …
 NAME: 0_1                                0_2                                …
  LOC: x 0.0, y 0.0, z 0.0                x 0.0, y 0.0, z 0.0                …
   FS: 1000.0                             1000.0                             …
 GAIN: 0.0                                0.0                                …
 RESP: a0 1.0, f0 1.0, 1z, 1p             a0 1.0, f0 1.0, 1z, 1p             …
UNITS: m/m                                m/m                                …
  SRC: SEG-Y/DASH/20160312/PoroTomo_iDAS… SEG-Y/DASH/20160312/PoroTomo_iDAS… …
 MISC: 92 entries                         92 entries                         …
NOTES: 0 entries                          0 entries                          …
    T: 2016-03-12T22:18:18 (0 gaps)       2016-03-12T22:18:18 (0 gaps)       …
    X: +3.117e-03                         +3.554e-03                         …
       +2.802e-03                         +4.045e-03                         …
           ...                                ...                            …
       +4.316e-04                         +4.164e-04                         …
       (nx = 30000)                       (nx = 30000)                       …

We can download files to disk using the porotomo_download function:

julia> OUTDIR = "~/DASDATA" # data to store SEGY files 

julia> porotomo_download(paths[1],OUTDIR)

The file PoroTomo_iDAS16043_160312220018.sgy is now stored at ~/DASDATA/SEG-Y/DASH/20160312/PoroTomo_iDAS16043_160312220018.sgy.

It can be read with SeisIO:

julia> using SeisIO, SeisIO.Nodal 

julia> N = read_nodal("segy", expanduser("~/DASDATA/SEG-Y/DASH/20160312/PoroTomo_iDAS16043_160312220018.sgy"))
NodalData with 8721 channels (2 shown)
COORDS: X = 0.0, Y = 0.0, Z = 0.0
  INFO: 32 entries
    ID: N0.00001..OG0                      N0.00002..OG0                      …
  NAME: 0_1                                0_2                                …
   LOC: x 0.0, y 0.0, z 0.0                x 0.0, y 0.0, z 0.0                …
    FS: 1000.0                             1000.0                             …
  GAIN: 0.0                                0.0                                …
  RESP: a0 1.0, f0 1.0, 1z, 1p             a0 1.0, f0 1.0, 1z, 1p             …
 UNITS: m/m                                m/m                                …
   SRC: /home/timclements/DASDATA/SEG-Y/D… /home/timclements/DASDATA/SEG-Y/D… …
  MISC: 92 entries                         92 entries                         …
 NOTES: 0 entries                          0 entries                          …
     T: 2016-03-12T22:18:18 (0 gaps)       2016-03-12T22:18:18 (0 gaps)       …
     X: +3.117e-03                         +3.554e-03                         …
        +2.802e-03                         +4.045e-03                         …
            ...                                ...                            …
        +4.316e-04                         +4.164e-04                         …
        (nx = 30000)                       (nx = 30000)                       …