-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathhrrr_FAQ_old.html
1036 lines (962 loc) · 65.5 KB
/
hrrr_FAQ_old.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html>
<!--
This file `hrrr_FAQ.html` and `index.html` are identical (linked) so that
whatever CHPC did, the URL http://hrrr.chpc.utah.edu directs to this page.
-->
<head>
<title>HRRR Archive FAQ</title>
<script src="./js/site/siteopen.js"></script>
<!--
The following script makes the HRRR archive findable by Google's dataset search
https://toolbox.google.com/datasetsearch/search?query=HRRR&docid=Klwq8fZSQuBEtzuGAAAAAA%3D%3D
-->
<script type="application/ld+json">
{
"@context":"http://schema.org/",
"@type":"Dataset",
"name":"High Resolution Rapid Refresh Model",
"description":"The High Resolution Rapid Refresh Model archive at the University of Utah",
"url":"http://hrrr.chpc.utah.edu/",
"creator":{
"@type":"Organization",
"url": "http://www.emc.ncep.noaa.gov/",
"name":"NOAA/NCEP/EMC > Environmental Modeling Center, National Centers For Environmental Prediction, NOAA, U.S. Department of Commerce",
"contactPoint": {
"@type": "ContactPoint",
"email": "[email protected]",
"url": "https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/home.html",
"contactType": "Customer Service"
}
},
"distribution":[
{
"@type":"DataDownload",
"encodingFormat":"GRIB2",
"contentUrl":"https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_script_tips.html"
}
],
"temporalCoverage":"2016-07-15/Present",
"spatialCoverage":{
"@type":"Place",
"geo":{
"@type":"GeoShape",
"box":"21.14 -134.10 52.62 -60.72"
}
}
}
</script>
</head>
<body>
<a name="TOP"></a>
<script src="./js/site/sitemenu.js"></script>
<br>
<div id="content" itemscope itemtype="http://schema.org/Dataset">
<h1 align="center">HRRR Archive at the University of Utah</h1>
<h2 align="center"><i class="fa fa-info-circle fa-fw"></i>Frequently Asked Questions</h2>
<script src='./js/pando_status.js'></script>
<script src='./js/HRRR_status.js'></script>
<div class="row" id="content">
<div class=" col-md-3">
<a href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_download_register.html"
class="btn btn-danger btn-block">
<i class="fa fa-user-plus"></i> Have you Registered?</a>
</div>
<div class="col-md-3">
<a href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_practices.html"
class="btn btn-warning btn-block">
<i class="far fa-handshake"></i> Best Practices</a>
</div>
<div class="col-md-3">
<a href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_FAQ.html"
class="btn btn-success btn-block active">
<i class="fa fa-info-circle"></i> HRRR FAQ</a>
</div>
<div class="col-md-3">
<a href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/hrrr_download.cgi"
class="btn btn-primary btn-block">
<i class="fa fa-cloud-download-alt"></i> Web Download Page</a>
</div>
</div>
<br>
<!-- Tabs -->
<ul class="nav nav-tabs">
<li class="active"><a data-toggle="tab" href="#tab1"><i class="fa fa-globe"></i> About the HRRR archive</a>
</li>
<li><a data-toggle="tab" href="#tab2"><i class="fa fa-user"></i> Who archives HRRR?</a></li>
<li><a data-toggle="tab" href="#tab3"><i class="fa fa-database"></i> Where is the archive?</a></li>
<li><a data-toggle="tab" href="#tab4"><i class="fa fa-calendar-alt"></i> What days are available?</a></li>
<li><a data-toggle="tab" href="#tab5"><i class="fa fa-users"></i> Who uses the HRRR archive?</a></li>
<li><a data-toggle="tab" href="#tab6"><i class="fa fa-paint-brush"></i> Gallery</a></li>
<li><a data-toggle="tab" href="#tab7"><i class="fa fa-desktop"></i> Tips for CHPC users</a></li>
</ul>
<div class="tab-content">
<div id="tab1" class="tab-pane fade in active">
<!--<img align=right width="450px" style="padding:5px 15px 5px 5px" src="https://rapidrefresh.noaa.gov/hrrr/hrrrcrefimage">-->
<iframe align=right width="460" height="322" src="https://www.youtube.com/embed/R8d10nMWY1I?rel=0"
frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
<h3>Just tell me how to get the data...</h3>
<a class='btn btn-primary'
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/hrrr_download.cgi">Web Download
Interface</a>
<a class='btn btn-primary'
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_script_tips.html">Scripting Tips</a>
<br>
<h2><i class="fa fa-globe"></i> About the HRRR archive</h2>
<h3>What is this HRRR archive?</h3>
<p>This HRRR archive is a collection of output from NCEP's High Resolution Rapid Refresh model. This
is a model developed by <a href="https://rapidrefresh.noaa.gov/hrrr/" target='_blank'>NOAA ESRL</a>
and is run operationally every hour at NCEP's Environmental Modeling Center.
It continues to be developed by ESRL.
<p>The operational HRRR generates hourly forecasts gridded at 3 km
for 18 hours over the contiguous United States making it the
highest spatial and temporal resolution forecast system run by NCEP.
<p><a class="btn btn-info"
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/HRRR_archive/hrrr_sfc_table_f00-f01.html">
HRRR Variable Details
</a>
<a class="btn btn-info" href="https://rapidrefresh.noaa.gov/RAP_var_diagnosis.html">
RAP/HRRR Diagnostic Details
</a>
<p>HRRR analyses and forecasts are exceptionally valuable to the research community. However, an
official HRRR archive does not exist. We began archiving HRRR data in April 2015 to support research
efforts at the University of Utah. The archive system <a
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/Pando_archive/Pando_Failure.html">failed</a>
and we lost all the data. We rebuilt the archive with data beginning 15 July 2016.
<script src='./js/climate_acknowledgement.js'></script>
<p>Instead of storing all available HRRR files, we only download the files most useful to accomplish our
research efforts. We realize this data is valuable to many others and have made the archive publicly
accessible for research purposes. We currently have 60 terabytes of storage space at our disposal.
<div class='alert alert-sm alert-info'>
<p><i class="fa fa-fw fa-book"></i> Additional details about this archive are published in <a
class="alert-link" href="https://doi.org/10.1016/j.cageo.2017.08.005"
target="_blank">Computers and Geosciences</a>
<p><i class="fa fa-fw fa-database"></i> This HRRR archive has an official doi: <a class="alert-link"
href="https://doi.org/10.7278/S5JQ0Z5B"> 10.7278/S5JQ0Z5B</a>
</div>
<hr>
<h3>What files are contained in the HRRR archive?</h3>
<p>Output files, in GRIB2 format, contained in the archive include:
<div class="panel-group" id="accordion" style="max-width:700px; margin:auto">
<div class="panel panel-default">
<div class="panel-heading">
<h4 class="panel-title">
<a data-toggle="collapse" data-parent="#accordion" href="#collapse1"><i
class="fa fa-fw fa-globe"></i> Operational HRRR</a>
</h4>
</div>
<div id="collapse1" class="panel-collapse collapse">
<div class="panel-body">
<ul style="padding-left:40px">
<li><b>sfc</b>: Surface fields for analyses and forecasts
<ul style="padding-left:40px">
<li>File format: <span
style="font-family:monospace">hrrr.t[00-23]z.wrfsfcf[00-18].grib2</span>
<li>File size: ~120 MB
</ul>
<li><b>prs</b>: Pressure fields for analyses
<ul style="padding-left:40px">
<li>File format: <span
style="font-family:monospace">hrrr.t[00-23]z.wrfprsf00.grib2</span>
<li>File size: ~380 MB
</ul>
<li><strike><b>bufr</b>: Vertical profiles available for KSLC, KODG, and KPVU
<ul style="padding-left:40px">
<li>File format: <span
style="font-family:monospace">[KSLC,KODG,KPVU]_[YYYYMMDDHH].buf</span>
<li>File size: ~75 KB
</ul></strike>
</ul>
</div>
</div>
</div>
<div class="panel panel-default">
<div class="panel-heading">
<h4 class="panel-title">
<a data-toggle="collapse" data-parent="#accordion" href="#collapse2"><i
class="fa fa-fw fa-flask"></i> Experimental HRRR</a>
</h4>
</div>
<div id="collapse2" class="panel-collapse collapse">
<div class="panel-body">
<ul style="padding-left:40px">
<li><b>sfc</b>: Surface fields for analyses
<ul style="padding-left:40px">
<li>File format: <span
style="font-family:monospace">hrrr.t[00-23]z.wrfsfcf00.grib2</span>
<li>File size: ~120 MB
</ul>
</ul>
<p>Note: Not all hours are available for the experimental runs.
</div>
</div>
</div>
<div class="panel panel-default">
<div class="panel-heading">
<h4 class="panel-title">
<a data-toggle="collapse" data-parent="#accordion" href="#collapse3"><i
class="fa fa-fw fa-cogs"></i> HRRR - Alaska</a>
</h4>
</div>
<div id="collapse3" class="panel-collapse collapse">
<div class="panel-body">
<p>Became operational after July 12, 2018.
<ul style="padding-left:40px">
<li><b>sfc</b>: Surface fields for analyses and forecast hours
<ul style="padding-left:40px">
<li>File format: <span
style="font-family:monospace">hrrr.t[00,03,06,09,12,15,18,21]z.wrfsfcf[00-36].grib2</span>
<li>File size: ~100 MB
</ul>
<strike>
<li><b>prs</b>: Pressure fields for analyses
<ul style="padding-left:40px">
<li>File format: <span
style="font-family:monospace">hrrr.t[00,03,06,09,12,15,18,21]z.wrfprsf00.grib2</span>
<li>File size: ~205 MB
</ul>
</strike>
</ul>
<p>Note: Not all hours are available for the experimental runs.
</div>
</div>
</div>
</div>
<p>Note: Some days and hours in our archive may not be available. Either the forecast wasn't run that
hour (typical for the experimental models), or our download scripts failed to download everything.
<hr>
<h3>What are GRIB2 files?</h3>
<p><a href="https://en.wikipedia.org/wiki/GRIB">GRIB2</a>, or Gridded Binary Version 2, is a standard
file format used by meteorologists for model data sets. There are several useful tools for working
with the data.
<ul style="padding-left:50px">
<li><a href="http://www.cpc.noaa.gov/products/wesley/wgrib2/">wgrib2</a>: a command line utility
used to read GRIB2 files.
<li><a href="https://pypi.python.org/pypi/pygrib">pygrib</a>: a Python module used to read GRIB2
files.
<li><a href="https://pypi.org/project/cfgrib/">cfgrib</a>: a new Python module used to read GRIB2
files (version > 0.9.7 works with HRRR data). Works on Windows 😎
<li><a href="https://www.unidata.ucar.edu/software/idv/">IDV</a>: a desktop application that can
view many types of weather data.
<li><a href="https://www.ncdc.noaa.gov/wct/">NOAA Toolkit</a>: graphical software that can read and
visualize GRIB2 files. <span style='color:red'>I highly recommend this tool if you haven't used
GRIB2 files before</span>.
</ul>
<hr>
<h3>Where do the HRRR output files come from?</h3>
<p>The operational HRRR (hrrr) and HRRR Alaska (hrrrak) is downloaded via HTTP from the <a
href="http://nomads.ncep.noaa.gov/">NOMADS server</a>.
<p>Experimental HRRR (hrrrX) is downloaded via FTP from NOAA ESRL (credentials required).
<hr>
<h3>What version of HRRR is in this archive?</h3>
<p><b>HRRRv1</b> was the operational model prior to August 23, 2016.
<p><b>HRRRv2</b> was implemented at NCEP on August 23, 2016 beginning
with the 12z run. We download the operational version from <a
href="http://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/">NOMADS</a>.
For more details on the HRRRv2 implementation, read the <a
href="http://www.nws.noaa.gov/om/notification/tin16-26rap_hrrrraaa.htm">announcement</a>
<p><b>HRRRv3</b> was implemented at NCEP on July 12, 2018 beginning with the 12z run.
Version 3 extends the forecast period for the 00, 06, 12, and 18 run out to 36 hours, but we have
decided not to archive the f19-f36 files. Improvements in version 3 include updated Thompson
microphysics, improved MYNN PBL scheme, updated land surface model with 15 second MODIS data,
refined roughness lengths for certain land use types, and additional improvements in assimilation
methods including lightning, radar radial velocity, and TAMDAR.
<p><b>HRRRv4</b> was implemented at NCEP on Dec 2, 2020 beginning with the 12z run.
This version makes more use of ensemble
modeling methods and many improvements to model physics.
Check out the <a href="https://rapidrefresh.noaa.gov/hrrr/">Official HRRR Webpage</a>
for more information. Key upgrades include:
<ul>
<li>Forecasts for the 00/06/12/18z are made out to 48 hours</li>
<li>MYNN PBL update for better sub-grid clouds</li>
<li>Aerosols sources/sinks</li>
<li>Improved Land surface</li>
<li>Assimilate GOES-16 Radiances and Motion Vectors and VIIRS/MODIS fire radiative power</li>
<li>Ensemble 3km data assimilation</li>
<li>Addition of smoke forecast variables (more information on these are available via <a href="https://rapidrefresh.noaa.gov/hrrr/HRRRsmoke/HRRR_Smoke_IMET_Fire_QuickGuide_Aug14.pdf" target="_blank">this guide</a>)</li>
</ul>
<p>What's next for the HRRR? The HRRR model is the foundation for the upcoming Rapid Refresh Forecast System anticipated in 2022. Check out this article of <a href="https://research.noaa.gov/article/ArtMID/587/ArticleID/2702/The-amazing-research-resume-of-High-Resolution-Rapid-Refresh-Model">HRRR's great achievements</a>.
<hr>
<h3>When do we download the HRRR data?</h3>
<p><b>HRRR</b>, <b>HRRR-X</b>, and <b>HRRR-Alaska</b>
is downloaded from NOMADS and ESRL 8 times a day at
00:30, 03:30, 06:30, 09:30, 12:30, 15:30, 18:30, and 21:30 UTC.
<p>We only download HRRR data eight times a day to prevent anyone
from grabbing real-time forecasts because this HRRR
archive is not meant to be used for real-time or operational
products.
<div class="alert alert-warning">
<p style="text-align:center"> If you are looking for today's HRRR output, then download directly
from
<a class="alert-link"
href="http://www.nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/">NOMADS</a>.
</div>
<hr>
<img align=right \ width='350px' \ class='style11' \ style="margin-left:15px; margin-bottom:15px" \
src='./images/mouse.jpg' \ title="We give people a little, and they start asking for a lot.">
<h3>Vision for the Future</h3>
<p>This project is turning into a classic
case of one of my favorite children's books <i>If You Give A Mouse A
Cookie</i> where starting one thing has grown and continues to grow
into something bigger.
<p>At the 2017 Annual AMS meeting I talked with a guy on the the
AMS Board on Data Stewardship. He said the lack of
a HRRR archive was one concern brought up at their board meeting.
<p>Archiving high resolution model data is an expensive task.
This is a growing issue in the atmospheric science community
as we move to higher resolution models that generate terabytes of
data annually. At the 2018 Annual AMS meeting, it sounds like
NOAA is interested in pushing the HRRR output to its
<a href='http://www.noaa.gov/data-community-resources'>Big Data</a>
partners.
<p>There are many opportunities with cloud computing where we "bring the
data to the computing." An archive on the cloud can be
kept in one place and researchers can use cloud resources to
process the data, perform analyses, and initialize WRF simulations
without downloading the model output on their own computers.
The benefits and complications of
cloud computing were discussed at the
<a href="http://www.unidata.ucar.edu/events/2017CloudModelingWorkshop/#schedule">
2017 Modeling Research in the Cloud</a> Workshop.
It would especially be beneficial if it were possible to efficiently mine the large
dataset for a specific variable at a point or subgrid
for a range of dates or times. Perhaps this model data needs to be stored in
HDF5 format rather than grib2. It would be nice if there was fewer
redundant data fields between the files. For instance, some fields
are available in the prs and sfc and nat files.
<p>Until data storage challenges are overcome, I would like to make this
HRRR archive easily available to everyone to demonstrate the
interest and need for having such an archive.
This archive has already proven beneficial to many researchers
and has served in many applications.
<p>In the future, it is possible to have server-side data processing
that generate time series or wind roses for point locations
in the HRRR model. My current methods for doing this are extremely
inefficient because they still require downloading a temporary file
for a single variable of the entire grid (which contains about 1.9
grid points over the contiguous United States).
<p>One day, we hope this data will be archived elsewhere that is more
accessible to everyone. Perhaps soon it will be hosted by Amazon by
their Opendata initiative. I would advocate to keep it in the
GRIB2 format (the original format it is output as), but it
would also be nice to store the data in a "cloud-friendly"
format such as zarr. Some extensive experimentation is still
needed to understand how best to chunk the data
in a way that will be useful for all users while preserving
the same compression of the data that is enjoyed by the GRIB2 format.
<hr>
<p> If I were in charge of the HRRR, I would like to see a more
rapid data assimilation cycle and run the model every 15 minutes
out to a 48 hour forecast. This would ultimately create 7 times as
much data while providing a 96 members for time-lagged ensembles.
Maybe different physics parameters could be used in each 15 minute
model run to produce more model spread.
My dream for a future weather prediction system is a model that is
<i>continuously updating</i> rather than rapidly updating.
<hr>
<p>The main concept of this archive is to create a HRRR archive in
one place that is easily accessible so researchers don't have
to redundantly store local archives, with the purpose to provide
the data you need, when you need it.
</div>
<div id="tab2" class="tab-pane fade">
<h3><i class="fa fa-user"></i> Who archives the HRRR data?</h3>
<p>The HRRR archive at the University of Utah is managed by Brian Blaylock and the
<a href="http://meso1.chpc.utah.edu/mesowest_overview/" target="_blank">MesoWest group</a>
on resources at Utah's <a href="https://www.chpc.utah.edu/" target="_blank">Center for High
Performance Computing</a>.
It can also be found in the University of Utah Hive.
Please contact <a href="mailto:[email protected]">Brian</a> with questions regarding the
archive.
<div align="center">
<a href="https://www.atmos.utah.edu/"><img align="center" src="./images/UU_logo.png"></a>
<a href="https://mesowest.org/"><img align="center" src="./images/MesoWest/MesoWest_20th_black.png"
style="padding-left:15px"></a>
<a href="https://www.chpc.utah.edu/"><img align="center" src="./images/chpc_logo3.jpg"
style="height:110px;padding-left:15px"></a>
<a href="https://hive.utah.edu/"><img align="center" src="./images/hive.png"
style="height:100px;padding-left:15px"></a>
</div>
<hr>
<h3>Why do we archive the HRRR?</h3>
<img align=right class='style11' width=400 style="margin-left:20px" src="./images/June2017_NCAR.PNG">
<p>While you can find current HRRR output at <a href="http://www.nomads.ncep.noaa.gov/">NOMADS</a>,
there is no official HRRR
archive that is publicly and easily available, to our knowledge.
This "gap" in the NOAA data archives will hopefully be filled in
the future, seeing that their vision is that all NOAA environmental
data shall be <i>Discoverable, Accessible, Useable, and Preserved</i>.
We hope to be part of the solution to that goal.
<p>This HRRR archive has been created to support various research
endeavors in the
Department of Atmospheric Sciences at the University of Utah.
Things we do include:
<ol style="padding-left: 60;">
<li>Initialize the Weather Research and Forecast (WRF) model with HRRR analyses as initial
and boundary conditions. <a
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr.html">More info here</a>.
<li>Model verification, where HRRR analyses and forecasts are compared to observed conditions.
<li>Retrospective analysis of high-impact weather events.
<li>Basic statistics of variables (max, min, mean, percentiles).
<li>Potential work with machine learning applications.
</ol>
<p> This archive allows us to perform analysis of the HRRR model performance, independent of the ESRL
folks, for our own research purposes and goals.
<p>Archive methods are becoming more important with increasing requirements by funding agencies to
maintain archives of the data used in research grants. The University of Utah is setting up a
framework for handling such requirements through what they call <a
href="https://hive.utah.edu/concern/generic_works/j9602061v">The Hive</a>. This HRRR archive was
used as a test case for how that framework can be used for very large data sets.</p>
<p>We hope this archive will increase the volume of research and model validation using the HRRR model
that otherwise could not be done. In the future, the HRRR may be archived as part of NOAA's Big Data
Project. This will lead to additional uses of model forecasts and analyses in cloud computing
environments and research done with rapidly updating forecast systems.
<br>
<p> You are welcome to use the archive for your own research, but as a courtesy, please register and
read the "Best Practices" document before downloading from the archive.
<hr>
<h3>Where else can I find archived HRRR data?</h3>
<ul>
<li><a href="http://www.nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/", target='_blank'>
NOMADS</a>: Last two days (main source)
<li><a href="https://console.cloud.google.com/marketplace/product/noaa-public/hrrr", target='_blank'>
<b><i class="fab fa-google"></i> Google Cloud Platform</b></a>:
<b>MOST COMPLETE.</b> 2014-Present. Wow! When did NOAA start this HRRR archive on Google?
They have all <code>sfc</code>, <code>prs</code>, <code>nat</code>, <code>subh</code> and other fields
for the Alaska and CONUS domains that were available.
Unfortunately, the .idx files are not available for dates earlier than around Oct. 2018,
but hey, they have it all!</li>
<li><a href="https://registry.opendata.aws/noaa-hrrr-pds/", target='_blank'>
<b><i class="fab fa-aws"></i> Amazon AWS S3</b></a>: NOAA Big Data Program HRRR archive back to 2014 (missing .idx before January 14th??) (Zarr format coming soon)</li>
<li><a href="https://esrl.noaa.gov/gsd/thredds/catalog/data/retro/hrrr/conus/wrftwo/catalog.html", target='_blank'>
ESRL</a>: 2013-2017, only F01 for 80-m winds and downward solar radiation</li>
<li><a href="https://www.ready.noaa.gov/data/archives/hrrr/README.TXT", target='_blank'>
NOAA ARL</a>: June 2015 to present, selected variables and levels in ARL format for f01</li>
</ul>
</div>
<div id="tab3" class="tab-pane fade">
<img src="./images/HRRR_pando.jpg" align="right" width="250px" class="style11" style="margin-left:20px">
<h3><i class="fa fa-database"></i> Where is the HRRR archive?</h3>
<p>The archive is physically located at the University of Utah
Downtown Data Center in Salt Lake City, Utah. It is hosted on
Utah's Center for High Performance Computing Pando archive storage
system.
<p>This object storage system is similar to Amazon's S3 storage,
but this is less expensive. We currently have a 60 TB allocation
with a storage rate of ~50 GB/day.
<a class="btn btn-danger" href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/Pando_archive/">
Pando Archive Usage
</a>
<p>But you probably aren't interested in it's physical location.
You want to know how to download from the archive. Lucky for you,
the data is publicly available to those outside the University of Utah.
We only ask that you fill out the registration form before downloading.
The registration form helps us keep track of the number of people
who find this data useful, which helps us justify making the archive
available to you.
<a href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_download_register.html"
class="btn btn-danger">
<i class="fa fa-user-plus"></i> Archive Registration</a>
<p>After registering, you will be redirected to the interactive
download page. Click the "Scripting Tips" button for how-to instructions
to download from Pando with wget or cURL. These tips also show how
you can target specific variables you are interested without downloading
the entire file. This will save you space an download time.
<hr>
<h3><img class="style11" style="margin-left:20px" src="./images/FallPando.jpg" align="right"
width="550px">
<i class="fa fa-server"></i> What is the Pando archive?</h3>
<p><b>Pando</b> is a colony of quaking aspen trees in southern Utah,
thought to be the oldest and largest living organism in the world,
linked together by the same root system.
According to <a href="https://en.wikipedia.org/wiki/Pando_%28tree%29">Wikipedia</a>,
Pando is Latin for "I spread".
The name is fitting for the CHPC object storage system
because of its resilience and scalability, like
the tree system.
<p>The underlying software running Pando is
<a href="http://ceph.com/">Ceph</a>, a project supported by RedHat.
While Ceph can be configured in a few ways, in this archive Ceph
manages the data objects with the Amazon S3 API. It is a separate
"island" from the rest of CHPC file system so that output from
other programs cannot write directly to it. Instead, files are
copied to Pando through utilities like rclone or s3cmd.
<p>Pando is built in three parts. All of these can be scaled to
meet needs of the growing archive.
<ul style="padding-left:50px">
<li>9 OSD Servers (16 8TB drives, each) - These contain the data objects.
<li>3 Monitors - Monitors keep a map of the data objects. When a request for data is made, these
monitors are contacted for the object map and return the object's ID.
<li>1 RADOS Gateway node - You, as the client, make requests through this gateway for data
downloads.
</ul>
<center><small><img src="./images/PANDO_Diagram.png" width="550px"><br>Adapted from Sam Liston</small>
</center>
<br>
</div>
<div id="tab4" class="tab-pane fade">
<h3><i class="fa fa-calendar-alt"></i> What days are available to download?</h3>
<p>The HRRR archive begins on 15 July 2016. Browse the interactive
<a href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/hrrr_download.cgi">web download
page</a>
for a visual sense of what data files are available each day.
<p>We store all the operational analyses and forecast for the surface
fields. Unfortunately, we decided the pressure field files
(~315 MB) are too large for us to save the forecast hours. We
store analyses of the experimental HRRR and HRRR Alaska models.
<p> Timeline:
<ul style="padding-left: 60px;">
<strike>
<li>2015-04-18: The HRRR archive is born! Only HRRR analysis hours (f00) were stored for sfc and
prs files.
<li>2016-07-27: Began storing HRRR 15 hour sfc forecasts.
<li>2016-08-24: Began storing HRRR 18 hour sfc forecasts (HRRRv2 became operational).
<li>2016-12-01: Began storing experimental HRRR-Alaska. prs analyses and select sfc variables
for analyses and forecasts.
<li>2016-12-01: Began storing available experimental HRRR surface analyses.
<li>2017-03-01: Moved HRRR archive from local file system to Pando storage.
<li>2017-06-08: Began storing all available HRRR-Alaska sfc variables.
</strike>
<li>2017-12-27: <b>Pando Failed</b> All prior data was lost.</li>
<li>2017-12-28: <b>Pando Rebuilt</b>
<li>2017-02-26: <b>Backfilled to 2016-07-15</b> HRRRv2 data from The Climate Corporation.
<li>2018-07-12: <b>HRRRv3 Upgrade</b> and HRRR-Alaska becomes operational.
<li>2020-12-02: <b>HRRRv4 Upgrade</b>.
</ul>
<a class='btn btn-default btn-lg'
href='https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/HRRR_archive/PRS_missing_incomplete_files.txt'>
<i class="fab fa-firstdraft"></i>
List of Missing PRS Files</a>
<a class='btn btn-default btn-lg'
href='https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/HRRR_archive/SFC_missing_incomplete_files.txt'>
<i class="fab fa-firstdraft"></i>
List of Missing SFC Files</a>
</div>
<div id="tab5" class="tab-pane fade">
<h3><i class="fa fa-users"></i> Who uses the HRRR archive?</h3>
<img align=right class='style11' style="margin-left:15px;margin-bottom:15px"
src='./images/hrrr_users.png' width='550px'>
<p>Many people are interested in the HRRR archive, including yourself
(obviously, or else you wouldn't have Googled <i>HRRR archive</i>).
<p>The initial purpose of this HRRR archive was to serve atmospheric science research at the University
of Utah, particularly my own research for my Master and PhD degrees.
For example, I used HRRR analyses to
<a href="http://journals.ametsoc.org/doi/abs/10.1175/JAMC-D-16-0216.1">initialize WRF
simulations</a>.
<p>The archive is searched for quite often by people like you.
Since this archive has been published online I have received
inquiries and download requests from
National Weather Service employees,
NASA scientists,
researchers at the National Institute of Standards and Technology,
a United State Air Force Captain,
employees at NVIDIA, Lockheed Martin,
commercial wind power companies,
university professors,
graduate students,
airline forecasters,
post docs,
the USDA Agriculture Research Services,
dozens of students working on capstone project,
and many others.
<p>The HRRR archive allows researchers to
use past weather data at high temporal and spatial resolution
without the need to run their own WRF simulations. This is a
huge time saver.
<p>How is the HRRR archive useful to you? Send me an email and let me know.
<div class="alert alert-success">
If this archive is useful to you, please contact the most
important person you know at NCEP and request they develop an
official HRRR archive. Thank you <i class="far fa-smile"></i>
<p><b>Update:</b> NOAA now has an archive on
<a href="https://console.cloud.google.com/marketplace/product/noaa-public/hrrr" target='_blank'>
Google Cloud Platform</a>.
</div>
<div class='well'>
<h3>Known published works that used this HRRR archive:</h3>
<p style='text-indent:-20px'>
Abatzoglou, A.T., C.M. Smith, D.L. Swain, T. Ptak, and C.A. Kolden, 2020:
Population exposure to pre-emptive de-energization aimed at averting wildfires in Northern
California.
<i>Environ. Res. Lett.</i>,
<b>15</b>, 1-8,
<a href='https://doi.org/10.1088/1748-9326/aba135'>https://doi.org/10.1088/1748-9326/aba135</a>
<p style='text-indent:-20px'>
Blaylock, B.K., J.D. Horel, and E.T. Crosman, 2017:
Impact of Lake Breezes on Summer Ozone Concentrations in the Salt Lake Valley.
<i>J. Appl. Meteor. Climatol.</i>,
<b>56</b>, 353-370,
<a href='https://doi.org/10.1175/JAMC-D-16-0216.1'>https://doi.org/10.1175/JAMC-D-16-0216.1</a>
<p style='text-indent:-20px'>
Blaylock, B.K., J.D. Horel, and C. Galli, 2018:
High-Resolution Rapid Refresh Model Data Analytics Derived on the Open Science Grid to Assist
Wildland Fire Weather Assessment.
<i>J. Atmos. Oceanic Technol.</i>,
<b>35</b>, 2213–2227,
<a
href='https://doi.org/10.1175/JTECH-D-18-0073.1'>https://doi.org/10.1175/JTECH-D-18-0073.1</a>.
<p style='text-indent:-20px'>
Blaylock, B.K. and J.D. Horel, 2020:
Comparison of Lightning Forecasts from the High-Resolution Rapid Refresh Model to Geostationary
Lightning Mapper Observations.
<i>Wea. Forecasting</i>,
<b>35</b> , 401-416,
<a href='https://doi.org/10.1175/WAF-D-19-0141.1'>https://doi.org/10.1175/WAF-D-19-0141.1</a>.
<p style="text-indent:-20px">
Bowes, B., J. Sadler, M. Morsy, M. Behl, and J. Goodall, 2019:
Forecasting Groundwater Table in a Flood Prone Coastal City with Long Short-term Memory and
Recurrent Neural Networks.
<i>Water</i>,
<b>11</b>, 1-38,
<a href='https://doi.org/10.3390/w11051098'>https://doi.org/10.3390/w11051098</a>.
<p style="text-indent:-20px">
Cann, M.D. and K. Friedrich, 2020: The Role of Moisture Pathways on Snowfall Amount and
Distribution in the Payette Mountains of Idaho.
<i>Mon. Wea. Rev.</i>,
<b>148</b> 2033–2048,
<a href='https://doi.org/10.1175/MWR-D-19-0350.1'>https://doi.org/10.1175/MWR-D-19-0350.1</a>.
<p style="text-indent:-20px">
Cao, Y., and A.P. Barros, 2020: Weather-Dependent Nonlinear Microwave Behavior
of Seasonal High-Elevation Snowpacks.
<i>Remote Sens.</i>,
<b>12, 3422</b> 1-35,
<a href='http://dx.doi.org/10.3390/rs12203422'>http://dx.doi.org/10.3390/rs12203422</a>.
<p style="text-indent:-20px">
Ditchek, S.D., K.L. Corbosiero, R.G. Fovell, and J. Molinari, 2020:
Electrically Active Diurnal Pulses in Hurricane Harvey (2017).
<i>Mon. Wea. Rev.</i>,
<b>148</b> 2283-2305,
<a href='https://doi.org/10.1175/MWR-D-20-0022.1'>https://doi.org/10.1175/MWR-D-20-0022.1</a>.
<p style="text-indent:-20px">
Handler, S. L., H. D. Reeves, and A. McGovern, 2020:
Development of a Probabilistic Subfreezing Road Temperature Nowcast and Forecast Using Machine Learning.
<i>Wea. Forecasting</i>,
<b>35</b>, 1845-1863,
<a href="https://doi.org/10.1175/WAF-D-19-0159.1">https://doi.org/10.1175/WAF-D-19-0159.1</a>.
<p style="text-indent:-20px">
Harkema, S., 2019:
Improving Situational Awareness of Heavy-Snowfall Using the Geostationary Lightning Mapper. The
University of Alabama in Huntsville, ProQuest Dissertations Publishing, 2019. 22592216.
<a href="http://search.proquest.com/openview/599d54316495fd26d98d8e8e58bfcf72/1?pq-origsite=gscholar&cbl=18750&diss=y">
ProQuest link</a>
<p style="text-indent:-20px">
Harkema, S.S., E.B. Berndt, and C.J. Schultz, 2020:
Characterization of Snowfall Rates, Totals, and Snow-to-Liquid Ratios in Electrified Snowfall
Events Identified by the Geostationary Lightning Mapper.
<i>Wea. Forecasting</i>,
<b>35</b>, 673-689,
<a href='https://doi.org/10.1175/WAF-D-19-0126.1'> https://doi.org/10.1175/WAF-D-19-0126.1</a>
</p>
<p style="text-indent:-20px">
Jackson, R., S. Collis, T. Lang, C. Potvin, and T. Munson, 2020:
PyDDA: A Pythonic Direct Data Assimilation Framework for Wind Retrievals.
<i>Journal of Open Research Software</i>,
<b>8(1)</b>, 20.
<a href='http://doi.org/10.5334/jors.264'>http://doi.org/10.5334/jors.264</a>.
<p style='text-indent:-20px'>
Jain, R., Luo, X., Sever, G., Hong, T., and Catlett, C., 2018:
Representation and evolution of urban weather boundary conditions in downtown Chicago.
<i>J. of Building Performance Simulation</i>,
<a
href='https://doi.org/10.1080/19401493.2018.1534275'>https://doi.org/10.1080/19401493.2018.1534275</a>.
<p style="text-indent:-20px">
Lee, T.R., M. Buban, D.D. Turner, T.P. Meyers, and C.B. Baker, 2019:
Evaluation of the High-Resolution Rapid Refresh (HRRR) Model Using Near-Surface Meteorological
and Flux Observations from Northern Alabama.
<i>Wea.Forecasting</i>,
<b>34</b>, 635-663,
<a href='https://doi.org/10.1175/WAF-D-18-0184.1 '>https://doi.org/10.1175/WAF-D-18-0184.1 </a>.
<p style='text-indent:-20px'>
McCorkle, T.A., J.D. Horel, A.A. Jacques, and T. Alcott, 2018:
Evaluating the Experimental High-Resolution Rapid Refresh-Alaska Modeling System Using USArray
Pressure Observations.
<i>Wea. Forecasting</i>,
<b>33</b>, 933-953,
<a href='https://doi.org/10.1175/WAF-D-17-0155.1'>https://doi.org/10.1175/WAF-D-17-0155.1</a>.
<p style="text-indent:-20px">
McNicholas, C. and C.F. Mass, 2018: Impacts of Assimilating Smartphone Pressure Observations on
Forecast Skill during Two Case Studies in the Pacific Northwest.
<i>Wea. Forecasting</i>,
<b>33</b>, 1375-1396,
<a href='https://doi.org/10.1175/WAF-D-18-0085.1'>https://doi.org/10.1175/WAF-D-18-0085.1</a>.
<p style="text-indent:-20px">
Moore, B.J., A.B. White, D.J. Gottas, and P.J. Neiman, 2020:
Extreme precipitation events in northern California during winter 2016-2017: Multiscale analysis
and climatological perspective.
<i>Wea. Forecasting</i>,
<b>148</b>, 1049-1074,
<a href='https://doi.org/10.1175/MWR-D-19-0242.1'>https://doi.org/10.1175/MWR-D-19-0242.1</a>.
<p style="text-indent:-20px">
Poolla, C., and A. K. Ishihara, 2018:
Localized solar power prediction based on weather data from local history and global forecasts.
<i>45th IEEE Photovoltaic Specialists Conference</i>.
<a href='https://arxiv.org/abs/1808.08657'>Link</a>.
<p style="text-indent:-20px">
Radford, J.T., G.M. Lackmann, and M.A. Baxter, 2019:
An evaluation of snowband predictability in the High-Resolution Rapid Refresh.
<i>Wea. Forecasting</i>,
<b>34</b>, 1477-1494,
<a href='https://doi.org/10.1175/WAF-D-19-0089.1'>https://doi.org/10.1175/WAF-D-19-0089.1</a>.
<p style="text-indent:-20px">
Radhakrishnan, C. and V. Chandrasekar, 2020:
CASA Prediction System over Dallas Fort Worth Urban network: Blending of nowcasting and
high-resolution Numerical Weather Prediction model.
<i>J. Atmos. Technol.</i>,
<b>37</b>, 211-228,
<a href='https://doi.org/10.1175/JTECH-D-18-0192.1'>https://doi.org/10.1175/JTECH-D-18-0192.1</a>.
<p style="text-indent:-20px">
Reynolds, D., J. M. Pflug, J. D. Lundquist, 2020:
Evaluating Wind Fields for Use in Basin-Scale Distributed Snow Models.
<i>Water Resources Research</i>,
<b>In Press</b>
<a href='https://doi.org/10.1029/2020WR028536'>https://doi.org/10.1029/2020WR028536</a>.
<p style="text-indent:-20px">
Rosenow, A.A., K. Howard, and J. Meitin, 2018:
Gap-Filling Mobile Radar Observations of a Snow Squall in the San Luis Valley.
<i>Mon. Wea. Rev.</i>,
<b>146</b>, 2469-2481,
<a href='https://doi.org/10.1175/MWR-D-17-0323.1'>https://doi.org/10.1175/MWR-D-17-0323.1</a>.
<p style="text-indent:-20px">
Zick, S. E., 2020:
Quantifying Extreme Precipitation Forecasting Skill in High-Resolution Models Using Spatial
Patterns: A Case Study of the 2016 and 2018 Ellicott City Floods
<i>Atmosphere</i>,
<b>11</b>, 1-25,
<a href='https://doi.org/10.3390/atmos11020136'>https://doi.org/10.3390/atmos11020136</a>.
</div>
</div>
<div id="tab6" class="tab-pane fade">
<h3><i class="fa fa-paint-brush"></i> Gallery</h3>
<h3>Online Tools</h3>
<center>
<div class="row" id="content">
<div class=" col-md-1">
</div>
<div class=" col-md-2">
<a class='btn btn-danger' role='button'
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_golf.html"
style="width:100%"> <i class="fa fa-map-marker"></i> Point Forecast</a>
</div>
<div class="col-md-2">
<a class='btn btn-danger' role='button'
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_fires.html"
style="width:100%"><i class="fas fa-fire-extinguisher"></i> Fires Forecast</a>
</div>
<div class="col-md-2">
<a class='btn btn-danger' role='button'
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_custom.html"
style="width:100%"> <i class="far fa-map"></i> Custom Maps</a>
</div>
<div class="col-md-2">
<a class='btn btn-danger' role='button'
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/hrrrX-hrrr.cgi"
style="width:100%"> <i class="fa fa-map"></i> Compare Maps</a>
</div>
</div>
</center>
<hr>
<img src='./images/OSG_logo.jpg' width='250px' align=RIGHT>
<h3>HRRR Statistics with Open Science Grid</h3>
<p>I'm using the Open Science Grid to calculate long term statistics,
calculated for +/- 15 days for every hour of the day for all
the HRRR files in our archive.
I calculate the following percentiles [0,1,2,3,4,5,10,25,33,50,66,75,90,95,96,97,98,99,100].
<p><a href="https://github.com/blaylockbk/pyBKB_v2/tree/master/OpenScienceGrid">Tell me more!</a>
<div class='row'>
<div class='col-md-6'>
<p style="text-align:center;">
<img src="./images/HRRR_gallery/HRRR_wind_95percentile.png" width="95%" class='style11'>
<figcaption style="text-align:center;"></figcaption>
</div>
<div class='col-md-6'>
<p style="text-align:center;">
<img src="./images/HRRR_gallery/HRRR_freezing_percentiles.png" width="95%" class='style11'>
<figcaption style="text-align:center;"></figcaption>
</div>
</div>
<p>The HRRR analyses mean winds are comparable with the results of
<a href="http://www.sciencedirect.com/science/article/pii/S0960148116309363" target="_blank">
James et al. 2017</a>.
<hr>
<h3>Plot HRRR Data with Python</h3>
<p>A handy-dandy <a
href="https://github.com/blaylockbk/pyBKB_v2/blob/master/demos/plot_HRRR_variables_on_map.ipynb">Jupyter
Notebook</a>
for making various maps.
<p>Also, a <a
href="https://github.com/blaylockbk/pyBKB_v2/blob/master/BB_goes16/mapping_GOES16_data.ipynb">
notebook</a> on overlaying GOES-16 images with HRRR data.
<blockquote class="twitter-tweet" data-lang="en">
<p lang="en" dir="ltr">Overlaying HRRR and <a
href="https://twitter.com/hashtag/GOES16?src=hash&ref_src=twsrc%5Etfw">#GOES16</a> data
in Python <a href="https://t.co/wYY4TaxWpQ">pic.twitter.com/wYY4TaxWpQ</a></p>— Brian
Blaylock (@blaylockbk) <a
href="https://twitter.com/blaylockbk/status/911327100560273408?ref_src=twsrc%5Etfw">September
22, 2017</a>
</blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<hr>
<h3>Initialize WRF</h3>
<p>I use the HRRR analyses to initialize WRF's boundary and initial conditions.
Check out my instructions for initializing WRF with the HRRR here:
<a href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr.html">
https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr.html</a>
<hr>
<h3>2017 Solar Eclipse</h3>
<blockquote class="twitter-tweet" data-lang="en">
<p lang="en" dir="ltr">Solar radiation from <a
href="https://twitter.com/mesowestutah?ref_src=twsrc%5Etfw">@mesowestutah</a>, operational
HRRR, and experimental HRRR during <a
href="https://twitter.com/hashtag/Eclipse2017?src=hash&ref_src=twsrc%5Etfw">#Eclipse2017</a>
Thanks, <a href="https://twitter.com/NOAA_ESRL?ref_src=twsrc%5Etfw">@NOAA_ESRL</a> <a
href="https://t.co/hHoBNWVW2m">pic.twitter.com/hHoBNWVW2m</a></p>— Brian Blaylock
(@blaylockbk) <a
href="https://twitter.com/blaylockbk/status/900034286916485120?ref_src=twsrc%5Etfw">August 22,
2017</a>
</blockquote>
<script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<hr>
<h3>Model Verification</h3>
<p><b>2-day Time Series</b>: Time series graphs of observed values for select
MesoWest stations and HRRR values for f00, f06, f12, and f18
for the last two days.
Check out the <a href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_golf.html">
HRRR Point Forecasts</a> page, and select the clock next to a
station to view verification of the HRRR model for the last two
days.
<center>
<div><img src='https://home.chpc.utah.edu/~u0553130/oper/HRRR_anlys/KSLC/h20.png' width="450px">
</div>
</center>
<br>
<p><b>"Hovmoller" Diagram:</b> Imagine a Hovmoller diagram with
forecast hour on the y-axis and valid time on the x-axis.
These are created for a variable over each HRRR
forecast hour and compare with the observed value.
On the <a href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/hrrr_golf.html">
HRRR Point Forecasts</a> page, select the clock next to a
station and click "Hovmoller".
<a href="https://github.com/blaylockbk/pyPlots_v2/blob/master/HRRR/HRRR_hovomoller.py"><i
class="fa fa-github"></i> GitHub Code</a>
<center>
<div><img src='https://home.chpc.utah.edu/~u0553130/oper/HRRR_hovmoller/KSLC/KSLC_air_temp.png'
width="450px">
<img src='https://home.chpc.utah.edu/~u0553130/oper/HRRR_hovmoller/KSLC/KSLC_wind_speed.png'
width="450px">
</div>
</center>
<hr>
<hr>
<p>Quantify inversion strength during the Utah Fine Particulate Air Quality Study, January 2017.</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/jup9D_xbG1o?rel=0" frameborder="0"
allowfullscreen></iframe>
<hr>
<p>New England Snow Storm, March 13-14 2017</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/73vZOPvtQDc?rel=0" frameborder="0"
allowfullscreen></iframe>
<hr>
<p>Great Salt Lake surface temperature (before HRRR assimilation was fixed)</p>
<iframe width="560" height="315" src="https://www.youtube.com/embed/PC72-dsTrS4?rel=0" frameborder="0"
allowfullscreen></iframe>
<hr>
<h3>Other People</h3>
<p> Matt Lammers from NASA combined HRRR 925 mb winds with
GMI and IMERG precipitation data for visualization of Hurricanes.
<ul style="padding-left:50px">
<li><a href="https://storm.pps.eosdis.nasa.gov/storm/cesium/Spiral.html">Harvey</a>
<li><a href="https://storm.pps.eosdis.nasa.gov/storm/cesium/SpiralLongMatthew.html">Matthew</a>
</ul>
</div>
<div id="tab7" class="tab-pane fade">
<h3><i class="fa fa-desktop"></i> Tips for CHPC Users</h3>
<p>Do you work/study at the University of Utah, too? Cool! Go Utes!!
Here are a few helpful tips that will make your life easier. That is,
if your life involves using HRRR data from our handy archive.
Please share with me cool stuff you learn!
<hr>
<h3>I have stuff in my own Pando buckets. How can I view those files in a browser?</h3>
<p>The HRRR download page is a custom-built interface for accessing files in the HRRR buckets,
but what if you have your own buckets in Pando? You're in luck,
because I created this more generic web interface for viewing and downloading
file objects in your Pando buckets that look more like the Apache file explorer.
<p>
<center><a
href='https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/cgi-bin/generic_pando_download.cgi'
class='btn btn-success'>
<i class="fas fa-list-ul"></i> Pando Content Viewer
</a></center>
<hr>
<h3>How full is our Pando allocation?</h3>
<p> View the amount of space used on our Pando. <a class='btn btn-primary'
href="https://home.chpc.utah.edu/~u0553130/Brian_Blaylock/Pando_archive/">Pando Allocation</a>
</p>
<hr>
<h3>Is the archive on the local file system?</h3>
<p>Yes, we keep a backup of the HRRR archive on horel-group7.
<div class="well well-sm"><span
style="font-family:monospace">/uufs/chpc.utah.edu/common/home/horel-group7/Pando/</span></div>
<p>Then navigate to the HRRR directory you are interested:
<ul style="padding-left: 60;">
<li>hrrr
<li>hrrrX
<li>hrrrak
<li>GOES16
</ul>
<hr>
<h3>Want a behind-the-scenes tour of the HRRR archive?</h3>
<p>The process we use to downloading HRRR and move HRRR to the
Pando archive system is documented on GitHub.
<a class='btn btn-primary' href="https://github.com/blaylockbk/HorelS3-Archive/">Horel-S3 <i
class="fab fa-github fa-fw"></i></a>
<hr>
<h3>How do I get the HRRR data if what I'm looking for is compressed?</h3>
<div class="alert alert-danger">
There is still some HRRR data that is compressed on the local file system
but it's much quicker to get it from PANDO. Still, this may be
useful for getting other stuff out of the compressed archive, so I'll
keep it here as a reference. (It took me many web searches before I
figured out how to make this work, so I don't want to lose this
token of knowledge.)
</div>
<p>If you find that what you're looking for is compressed, the
following instructions will help get what you need:
<ol style="padding-left:60px">
<li>
<p>Copy the <b>models.tar.gz</b> file to your own space.
Yes, this is large (~20 GB). It contains
all the model data we archive (I warned you that this was
inefficient).
<li>
<p>Before untarring the entire directory, you can check if
it contains the files you need with something like this:
<div class="well well-sm" style="font-family:'Courier';">less -p /hrrr/hrrr.t models.tar.gz
</div>