ResStock-HPXML: HES Workflow Generator #252

aspeake · 2021-10-27T18:40:03Z

Fixes #243

Pull Request Description

New workflow generator that translates ResStock-generated xml files to HES json inputs, which are passed to the OS-HEScore workflow to simulate buildings.

Translator: https://github.com/NREL/hescore-hpxml
OS Workflow: https://github.com/NREL/OpenStudio-HEScore

This PR adds a new workflow generator that requires a new argument os_hescore_directory to point to a local OpenStudio-HEScore branch. The hescore-hpxml translator is installed as part of the singularity/Docker image (https://github.com/NREL/docker-hescore-hpxml-openstudio/blob/main/Dockerfile). This is accompanied by changes to the resstock repo from where the hescore-hpxml translator will be called (PR is TBA)

Checklist

Not all may apply

Code changes (must work)
Tests exercising your feature/bug fix (check coverage report on CircleCI build -> Artifacts)
All other unit tests passing
Update validation for project config yaml file changes
Update existing documentation
Run a small batch run to make sure it all works (local is fine, unless an Eagle specific feature)
Add to the changelog_dev.rst file and propose migration text in the pull request

…es workflow

…or args

…-v3-hes

aspeake · 2021-12-10T23:25:41Z

@nmerket I think this is ready for an initial review. The new workflow generator has been built to work just with local buildstock_docker at the moment, and I plan to have the next round of changes be focused on generalizing for Eagle/AWS.

There are some CI errors right now which appear to spawn from the switch to OS-3.3.0 on the restructure-v3 branch.

nmerket

A lot of these comments are for me. There's definitely some growing pains in buildstockbatch to accommodate HEScore here. It turns out making a fully extensible batch model articulation and simulation platform is kind of hard.

nmerket · 2021-12-14T19:05:56Z

buildstockbatch/base.py

+        if self.cfg['workflow_generator']['type'] == 'residential_hpxml_hes':
+            self.os_hescore_dir = path_rel_to_file('',self.cfg['workflow_generator']['args']['openstudio_hescore']['os_hescore_directory'])
+        else:
+            self.os_hescore_dir = None


Having this kind of conditional thing in the base class is concerning. I'm not sure what to do about it, though. Ideally, this would be handled with some elegant inheritance mechanism, but we're pretty far off from that here.

nmerket · 2021-12-14T19:08:08Z

buildstockbatch/base.py

-        if self.cfg['workflow_generator']['type'] == 'residential_hpxml':
+        if 'residential_hpxml' in self.cfg['workflow_generator']['type']:


I learned something today that you can search for a substring using in.

nmerket · 2021-12-14T19:09:28Z

buildstockbatch/localdocker.py

+        # FIXME temporary docker image for testing
+        # return 'nrel/hescore-hpxml-openstudio'
        return 'nrel/openstudio:{}'.format(self.os_version)


Need to figure out something more permanent here.

nmerket · 2021-12-14T19:10:17Z

buildstockbatch/localdocker.py

-    def run_building(cls, project_dir, buildstock_dir, weather_dir, docker_image, results_dir, measures_only,
-                     n_datapoints, cfg, i, upgrade_idx=None):
+    def run_building(cls, project_dir, buildstock_dir, weather_dir, os_hescore_dir, docker_image, results_dir,
+                     measures_only, n_datapoints, cfg, i, upgrade_idx=None):


I see this was necessary to get it working, but I don't like it.

nmerket · 2021-12-14T19:15:33Z

buildstockbatch/localdocker.py

+        # OS-HEScore measure directories
+        if os_hescore_dir and os.path.exists(os_hescore_dir):
+            bind_mounts += [(os.path.join(os_hescore_dir, 'rulesets'), 'OpenStudio-HEScore/rulesets', 'ro'),
+                            (os.path.join(os_hescore_dir, 'hpxml-measures'), 'OpenStudio-HEScore/hpxml-measures', 'ro'),
+                            (os.path.join(os_hescore_dir, 'weather'), 'OpenStudio-HEScore/weather', 'ro')]


Just thinking out loud here: I wonder if we could have the workflow generator define bind mounts in some generic way that could be used by both eagle.py and localdocker.py.

nmerket · 2021-12-14T19:16:12Z

buildstockbatch/localdocker.py

-        sim_out_tarfile_name = os.path.join(sim_out_dir, 'simulations_job0.tar.gz')
-        logger.debug(f'Compressing simulation outputs to {sim_out_tarfile_name}')
-        with tarfile.open(sim_out_tarfile_name, 'w:gz') as tarf:
-            for dirname in os.listdir(sim_out_dir):
-                if re.match(r'up\d+', dirname) and os.path.isdir(os.path.join(sim_out_dir, dirname)):
-                    tarf.add(os.path.join(sim_out_dir, dirname), arcname=dirname)
-                    shutil.rmtree(os.path.join(sim_out_dir, dirname))
+        # FIXME temporarily comment out for testing
+        # sim_out_tarfile_name = os.path.join(sim_out_dir, 'simulations_job0.tar.gz')
+        # logger.debug(f'Compressing simulation outputs to {sim_out_tarfile_name}')
+        # with tarfile.open(sim_out_tarfile_name, 'w:gz') as tarf:
+        #     for dirname in os.listdir(sim_out_dir):
+        #         if re.match(r'up\d+', dirname) and os.path.isdir(os.path.join(sim_out_dir, dirname)):
+        #             tarf.add(os.path.join(sim_out_dir, dirname), arcname=dirname)
+        #             shutil.rmtree(os.path.join(sim_out_dir, dirname))


Remember to uncomment this at some point and if it breaks something, fix it.

buildstockbatch/workflow_generator/test_workflow_generator.py

buildstockbatch/workflow_generator/residential_hpxml_hes.py

joseph-robertson · 2022-01-26T16:22:19Z

buildstockbatch/workflow_generator/residential_hpxml_hes.py

+            osw['steps'][0]['arguments']['os_hescore_directory'] = '../../OpenStudio-HEScore'
+
+        # Add measure path for reporting measure
+        osw['measure_paths'] = ['OpenStudio-HEScore/hpxml-measures'] + osw['measure_paths']


osw['measure_paths'] += ['OpenStudio-HEScore/hpxml-measures']?

OpenStudio-HEScore path needs to be first to prioritize the OS-HEScore reporting measure

buildstockbatch/workflow_generator/test_workflow_generator.py

…-v3-hes

github-actions · 2022-02-17T00:44:43Z

File	Coverage
All files	`85%`	✅
base.py	`87%`	✅
eagle.py	`74%`	✅
exc.py	`57%`	✅
localdocker.py	`61%`	✅
postprocessing.py	`84%`	✅
utils.py	`96%`	✅
sampler/base.py	`79%`	✅
sampler/downselect.py	`33%`	✅
sampler/precomputed.py	`93%`	✅
sampler/residential_quota.py	`83%`	✅
test/test_validation.py	`96%`	✅
workflow_generator/base.py	`90%`	✅
workflow_generator/commercial.py	`53%`	✅
workflow_generator/residential.py	`96%`	✅
workflow_generator/residential_hpxml.py	`71%`	✅
workflow_generator/residential_hpxml_hes.py	`75%`	✅

Minimum allowed coverage is 33%

Generated by 🐒 cobertura-action against d8fa15f

aspeake · 2022-02-17T01:13:36Z

To run the workflow using this branch:

Branches:
resstock: develop
Openstudio-HEScore: hes_translator https://github.com/NREL/OpenStudio-HEScore/pull/337 (subtrees NREL/hescore-hpxml#151)

Project File: https://github.com/NREL/buildstockbatch/blob/restructure-v3-hes/project_resstock_national_hes.yml

Docker Image: https://github.com/NREL/docker-hescore-hpxml-openstudio/blob/main/Dockerfile

@nmerket

nmerket · 2022-03-01T22:12:55Z

@aspeake Got it to work with some minor changes. Thanks for getting it to this point.

I'd like to make some more structural changes to buildstockbatch to accommodate this workflow and hopefully provide more flexibility for other workflows such as ComStock. I'd like to run this by @asparke2 and @joseph-robertson to get their thoughts.

Bind Mounts

What directories get mounted where in the container is driven by what workflow you're running. ResStock legacy, ResStock-HPXML, ComStock, and now ResStock-HEScore all have potentially slightly different folders that need to be mounted in. I'm a little murky on how we're handling this in all the implementations we have going already. It's probably a lot of if blocks detecting different workflows. I think we can move these bind mount definitions into the workflow generator classes. It makes sense that they live there. Then in localdocker.py and eagle.py we read them and build the container mount arguments for whatever container platform we're using (docker or singularity).

Custom container image built at runtime

We'd like to use the hescore-hpxml that comes with OpenStudio-HEScore. That way everything is in sync. For it to work well it needs to be installed in the docker image so the right dependencies and module lookups happen correctly. The hescore-hpxml package isn't really meant to be run as a standalone script. The problem is that we don't know the hescore-hpxml version until runtime, so we have a complicated multistep process that's prone to error to build a docker image with the hescore-hpxml package in it.

I think we can make a general capability in buildstockbatch that allows defining the image to run in three ways:

No image provided. Uses default.
User specifies a docker image from Docker Hub or another container registry. This is pretty easy.
User points to a Dockerfile somewhere that is used to build an image before running it.

Another idea is that the docker/singularity image should be defined in the workflow generator class. In general, the workflow you're running is what drives the image needed. Not sure if this is a good idea or bad idea.

Building a singularity image on the fly is a little trickier, but might still be possible. I know ComStock is using custom Singularity images on Eagle. I'm pretty sure they pre-build them. We will probably have to do the same.

This reverts commit 0ac675a.

aspeake · 2022-09-22T21:06:34Z

Some minor changes may be needed in the workflow generator to align with the latest approach for in residential_hpxml: #302

nmerket · 2022-09-22T22:31:01Z

Thanks for the heads up @aspeake. Is that something you have time to do?

aspeake · 2022-09-22T22:35:03Z

@nmerket I can probably find some time in October to do so, however I think it may require updating all the relevant branches so that we can properly test the HES workflow.

nmerket · 2023-07-07T23:05:00Z

Replaced with #378

aspeake added 2 commits October 5, 2021 13:08

make copy of residential_hpxml workflow generator to get started on h…

45ee2f9

…es workflow

first pass at connecting OS-HEScore using filepath yml argument

a1efb5a

aspeake linked an issue Oct 27, 2021 that may be closed by this pull request

ResStock-HEScore workflow generator #243

Open

aspeake added 10 commits November 11, 2021 08:51

try to pull local Docker image first

a270ff6

update yml field name; some updates to HES workflow generator

d8cd306

fix a couple of osw arguments; temporary changes for testing

7483a12

mount OS-HEScore weather directory; update hardcoded workflow generat…

a748763

…or args

new argument passed to BuildExistingModel

570d5cd

style fixes

633ae54

fix check for OS_HEScore directory path

bd2ba32

Merge remote-tracking branch 'origin/restructure-v3' into restructure…

21a74ef

…-v3-hes

output timeseries for hes workflow generator

ced0abc

change opestudio docker image

0b5324e

aspeake requested a review from nmerket December 10, 2021 23:21

nmerket reviewed Dec 14, 2021

View reviewed changes

aspeake added 6 commits December 15, 2021 12:24

apply defaults in BuildExistingModel

553d5fe

updated HEScoreHPXML arguments

20317ea

unit test for hes workflow generator

ed499b4

use bool types in argument dict

110742e

inherit residential_hpxml in residential_hpxml_hes workflow generator

708b8ca

specify OS-HEScore measure path to use the correct reporting measure

f81e138

joseph-robertson reviewed Jan 26, 2022

View reviewed changes

buildstockbatch/workflow_generator/residential_hpxml_hes.py Outdated Show resolved Hide resolved

joseph-robertson reviewed Jan 26, 2022

View reviewed changes

buildstockbatch/workflow_generator/test_workflow_generator.py Outdated Show resolved Hide resolved

aspeake added 3 commits January 26, 2022 10:57

update test for hpxml hes workflow generator

a209fe6

style fixes

54f7f22

Merge remote-tracking branch 'origin/restructure-v3' into restructure…

c001c97

…-v3-hes

aspeake requested a review from nmerket February 1, 2022 21:26

aspeake marked this pull request as ready for review February 8, 2022 20:22

aspeake added 2 commits February 16, 2022 17:42

temporary yml for hes workflow testing

8bc4b4f

fix relative path for OS-HEScore directory

a22e02c

mounting into /opt/hescore-hpxml

5bdb4e5

Base automatically changed from restructure-v3 to develop March 2, 2022 18:06

nmerket and others added 7 commits March 2, 2022 11:39

keeping the whole batch from failing on a single simulation failure

955f610

Merge branch 'develop' into restructure-v3-hes

0a91a21

Merge remote-tracking branch 'origin/develop' into restructure-v3-hes

21fcba0

minor formatting change

5a141be

building hescore image on the fly

2a00232

adding resstock hescore reporting measure

0ac675a

Revert "adding resstock hescore reporting measure"

82f4c7f

This reverts commit 0ac675a.

aspeake added 9 commits October 25, 2022 14:57

Merge remote-tracking branch 'origin/develop' into restructure-v3-hes

afeaa40

add buildarg to hes docker image, minor updates to hes workflow yml

35ba146

break up yml validation method in order to edit schema in child class

7cd801c

Merge remote-tracking branch 'origin/develop' into restructure-v3-hes

e256c7e

style fixes

653f6d2

more style fixes

7bc8030

update hes workflow generator test

50125e3

Fix missing os_hes_dir variable in some cases

0af1325

remove debug statement

d8fa15f

nmerket mentioned this pull request Jul 7, 2023

HES Workflow Generator, try 2 #378

Open

8 tasks

nmerket added a commit that referenced this pull request Jul 7, 2023

manually porting over changes from #252

3bf2adb

nmerket closed this Jul 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ResStock-HPXML: HES Workflow Generator #252

ResStock-HPXML: HES Workflow Generator #252

aspeake commented Oct 27, 2021 •

edited

Loading

aspeake commented Dec 10, 2021

nmerket left a comment

nmerket Dec 14, 2021

nmerket Dec 14, 2021

nmerket Dec 14, 2021

nmerket Dec 14, 2021

nmerket Dec 14, 2021

nmerket Dec 14, 2021

joseph-robertson Jan 26, 2022

aspeake Jan 26, 2022

github-actions bot commented Feb 17, 2022 •

edited

Loading

aspeake commented Feb 17, 2022 •

edited

Loading

nmerket commented Mar 1, 2022

aspeake commented Sep 22, 2022 •

edited

Loading

nmerket commented Sep 22, 2022

aspeake commented Sep 22, 2022

nmerket commented Jul 7, 2023

		if self.cfg['workflow_generator']['type'] == 'residential_hpxml':
		if 'residential_hpxml' in self.cfg['workflow_generator']['type']:

ResStock-HPXML: HES Workflow Generator #252

ResStock-HPXML: HES Workflow Generator #252

Conversation

aspeake commented Oct 27, 2021 • edited Loading

Pull Request Description

Checklist

aspeake commented Dec 10, 2021

nmerket left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Feb 17, 2022 • edited Loading

aspeake commented Feb 17, 2022 • edited Loading

nmerket commented Mar 1, 2022

Bind Mounts

Custom container image built at runtime

aspeake commented Sep 22, 2022 • edited Loading

nmerket commented Sep 22, 2022

aspeake commented Sep 22, 2022

nmerket commented Jul 7, 2023

aspeake commented Oct 27, 2021 •

edited

Loading

github-actions bot commented Feb 17, 2022 •

edited

Loading

aspeake commented Feb 17, 2022 •

edited

Loading

aspeake commented Sep 22, 2022 •

edited

Loading