Failed to Infer Measure Name

[kflemin]

When running the openstudio CLI measure -u call on a measure, I often get this error: "Failed to infer measure name from '/path/to/measure/measure.rb'". I get it on Python measures as well as Ruby measures. This error also ends up in the xml file in an tag. For example: https://github.com/NREL/OpenStudio-workflow-gem/blob/39c84e45f152446a92379a501807e2691dba15ab/spec/files/script_error_osw/measures/IncreaseWallRValue/measure.xml#L4

Measures contains such errors do not get uploaded to the BCL. I don't think they work in the OS apps either.

Ruby code that triggers the error:

OpenStudio/ruby/engine/RubyEngine.cpp

Lines 154 to 203 in ba1b94b

	std::string RubyEngine::inferMeasureClassName(const openstudio::path& measureScriptPath) {
	auto inferClassNameCmd = fmt::format(R"ruby(
	ObjectSpace.garbage_collect
	ObjectSpace.garbage_collect
	# Measure should be at root level (not inside a module) so we can just get constants
	measurePath = '{}'
	prev = Object.constants
	# puts "prev = #{{prev}}"
	load measurePath # need load in case have seen this script before
	just_defined = Object.constants - prev
	# puts "just_defined = #{{just_defined}}"
	just_defined.select!{{|c| Object.const_get(c).ancestors.include?(OpenStudio::Measure::OSMeasure)}}
	# puts "just_defined, filtered = #{{just_defined}}"
	if just_defined.empty?
	raise "Unable to extract OpenStudio::Measure::OSMeasure object from " +
	measurePath + ". The script should contain a class that derives " +
	"from OpenStudio::Measure::OSMeasure and should close with a line stating " +
	"the class name followed by .new.registerWithApplication."
	end
	if just_defined.size > 1
	raise "Found more than one OSMeasure at #{{measurePath}}: #{{just_defined}}"
	end
	c = just_defined[0]
	class_info = Object.const_get(c)
	$measure_name = class_info.to_s
	# Undef what we loaded
	just_defined.each {{|x| Object.send(:remove_const, x) }}
	ObjectSpace.garbage_collect
	ObjectSpace.garbage_collect
	)ruby",
	measureScriptPath.generic_string());
	std::string className;
	try {
	exec(inferClassNameCmd);
	ScriptObject measureClassNameObject = eval("$measure_name");
	// measureClassNameObject = rubyEngine->eval(fmt::format("{}.new()", className));
	// ScriptObject measureClassNameObject = rubyEngine->eval(inferClassName);
	className = getAs<std::string>(measureClassNameObject);
	} catch (const RubyException& e) {
	auto msg = fmt::format("Failed to infer measure name from {}: {}\nlocation={}", measureScriptPath.generic_string(), e.what(), e.location());
	fmt::print(stderr, "{}\n", msg);
	}
	return className;
	}

Python code that triggers the error:

OpenStudio/python/engine/PythonEngine.cpp

Lines 286 to 310 in eb043d2

	auto inferClassNameCmd = fmt::format(R"python(
	import importlib.util
	import inspect
	spec = importlib.util.spec_from_file_location(f"throwaway", "{}")
	module = importlib.util.module_from_spec(spec)
	spec.loader.exec_module(module)
	class_members = inspect.getmembers(module, lambda x: inspect.isclass(x) and issubclass(x, openstudio.measure.OSMeasure))
	assert len(class_members) == 1
	measure_name, measure_typeinfo = class_members[0]
	)python",
	measureScriptPath.generic_string());
	std::string className;
	try {
	exec(inferClassNameCmd);
	ScriptObject measureClassNameObject = eval("measure_name");
	className = *getAs<std::string*>(measureClassNameObject);
	} catch (const std::runtime_error& e) {
	auto msg = fmt::format("Failed to infer measure name from {}: {}", measureScriptPath.generic_string(), e.what());
	fmt::print(stderr, "{}\n", msg);
	}
	return className;
	}

I'd like to understand better why this error is being triggered...I can't tell exactly from the code. It would seem like there's a typo or not-recommended pattern in the class name, but that doesn't actually seem to be the case.

Another piece of information: I can sometimes update a measure with openstudio measure -u which triggers the error (in addition to doing other cleanup with the files and checksums but doesn't touch name/class_name), remove the error from the measure.xml, re-update the measure with openstudio measure -u again, and now it works?!

thanks!

[macumber]

I think this is related to #5212

[DavidGoldwasser]

I've added this and related measure to OS GItHub project and tagged for 3.9.0. One possible related behavior I saw in the past and I want to test if still happens, is that once an error is in the XML file (link example described at top of this issue) you can't clear it out even after measure is fixed with openstudio measure -u Instead the XML file has to be opened in text editor and the error removed by hand. This has created confusion at times when not know why fix to measure wasn't working.

[kbenne]

@kflemin the Measure that you linked as an example does not have an end statement closing the Measure class. There is even a comment about not closing the class, so it seems deliberate. Do you know why this is? When I close the class I am not able to reproduce the issue, but when I leave the class open I arrive at the same kind of failure that is descrbed here. And I would expect that failure, because the syntax is not valid as it is written.

[kbenne]

Expanding on this a bit, it looks like all of the measures in https://github.com/NREL/OpenStudio-workflow-gem/blob/39c84e45f152446a92379a501807e2691dba15ab/spec/files/script_error_osw intentionally have various errors for testing purposes. I don't think all of these will cause fatal errors (I tested a number of them by calling measure -u after making small changes), but in the case when the measure syntax is invalid, the measure update implementation breaks down because the core of the approach is to parse and load the measure script.

Having said all of this, if someone has an example measure that appears to be valid that we can use to reproduce this issue, I would be interested in it.

[kbenne]

From @kflemin, another measure that bubbles up the reported issue is https://github.com/pnnl/openstudio-building-energy-standard-measures-gem/tree/master/lib/measures/GenerateConStrainReport.

I was able to reproduce the problem and found that this Measure is trying to load the ConStrain, which we do not include with OpenStudio. As a result OpenStudio fails to load/parse the Measure and the Measure update process fails with the following output.

One solution is to use the --python_path option and point to the ConStrain library.

openstudio --python_path /path/to/constrain measure update -u /path/to/measure/dir

[kbenne]

Based on what I've seen I think this issue consistently stems from issues in the Measure itself. We can either:

1. Close and do nothing on the OpenStudio side.
2. Consider some adjustments to OpenStudio to help people understand what is going on. There is a callstack reported on standard output, but that level of detail does not make it into the measure.xml file. Should it?

[DavidGoldwasser]

When a measure (Ruby or Python) requires custom gems we have put the include outside of the run method. When that library isn't found it can be hard for user to see what is going on. If we have people move those inside of run, then we can add a rescue and check to see if it was loaded, and then runner false with runner.registerError message to user explaining why it failed.

If we can add to measure.xml output that could be useful, or make sure it shows up on verbose log for runner. But neither of those changes seem necessary for 3.9.0. I do want to make sure UO tests run. If we don't have any other specific change to make now we can always make an RC1 and just test it right away so if we need fix in RC2 we can do that.