-
Notifications
You must be signed in to change notification settings - Fork 21
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parameterized notebook examples #108
Changes from all commits
ac26c38
910be93
59fa600
0d20ac7
dc3d043
014a634
b19c86d
daeef8b
9a3a865
095c259
21087d9
a0d576b
6eb677a
6547fb2
8a48011
14850f1
5361524
6a3e26f
a370b92
46c21c8
9e38275
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
import param | ||
|
||
def params_from_kwargs(**kwargs): | ||
""" | ||
Utility to promote keywords with literal values to the appropriate | ||
parameter type with the specified default value unless the value is | ||
already a parameter. | ||
""" | ||
params = {} | ||
for k, v in kwargs.items(): | ||
kws = dict(default=v) | ||
if isinstance(v, param.Parameter): | ||
params[k] = v | ||
elif isinstance(v, bool): | ||
params[k] = param.Boolean(**kws) | ||
elif isinstance(v, int): | ||
params[k] = param.Integer(**kws) | ||
elif isinstance(v, float): | ||
params[k] = param.Number(**kws) | ||
elif isinstance(v, str): | ||
params[k] = param.String(**kws) | ||
elif isinstance(v, dict): | ||
params[k] = param.Dict(**kws) | ||
elif isinstance(v, tuple): | ||
params[k] = param.Tuple(**kws) | ||
elif isinstance(v, list): | ||
params[k] = param.List(**kws) | ||
elif isinstance(v, np.ndarray): | ||
params[k] = param.Array(**kws) | ||
else: | ||
params[k] = param.Parameter(**kws) | ||
return params | ||
|
||
|
||
def parameters(**kwargs): | ||
""" | ||
Utility to easily define a parameterized class with a chosen set of | ||
parameters specified as keyword arguments. The resulting object can | ||
be used to parameterize a notebook, display corresponding widgets | ||
with parambokeh and control the workflow from the command line. | ||
""" | ||
name = kwargs.get('name', 'Parameters') | ||
params = params_from_kwargs(**kwargs) | ||
params['name'] = param.String(default=name) | ||
return type(name, (param.Parameterized,), params) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
""" | ||
Prototype of a param script that can set global_params via the | ||
appropriate environment variable to execute an arbitrary command. | ||
|
||
Example usage: | ||
|
||
param -cmd 'jupyter nbconvert --execute GSSHA_Workflow_Batched_Example2.ipynb' -p rain_intensity=25 -p rain_duration=3600 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Presumably 'param' is 'earthsim' in this case? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I wasn't sure what to call it: right now it is in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oh, I see -- I thought the command name would default to the module name, but I see later that it's explicitly declared There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't believe their is a unix/windows command called If we have to worry about a clash, I suppose I might consider something like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Both of those are equally generic. If the intent is for param to supply this command, then calling it There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
That was what I was thinking at least. |
||
""" | ||
|
||
import os | ||
import sys | ||
import json | ||
import argparse | ||
import subprocess | ||
from distutils import spawn | ||
|
||
def main(): | ||
parser = argparse.ArgumentParser() | ||
parser.add_argument('-cmd',type=str) | ||
parser.add_argument("-p", nargs=1, action='append') | ||
args = parser.parse_args() | ||
if args.p is None: | ||
print('Please supply parameters using the -p flag') | ||
sys.exit(1) | ||
|
||
if args.cmd is None: | ||
print('Please supply a command string using the -cmd flag') | ||
sys.exit(1) | ||
|
||
execute(args) | ||
|
||
def execute(args): | ||
split_strings = dict(el[0].split('=') for el in args.p) | ||
json_dict = {k:eval(v) for k,v in split_strings.items()} | ||
env = os.environ.copy() # Required on windows | ||
env['PARAM_JSON_INIT'] = json.dumps(json_dict) | ||
cmd = args.cmd.split() | ||
p = subprocess.Popen(cmd, env=env, | ||
stdout=subprocess.PIPE, stderr=subprocess.PIPE) | ||
out, err = p.communicate() | ||
|
||
if __name__ == "__main__": | ||
main() |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,6 +9,7 @@ channels: | |
- defaults | ||
dependencies: | ||
- python>=3.5 | ||
- lancet | ||
- fiona | ||
- rasterio | ||
- gdal | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd greatly prefer this to be done using a lookup table. Something like:
That way other types could be registered by adding them to this dictionary.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is code I've used in at least three separate contexts, once in another project and once in holoviews itself. I would like to see it moved to param instead of duplicating it in different places. It is often useful!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm happy for it to move to param, though if it's done there, it will need a good description mentioning the limitations (i.e. that only a few Parameter types will ever be supported in this way).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll submit a PR to param for this code but I don't think this PR should wait on that. Once this is in param, I can update earthsim to import it accordingly.