Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCB to TDB conversion #1531

Merged
merged 56 commits into from
Apr 13, 2023
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
742b8df
allow_tcb option
abhisrkckl Feb 10, 2023
2e961eb
improved error message
abhisrkckl Feb 10, 2023
aabd2b8
IFTE constants
abhisrkckl Feb 10, 2023
199cce7
tcb conversion code
abhisrkckl Feb 11, 2023
717ac24
conversion script
abhisrkckl Feb 11, 2023
8480903
warning
abhisrkckl Feb 11, 2023
8fe3681
setup
abhisrkckl Feb 11, 2023
0f0c03e
welcome message
abhisrkckl Feb 11, 2023
58677bc
typo
abhisrkckl Feb 11, 2023
2a638b2
black
abhisrkckl Feb 11, 2023
3ddcc31
sourcery
abhisrkckl Feb 13, 2023
9764280
message
abhisrkckl Feb 13, 2023
88d60ba
script works.
abhisrkckl Feb 13, 2023
eb0f52c
test_tcb2tdb
abhisrkckl Feb 13, 2023
824bf8a
refact error msg
abhisrkckl Feb 13, 2023
02a3f6a
doc entry
abhisrkckl Feb 13, 2023
44d64f9
docs
abhisrkckl Feb 13, 2023
382b559
sourcery
abhisrkckl Feb 13, 2023
0cea703
sourcery
abhisrkckl Feb 13, 2023
9638fc4
docs
abhisrkckl Feb 13, 2023
97ce07e
docs and sourcery
abhisrkckl Feb 13, 2023
1ad4153
improve test
abhisrkckl Feb 13, 2023
a41c8fd
improve test
abhisrkckl Feb 13, 2023
145a395
assert
abhisrkckl Feb 13, 2023
412ed7b
docs
abhisrkckl Feb 13, 2023
6c2b249
typos
abhisrkckl Feb 13, 2023
d4ce402
isort
abhisrkckl Feb 13, 2023
f4b0e72
Update CHANGELOG.md
abhisrkckl Feb 14, 2023
805c5e9
welcome msg as warning
abhisrkckl Feb 14, 2023
0a1cd02
convert DM and DM derivatives
abhisrkckl Feb 14, 2023
eb66a35
docstrings
abhisrkckl Feb 15, 2023
96f7862
warning
abhisrkckl Feb 15, 2023
8845e16
warning and help message
abhisrkckl Feb 15, 2023
41b1198
get_prefix_mapping
abhisrkckl Feb 15, 2023
920b749
fix power
abhisrkckl Feb 15, 2023
ebd1d5f
fix for loop
abhisrkckl Feb 16, 2023
5e0229f
get_model and get_model_and_toas do not need allow_tcb option.
abhisrkckl Feb 16, 2023
fd18592
back conversion and roundtrip test
abhisrkckl Feb 16, 2023
dd6622e
docstring
abhisrkckl Feb 16, 2023
dc9b39e
Merge branch 'nanograv:master' into tcb2tdb
abhisrkckl Feb 16, 2023
e7d5d4f
minor corrections
abhisrkckl Feb 17, 2023
6cb11dd
convert upon read option
abhisrkckl Feb 17, 2023
c9e70d5
target_units
abhisrkckl Feb 17, 2023
570eb8e
target_units skip
abhisrkckl Feb 17, 2023
7b1399c
convert tcb upon read in ModelBuilder
abhisrkckl Feb 17, 2023
1e650c8
Merge branch 'nanograv:master' into tcb2tdb
abhisrkckl Feb 18, 2023
3ea3ad7
Merge branch 'nanograv:master' into tcb2tdb
abhisrkckl Mar 2, 2023
307f69a
fix roundtrip test
abhisrkckl Mar 23, 2023
9a114f7
Merge branch 'nanograv:master' into tcb2tdb
abhisrkckl Mar 30, 2023
d348777
Merge branch 'master' into tcb2tdb
abhisrkckl Apr 3, 2023
628f78e
don't (try to) convert twice
abhisrkckl Apr 6, 2023
296ce2c
space
abhisrkckl Apr 6, 2023
91daff7
space
abhisrkckl Apr 6, 2023
458a044
log msg
abhisrkckl Apr 6, 2023
664cdd8
update tcb explanation
abhisrkckl Apr 6, 2023
0976fa4
fix test
abhisrkckl Apr 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ and this project, at least loosely, adheres to [Semantic Versioning](https://sem
- `funcParameters` defined as functions operating on other parameters
- Option to save `emcee` backend chains in `event_optimize`
- Documentation on how to extract a covariance matrix
- TCB to TDB conversion script (`tcb2tdb`)
### Fixed
- Broken notebooks CI test
- BIPM correction for simulated TOAs
Expand Down
8 changes: 8 additions & 0 deletions docs/command-line.rst
Original file line number Diff line number Diff line change
Expand Up @@ -111,3 +111,11 @@ the examples subdirectory of the PINT distro.

event_optimize J0030+0451_P8_15.0deg_239557517_458611204_ft1weights_GEO_wt.gt.0.4.fits PSRJ0030+0451_psrcat.par templateJ0030.3gauss --weightcol=PSRJ0030+0451 --minWeight=0.9 --nwalkers=100 --nsteps=500

tcb2tdb
-------

A command line tool that converts par files from TCB timescale to TDB timescale.

::

tcb2tdb J0030+0451_tcb.par J0030+0451_tdb.par
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ console_scripts =
pintk = pint.scripts.pintk:main
convert_parfile = pint.scripts.convert_parfile:main
compare_parfiles = pint.scripts.compare_parfiles:main
tcb2tdb = pint.scripts.tcb2tdb:main


# See the docstring in versioneer.py for instructions. Note that you must
Expand Down
161 changes: 87 additions & 74 deletions src/pint/models/model_builder.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
"""Building a timing model from a par file."""
abhisrkckl marked this conversation as resolved.
Show resolved Hide resolved

import copy
import warnings
from io import StringIO
Expand Down Expand Up @@ -73,7 +75,7 @@ def __init__(self):
self._validate_components()
self.default_components = ["SolarSystemShapiro"]

def __call__(self, parfile, allow_name_mixing=False):
def __call__(self, parfile, allow_name_mixing=False, allow_tcb=False):
"""Callable object for making a timing model from .par file.

Parameters
Expand All @@ -87,6 +89,9 @@ def __call__(self, parfile, allow_name_mixing=False):
T2EFAC and EFAC, both of them maps to PINT parameter EFAC, present
in the parfile at the same time.

allow_tcb : bool, optional
Whether to allow reading TCB par files

Returns
-------
pint.models.timing_model.TimingModel
Expand All @@ -103,12 +108,20 @@ def __call__(self, parfile, allow_name_mixing=False):
# Make timing model
cps = [self.all_components.components[c] for c in selected]
tm = TimingModel(components=cps)
self._setup_model(tm, pint_param_dict, original_name, setup=True, validate=True)
self._setup_model(
tm,
pint_param_dict,
original_name,
setup=True,
validate=True,
allow_tcb=allow_tcb,
)
# Report unknown line
for k, v in unknown_param.items():
p_line = " ".join([k] + v)
warnings.warn(f"Unrecognized parfile line '{p_line}'", UserWarning)
# log.warning(f"Unrecognized parfile line '{p_line}'")

return tm

def _validate_components(self):
Expand Down Expand Up @@ -166,9 +179,9 @@ def _get_component_param_overlap(self, component):
# Add aliases compare
overlap = in_param & cpm_param
# translate to PINT parameter
overlap_pint_par = set(
[self.all_components.alias_to_pint_param(ovlp)[0] for ovlp in overlap]
)
overlap_pint_par = {
self.all_components.alias_to_pint_param(ovlp)[0] for ovlp in overlap
}
# The degree of overlapping for input component and compared component
overlap_deg_in = len(component.params) - len(overlap_pint_par)
overlap_deg_cpm = len(cp.params) - len(overlap_pint_par)
Expand Down Expand Up @@ -246,38 +259,40 @@ def _pintify_parfile(self, parfile, allow_name_mixing=False):
try:
pint_name, init0 = self.all_components.alias_to_pint_param(k)
except UnknownParameter:
if k in ignore_params: # Parameter is known but in the ingore list
if k in ignore_params:
# Parameter is known but in the ignore list
continue
else: # Check ignored prefix
try:
pfx, idxs, idx = split_prefixed_name(k)
if pfx in ignore_prefix: # It is an ignored prefix.
continue
else:
unknown_param[k] += v
except PrefixError:
# Check ignored prefix
try:
pfx, idxs, idx = split_prefixed_name(k)
if pfx in ignore_prefix: # It is an ignored prefix.
continue
else:
unknown_param[k] += v
except PrefixError:
unknown_param[k] += v
continue
pint_param_dict[pint_name] += v
original_name_map[pint_name].append(k)
repeating[pint_name] += len(v)
# Check if this parameter is allowed to be repeated by PINT
if len(pint_param_dict[pint_name]) > 1:
if pint_name not in self.all_components.repeatable_param:
raise TimingModelError(
f"Parameter {pint_name} is not a repeatable parameter. "
f"However, multiple line use it."
)
if (
len(pint_param_dict[pint_name]) > 1
and pint_name not in self.all_components.repeatable_param
):
raise TimingModelError(
f"Parameter {pint_name} is not a repeatable parameter. "
f"However, multiple line use it."
)
# Check if the name is mixed
for p_n, o_n in original_name_map.items():
if len(o_n) > 1:
if not allow_name_mixing:
raise TimingModelError(
f"Parameter {p_n} have mixed input names/alias "
f"{o_n}. If you want to have mixing names, please use"
f" 'allow_name_mixing=True', and the output .par file "
f"will use '{original_name_map[pint_name][0]}'."
)
if len(o_n) > 1 and not allow_name_mixing:
raise TimingModelError(
f"Parameter {p_n} have mixed input names/alias "
f"{o_n}. If you want to have mixing names, please use"
f" 'allow_name_mixing=True', and the output .par file "
f"will use '{original_name_map[pint_name][0]}'."
)
original_name_map[p_n] = o_n[0]

return pint_param_dict, original_name_map, unknown_param
Expand Down Expand Up @@ -347,12 +362,11 @@ def choose_model(self, param_inpar):
if p_name != first_init:
param_not_in_pint.append(pp)

p_cp = self.all_components.param_component_map.get(first_init, None)
if p_cp:
if p_cp := self.all_components.param_component_map.get(first_init, None):
param_components_inpar[p_name] = p_cp
# Back map the possible_components and the parameters in the parfile
# This will remove the duplicate components.
conflict_components = defaultdict(set) # graph for confilict
conflict_components = defaultdict(set) # graph for conflict
for k, cps in param_components_inpar.items():
# If `timing_model` in param --> component mapping skip
# Timing model is the base.
Expand Down Expand Up @@ -386,10 +400,10 @@ def choose_model(self, param_inpar):
temp_cf_cp.remove(cp)
conflict_components[cp].update(set(temp_cf_cp))
continue
# Check if the selected component in the confilict graph. If it is
# remove the selected componens with its conflict components.
# Check if the selected component in the conflict graph. If it is
# remove the selected components with its conflict components.
for ps_cp in selected_components:
cf_cps = conflict_components.get(ps_cp, None)
cf_cps = conflict_components.get(ps_cp)
if cf_cps is not None: # Had conflict, but resolved.
for cf_cp in cf_cps:
del conflict_components[cf_cp]
Expand All @@ -398,17 +412,17 @@ def choose_model(self, param_inpar):
selected_cates = {}
for cp in selected_components:
cate = self.all_components.component_category_map[cp]
if cate not in selected_cates.keys():
selected_cates[cate] = cp
else:
exisit_cp = selected_cates[cate]
if cate in selected_cates:
exist_cp = selected_cates[cate]
raise TimingModelError(
f"Component '{cp}' and '{exisit_cp}' belong to the"
f"Component '{cp}' and '{exist_cp}' belong to the"
f" same category '{cate}'. Only one component from"
f" the same category can be used for a timing model."
f" Please check your input (e.g., .par file)."
)

else:
selected_cates[cate] = cp
return selected_components, conflict_components, param_not_in_pint

def _setup_model(
Expand All @@ -418,6 +432,7 @@ def _setup_model(
original_name=None,
setup=True,
validate=True,
allow_tcb=False,
):
"""Fill up a timing model with parameter values and then setup the model.

Expand All @@ -443,29 +458,28 @@ def _setup_model(
Whether to run the setup function in the timing model.
validate : bool, optional
Whether to run the validate function in the timing model.
allow_tcb : bool, optional
Whether to allow reading TCB par files
"""
if original_name is not None:
use_alias = True
else:
use_alias = False
use_alias = original_name is not None
for pp, v in pint_param_dict.items():
try:
par = getattr(timing_model, pp)
except AttributeError:
# since the input is pintfied, it should be an uninitized indexed parameter
# since the input is pintfied, it should be an uninitialized indexed parameter
# double check if the missing parameter an indexed parameter.
pint_par, first_init = self.all_components.alias_to_pint_param(pp)
try:
prefix, _, index = split_prefixed_name(pint_par)
except PrefixError:
except PrefixError as e:
par_hosts = self.all_components.param_component_map[pint_par]
currnt_cp = timing_model.components.keys()
current_cp = timing_model.components.keys()
raise TimingModelError(
f"Parameter {pint_par} is recognized"
f" by PINT, but not used in the current"
f" timing model. It is used in {par_hosts},"
f" but the current timing model uses {currnt_cp}."
)
f" but the current timing model uses {current_cp}."
) from e
# TODO need to create a better API for _locate_param_host
host_component = timing_model._locate_param_host(first_init)
timing_model.add_param_from_top(
Expand All @@ -477,10 +491,7 @@ def _setup_model(
# Fill up the values
param_line = len(v)
if param_line < 2:
if use_alias:
name = original_name[pp]
else:
name = pp
name = original_name[pp] if use_alias else pp
par.from_parfile_line(" ".join([name] + v))
else: # For the repeatable parameters
lines = copy.deepcopy(v) # Line queue.
Expand Down Expand Up @@ -516,20 +527,20 @@ def _setup_model(

# There is no current repeatable parameter matching the new line
# First try to fill up an empty space.
if empty_repeat_param != []:
emt_par = empty_repeat_param.pop(0)
emt_par.from_parfile_line(" ".join([emt_par.name, li]))
if use_alias: # Use the input alias as input
emt_par.use_alias = original_name[pp]
else:
if not empty_repeat_param:
# No empty space, add a new parameter to the timing model.
host_component = timing_model._locate_param_host(pp)
timing_model.add_param_from_top(temp_par, host_component[0][0])

else:
emt_par = empty_repeat_param.pop(0)
emt_par.from_parfile_line(" ".join([emt_par.name, li]))
if use_alias: # Use the input alias as input
emt_par.use_alias = original_name[pp]
if setup:
timing_model.setup()
if validate:
timing_model.validate()
timing_model.validate(allow_tcb=allow_tcb)
return timing_model

def _report_conflict(self, conflict_graph):
Expand All @@ -538,12 +549,10 @@ def _report_conflict(self, conflict_graph):
# Put all the conflict components together from the graph
cf_cps = list(v)
cf_cps.append(k)
raise ComponentConflict(
"Can not decide the one component from:" " {}".format(cf_cps)
)
raise ComponentConflict(f"Can not decide the one component from: {cf_cps}")


def get_model(parfile, allow_name_mixing=False):
def get_model(parfile, allow_name_mixing=False, allow_tcb=False):
"""A one step function to build model from a parfile.

Parameters
Expand All @@ -557,6 +566,9 @@ def get_model(parfile, allow_name_mixing=False):
T2EFAC and EFAC, both of them maps to PINT parameter EFAC, present
in the parfile at the same time.

allow_tcb : bool, optional
Whether to allow reading TCB par files

Returns
-------
Model instance get from parfile.
Expand All @@ -566,16 +578,14 @@ def get_model(parfile, allow_name_mixing=False):
contents = parfile.read()
except AttributeError:
contents = None
if contents is None:
# # parfile is a filename and can be handled by ModelBuilder
# if _model_builder is None:
# _model_builder = ModelBuilder()
model = model_builder(parfile, allow_name_mixing)
model.name = parfile
return model
else:
tm = model_builder(StringIO(contents), allow_name_mixing)
return tm
if contents is not None:
return model_builder(StringIO(contents), allow_name_mixing, allow_tcb=allow_tcb)
# # parfile is a filename and can be handled by ModelBuilder
# if _model_builder is None:
# _model_builder = ModelBuilder()
model = model_builder(parfile, allow_name_mixing, allow_tcb=allow_tcb)
model.name = parfile
return model


def get_model_and_toas(
Expand All @@ -592,6 +602,7 @@ def get_model_and_toas(
picklefilename=None,
allow_name_mixing=False,
limits="warn",
allow_tcb=False,
):
"""Load a timing model and a related TOAs, using model commands as needed

Expand Down Expand Up @@ -630,12 +641,14 @@ def get_model_and_toas(
in the parfile at the same time.
limits : "warn" or "error"
What to do when encountering TOAs for which clock corrections are not available.
allow_tcb : bool, optional
Whether to allow reading TCB par files

Returns
-------
A tuple with (model instance, TOAs instance)
"""
mm = get_model(parfile, allow_name_mixing)
mm = get_model(parfile, allow_name_mixing, allow_tcb=allow_tcb)
tt = get_TOAs(
timfile,
include_pn=include_pn,
Expand Down
6 changes: 3 additions & 3 deletions src/pint/models/solar_system_shapiro.py
Original file line number Diff line number Diff line change
Expand Up @@ -111,13 +111,13 @@ def solar_system_shapiro_delay(self, toas, acc_delay=None):
if self.PLANET_SHAPIRO.value:
for pl in ("jupiter", "saturn", "venus", "uranus", "neptune"):
delay[grp] += self.ss_obj_shapiro_delay(
tbl[grp]["obs_" + pl + "_pos"],
tbl[grp][f"obs_{pl}_pos"],
psr_dir,
self._ss_mass_sec[pl],
)
except KeyError:
except KeyError as e:
raise KeyError(
"Planet positions not found when trying to compute Solar System Shapiro delay. "
"Make sure that you include `planets=True` in your `get_TOAs()` call, or use `get_model_and_toas()`."
)
) from e
return delay * u.second
Loading