Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-42157: Use new multiprofit config classes in pipelines #8

Merged
merged 16 commits into from
Mar 14, 2024
+894 −329
Merged

DM-42157: Use new multiprofit config classes in pipelines #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
e62bfb8
Add PSF fit_parents config option
taranu Dec 19, 2023
cb91ae7
Add InputConfig.get_connection
taranu Jan 11, 2024
ff9f6f4
Implement joining for table subset consolidation
taranu Jan 11, 2024
e3ab413
Update TODO for the more distant future
taranu Jan 11, 2024
ed28112
Improve expected error handling in fit tasks
taranu Jan 11, 2024
beb5d65
Disable eval_residual in PSF fitting
taranu Jan 11, 2024
cbc53ee
Formatting only
taranu Jan 11, 2024
a9c9448
Parameterize each model to be merged
taranu Jan 11, 2024
b4725f3
Use CoordinateSystem in source fitting
taranu Jan 18, 2024
559da45
Refactor to use new multiprofit Config classes
taranu Jan 31, 2024
21eafb2
Add psf_sigma_subtract option
taranu Feb 7, 2024
b41d103
Fix bugs in loading images of deblended children
taranu Feb 17, 2024
35c99d0
Add MultiProFitSourceTask.make_CatalogExposurePsfs
taranu Feb 20, 2024
8d6ae56
Copy cen_x/cen_y columns when merging
taranu Feb 20, 2024
3a38799
Add detect_isPatchInner to copied columns on merge
taranu Mar 2, 2024
a93d281
Add size-mag plots in extended pipeline
taranu Mar 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions pipelines/coaddQualityExtended.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
description: |
Tier2 atools and metrics to assess coadd quality
tasks:
analyzeObjectTableExtended:
class: lsst.analysis.tools.tasks.ObjectTableTractAnalysisTask
config:
connections.data: objectTable_tract_multiprofit
connections.outputName: objectTableMultiprofitExtended
# set plots to run
atools.serReffVsMag: SizeMagnitudePlot
atools.serReffVsMag.fluxes_default.cmodel_err: model_ser
atools.serReffVsMag.sizes_default.shape_slot: sizes_ser
atools.serReffVsMag.config_moments: moments_ser
atools.serReffVsMag.size_type: "determinantRadius"
atools.serReffVsMag.mag_x: "cmodel_err"
atools.serReffVsMag.size_y: "shape_slot"
atools.serReffVsMag.is_covariance: False
atools.serReffVsMag.applyContext: CoaddContext
atools.serReffVsMag.prep.selectors.flagSelector.selectWhenFalse: flags_ser
atools.serReffVsMag.prep.selectors.flagSelector.selectWhenTrue: []
atools.serReffVsMag.produce.plot.xLims: (17, 29)
atools.serReffVsMag.produce.plot.yLims: (-4, 3)
python: |
from lsst.analysis.tools.atools import *
from lsst.analysis.tools.contexts import *
from lsst.analysis.tools.atools.genericBuild import FluxConfig, MomentsConfig, SizeConfig

models_mag = {}
flags_model = {}
for name_model, label_model in (("ser", "MPF Ser"), ("expdev", "MPF Exp+Dev")):
models_mag[name_model] = FluxConfig(
key_flux = f"mpf_{name_model}_ser_{{band}}_flux",
key_flux_error = f"mpf_{name_model}_ser_{{band}}_flux_err",
name_flux = label_model,
)
flags_model[name_model] = [
'mpf_ser_fixedcen_unknown_flag',
'mpf_ser_fixedcen_is_parent_flag',
'mpf_ser_fixedcen_not_primary_flag',
'mpf_ser_fixedcen_psf_fit_flag',
]
flags_ser = flags_model["ser"]
model_ser = models_mag["ser"]
sizes_ser = SizeConfig(key_size="mpf_ser_ser_{suffix}", name_size="Sersic $R_{eff}$")
moments_ser = MomentsConfig(xx="reff_x", yy="reff_y", xy="rho")
271 changes: 203 additions & 68 deletions pipelines/fit.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,120 +7,255 @@ tasks:
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_psf import MultiProFitPsfTask
config.fit_coadd_psf.retarget(MultiProFitPsfTask)
config.fit_coadd_psf.config_fit.eval_residual = False
fit_src_psgauss_multiprofit:
class: lsst.pipe.tasks.fit_coadd_multiband.CoaddMultibandFitTask
config:
connections.cat_output: deepCoadd_psgauss_multiprofit
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_multiband import MultiProFitSourceTask
from lsst.multiprofit.componentconfig import SersicConfig, SersicIndexConfig
from lsst.multiprofit.componentconfig import (
GaussianComponentConfig, ParameterConfig, SersicComponentConfig, SersicIndexParameterConfig,
)
from lsst.multiprofit.modelconfig import ModelConfig
from lsst.multiprofit.sourceconfig import ComponentGroupConfig, SourceConfig
config.fit_coadd_multiband.retarget(MultiProFitSourceTask)
config.fit_coadd_multiband.n_pointsources = 1
config.fit_coadd_multiband.sersics = {
"gauss": SersicConfig(
sersicindex=SersicIndexConfig(fixed=True, value_initial=0.5),
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
)
}
config.fit_coadd_multiband.config_model = ModelConfig(
sources={"": SourceConfig(component_groups={"": ComponentGroupConfig(
components_gauss={
"ps": GaussianComponentConfig(
size_x=ParameterConfig(value_initial=0., fixed=True),
size_y=ParameterConfig(value_initial=0., fixed=True),
rho=ParameterConfig(value_initial=0., fixed=True),
),
},
# It could be in components_gauss, but this keeps the size field
# conveniently named reff for consistency (and uses a Gaussian
# component under the hood anyway).
components_sersic={
"gauss": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
sersic_index=SersicIndexParameterConfig(value_initial=0.5, fixed=True),
),
},
)})}
)
fit_src_psexp_multiprofit:
class: lsst.pipe.tasks.fit_coadd_multiband.CoaddMultibandFitTask
config:
connections.cat_output: deepCoadd_psexp_multiprofit
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_multiband import MultiProFitSourceTask
from lsst.multiprofit.componentconfig import SersicConfig, SersicIndexConfig
from lsst.multiprofit.componentconfig import (
GaussianComponentConfig, ParameterConfig, SersicComponentConfig, SersicIndexParameterConfig,
)
from lsst.multiprofit.modelconfig import ModelConfig
from lsst.multiprofit.sourceconfig import ComponentGroupConfig, SourceConfig
config.fit_coadd_multiband.retarget(MultiProFitSourceTask)
config.fit_coadd_multiband.n_pointsources = 1
config.fit_coadd_multiband.sersics = {
"exp": SersicConfig(
sersicindex=SersicIndexConfig(fixed=True, value_initial=1.0),
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
)
}
config.fit_coadd_multiband.config_model = ModelConfig(
sources={"": SourceConfig(component_groups={"": ComponentGroupConfig(
components_gauss={
"ps": GaussianComponentConfig(
size_x=ParameterConfig(value_initial=0., fixed=True),
size_y=ParameterConfig(value_initial=0., fixed=True),
rho=ParameterConfig(value_initial=0., fixed=True),
)
},
components_sersic={
"exp": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
sersic_index=SersicIndexParameterConfig(value_initial=1., fixed=True),
),
},
)})}
)
fit_src_exp_multiprofit:
class: lsst.pipe.tasks.fit_coadd_multiband.CoaddMultibandFitTask
config:
connections.cat_output: deepCoadd_exp_multiprofit
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_multiband import MultiProFitSourceTask
from lsst.multiprofit.componentconfig import SersicConfig, SersicIndexConfig
from lsst.multiprofit.componentconfig import (
GaussianComponentConfig, ParameterConfig, SersicComponentConfig, SersicIndexParameterConfig,
)
from lsst.multiprofit.modelconfig import ModelConfig
from lsst.multiprofit.sourceconfig import ComponentGroupConfig, SourceConfig
config.fit_coadd_multiband.retarget(MultiProFitSourceTask)
config.fit_coadd_multiband.sersics = {
"exp": SersicConfig(
sersicindex=SersicIndexConfig(fixed=True, value_initial=1.0),
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
)
}
config.fit_coadd_multiband.config_model = ModelConfig(
sources={"": SourceConfig(component_groups={"": ComponentGroupConfig(
components_sersic={
"exp": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
sersic_index=SersicIndexParameterConfig(value_initial=1., fixed=True),
),
},
)})}
)
fit_src_exp_fixedcen_multiprofit:
class: lsst.pipe.tasks.fit_coadd_multiband.CoaddMultibandFitTask
config:
connections.cat_output: deepCoadd_exp_fixedcen_multiprofit
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_multiband import MultiProFitSourceTask
from lsst.multiprofit.componentconfig import SersicConfig, SersicIndexConfig
from lsst.multiprofit.componentconfig import (
CentroidConfig, ParameterConfig, SersicComponentConfig, SersicIndexParameterConfig,
)
from lsst.multiprofit.modelconfig import ModelConfig
from lsst.multiprofit.sourceconfig import ComponentGroupConfig, SourceConfig
config.fit_coadd_multiband.retarget(MultiProFitSourceTask)
config.fit_coadd_multiband.fit_cen_x = False
config.fit_coadd_multiband.fit_cen_y = False
config.fit_coadd_multiband.sersics = {
"exp": SersicConfig(
sersicindex=SersicIndexConfig(fixed=True, value_initial=1.0),
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
)
}
config.fit_coadd_multiband.config_model = ModelConfig(
sources={"": SourceConfig(component_groups={"": ComponentGroupConfig(
centroids={"default": CentroidConfig(
x=ParameterConfig(fixed=True),
y=ParameterConfig(fixed=True),
)},
components_sersic={
"exp": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
sersic_index=SersicIndexParameterConfig(value_initial=1., fixed=True),
),
},
)})}
)
fit_src_ser_fixedcen_multiprofit:
class: lsst.pipe.tasks.fit_coadd_multiband.CoaddMultibandFitTask
config:
connections.cat_output: deepCoadd_ser_fixedcen_multiprofit
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_multiband import MultiProFitSourceTask
from lsst.multiprofit.componentconfig import (
CentroidConfig, ParameterConfig, SersicComponentConfig
)
from lsst.multiprofit.modelconfig import ModelConfig
from lsst.multiprofit.sourceconfig import ComponentGroupConfig, SourceConfig
config.fit_coadd_multiband.retarget(MultiProFitSourceTask)
config.fit_coadd_multiband.config_model = ModelConfig(
sources={"": SourceConfig(component_groups={"": ComponentGroupConfig(
centroids={"default": CentroidConfig(
x=ParameterConfig(fixed=True),
y=ParameterConfig(fixed=True),
)},
components_sersic={
"ser": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
),
},
)})}
)
fit_src_expdev_multiprofit:
class: lsst.pipe.tasks.fit_coadd_multiband.CoaddMultibandFitTask
config:
connections.cat_output: deepCoadd_expdev_multiprofit
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_multiband import MultiProFitSourceTask
from lsst.multiprofit.componentconfig import (
GaussianComponentConfig, ParameterConfig, SersicComponentConfig, SersicIndexParameterConfig,
)
from lsst.multiprofit.modelconfig import ModelConfig
from lsst.multiprofit.sourceconfig import ComponentGroupConfig, SourceConfig
config.fit_coadd_multiband.retarget(MultiProFitSourceTask)
config.fit_coadd_multiband.config_model = ModelConfig(
sources={"": SourceConfig(component_groups={"": ComponentGroupConfig(
components_sersic={
"exp": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
sersic_index=SersicIndexParameterConfig(value_initial=1., fixed=True),
),
"dev": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
sersic_index=SersicIndexParameterConfig(value_initial=4., fixed=True),
),
},
)})}
)
fit_src_ser_multiprofit:
class: lsst.pipe.tasks.fit_coadd_multiband.CoaddMultibandFitTask
config:
connections.cat_output: deepCoadd_ser_multiprofit
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_multiband import MultiProFitSourceTask
from lsst.multiprofit.componentconfig import SersicConfig, SersicIndexConfig
from lsst.multiprofit.componentconfig import (ParameterConfig, SersicComponentConfig)
from lsst.multiprofit.modelconfig import ModelConfig
from lsst.multiprofit.sourceconfig import ComponentGroupConfig, SourceConfig
config.fit_coadd_multiband.retarget(MultiProFitSourceTask)
config.fit_coadd_multiband.n_pointsources = 0
config.fit_coadd_multiband.sersics = {
"ser": SersicConfig(
sersicindex=SersicIndexConfig(fixed=False, value_initial=1.0),
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
)
}
config.fit_coadd_multiband.config_model = ModelConfig(
sources={"": SourceConfig(component_groups={"": ComponentGroupConfig(
components_sersic={
"ser": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
),
},
)})}
)
fit_src_psser_multiprofit:
class: lsst.pipe.tasks.fit_coadd_multiband.CoaddMultibandFitTask
config:
connections.cat_output: deepCoadd_psser_multiprofit
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_multiband import MultiProFitSourceTask
from lsst.multiprofit.componentconfig import SersicConfig, SersicIndexConfig
from lsst.multiprofit.componentconfig import (
GaussianComponentConfig, ParameterConfig, SersicComponentConfig,
)
from lsst.multiprofit.modelconfig import ModelConfig
from lsst.multiprofit.sourceconfig import ComponentGroupConfig, SourceConfig
config.fit_coadd_multiband.retarget(MultiProFitSourceTask)
config.fit_coadd_multiband.n_pointsources = 1
config.fit_coadd_multiband.sersics = {
"ser": SersicConfig(
sersicindex=SersicIndexConfig(fixed=False, value_initial=1.63),
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
)
}
config.fit_coadd_multiband.config_model = ModelConfig(
sources={"": SourceConfig(component_groups={"": ComponentGroupConfig(
components_gauss={
"ps": GaussianComponentConfig(
size_x=ParameterConfig(value_initial=0., fixed=True),
size_y=ParameterConfig(value_initial=0., fixed=True),
rho=ParameterConfig(value_initial=0., fixed=True),
)
},
components_sersic={
"ser": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
),
},
)})}
)
fit_src_psexpdev_multiprofit:
class: lsst.pipe.tasks.fit_coadd_multiband.CoaddMultibandFitTask
config:
connections.cat_output: deepCoadd_psexpdev_multiprofit
python: |
from lsst.meas.extensions.multiprofit.fit_coadd_multiband import MultiProFitSourceTask
from lsst.multiprofit.componentconfig import SersicConfig, SersicIndexConfig
from lsst.multiprofit.componentconfig import (
GaussianComponentConfig, ParameterConfig, SersicComponentConfig, SersicIndexParameterConfig,
)
from lsst.multiprofit.modelconfig import ModelConfig
from lsst.multiprofit.sourceconfig import ComponentGroupConfig, SourceConfig
config.fit_coadd_multiband.retarget(MultiProFitSourceTask)
config.fit_coadd_multiband.n_pointsources = 1
config.fit_coadd_multiband.sersics = {
"exp": SersicConfig(
sersicindex=SersicIndexConfig(fixed=True, value_initial=1.0),
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
),
"dev": SersicConfig(
sersicindex=SersicIndexConfig(fixed=True, value_initial=4.0),
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
)
}
config.fit_coadd_multiband.config_model = ModelConfig(
sources={"": SourceConfig(component_groups={"": ComponentGroupConfig(
components_gauss={
"ps": GaussianComponentConfig(
size_x=ParameterConfig(value_initial=0., fixed=True),
size_y=ParameterConfig(value_initial=0., fixed=True),
rho=ParameterConfig(value_initial=0., fixed=True),
)
},
components_sersic={
"exp": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
sersic_index=SersicIndexParameterConfig(value_initial=1., fixed=True),
),
"dev": SersicComponentConfig(
prior_axrat_stddev=0.8,
prior_size_stddev=0.3,
sersic_index=SersicIndexParameterConfig(value_initial=4., fixed=True),
),
},
)})}
)
Loading
Loading