Skip to content
This repository has been archived by the owner on Nov 26, 2023. It is now read-only.

Deal with bad input files #161

Open
ktfhale opened this issue Sep 1, 2014 · 0 comments
Open

Deal with bad input files #161

ktfhale opened this issue Sep 1, 2014 · 0 comments

Comments

@ktfhale
Copy link
Contributor

ktfhale commented Sep 1, 2014

As documented in Ticket #134, there are 83 files in mtpipeline_outputs known to cause the pipeline to fail. 80 of these have an EXPTIME flag of 0, which causes AstroDrizzle to fail, and 3 others have an unknown problem

The EXPTIME flag problem

wfc3/11573_uranus/ib3805v2q_flt.fits
wfc3/11573_uranus/ib3802tpq_flt.fits
wfc3/11573_uranus/ib3802ttq_flt.fits
wfc3/11573_uranus/ib3805v1q_flt.fits
wfc3/11573_uranus/ib3802tsq_flt.fits
wfc3/12237_174567/ibjb04p0q_flt.fits
wfc3/12237_174567/ibjb04p1q_flt.fits
wfc3/12237_2005ef298/ibjb07jdq_flt.fits
wfc3/12237_2005ef298/ibjb07jcq_flt.fits
wfc3/12237_2005ef298/ibjb07jbq_flt.fits
wfc3/12468_04vy130/ibtp14dxq_flt.fits
wfc3/12468_04vy130/ibtp14dyq_flt.fits
wfc3/12468_04vy130/ibtp14e2q_flt.fits
wfc3/12468_04vy130/ibtp14e0q_flt.fits
wfc3/12468_08so266/ibtp22weq_flt.fits
wfc3/12468_08so266/ibtp22wgq_flt.fits
wfc3/12468_08so266/ibtp22wdq_flt.fits
wfc3/12468_08so266/ibtp22wiq_flt.fits
acs/11970_titan-pre-inf-conj/jb9z04dbq_flt.fits
acs/11970_titan-pre-inf-conj/jb9z04dcq_flt.fits
acs/11970_titan-pre-inf-conj/jb9z04d6q_flt.fits
acs/11970_titan-pre-inf-conj/jb9z04d7q_flt.fits
acs/11970_titan-pre-inf-conj/jb9z04daq_flt.fits
acs/11970_titan-pre-inf-conj/jb9z04d8q_flt.fits
acs/10805_uranus/j9q301g4q_flt.fits
acs/10805_uranus/j9q308lyq_flt.fits
acs/10805_uranus/j9q301g3q_flt.fits
acs/10805_uranus/j9q301fyq_flt.fits
acs/10805_uranus/j9q301g1q_flt.fits
acs/10805_uranus/j9q310mkq_flt.fits
acs/10805_uranus/j9q310mhq_flt.fits
acs/10805_uranus/j9q310mlq_flt.fits
acs/10805_uranus/j9q308m0q_flt.fits
acs/10805_uranus/j9q308m2q_flt.fits
acs/10805_uranus/j9q308lvq_flt.fits
acs/10805_uranus/j9q306l2q_flt.fits
acs/10805_uranus/j9q306l7q_flt.fits
acs/10805_uranus/j9q310mmq_flt.fits
acs/10805_uranus/j9q306l6q_flt.fits
acs/10805_uranus/j9q306l1q_flt.fits
acs/10805_uranus/j9q308m1q_flt.fits
acs/10805_uranus/j9q301fzq_flt.fits
acs/10805_uranus/j9q306l4q_flt.fits
acs/10860_2002ux25/j9qs12hrq_flt.fits
acs/10860_2002ux25/j9qs12hsq_flt.fits
acs/10800_01og109/j9rpa3ghq_flt.fits
acs/10800_01og109/j9rpa3ggq_flt.fits
acs/10800_01og109/j9rpa3giq_flt.fits
acs/10800_01og109/j9rpa3gjq_flt.fits
acs/10514_any/j9fw33qzq_flt.fits
acs/10514_any/j9fw33r1q_flt.fits
acs/10514_any/j9fw05scq_flt.fits
acs/10514_any/j9fw33r4q_flt.fits
acs/10514_any/j9fw05s9q_flt.fits
acs/10514_any/j9fw05s7q_flt.fits
acs/10800_04pa112/j9rpd2n4q_flt.fits
acs/10545_ixion/j9fs14wcq_flt.fits
acs/10545_ixion/j9fs14wfq_flt.fits
acs/10545_ixion/j9fs14weq_flt.fits
acs/10545_ixion/j9fs14wdq_flt.fits
acs/10508_2002cr46b/j9f608yvq_flt.fits
acs/10508_2002cr46b/j9f608yxq_flt.fits
acs/10508_2002cr46b/j9f608yzq_flt.fits
acs/10508_2002cr46b/j9f608ywq_flt.fits
acs/10508_2002cr46b/j9f608z0q_flt.fits
acs/10508_2002cr46b/j9f608yyq_flt.fits
acs/10508_2002cr46b/j9f608yuq_flt.fits
acs/10508_2002cr46b/j9f608z1q_flt.fits
acs/10800_05ef304/j9rp26qaq_flt.fits
acs/10800_05ef304/j9rp26q9q_flt.fits
acs/10800_05ef304/j9rp26q8q_flt.fits
acs/10800_05ef304/j9rp26qbq_flt.fits
acs/10557_asteroid-7719/j9f304xcq_flt.fit
acs/10557_asteroid-7719/j9f304xbq_flt.fit
acs/10514_82075/j9fw05s6q_flt.fits
acs/10514_82075/j9fw05sbq_flt.fits
acs/10514_82075/j9fw05s8q_flt.fits
acs/10514_99rb216/j9fw33r0q_flt.fits
acs/10514_99rb216/j9fw33qyq_flt.fits
acs/10514_99rb216/j9fw33r3q_flt.fits

The unknown problem:

wfc3/12537_any/ibu521krq_flt.fits
wfc3/12537_any/ibu521kvq_flt.fits
wfc3/12537_any/ibu521kyq_flt.fits

In Ticket #134, we looked at these last three images and determined they were of little value. The value of the other 80 is harder to determine, although having an EXPTIME flag of 0 likely indicates they are bad observations.

One way to deal with these files in the pipeline is to have something that checks the proposalid (like j9fw33r3q) against this list, and to abort after the cosmic-ray rejection step if it finds it. That's effectively what the pipeline already does, but this way we can throw in an error message into the log that this is a known bad file.

I'll create a 'bad-files' branch to add this change.

@ktfhale ktfhale changed the title Deal with bad files Deal with bad input files Sep 1, 2014
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant