From 827bd4e7ac66a0fc837a3dbd18386a7c2f1cd03e Mon Sep 17 00:00:00 2001 From: Sam Bianco Date: Tue, 25 Jun 2024 21:38:36 -0400 Subject: [PATCH 1/4] MAST metadata search notebook --- .../Querying MAST for JWST Metadata.ipynb | 538 ++++++++++++++++++ .../MAST_metadata_search/requirements.txt | 2 + 2 files changed, 540 insertions(+) create mode 100644 notebooks/JWST/MAST_metadata_search/Querying MAST for JWST Metadata.ipynb create mode 100644 notebooks/JWST/MAST_metadata_search/requirements.txt diff --git a/notebooks/JWST/MAST_metadata_search/Querying MAST for JWST Metadata.ipynb b/notebooks/JWST/MAST_metadata_search/Querying MAST for JWST Metadata.ipynb new file mode 100644 index 000000000..b0156df26 --- /dev/null +++ b/notebooks/JWST/MAST_metadata_search/Querying MAST for JWST Metadata.ipynb @@ -0,0 +1,538 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "\n", + "# Astroquery: Exploring Metadata from the James Webb Space Telescope\n", + "***\n", + "## Learning Goals\n", + "\n", + "By the end of this tutorial, you will:\n", + "\n", + "- Understand how to use the `astroquery.mast` module to access metadata from the James Webb Space Telescope (JWST).\n", + "- Run metadata queries based on coordinates, an object name, or non-positional criteria.\n", + "- Use optional search parameters to further refine query results." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Table of Contents\n", + "* [Introduction](#Introduction)\n", + "* [Querying MAST for JWST Metadata](#Querying-MAST-for-JWST-Metadata)\n", + " * [Setup](#Setup)\n", + " * [Optional Search Parameters](#Optional-Search-Parameters)\n", + " * [Query by Object Name](#Query-by-Object-Name)\n", + " * [Query by Region](#Query-by-Region)\n", + " * [Query by Criteria](#Query-by-Criteria)\n", + "* [Additional Resources](#Additional-Resources)\n", + "\n", + "## Introduction\n", + "\n", + "Welcome! This tutorial focuses on using the `astroquery.mast` module to search for metadata from the [James Webb Space Telescope (JWST)](https://webb.nasa.gov/). Launched in December of 2021, JWST is an advanced space observatory designed for observations in the infrared light spectrum.\n", + "\n", + "The [Mikulski Archive for Space Telescopes (MAST)](https://archive.stsci.edu/) hosts publicly accessible data products from space telescopes like JWST. `astroquery.mast` provides access to a broad set of JWST metadata, including header keywords, proposal information, and observational parameters. The available metadata can also be found using the [MAST JWST Search](https://mast.stsci.edu/search/ui/#/jwst) interface." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## Imports\n", + "\n", + "This notebook uses the following packages:\n", + "- *astroquery.mast* to query the MAST Archive\n", + "- *astropy.coordinates* to assign coordinates of interest" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "slideshow": { + "slide_type": "fragment" + } + }, + "outputs": [], + "source": [ + "from astroquery.mast import MastMissions\n", + "from astropy.coordinates import SkyCoord" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "***" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Querying MAST for JWST Metadata" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Setup" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In order to make queries on JWST metadata, we will have to perform some setup. First, we will instantiate an object of the `MastMissions` class and assign its `mission` to be `'jwst'`. Its `service` is set to the default of `'search'`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create MastMissions object and assign mission to 'jwst'\n", + "missions = MastMissions(mission='jwst')\n", + "\n", + "print(f'Mission: {missions.mission}')\n", + "print(f'Service: {missions.service}')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "When writing queries, keyword arguments can be used to specify output characteristics (see the following section) and filter on values like instrument, exposure type, and proposal ID. The available column names for a mission are returned by the `get_column_list` function. Below, we will print out the name, data type, and description for the first 10 columns in JWST metadata." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Get available columns for JWST mission\n", + "columns = missions.get_column_list()\n", + "columns[:10]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Optional Search Parameters\n", + "\n", + "Before we dive in to the actual queries, it's important to know how we can refine our results with optional keyword arguments. The following parameters are available:\n", + "\n", + "- `radius`: For positional searches only. Only return results within a certain distance from an object or set of coordinates. Default is 3 arcminutes. \n", + "\n", + "- `limit`: The maximum number of results to return. Default is 5000.\n", + "- `offset`: Skip the first ***n*** results. Useful for paging through results.\n", + "- `select_cols`: A list of columns to be returned in the response.\n", + "\n", + "As we walk through different types of queries, we will see these parameters in action!" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Query by Object Name\n", + "\n", + "We've reached our first query! We can use object names to perform metadata queries using the `query_object` function.\n", + "\n", + "To start, let's query for the Messier 1 object, a supernova remnant in the Taurus constellation. You may know it better as the Crab Nebula!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Query for Messier 1 ('M1')\n", + "results = missions.query_object('M1')\n", + "\n", + "# Display the first 5 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "There were 250 total results, meaning that 250 JWST datasets were targeting the Crab Nebula. Now, let's try refining our search a bit more.\n", + "\n", + "- Each dataset is associated with a celestial coordinate, given by `targ_ra` (right ascension) and `targ_dec` (declination). By default, the query returns all datasets that fall within 3 arcminutes from the object's coordinates. Let's set the `radius` parameter to be 1 arcminute instead.\n", + "- Say that we're not interested in the first 4 results. We can assign `offset` to skip a certain number of rows.\n", + "- By default, a subset of recommended columns are returned for each query. However, we can specify exactly which columns to return using the `select_cols` keyword argument. The `ArchiveFileID` column is included automatically." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Refined query for Messier 1 ('M1')\n", + "results = missions.query_object('M1',\n", + " radius=1, # Search within a 1 arcminute radius\n", + " offset=4, # Skip the first 4 results\n", + " select_cols=['fileSetName', 'targprop', 'date_obs']) # Select certain columns\n", + "\n", + "# Display the first 5 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Exercise**: Now it's your turn! Try querying for the Whirlpool Galaxy object. Search within a radius of 1 arcminute, skip the first 300 results, and select the `fileSetName` and `opticalElements` columns." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Query for Whirlpool Galaxy\n", + "results = missions.query_object(...) # Write your query!\n", + "\n", + "# Display the first 5 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Query by Region\n", + "\n", + "The `missions` object also allows us to query by a region in the sky. By passing in a set of coordinates to the `query_region` function, we can return datasets that fall within a certain `radius` value of that point. This type of search is also known as a cone search." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create coordinate object\n", + "coords = SkyCoord(210.80227, 54.34895, unit=('deg'))\n", + "\n", + "# Query for results within 10 arcminutes of coords\n", + "results = missions.query_region(coords, radius=10)\n", + "\n", + "# Display results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "395 JWST datasets fall within our cone search. In other words, their target coordinates are within 10 arcminutes of the coordinates that we defined." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Exercise**: JWST has observed the star Vega, which has a right ascension of 279.23473 degrees and a declination of 38.78369 degrees. Use the `query_region` function to search for datasets within 15 arcminutes of Vega. Select the `fileSetName`, `targprop`, `targ_ra`, and `targ_dec` columns." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Vega coordinates\n", + "vega = SkyCoord(_, _, unit=('deg')) # Fill in with Vega's coordinates\n", + "\n", + "# Query for datasets around Vega\n", + "results = missions.query_region(...) # Write your query!\n", + "\n", + "# Display the first 5 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Query by Criteria" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In some cases, we may want to run queries with non-positional parameters. To accomplish this, we use the `query_criteria` function.\n", + "\n", + "For any of our query functions, we can filter our results by the value of columns in the dataset.\n", + "\n", + "Let's say that we only want observations from JWST's Near Infrared Camera (NIRCam) instrument, and that we only want datasets connected to program number 1189." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Query with column criteria\n", + "results = missions.query_criteria(instrume='NIRCAM', # From Near Infrared Camera\n", + " program=1189,\n", + " select_cols=['fileSetName', 'instrume', 'exp_type', 'program', 'pi_name'])\n", + "\n", + "# Display the first 5 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To exclude and filter out a certain value from the results, we can prepend the value with `!`.\n", + "\n", + "Let's run the same query as above, but this time, we will filter out datasets coming from the NIRCam instrument." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Filtered query, excluding NIRCam datasets\n", + "results = missions.query_criteria(program=1189,\n", + " instrume='!NIRCAM', # Exclude datasets from the NIRCam instrument\n", + " select_cols=['fileSetName', 'instrume', 'exp_type', 'program', 'pi_name'])\n", + "\n", + "# Display the first 5 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also use wildcards for more advanced filtering. Let's use the same query from above, but we will add an exposure type filter for fixed slits (FS) spectroscopy." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Filtered query with wildcard\n", + "results = missions.query_criteria(program=1189,\n", + " instrume='!NIRCAM', # Exclude datasets from the NIRCam instrument\n", + " exp_type='*FIXEDSLIT*', # Any exposure type that contains 'FIXEDSLIT'\n", + " select_cols=['fileSetName', 'instrume', 'exp_type', 'program', 'pi_name'])\n", + "\n", + "# Display the first 10 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:10]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To filter by multiple values for a single column, we use a string of the values delimited by commas.\n", + "\n", + "To illustrate this, we will use a slightly different query. We query for datasets that have a fixed slits spectroscopy exposure type and targets with moving coordinates (`targtype='MOVING'`). We will add another filter to match three different last names for principal investigators (PIs)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Filtered query with multiple values\n", + "results = missions.query_criteria(exp_type='*FIXEDSLIT*', # Any exposure type that contains 'FIXEDSLIT'\n", + " targtype='MOVING', # Only return moving targets\n", + " pi_name='Stansberry, Parker, Lunine', # Last name of PI can be any of these 3 values\n", + " select_cols=['fileSetName', 'targtype', 'instrume', 'exp_type', 'program', 'pi_name'])\n", + "\n", + "# Display the first 10 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:10]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For columns with numeric or date values, we can filter using comparison values:\n", + "\n", + "- `<`: Return values less than or before the given number/date\n", + "- `>`: Return values greater than or after the given number/date\n", + "- `<=`: Return values less than or equal to the given number/date\n", + "- `>=`: Return values greater than or equal to the given number/date\n", + "\n", + "As an example, let's write a query to return all datasets with an observation date before February 1, 2022." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Query using comparison operator\n", + "results = missions.query_criteria(date_obs='<2022-02-01', # Must be observed before February 1, 2022\n", + " select_cols=['fileSetName', 'program', 'date_obs'])\n", + "\n", + "# Display results\n", + "print(f'Total number of results: {len(results)}')\n", + "results" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For numeric or date data types, we can also filter with ranges. This requires the following syntax: `'#..#'`.\n", + "\n", + "Let's write a query that uses range syntax to return datasets that belong to a program number between 1150 and 1155. We will also select for exposure durations that are greater than or equal to 100 seconds." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Query using range operator\n", + "results = missions.query_criteria(program='1150..1155', # Program number between 1150 and 1155\n", + " duration='>100', # Exposure duration is greater than or equal to 100 seconds\n", + " select_cols=['fileSetName', 'program', 'duration'])\n", + "\n", + "# Display results\n", + "print(f'Total number of results: {len(results)}')\n", + "results" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Exercise**: It's time to apply all that you've learned! Write a non-positional query based on the following:\n", + "\n", + "- Fixed targets (*HINT*: `targtype='FIXED'`)\n", + "- Instument is Mid-Infrared Instrument (MIRI) or Fine Guidance Sensor (FGS)\n", + "- Proposal type should NOT include General Observers (`GO`)\n", + "- Exposure type includes the string `'IMAGE'`\n", + "- Right ascension is between 70 and 75 degrees\n", + "- Program number is less than 1200.\n", + "- Skip the first 5 entries.\n", + "- Select the following columns: `targtype`, `instrume`, `proposal_type`, `exp_type`, `targ_ra`, `program`" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# A non-positional query with column criteria\n", + "results = missions.query_criteria(...) # Write your query here!\n", + "\n", + "# Display results\n", + "print(f'Total number of results: {len(results)}')\n", + "results" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Additional Resources\n", + "\n", + "- [MAST JWST Search Form](https://mast.stsci.edu/search/ui/#/jwst)\n", + "- [MAST JWST Search API](https://mast.stsci.edu/search/docs/?urls.primaryName=jwst_api)\n", + "- [`astroquery.mast` Documentation for Mission-Specific Searches](https://astroquery.readthedocs.io/en/latest/mast/mast_missions.html#mission-specific-search-queries)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Citations\n", + "\n", + "If you use `astroquery` for published research, please cite the\n", + "authors. Follow these links for more information about citing `astroquery`:\n", + "\n", + "* [Citing `astroquery`](https://github.com/astropy/astroquery/blob/main/astroquery/CITATION)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "slideshow": { + "slide_type": "slide" + } + }, + "source": [ + "## About this Notebook\n", + "\n", + "**Author(s):** Sam Bianco
\n", + "**Keyword(s):** Tutorial, JWST, Astroquery, MastMissions
\n", + "**First published:** June 2024
\n", + "**Last updated:** June 2024
\n", + "\n", + "***\n", + "[Top of Page](#top)\n", + "\"Space " + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.12.2" + } + }, + "nbformat": 4, + "nbformat_minor": 4 +} diff --git a/notebooks/JWST/MAST_metadata_search/requirements.txt b/notebooks/JWST/MAST_metadata_search/requirements.txt new file mode 100644 index 000000000..7087fc249 --- /dev/null +++ b/notebooks/JWST/MAST_metadata_search/requirements.txt @@ -0,0 +1,2 @@ +astroquery==0.4.7 +astropy==6.1.0 \ No newline at end of file From 873a18231147ec45fb7218bf3248a71b43e1eaf0 Mon Sep 17 00:00:00 2001 From: Sam Bianco Date: Tue, 25 Jun 2024 21:44:37 -0400 Subject: [PATCH 2/4] Rename notebook --- ...T Metadata.ipynb => Querying_MAST_for_JWST_Metadata.ipynb} | 0 notebooks/JWST/MAST_metadata_search/requirements.txt | 4 ++-- 2 files changed, 2 insertions(+), 2 deletions(-) rename notebooks/JWST/MAST_metadata_search/{Querying MAST for JWST Metadata.ipynb => Querying_MAST_for_JWST_Metadata.ipynb} (100%) diff --git a/notebooks/JWST/MAST_metadata_search/Querying MAST for JWST Metadata.ipynb b/notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb similarity index 100% rename from notebooks/JWST/MAST_metadata_search/Querying MAST for JWST Metadata.ipynb rename to notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb diff --git a/notebooks/JWST/MAST_metadata_search/requirements.txt b/notebooks/JWST/MAST_metadata_search/requirements.txt index 7087fc249..87de7281e 100644 --- a/notebooks/JWST/MAST_metadata_search/requirements.txt +++ b/notebooks/JWST/MAST_metadata_search/requirements.txt @@ -1,2 +1,2 @@ -astroquery==0.4.7 -astropy==6.1.0 \ No newline at end of file +astroquery>=0.4.7 +astropy>=6.1.0 \ No newline at end of file From effcf000a7242a38d104dd4f0240cd67ffe5c5cb Mon Sep 17 00:00:00 2001 From: Sam Bianco Date: Wed, 26 Jun 2024 12:58:34 -0400 Subject: [PATCH 3/4] add exercise solutions, download warning --- .../Querying_MAST_for_JWST_Metadata.ipynb | 192 +++++++++++++++--- 1 file changed, 163 insertions(+), 29 deletions(-) diff --git a/notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb b/notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb index b0156df26..ae39b1693 100644 --- a/notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb +++ b/notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb @@ -33,12 +33,15 @@ " * [Query by Region](#Query-by-Region)\n", " * [Query by Criteria](#Query-by-Criteria)\n", "* [Additional Resources](#Additional-Resources)\n", + "* [Exercise Solutions](#Exercise-Solutions)\n", "\n", "## Introduction\n", "\n", "Welcome! This tutorial focuses on using the `astroquery.mast` module to search for metadata from the [James Webb Space Telescope (JWST)](https://webb.nasa.gov/). Launched in December of 2021, JWST is an advanced space observatory designed for observations in the infrared light spectrum.\n", "\n", - "The [Mikulski Archive for Space Telescopes (MAST)](https://archive.stsci.edu/) hosts publicly accessible data products from space telescopes like JWST. `astroquery.mast` provides access to a broad set of JWST metadata, including header keywords, proposal information, and observational parameters. The available metadata can also be found using the [MAST JWST Search](https://mast.stsci.edu/search/ui/#/jwst) interface." + "The [Mikulski Archive for Space Telescopes (MAST)](https://archive.stsci.edu/) hosts publicly accessible data products from space telescopes like JWST. `astroquery.mast` provides access to a broad set of JWST metadata, including header keywords, proposal information, and observational parameters. The available metadata can also be found using the [MAST JWST Search](https://mast.stsci.edu/search/ui/#/jwst) interface.\n", + "\n", + "***Please note that `astroquery.mast.MastMissions` and the MAST JWST Search API do not yet support data product downloads.***" ] }, { @@ -58,7 +61,7 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": { "slideshow": { "slide_type": "fragment" @@ -100,9 +103,18 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Mission: jwst\n", + "Service: search\n" + ] + } + ], "source": [ "# Create MastMissions object and assign mission to 'jwst'\n", "missions = MastMissions(mission='jwst')\n", @@ -120,9 +132,50 @@ }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
Table length=10\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "
namedata_typedescription
str17str9str1594
search_posstringSearch Position (RA and Dec)
ArchiveFileIDintegerArchiveFileID
fileSetNamestringOf the form jwpppppooovvv_ggsaa_eeeee where ppppp - Program Number, ooo - Observation Number, vvv - Visit Number, gg - Visit Group, s - Parallel Sequence id (1 prime, 2-5 parallel), aa - Activity (base 36), and eeeee - Exposure Number, all padded by zeros where necessary. Source-based data set names are expressed as jwppppp-oOOO_tTTT_instrument, where: ppppp - Program number, OOO - observation number, TTT - target number, and instrument - Instrument name
productLevelstringProduct levels, 1b is an Uncal FITS file, 2a = Countrate exposure, 2b = calibrated exposure, 2c = cosmic-ray-flagged exposure, and 3 = combined data.
targpropstringProposer prefered name for the target
targnamestringStandard astronomical catalog name for the target
targ_raraRight Ascension (J2000), from 0 to 360, in degrees
targ_decdecDeclination (J2000), from -90 to +90, in degrees
instrumestringIdentifies the instrument used to acquire the data.
exp_typestringExposure type is used by DMS to direct Science Data Processing code (SDP) and calibration, as well as provide metadata for archive searches. For a given template the EXP_TYPE keyword value is obtained for each exposure from the EXPOSURE_TYPE exposure-level parameter in the Observatory Status File (OSF). In a few cases, additional information is required to set the value for the EXP_TYPE keyword
" + ], + "text/plain": [ + "\n", + " name ...\n", + " str17 ...\n", + "------------- ...\n", + " search_pos ...\n", + "ArchiveFileID ...\n", + " fileSetName ...\n", + " productLevel ...\n", + " targprop ...\n", + " targname ...\n", + " targ_ra ...\n", + " targ_dec ...\n", + " instrume ...\n", + " exp_type ..." + ] + }, + "execution_count": 4, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# Get available columns for JWST mission\n", "columns = missions.get_column_list()\n", @@ -203,7 +256,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Exercise**: Now it's your turn! Try querying for the Whirlpool Galaxy object. Search within a radius of 1 arcminute, skip the first 300 results, and select the `fileSetName` and `opticalElements` columns." + "**Exercise 1**: Now it's your turn! Try querying for the Whirlpool Galaxy object. Search within a radius of 1 arcminute, skip the first 300 results, and select the `fileSetName` and `opticalElements` columns." ] }, { @@ -212,12 +265,12 @@ "metadata": {}, "outputs": [], "source": [ - "# Query for Whirlpool Galaxy\n", - "results = missions.query_object(...) # Write your query!\n", + "# # Query for Whirlpool Galaxy\n", + "# results = missions.query_object(...) # Write your query!\n", "\n", - "# Display the first 5 results\n", - "print(f'Total number of results: {len(results)}')\n", - "results[:5]" + "# # Display the first 5 results\n", + "# print(f'Total number of results: {len(results)}')\n", + "# results[:5]" ] }, { @@ -257,7 +310,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Exercise**: JWST has observed the star Vega, which has a right ascension of 279.23473 degrees and a declination of 38.78369 degrees. Use the `query_region` function to search for datasets within 15 arcminutes of Vega. Select the `fileSetName`, `targprop`, `targ_ra`, and `targ_dec` columns." + "**Exercise 2**: JWST has observed the star Vega, which has a right ascension of 279.23473 degrees and a declination of 38.78369 degrees. Use the `query_region` function to search for datasets within 15 arcminutes of Vega. Select the `fileSetName`, `targprop`, `targ_ra`, and `targ_dec` columns." ] }, { @@ -266,15 +319,15 @@ "metadata": {}, "outputs": [], "source": [ - "# Vega coordinates\n", - "vega = SkyCoord(_, _, unit=('deg')) # Fill in with Vega's coordinates\n", + "# # Vega coordinates\n", + "# vega = SkyCoord(_, _, unit=('deg')) # Fill in with Vega's coordinates\n", "\n", - "# Query for datasets around Vega\n", - "results = missions.query_region(...) # Write your query!\n", + "# # Query for datasets around Vega\n", + "# results = missions.query_region(...) # Write your query!\n", "\n", - "# Display the first 5 results\n", - "print(f'Total number of results: {len(results)}')\n", - "results[:5]" + "# # Display the first 5 results\n", + "# print(f'Total number of results: {len(results)}')\n", + "# results[:5]" ] }, { @@ -444,7 +497,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Exercise**: It's time to apply all that you've learned! Write a non-positional query based on the following:\n", + "**Exercise 3**: It's time to apply all that you've learned! Write a non-positional query based on the following:\n", "\n", "- Fixed targets (*HINT*: `targtype='FIXED'`)\n", "- Instument is Mid-Infrared Instrument (MIRI) or Fine Guidance Sensor (FGS)\n", @@ -453,7 +506,7 @@ "- Right ascension is between 70 and 75 degrees\n", "- Program number is less than 1200.\n", "- Skip the first 5 entries.\n", - "- Select the following columns: `targtype`, `instrume`, `proposal_type`, `exp_type`, `targ_ra`, `program`" + "- Select the following columns: `fileSetName`, `targtype`, `instrume`, `proposal_type`, `exp_type`, `targ_ra`, `program`" ] }, { @@ -462,12 +515,12 @@ "metadata": {}, "outputs": [], "source": [ - "# A non-positional query with column criteria\n", - "results = missions.query_criteria(...) # Write your query here!\n", + "# # A non-positional query with column criteria\n", + "# results = missions.query_criteria(...) # Write your query here!\n", "\n", - "# Display results\n", - "print(f'Total number of results: {len(results)}')\n", - "results" + "# # Display results\n", + "# print(f'Total number of results: {len(results)}')\n", + "# results" ] }, { @@ -481,6 +534,87 @@ "- [`astroquery.mast` Documentation for Mission-Specific Searches](https://astroquery.readthedocs.io/en/latest/mast/mast_missions.html#mission-specific-search-queries)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Exercise Solutions\n", + "\n", + "**Exercise 1:**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Query for Whirlpool Galaxy\n", + "results = missions.query_object('Whirlpool',\n", + " radius=1, # Search radius of 1 arcminute\n", + " offset=300, # Skip the first 300 rows\n", + " select_cols=['fileSetName', 'opticalElements'])\n", + "\n", + "# Display the first 5 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Exercise 2:**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Vega coordinates\n", + "vega = SkyCoord(279.23473, 38.78369, unit=('deg'))\n", + "\n", + "# Query for datasets around Vega\n", + "results = missions.query_region(vega,\n", + " radius=15, # Search radius of 15 arcminutes\n", + " select_cols=['fileSetName', 'targprop', 'targ_ra', 'targ_dec'])\n", + "\n", + "# Display the first 5 results\n", + "print(f'Total number of results: {len(results)}')\n", + "results[:5]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Exercise 3:**" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# A non-positional query with column criteria\n", + "results = missions.query_criteria(targtype='FIXED', # Fixed target\n", + " instrume='MIRI, FGS', # Select MIRI and FGS observations\n", + " proposal_type='!GO', # Not from a general observer proposal\n", + " exp_type='*IMAGE*', # Contains the string \"IMAGE\"\n", + " targ_ra='70..75', # Between 70 and 75\n", + " program='<1200', # Less than 1200\n", + " offset=5, # Skip the first 5 results\n", + " select_cols=['fileSetName', 'targtype', 'instrume', 'proposal_type', \n", + " 'exp_type', 'targ_ra', 'program'])\n", + "\n", + "# Display results\n", + "print(f'Total number of results: {len(results)}')\n", + "results" + ] + }, { "cell_type": "markdown", "metadata": {}, From 5851795bff27fde674ed2e0f0fd0071b65a02eab Mon Sep 17 00:00:00 2001 From: Thomas Dutkiewicz <106269091+ttdu@users.noreply.github.com> Date: Wed, 26 Jun 2024 15:21:24 -0400 Subject: [PATCH 4/4] minor style updates --- .../Querying_MAST_for_JWST_Metadata.ipynb | 84 ++++--------------- 1 file changed, 18 insertions(+), 66 deletions(-) diff --git a/notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb b/notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb index ae39b1693..53787c4c7 100644 --- a/notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb +++ b/notebooks/JWST/MAST_metadata_search/Querying_MAST_for_JWST_Metadata.ipynb @@ -61,7 +61,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": null, "metadata": { "slideshow": { "slide_type": "fragment" @@ -103,18 +103,9 @@ }, { "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Mission: jwst\n", - "Service: search\n" - ] - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "# Create MastMissions object and assign mission to 'jwst'\n", "missions = MastMissions(mission='jwst')\n", @@ -132,50 +123,9 @@ }, { "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "data": { - "text/html": [ - "
Table length=10\n", - "
\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "
namedata_typedescription
str17str9str1594
search_posstringSearch Position (RA and Dec)
ArchiveFileIDintegerArchiveFileID
fileSetNamestringOf the form jwpppppooovvv_ggsaa_eeeee where ppppp - Program Number, ooo - Observation Number, vvv - Visit Number, gg - Visit Group, s - Parallel Sequence id (1 prime, 2-5 parallel), aa - Activity (base 36), and eeeee - Exposure Number, all padded by zeros where necessary. Source-based data set names are expressed as jwppppp-oOOO_tTTT_instrument, where: ppppp - Program number, OOO - observation number, TTT - target number, and instrument - Instrument name
productLevelstringProduct levels, 1b is an Uncal FITS file, 2a = Countrate exposure, 2b = calibrated exposure, 2c = cosmic-ray-flagged exposure, and 3 = combined data.
targpropstringProposer prefered name for the target
targnamestringStandard astronomical catalog name for the target
targ_raraRight Ascension (J2000), from 0 to 360, in degrees
targ_decdecDeclination (J2000), from -90 to +90, in degrees
instrumestringIdentifies the instrument used to acquire the data.
exp_typestringExposure type is used by DMS to direct Science Data Processing code (SDP) and calibration, as well as provide metadata for archive searches. For a given template the EXP_TYPE keyword value is obtained for each exposure from the EXPOSURE_TYPE exposure-level parameter in the Observatory Status File (OSF). In a few cases, additional information is required to set the value for the EXP_TYPE keyword
" - ], - "text/plain": [ - "\n", - " name ...\n", - " str17 ...\n", - "------------- ...\n", - " search_pos ...\n", - "ArchiveFileID ...\n", - " fileSetName ...\n", - " productLevel ...\n", - " targprop ...\n", - " targname ...\n", - " targ_ra ...\n", - " targ_dec ...\n", - " instrume ...\n", - " exp_type ..." - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], + "execution_count": null, + "metadata": {}, + "outputs": [], "source": [ "# Get available columns for JWST mission\n", "columns = missions.get_column_list()\n", @@ -191,7 +141,6 @@ "Before we dive in to the actual queries, it's important to know how we can refine our results with optional keyword arguments. The following parameters are available:\n", "\n", "- `radius`: For positional searches only. Only return results within a certain distance from an object or set of coordinates. Default is 3 arcminutes. \n", - "\n", "- `limit`: The maximum number of results to return. Default is 5000.\n", "- `offset`: Skip the first ***n*** results. Useful for paging through results.\n", "- `select_cols`: A list of columns to be returned in the response.\n", @@ -256,7 +205,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Exercise 1**: Now it's your turn! Try querying for the Whirlpool Galaxy object. Search within a radius of 1 arcminute, skip the first 300 results, and select the `fileSetName` and `opticalElements` columns." + "### Exercise 1\n", + "Now it's your turn! Try querying for the Whirlpool Galaxy object. Search within a radius of 1 arcminute, skip the first 300 results, and select the `fileSetName` and `opticalElements` columns." ] }, { @@ -310,7 +260,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Exercise 2**: JWST has observed the star Vega, which has a right ascension of 279.23473 degrees and a declination of 38.78369 degrees. Use the `query_region` function to search for datasets within 15 arcminutes of Vega. Select the `fileSetName`, `targprop`, `targ_ra`, and `targ_dec` columns." + "### Exercise 2: \n", + "JWST has observed the star Vega, which has a right ascension of 279.23473 degrees and a declination of 38.78369 degrees. Use the `query_region` function to search for datasets within 15 arcminutes of Vega. Select the `fileSetName`, `targprop`, `targ_ra`, and `targ_dec` columns." ] }, { @@ -497,7 +448,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Exercise 3**: It's time to apply all that you've learned! Write a non-positional query based on the following:\n", + "### Exercise 3\n", + "It's time to apply all that you've learned! Write a non-positional query based on the following:\n", "\n", "- Fixed targets (*HINT*: `targtype='FIXED'`)\n", "- Instument is Mid-Infrared Instrument (MIRI) or Fine Guidance Sensor (FGS)\n", @@ -540,7 +492,7 @@ "source": [ "## Exercise Solutions\n", "\n", - "**Exercise 1:**" + "### Exercise 1" ] }, { @@ -564,7 +516,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Exercise 2:**" + "### Exercise 2" ] }, { @@ -590,7 +542,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "**Exercise 3:**" + "### Exercise 3" ] }, { @@ -644,7 +596,7 @@ "\n", "***\n", "[Top of Page](#top)\n", - "\"Space " + "\"Space " ] } ], @@ -664,7 +616,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.12.2" + "version": "3.11.5" } }, "nbformat": 4,