MAST Queries (astroquery.mast
)¶
Getting Started¶
This module can be used to query the Barbara A. Mikulski Archive for Space Telescopes (MAST). Below are examples of the types of queries that can be used, and how to access data products.
Positional Queries¶
Positional queries can be based on a sky position or a target name. The observation fields are documented here.
>>> from astroquery.mast import Observations
>>> obsTable = Observations.query_region("322.49324 12.16683")
>>> print(obsTable[:10])
dataproduct_type obs_collection instrument_name ... distance
---------------- -------------- --------------- ... --------
cube SWIFT UVOT ... 0.0
cube SWIFT UVOT ... 0.0
cube SWIFT UVOT ... 0.0
cube SWIFT UVOT ... 0.0
cube SWIFT UVOT ... 0.0
cube SWIFT UVOT ... 0.0
cube SWIFT UVOT ... 0.0
cube SWIFT UVOT ... 0.0
cube SWIFT UVOT ... 0.0
cube SWIFT UVOT ... 0.0
Radius is an optional parameter and the default is 0.2 degrees.
>>> from astroquery.mast import Observations
>>> observations = Observations.query_object("M8",radius=".02 deg")
>>> print(observations[:10])
dataproduct_type obs_collection instrument_name ... distance
---------------- -------------- --------------- ... -------------
cube K2 Kepler ... 39.4914065162
spectrum IUE LWP ... 0.0
spectrum IUE LWP ... 0.0
spectrum IUE LWP ... 0.0
spectrum IUE LWR ... 0.0
spectrum IUE LWR ... 0.0
spectrum IUE LWR ... 0.0
spectrum IUE LWR ... 0.0
spectrum IUE LWR ... 0.0
spectrum IUE LWR ... 0.0
Observation Criteria Queries¶
To search for observations based on parameters other than position or target name,
use query_criteria
.
Criteria are supplied as keyword arguments, where valid criteria are “coordinates”,
“objectname”, “radius” (as in query_region
and
query_object
), and all observation fields listed
here.
Argument values are one or more acceptable values for the criterion, except for fields with a float datatype where the argument should be in the form [minVal, maxVal]. For non-float type criteria, wildcards (both * and %) may be used. However, only one wildcarded value can be processed per criterion.
RA and Dec must be given in decimal degrees, and datetimes in MJD.
>>> from astroquery.mast import Observations
>>> obsTable = Observations.query_criteria(dataproduct_type=["image"],
proposal_pi="Osten",
s_dec=[43.5,45.5])
>>> print(obsTable)
dataproduct_type calib_level obs_collection ... dataURL obsid objID
---------------- ----------- -------------- ... ------- ---------- ----------
image 1 HST ... None 2003520266 2011133418
image 1 HST ... None 2003520267 2011133419
image 1 HST ... None 2003520268 2011133420
>>> obsTable = Observations.query_criteria(filters=["*UV","Kepler"],objectname="M101")
>>> print(obsTable)
dataproduct_type calib_level obs_collection ... objID1 distance
---------------- ----------- -------------- ... ---------- -------------
image 2 GALEX ... 1000055044 0.0
image 2 GALEX ... 1000004937 3.83290685323
image 2 GALEX ... 1000045953 371.718371962
image 2 GALEX ... 1000055047 229.810616011
image 2 GALEX ... 1000016644 229.810616011
image 2 GALEX ... 1000045952 0.0
image 2 GALEX ... 1000048357 0.0
image 2 GALEX ... 1000001326 0.0
image 2 GALEX ... 1000001327 371.718371962
image 2 GALEX ... 1000004203 0.0
image 2 GALEX ... 1000016641 0.0
image 2 GALEX ... 1000048943 3.83290685323
Getting Observation Counts¶
To get the number of observations and not the observations themselves, query_counts functions are available. This can be useful if trying to decide whether the available memory is sufficient for the number of observations.
>>> from astroquery.mast import Observations
>>> print(Observations.query_region_count("322.49324 12.16683"))
1804
>>> print(Observations.query_object_count("M8",radius=".02 deg"))
196
>>> print(Observations.query_criteria_count(dataproduct_type="image",
filters=["NUV","FUV"],
t_max=[52264.4586,54452.8914]))
59033
Listing Available Missions¶
To list data missions archived by MAST and avaiable through astroquery.mast
, use the list_missions
function.
>>> from astroquery.mast import Observations
>>> print(Observations.list_missions())
['IUE', 'Kepler', 'K2FFI', 'EUVE', 'HLA', 'KeplerFFI','FUSE',
'K2', 'HST', 'WUPPE', 'BEFS', 'GALEX', 'TUES','HUT', 'SWIFT']
Downloading Data¶
Getting Product Lists¶
Each observation returned from a MAST query can have one or more associated data products.
Given one or more observations or observation ids (“obsid”)
get_product_list
will return
a Table
containing the associated data products.
The product fields are documented here.
>>> from astroquery.mast import Observations
>>> obsTable = Observations.query_object("M8",radius=".02 deg")
>>> dataProductsByObservation = Observations.get_product_list(obsTable[0:2])
>>> print(dataProductsByObservation)
obsID obs_collection ... productFilename size
---------- -------------- ... ---------------------------------- --------
3000007760 IUE ... lwp13058.elbll.gz 185727
3000007760 IUE ... lwp13058.elbls.gz 183350
3000007760 IUE ... lwp13058.lilo.gz 612715
3000007760 IUE ... lwp13058.melol.gz 12416
3000007760 IUE ... lwp13058.melos.gz 12064
3000007760 IUE ... lwp13058.raw.gz 410846
3000007760 IUE ... lwp13058.rilo.gz 416435
3000007760 IUE ... lwp13058.silo.gz 100682
3000007760 IUE ... lwp13058.gif 8971
3000007760 IUE ... lwp13058.mxlo.gz 18206
3000007760 IUE ... lwp13058mxlo_vo.fits 48960
3000007760 IUE ... lwp13058.gif 3967
9500243833 K2 ... k2-tpf-only-target_bw_large.png 9009
9500243833 K2 ... ktwo200071160-c91_lpd-targ.fits.gz 39930404
9500243833 K2 ... ktwo200071160-c92_lpd-targ.fits.gz 62213068
9500243833 K2 ... k2-tpf-only-target_bw_thumb.png 1301
>>> obsids = obsTable[0:2]['obsid']
>>> dataProductsByID = Observations.get_product_list(obsids)
>>> print(dataProductsByID)
obsID obs_collection ... productFilename size
---------- -------------- ... ---------------------------------- --------
3000007760 IUE ... lwp13058.elbll.gz 185727
3000007760 IUE ... lwp13058.elbls.gz 183350
3000007760 IUE ... lwp13058.lilo.gz 612715
3000007760 IUE ... lwp13058.melol.gz 12416
3000007760 IUE ... lwp13058.melos.gz 12064
3000007760 IUE ... lwp13058.raw.gz 410846
3000007760 IUE ... lwp13058.rilo.gz 416435
3000007760 IUE ... lwp13058.silo.gz 100682
3000007760 IUE ... lwp13058.gif 8971
3000007760 IUE ... lwp13058.mxlo.gz 18206
3000007760 IUE ... lwp13058mxlo_vo.fits 48960
3000007760 IUE ... lwp13058.gif 3967
9500243833 K2 ... k2-tpf-only-target_bw_large.png 9009
9500243833 K2 ... ktwo200071160-c91_lpd-targ.fits.gz 39930404
9500243833 K2 ... ktwo200071160-c92_lpd-targ.fits.gz 62213068
9500243833 K2 ... k2-tpf-only-target_bw_thumb.png 1301
>>> print((dataProductsByObservation == dataProductsByID).all())
True
Downloading Data Products¶
Products can be downloaded by using download_products
,
with a Table
of data products, or a list (or single) obsid as the argument.
By default only Minimum Recommended Products will be downloaded.
>>> from astroquery.mast import Observations
>>> obsid = '3000007760'
>>> dataProductsByID = Observations.get_product_list(obsid)
>>> manifest = Observations.download_products(dataProductsByID)
Downloading URL http://archive.stsci.edu/pub/iue/data/lwp/13000/lwp13058.mxlo.gz to ./mastDownload/IUE/lwp13058/lwp13058.mxlo.gz ... [Done]
Downloading URL http://archive.stsci.edu/pub/vospectra/iue2/lwp13058mxlo_vo.fits to ./mastDownload/IUE/lwp13058/lwp13058mxlo_vo.fits ... [Done]
>>> print(manifest)
Local Path Status Message URL
------------------------------------------------ -------- ------- ----
./mastDownload/IUE/lwp13058/lwp13058.mxlo.gz COMPLETE None None
./mastDownload/IUE/lwp13058/lwp13058mxlo_vo.fits COMPLETE None None
As an alternative to downloading the data files now, the curl_flag can be used instead to instead get a curl script that can be used to download the files at a later time.
>>> from astroquery.mast import Observations
>>> Observations.download_products('2003839997',
mrp_only=False,
productType="SCIENCE",
curl_flag=True)
Downloading URL https://mast.stsci.edu/portal/Download/stage/anonymous/public/514cfaa9-fdc1-4799-b043-4488b811db4f/mastDownload_20170629162916.sh to ./mastDownload_20170629162916.sh ... [Done]
Filtering¶
Filter keyword arguments can be applied to download only data products that meet the given criteria. Available filters are “mrp_only” (Minium Recommended Products), “extension” (file extension), and all products fields listed here.
Important: mrp_only defaults to True.
The ‘AND’ operation is performed for a list of filters, and the ‘OR’ operation is performed within a filter set. The below example illustrates downloading all product files with the extension “fits” that are either “RAW” or “UNCAL.”
>>> from astroquery.mast import Observations
>>> Observations.download_products('2003839997',
mrp_only=False,
productSubGroupDescription=["RAW", "UNCAL"],
extension="fits")
Downloading URL https://mast.stsci.edu/api/v0/download/file/HST/product/ib3p11p7q_raw.fits to ./mastDownload/HST/IB3P11P7Q/ib3p11p7q_raw.fits ... [Done]
Downloading URL https://mast.stsci.edu/api/v0/download/file/HST/product/ib3p11p8q_raw.fits to ./mastDownload/HST/IB3P11P8Q/ib3p11p8q_raw.fits ... [Done]
Downloading URL https://mast.stsci.edu/api/v0/download/file/HST/product/ib3p11phq_raw.fits to ./mastDownload/HST/IB3P11PHQ/ib3p11phq_raw.fits ... [Done]
Downloading URL https://mast.stsci.edu/api/v0/download/file/HST/product/ib3p11q9q_raw.fits to ./mastDownload/HST/IB3P11Q9Q/ib3p11q9q_raw.fits ... [Done]
Product filtering can also be appllied directly to a table of products without proceeding to the download step.
>>> from astroquery.mast import Observations
>>> dataProductsByID = Observations.get_product_list('2003839997')
>>> print(len(dataProductsByID))
31
>>> dataProductsByID = Observations.filter_products(dataProductsByID,
mrp_only=False,
productSubGroupDescription=["RAW", "UNCAL"],
extenstion="fits")
>>> print(len(dataProductsByID))
4
Accessing Proprietary Data¶
To access data that is not publicly available users may log into their
MyST Account.
This can be done by using the login
function,
or by initializing a class instance with a username/password.
If a password is not supplied, the user will be prompted to enter one.
>>> from astroquery.mast import Observations
>>> Observations.login(username="testUser@stsci.edu",password="testPwd")
Authentication successful!
Session Expiration: 600 minute(s)
>>> sessionInfo = Observations.session_info()
Session Expiration: 559.0 min
Username: testUser@stsci.edu
First Name: Test
Last Name: User
>>> from astroquery.mast import Observations
>>> mySession = Observations(username="testUser@stsci.edu",password="testPwd")
Authentication successful!
Session Expiration: 600 minute(s)
>>> sessionInfo = mySession.session_info()
Session Expiration: 559.0 min
Username: testUser@stsci.edu
First Name: Test
Last Name: User
* For security passwords should not be typed into a terminal or Jupyter notebook
but instead input using a more secure method such as getpass
.
MAST login can also be achieved with a “session token,” from an existing valid MAST session.
>>> from astroquery.mast import Observations
>>> from astroquery.mast import Mast
>>> myObsSession = Observations(username="testUser@stsci.edu",password="testPwd")
Authentication successful!
Session Expiration: 600 minute(s)
>>> myToken = myObsSession.get_token()
>>> Mast.login(session_token=myToken)
Authentication successful!
Session Expiration: 599 minute(s)
MAST sessions expire after 600 minutes, at which point the user must login again.
The store_password
argument can be used to store the username and password securely in the user’s keyring.
If the username/password are thus stored, only the username need be entered to login.
This password can be overwritten using the reenter_password
argument.
To logout before a session expires, the logout
method may be used.
Direct Mast Queries¶
The Mast class provides more direct access to the MAST interface. It requires more knowledge of the inner workings of the MAST API, and should be rarely needed. However in the case of new functionality not yet implemented in astroquery, this class does allow access. See the MAST api documentation for more information.
The basic MAST query function returns query results as an Table
.
>>> from astroquery.mast import Mast
>>> service = 'Mast.Caom.Cone'
>>> params = {'ra':184.3,
'dec':54.5,
'radius':0.2}
>>> observations = Mast.mashup_request(service, params)
>>> print(observations)
dataproduct_type obs_collection instrument_name ... distance _selected_
---------------- -------------- --------------- ... ------------- ----------
image GALEX GALEX ... 0.0 False
image GALEX GALEX ... 0.0 False
image GALEX GALEX ... 0.0 False
image GALEX GALEX ... 0.0 False
image GALEX GALEX ... 0.0 False
image GALEX GALEX ... 302.405835798 False
image GALEX GALEX ... 302.405835798 False
If the output is not the MAST json result type it cannot be properly parsed into a Table
.
In this case, the async method should be used to get the raw http response, which can then be manually parsed.
>>> from astroquery.mast import Mast
>>> service = 'Mast.Name.Lookup'
>>> params ={'input':"M8",
'format':'json'}
>>> response = Mast.mashup_request_async(service,params)
>>> result = response[0].json()
>>> print(result)
{'resolvedCoordinate': [{'cacheDate': 'Apr 12, 2017 9:28:24 PM',
'cached': True,
'canonicalName': 'MESSIER 008',
'decl': -24.38017,
'objectType': 'Neb',
'ra': 270.92194,
'resolver': 'NED',
'resolverTime': 113,
'searchRadius': -1.0,
'searchString': 'm8'}],
'status': ''}
Reference/API¶
astroquery.mast Package¶
MAST Query Tool¶
This module contains various methods for querying the MAST Portal.
Classes¶
ObservationsClass ([username, password, …]) |
MAST Observations query class. |
MastClass ([username, password, session_token]) |
MAST query class. |
Conf |
Configuration parameters for astroquery.mast . |