This tutorial demonstrates how to use R to connect to the AppEEARS API The Application for Extracting and Exploring Analysis Ready Samples (AppEEARS) offers a simple and efficient way to access and transform geospatial data from a variety of federal data archives in an easy-to-use web application interface. AppEEARS enables users to subset geospatial data spatially, temporally, and by band/layer for point and area samples. AppEEARS returns not only the requested data, but also the associated quality values, and offers interactive visualizations with summary statistics in the web interface. The AppEEARS API offers users programmatic access to all features available in AppEEARS, with the exception of visualizations. The API features are demonstrated in this tutorial.


Example: Submit a point request with multiple points in U.S. National Parks for extracting vegetation and land surface temperature data

In this tutorial, Connecting to the AppEEARS API, querying the list of available products, submitting a point sample request, downloading the request, working with the AppEEARS Quality API, and loading the results into R for visualization are covered. AppEEARS point requests allow users to subset their desired data using latitude/longitude geographic coordinate pairs (points) for a time period of interest, and for specific data layers within data products. AppEEARS returns the valid data from the parameters defined within the sample request.

Data Used in the Example:

  • Data layers:
    • Combined MODIS Leaf Area Index (LAI)
    • Terra MODIS Land Surface Temperature
      • MOD11A2.061, 1000m, 8 day: ‘LST_Day_1km’, ‘LST_Night_1km’

Topics Covered in this tutorial:

  1. Getting Started
    1a. Load Packages 1b. Set Up the Output Directory
    1c. Login
  2. Query Available Products
    2a. Search and Explore Available Products
    2b. Search and Explore Available Layers
  3. Submit a Point Request
    3a. Compile a JSON Object 3b. Submit a Task Request 3c. Retrieve Task Status
  4. Download a Request
    4a. Explore Files in Request Output
    4b. Download Files in a Request (Automation)
  5. Explore AppEEARS Quality API
    5a. List Quality Layers
    5b. Show Quality Values
    5c. Decode Quality Values
  6. BONUS: Load Request Output and Visualize
    6a. Load CSV
    6b. Plot Results (Line/Scatter Plots)

Prerequisites:

  • A NASA Earthdata Login account is required to login to the AppEEARS API and submit a request . You can create an account at the link provided.

  • Install R and RStudio. These tutorials have been tested on Windows and MAc systems using R Version 4.0.0, RStudio version 1.1.463, and the specifications listed below.

    • Required packages:
      • getPass
      • httr
      • jsonlite
      • warnings
    • To read and visualize the tabular data:
      • dplyr
      • tidyr
      • readr
      • ggplot2

Procedures:

Getting Started:

  1. Clone/download AppEEARS API Getting Started in R Repository from the LP DAAC Data User Resources Repository.

  2. Open the AppEEARS_API_R.Rproj file to directly open the project. Next, select the AppEEARS_API_Point_R.Rmd from the files list and open it.

Environment Setup:

1. Check the version of R using version and RStudio using RStudio.Version() and update them if needed.

  • Windows
    • Install and load installr:
      • Install.packages("installr");library(installr)
    • Copy/Update the existing packages to the new R installation:
    • UpdateR()
    • Open RStudio, go to Help > Check for Updates to install newer version of RStudio (if available).
  • Mac
    • Go to https://cloud.r-project.org/bin/macosx/.
      • Download the latest release (R-4.0.1.pkg) and finish the installation.
      • Open RStudio, go to Help > Check for Updates to install newer version of RStudio (if available).
      • To update packages, go to Tools > Check for Package Updates. If updates are available, select All, and click Install Updates.

2. Download any required packages that you are missing using either options below.

  • Use install.packages('package name') command in the console. To download multiple packages use install.packages(c('package #1', 'package #2',...)).

  • Run AppEEARS_API_install.R to find the missing packages and install them.

source("../Scripts/AppEEARS_API_Point_Install.R")
## All packages already installed

AppEEARS Information:

To access AppEEARS, visit: https://appeears.earthdatacloud.nasa.gov/.

For comprehensive documentation of the full functionality of the AppEEARS API, please see the AppEEARS API Documentation.


1. Getting Started

1a. Load Packages

First, load the R packages necessary to run the tutorial.

# Load necessary packages into R                                               
library(getPass)            # A micro-package for reading passwords
library(httr)               # To send a request to the server/receive a response from the server
library(jsonlite)           # Implements a bidirectional mapping between JSON data and the most important R data types
library(ggplot2)            # Functions for graphing and mapping
library(tidyr)              # Function for working with tabular data
library(dplyr)              # Function for working with tabular data
library(readr)              # Read rectangular data like CSV

1b. Set Up the Output Directory

Set your input directory, and create an output directory for the results.

outDir <- file.path("R_output")                 # Create an output directory if it doesn't exist
suppressWarnings(dir.create(outDir))                 

1c. Login

To submit a request, you must first login to the AppEEARS API. Use the getpass package to enter your NASA Earthdata login Username and Password. When prompted after executing the code block below, enter your username followed by your password.

user <- getPass(msg = "Enter NASA Earthdata Login Username: ")     # Enter NASA Earthdata Login Username
## Please enter password in TK window (Alt+Tab)
password <- getPass(msg = "Enter NASA Earthdata Login Password: ") # Enter NASA Earthdata Login Password
## Please enter password in TK window (Alt+Tab)

Decode the username and password to be used to post login request.

secret <- jsonlite::base64_enc(paste(user, password, sep = ":"))  # Encode the string of username and password

Next, assign the AppEEARS API URL to a static variable.

API_URL = 'https://appeears.earthdatacloud.nasa.gov/api/'  # Set the AppEEARS API to a variable

Use the httr package to post your username and password. A successful login will provide you with a token to be used later in this tutorial to submit a request. For more information or if you are experiencing difficulties, please see the API Documentation.

# Insert API URL, call login service, set the component of HTTP header, and post the request to the server
response <- httr::POST(paste0(API_URL,"login"), 
                       add_headers("Authorization" = paste("Basic", gsub("\n", "", secret)),
                                   "Content-Type" ="application/x-www-form-urlencoded;charset=UTF-8"),
                       body = "grant_type=client_credentials")

response_content <- content(response)                          # Retrieve the content of the request
token_response <- toJSON(response_content, auto_unbox = TRUE)  # Convert the response to the JSON object
remove(user, password, secret, response)                       # Remove the variables that are not needed anymore 
prettify(token_response)                                       # Print the prettified response
## {
##     "token_type": "Bearer",
##     "token": "EXIW5BKOfm6GyO99WnD3Acwexo-JN5cNdlksZcdRB34C9x9kmCVxI34hwyjYKyO3hfXfmHk32ZdrcAg59QVb2Q",
##     "expiration": "2022-04-06T21:08:05Z"
## }
## 

Above, you should see a Bearer token. Notice that this token will expire approximately 48 hours after being acquired.


2. Query Available Products

The product API provides details about all of the products and layers available in AppEEARS. For more information, please see the API Documentation.

Below, call the product API to list all of the products available in AppEEARS.

prods_req <- GET(paste0(API_URL, "product"))             # Request the info of all products from product service
prods_content <- content(prods_req)                      # Retrieve the content of request 
all_Prods <- toJSON(prods_content, auto_unbox = TRUE)    # Convert the info to JSON object
remove(prods_req, prods_content)                         # Remove the variables that are not needed anymore
# prettify(all_Prods)                                      # Print the prettified product response

2a. Search and Explore Available Products

Create a list indexed by product name to make it easier to query a specific product.

# Divides information from each product.
divided_products <- split(fromJSON(all_Prods), seq(nrow(fromJSON(all_Prods))))
# Create a list indexed by the product name and version
products <- setNames(divided_products,fromJSON(all_Prods)$ProductAndVersion)
# Print no. products available in AppEEARS
sprintf("AppEEARS currently supports %i products." ,length(products))   
## [1] "AppEEARS currently supports 155 products."

Next, look at the product’s names and descriptions. Below, the ‘ProductAndVersion’ and ‘Description’ are printed for all products.

# Loop through the products in the list and print the product name and description
for (p in products){                                   
  print(paste0(p$ProductAndVersion," is ",p$Description," from ",p$Source))  
}
## [1] "GPW_DataQualityInd.004 is Quality of Input Data for Population Count and Density Grids from SEDAC"
## [1] "GPW_UN_Adj_PopCount.004 is UN-adjusted Population Count from SEDAC"
## [1] "GPW_UN_Adj_PopDensity.004 is UN-adjusted Population Density from SEDAC"
## [1] "MCD12Q1.006 is Land Cover Type from LP DAAC"
## [1] "MCD12Q2.006 is Land Cover Dynamics from LP DAAC"
## [1] "MCD15A2H.006 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MCD15A3H.006 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MCD43A1.006 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A1.061 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A2.061 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A3.006 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A3.061 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A4.006 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD43A4.061 is Bidirectional Reflectance Distribution Function (BRDF) and Albedo from LP DAAC"
## [1] "MCD64A1.006 is Burned Area (fire) from LP DAAC"
## [1] "MCD64A1.061 is Burned Area (fire) from LP DAAC"
## [1] "MOD09A1.006 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MOD09A1.061 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MOD09GA.006 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MOD09GA.061 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MOD09GQ.006 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MOD09GQ.061 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MOD09Q1.006 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MOD09Q1.061 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MOD10A1.005 is Snow Cover from NSIDC DAAC"
## [1] "MOD10A1.006 is Snow Cover (NDSI) from NSIDC DAAC"
## [1] "MOD10A2.005 is Snow Cover from NSIDC DAAC"
## [1] "MOD10A2.006 is Snow Cover from NSIDC DAAC"
## [1] "MOD11A1.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MOD11A1.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MOD11A2.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MOD11A2.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MOD13A1.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A1.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A2.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A2.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A3.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13A3.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13Q1.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD13Q1.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MOD14A2.006 is Thermal Anomalies and Fire from LP DAAC"
## [1] "MOD14A2.061 is Thermal Anomalies and Fire from LP DAAC"
## [1] "MOD15A2H.006 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MOD15A2H.061 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MOD16A2.006 is Evapotranspiration (ET & LE) from LP DAAC"
## [1] "MOD16A2.061 is Evapotranspiration (ET & LE) from LP DAAC"
## [1] "MOD16A2GF.006 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MOD16A2GF.061 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MOD16A3GF.006 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MOD16A3GF.061 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MOD17A2H.006 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MOD17A2H.061 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MOD17A2HGF.006 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MOD17A2HGF.061 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MOD17A3HGF.006 is Net Primary Production (NPP) Gap-Filled from LP DAAC"
## [1] "MOD17A3HGF.061 is Net Primary Production (NPP) Gap-Filled from LP DAAC"
## [1] "MOD21A1D.061 is Temperature and Emissivity (LST&E) from LP DAAC"
## [1] "MOD21A1N.061 is Temperature and Emissivity (LST&E) from LP DAAC"
## [1] "MOD21A2.061 is Temperature and Emissivity (LST&E) from LP DAAC"
## [1] "MOD44B.006 is Vegetation Continuous Fields (VCF) from LP DAAC"
## [1] "MOD44W.006 is Land/Water Mask from LP DAAC"
## [1] "MODOCGA.006 is Ocean Reflectance Bands 8-16 from LP DAAC"
## [1] "MODTBGA.006 is Thermal Bands and Albedo from LP DAAC"
## [1] "MYD09A1.006 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MYD09A1.061 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MYD09GA.006 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MYD09GA.061 is Surface Reflectance Bands 1-7 from LP DAAC"
## [1] "MYD09GQ.006 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MYD09GQ.061 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MYD09Q1.006 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MYD09Q1.061 is Surface Reflectance Bands 1-2 from LP DAAC"
## [1] "MYD10A1.005 is Snow Cover from NSIDC DAAC"
## [1] "MYD10A1.006 is Snow Cover (NDSI) from NSIDC DAAC"
## [1] "MYD10A2.005 is Snow Cover from NSIDC DAAC"
## [1] "MYD10A2.006 is Snow Cover from NSIDC DAAC"
## [1] "MYD11A1.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD11A1.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD11A2.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD11A2.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD13A1.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A1.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A2.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A2.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A3.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13A3.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13Q1.006 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD13Q1.061 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "MYD14A2.006 is Thermal Anomalies and Fire from LP DAAC"
## [1] "MYD14A2.061 is Thermal Anomalies and Fire from LP DAAC"
## [1] "MYD15A2H.006 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MYD15A2H.061 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "MYD16A2.006 is Evapotranspiration (ET & LE) from LP DAAC"
## [1] "MYD16A2.061 is Evapotranspiration (ET & LE) from LP DAAC"
## [1] "MYD16A2GF.006 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MYD16A2GF.061 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MYD16A3GF.006 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MYD16A3GF.061 is Net Evapotranspiration Gap-Filled (ET & LE) from LP DAAC"
## [1] "MYD17A2H.006 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MYD17A2H.061 is Gross Primary Productivity (GPP) from LP DAAC"
## [1] "MYD17A2HGF.006 is Gross Primary Productivity (GPP) Gap-Filled from LP DAAC"
## [1] "MYD17A2HGF.061 is Gross Primary Productivity (GPP) Gap-Filled from LP DAAC"
## [1] "MYD17A3HGF.006 is Net Primary Production (NPP) Gap-Filled from LP DAAC"
## [1] "MYD17A3HGF.061 is Net Primary Production (NPP) Gap-Filled from LP DAAC"
## [1] "MYD21A1D.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A1D.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A1N.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A1N.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A2.006 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYD21A2.061 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "MYDOCGA.006 is Ocean Reflectance Bands 8-16 from LP DAAC"
## [1] "MYDTBGA.006 is Thermal Bands and Albedo from LP DAAC"
## [1] "NASADEM_NC.001 is Elevation from LP DAAC"
## [1] "NASADEM_NUMNC.001 is Source from LP DAAC"
## [1] "SPL3SMP_E.004 is Enhanced L3 Radiometer Soil Moisture from NSIDC DAAC"
## [1] "SPL3SMP.007 is Soil Moisture from NSIDC DAAC"
## [1] "SPL3FTP.003 is Freeze/Thaw State from NSIDC DAAC"
## [1] "SRTMGL1_NC.003 is Elevation (DEM) from LP DAAC"
## [1] "SRTMGL1_NUMNC.003 is Source (DEM) from LP DAAC"
## [1] "SRTMGL3_NC.003 is Elevation (DEM) from LP DAAC"
## [1] "SRTMGL3_NUMNC.003 is Source (DEM) from LP DAAC"
## [1] "ASTGTM_NC.003 is Elevation from LP DAAC"
## [1] "ASTGTM_NUMNC.003 is Source from LP DAAC"
## [1] "ASTWBD_ATTNC.001 is Water Bodies Database Attributes from LP DAAC"
## [1] "ASTWBD_NC.001 is Water Bodies Database Elevation from LP DAAC"
## [1] "VNP09H1.001 is Surface Reflectance from LP DAAC"
## [1] "VNP09A1.001 is Surface Reflectance from LP DAAC"
## [1] "VNP09GA.001 is Surface Reflectance from LP DAAC"
## [1] "VNP13A1.001 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "VNP13A2.001 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "VNP13A3.001 is Vegetation Indices (NDVI & EVI) from LP DAAC"
## [1] "VNP14A1.001 is Thermal Anomalies/Fire from LP DAAC"
## [1] "VNP15A2H.001 is Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR) from LP DAAC"
## [1] "VNP21A1D.001 is Land Surface Temperature & Emissivity Day (LST&E) from LP DAAC"
## [1] "VNP21A1N.001 is Land Surface Temperature & Emissivity Night (LST&E) from LP DAAC"
## [1] "VNP21A2.001 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "VNP22Q2.001 is Global Land Surface Phenology (GLSP) from LP DAAC"
## [1] "VNP43IA1.001 is BRDF-Albedo Model Parameters from LP DAAC"
## [1] "VNP43IA2.001 is BRDF-Albedo Quality from LP DAAC"
## [1] "VNP43IA3.001 is Albedo (BRDF) from LP DAAC"
## [1] "VNP43IA4.001 is Nadir BRDF-Adjusted Reflectance from LP DAAC"
## [1] "VNP43MA1.001 is BRDF-Albedo Model Parameters from LP DAAC"
## [1] "VNP43MA2.001 is BRDF-Albedo Quality from LP DAAC"
## [1] "VNP43MA3.001 is Albedo (BRDF) from LP DAAC"
## [1] "VNP43MA4.001 is Nadir BRDF-Adjusted Reflectance from LP DAAC"
## [1] "DAYMET.004 is Daily Surface Weather Data for North America from ORNL"
## [1] "ECO2LSTE.001 is Land Surface Temperature & Emissivity (LST&E) from LP DAAC"
## [1] "ECO2CLD.001 is Cloud Mask from LP DAAC"
## [1] "ECO3ETPTJPL.001 is Evapotranspiration PT-JPL from LP DAAC"
## [1] "ECO3ANCQA.001 is L3/L4 Ancillary Data Quality Assurance (QA) Flags from LP DAAC"
## [1] "ECO4ESIPTJPL.001 is Evaporative Stress Index PT-JPL from LP DAAC"
## [1] "ECO4WUE.001 is Water Use Efficiency from LP DAAC"
## [1] "ECO1BGEO.001 is Geolocation from LP DAAC"
## [1] "ECO1BMAPRAD.001 is Resampled Radiance from LP DAAC"
## [1] "ECO3ETALEXI.001 is Evapotranspiration dis-ALEXI from LP DAAC"
## [1] "ECO4ESIALEXI.001 is Evaporative Stress Index dis-ALEXI from LP DAAC"

The product service provides many useful details, including if a product is currently available in AppEEARS, a description, and information on the spatial and temporal resolution. Below, the product details are retrieved using ‘ProductAndVersion’.

# Convert the MCD15A3H.006 info to JSON object and print the prettified info
prettify(toJSON(products$"MCD15A3H.006")) 
## [
##     {
##         "Product": "MCD15A3H",
##         "Platform": "Combined MODIS",
##         "Description": "Leaf Area Index (LAI) and Fraction of Photosynthetically Active Radiation (FPAR)",
##         "RasterType": "Tile",
##         "Resolution": "500m",
##         "TemporalGranularity": "4 day",
##         "Version": "006",
##         "Available": true,
##         "DocLink": "https://doi.org/10.5067/MODIS/MCD15A3H.006",
##         "Source": "LP DAAC",
##         "TemporalExtentStart": "2002-07-04",
##         "TemporalExtentEnd": "Present",
##         "Deleted": false,
##         "DOI": "10.5067/MODIS/MCD15A3H.006",
##         "ProductAndVersion": "MCD15A3H.006"
##     }
## ]
## 

Also, the products can be searched using their description. Below, search for products containing Leaf Area Index in their description and make a list of their productAndVersion.

LAI_Products <- list()                                        # Create an empty list 
for (p in products){                                          # Loop through the product list
  if (grepl('Leaf Area Index', p$Description )){              # Look through the product description for a keyword 
    LAI_Products <- append(LAI_Products, p$ProductAndVersion) # Append the LAI products to the list
  }
}
LAI_Products
## [[1]]
## [1] "MCD15A2H.006"
## 
## [[2]]
## [1] "MCD15A3H.006"
## 
## [[3]]
## [1] "MOD15A2H.006"
## 
## [[4]]
## [1] "MOD15A2H.061"
## 
## [[5]]
## [1] "MYD15A2H.006"
## 
## [[6]]
## [1] "MYD15A2H.061"
## 
## [[7]]
## [1] "VNP15A2H.001"

Using the info above, Create a list of desired products.

desired_products <- c('MCD15A3H.006','MOD11A2.061')   # Create a vector of desired products 
desired_products
## [1] "MCD15A3H.006" "MOD11A2.061"

2b. Search and Explore Available Layers

This API call will list all of the layers available for a given product. Each product is referenced by its ProductAndVersion property which is also referred to as the product_id. First, request the layers for the MCD15A3H.006 product.

# Request layers for the 1st product in the list: MCD15A3H.006
MCD15A3H_req <- GET(paste0(API_URL,"product/", desired_products[1]))  # Request the info of a product from product URL
MCD15A3H_content <- content(MCD15A3H_req)                             # Retrieve content of the request 
MCD15A3H_response <- toJSON(MCD15A3H_content, auto_unbox = TRUE)      # Convert the content to JSON object
remove(MCD15A3H_req, MCD15A3H_content)                                # Remove the variables that are not needed anymore
#prettify(MCD15A3H_response)                                          # Print the prettified response
names(fromJSON(MCD15A3H_response))                                    # print the layer's names    
## [1] "FparExtra_QC"    "FparLai_QC"      "FparStdDev_500m" "Fpar_500m"      
## [5] "LaiStdDev_500m"  "Lai_500m"

Next, request the layers for the MOD11A2.061 product.

MOD11_req <- GET(paste0(API_URL,"product/", desired_products[2]))  # Request the info of a product from product URL
MOD11_content <- content(MOD11_req)                                # Retrieve content of the request
MOD11_response <- toJSON(MOD11_content, auto_unbox = TRUE)         # Convert the content to JSON object
remove(MOD11_req, MOD11_content)                                   # Remove the variables that are not needed anymore
names(fromJSON(MOD11_response))                                    # print the layer names
##  [1] "Clear_sky_days"   "Clear_sky_nights" "Day_view_angl"    "Day_view_time"   
##  [5] "Emis_31"          "Emis_32"          "LST_Day_1km"      "LST_Night_1km"   
##  [9] "Night_view_angl"  "Night_view_time"  "QC_Day"           "QC_Night"

Lastly, select the desired layers and pertinent products and make a data frame using this information. This data frame will be inserted into the nested data frame that will be used to create a JSON object to submit a request in Section 3.

desired_layers <- c("LST_Day_1km","LST_Night_1km","Lai_500m")   # Create a vector of desired layers
desired_prods <- c("MOD11A2.061","MOD11A2.061","MCD15A3H.006")  # Create a vector of products including the desired layers
# Create a data frame including the desired data products and layers
layers <- data.frame(product = desired_prods, layer = desired_layers) 

3. Submit a Point Request

The Submit task API call provides a way to submit a new request to be processed. It can accept data via JSON or query string. In the example below, create a JSON object and submit a request. Tasks in AppEEARS correspond to each request associated with your user account. Therefore, each of the calls to this service requires an authentication token (see Section 1c.).


3a. Compile a JSON Object

In this section, begin by setting up the information needed for a nested data frame that will be later converted to a JSON object for submitting an AppEEARS point request. For detailed information on required JSON parameters, see the API Documentation.

For point requests, beside the date range and desired layers information, the coordinates property must also be inside the task object. Optionally, set id and category properties to further identify your selected coordinates.

startDate <- "01-01-2018"       # Start of the date range for  which to extract data: MM-DD-YYYY
endDate <- "12-31-2018"         # End of the date range for  which to extract data: MM-DD-YYYY

# If you are interested in submitting a request using a recurring time in a year, set the time variables as below:
# startDate <- '05-01'                 # change start/end date to MM-DD
# endDate <- '06-30'
# recurring <- TRUE                    # Specify True for a recurring date range
# fromYear <- 2018
# toYear <- 2020

lat <- c(36.206228, 37.289327)        # Latitude of the point sites 
lon <- c(-112.127134, -112.973760)    # Longitude of the point sites
id <- c("0","1")                      # ID for the point sites
category <- c("Grand Canyon", "Zion") # Category for point sites

taskName <- 'NPS Vegetation'           # Enter name of the task, 'NPS Vegetation' used here
taskType <- 'point'                    # Specify the task type, it can be either "area" or "point"

To be able to successfully submit a task, the JSON object should be structured in a certain way. The code chunk below uses the information from the previous chunk to create a nested data frame. This nested data frame will be converted to JSON object that can be used to complete the request.

# Create a data frame including the date range for the request
date <- data.frame(startDate = startDate, endDate = endDate)
# If you set the recurring to TRUE 
# date <- data.frame(startDate = startDate, endDate = endDate , recurring = recurring)
# date$yearRange <- list(c(fromYear,toYear))

# Create a data frame including lat and long coordinates. ID and category name is optional.
coordinates <- data.frame(id = id, longitude = lon, latitude = lat, category = category)

task_info <- list(date,layers, coordinates)               # Create a list of data frames 
names(task_info) <- c("dates", "layers", "coordinates")   # Assign names

task <- list(task_info, taskName, taskType)               # Create a nested list 
names(task) <- c("params", "task_name", "task_type")      # Assign names 
remove(date, layers, coordinates, task_info)              # Remove the variables that are not needed anymore

toJSON function from jsonlite package converts the type of data frame to a string that can be recognized as a JSON object to be submitted as a point request.

task_json <- toJSON(task,auto_unbox = TRUE)   # Convert to JSON object

3b. Submit a Task Request

Token information is needed to submit a request. Below the login token is assigned to a variable.

token <- paste("Bearer", fromJSON(token_response)$token)     # Save login token to a variable

Below, post a call to the API task service, using the task_json created above.

# Post the point request to the API task service
response <- POST(paste0(API_URL, "task"), 
                 body = task_json , 
                 encode = "json", 
                 add_headers(Authorization = token, "Content-Type" = "application/json"))

task_content <- content(response)                                 # Retrieve content of the request 
task_response <- prettify(toJSON(task_content, auto_unbox = TRUE))# Convert the content to JSON object
remove(response, task_content)                                    # Remove the variables that are not needed anymore
task_response                                                     # Print the prettified task response
## {
##     "task_id": "0dd57143-d242-46f7-b608-e31e783b7070",
##     "status": "pending"
## }
## 

3c. Retrieve Task Status

This API call will list all of the requests associated with your user account, automatically sorted by date descending with the most recent requests listed first. The AppEEARS API contains some helpful formatting resources. Below, limit the API response to 2 entries for the last 2 requests and set pretty to True to format the response as an organized JSON object to make it easier to read. Additional information on AppEEARS API retrieve task, pagination, and formatting can be found in the API documentation.

params <- list(limit = 2, pretty = TRUE)                            # Set up query parameters
# Request the task status of last 2 requests from task URL
response_req <- GET(paste0(API_URL,"task"), query = params, add_headers(Authorization = token))
response_content <- content(response_req)                           # Retrieve content of the request
status_response <- toJSON(response_content, auto_unbox = TRUE)      # Convert the content to JSON object
remove(response_req, response_content)                              # Remove the variables that are not needed anymore                         
prettify(status_response)                                           # Print the prettified response
## [
##     {
##         "params": {
##             "dates": [
##                 {
##                     "endDate": "12-31-2018",
##                     "startDate": "01-01-2018"
##                 }
##             ],
##             "layers": [
##                 {
##                     "layer": "LST_Day_1km",
##                     "product": "MOD11A2.061"
##                 },
##                 {
##                     "layer": "LST_Night_1km",
##                     "product": "MOD11A2.061"
##                 },
##                 {
##                     "layer": "Lai_500m",
##                     "product": "MCD15A3H.006"
##                 }
##             ]
##         },
##         "status": "pending",
##         "created": "2022-04-06T03:47:54.947711",
##         "task_id": "0dd57143-d242-46f7-b608-e31e783b7070",
##         "updated": "2022-04-06T03:47:55.124910",
##         "user_id": "user@email.com",
##         "estimate": {
##             "request_size": 272
##         },
##         "has_swath": false,
##         "task_name": "NPS Vegetation",
##         "task_type": "point",
##         "api_version": "v1",
##         "svc_version": "3.1",
##         "web_version": {
## 
##         },
##         "has_nsidc_daac": false,
##         "expires_on": "2022-05-06T03:47:55.124910"
##     },
##     {
##         "params": {
##             "dates": [
##                 {
##                     "endDate": "12-31-2019",
##                     "startDate": "01-01-2018"
##                 }
##             ],
##             "layers": [
##                 {
##                     "layer": "LST_Day_1km",
##                     "product": "MOD11A2.061"
##                 },
##                 {
##                     "layer": "LST_Night_1km",
##                     "product": "MOD11A2.061"
##                 },
##                 {
##                     "layer": "Lai_500m",
##                     "product": "MCD15A3H.006"
##                 },
##                 {
##                     "layer": "SRTMGL1_DEM",
##                     "product": "SRTMGL1_NC.003"
##                 }
##             ],
##             "output": {
##                 "format": {
##                     "type": "geotiff"
##                 },
##                 "projection": "geographic"
##             }
##         },
##         "status": "pending",
##         "created": "2022-04-06T03:13:04.825063",
##         "task_id": "a359d60a-f877-4355-9f47-ace74656a9f0",
##         "updated": "2022-04-06T03:13:14.813119",
##         "user_id": "user@email.com",
##         "estimate": {
##             "request_size": 368451761.6404
##         },
##         "has_swath": false,
##         "task_name": "Grand Canyon",
##         "task_type": "area",
##         "api_version": "v1",
##         "svc_version": "3.1",
##         "web_version": {
## 
##         },
##         "has_nsidc_daac": false,
##         "expires_on": "2022-05-06T03:13:14.813119"
##     }
## ]
## 

The task_id that was generated when submitting your request can also be used to retrieve a task status.

task_id <- fromJSON(task_response)$task_id                 # Extract the task_id of submitted point request
# Request the task status of a task with the provided task_id from task URL
status_req <- GET(paste0(API_URL,"task/", task_id), add_headers(Authorization = token)) 
status_content <- content(status_req)                       # Retrieve content of the request       
statusResponse <-toJSON(status_content, auto_unbox = TRUE)  # Convert the content to JSON object
stat <- fromJSON(statusResponse)$status                     # Assign the task status to a variable  
remove(status_req, status_content)                          # Remove the variables that are not needed anymore
prettify(statusResponse)                                    # Print the prettified response
## {
##     "params": {
##         "dates": [
##             {
##                 "endDate": "12-31-2018",
##                 "startDate": "01-01-2018"
##             }
##         ],
##         "layers": [
##             {
##                 "layer": "LST_Day_1km",
##                 "product": "MOD11A2.061"
##             },
##             {
##                 "layer": "LST_Night_1km",
##                 "product": "MOD11A2.061"
##             },
##             {
##                 "layer": "Lai_500m",
##                 "product": "MCD15A3H.006"
##             }
##         ],
##         "coordinates": [
##             {
##                 "id": "0",
##                 "category": "Grand Canyon",
##                 "latitude": 36.2062,
##                 "longitude": -112.1271
##             },
##             {
##                 "id": "1",
##                 "category": "Zion",
##                 "latitude": 37.2893,
##                 "longitude": -112.9738
##             }
##         ]
##     },
##     "status": "pending",
##     "created": "2022-04-06T03:47:54.947711",
##     "task_id": "0dd57143-d242-46f7-b608-e31e783b7070",
##     "updated": "2022-04-06T03:47:55.124910",
##     "user_id": "use@email.com",
##     "estimate": {
##         "request_size": 272
##     },
##     "has_swath": false,
##     "task_name": "NPS Vegetation",
##     "task_type": "point",
##     "api_version": "v1",
##     "svc_version": "3.1",
##     "web_version": {
## 
##     },
##     "has_nsidc_daac": false,
##     "expires_on": "2022-05-06T03:47:55.124910"
## }
## 

Retrieve the task status every 5 seconds. The task status should be done to be able to download the output.

while (stat != 'done') {
  Sys.sleep(5)
  # Request the task status and retrieve content of request from task URL
  stat_content <- content(GET(paste0(API_URL,"task/", task_id), add_headers(Authorization = token)))
  stat <-fromJSON(toJSON(stat_content, auto_unbox = TRUE))$status    # Get the status
  remove(stat_content) 
  print(stat)
}
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "pending"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "processing"
## [1] "done"

4. Download a Request

4a. Explore Files in Request Output

Before downloading the request output, examine the files contained in the request output.

# Request the task bundle info from API bundle URL
response <- GET(paste0(API_URL, "bundle/", task_id), add_headers(Authorization = token))
response_content <- content(response)                          # Retrieve content of the request
bundle_response <- toJSON(response_content, auto_unbox = TRUE)  # Convert the content to JSON object
prettify(bundle_response)                                       # Print the prettified response
## {
##     "files": [
##         {
##             "sha256": "4704c2b1563780ce3a6b9d5a0532e850865bf7df0266158fc8abe2ab9f0536d7",
##             "file_id": "15815f7e-110b-45c0-924e-06b1c5e43b8c",
##             "file_name": "NPS-Vegetation-MCD15A3H-006-results.csv",
##             "file_size": 104038,
##             "file_type": "csv"
##         },
##         {
##             "sha256": "2a6825d3ca77b8c00c9a33c5dbd8956f7fa33e3397b03a771288995138f5bf0b",
##             "file_id": "66b96600-96e5-4b49-83c7-feb1412830fb",
##             "file_name": "NPS-Vegetation-MOD11A2-061-results.csv",
##             "file_size": 42948,
##             "file_type": "csv"
##         },
##         {
##             "sha256": "8ba6ca914fb19e42cfb2a2dc4a3ac415cd9d6346fc4e49c1fe9dcc59ce0d819e",
##             "file_id": "da09564d-19ee-478c-a273-e9dee144995a",
##             "file_name": "NPS-Vegetation-granule-list.txt",
##             "file_size": 32008,
##             "file_type": "txt"
##         },
##         {
##             "sha256": "09931fe16f995c743dc46bd0b40860cd237c13302fb90682e49a1f9d98ac2d72",
##             "file_id": "836e0149-aed9-4642-af10-bf3c1c2adf5b",
##             "file_name": "NPS-Vegetation-request.json",
##             "file_size": 935,
##             "file_type": "json"
##         },
##         {
##             "sha256": "77d5f403d970ea5848bef71dadf7bdf5a8807f692ebb7743347d81f4af04e168",
##             "file_id": "d96fe995-2741-47f9-b575-7538d6cc96c0",
##             "file_name": "NPS-Vegetation-MCD15A3H-006-metadata.xml",
##             "file_size": 17237,
##             "file_type": "xml"
##         },
##         {
##             "sha256": "1eb0396add64eda3b5b421428420cd723d02467b40db28873a27f5c0e535a443",
##             "file_id": "039dba16-73ea-42ae-bc45-23f3c8bf825e",
##             "file_name": "NPS-Vegetation-MOD11A2-061-metadata.xml",
##             "file_size": 17191,
##             "file_type": "xml"
##         },
##         {
##             "sha256": "19c972972442be97608914a6f862fce499ba6915bda479901b2e8b7ad8f3d32c",
##             "file_id": "f0ef69fb-0175-43cb-8aa0-7bff8ea4bc39",
##             "file_name": "README.md",
##             "file_size": 18785,
##             "file_type": "txt"
##         }
##     ],
##     "created": "2022-04-06T03:51:08.622651",
##     "task_id": "0dd57143-d242-46f7-b608-e31e783b7070",
##     "updated": "2022-04-06T03:51:57.033457",
##     "bundle_type": "point"
## }
## 

4b. Download Files in a Request (Automation)

The bundle API provides information about completed tasks. For any completed task, a bundle can be queried to return the files contained as a part of the task request. Below, call the bundle API and return all of the output files. Next, read the contents of the bundle in JSON format and loop through file_id to automate downloading all of the output files into the output directory. For more information, please see AppEEARS API Documentation.

bundle <- fromJSON(bundle_response)$files
for (id in bundle$file_id){
  # retrieve the filename from the file_id
  filename <- bundle[bundle$file_id == id,]$file_name     
  # create a destination directory to store the file in
  filepath <- paste(outDir,filename, sep = "/")
  suppressWarnings(dir.create(dirname(filepath)))
  # write the file to disk using the destination directory and file name 
  response <- GET(paste0(API_URL, "bundle/", task_id, "/", id),
                  write_disk(filepath, overwrite = TRUE),
                  progress(),
                  add_headers(Authorization = token))
  }

5. Explore AppEEARS Quality Service

The quality API provides quality details about all of the data products available in AppEEARS. Below are examples of how to query the quality API for listing quality products, layers, and values. The final example (Section 5c.) demonstrates how AppEEARS quality services can be leveraged to decode pertinent quality values for your data. For more information visit AppEEARS API documentation.

First, reset pagination to include offset which allows you to set the number of results to skip before starting to return entries. Next, make a call to list all of the data product layers and the associated quality product and layer information.

params <- list(limit = 6, offset = 20, pretty = TRUE)     # Set up the query parameters
q_req <- GET(paste0(API_URL, "quality"), query = params)  # Request the quality info from quality API_URL
q_content <- content(q_req)                               # Retrieve the content of request
q_response <- toJSON(q_content, auto_unbox = TRUE)        # Convert the info to JSON object
remove(params, q_req, q_content)                          # Remove the variables that are not needed 
prettify(q_response)                                      # Print the prettified quality information
## [
##     {
##         "ProductAndVersion": "CU_LT05.001",
##         "Layer": "SRB1",
##         "QualityProductAndVersion": "CU_LT05.001",
##         "QualityLayers": [
##             "PIXELQA"
##         ],
##         "Continuous": false,
##         "VisibleToWorker": true
##     },
##     {
##         "ProductAndVersion": "CU_LT05.001",
##         "Layer": "SRB2",
##         "QualityProductAndVersion": "CU_LT05.001",
##         "QualityLayers": [
##             "PIXELQA"
##         ],
##         "Continuous": false,
##         "VisibleToWorker": true
##     },
##     {
##         "ProductAndVersion": "CU_LT05.001",
##         "Layer": "SRB3",
##         "QualityProductAndVersion": "CU_LT05.001",
##         "QualityLayers": [
##             "PIXELQA"
##         ],
##         "Continuous": false,
##         "VisibleToWorker": true
##     },
##     {
##         "ProductAndVersion": "CU_LT05.001",
##         "Layer": "SRB4",
##         "QualityProductAndVersion": "CU_LT05.001",
##         "QualityLayers": [
##             "PIXELQA"
##         ],
##         "Continuous": false,
##         "VisibleToWorker": true
##     },
##     {
##         "ProductAndVersion": "CU_LT05.001",
##         "Layer": "SRB5",
##         "QualityProductAndVersion": "CU_LT05.001",
##         "QualityLayers": [
##             "PIXELQA"
##         ],
##         "Continuous": false,
##         "VisibleToWorker": true
##     },
##     {
##         "ProductAndVersion": "CU_LT05.001",
##         "Layer": "SRB7",
##         "QualityProductAndVersion": "CU_LT05.001",
##         "QualityLayers": [
##             "PIXELQA"
##         ],
##         "Continuous": false,
##         "VisibleToWorker": true
##     }
## ]
## 

5a. List Quality Layers

This API call will list all of the quality layer information for a product. For more information visit AppEEARS API documentation

productAndVersion <- 'MCD15A3H.006'                            # Assign ProductAndVersion to a variable 
# Request the quality info from quality API for a specific product
MCD15A3H_req <- GET(paste0(API_URL, "quality/", productAndVersion))
MCD15A3H_content <- content(MCD15A3H_req)                      # Retrieve the content of request
MCD15A3H_quality <- toJSON(MCD15A3H_content, auto_unbox = TRUE)# Convert the info to JSON object
remove(MCD15A3H_req, MCD15A3H_content)                         # Remove the variables that are not needed anymore
prettify(MCD15A3H_quality)                                     # Print the prettified quality information
## [
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "Layer": "Fpar_500m",
##         "QualityProductAndVersion": "MCD15A3H.006",
##         "QualityLayers": [
##             "FparLai_QC",
##             "FparExtra_QC"
##         ],
##         "VisibleToWorker": true
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "Layer": "FparStdDev_500m",
##         "QualityProductAndVersion": "MCD15A3H.006",
##         "QualityLayers": [
##             "FparLai_QC",
##             "FparExtra_QC"
##         ],
##         "VisibleToWorker": true
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "Layer": "Lai_500m",
##         "QualityProductAndVersion": "MCD15A3H.006",
##         "QualityLayers": [
##             "FparLai_QC",
##             "FparExtra_QC"
##         ],
##         "VisibleToWorker": true
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "Layer": "LaiStdDev_500m",
##         "QualityProductAndVersion": "MCD15A3H.006",
##         "QualityLayers": [
##             "FparLai_QC",
##             "FparExtra_QC"
##         ],
##         "VisibleToWorker": true
##     }
## ]
## 

5b. Show Quality Values

This API call will list all of the values for a given quality layer.

quality_layer <- 'FparLai_QC'                                 # assign a quality layer to a variable
# Request the specified quality layer info from quality API
quality_req <- GET(paste0(API_URL, "quality/",  productAndVersion, "/", quality_layer, sep = ""))
quality_content <- content(quality_req)                        # Retrieve the content of request
quality_response <- toJSON(quality_content, auto_unbox = TRUE) # Convert the info to JSON object
remove(quality_req, quality_content)                           # Remove the variables that are not needed 
prettify(quality_response)                                     # Print the quality response as a data frame
## [
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "MODLAND",
##         "Value": 0,
##         "Description": "Good quality (main algorithm with or without saturation)",
##         "Acceptable": true
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "MODLAND",
##         "Value": 1,
##         "Description": "Other Quality (back-up algorithm or fill values)",
##         "Acceptable": false
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "Sensor",
##         "Value": 0,
##         "Description": "Terra",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "Sensor",
##         "Value": 1,
##         "Description": "Aqua",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "DeadDetector",
##         "Value": 0,
##         "Description": "Detectors apparently fine for up to 50% of channels 1, 2",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "DeadDetector",
##         "Value": 1,
##         "Description": "Dead detectors caused >50% adjacent detector retrieval",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "CloudState",
##         "Value": 0,
##         "Description": "Significant clouds NOT present (clear)",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "CloudState",
##         "Value": 1,
##         "Description": "Significant clouds WERE present",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "CloudState",
##         "Value": 2,
##         "Description": "Mixed cloud present in pixel",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "CloudState",
##         "Value": 3,
##         "Description": "Cloud state not defined, assumed clear",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "SCF_QC",
##         "Value": 0,
##         "Description": "Main (RT) method used, best result possible (no saturation)",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "SCF_QC",
##         "Value": 1,
##         "Description": "Main (RT) method used with saturation. Good, very usable",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "SCF_QC",
##         "Value": 2,
##         "Description": "Main (RT) method failed due to bad geometry, empirical algorithm used",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "SCF_QC",
##         "Value": 3,
##         "Description": "Main (RT) method failed due to problems other than geometry, empirical algorithm used",
##         "Acceptable": {
## 
##         }
##     },
##     {
##         "ProductAndVersion": "MCD15A3H.006",
##         "QualityLayer": "FparLai_QC",
##         "Name": "SCF_QC",
##         "Value": 4,
##         "Description": "Pixel not produced at all, value couldn't be retrieved (possible reasons: bad L1B data, unusable MOD09GA data)",
##         "Acceptable": {
## 
##         }
##     }
## ]
## 

5c. Decode Quality Values

This API call will decode the bits for a given quality value.

quality_value <- 1                        # Assign a quality value to a variable 
# Request and retrieve information for provided quality value from quality API URL 
response <- content(GET(paste0(API_URL, "quality/", productAndVersion, "/", quality_layer, "/", quality_value)))
q_response <- toJSON(response, auto_unbox = TRUE)     # Convert the info to JSON object
remove(response)                                      # Remove the variables that are not needed anymore
prettify(q_response)                                  # Print the prettified response
## {
##     "Binary Representation": "0b00000001",
##     "MODLAND": {
##         "bits": "0b1",
##         "description": "Other Quality (back-up algorithm or fill values)"
##     },
##     "Sensor": {
##         "bits": "0b0",
##         "description": "Terra"
##     },
##     "DeadDetector": {
##         "bits": "0b0",
##         "description": "Detectors apparently fine for up to 50% of channels 1, 2"
##     },
##     "CloudState": {
##         "bits": "0b00",
##         "description": "Significant clouds NOT present (clear)"
##     },
##     "SCF_QC": {
##         "bits": "0b000",
##         "description": "Main (RT) method used, best result possible (no saturation)"
##     }
## }
## 

6. BONUS: Load Request Output and Visualize

Here, load the CSV file containing the results from your request using readr package, and create some basic visualizations using the ggplot2 package.

6a. Load a CSV

Use the readr package to load the CSV file containing the results from the AppEEARS request.

# Make a list of csv files in the output directory
files <- list.files(outDir, pattern = "\\MOD11A2-061-results.csv$") 
# Read the MOD11A2 results
df <- read_csv(paste0(outDir,"/", files), show_col_types = FALSE)

Select the MOD11A2.061 LST Day column for the data from Grand Canyon National Park using dplyr package.

lst_GC <- df %>%
  # Filter df for the point from GC
  filter(Category == "Grand Canyon") %>%  
  # Select desired columns
  select(Latitude, Longitude, Date ,MOD11A2_061_LST_Day_1km, MOD11A2_061_LST_Night_1km) 

Extract information for LST_DAY_1KM from MOD11_response of product service call from earlier in the tutorial.

#fromJSON(MOD11_response)$LST_Day_1km                        # Extract all the info for LST_Day_1km layer

fillValue <- fromJSON(MOD11_response)$LST_Day_1km$FillValue  # Assign fill value to a variable
unit <- fromJSON(MOD11_response)$LST_Day_1km$Units           # Assign unit to a variable
sprintf("Fill value for LST_DAY_1KM is: %i" ,fillValue)      # Print LST_DAY_1KM fill value
## [1] "Fill value for LST_DAY_1KM is: 0"
sprintf("Unit for LST_DAY_1KM is: %s" ,unit)                 # Print LST_DAY_1KM unit 
## [1] "Unit for LST_DAY_1KM is: Kelvin"

6b. Plot Results (Line/Scatter Plots)

Next, plot a time series of daytime LST for the selected point in Grand Canyon National Park for 2018. Below, filter the LST data to exclude fill values.

lst_GC <- lst_GC %>%
  # exclude NoData
  filter(MOD11A2_061_LST_Day_1km != fillValue)%>%
  filter(MOD11A2_061_LST_Night_1km != fillValue)

Next, plot LST Day as a time series with some additional formatting using ggplot2.

ggplot(lst_GC)+
  geom_line(aes(x= Date, y = MOD11A2_061_LST_Day_1km), size=1, color="blue")+
  geom_point(aes(x= Date, y = MOD11A2_061_LST_Day_1km), shape=18 , size = 3, color="blue")+
  labs(title = "Time Series",
       x = "Date",
       y = sprintf( "LST_Day_1km (%s)", unit))+
  scale_x_date(date_breaks = "16 day", limits = as.Date(c('2018-01-01','2019-01-01')))+ 
  scale_y_continuous(limits = c(250, 325), breaks = seq(250, 325, 10))+
  theme(plot.title = element_text(face = "bold",size = rel(2.5),hjust = 0.5),
        axis.title = element_text(face = "bold",size = rel(1)),
        panel.background = element_rect(fill = "lightgray", colour = "black"),
        axis.text.x = element_text(face ="bold",color="black", angle= 315 , size = 10),
        axis.text.y = element_text(face ="bold",color="black", angle= 0, size = 10)
        )
## Warning: Removed 1 row(s) containing missing values (geom_path).
## Warning: Removed 1 rows containing missing values (geom_point).

Using the tidyr package, the LST Day and Night values for Grand Canyon NP are being gathered in a single column to be used to make a plot including both LST_Day_1km and LST_Night_1km.

lst_GC_DN <- tidyr::gather(lst_GC, key = Tstat , value = LST, MOD11A2_061_LST_Day_1km, MOD11A2_061_LST_Night_1km)
lst_GC_DN[1:5,]                     # print the five first observations 

Next, plot LST Day and Night as a time series with some additional formatting.

ggplot(lst_GC_DN)+
  geom_line(aes(x= Date, y = LST, color = Tstat), size=1)+
  geom_point(aes(x= Date, y = LST, color = Tstat), shape=18 , size = 3)+
  scale_fill_manual(values=c("red", "blue"))+
  scale_color_manual(values=c('red','blue'))+
  labs(title = "Time Series",
       x = "Date",
       y = sprintf( "LST_Day_1km (%s)",unit))+
  scale_x_date(date_breaks = "16 day", limits = as.Date(c('2018-01-01','2019-01-01')))+  
  scale_y_continuous(limits = c(250, 325), breaks = seq(250, 325, 10))+
  theme(plot.title = element_text(face = "bold",size = rel(2.5), hjust = 0.5),
        axis.title = element_text(face = "bold",size = rel(1)),
        panel.background = element_rect(fill = "lightgray", colour = "black"),
        axis.text.x = element_text(face ="bold",color="black", angle= 315 , size = 10),
        axis.text.y = element_text(face ="bold",color="black", angle= 0, size = 10),
        legend.position = "bottom",
        legend.title = element_blank()
        )
## Warning: Removed 2 row(s) containing missing values (geom_path).
## Warning: Removed 2 rows containing missing values (geom_point).

Finally, bring in the daytime LST data from Zion National Park, and compare with daytime LST at Grand Canyon National Park, shown below in a scatterplot using ggplot2 package. Here, the dplyr is used to extract the LST_DAY_1km for Zion National Park.

lst_Z <- df %>%
  filter(MOD11A2_061_LST_Day_1km != fillValue) %>%        # Filter fill value 
  filter(Category == "Zion")%>%                           # Filter Zion national park
  select(Date, MOD11A2_061_LST_Day_1km)                   # Select desired columns

Make a scatterplot.

ggplot()+
  geom_point(aes(x=lst_Z$MOD11A2_061_LST_Day_1km, y=lst_GC$MOD11A2_061_LST_Day_1km), shape=18 , size = 3, color="blue")+
  labs(title = "MODIS LST: Grand Canyon vs. Zion National Park, 2018",
       x = sprintf("Zion: LST_Day_1km (%s)",unit),
       y = sprintf( "Grand Canyon: LST_Day_1km (%s)",unit))+
  theme(plot.title = element_text(face = "bold",size = rel(1.5), hjust = 0.5),
        axis.title = element_text(face = "bold",size = rel(1)),
        panel.background = element_rect(fill = "lightgray", colour = "black"),
        axis.text.x = element_text(face ="bold",color="black", size = 10),
        axis.text.y = element_text(face ="bold",color="black", size = 10)
        )

This example can provide a template to use for your own research workflows. Leveraging the AppEEARS API for searching, extracting, and formatting analysis ready data, and loading it directly into R means that you can keep your entire research workflow in a single software program, from start to finish.


Contact Information

Material written by Mahsa Jami1 and Cole Krehbiel1
Contact:
Voice: +1-866-573-3222
Organization: Land Processes Distributed Active Archive Center (LP DAAC)
Website: https://lpdaac.usgs.gov/
Date last modified: 06-12-2020

1 KBR, Inc., contractor to the U.S. Geological Survey, Earth Resources Observation and Science (EROS) Center,
Sioux Falls, South Dakota, USA. Work performed under USGS contract G15PD00467 for LP DAAC2.

2 LP DAAC Work performed under NASA contract NNG14HH33I.