Commits
6 6 | ```{r setup, include=FALSE} |
7 7 | knitr::opts_chunk$set(echo = TRUE) |
8 8 | ``` |
9 9 | *** |
10 10 | |
11 11 | ### This tutorial demonstrates how to use R to connect to the AppEEARS API |
12 12 | The Application for Extracting and Exploring Analysis Ready Samples ([AppEEARS](https://lpdaacsvc.cr.usgs.gov/appeears/)) offers a simple and efficient way to access and transform [geospatial data](https://lpdaacsvc.cr.usgs.gov/appeears/products) from a variety of federal data archives in an easy-to-use web application interface. AppEEARS enables users to subset geospatial data spatially, temporally, and by band/layer for point and area samples. AppEEARS returns not only the requested data, but also the associated quality values, and offers interactive visualizations with summary statistics in the web interface. The [AppEEARS API](https://lpdaacsvc.cr.usgs.gov/appeears/api/) offers users **programmatic access** to all features available in AppEEARS, with the exception of visualizations. The API features are demonstrated in this tutorial. |
13 13 | |
14 14 | *** |
15 15 | ### Example: Submit a point request with multiple points in U.S. National Parks for extracting vegetation and land surface temperature data |
16 - | In this tutorial Connecting to the AppEEARS API, querying the list of available products, submitting a point sample request, downloading the request, working with the AppEEARS Quality API, and loading the results into R for visualization are covered. AppEEARS point requests allow users to subset their desired data using latitude/longitude geographic coordinate pairs (points) for a time period of interest, and for specific data layers within data products. AppEEARS returns the valid data from the parameters defined within the sample request. |
16 + | In this tutorial, Connecting to the AppEEARS API, querying the list of available products, submitting a point sample request, downloading the request, working with the AppEEARS Quality API, and loading the results into R for visualization are covered. AppEEARS point requests allow users to subset their desired data using latitude/longitude geographic coordinate pairs (points) for a time period of interest, and for specific data layers within data products. AppEEARS returns the valid data from the parameters defined within the sample request. |
17 17 | |
18 18 | #### Data Used in the Example: |
19 19 | - Data layers: |
20 20 | - Combined MODIS Leaf Area Index (LAI) |
21 21 | - [MCD15A3H.006](https://doi.org/10.5067/MODIS/MCD15A3H.006), 500m, 4 day: 'Lai_500m' |
22 22 | - Terra MODIS Land Surface Temperature |
23 23 | - [MOD11A2.006](https://doi.org/10.5067/MODIS/MOD11A2.006), 1000m, 8 day: 'LST_Day_1km', 'LST_Night_1km' |
24 24 | |
25 25 | *** |
26 26 | ## Topics Covered in this tutorial: |
148 148 | *** |
149 149 | ## 2. **[Query Available Products]{#section2}** |
150 150 | The product API provides details about all of the products and layers available in AppEEARS. For more information, please see the [API Documentation](https://lpdaacsvc.cr.usgs.gov/appeears/api/?language=R#product). |
151 151 | |
152 152 | Below, call the product API to list all of the products available in AppEEARS. |
153 153 | ```{r} |
154 154 | prods_req <- GET(paste0(API_URL, "product")) # Request the info of all products from product service |
155 155 | prods_content <- content(prods_req) # Retrieve the content of request |
156 156 | all_Prods <- toJSON(prods_content, auto_unbox = TRUE) # Convert the info to JSON object |
157 157 | remove(prods_req, prods_content) # Remove the variables that are not needed anymore |
158 - | #prettify(all_Prods) # Print the prettified product response |
158 + | # prettify(all_Prods) # Print the prettified product response |
159 159 | ``` |
160 160 | *** |
161 161 | ## 2a. Search and Explore Available Products |
162 162 | Create a list indexed by product name to make it easier to query a specific product. |
163 163 | ```{r} |
164 - | # Devides the individual products information |
165 - | devided_products <- split(fromJSON(all_Prods), seq(nrow(fromJSON(all_Prods)))) |
164 + | # Divides information from each product. |
165 + | divided_products <- split(fromJSON(all_Prods), seq(nrow(fromJSON(all_Prods)))) |
166 166 | # Create a list indexed by the product name and version |
167 - | products <- setNames(devided_products,fromJSON(all_Prods)$ProductAndVersion) |
167 + | products <- setNames(divided_products,fromJSON(all_Prods)$ProductAndVersion) |
168 168 | # Print no. products available in AppEEARS |
169 169 | sprintf("AppEEARS currently supports %i products." ,length(products)) |
170 170 | ``` |
171 171 | Next, look at the product's names and descriptions. Below, the 'ProductAndVersion' and 'Description' are printed for all products. |
172 172 | ```{r} |
173 - | # Loop through the products in the list and Print the product name and description |
173 + | # Loop through the products in the list and print the product name and description |
174 174 | for (p in products){ |
175 175 | print(paste0(p$ProductAndVersion," is ",p$Description," from ",p$Source)) |
176 176 | } |
177 177 | ``` |
178 178 | The product service provides many useful details, including if a product is currently available in AppEEARS, a description, and information on the spatial and temporal resolution. Below, the product details are retrieved using 'ProductAndVersion'. |
179 179 | ```{r} |
180 180 | # Convert the MCD15A3H.006 info to JSON object and print the prettified info |
181 181 | prettify(toJSON(products$"MCD15A3H.006")) |
182 182 | ``` |
183 183 | |
219 219 | names(fromJSON(MOD11_response)) # print the layer names |
220 220 | ``` |
221 221 | Lastly, select the desired layers and pertinent products and make a data frame using this information. This data frame will be inserted into the nested data frame that will be used to create a JSON object to submit a request in [Section 3](#section3). |
222 222 | ```{r} |
223 223 | desired_layers <- c("LST_Day_1km","LST_Night_1km","Lai_500m") # Create a vector of desired layers |
224 224 | desired_prods <- c("MOD11A2.006","MOD11A2.006","MCD15A3H.006") # Create a vector of products including the desired layers |
225 225 | # Create a data frame including the desired data products and layers |
226 226 | layers <- data.frame(product = desired_prods, layer = desired_layers) |
227 227 | ``` |
228 228 | *** |
229 - | ## 3. **[Submit an Point Request]{#section3}** |
229 + | ## 3. **[Submit a Point Request]{#section3}** |
230 230 | The Submit task API call provides a way to submit a new request to be processed. It can accept data via JSON or query string. In the example below, create a JSON object and submit a request. Tasks in AppEEARS correspond to each request associated with your user account. Therefore, each of the calls to this service requires an authentication token (see [Section 1c.](#login)). |
231 231 | |
232 232 | *** |
233 233 | ## 3a. Compile a JSON Object |
234 - | In this section, begin by setting up the information needed for a nested data frame that will be later convertd to a JSON object for submitting an AppEEARS point request. |
234 + | In this section, begin by setting up the information needed for a nested data frame that will be later converted to a JSON object for submitting an AppEEARS point request. |
235 235 | For detailed information on required JSON parameters, see the [API Documentation](https://lpdaacsvc.cr.usgs.gov/appeears/api/?language=R#tasks). |
236 236 | |
237 237 | For point requests, beside the date range and desired layers information, the coordinates property must also be inside the task object. Optionally, set `id` and `category` properties to further identify your selected coordinates. |
238 238 | ```{r} |
239 239 | startDate <- "01-01-2018" # Start of the date range for which to extract data: MM-DD-YYYY |
240 240 | endDate <- "12-31-2018" # End of the date range for which to extract data: MM-DD-YYYY |
241 241 | recurring <- FALSE # Specify True for a recurring date range |
242 242 | #yearRange <- [2000,2016] # If recurring = True, set yearRange, change start/end date to MM-DD |
243 243 | |
244 244 | lat <- c(36.206228, 37.289327) # Latitude of the point sites |
308 308 | task_id <- fromJSON(task_response)$task_id # Extract the task_id of submitted point request |
309 309 | # Request the task status of a task with the provided task_id from task URL |
310 310 | status_req <- GET(paste0(API_URL,"task/", task_id), add_headers(Authorization = token)) |
311 311 | status_content <- content(status_req) # Retrieve content of the request |
312 312 | statusResponse <-toJSON(status_content, auto_unbox = TRUE) # Convert the content to JSON object |
313 313 | stat <- fromJSON(statusResponse)$status # Assign the task status to a variable |
314 314 | remove(status_req, status_content) # Remove the variables that are not needed anymore |
315 315 | prettify(statusResponse) # Print the prettified response |
316 316 | ``` |
317 317 | |
318 - | Retrieve the task satus every 5 seconds. The task status should be `done` to be able to download the output. |
318 + | Retrieve the task status every 5 seconds. The task status should be `done` to be able to download the output. |
319 319 | ```{r} |
320 320 | while (stat != 'done') { |
321 321 | Sys.sleep(5) |
322 - | # Request the task status and Retrieve content of request from task URL |
322 + | # Request the task status and retrieve content of request from task URL |
323 323 | stat_content <- content(GET(paste0(API_URL,"task/", task_id), add_headers(Authorization = token))) |
324 324 | stat <-fromJSON(toJSON(stat_content, auto_unbox = TRUE))$status # Get the status |
325 325 | remove(stat_content) |
326 326 | print(stat) |
327 327 | } |
328 328 | ``` |
329 329 | |
330 330 | *** |
331 331 | ## 4. **[Download a Request]{#section4}** |
332 332 | ## 4a. Explore Files in Request Output |
415 415 | Use the `readr` package to load the CSV file containing the results from the AppEEARS request. |
416 416 | ```{r, message= FALSE, warning= FALSE} |
417 417 | # Make a list of csv files in the output directory |
418 418 | files <- list.files(paste0(inDir,"/R_output"), pattern = "\\MOD11A2-006-results.csv$") |
419 419 | # Read the MOD11A2 results |
420 420 | df <- read_csv(paste0(inDir,"/R_output/", files)) |
421 421 | ``` |
422 422 | Select the MOD11A2.006 LST Day column for the data from Grand Canyon National Park using `dplyr` package. |
423 423 | ```{r} |
424 424 | lst_GC <- df %>% |
425 - | # Filter df for the pixel from GC |
425 + | # Filter df for the point from GC |
426 426 | filter(Category == "Grand Canyon") %>% |
427 427 | # Select desired columns |
428 428 | select(Latitude, Longitude, Date ,MOD11A2_006_LST_Day_1km, MOD11A2_006_LST_Night_1km) |
429 429 | ``` |
430 430 | Extract information for LST_DAY_1KM from MOD11_response of product service call from earlier in the tutorial. |
431 431 | ```{r} |
432 432 | #fromJSON(MOD11_response)$LST_Day_1km # Extract all the info for LST_Day_1km layer |
433 433 | |
434 434 | fillValue <- fromJSON(MOD11_response)$LST_Day_1km$FillValue # Assign fill value to a variable |
435 435 | unit <- fromJSON(MOD11_response)$LST_Day_1km$Units # Assign unit to a variable |
514 514 | panel.background = element_rect(fill = "lightgray", colour = "black"), |
515 515 | axis.text.x = element_text(face ="bold",color="black", size = 10), |
516 516 | axis.text.y = element_text(face ="bold",color="black", size = 10) |
517 517 | ) |
518 518 | ``` |
519 519 | |
520 520 | This example can provide a template to use for your own research workflows. Leveraging the AppEEARS API for searching, extracting, and formatting analysis ready data, and loading it directly into R means that you can keep your entire research workflow in a single software program, from start to finish. |
521 521 | |
522 522 | *** |
523 523 | ## Contact Information |
524 - | Material written by Mahsa Jami^1^ ,Cole Krehbiel^1^ |
524 + | Material written by Mahsa Jami^1^ and Cole Krehbiel^1^ |
525 525 | Contact: LPDAAC@usgs.gov |
526 526 | Voice: +1-866-573-3222 |
527 527 | Organization: Land Processes Distributed Active Archive Center (LP DAAC) |
528 528 | Website: https://lpdaac.usgs.gov/ |
529 529 | Date last modified: 06-12-2020 |
530 530 | |
531 531 | ^1^ KBR, Inc., contractor to the U.S. Geological Survey, Earth Resources Observation and Science (EROS) Center, |
532 532 | Sioux Falls, South Dakota, USA. Work performed under USGS contract G15PD00467 for LP DAAC^2^. |
533 533 | |
534 534 | ^2^ LP DAAC Work performed under NASA contract NNG14HH33I. |