targets pipeline with RMarkdown to download and visualize USGS data

rOpenSci package or resource used*

targets

What did you do?

I was looking for a simple way to share code for a unique figure of cartogram bar charts related to the frequency of “ice” flags in public U.S. Geological Survey (USGS) streamgage data.

I could have written a regular script but I needed the code to download a lot of data for all 50 U.S. states. These downloads can sometimes timeout and trigger you to have to start again. Due to this concern, I decided to use targets to build a pipeline that could handle skipping downloads that were already successful if I needed to start again.

A regular targets setup using the _targets.R file to write and orchestrate your code would certainly work; however, I noticed that a section related to using targets and R Markdown was recently added to the User Manual and I wanted to give that a try. This was my first time using targets with R Markdown, but it wasn’t hard to figure out. I love that in the end, it is just one button click (or maybe a few if there is a network failure during the downloads) to build the full pipeline and show the results. I am excited for using this more in the future.

URL or code snippet for your use case*

You can find the R Markdown file at blitz-FY21Q3/ice_pipeline.Rmd at 835a05bf725d53ace5bf9884435b53402b97ec6e · USGS-VIZLAB/blitz-FY21Q3 · GitHub. The Rmd file produces the basic cartogram behind the visualization featured in this Tweet. We added the text later using InkScape to save on time since this was a very small, constrained project.

Note that this Rmd will download 9 months of daily data for all 50 U.S. states by default, which takes awhile. Read the text in the Rmd to see how to adjust for a shorter time period or fewer states if you are interested in testing this but can’t wait a few hours for the data!

Image

Sector

government

Field(s) of application

hydrology

Comments

We did learn that you need to use targets v0.5.0 or greater to build the pipeline via Markdown (it broke when a colleague was reviewing the code using targets v0.4.2). So, just a reminder that you should keep your packages up-to-date because you never know what cool new features (or bug fixes) a developer may have added.

Twitter handle

@USGS_DataSci

3 Likes