ghcn and meteo_tidy_ghcnd not pulling most recent data

Package Use Questions
1

Hi all, I’m using the excellent meteo_tidy_ghcnd function to pull daily data from the local airport. I notice that today (May 4, 2017) that all the data after April 12, 2017 are NA despite my observation that the raw data list values for TMIN, TMAX, and PRCP through the end of the month. I can’t see anything in the raw data (e.g., flags) that would preclude data being pulled but that might well be a misunderstanding on my part.

So, how do ghcn and meteo_tidy_ghcnd truncate data? Advice appreciated. -Andy

airport <- data.frame(meteo_tidy_ghcnd(stationid="USW00024217", 
                        keep_flags = FALSE, 
                        var = c("TMIN","TMAX","PRCP")))
tail(airport,20) # why only to  12-apr?
# or
dat <- ghcnd(stationid = "USW00024217")
alldat <- ghcnd_splitvars(dat)
prcp <- data.frame(alldat$prcp)
prcp <- prcp[order(prcp$date),]
tail(prcp,20)

raw data link: ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/daily/all/USW00024217.dly (wasn’t showing up in embedded link above)

2

Thanks for raising the issue @AndyBunn

Opened an issue https://github.com/ropensci/rnoaa/issues/221 and fixed that problem, reinstall devtools::install_github("ropensci/rnoaa") and try again, let me know if it works.

3

Thanks @sckott but…nope. Broke it even worse I fear. Looks like a character is getting passed out when an integer is expected?

R> airport ← data.frame(meteo_tidy_ghcnd(stationid=“USW00024217”,

  •                     keep_flags = FALSE, 

  •                     var = c("TMIN","TMAX","PRCP")))

Error in scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
scan() expected ‘an integer’, got ‘“1949”’
R> dat ← ghcnd(stationid = “USW00024217”)
Error in scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
scan() expected ‘an integer’, got ‘“1949”’
R>

4

Works good for me. Did you update to the latest version and restart your session before trying again?

Can you share you session_info() or sessionInfo()

5

Fresh session post the devtools install. No other packages loaded.

R> rm(list=ls())
R> require(rnoaa)
R> dat ← ghcnd(stationid = “USW00024217”)
Error in scan(file = file, what = what, sep = sep, quote = quote, dec = dec, :
scan() expected ‘an integer’, got ‘“1949”’
R> sessionInfo()
R version 3.3.3 (2017-03-06)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.4

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] rnoaa_0.6.7.9219

loaded via a namespace (and not attached):
[1] Rcpp_0.12.10 lubridate_1.6.0 tidyr_0.6.1 XML_3.98-1.6
[5] hoardr_0.1.0 dplyr_0.5.0 assertthat_0.2.0 rappdirs_0.3.1
[9] grid_3.3.3 R6_2.2.0 plyr_1.8.4 jsonlite_1.4
[13] gtable_0.2.0 DBI_0.6-1 magrittr_1.5 scales_0.4.1
[17] httr_1.2.1 ggplot2_2.2.1 stringi_1.1.5 lazyeval_0.2.0
[21] xml2_1.1.1 fortunes_1.5-4 tools_3.3.3 stringr_1.2.0
[25] munsell_0.4.3 colorspace_1.3-2 gridExtra_2.2.1 tibble_1.3.0
R>

6

after that error, can you run traceback() and paste in what that shows, trying to locate what exact line of code that comes from

7

Here it is. Sorry for the hassle. I/O is the worst thing ever. It doesn’t like the ghcnd_col_classes on line 189.

R> traceback()
4: scan(file = file, what = what, sep = sep, quote = quote, dec = dec,
nmax = nrows, skip = 0, na.strings = na.strings, quiet = TRUE,
fill = fill, strip.white = strip.white, blank.lines.skip = blank.lines.skip,
multi.line = FALSE, comment.char = comment.char, allowEscapes = allowEscapes,
flush = flush, encoding = encoding, skipNul = skipNul)
3: read.table(file = file, header = header, sep = sep, quote = quote,
dec = dec, fill = fill, comment.char = comment.char, …)
2: read.csv(csvpath, stringsAsFactors = FALSE, colClasses = ghcnd_col_classes)
1: ghcnd(stationid = “USW00024217”)
R>

8

can you run ghcnd_clear_cache() (may need to do ghcnd_clear_cache(force=TRUE) )

then try the ex again

i think the cached files may be used and so you’re reading in cached files that were written with old version of rnoaa

9

Success! Thanks for figuring this out!

1 Like
10

great, glad it worked. will make sure that’s highlighted in the NEWS for the new version going out today