Extracting tweets from Twitter in Rstudio

General Q&A
, ,
1

Hi!

I was be able to extract tweets from twitter into R, using search_30days. However, I wanted to exclude the retweets, but got the error code 422 which means that something is wrong with the way I called it.

This is what I use:

´´´
dataBTC1 ← search_30day(“Bitcoin analysis lang:en -is:retweet”, n = 100, env_name = “Tweets30”, fromDate = “20220915000”, toDate = “202209152359”, parse = FALSE)
´´´
Moreover, I have converted the tweets from a list into a data frame by doing the following:

´´´
dataBTC1 ← search_30day(“Bitcoin analysis future lang:en”, n = 100, env_name = “Tweets30”, fromDate = “20220915000”, toDate = “202209152359”, parse = FALSE)

dfbtc1 ← do.call(rbind.data.frame, dataBTC1)
´´´

However, when I change my search query to:

´´´
dataBTC1 ← search_30day(“Bitcoin analysis on the market lang:en”, n = 100, env_name = “Tweets30”, fromDate = “20220915000”, toDate = “202209152359”, parse = FALSE)

dfbtc1 ← do.call(rbind.data.frame, dataBTC1)
´´´
I get the following error:

´´´
Warning: non-unique values when setting ‘row.names’: ‘1’, ‘2’, ‘3’, ‘4’, ‘5’, ‘6’, ‘7’Error in .rowNamesDF<-(x, value = value) :
duplicate ‘row.names’ are not allowed
´´´

How can this be fixed? I also noticed that the first code results in a large list approx 2 mb and the second code results in a large list (16 elements, 2.9 mb). I assume it has something to do with this, but I dont know how to fix it.

2

Be aware that with parse = FALSE the returned data is not parsed by rtweet and it is in json format. I’m not sure why you expect it to work with the rbind.data.frame or why it worked in the first case.
You might be using parse = FALSE to avoid the issue with the new edit* fields that the API is returning and that the current released rtweet version doesn’t know how to handle. I’ve set a workaround in the devel version of rtweet in github, that I plan to release soon.

1 Like
3

Any news on this yet? Or should I install devel version?