Some time ago I saw a blog post on [visualising your twitter history using streamgraphs]1 in R. As my tweets moved from mostly psychology/philosophy tweeting around my teaching, through to my current mish-mash of learning analytics stuff, I thought it’d be interesting to play with this. So, here’s the code reproduced, plus my streamgraph. If anyone has any other ideas for analysing my twitter data, now I’ve got the archive it’d be easy enough to do new stuff

    # Adapted from
    # Script for producing a streamgraph of tweet hashtags

    #note: You have to install 'devtools' package first, to install streamgraph from github
    #to use devtools you also need rtools from
    #I also had to install withr package 

    # Load packages

    # Read my tweets
    tweets_df <- read_csv("tweets.csv")
    tweets_df\$text <- sapply(tweets_df\$text,function(x) iconv(x, "", "UTF-8"))
    tweets_df\$text <- tolower(tweets_df\$text)

    # Pick hashtags with regexp
    hashtags_list <- regmatches(tweets_df\$text, gregexpr("#[[:alnum:]]+", tweets_df\$text))

    #if you want to play with expanding the hashtags to other keywords one way might be to look at a tfidf matrix, removing very infrequent and very frequent terms

    # Create a new data_frame with (timestamp, hashtag) -pairs
    hashtags_df <- data_frame()
    for (i in which(sapply(hashtags_list, length) > 0)) {
      hashtags_df <- bind_rows(hashtags_df, data_frame(timestamp = tweets_df\$timestamp[i],
                                                       hashtag = hashtags_list[[i]]))

    # Process data for plotting
    hashtags_df <- hashtags_df %>%
      # Pick top 20 hashtags
      filter(hashtag %in% names(sort(table(hashtag), decreasing=TRUE))[1:20]) %>%
      # Group by year-month (daily is too messy)
      # Need to add '-01' to make it a valid date for streamgraph
      mutate(yearmonth = paste0(format(as.Date(timestamp), format="%Y-%m"), "-01")) %>%
      group_by(yearmonth, hashtag) %>%
      summarise(value = n())

    # Create streamgraph
    sg <- streamgraph(data = hashtags_df, key = "hashtag", value = "value", date = "yearmonth",
                      offset = "silhouette", interpolate = "cardinal",
                      width = "700", height = "400") %>%
      sg_legend(TRUE, "hashtag: ") %>%
      sg_axis_x(tick_interval = 1, tick_units = "year", tick_format = "%Y")

    # Save it for viewing in the blog post
    # Using original code to save here (with file.rename()...but not sure if I have the same problem they encountered
    saveWidget(sg, file="twitter_streamgraph.html", selfcontained = TRUE)
    file.rename("twitter_streamgraph.html", "twitter_streamgraph.html")

The live version can be found [here]2 (I recommend zooming in), with a screenshot highlighting ‘learning analytics’ below. As I note in the code above, another way of visualising this would be to select different terms, but for now this is quite nice :-). One thing it indicates is that I probably am not using hashtags very effectively(!) except for events like conferences – most of the peaks here are around particular, often very short term events. My tweeting certainly goes up then, but it shouldn’t be quite so dramatic as it is.