{"id":373,"date":"2015-03-31T08:29:39","date_gmt":"2015-03-31T08:29:39","guid":{"rendered":"http:\/\/codata.org\/blog\/?p=373"},"modified":"2020-02-27T08:45:20","modified_gmt":"2020-02-27T08:45:20","slug":"isi-codata-big-data-workshop-as-word-clouds","status":"publish","type":"post","link":"https:\/\/codata.org\/blog\/2015\/03\/31\/isi-codata-big-data-workshop-as-word-clouds\/","title":{"rendered":"ISI-CODATA Big Data Workshop as Word Clouds"},"content":{"rendered":"

\"Shiva_Khanal\"<\/a>This post was written by Shiva Khanal<\/a>,\u00a0Research Officer with the\u00a0Department of Forest Research and Survey<\/a> in Nepal. \u00a0Shiva was one of the international scholars sponsored by CODATA to attend the ISI-CODATA International Training Workshop on Big Data.<\/em><\/p>\n

This March two associated events were co-organized by the\u00a0Committee on Data for Science and Technology (CODATA)<\/a> and the Indian Statistical Institute (ISI): the International Seminar on Data Science (19-20, Mar 2015)\u00a0 and ISI CODATA International Training Workshop on Big Data (9-18, Mar 2015). Those events in ISI Bangalore, India, covered a wide range of talks and presentations related to big data with presenters from diverse background such as academic community, business sectors and data scientists.<\/p>\n

One way to visualize the focus of the program would be to make plot of terms that were more frequent. I obtained the schedule of presentations and tutorials during seminar (http:\/\/drtc1.isibang.ac.in\/datascience\/schedule.html)<\/span><\/a> and training workshop (http:\/\/drtc1.isibang.ac.in\/bdworkshop\/schedule.html)<\/span><\/a> and generated a word cloud using the R package – wordcloud.<\/p>\n

The R code along with the dropbox link to the data is provided. Pasting this on R console will give the word cloud shown here:<\/p>\n

\"isi_codata_word_cloud\"<\/a><\/p>\n

Here is the code:<\/p>\n

###########################################<\/p>\n

#load packages<\/p>\n

library(wordcloud)<\/p>\n

library(tm)<\/p>\n

#read the presentation details<\/p>\n

textf = readLines(“http:\/\/dl.dropboxusercontent.com\/u\/111213395\/text_file_presentation_titles.txt”)<\/span><\/a><\/span><\/p>\n

# get a column of strings.<\/p>\n

text_corpus <- Corpus(VectorSource(textf))<\/p>\n

# create document term matrix and apply transformations<\/p>\n

tdm = TermDocumentMatrix(text_corpus,<\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0 control = list(removePunctuation = TRUE, stripWhitespace=TRUE,<\/p>\n

\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 stopwords = c(stopwords()),PlainTextDocument =TRUE, removeNumbers = TRUE, tolower = TRUE))<\/p>\n

m <- as.matrix(tdm)<\/p>\n

v <- sort(rowSums(m),decreasing=TRUE)<\/p>\n

d <- data.frame(word = names(v),freq=v)<\/p>\n

pal <- brewer.pal(6,”Dark2″)<\/p>\n

pal <- pal[-(1)]<\/p>\n

#plot the word cloud and save as png<\/p>\n

png(“test.png”,width=3.25,height=3.25,units=”in”,res=1200)<\/p>\n

wordcloud(d$word,d$freq,c(4,.3),2,,TRUE,TRUE,.15,pal)<\/p>\n

dev.off()<\/p>\n

###########################################<\/p>\n

I also\u00a0created a word cloud from twitter using #isibigdata (total ~100 tweets). Unlike, the \u00a0text based word cloud above, twitter extraction required little bit of customization (setting up credentials for a twitteR session), but otherwise the plotting codes are almost the same).<\/p>\n

\"isibigdata<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

This post was written by Shiva Khanal,\u00a0Research Officer with the\u00a0Department of Forest Research and Survey in Nepal. \u00a0Shiva was one of the international scholars sponsored by CODATA to attend the ISI-CODATA International Training Workshop on Big Data. This March two associated events were co-organized by the\u00a0Committee on Data for Science and Technology (CODATA) and the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-373","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/posts\/373","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/comments?post=373"}],"version-history":[{"count":5,"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/posts\/373\/revisions"}],"predecessor-version":[{"id":1916,"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/posts\/373\/revisions\/1916"}],"wp:attachment":[{"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/media?parent=373"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/categories?post=373"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codata.org\/blog\/wp-json\/wp\/v2\/tags?post=373"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}