Visualizing your Data

Start digging in the data mines for this module here

'Shaft', by Kačka a Ondra, Flickr

<iframe width="420" height="315" src="https://www.youtube.com/embed/jIfu2A0ezq0" frameborder="0" allowfullscreen></iframe>

#Qualitative data vs Quantitative

how do you see patterns in words?

You count 'em.

or, you realize that you can count the patterns that clusters of words make up

http://biodiversitylibrary.org/page/37047310#page/496/mode/1up

Many tools available

and many approaches
traditionally, easier to get money to develop a new tool than to do research using that tool

See

http://dirtdirectory.org/

Voyant

voyant-tools.org

So let's see what word counts can get us

Colonial Newspapers Database

Voyant contains many tools

Cogwheel icon at top-right
Especially useful: RezoViz, Bubble Lines
see my Ferfuson post

OverView Project

overviewproject.org
Look at distributions of words over a corpus
TF-IDF

How it works

The cat sat on the mat. Then the cat chased the rat.
The cat slept all day on the mat.
The rat ran across the floor.

Strip the stopwords

cat sat mat cat chased rat
cat slept all day mat
rat ran across floor

Count what remains

TF-IDF

compare every pair of documents
multiply the frequencies of corresponding words

Visualize

Tag, Visualize

Topic Modeling

many approaches go under the term 'topic modeling'
most common in humanities approaches: LDA

Topic Modeling Gettysburg Address

War v Governance

What % of this text is composed by a 'war' topic?
How do you know what 'war' words are?

Teaching the computer

supervised vs. unsupervised

How the world works

we all just pull from bags of words, right?

State library of Australia

Topic Model Gui

Typical output

Other ways of representing the conenction b/w topics & documents?

Heatmap
Network

MALLET in R

Using a MALLET on the CND
Rstudio.org

Next Day

AntConc
NER
SNA with Gephi