Uncharted: Big Data as a Lens on Human Culture by Erez Aiden & Jean-Baptiste Michel

Here we investigate missing data and what it means. Over time, there have been a number of cultures that have suppressed and censored people, information, and artifacts. The primary example here is the impact the Nazis had on this data set. As you might expect, references to all things jewish starts to ebb as the Nazis take over in the 1930’s. By one estimate, 69% of the books were removed from one library. In Russia, Stalin did a fine job of making his opponents disappear from the written record and from the face of the Earth. In China to this day it’s hard to find references to what really happened in Tiananmen Square. Even in the US we had ten movie makers silenced due to the actions of the House Un-American Activities Committee in the 1950’s.
The sudden disappearance of words and phrases is the characteristic mark of censorship and suppression, and the author’s method is more accurate and much faster that traditional historical techniques. Let’s hope this will encourage governments to be more honest and let their people and ideas free.

Here we start with the notion that memory is something very difficult to measure. This is due to the fact that everything you remember is associated with many other things. For the same reason, it is also hard to measure forgetting. With Ngrams, you can graph the collective memory of the English speaking world. The authors provide many learning curves, which show how fast a word or phrase grows and later shrinks in use. They also note that Ngrams don’t work for breaking news as it takes a while for things to show up in books. A study of inventions that take longer to spread than ordinary news shows that society is learning faster. In 2010, the viewer was made available to others and it turned out to be a big hit. As a result, the authors see it as a nerdy version of heroin that people often quickly become addicted to.

The final chapter serves as a warning about the possible benefits and costs of big data. As wearable technology proliferates, we can look forward a future where our health data is automatically uploaded to our doctors, and if something doesn’t look right, the doctor just might call you. We can also worry about what the government might do as it has access to more and more information about what we do and think. There is also hope that the sciences and the humanities will draw closer and that more research will incorporate lessons from both. The authors also think there is promise that their Ngram data hunt can be used to predict where our culture is going and help us avoid negative situations.
The book ends with over 20 pages of graphs that compare one Ngram to another such as satan/santa and Beatles/Jesus Christ. There is also a rich set of notes for each chapter and a comprehensive index.

Book Summaries