Encyclopedia Britannica data visualisation

Because the data was composed of unstructured OCR text, we needed to create our own structured framework. We focused on identifying simple definitions that followed a “TERM, definition” format, as well as cross-references such as “See x.” To gauge the relative popularity of topics across editions, we used the number of cross-references as a proxy.Working with unclean OCR data presented challenges. Regular expressions couldn’t consistently extract entries without noise or omissions. However, because these inconsistencies appeared uniformly across editions, we were still able to draw proportional insights from the processed data. Extracting the most referenced or lengthy terms required manual review of the top entries, since the data was too noisy to be parsed accurately through automation alone.Due to the data being unstructured text, we had to generate our own structured data from it.
We decided to focus on very specific aspects of the data: simple entries of the form “TERM, definition”, as well as references to topics of the form “See x”, and used reference counts as a proxy for the popularity of a topic.

Our audience is the general public. These are curious individuals who are interested in the Encyclopaedia Britannica but may feel overwhelmed by the sheer volume and density of its content.