Shannon's Experiment

Over the past few days I've worked through Chapter 2 of Foundations (Manning and Schuetze), including writing a Python program to reproduce Shannon's Experiment.

Program for administering Shannon's experiment to test the entropy of written English

My solutions to the Chapter 2 exercises, including some rants about computing entropy empirically, are to be found in the PDF for chapter 2 on my FSNLP page.

It is still unclear to me how exactly Shannon computed entropy in his experiment. I've experimented with a few different methods, but I'm not quite satisfied. Perhaps the original paper would help?

Originally published on Quasiphysics.