A Maximum Entropy Approach to Natural Language Processing

Authors: Adam L. Berger, Stephen A. Della Pietra, Vincent J. Della Pietra
Year: 1996
Summary: A foundational paper on the principle of maximum entropy in computational linguistics. It argues that the best statistical model, given a set of constraints, is the one that is otherwise as uniform as possible (i.e., has the highest entropy). This principle is crucial for defining unbiased prior probabilities in Bayesian models of language and memory.
Link: https://aclanthology.org/J96-1002/