Frederick Jelinek

Frederick Jelinek (November 18, 1932 – September 14, 2010) was a Czech-American researcher in information theory, automatic speech recognition, and natural language processing. He was a leading figure in the development of statistical approaches to speech and language processing. At IBM Research, where he led the Continuous Speech Recognition Group, Jelinek applied information-theoretic and probabilistic methods to speech recognition and later statistical machine translation.

Jelinek was the director of the Center for Language and Speech Processing at Johns Hopkins University from 1993 till 2010. He was elected to the National Academy of Engineering in 2006 for his contributions to statistical language processing and automatic speech recognition, and received the Association for Computational Linguistics Lifetime Achievement Award in 2009.

Early life and education quote = It is generally believed that scientific talent reveals itself in early youth. ... This was certainly not my case. I somehow slid into my scientific profession. My mother wished for me to become a physician, just like my father. ... I myself wanted to be a lawyer, defender of the unjustly accused. But my career is the result of political circumstances, academic possibilities, and lucky accidents. — —Talking about his life in a 2001 speech. His father was a Jewish dentist. His mother, was born in Switzerland to Czech Catholic parents and later converted to Judaism.

Career After receiving his doctorate in 1962, Jelinek joined the faculty of Cornell University, where he taught and conducted research in information theory. He had hoped to collaborate with linguist Charles F. Hockett on applying information theory to language, but the proposed research did not materialize. Jelinek nevertheless helped establish the prominence of Cornell's information-theory program.

In 1972, Jelinek joined IBM Research at the Thomas J. Watson Research Center, initially while on leave from Cornell; he left Cornell permanently in 1974. The group applied information-theoretic and statistical methods to speech recognition and later to machine translation.

Research and legacy In the 1950s, information theory attracted growing interest as a way to study language and communication, although Claude Shannon warned against applying it too broadly. During the 1960s, probabilistic approaches to language became less influential as formal linguistics, associated with Noam Chomsky, gained prominence. Research in machine translation was also curtailed after the 1966 ALPAC report led to reduced U.S. government funding.

Around 1978, the group began developing the Tangora system for more realistic dictation tasks. It used a 5,000-word vocabulary, statistical language models based on trigrams, and hidden Markov models for acoustic modeling.

The IBM group also extended its probabilistic approach beyond speech recognition. In 1987, it began research on statistical machine translation, developing models for translation from French into English using parallel text from the proceedings of the Parliament of Canada.

Honors and awards Jelinek's works won "best paper" awards on several occasions, and he received a number of company awards while he worked at IBM.

Selected publications

Jelinek, Frederick (1968). Probabilistic Information Theory: Discrete and memoryless models. McGraw-Hill series in systems science. New York: McGraw-Hill. 689p. (review) ———————- (1969). "Fast sequential decoding algorithm using a stack". IBM Journal of Research and Development 13(6):675–685. . ———————- (1969). "Tree encoding of memoryless time-discrete sources with a fidelity criterion". IEEE Transactions on Information Theory 15(5):584–590. . (received 1971 "Best Paper" award) Bahl, Lalit R.; John Cocke, Frederick Jelinek, Josef Raviv (1974). "Optimal decoding of linear codes for minimizing symbol error rate". IEEE Transactions on Information Theory 20(2):284–287. . (received Information Theory Society Golden Jubilee paper award) ———————- (1976). "Continuous speech recognition by statistical methods". Proceedings of the IEEE 64(4):532–556. . Brown, P.; J. Cocke, S. Della Pietra, V. Della Pietra, F. Jelinek, R, Mercer and P. Roossin (1988). "A statistical approach to language translation" . In Dénes Vargha, ed. Coling 88: Proceedings of the 12th conference on Computational linguistics, volume 1. Budapest: John Von Neumann society for computing sciences. pp. 71–76. . . ———————- (1990). "Self-Organized Language Modeling for Speech Recognition". In Alex Waibel & Kai-Fu Lee, eds. Readings in speech recognition. San Mateo: Morgan Kaufmann. 629p. . ———————-; John D. Lafferty and Robert L. Mercer. (1990) "Basic methods of probabilistic context free grammars". Technical Report RC 16374 (72684), IBM. Reprinted in Laface, Pietro; Renato De Mori (1992). Speech Recognition and Understanding: Recent advances, trends, and applications. NATO ASI series. Series F, Computer and systems sciences, 75. New York: Springer-Verlag. pp. 345–360. . ———————- (1997). Statistical Methods for Speech Recognition. Cambridge, Mass.: MIT Press. 283p. . (review) (review 2) Chelba, Ciprian; Frederick Jelinek (2000). "Structured Language Modeling". Computer Speech & Language 14(4):283–332. (received 2002 "Best Paper" award). Expanded version of a presentation at NLDB'99. Klagenfurt, Austria, June 17–19, 1999 . Xu, Peng; Ahmad Emami and Frederick Jelinek (2003). "Training Connectionist Models for the Structured Language Model". In Michael Collins and Mark Steedman, eds. EMNLP '03 Proceedings of the 2003 conference on Empirical methods in natural language processing. East Stroudsburg, Penn.: Association for Computational Linguistics. pp. 160–167. . . (won "best paper" award)