Ian Eslick & Hugo Liu (2005). Langutils: a natural language toolkit for common lisp. Proceedings of the International Lisp Conference (ILC 2005), Stanford, CA.
Eslick, I., & Liu, H. (2005). Langutils: a natural language toolkit for common lisp. In Proceedings of the International Lisp Conference (ILC 2005), Stanford, CA.
@inproceedings{eslick2005langutils,
author = {Ian Eslick and Hugo Liu},
title = {Langutils: a natural language toolkit for common lisp},
booktitle = {Proceedings of the International Lisp Conference (ILC 2005), Stanford, CA},
url = {https://starheartsong.com/papers/pdf/LISP2005-LangUtils.pdf},
year = {2005}
}Langutils carries a MontyLingua-derived pipeline into Common Lisp while preserving the language's high-level, inspectable style. It combines contextual tokenization, a rule-based Brill tagger, and shallow phrase chunking with compact integer-token arrays and macros that compile recognizers into specialized code. On a 1.67 GHz PowerBook, the tokenizer processed about 30,000 words per second, tagging reached 15,000–21,000, and chunking 250,000 tokens per second. Example applications include command mapping, ConceptNet extraction, and web mining; the last produced only about 30 percent high-quality phrases. The paper measures throughput, not independent linguistic accuracy. Brill training was absent, token representation was unbounded, multilingual handling incomplete, genre sensitivity substantial, and shallow chunks were a deliberate trade for speed.