Debian Science Linguistics packages

An open-source shallow-transfer machine translation engine, Apertium is initially aimed at related-language pairs.

It uses finite-state transducers for lexical processing, hidden Markov models for part-of-speech tagging, and finite-state based chunking for structural transfer.

The system is largely based upon systems already developed by the Transducens group at the Universitat d'Alacant, such as interNOSTRUM (Spanish-Catalan, http://www.internostrum.com/welcome.php) and Traductor Universia (Spanish-Portuguese, http://traductor.universia.net).

It will be possible to use Apertium to build machine translation systems for a variety of related-language pairs simply providing the linguistic data needed in the right format.

Artha is a off-line English thesaurus with distinct features like:

• hot-key press word look-up (select text on any window and press a preset hot-key for look-up)
• regular expressions based search (broaden search using wild-cards like *, ?, etc.)
• passive desktop notifications (of word definitions for uninterrupted work-flow)
• spelling suggestions (when the exact spelling is vague/not known)

Once launched, it monitors for a preset hot-key combination. When some text is selected on any window and the hot-key is pressed, it pops-up with the word looked-up. Should the user prefer passive notifications, this can be done by enabling the notifications option.

When the term looked for is vague/not known, then either the search can be broadened with the use of regular expressions (*, ?, etc.) in the search string or spelling suggestions when a term is incorrect.

For regular expressions based search to work, wordnet-sense-index package is required.

Dimbl is a wrapper around the k-nearest neighbor classifier in TiMBL, offering parallel classification on multi-CPU machines. Dimbl splits the original training set, builds separate TiMBL classifiers per training subset, and merges their nearest-neighbor sets per classified instance

Dimbl's features are:

• Wraps neatly around TiMBL, retaining all command line options;
• Knows what to do with your multiple, duo, or quad cores;
• Makes use of the OpenMP specification for parallel programming;
• Can attain superlinear speed gains compared to standard TiMBL.

Dimbl is a product of the ILK Research Group (Tilburg University, The Netherlands).

If you do scientific research in Natural Language Processing using the Memory-Based Learning technique, Dimbl will likely be of use to you.

Memory-Based Learning (MBL) is a machine-learning method applicable to a wide range of tasks in Natural Language Processing (NLP).

Frog is a modular system integrating a morphosyntactic tagger, lemmatizer, morphological analyzer, and dependency parser for the Dutch language. It is based upon it's predecessor TADPOLE (TAgger, Dependency Parser, and mOrphoLogical analyzEr). Using Memory-Based Learning techniques, Tadpole tokenizes, tags, lemmatizes, and morphologically segments word tokens in incoming Dutch UTF-8 text files, and assigns a dependency graph to each sentence. Tadpole is particularly targeted at the increasing need for fast, automatic NLP systems applicable to very large (multi-million to billion word) document collections that are becoming available due to the progressive digitization of both new and old textual data.

NB: Frog can be considered alpha software, and is in a fair state of flux.

Frog is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in NLP, Frog will likely be of use to you.

In Selator, D. and Temperly, D. "Parsing English with a Link Grammar" (1991), the authors defined a new formal grammatical system called a "link grammar". A sequence of words is in the language of a link grammar if there is a way to draw "links" between words in such a way that the local requirements of each word are satisfied, the links do not cross, and the words form a connected graph. The authors encoded English grammar into such a system, and wrote this program to parse English using this grammar.

link-grammar can be used for linguistic parsing for information retrieval or extraction from natural language documents. It can also be used as a grammar checker.

This package contains the user-executable binary.

MBT is a memory-based tagger-generator and tagger in one. The tagger-generator part can generate a sequence tagger on the basis of a training set of tagged sequences; the tagger part can tag new sequences. MBT can, for instance, be used to generate part-of-speech taggers or chunkers for natural language processing. Features:

• Tagger generation: tagged text in, tagger out,
• Optional feedback loop: feed previous tag decision back to input of next decision,
• Easily customizable feature representation; can incorporate user-provided features,
• Automatic generation of separate sub-taggers for known words and unknown words,
• Can make use of full algorithmic parameters of TiMBL.

MBT is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in natural language processing, MBT will likely be of use to you.

MbtServer extends Mbt with a server layer, running as a TCP server. Mbt is a memory-based tagger-generator and tagger for natural language processing. MbtServer provides the possibility to access a trained tagger from multiple sessions. It also allows one to run and access different taggers in parallel.

MbtServer is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in natural language processing, MbtServer will likely be of use to you.

Memory-Based Learning (MBL) is a machine-learning method applicable to a wide range of tasks in Natural Language Processing (NLP).

The Tilburg Memory Based Learner, TiMBL, is a tool for NLP research, and for many other domains where classification tasks are learned from examples. It is an efficient implementation of k-nearest neighbor classifier.

TiMBL's features are:

• Fast, decision-tree-based implementation of k-nearest neighbor classification;

• Implementations of IB1 and IB2, IGTree, TRIBL, and TRIBL2 algorithms;

• Similarity metrics: Overlap, MVDM, Jeffrey Divergence, Dot product, Cosine;

• Feature weighting metrics: information gain, gain ratio, chi squared, shared variance;

• Distance weighting metrics: inverse, inverse linear, exponential decay;

• Extensive verbosity options to inspect nearest neighbor sets;
• Server functionality and extensive API;
• Fast leave-one-out testing and internal cross-validation;
• and Handles user-defined example weighting.

TiMBL is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in NLP, timbl will likely be of use to you.

timblserver is a TiMBL wrapper; it adds server functionality to TiMBL. It allows TiMBL to run multiple experiments as a TCP server, optionally via HTTP.

The Tilburg Memory Based Learner, TiMBL, is a tool for Natural Language Processing research, and for many other domains where classification tasks are learned from examples.

TimblServer is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in NLP, TimblServer will likely be of use to you.

Ucto can tokenize UTF-8 encoded text files (i.e. separate words from punctuation, split sentences, generate n-grams), and offers several other basic preprocessing steps (change case, count words/characters and reverse lines) that make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation.

Ucto is a product of the ILK Research Group, Tilburg University (The Netherlands).

If you are interested in machine parsing of UTF-8 encoded text files, e.g. to do scientific research in natural language processing, ucto will likely be of use to you.

WordNet(C) is an on-line lexical reference system whose design is inspired by current psycholinguistic theories of human lexical memory. English nouns, verbs, adjectives and adverbs are organized into synonym sets, each representing one underlying lexical concept. Different relations link the synonym sets.

WordNet was developed by the Cognitive Science Laboratory at Princeton University under the direction of Professor George A. Miller (Principal Investigator).

WordNet is considered to be the most important resource available to researchers in computational linguistics, text analysis, and many related areas. Its design is inspired by current psycholinguistic and computational theories of human lexical memory.

Binary and manpages of WordNet as well as general manpages.

FoLiA is an XML-based format for Linguistic Annotation suitable for representing written language resources such as corpora. Its goal is to unify a variety of linguistic annotations in one single rich format, without committing to any particular standard annotation set. Instead, it seeks to accommodate any desired system or tagset, and so offer maximum flexibility. This makes FoLiA language independent. see http://ilk.uvt.nl/folia/ for more information.

libfolia is a product of the ILK Research Group, Tilburg University (The Netherlands).

This package provides the FoLiA header files required to compile C++ programs that use libfolia.

MBT is a memory-based tagger-generator and tagger in one. The tagger-generator part can generate a sequence tagger on the basis of a training set of tagged sequences; the tagger part can tag new sequences. MBT can, for instance, be used to generate part-of-speech taggers or chunkers for natural language processing.

MBT is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

If you do scientific research in natural language processing, MBT will likely be of use to you.

This package provides the header files required to compile C++ programs that use libmbt.

The Tilburg Memory Based Learner, TiMBL, is a tool for Natural Language Processing research, and for many other domains where classification tasks are learned from examples. It is an efficient implementation of k-nearest neighbor classifier.

TiMBL is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

This package provides the TiMBL header files required to compile C++ programs that use TiMBL.

timblserver is a TiMBL wrapper; it adds server functionality to TiMBL. It allows TiMBL to run multiple experiments as a TCP server, optionally via HTTP.

The Tilburg Memory Based Learner, TiMBL, is a tool for Natural Language Processing research, and for many other domains where classification tasks are learned from examples.

TimblServer is a product of the ILK Research Group (Tilburg University, The Netherlands) and the CLiPS Research Centre (University of Antwerp, Belgium).

This package provides the header files required to compile C++ programs that use timblserver

Ucto can tokenize UTF-8 encoded text files (i.e. separate words from punctuation, split sentences, generate n-grams), and offers several other basic preprocessing steps (change case, count words/characters and reverse lines) that make your text suited for further processing such as indexing, part-of-speech tagging, or machine translation.

Ucto is a product of the ILK Research Group, Tilburg University (The Netherlands).

This package provides the ucto header files required to compile C++ programs that use ucto.

Sequitur G2P is a data-driven grapheme-to-phoneme converter. It can be applied to any monotonous sequence translation problem, provided the source and target alphabets are small (less than 255 symbols). Data-driven means that you need to train it with example pronunciations. Training takes a pronunciation dictionary and creates a model file. The model file can then be used to transcribe words that where not in the dictionary.

WordNet SQL Builder is a Java utility to generate SQL database from WordNet standard database as released by the WordNet Project (Princeton University)

Features

• Support for MySql and PostGreSQL.
• Complete port (however, orphaned morphological forms are dropped, and so are VerbNet/XWordNet data that cannot be linked to WordNet entries).
• Incremental build support.
• Retains synset index as primary key allowing easy reference to wordnet original database
• Includes support for WordNet 3.0
• Includes support for WordNet 2.0 to 2.1, 2.1 to 3.0, 2.0 to 3.0 sense maps
• Includes support for VerbNet 2.3
• Includes support for XWordNet 2.0-1.1
• Ready-to-use database (see wnsqldatabase package in download section) including
• WordNet 3.0
• WordNet 2.0 to 2.1, 2.1 to 3.0, 2.0 to 3.0 sense maps
• VerbNet 2.3
• XWordNet 2.0-1.1
• British National Corpus statistical data (for commonly used-words)