BIST Dependency Parser

Graph-based dependency parser using BiLSTM feature extractors

The techniques behind the parser are described in the Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations [1].

Usage

To use the module, import it like so:

from nlp_architect.models.bist_parser import BISTModel

Training

Training the parser requires having a train.conllu file formatted according to the CoNLL-U data format, annotated with part-of-speech tags and dependencies. The benchmark was performed on a Mac book pro with i7 processor. The parser achieves an accuracy of 93.8 UAS on the standard Penn Treebank dataset (Universal Dependencies).

Basic Example

To train a parsing model with default parameters, type the following:

parser = BISTModel()
parser.fit('/path/to/train.conllu')

Exhaustive Example

Optionally, the following model/training parameters can be supplied (overriding their default values listed below):

parser = BISTModel(activation='tanh', lstm_layers=2, lstm_dims=125, pos_dims=25)
parser.fit('/path/to/train.conllu', epochs=10)

Conducting Intermediate Evaluations

If a path to a development dataset file (annotated with POS tags and dependencies) is supplied, intermediate model evaluations are conducted:

parser = BISTModel()
parser.fit('/path/to/train.conllu', dev='/path/to/dev.conllu')

For each completed epoch, denoted by n, the following files will be created in the dataset’s directory:

  • dev_epoch_n_pred.conllu - prediction results on dev file after n iterations.
  • dev_epoch_n_pred_eval.txt - accuracy results of the above predictions.

Inference

Once you have a trained BISTModel, there are two acceptable input modes for running inference with it. For both modes, the input must be annotated with part-of-speech tags.

File Input Mode

Supply a path to a dataset file in the CoNLL-U data format.

predictions = parser.predict(dataset='/path/to/test.conllu')

After running the above example, predictions will hold the input sentences with annotated dependencies, as a collection of ConllEntry objects, where each ConllEntry represents an annotated token.

ConllEntry Input Mode

Supply a list of sentences, where each sentence is a list of annotated tokens, represented by ConllEntry instances.

predictions = parser.predict(conll='/path/to/test.conllu')

The output format is the same as in file input mode.

Evaluating Predictions

Running an evaluation requires the following: - Inference must be run in file input mode - The input file must be annotated with dependencies as well

To evaluate predictions immediately after they’re generated, type the following:

predictions = parser.predict(dataset='/path/to/test.conllu', evaluate=True)

This will produce 2 files in your input dataset’s directory:

  • test_pred.conllu - predictions file in CoNLL-U format
  • test_pred_eval.txt - evaluation report text file

Saving and Loading a Model

To save a BISTModel to some path, type:

parser.save('/path/to/bist.model')

This operation will also produce a model parameters file named params.json, in the same directory. This file is required for loading the model afterwards.

To load a BISTModel from some path, type:

parser.load('/path/to/bist.model')

Note that this operation will also look for the params.json in the same directory.

References

[1]Kiperwasser, E., & Goldberg, Y. (2016). Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations. Transactions Of The Association For Computational Linguistics, 4, 313-327. https://transacl.org/ojs/index.php/tacl/article/view/885/198