Bidirectional LSTM/CRF (BiLTSM-CRF) Training System

From GM-RKB
(Redirected from BiLSTM-CRF Training System)
Jump to navigation Jump to search

A Bidirectional LSTM/CRF (BiLTSM-CRF) Training System is a bidirectional LSTM training system that includes a CRF training system and implements a bi-directional LSTM/CRF training algorithm to train a biLSTM-CRF model.

import torch

import torch.autograd as autograd

import torch.nn as nn

import torch.optim as optim

def argmax(vec):

(...)

class BiLSTM_CRF(nn.Module):

(...)

model = BiLSTM_CRF(len(word_to_ix), tag_to_ix, EMBEDDING_DIM, HIDDEN_DIM)

optimizer = optim.SGD(model.parameters(), lr=0.01, weight_decay=1e-4)

(...)



References

2018a

The implementation is highly configurable, so you can tune the different hyperparameters easily. You can use it for Single Task Learning as well as different options for Multi-Task Learning. You can also use it for Multilingual Learning by using multilingual word embeddings. …

2018b

(sents.txt)

the fat rat sat on a mat

...

(post.txt)

det adj noun verb prep det noun

...

(labels.txt)

O O B-Animal O O O B-Object

...

Then above input and label files are provided to train.py using --input-path and --label-path respectively.

python train.py --input-path sents.txt --input-path pos.txt --label-path labels.txt

You might need to setup several more parameters in order to make it work. Checkout examples/atis for an example of training a simple BiLSTM-CRF model with ATIS dataset. Run python preprocess.py at the example directory to convert to the dataset to train.py-friendly format, then run

python ../../train.py --config train-atis.yml

to see a running example. The example configuration assumes that standalone tensorboard is installed (you could turn it off in the configuration file).

For more information on the configurations, check out python train.py --help.

2017a

2017b

2017c