Skip to content
This repository has been archived by the owner on Jan 15, 2024. It is now read-only.

dmlc/gluon-nlp

Folders and files

NameName
Last commit message
Last commit date

Latest commit

14553a0 · Dec 25, 2022
Apr 21, 2021
May 21, 2021
Dec 25, 2022
Dec 16, 2022
Aug 24, 2021
May 22, 2021
Jun 16, 2020
Jun 10, 2020
Jan 9, 2021
Apr 19, 2018
Jun 10, 2020
Aug 20, 2020
Aug 20, 2020
Aug 20, 2020
Jan 9, 2021
Apr 19, 2021
Sep 15, 2020
Jun 10, 2020
Dec 25, 2022

Repository files navigation

GluonNLP Logo

GluonNLP: Your Choice of Deep Learning for NLP

GluonNLP is a toolkit that helps you solve NLP problems. It provides easy-to-use tools that helps you load the text data, process the text data, and train models.

See our documents at https://nlp.gluon.ai/master/index.html.

Features

  • Easy-to-use Text Processing Tools and Modular APIs
  • Pretrained Model Zoo
  • Write Models with Numpy-like API
  • Fast Inference via Apache TVM (incubating) (Experimental)
  • AWS Integration via SageMaker

Installation

First of all, install the MXNet 2 release such as MXNet 2 Alpha. You may use the following commands:

# Install the version with CUDA 10.2
python3 -m pip install -U --pre "mxnet-cu102>=2.0.0a"

# Install the version with CUDA 11
python3 -m pip install -U --pre "mxnet-cu110>=2.0.0a"

# Install the cpu-only version
python3 -m pip install -U --pre "mxnet>=2.0.0a"

To install GluonNLP, use

python3 -m pip install -U -e .

# Also, you may install all the extra requirements via
python3 -m pip install -U -e ."[extras]"

If you find that you do not have the permission, you can also install to the user folder:

python3 -m pip install -U -e . --user

For Windows users, we recommend to use the Windows Subsystem for Linux.

Access the Command-line Toolkits

To facilitate both the engineers and researchers, we provide command-line-toolkits for downloading and processing the NLP datasets. For more details, you may refer to GluonNLP Datasets and GluonNLP Data Processing Tools.

# CLI for downloading / preparing the dataset
nlp_data help

# CLI for accessing some common data processing scripts
nlp_process help

# Also, you can use `python -m` to access the toolkits
python3 -m gluonnlp.cli.data help
python3 -m gluonnlp.cli.process help

Run Unittests

You may go to tests to see how to run the unittests.

Use Docker

You can use Docker to launch a JupyterLab development environment with GluonNLP installed.

# GPU Instance
docker pull gluonai/gluon-nlp:gpu-latest
docker run --gpus all --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 --shm-size=2g gluonai/gluon-nlp:gpu-latest

# CPU Instance
docker pull gluonai/gluon-nlp:cpu-latest
docker run --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 --shm-size=2g gluonai/gluon-nlp:cpu-latest

For more details, you can refer to the guidance in tools/docker.