state dictionary for indicxlit model

[subnkve479]

Hi
I have downloaded indicxlit model but could not see any state dictionary associated with it.I could only see corpus-bin and pytorch file in transformer folder. But to load any pytorch model,I need to give its state dictonary also. Can you please help me to load indicXlit pytorch model

Below is the code I have tried

Code 1

import torch
model=torch.jit.load('./transformer/indicxlit.pt')
model.eval()

Error:
RuntimeError: PytorchStreamReader failed locating file constants.pkl: file not found

Code 2

import torch

model=torch.load('./transformer/indicxlit.pt')

print(model.keys())

Above code is loading model as dictonary,but I cannot do any predictions.

Can you please help me here

Regards
Subbu

[anoopkunchukuttan]

@yashmadhani97, @GokulNC Can you check?

[GokulNC]

Hello Subrahmanyam

We use the fairseq library for transformers, not PyTorch directly.
Please check the inference section on our README on how to use our models:
https://github.com/AI4Bharat/IndicXlit#running-inference

[subnkve479]

Hi Gokul

Thanks for quick response.I am able to run inference for indicXlit using ai4bharat-transliteration and using fairseq. I am looking to reduce the inference time.(it is currently around 120ms using python package,340ms using fairseq) .I am trying to reduce it by having the model in my local directory and hence trying to download the model

Can you please let me know your thoughts to reduce the inference time as I want to integrate this with realtime application where inference time requirements is around 30ms

Regards
Subbu

[GokulNC]

If you want to do this quickly, you can try a couple of things like:

• Reducing beam-size
• Disabling reranking

You may also try torchscripting or quantizing the model.

Apart from this, I think the other option would be to train a smaller model using our data.
This is a transformer model trained for many languages.
You may train smaller variants of transformer (or any other seq2seq model), just focusing on the language you want.
But this could result in decrease in accuracy.

[subnkve479]

Thanks Gokul.This helps.Will try them

[subnkve479]

HI @GokulNC

I am trying to torchscript the IndicXlit model for which I need to load the model. Is the mode 'Indicxlit.pt' a jitted model?
If not,I think we need model class to load the state dictionary.
Can you help me to get the code for model class ?

Here I am not looking for inferencing the model but to load the model and torchscript the same.

Can you please help me here

Regards
Subbu

[GokulNC]

Hi @subnkve479 , I haven't yet tried torchscripting this model.
This is a fairseq model, so you will have to see how one should torchscript it.

FYI, this is how we directly run inference using the fairseq cli:
https://github.com/AI4Bharat/IndicXlit/blob/master/inference/cli/en-indic/interactive.sh