Skip to content

WindMillCode/WML-AI-Model-Managers

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

WML AI Model Managers

A package with several AI classes making it easy to train data based on PyTorch.

Initialization

To initialize:

# Make sure the test and train data come from the same dataset
myai = WMLTextModelManagerOne(
    model_file_name="AmazonReviewFull.pkl",
    dataloader_info ={
      "datapipe_fn":datasets.AmazonReviewFull,
      "vocab_folder_path":"data/AmazonReviewFull",
      "get_dataset":True
    }
  )

Training

To train:

  myai.download_train_and_test_data()
  myai.load_model_from_scratch()
  myai.train()
  myai.save_model_to_pickle()

Chat

To chat:

myai =  WMLTextModelManagerOne(
    model_file_name="AmazonReviewFull.pkl",
    dataloader_info ={
      "datapipe_fn":datasets.AmazonReviewFull,
      "vocab_folder_path":"data/AmazonReviewFull",
      "get_dataset":False
    },
  )

myai.download_train_and_test_data()
myai.load_model_from_file()
# the default is 150 but we increased to 500 the limit should be myai.batch_size * myai.block_size but see if you can get larger than this
myai.chat_with_model(500)

Class Initialization Properties

  • device: Specifies whether to use CUDA, CPU, or lets it be determined by available computer hardware.
  • dataloader_info: Important information about the pytorch dataset the model manager needs to retrieve the dataset
    • datapipe_fn: One of the various pytorch datapipes which can be found here https://pytorch.org/text/stable/datasets.html. You can implement your own custom datapipe fn as the model is looking for an IterDataPipe, make sure your custom fn have train and test splits.
    • vocab_folder_path: Where the vocab file for the dataset will be stored. it will store test-vocab.txt and train-vocab.txt.
    • get_dataset: Set to false if communicating with chatbot set to true if doing a training session.
  • max_iters: Number of iterations for the entire training run.
  • n_embd: Number of embeddings.
  • n_head: Number of heads for multihead attention.
  • n_layer: Number of layers (linear, activation, output functions).
  • dropout: Number of values to turn to zero to prevent the model from memorization.
  • model_file_name: The name of the file to save the model to (currently supports only pickle files, so please save as .pkl).
  • reporting_loss.
  • learning_rate.
  • block_size: Amount of characters in a section of text that represents 1 batch.
  • batch_size: Amount of batches the model gets to learn in 1 iteration during the training session. A training session is like a child going through pre-K through college, and each grade is 1 iteration. At the end, the model should be able to generalize (converge) well, like an adult who can meaningfully contribute to society.

Changelog

v0.0.2:

  • Corrected issues with the dataloader where the model_manager would not receive the training and test dataloaders.

v0.0.3:

  • Changed the default block and batch sizes so beginners can feel more tangible results.
  • Refactored

v1.0.0:

  • instead of providing a dataloader argument to the modelManger provide dataloader_info according to the example above

v1.0.1

  • abstracted more fns into

v1.1.0

  • changed from load_model_from_file to load_model_from_pickle_file
  • changed from save_model_to_pickle to save_model_to_pickle_file

v1.1.1

  • fixed a bug with v1.1.0