BASE／BC_H13 v20220210

Model

Encoder

• Type: Transformer encoder layers (the same network structure as the one used for BERT_BASE)
  • Dimension: 768
  • # of heads: 12
  • Dimension of feedforward networks: 3072
  • # of layers: 12
  • Activation function: GELU
  • Dropout rate in training: 0.1
  • Initialization: Random

Decoder

• Type: Single-layer position-wise feedforward network
  • Dimension: 3072
  • Activation function: GELU
  • Dropout rate in training: 0.1
  • Initialization: Random

Objective

• Type: Behavioral cloning (BC)
• Loss function: Cross entropy

Data

Crawled Data

• Period: 2020/07/11 ~ 2021/07/31
• Game style: 4-player, half- and quarter-length games
• Room: Gold, Jade, and Throne
• # of games:
• # of rounds:
• # of samples:

Training Examples

43002752 samples consisting of action selections only by Saint 2, Saint 3, and Celestial players, which are randomly sampled from the crawled data and shuffled.

Optimization

• Optimizer: LAMB
• Learning rate: 0.001
• ε: 1.0e-6
• Batch size: 4096
• # of training epochs: 1