- Python 3.7 or later
- MySQL
- Postgres
- config.cfg
- JUMAN++ or MeCab
- For Japanese tweets
You'll need a configuration file "config.cfg" with MySQL/Postgres connection information (replace * with your keys).
[X/Twitter database]
host = ****
user = ****
passwd = ****
db_name = ****
[GPS trajectory database]
host = ****
user = ****
passwd = ****
db_name = ****
tmux new -s prediction
tmux attach -t prediction
- control + b, d to detach
(assuming Nvidia libraries are properly configured)
- conda env create --name prediction --file environment.yml
- conda activate prediction
- Review
settings.py
- Check settings and parameters of models.
- What is your target area (in xy coordinates)?
- What is your target training time span? (e.g. 2012-07-25 00:00:00 ~ 2012-07-31 23:59:59)
- What is your target test time span? (e.g. 2012-07-25 08:00:00 ~ 2012-07-25 08:59:59)
- Preprocessing
- Embedding
- Run
preprocessing_1_embedding.py
- Run
- X/Twitter topic modeling
- Run
preprocessing_2_train_topic_modeling_local.py
- Run
preprocessing_3_create_topic_features.py
- Run
- GPS (mobile phones)
- Run
preprocessing_4_create_experiment_profile_table.py
- Load GPS trajectory from DB and create profile table.
- Run
preprocessing_5_interpolated_gps_db.py
- Load GPS from DB and process data for machine learning model.
- Run
- Embedding
- Run experiment (training and evaluation)
- Training
- Run
prediction_1_gps_grid_baseline_ngram.py
for runnning ngram baseline model. - RUN
prediction_2_gps_grid_baseline_rnn.py
for running RNN baseline model. - Run
prediction_3_gps_grid.py
for GPS only model. - Run
prediction_4_gps_grid_mode.py
for GPS and mode model. - Run
prediction_5_gps_grid_topic.py
for GPS and topic model. - Run
prediction_6_gps_grid_mode_topic.py
for GPS, mode, and topic model. - Run
prediction_7_gps_grid_mode_topic_attnention.py
for GPS, mode, and topic model with attention layer.
- Run
- Evaluation
- Run
evaluation_1_prediction.py
- Evaluate prediction with Loss
- Run
evaluation_2_prediction_cityemd.py
- Evaluate prediction with CityEMD
- Run
evaluation_3_prediction_small_target.py
- Evaluate prediction in small target area
- Run
- Training
directory | Description |
---|---|
DATA_DIR_RAW = "../data/raw/" | Raw data. Ignored. |
DATA_DIR_PROCESSED = "../data/processed/" | Processed data. Ignored. |
DATA_DIR_INTERIM = "../data/interim/" | Interim data. Ignored. |
OUTPUT_DIR = "../data/output/" + EXPERIMENT_PARAMETERS["EXPERIMENT_NAME"] + "/" | Output data. Ignored, but uploaded to Dropbox. |