You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First of all let me thank you for such a great library. I really like it!
But I have a question. I use my own dataset; I created the .item and .iter files, it loads and trains ok.
But it is quite big (882 000 users, 1.4 mln items) so it takes quite a lot of memory and time to train the model.
(I try to use the sequential recommender - SASRec and SHAN)
So I need to use the trained model for prediction (to make some recomendations) without loading the full dataset.
How can I do it?
I've read this topic: #1288 but I still can not understand, how to load the model at all?
For now, I train the model using run_recbole(model=my_model_name, dataset=DATASET_NAME, config_dict=parameter_dict)
and then config, model, dataset, train_data, valid_data, test_data = load_data_and_model(model_file='saved/my_model.pth')
but it still takes a lot of memory
I checked the code of the load_data_and_model function and tried to do:
but it is almost the same thing. As I can understand, it still loads the train_data, so it takes a lot of memory
Is there any way to avoid this?
And then, how should I create the interaction object for prediction ? I hardly can understand it. Could you explain it, please.
For today I use scores, topk_iid_list = full_sort_topk([user], model, test_data, k=N_PREDICTIONS, device=config['device'])
but it requires the test_data, so I have to use data_preparation, and therefore load all the data (am I right?)
Or maybe there are some code examples somewhere?
Sorry for the naive questions, I'm new to working with the Recbole.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hi,
First of all let me thank you for such a great library. I really like it!
But I have a question. I use my own dataset; I created the .item and .iter files, it loads and trains ok.
But it is quite big (882 000 users, 1.4 mln items) so it takes quite a lot of memory and time to train the model.
(I try to use the sequential recommender - SASRec and SHAN)
So I need to use the trained model for prediction (to make some recomendations) without loading the full dataset.
How can I do it?
I've read this topic: #1288 but I still can not understand, how to load the model at all?
For now, I train the model using
run_recbole(model=my_model_name, dataset=DATASET_NAME, config_dict=parameter_dict)
and then
config, model, dataset, train_data, valid_data, test_data = load_data_and_model(model_file='saved/my_model.pth')
but it still takes a lot of memory
I checked the code of the load_data_and_model function and tried to do:
but it is almost the same thing. As I can understand, it still loads the train_data, so it takes a lot of memory
Is there any way to avoid this?
And then, how should I create the interaction object for prediction ? I hardly can understand it. Could you explain it, please.
For today I use
scores, topk_iid_list = full_sort_topk([user], model, test_data, k=N_PREDICTIONS, device=config['device'])
but it requires the test_data, so I have to use data_preparation, and therefore load all the data (am I right?)
Or maybe there are some code examples somewhere?
Sorry for the naive questions, I'm new to working with the Recbole.
Sincerely yours,
Arseny
Beta Was this translation helpful? Give feedback.
All reactions