Glove cpu #57
Glove cpu #57
Conversation
| if 0: #torch.cuda.is_available(): | ||
| try: | ||
| self.model = SentenceTransformer(self.model_name) | ||
| except: | ||
| logger.warning(f"No model {self.model_name} is found.") | ||
|
|
There was a problem hiding this comment.
this is dead code. it will always evaluate to false
There was a problem hiding this comment.
please address the comments. looking at the metaflow run for benchmarking glove, i don't see the gains that were expected (seen previously here). Could it be due to the limited runtime?
| if 0: #torch.cuda.is_available(): | ||
| transformed_train_column = huggingface_run(self.model, np.transpose(train_X[series_name].to_numpy()).T) | ||
| transformed_test_column = huggingface_run(self.model, np.transpose(test_X[series_name].to_numpy()).T) |
| @@ -62,5 +91,5 @@ def _transform_dataframes(self, train_X: pd.DataFrame, test_X: pd.DataFrame) -> | |||
| ] | |||
| train_X = pd.concat([train_X.drop([series_name], axis=1), transformed_train_column], axis=1) | |||
| test_X = pd.concat([test_X.drop([series_name], axis=1), transformed_test_column], axis=1) | |||
|
|
|||
| return train_X, test_X | |||
|
|
|||
|
I don't think that's the case, since DBInfer also uses glove (see here) and our previous benchmarks with their preprocessing showed that specifically stumbleupon competition was heavily benefitted even with glove embeddings and them running on CPU. Anyway, my benchmark that is running at the moment will get the 4hrs results for this as well, but i think glove embeddings are capable to push the scores in this competition based on previous benchmarks. |
I'm using the same model as their setting. I think it's likely that you were previously looking at the public leaderboard. it was 100% for public one, and 60% on private. |
|
please focus on your task of benchmarking. I'll take the weekend to look at the results and I'll prioritize the tasks based on the results. Thanks! Depends on the results, I might priority on this task or not |
|
sorted_competitions_341.csv |
I appreciate the results. however, it don't have any context to make a good call on the results. What was the code that ran it? and why isn't that code being implemented instead? Overall, all I wanted to say:
|
|
I think it is not a single benchmark, but all the datasets in the future that will have text columns, all of them will benefit from this. As for the code used, here: #2 It is the code from dbinfer/tab2graph library. |
Issue #, if available:
Description of changes:
In this PR, when no cuda is detected, it will automatically use
glove-twitter-100.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.