Skip to content

Commit

Permalink
rename evaluate templates
Browse files Browse the repository at this point in the history
  • Loading branch information
@kervias
kervias committed Feb 3, 2024
1 parent 85536e0 commit 6eec637
Show file tree
Hide file tree
Showing 62 changed files with 124 additions and 124 deletions.
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ run_edustudio(
'cls': 'NCDM',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
}
)

Expand Down
8 changes: 4 additions & 4 deletions docs/source/developer_guide/customize_evaltpl.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,14 @@ The protocols in ``BaseEvalTPL`` are listed as follows.
EvalTPLs
----------------------

EduStudio provides ``BinaryClassificationEvalTPL`` and ``CognitiveDiagnosisEvalTPL``, which inherent ``BaseEvalTPL``.
EduStudio provides ``PredictionEvalTPL`` and ``InterpretabilityEvalTPL``, which inherent ``BaseEvalTPL``.

### BinaryClassificationEvalTPL
### PredictionEvalTPL
This EvalTPL is for the model evaluation using binary classification metrics.
The protocols in ``BinaryClassificationEvalTPL`` are listed as follows.
The protocols in ``PredictionEvalTPL`` are listed as follows.


### CognitiveDiagnosisEvalTPL
### InterpretabilityEvalT
This EvalTPL is for the model evaluation for interpretability. It uses states of students and Q matrix for ``eval``, which are domain-specific in student assessment.

## Develop a New EvalTPL in EduStudio
Expand Down
8 changes: 4 additions & 4 deletions docs/source/features/dataset_folder_protocol.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ run_edustudio(
'cls': 'KaNCD',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
}
)
```
Expand Down Expand Up @@ -90,7 +90,7 @@ run_edustudio(
'cls': 'KaNCD',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
}
)
```
Expand Down Expand Up @@ -118,7 +118,7 @@ run_edustudio(
'cls': 'KaNCD',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
}
)
```
Expand Down Expand Up @@ -154,7 +154,7 @@ run_edustudio(
'cls': 'KaNCD',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
}
)
```
2 changes: 1 addition & 1 deletion docs/source/get_started/quick_start.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ run_edustudio(
'cls': 'KaNCD',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
}
)
```
Expand Down
88 changes: 44 additions & 44 deletions docs/source/user_guide/reference_table.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,52 +4,52 @@

| Model | DataTPL | TrainTPL | EvalTPL |
| :------ | ---------------------: | :-------------: | ------------------------------------------------------ |
| IRT | CDInterDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| MIRT | CDInterDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| NCDM | CDInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL、CognitiveDiagnosisEvalTPL |
| CNCD_Q | CNCDQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| CNCD_F | CNCDFDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| DINA | CDInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL、CognitiveDiagnosisEvalTPL |
| HierCDF | HierCDFDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL、CognitiveDiagnosisEvalTPL |
| CDGK | CDGKDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL、CognitiveDiagnosisEvalTPL |
| CDMFKC | CDInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| ECD | ECDDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| IRR | IRRDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| KaNCD | CDInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL、CognitiveDiagnosisEvalTPL |
| KSCD | CDInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| MGCD | MGCDDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| RCD | RCDDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| IRT | CDInterDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| MIRT | CDInterDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| NCDM | CDInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL、InterpretabilityEvalTPL |
| CNCD_Q | CNCDQDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| CNCD_F | CNCDFDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| DINA | CDInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL、InterpretabilityEvalTPL |
| HierCDF | HierCDFDataTPL | GeneralTrainTPL | PredictionEvalTPL、InterpretabilityEvalTPL |
| CDGK | CDGKDataTPL | GeneralTrainTPL | PredictionEvalTPL、InterpretabilityEvalTPL |
| CDMFKC | CDInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| ECD | ECDDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| IRR | IRRDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| KaNCD | CDInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL、InterpretabilityEvalTPL |
| KSCD | CDInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| MGCD | MGCDDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| RCD | RCDDataTPL | GeneralTrainTPL | PredictionEvalTPL |

## KT models

| Model | DataTPL | TrainTPL | EvalTPL |
| :----------- | ----------------------: | :-------------: | --------------------------- |
| AKT | KTInterDataTPLCptUnfold | GeneralTrainTPL | BinaryClassificationEvalTPL |
| ATKT | KTInterDataTPLCptUnfold | AtktTrainTPL | BinaryClassificationEvalTPL |
| CKT | KTInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| CL4KT | CL4KTDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| CT_NCM | KTInterCptUnfoldDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| DeepIRT | KTInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| DIMKT | DIMKTDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| DKT | KTInterDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| DKTDSC | DKTDSCDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| DKTForget | DKTForgetDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| DKT_plus | KTInterDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| DKVMN | KTInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| DTransformer | KTInterCptUnfoldDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| EERNN | EERNNDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| EKT | EKTDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| GKT | GKTDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| HawkesKT | KTInterCptUnfoldDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| IEKT | KTInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| KQN | KTInterDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| LPKT | LPKTDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| LPKT_S | LPKTDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| QDKT | QDKTDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| QIKT | KTInterExtendsQDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| RKT | RKTDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| SAINT | KTInterCptUnfoldDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| SAINT_plus | KTInterCptUnfoldDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| SAKT | KTInterDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| SimpleKT | KTInterCptUnfoldDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| SKVMN | KTInterDataTPL | GeneralTrainTPL | BinaryClassificationEvalTPL |
| AKT | KTInterDataTPLCptUnfold | GeneralTrainTPL | PredictionEvalTPL |
| ATKT | KTInterDataTPLCptUnfold | AtktTrainTPL | PredictionEvalTPL |
| CKT | KTInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| CL4KT | CL4KTDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| CT_NCM | KTInterCptUnfoldDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| DeepIRT | KTInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| DIMKT | DIMKTDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| DKT | KTInterDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| DKTDSC | DKTDSCDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| DKTForget | DKTForgetDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| DKT_plus | KTInterDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| DKVMN | KTInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| DTransformer | KTInterCptUnfoldDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| EERNN | EERNNDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| EKT | EKTDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| GKT | GKTDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| HawkesKT | KTInterCptUnfoldDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| IEKT | KTInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| KQN | KTInterDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| LPKT | LPKTDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| LPKT_S | LPKTDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| QDKT | QDKTDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| QIKT | KTInterExtendsQDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| RKT | RKTDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| SAINT | KTInterCptUnfoldDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| SAINT_plus | KTInterCptUnfoldDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| SAKT | KTInterDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| SimpleKT | KTInterCptUnfoldDataTPL | GeneralTrainTPL | PredictionEvalTPL |
| SKVMN | KTInterDataTPL | GeneralTrainTPL | PredictionEvalTPL |
4 changes: 2 additions & 2 deletions docs/source/user_guide/usage/aht.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ search_space= {
'traintpl_cfg.cls': tune.grid_search(['GeneralTrainTPL']),
'datatpl_cfg.cls': tune.grid_search(['CDInterExtendsQDataTPL']),
'modeltpl_cfg.cls': tune.grid_search(['KaNCD']),
'evaltpl_cfg.clses': tune.grid_search([['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL']]),
'evaltpl_cfg.clses': tune.grid_search([['PredictionEvalTPL', 'InterpretabilityEvalTPL']]),


'traintpl_cfg.batch_size': tune.grid_search([256,]),
Expand Down Expand Up @@ -118,7 +118,7 @@ space = {
'traintpl_cfg.cls': hp.choice('traintpl_cfg.cls', ['GeneralTrainTPL']),
'datatpl_cfg.cls': hp.choice('datapl_cfg.cls', ['CDInterExtendsQDataTPL']),
'modeltpl_cfg.cls': hp.choice('modeltpl_cfg.cls', ['KaNCD']),
'evaltpl_cfg.clses': hp.choice('evaltpl_cfg.clses', [['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL']]),
'evaltpl_cfg.clses': hp.choice('evaltpl_cfg.clses', [['PredictionEvalTPL', 'InterpretabilityEvalTPL']]),


'traintpl_cfg.batch_size': hp.choice('traintpl_cfg.batch_size', [256,]),
Expand Down
4 changes: 2 additions & 2 deletions docs/source/user_guide/usage/run_edustudio.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ run_edustudio(
'cls': 'KaNCD',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
}
)
```
Expand Down Expand Up @@ -55,7 +55,7 @@ modeltpl_cfg:
cls: NCDM

evaltpl_cfg:
clses: [BinaryClassificationEvalTPL, CognitiveDiagnosisEvalTPL]
clses: [PredictionEvalTPL, InterpretabilityEvalT]
```
then, run command:
Expand Down
10 changes: 5 additions & 5 deletions docs/source/user_guide/usage/use_case_of_config.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,15 +34,15 @@ run_edustudio(
'cls': 'KaNCD',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
}
)
```

## Q2: How to specify the config of evaluate template
The default_cfg of `BinaryClassificationEvalTPL` is as follows:
The default_cfg of `PredictionEvalTPL` is as follows:
```python
class BinaryClassificationEvalTPL(BaseEvalTPL):
class PredictionEvalTPL(BaseEvalTPL):
default_cfg = {
'use_metrics': ['auc', 'acc', 'rmse']
}
Expand Down Expand Up @@ -70,8 +70,8 @@ run_edustudio(
'cls': 'KaNCD',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'CognitiveDiagnosisEvalTPL': {
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
'InterpretabilityEvalTPL': {
'use_metrics': {"auc"} # look here
}
}
Expand Down
4 changes: 2 additions & 2 deletions edustudio/evaltpl/__init__.py
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from .base_evaltpl import BaseEvalTPL
from .bc_evaltpl import BinaryClassificationEvalTPL
from .cd_evaltpl import CognitiveDiagnosisEvalTPL
from .prediction_evaltpl import PredictionEvalTPL
from .interpretability_evaltpl import InterpretabilityEvalTPL
from .fairness_evaltpl import FairnessEvalTPL
2 changes: 2 additions & 0 deletions edustudio/evaltpl/fairness_evaltpl.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@


class FairnessEvalTPL(BaseEvalTPL):
"""Fairness Cognitive Evaluation
"""
default_cfg = {
'use_sensi_attrs': ['gender:token'],
'use_metrics': ['EO', 'DP', 'FCD']
Expand Down
10 changes: 3 additions & 7 deletions edustudio/evaltpl/cd_evaltpl.py → ...tudio/evaltpl/interpretability_evaltpl.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@
from edustudio.utils.callback import ModeState


class CognitiveDiagnosisEvalTPL(BaseEvalTPL):
class InterpretabilityEvalTPL(BaseEvalTPL):
"""Student Cogntive Representation Interpretability Evaluation
"""
default_cfg = {
'use_metrics': ['doa_all'],
'test_only_metrics': ['doa_all']
Expand Down Expand Up @@ -189,10 +191,4 @@ def doa_eval(self, y_true, y_pred):
doa.append(_doa / _z)
z_support += _z # 有效pair个数
doa_support += 1 # 有效doa
# return {
# "doa": np.mean(doa),
# "doa_know_support": doa_support,
# "doa_z_support": z_support,
# "doa_list": doa,
# }
return float(np.mean(doa))
4 changes: 3 additions & 1 deletion edustudio/evaltpl/bc_evaltpl.py → edustudio/evaltpl/prediction_evaltpl.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@
from sklearn.metrics import mean_squared_error, roc_auc_score, accuracy_score, f1_score, label_ranking_loss, coverage_error


class BinaryClassificationEvalTPL(BaseEvalTPL):
class PredictionEvalTPL(BaseEvalTPL):
"""Student Performance Prediction Evaluation
"""
default_cfg = {
'use_metrics': ['auc', 'acc', 'rmse']
}
Expand Down
2 changes: 1 addition & 1 deletion examples/1.run_cd_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@
'cls': 'KaNCD',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL'],
'clses': ['PredictionEvalTPL', 'InterpretabilityEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/2.run_kt_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@
'cls': 'DKT',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/3.run_with_customized_tpl.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@
'cls': DKT,
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/5.run_with_hyperopt.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def objective_function(args):
'traintpl_cfg.cls': hp.choice('traintpl_cfg.cls', ['GeneralTrainTPL']),
'datatpl_cfg.cls': hp.choice('datapl_cfg.cls', ['CDInterExtendsQDataTPL']),
'modeltpl_cfg.cls': hp.choice('modeltpl_cfg.cls', ['KaNCD']),
'evaltpl_cfg.clses': hp.choice('evaltpl_cfg.clses', [['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL']]),
'evaltpl_cfg.clses': hp.choice('evaltpl_cfg.clses', [['PredictionEvalTPL', 'InterpretabilityEvalTPL']]),


'traintpl_cfg.batch_size': hp.choice('traintpl_cfg.batch_size', [256,]),
Expand Down
2 changes: 1 addition & 1 deletion examples/6.run_with_ray.tune.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ def objective_function(args):
'traintpl_cfg.cls': tune.grid_search(['GeneralTrainTPL']),
'datatpl_cfg.cls': tune.grid_search(['CDInterExtendsQDataTPL']),
'modeltpl_cfg.cls': tune.grid_search(['KaNCD']),
'evaltpl_cfg.clses': tune.grid_search([['BinaryClassificationEvalTPL', 'CognitiveDiagnosisEvalTPL']]),
'evaltpl_cfg.clses': tune.grid_search([['PredictionEvalTPL', 'InterpretabilityEvalTPL']]),


'traintpl_cfg.batch_size': tune.grid_search([256,]),
Expand Down
2 changes: 1 addition & 1 deletion examples/single_model/run_akt_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@
'cls': 'AKT',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/single_model/run_atkt_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@
'cls': 'ATKT',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/single_model/run_cdgk_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,6 @@
'cls': 'CDGK_MULTI',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/single_model/run_cdmfkc_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@
'cls': 'CDMFKC',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/single_model/run_ckt_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,6 @@
'cls': 'CKT',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/single_model/run_cl4kt_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@
'cls': 'CL4KT',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/single_model/run_cncd_f_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@
'cls': 'CNCD_F',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/single_model/run_cncdq_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@
'cls': 'CNCD_Q',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)
2 changes: 1 addition & 1 deletion examples/single_model/run_ctncm_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
'cls': 'CT_NCM',
},
evaltpl_cfg_dict={
'clses': ['BinaryClassificationEvalTPL'],
'clses': ['PredictionEvalTPL'],
}
)

Loading

0 comments on commit 6eec637

Please sign in to comment.