Code

Metrics redesign#1389

Code

Merged

Liraim merged 150 commits intomainevidentlyai/evidently:mainfrom feature/metrics-redesignevidentlyai/evidently:feature/metrics-redesignCopy head branch name to clipboardJan 10, 2025

+21,608-325

Commits

Commits on Dec 3, 2024

• First draft of redesign.

  Liraim

  committed
  07d3a19

  Copy full SHA for 07d3a19

• Add examples

  Liraim

  committed
  368724f

  Copy full SHA for 368724f

• Add examples

  Liraim

  committed
  cf38a34

  Copy full SHA for cf38a34

Commits on Dec 6, 2024

• Add FeatureScorers.

  Liraim

  committed
  371aa0d

  Copy full SHA for 371aa0d

Commits on Dec 9, 2024

• Add metric container, group by and support for llm_judge in scorers.

  Liraim

  committed
  4953ef5

  Copy full SHA for 4953ef5

• Fix example.

  Liraim

  committed
  6ccffa6

  Copy full SHA for 6ccffa6

• Fix example.

  Liraim

  committed
  7644cb1

  Copy full SHA for 7644cb1

• Add custom scorers.

  Liraim

  committed
  ede9486

  Copy full SHA for ede9486

Commits on Dec 11, 2024

• Add first version of presets and report.

  Liraim

  committed
  c194b4b

  Copy full SHA for c194b4b

Commits on Dec 12, 2024

• Fix column name for FeatureScorer.

  Show description for c350d95
  Liraim

  committed
  c350d95

  Copy full SHA for c350d95

• Add implementations for column_max, column_mean, column_min and column_quantile.

  Liraim

  committed
  92bd2f5

  Copy full SHA for 92bd2f5

Commits on Dec 13, 2024

• Support MetricPreset and MetricContainer in Reports.

  Liraim

  committed
  e142196

  Copy full SHA for e142196

• codegen scorers (#1394)

  Show description for ccab7fc
  mike0sv

  authored
  ccab7fc

  Copy full SHA for ccab7fc

Commits on Dec 16, 2024

• Add separate render for metrics and checks.

  Liraim

  committed
  e35ef8e

  Copy full SHA for e35ef8e

Commits on Dec 17, 2024

• Merge branch 'refs/heads/main' into feature/metrics-redesign

  Liraim

  committed
  d451be5

  Copy full SHA for d451be5

• rename scoreres->descriptors, add showcase.ipynb and add sql descriptor

  mike0sv

  committed
  6a52347

  Copy full SHA for 6a52347

Commits on Dec 18, 2024

• Collect checks in single list.

  Show description for a61f7af
  Liraim

  committed
  a61f7af

  Copy full SHA for a61f7af

Commits on Dec 20, 2024

• Add example F1 metric (from legacy metrics) in new reports.

  Liraim

  committed
  cca2ad3

  Copy full SHA for cca2ad3

Commits on Dec 22, 2024

• Add metrics ROC AUC, f1, precision, recall.

  Liraim

  committed
  863f12a

  Copy full SHA for 863f12a

Commits on Dec 23, 2024

• Add same render as in old metrics to RocAucMetric.

  Liraim

  committed
  e8556db

  Copy full SHA for e8556db

• Rename checks to tests and reorganize code.

  Liraim

  committed
  51af315

  Copy full SHA for 51af315

• Bring back column_summary metrics.

  Liraim

  committed
  a4903b1

  Copy full SHA for a4903b1

Commits on Dec 24, 2024

• Add quality by class preset.

  Liraim

  committed
  5b765b0

  Copy full SHA for 5b765b0

• Remove tabs title and rename Checks to Tests.

  Liraim

  committed
  cb3caa8

  Copy full SHA for cb3caa8

• support v2 metrics in v1 snapshots (#1398)

  Show description for f894998
  mike0sv

  authored
  f894998

  Copy full SHA for f894998

Commits on Dec 25, 2024

• add mylabelvalue

  mike0sv

  committed
  81029c4

  Copy full SHA for 81029c4

• fix ipython import

  mike0sv

  committed
  2ce0121

  Copy full SHA for 2ce0121

• fix

  mike0sv

  committed
  c831a51

  Copy full SHA for c831a51

• fix deduplication

  mike0sv

  committed
  a80e253

  Copy full SHA for a80e253

• Add support for ByLabel tests.

  Liraim

  committed
  30e1e79

  Copy full SHA for 30e1e79

• Update column statistics metrics: min, max, mean,median, quantile and std.

  Liraim

  committed
  7fca28e

  Copy full SHA for 7fca28e

• Fix group_by metrics and metric_workbench examples.

  Liraim

  committed
  71ebb54

  Copy full SHA for 71ebb54

Commits on Dec 26, 2024

• make all reports

  mike0sv

  committed
  b9e04fa

  Copy full SHA for b9e04fa

Commits on Dec 27, 2024

• Add distribution widgets to metrics.

  Liraim

  committed
  79341e9

  Copy full SHA for 79341e9

• Merge remote-tracking branch 'origin/feature/metrics-redesign' into feature/metrics-redesign

  mike0sv

  committed
  d5732b3

  Copy full SHA for d5732b3

• test widgets

  mike0sv

  committed
  4ce3ea8

  Copy full SHA for 4ce3ea8

• Add metrics InRange, OutRange, InList, OutList and fix some bugs.

  Liraim

  committed
  1368ba1

  Copy full SHA for 1368ba1

• add count value support

  mike0sv

  committed
  5c123bc

  Copy full SHA for 5c123bc

Commits on Dec 30, 2024

• Move from evidently.v2 to evidently.future.

  Liraim

  committed
  f7c5e19

  Copy full SHA for f7c5e19

• Rewrite tests.

  Liraim

  committed
  1312759

  Copy full SHA for 1312759

• Update examples.

  Liraim

  committed
  788df14

  Copy full SHA for 788df14

• Add dataset statistics metrics: RowCount, ColumnCount.

  Liraim

  committed
  4853eb1

  Copy full SHA for 4853eb1

• Rename v2 to future fixes.

  Liraim

  committed
  c5f07d6

  Copy full SHA for c5f07d6

• Add example of dashboard with new metrics.

  Liraim

  committed
  03ead97

  Copy full SHA for 03ead97

Commits on Dec 31, 2024

• Add dataset stats calculations on creation.

  Liraim

  committed
  efe5fc6

  Copy full SHA for efe5fc6

• Add value drift first version.

  Liraim

  committed
  86f316d

  Copy full SHA for 86f316d

Commits on Jan 1, 2025

• Add ValueStats and DatasetStats.

  Liraim

  committed
  c6c32bf

  Copy full SHA for c6c32bf

• Support for dict / json for snapshots.

  Liraim

  committed
  2ae1833

  Copy full SHA for 2ae1833

Commits on Jan 2, 2025

• Support for legacy metrics.

  Show description for 8b2bda9
  Liraim

  committed
  8b2bda9

  Copy full SHA for 8b2bda9

• Add empty rows and columns, constant columns metrics.

  Liraim

  committed
  bc3f164

  Copy full SHA for bc3f164

• Add MissingValuesCount metrics.

  Liraim

  committed
  ada2b2b

  Copy full SHA for ada2b2b

• update descriptors

  mike0sv

  committed
  90ea710

  Copy full SHA for 90ea710

• fix init

  mike0sv

  committed
  6355f18

  Copy full SHA for 6355f18

Commits on Jan 3, 2025

• fix descriptors showcase

  mike0sv

  committed
  c2ce94d

  Copy full SHA for c2ce94d

• Add metric config (#1403)

  mike0sv

  authored
  442e1db

  Copy full SHA for 442e1db

Commits on Jan 4, 2025

• Remove LegacyMetric and make dataset stats metrics simple single value metrics.

  Liraim

  committed
  197d092

  Copy full SHA for 197d092

• Update Classification metrics F1 by label, Precision by label, Recall by label, RocAuc by label and Classification Quality by Class preset.

  Liraim

  committed
  9511471

  Copy full SHA for 9511471

• Reorganize code with base types.

  Liraim

  committed
  c7ec559

  Copy full SHA for c7ec559

• Add classification quality metrics: F1Score, Accuracy, Precision, Recall, TPR, TNR, FPR, FNR.

  Liraim

  committed
  3c3b4d9

  Copy full SHA for 3c3b4d9

• Add better example for custom descriptors.

  Liraim

  committed
  f8e0742

  Copy full SHA for f8e0742

Commits on Jan 5, 2025

• First version of new DataDefinition.

  Liraim

  committed
  dd9e401

  Copy full SHA for dd9e401

Commits on Jan 6, 2025

• Add ValueDrift example for text column.

  Show description for 1f793e4
  Liraim

  committed
  1f793e4

  Copy full SHA for 1f793e4

• Remove test results from metric config to calculations.

  Liraim

  committed
  386d2c1

  Copy full SHA for 386d2c1

• Fix errors after moving get_tests to calculations.

  Liraim

  committed
  8135b9d

  Copy full SHA for 8135b9d

• renew registry

  mike0sv

  committed
  c08c122

  Copy full SHA for c08c122

• ensure registry import

  mike0sv

  committed
  294f93e

  Copy full SHA for 294f93e

• Add RocAuc metric.

  Liraim

  committed
  b2f9217

  Copy full SHA for b2f9217

• Add LogLoss metric.

  Liraim

  committed
  1264cfe

  Copy full SHA for 1264cfe

• Merge branch 'main' into feature/metrics-redesign

  mike0sv

  committed
  ff31bc1

  Copy full SHA for ff31bc1

• Added a draft of a full list of descriptors, including non obvious ones

  emeli-dral

  committed
  06de356

  Copy full SHA for 06de356

• Add classification quality presets.

  Liraim

  committed
  7cd182e

  Copy full SHA for 7cd182e

Commits on Jan 7, 2025

• Support add_descriptors in PandasDataset.

  Liraim

  committed
  f24e2b5

  Copy full SHA for f24e2b5

• Add ClassificationDummyQuality.

  Liraim

  committed
  e89e4f9

  Copy full SHA for e89e4f9

• updated descriptor list example

  emeli-dral

  committed
  47250a0

  Copy full SHA for 47250a0

• Added custom callable examples

  emeli-dral

  committed
  70563c6

  Copy full SHA for 70563c6

• Add Dataset.add_descriptors() method.

  Liraim

  committed
  720ab08

  Copy full SHA for 720ab08

• Added more descriptor examples

  emeli-dral

  committed
  ff69ba1

  Copy full SHA for ff69ba1

• Added more descriptor examples

  emeli-dral

  committed
  373c691

  Copy full SHA for 373c691

• Added more descriptor examples

  emeli-dral

  committed
  32cd288

  Copy full SHA for 32cd288

• Fix pandas dataset creation without descriptors.

  Liraim

  committed
  5079133

  Copy full SHA for 5079133

Commits on Jan 8, 2025

• Add regression metrics.

  Show description for 7b9614b
  Liraim

  committed
  7b9614b

  Copy full SHA for 7b9614b

• add snapshot version

  mike0sv

  committed
  36c31ec

  Copy full SHA for 36c31ec

• Add metric_id property for dashboards.

  Show description for d19454a
  Liraim

  committed
  d19454a

  Copy full SHA for d19454a

• Fix typing in tests.

  Liraim

  committed
  9e64f27

  Copy full SHA for 9e64f27

• Update regression metrics.

  Liraim

  committed
  af193ba

  Copy full SHA for af193ba

• Add RegressionQuality preset.

  Liraim

  committed
  187bc40

  Copy full SHA for 187bc40

• Fix classification method and create column_mapping from data_definition.

  Liraim

  committed
  3cb83aa

  Copy full SHA for 3cb83aa

• fix DataDefinition and get_legacy_metric

  DimaAmega

  committed
  0601485

  Copy full SHA for 0601485

• add new bikes-v2demo project

  DimaAmega

  committed
  802cf50

  Copy full SHA for 802cf50

• bit refactor: list(DEMO_PROJECTS.keys()) -> DEMO_PROJECTS_NAMES

  DimaAmega

  committed
  a349d2b

  Copy full SHA for a349d2b

• Added more descriptor examples

  emeli-dral

  committed
  3189851

  Copy full SHA for 3189851

• updated more descriptor example

  emeli-dral

  committed
  c64f32e

  Copy full SHA for c64f32e

• fix CategoryCountCalculation calculation

  emeli-dral

  committed
  60bc16a

  Copy full SHA for 60bc16a

• add llm descriptors

  mike0sv

  committed
  b50eceb

  Copy full SHA for b50eceb

• add llm descriptors

  mike0sv

  committed
  dbba5fc

  Copy full SHA for dbba5fc

• add snapshot version into @get("/{project_id:uuid}/{snapshot_id:uuid}/data") handler and fix bikes-v2 demo project

  DimaAmega

  committed
  1ceedaf

  Copy full SHA for 1ceedaf

• Feature/metrics redesign test configs (#1409)

  mike0sv

  authored
  07298ce

  Copy full SHA for 07298ce

• Fix tests method.

  Liraim

  committed
  ab370d3

  Copy full SHA for ab370d3

• add aliases to registry

  mike0sv

  committed
  aa890b7

  Copy full SHA for aa890b7

• added reviews demo project, generated ober future reports

  emeli-dral

  committed
  751785d

  Copy full SHA for 751785d

• Add Regression dummy

  Liraim

  committed
  279f96b

  Copy full SHA for 279f96b

• Add DataDrift preset and DriftedColumnsCount.

  Liraim

  committed
  25501a3

  Copy full SHA for 25501a3

• Add TextEvals.

  Liraim

  committed
  b09210b

  Copy full SHA for b09210b

Commits on Jan 9, 2025

• support metrics and tests tabs

  DimaAmega

  committed
  38d8f31

  Copy full SHA for 38d8f31

• commit UI assets

  DimaAmega

  committed
  64a07d1

  Copy full SHA for 64a07d1

• use metric v2 fingerprint in adapter

  mike0sv

  committed
  075962b

  Copy full SHA for 075962b

• Merge remote-tracking branch 'origin/feature/metrics-redesign' into feature/metrics-redesign

  mike0sv

  committed
  ca1d817

  Copy full SHA for ca1d817

• Make a copy of data_definition on Dataset creation.

  Liraim

  committed
  bcbffa2

  Copy full SHA for bcbffa2

• fix descriptors alias

  mike0sv

  committed
  d1b1c8e

  Copy full SHA for d1b1c8e

• Add data_definition property for datasets.

  Liraim

  committed
  097d2b2

  Copy full SHA for 097d2b2

• fix HF descriptors alias

  mike0sv

  committed
  4e1610b

  Copy full SHA for 4e1610b

• Merge remote-tracking branch 'origin/feature/metrics-redesign' into feature/metrics-redesign

  mike0sv

  committed
  c3b9302

  Copy full SHA for c3b9302

• Refactor TextEvals to include RowColumns only once.

  Liraim

  committed
  1600d13

  Copy full SHA for 1600d13

• Make a copy of data_definition on datset creation.

  Liraim

  committed
  de81bc3

  Copy full SHA for de81bc3

• Do not throw exception if column not in column list.

  Liraim

  committed
  d171938

  Copy full SHA for d171938

• Last attempt to copy data_definitions.

  Liraim

  committed
  0fddf73

  Copy full SHA for 0fddf73

• added examples with LLMEvals

  emeli-dral

  committed
  a766ade

  Copy full SHA for a766ade

• added llm judge descriptor example

  emeli-dral

  committed
  f81a41f

  Copy full SHA for f81a41f

• added llm judge descriptor example

  emeli-dral

  committed
  4ac928c

  Copy full SHA for 4ac928c

• added descriptor tutorial for users

  emeli-dral

  committed
  14f4487

  Copy full SHA for 14f4487

• added descriptor tutorial for users

  emeli-dral

  committed
  711142d

  Copy full SHA for 711142d

• add datadefinition backport

  mike0sv

  committed
  39fc32f

  Copy full SHA for 39fc32f

• Merge remote-tracking branch 'origin/feature/metrics-redesign' into feature/metrics-redesign

  mike0sv

  committed
  d9e9f1f

  Copy full SHA for d9e9f1f

• support snapshotv2 in ws

  mike0sv

  committed
  d3c2bf1

  Copy full SHA for d3c2bf1

Commits on Jan 10, 2025

• Add UniqueValueCount and new DistributionPanel.

  Liraim

  committed
  d84c97b

  Copy full SHA for d84c97b

• better SnapshotWidgets component

  DimaAmega

  committed
  c2a20d3

  Copy full SHA for c2a20d3

• Add target, prediction params in Regression.

  Liraim

  committed
  af3d668

  Copy full SHA for af3d668

• revert using version on UI

  DimaAmega

  committed
  d2e5ab0

  Copy full SHA for d2e5ab0

• add reviews-v2 demo project

  DimaAmega

  committed
  e1c865d

  Copy full SHA for e1c865d

• move version to metadata

  mike0sv

  committed
  df1d96e

  Copy full SHA for df1d96e

• Merge remote-tracking branch 'origin/feature/metrics-redesign' into feature/metrics-redesign

  Show description for a1a062d
  mike0sv

  committed
  a1a062d

  Copy full SHA for a1a062d

• move version to metadata

  mike0sv

  committed
  62880ff

  Copy full SHA for 62880ff

• Add save_html, save_json methods.

  Liraim

  committed
  22ef094

  Copy full SHA for 22ef094

• Fix group_by metric.

  Liraim

  committed
  1d6d1e5

  Copy full SHA for 1d6d1e5

• tmp names in data definition

  emeli-dral

  committed
  ee1530e

  Copy full SHA for ee1530e

• minor rename in data definition

  emeli-dral

  committed
  56027ba

  Copy full SHA for 56027ba

• fix test panels

  mike0sv

  committed
  48a2919

  Copy full SHA for 48a2919

• Merge remote-tracking branch 'origin/feature/metrics-redesign' into feature/metrics-redesign

  mike0sv

  committed
  4773f5a

  Copy full SHA for 4773f5a

• Fix mypy issues

  Liraim

  committed
  f2d9808

  Copy full SHA for f2d9808

• fix ui types

  mike0sv

  committed
  b37bfa7

  Copy full SHA for b37bfa7

• Fix mypy issues

  Liraim

  committed
  fc9bc06

  Copy full SHA for fc9bc06

• Merge remote-tracking branch 'origin/feature/metrics-redesign' into feature/metrics-redesign

  Show description for 23730bf
  mike0sv

  committed
  23730bf

  Copy full SHA for 23730bf

• Merge remote-tracking branch 'origin/feature/metrics-redesign' into feature/metrics-redesign

  mike0sv

  committed
  4d82521

  Copy full SHA for 4d82521

• fix mypy

  mike0sv

  committed
  e447c71

  Copy full SHA for e447c71

• comment new v2 demo projects

  DimaAmega

  committed
  4d46877

  Copy full SHA for 4d46877

• increase timeout

  DimaAmega

  committed
  dec649b

  Copy full SHA for dec649b

• Fix mypy issues

  Liraim

  committed
  03884a6

  Copy full SHA for 03884a6

• dvc: add new screenshots

  DimaAmega

  committed
  1fd7aa0

  Copy full SHA for 1fd7aa0

• Fix tests.

  Liraim

  committed
  629f9df

  Copy full SHA for 629f9df

• Fix tests.

  Liraim

  committed
  4beb9e2

  Copy full SHA for 4beb9e2