main conflicts fixes

README.md

@@ -87,14 +87,37 @@
 
 
 ## Примеры и тьюториалы
-Примеры использования API библиотеки:
-* [Визуализация](/examples/minimal/Visualization.ipynb)
-* [Гомологии](/examples/minimal/Homologies.ipynb)
-* [Байесианизация](/examples/minimal/Bayesianization.ipynb)
-
-Примеры использования библиотеки для индустриальных задач:
-* [CIFAR10 classification](/examples/CIFAR10)
-* [casting defect detection](/examples/casting)
+Мы предоставляем примеры разного уровня сложности:
+* [минимальные] минималистичные примеры, представляющие наш API
+* [базовые] применение eXNN для простых задач, таких как классификация MNIST
+* [сценарии использования] демонстрация использования eXplain-NN для решения различных проблем, возникающих в промышленных задачах.
+
+### Минимальные
+Этот колаб содержит минималистическую демонстрацию нашего API на фиктивных объектах:
+
+[![minimal](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1lOiB50LppDiiRHTv184JMuQ2IvZ4I4rp?usp=sharing)
+
+### Базовые
+Вот колабы, демонстрирующие, как работать с разными модулями нашего API на простых задачах:
+
+| Colab Link | Module |
+| ------------- | ------------- |
+| [![bayes](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1Ayd0IronxUIfnbAmWQLHiILG2qtBBpF4?usp=sharing)| bayes |
+| [![topology](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1T5ENfNaCIRI61LM2ZhtU8lfmvRmlfiEo?usp=sharing)| topology |
+| [![visualization](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1LJVdWTv-wcASSMX4is_E15TR7XJsT7W3?usp=sharing)| visualization |
+
+### Сценарии использования
+В этом блоке представлены примеры использования eXplain-NN для решения различных вариантов использования в промышленных задачах. Для демонстрационных целей используются 3 задачи:
+* [спутник] классификация ландшафтов по спутниковым снимкам.
+* [электроника] классификация электронных компонентов и устройств
+* [ЭКГ] диагностика ЭКГ
+
+| Colab Link | Task | Use Case |
+| ------------- | ------------- | ------------- |
+| [![CNN_viz](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/12ZJigH-0geGTefNXnCM5dQ71d4tqlf6L?usp=sharing)| спутник | Визуализация изменения многообразия данных от слоя к слою |
+| TBD | спутник | Детекция adversarial данных |
+| TBD | электроника | Оценка обобщающей способности нейронной сети |
+| TBD | ЭКГ | Визуализация изменения многообразия данных от слоя к слою |
 
 ## Как помочь проекту
 [Инструкции](/docs/contribution.md).

README_eng.md

@@ -87,14 +87,36 @@ eXplain-NNs Library API is available [here](https://med-ai-lab.github.io/eXplain
 
 
 ## Examples & Tutorials
-We provide tutorials demonstrating suggested usage pipeline:
-* [Visualization](/examples/minimal/Visualization.ipynb)
-* [Homologies](/examples/minimal/Homologies.ipynb)
-* [Bayesianization](/examples/minimal/Bayesianization.ipynb)
-
-We also provide examples of application of our library to different tasks:
-* [CIFAR10 classification](/examples/CIFAR10)
-* [casting defect detection](/examples/casting)
+We provides examples of different levels of complexity:
+* [minimal] minimalistic examples presenting our API
+* [basic] applying eXNN to simple tasks like MNIST classification
+* [use cases] demonstation of eXplain-NN usage for solving different use cases in industrial tasks
+
+### Minimal
+This colab contains minimalistic demonstration of our API on dummy objects:
+
+[![minimal](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1lOiB50LppDiiRHTv184JMuQ2IvZ4I4rp?usp=sharing)
+
+### Basic
+Here are colabs demonstrating how to work with different modules of our API on simple tasks:
+| Colab Link | Module |
+| ------------- | ------------- |
+| [![bayes](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1Ayd0IronxUIfnbAmWQLHiILG2qtBBpF4?usp=sharing)| bayes |
+| [![topology](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1T5ENfNaCIRI61LM2ZhtU8lfmvRmlfiEo?usp=sharing)| topology |
+| [![visualization](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1LJVdWTv-wcASSMX4is_E15TR7XJsT7W3?usp=sharing)| visualization |
+
+### Use Cases
+This block provides examples how eXplain-NNs can be used to solve different use cases in industrial tasks. For demonstration purposed 3 tasks are used:
+* [satellite] landscape classification from satellite imagery
+* [electronics] electronic components and devices classification
+* [ECG] ECG diagnostics
+
+| Colab Link | Task | Use Case |
+| ------------- | ------------- | ------------- |
+| [![CNN_viz](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/12ZJigH-0geGTefNXnCM5dQ71d4tqlf6L?usp=sharing)| satellite | Visualization of data manifold evolution from layer to layer |
+| TBD | satellite | Detecting adversarial examples |
+| TBD | electronics | Estimating generalization of a NN |
+| TBD | ECG | Visualization of data manifold evolution from layer to layer |
 
 ## Contribution Guide
 The contribution guide is available in the [repository](/docs/contribution.md).

eXNN/topology/__init__.py

@@ -1 +1 @@
-from .api import get_data_barcode, get_nn_barcodes, plot_barcode
+from .api import evaluate_barcode, get_data_barcode, get_nn_barcodes, plot_barcode

eXNN/topology/api.py

@@ -1,10 +1,10 @@
-from typing import Dict, List
+from typing import Dict, List, Optional, Union
 
 import matplotlib
 import numpy as np
 import torch
 
-from eXNN.topology import homologies
+from eXNN.topology import homologies, metrics
 
 
 def get_data_barcode(
@@ -68,3 +68,21 @@ def plot_barcode(barcode: Dict[str, np.ndarray]) -> matplotlib.figure.Figure:
         matplotlib.figure.Figure: a plot of the barcode
     """
     return homologies.plot_barcode(barcode)
+
+
+def evaluate_barcode(
+    barcode: Dict[str, np.ndarray], metric_name: Optional[str] = None
+) -> Union[float, Dict[str, float]]:
+    """
+    The function evaluates a persistent homologies barcode with a metric.
+
+    Args:
+        barcode (Dict[str, np.ndarray]): barcode
+        metric_name (Optional[str]): metric name
+            (if `None` all available metrics values are computed)
+
+    Returns:
+        Union(float, Dict[str, float]): float if metric is specified
+            or a dictionary with value of each available metric
+    """
+    return metrics.compute_metric(barcode, metric_name)

eXNN/topology/metrics.py

@@ -0,0 +1,146 @@
+import heapq
+
+import numpy as np
+
+
+def _get_available_metrics():
+    return {
+        # absolute length based metrics
+        "max_length": _compute_longest_interval_metric,
+        "mean_length": _compute_length_mean_metric,
+        "median_length": _compute_length_median_metric,
+        "stdev_length": _compute_length_stdev_metric,
+        "sum_length": _compute_length_sum_metric,
+        # relative length based metrics
+        "ratio_2_1": _compute_two_to_one_ratio_metric,
+        "ratio_3_1": _compute_three_to_one_ratio_metric,
+        # entopy based metrics
+        "h": _compute_entropy_metric,
+        "normh": _compute_normed_entropy_metric,
+        # signal to noise ration
+        "snr": _compute_snr_metric,
+        # birth-death based metrics
+        "mean_birth": _compute_births_mean_metric,
+        "stdev_birth": _compute_births_stdev_metric,
+        "mean_death": _compute_deaths_mean_metric,
+        "stdev_death": _compute_deaths_stdev_metric,
+    }
+
+
+def compute_metric(barcode, metric_name=None):
+    metrics = _get_available_metrics()
+    if metric_name is None:
+        return {name: fn(barcode) for (name, fn) in metrics.items()}
+    else:
+        return metrics[metric_name](barcode)
+
+
+def _get_lengths(barcode):
+    diag = barcode["H0"]
+    return [d[1] - d[0] for d in diag]
+
+
+def _compute_longest_interval_metric(barcode):
+    lengths = _get_lengths(barcode)
+    return np.max(lengths).item()
+
+
+def _compute_length_mean_metric(barcode):
+    lengths = _get_lengths(barcode)
+    return np.mean(lengths).item()
+
+
+def _compute_length_median_metric(barcode):
+    lengths = _get_lengths(barcode)
+    return np.median(lengths).item()
+
+
+def _compute_length_stdev_metric(barcode):
+    lengths = _get_lengths(barcode)
+    return np.std(lengths).item()
+
+
+def _compute_length_sum_metric(barcode):
+    lengths = _get_lengths(barcode)
+    return np.sum(lengths).item()
+
+
+# Proportion between the longest intervals: 2/1 ratio, 3/1 ratio
+def _compute_two_to_one_ratio_metric(barcode):
+    lengths = _get_lengths(barcode)
+    value = heapq.nlargest(2, lengths)[1] / lengths[0]
+    return value.item()
+
+
+def _compute_three_to_one_ratio_metric(barcode):
+    lengths = _get_lengths(barcode)
+    value = heapq.nlargest(3, lengths)[2] / lengths[0]
+    return value.item()
+
+
+# Compute the persistent entropy and normed persistent entropy
+def _get_entropy(values, normalize: bool):
+    values_sum = np.sum(values)
+    entropy = (-1) * np.sum(np.divide(values, values_sum) * np.log(np.divide(values, values_sum)))
+    if normalize:
+        entropy = entropy / np.log(values_sum)
+    return entropy
+
+
+def _compute_entropy_metric(barcode):
+    return _get_entropy(_get_lengths(barcode), normalize=False).item()
+
+
+def _compute_normed_entropy_metric(barcode):
+    return _get_entropy(_get_lengths(barcode), normalize=True).item()
+
+
+# Compute births
+def _get_births(barcode):
+    diag = barcode["H0"]
+    return np.array([x[0] for x in diag])
+
+
+# Comput deaths
+def _get_deaths(barcode):
+    diag = barcode["H0"]
+    return np.array([x[1] for x in diag])
+
+
+# def _get_birth(barcode, dim):
+#     diag = barcode['H0']
+#     temp = np.array([x[0] for x in diag if x[2] == dim])
+#     return temp[0]
+
+
+# def _get_death(barcode, dim):
+#     diag = barcode['H0']
+#     temp = np.array([x[1] for x in diag if x[2] == dim])
+#     return temp[-1]
+
+
+# Compute SNR
+def _compute_snr_metric(barcode):
+    births = _get_births(barcode)
+    deaths = _get_deaths(barcode)
+    signal = np.mean(deaths - births)
+    noise = np.std(births)
+    snr = signal / noise
+    return snr.item()
+
+
+# Compute the birth-death pair indices: Birth mean, birth stdev, death mean, death stdev
+def _compute_births_mean_metric(barcode):
+    return np.mean(_get_births(barcode)).item()
+
+
+def _compute_births_stdev_metric(barcode):
+    return np.std(_get_births(barcode)).item()
+
+
+def _compute_deaths_mean_metric(barcode):
+    return np.mean(_get_deaths(barcode)).item()
+
+
+def _compute_deaths_stdev_metric(barcode):
+    return np.std(_get_deaths(barcode)).item()

eXNN/visualization/__init__.py

@@ -1 +1,6 @@
-from .api import get_random_input, reduce_dim, visualize_layer_manifolds
+from .api import (
+    get_random_input,
+    reduce_dim,
+    visualize_layer_manifolds,
+    visualize_recurrent_layer_manifolds,
+)