From a2d0ed3337203a1b56d320e1afd35a3ab4fa3a76 Mon Sep 17 00:00:00 2001 From: "Documenter.jl" Date: Fri, 26 Jan 2024 21:03:22 +0000 Subject: [PATCH] build based on 0fb4bd3 --- dev/.documenter-siteinfo.json | 2 +- dev/api/index.html | 10 +++++----- dev/generated/augmentations.ipynb | 24 ++++++++++++------------ dev/generated/augmentations/index.html | 10 +++++----- dev/generated/example/index.html | 2 +- dev/generated/heatmapping/index.html | 2 +- dev/index.html | 2 +- dev/search_index.js | 2 +- 8 files changed, 27 insertions(+), 27 deletions(-) diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index 484fb57..0bd85dc 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-01-26T20:26:44","documenter_version":"1.2.1"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-01-26T21:03:17","documenter_version":"1.2.1"}} \ No newline at end of file diff --git a/dev/api/index.html b/dev/api/index.html index 3c947e3..9bb5a82 100644 --- a/dev/api/index.html +++ b/dev/api/index.html @@ -1,6 +1,6 @@ -API Reference · ExplainableAI.jl
GitHub

Basic API

All methods in ExplainableAI.jl work by calling analyze on an input and an analyzer:

XAIBase.analyze — Function
analyze(input, method)
-analyze(input, method, neuron_selection)

Apply the analyzer method for the given input, returning an Explanation. If neuron_selection is specified, the explanation will be calculated for that neuron. Otherwise, the output neuron with the highest activation is automatically chosen.

See also Explanation and heatmap.

Keyword arguments

  • add_batch_dim: add batch dimension to the input without allocating. Default is false.
source
XAIBase.Explanation — Type

Return type of analyzers when calling analyze.

Fields

  • val: numerical output of the analyzer, e.g. an attribution or gradient
  • output: model output for the given analyzer input
  • output_selection: index of the output used for the explanation
  • analyzer: symbol corresponding the used analyzer, e.g. :Gradient or :LRP
  • heatmap: symbol indicating a preset heatmapping style, e.g. :attibution, :sensitivity or :cam
  • extras: optional named tuple that can be used by analyzers to return additional information.
source
XAIBase.heatmap — Function
heatmap(explanation)

Visualize Explanation from XAIBase as a vision heatmap. Assumes WHCN convention (width, height, channels, batchsize) for explanation.val.

Keyword arguments

  • colorscheme::Union{ColorScheme,Symbol}: color scheme from ColorSchemes.jl. Defaults to :seismic.
  • reduce::Symbol: Selects how color channels are reduced to a single number to apply a color scheme. The following methods can be selected, which are then applied over the color channels for each "pixel" in the array:
    • :sum: sum up color channels
    • :norm: compute 2-norm over the color channels
    • :maxabs: compute maximum(abs, x) over the color channels
    Defaults to :sum.
  • rangescale::Symbol: Selects how the color channel reduced heatmap is normalized before the color scheme is applied. Can be either :extrema or :centered. Defaults to :centered.
  • process_batch::Bool: When heatmapping a batch, setting process_batch=true will apply the rangescale normalization to the entire batch instead of computing it individually for each sample in the batch. Defaults to false.
  • permute::Bool: Whether to flip W&H input channels. Default is true.
  • unpack_singleton::Bool: If false, heatmap will always return a vector of images. When heatmapping a batch with a single sample, setting unpack_singleton=true will unpack the singleton vector and directly return the image. Defaults to true.
source
heatmap(input, analyzer)

Compute an Explanation for a given input using the method analyzer and visualize it as a vision heatmap.

Any additional arguments and keyword arguments are passed to the analyzer. Refer to analyze for more information on available keyword arguments.

To customize the heatmapping style, first compute an explanation using analyze and then call heatmap on the explanation.

source
heatmap(explanation, text)

Visualize Explanation from XAIBase as text heatmap. Text should be a vector containing vectors of strings, one for each input in the batched explanation.

Keyword arguments

  • colorscheme::Union{ColorScheme,Symbol}: color scheme from ColorSchemes.jl. Defaults to :seismic.
  • rangescale::Symbol: selects how the color channel reduced heatmap is normalized before the color scheme is applied. Can be either :extrema or :centered. Defaults to :centered for use with the default color scheme :seismic.
source

Analyzers

ExplainableAI.Gradient — Type
Gradient(model)

Analyze model by calculating the gradient of a neuron activation with respect to the input.

source
ExplainableAI.InputTimesGradient — Type
InputTimesGradient(model)

Analyze model by calculating the gradient of a neuron activation with respect to the input. This gradient is then multiplied element-wise with the input.

source
ExplainableAI.SmoothGrad — Function
SmoothGrad(analyzer, [n=50, std=0.1, rng=GLOBAL_RNG])
-SmoothGrad(analyzer, [n=50, distribution=Normal(0, σ²=0.01), rng=GLOBAL_RNG])

Analyze model by calculating a smoothed sensitivity map. This is done by averaging sensitivity maps of a Gradient analyzer over random samples in a neighborhood of the input, typically by adding Gaussian noise with mean 0.

References

  • Smilkov et al., SmoothGrad: removing noise by adding noise
source
ExplainableAI.IntegratedGradients — Function
IntegratedGradients(analyzer, [n=50])
-IntegratedGradients(analyzer, [n=50])

Analyze model by using the Integrated Gradients method.

References

  • Sundararajan et al., Axiomatic Attribution for Deep Networks
source

Input augmentations

SmoothGrad and IntegratedGradients are special cases of the input augmentations NoiseAugmentation and InterpolationAugmentation, which can be applied as a wrapper to any analyzer:

ExplainableAI.NoiseAugmentation — Type
NoiseAugmentation(analyzer, n, [std=1, rng=GLOBAL_RNG])
-NoiseAugmentation(analyzer, n, distribution, [rng=GLOBAL_RNG])

A wrapper around analyzers that augments the input with n samples of additive noise sampled from distribution. This input augmentation is then averaged to return an Explanation.

source
ExplainableAI.InterpolationAugmentation — Type
InterpolationAugmentation(model, [n=50])

A wrapper around analyzers that augments the input with n steps of linear interpolation between the input and a reference input (typically zero(input)). The gradients w.r.t. this augmented input are then averaged and multiplied with the difference between the input and the reference input.

source

Input preprocessing

ExplainableAI.preprocess_imagenet — Function
preprocess_imagenet(img)

Preprocess an image for use with Metalhead.jl's ImageNet models using PyTorch weights. Uses PyTorch's normalization constants.

source

Index

+API Reference · ExplainableAI.jl
GitHub

Basic API

All methods in ExplainableAI.jl work by calling analyze on an input and an analyzer:

XAIBase.analyze — Function
analyze(input, method)
+analyze(input, method, neuron_selection)

Apply the analyzer method for the given input, returning an Explanation. If neuron_selection is specified, the explanation will be calculated for that neuron. Otherwise, the output neuron with the highest activation is automatically chosen.

See also Explanation and heatmap.

Keyword arguments

  • add_batch_dim: add batch dimension to the input without allocating. Default is false.
source
XAIBase.Explanation — Type

Return type of analyzers when calling analyze.

Fields

  • val: numerical output of the analyzer, e.g. an attribution or gradient
  • output: model output for the given analyzer input
  • output_selection: index of the output used for the explanation
  • analyzer: symbol corresponding the used analyzer, e.g. :Gradient or :LRP
  • heatmap: symbol indicating a preset heatmapping style, e.g. :attibution, :sensitivity or :cam
  • extras: optional named tuple that can be used by analyzers to return additional information.
source
XAIBase.heatmap — Function
heatmap(explanation)

Visualize Explanation from XAIBase as a vision heatmap. Assumes WHCN convention (width, height, channels, batchsize) for explanation.val.

Keyword arguments

  • colorscheme::Union{ColorScheme,Symbol}: color scheme from ColorSchemes.jl. Defaults to :seismic.
  • reduce::Symbol: Selects how color channels are reduced to a single number to apply a color scheme. The following methods can be selected, which are then applied over the color channels for each "pixel" in the array:
    • :sum: sum up color channels
    • :norm: compute 2-norm over the color channels
    • :maxabs: compute maximum(abs, x) over the color channels
    Defaults to :sum.
  • rangescale::Symbol: Selects how the color channel reduced heatmap is normalized before the color scheme is applied. Can be either :extrema or :centered. Defaults to :centered.
  • process_batch::Bool: When heatmapping a batch, setting process_batch=true will apply the rangescale normalization to the entire batch instead of computing it individually for each sample in the batch. Defaults to false.
  • permute::Bool: Whether to flip W&H input channels. Default is true.
  • unpack_singleton::Bool: If false, heatmap will always return a vector of images. When heatmapping a batch with a single sample, setting unpack_singleton=true will unpack the singleton vector and directly return the image. Defaults to true.
source
heatmap(input, analyzer)

Compute an Explanation for a given input using the method analyzer and visualize it as a vision heatmap.

Any additional arguments and keyword arguments are passed to the analyzer. Refer to analyze for more information on available keyword arguments.

To customize the heatmapping style, first compute an explanation using analyze and then call heatmap on the explanation.

source
heatmap(explanation, text)

Visualize Explanation from XAIBase as text heatmap. Text should be a vector containing vectors of strings, one for each input in the batched explanation.

Keyword arguments

  • colorscheme::Union{ColorScheme,Symbol}: color scheme from ColorSchemes.jl. Defaults to :seismic.
  • rangescale::Symbol: selects how the color channel reduced heatmap is normalized before the color scheme is applied. Can be either :extrema or :centered. Defaults to :centered for use with the default color scheme :seismic.
source

Analyzers

ExplainableAI.Gradient — Type
Gradient(model)

Analyze model by calculating the gradient of a neuron activation with respect to the input.

source
ExplainableAI.InputTimesGradient — Type
InputTimesGradient(model)

Analyze model by calculating the gradient of a neuron activation with respect to the input. This gradient is then multiplied element-wise with the input.

source
ExplainableAI.SmoothGrad — Function
SmoothGrad(analyzer, [n=50, std=0.1, rng=GLOBAL_RNG])
+SmoothGrad(analyzer, [n=50, distribution=Normal(0, σ²=0.01), rng=GLOBAL_RNG])

Analyze model by calculating a smoothed sensitivity map. This is done by averaging sensitivity maps of a Gradient analyzer over random samples in a neighborhood of the input, typically by adding Gaussian noise with mean 0.

References

  • Smilkov et al., SmoothGrad: removing noise by adding noise
source
ExplainableAI.IntegratedGradients — Function
IntegratedGradients(analyzer, [n=50])
+IntegratedGradients(analyzer, [n=50])

Analyze model by using the Integrated Gradients method.

References

  • Sundararajan et al., Axiomatic Attribution for Deep Networks
source

Input augmentations

SmoothGrad and IntegratedGradients are special cases of the input augmentations NoiseAugmentation and InterpolationAugmentation, which can be applied as a wrapper to any analyzer:

ExplainableAI.NoiseAugmentation — Type
NoiseAugmentation(analyzer, n, [std=1, rng=GLOBAL_RNG])
+NoiseAugmentation(analyzer, n, distribution, [rng=GLOBAL_RNG])

A wrapper around analyzers that augments the input with n samples of additive noise sampled from distribution. This input augmentation is then averaged to return an Explanation.

source
ExplainableAI.InterpolationAugmentation — Type
InterpolationAugmentation(model, [n=50])

A wrapper around analyzers that augments the input with n steps of linear interpolation between the input and a reference input (typically zero(input)). The gradients w.r.t. this augmented input are then averaged and multiplied with the difference between the input and the reference input.

source

Index

diff --git a/dev/generated/augmentations.ipynb b/dev/generated/augmentations.ipynb index 9c3cf92..b11794d 100644 --- a/dev/generated/augmentations.ipynb +++ b/dev/generated/augmentations.ipynb @@ -113,10 +113,10 @@ { "output_type": "execute_result", "data": { - "text/plain": "28×28 Array{RGB{Float64},2} with eltype ColorTypes.RGB{Float64}:\n RGB{Float64}(0.447882,0.445999,0.443883) … RGB{Float64}(0.449416,0.447535,0.44541)\n RGB{Float64}(0.449702,0.447822,0.445695) RGB{Float64}(0.449819,0.447939,0.445811)\n RGB{Float64}(0.453981,0.452106,0.449955) RGB{Float64}(0.447928,0.446045,0.443929)\n RGB{Float64}(0.449071,0.447189,0.445066) RGB{Float64}(0.447111,0.445227,0.443115)\n RGB{Float64}(0.441472,0.439579,0.4375) RGB{Float64}(0.44992,0.44804,0.445912)\n RGB{Float64}(0.43803,0.436132,0.434073) … RGB{Float64}(0.452557,0.450681,0.448538)\n RGB{Float64}(0.441404,0.439511,0.437433) RGB{Float64}(0.454033,0.452159,0.450007)\n RGB{Float64}(0.445346,0.443458,0.441357) RGB{Float64}(0.447119,0.445235,0.443123)\n RGB{Float64}(0.445333,0.443446,0.441345) RGB{Float64}(0.44025,0.438356,0.436284)\n RGB{Float64}(0.450603,0.448724,0.446592) RGB{Float64}(0.440059,0.438164,0.436093)\n ⋮ ⋱ \n RGB{Float64}(0.44501,0.443123,0.441023) RGB{Float64}(0.45547,0.453598,0.451438)\n RGB{Float64}(0.45497,0.453097,0.45094) … RGB{Float64}(0.444714,0.442826,0.440728)\n RGB{Float64}(0.449579,0.447699,0.445573) RGB{Float64}(0.444387,0.442499,0.440403)\n RGB{Float64}(0.442706,0.440815,0.438729) RGB{Float64}(0.444986,0.443098,0.440999)\n RGB{Float64}(0.44027,0.438375,0.436303) RGB{Float64}(0.451557,0.449679,0.447542)\n RGB{Float64}(0.444524,0.442636,0.440539) RGB{Float64}(0.454007,0.452133,0.449981)\n RGB{Float64}(0.444923,0.443035,0.440936) … RGB{Float64}(0.451695,0.449818,0.44768)\n RGB{Float64}(0.448136,0.446253,0.444136) RGB{Float64}(0.448775,0.446893,0.444772)\n RGB{Float64}(0.450134,0.448254,0.446125) RGB{Float64}(0.449176,0.447295,0.445172)", - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAIAAABJgmMcAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAACiJJREFUeAHtwc2SJNV5x+Hfe87JrKqumv6YoWFa7hGgkEPGsACHfBXWzVjXJl8BgbU1HxtpYIHExMB80VVdX5l58rxe/XNYVASbXObzpD//+b8ZOG8Zv84ZuDtSSkEcR4IFJKWEWDDEMKSUgrg7pziOGMYpZoY4zsAZOI4YxsAYGG+5c1JiMqrEZFSJyaiSYQwCA8MYGAPDGBi/KlhAYoyIBWPgDHKfEXNDSilI8YJ4cU6xYEgIAQkWEDdHggXEMAbGwN0ZuHNKYjKqxGRUicmokpkhZsZJxsDdGTiDGCJiZkiMkbcc6boOcXfE3ZG+75HSF6R4QdwdCRYQc0PcHQkpIDFGxDDEcaTPPVK8II5zSmIyqsRkVInJqJJ7QfriSCmFUwxDQghIigkxMyTnjDRNgxyORyTFyCmLxQJZzBdI13VIlzukazuk73ukrmvEMMTdkRgj0jQNkvvMwBmYGQNjkJiMKjEZVWIyqlSKIzlnpO97xMyQEANSVRVSVRWS+4w0bYPsdjvk2ByR0hckpYQ0TcPAGOQuIzFGJOeMvPvuu8hiMUeqqkLWmw1yPByRUgpiZkgIAbFgDJxBYjKqxGRUicmokrtzigVDDENSjIgFQ/q+R7q245S6rpH5Yo7s93skhIiUvkd22x1y++QWMQypqgqpZzWyXq+RYAHJOSOz2QwppSAhBqSUgszqGXJsjkhiMqrEZFSJyaiSmSExRqQKFWLBkJQSEiwgfemR3Gek6zok54zkY0ZKKUjbtEgpBVmtVsjFxQWyWq2Qtm2R4+GIrNdrJOeMLM+WDIyBYQwyg6qqkJwzYhiSmIwqMRlVYjKqZGZICAEJMSDujqxWKySGiMznc2Rzv0EOhwPSNi2yP+yRqqqQuq6RruuQGCJyeXmJXF1dIfvdDvnn9gckxYTsd3tke79F5vM58vDRQyTnjPR9j8znc6R4QRKTUSUmo0pMRpX4BccRw5AYI1JVFeLFkfVmjeQuI/f390hzbJB33nkHSVVC+twjq9UKqVKF7LY7JKWENMcjYmbI1cMr5Ng0SEoJ6boOaZsW6fseSTEhx+aIxBiRxGRUicmoEpNRJTNDQgiI48hut0O6rkMcR5bLJfL45jHy3uP3kLu7O+TVy1dIs26Q1WqFnC3PkKZpkNWDFdI0DdLljPzuw98hP734CdlsNshsNkN22x1Sz2rEMKRpG2QxX3BKYjKqxGRUicmoUowBiTEh7gVZLBbIdrdFFosFUvqCvH79GtlsNsjd3R3Sti3SHBukrmrkzZs3yMXFBQN3pG07ZH/YIz/88wcGxuD2X26Rqq6QruuQ/X6PdF2H1LOaU0opSGIyqsRkVInJqFJxRzxnpC894sWR5dkSWS6XyOF4QJInZLfbIU+fPkW6tkOePHmC/PjTT8ijhw+RGCNyd3eH3N/fI+v1Gtnt9sjN48fIfD5HHqweIJv7DTKfzZCmL0jOGalSxSmJyagSk1ElJqNKfd9zkjOYzWeckvuM7Pd75OXLl0hVVcjt7S3y9Om3yMXlBbLf7ZEff/wR+eCDD5CnT79Fzh88QD788EPk/PwcKX1Bvv/+e2Sz3iCr1QrJuUdCCEhVVUjTNEhd1UhiMqrEZFSJyahSKQVxd045Ho+IYUipCnJ1eYXc3t4il5eXyMcff4x89dVXyEcffYRcX18j//OXvyC///2/IsvVEnn//feRruuQn9/8jGwPW+TB+QPkzZs3yHa3RVJMSF96ZLlcInVdIzEGJDEZVWIyqsRkVMnMGDiDEAJiwZCqqpBZPUO2uy3y6NFD5MntLfLZp58i//HZZ8jibIFstzvk8eMb5I//+UfkxcsXyGKx4JR6ViPzbs4pq9UKWd+tES+OzOYz5GxxhoQQkOKOJCajSkxGlZiMKgULiCVD6rpmYAzquka6tkMuLy+Rn+/ukJ9evED+78svke+++w757NNPkbbrkEePHiJff/018tsnT5C//f3vSIoJqasKiTEiXhzp+x45Pz9HqrpC6rpGQgxICAHJOSOJyagSk1ElJqNKFgxJKSF1XSOlFGQ+nyP3m3sk95lTvvnmG+RssUB22x2y3myQD95/H/n888+R5dkS+cO//QF59PAhcmwa5PWr18j2foscDgckpcTAGZydnSG77Q4xM6SqK8TdkcRkVInJqBKTUSWck5qmQXLOiLsjTdsg+8MeOR6OyHwxR/73r39FNusN8sUXX3DK9bvXyM3NDfKP7/+BlFKQZ8+eIYf9AXn95jXy6uUrZDabIY8ePUJ22x2SqoSUUhB3R0IISGIyqsRkVInJqBK/kHPmFDNDDscDcn9/j7x6+Qrp+x558eIFst1tkZcvXyJ3d2vk3z/6iIEZ8qc//Rfy7dNvkdl8huz3e+SwPyDzxRx57/F7SIoJOVueccqsniG5zwyckxKTUSUmo0pMRpWKFyQQkL70SN/3SO4yYmbIzW9ukOPhiKxWK2S33yFeHLl+5xo5Pz9HHj58iHz15VdI27XI+fk5EkNEYopIXdXI5eUlMpvNkN1+h8QQkZQS4jiSc0bcHUlMRpWYjCoxGVWKMSKGIcULp1gwpG07pK5qTgkhINfvXCPr9QZ5sFohjiMWAnJ5fo6sNxukbVqkbVvk8vISubi8QObzOQNjsDxbIjlnxHGk9AXp+x5xdyQxGVViMqrEZFTJiyO5ZOR4OCJt1yIxJiTGiPR9jxybBmmaBokxItfX1wzckefPnyOXl5fIg5sbZLFYIGaGuDsSU0TatkXqqkYsGOLuvOVI13VIX3rEizMwBonJqBKTUSUmo0qlFKTve6R4Qbw4UqxH7u/vkdL3SN/3iJkhN7+5QbbbLeLFGRiDWT1Drq6ukPVmjaSYkKurKwbG4OLigoEzaNsW2R/2SNt1SNd2iAXj1yQmo0pMRpWYjCo5jhiGxBCRMAtIVVfIo0eO5C4j5xfnyOPHj5H5fI5s1hvk2bNnyCeffII8f/4cWZwtkP1hj5wtzpCcM2LBkO12i7RNi+Sckf1hj7g7kmJCUkiIReOUxGRUicmoEpNRJTNDQgxIbTWnxBSRFBNye3uLXFxcIO6OrJZLZH23RparJQNn8Nsnv0Xu7u6Q+WyO9KVH+tIjx+MRiSEibdciOWek9AWp6xqxyhALhoQQEC+OJCajSkxGlZiMKvELIQTEgnGKBUMePHiAVCkhu90Oubi4QPaHA9J1HRJCQHKfkf1+j8QQkb7vkbZrkWABKX1BSimImSFVqhhUDFJMSAyRk5yB40hiMqrEZFSJyaiSmSFmximGIWbGwBi0bYd0uUPatkW8OOLuSN/3SNd2SCkFsWBITBGZhRkSQ0TarkXcHYkhIiEGxMwQd2fgvOUMeu85JTEZVWIyqsRkVMkwBs5bxiAE4xQ3Bk3bIPv9HkkpIVVVccqsniGOIxYMyV1m4AwsGOLuSIoJKV6QYAExjIHzljNwnF9jGJKYjCoxGVViMqr/B6EqGNiXBH+0AAAAAElFTkSuQmCC", + "text/plain": "28×28 Array{RGB{Float64},2} with eltype ColorTypes.RGB{Float64}:\n RGB{Float64}(0.448294,0.446411,0.444292) … RGB{Float64}(0.450381,0.448501,0.446371)\n RGB{Float64}(0.45112,0.449242,0.447107) RGB{Float64}(0.450066,0.448186,0.446057)\n RGB{Float64}(0.455218,0.453346,0.451188) RGB{Float64}(0.448511,0.446628,0.444509)\n RGB{Float64}(0.448603,0.446721,0.444601) RGB{Float64}(0.447597,0.445713,0.443598)\n RGB{Float64}(0.439173,0.437277,0.435211) RGB{Float64}(0.450975,0.449096,0.446962)\n RGB{Float64}(0.438302,0.436405,0.434344) … RGB{Float64}(0.453376,0.451501,0.449353)\n RGB{Float64}(0.441507,0.439614,0.437535) RGB{Float64}(0.45547,0.453598,0.451438)\n RGB{Float64}(0.445287,0.4434,0.441299) RGB{Float64}(0.446933,0.445048,0.442937)\n RGB{Float64}(0.446011,0.444125,0.44202) RGB{Float64}(0.441522,0.43963,0.43755)\n RGB{Float64}(0.448543,0.446661,0.444541) RGB{Float64}(0.442114,0.440222,0.438139)\n ⋮ ⋱ \n RGB{Float64}(0.447101,0.445216,0.443105) RGB{Float64}(0.457581,0.455713,0.45354)\n RGB{Float64}(0.454298,0.452425,0.450272) … RGB{Float64}(0.44685,0.444965,0.442855)\n RGB{Float64}(0.44956,0.447679,0.445553) RGB{Float64}(0.443258,0.441368,0.439279)\n RGB{Float64}(0.444273,0.442384,0.440289) RGB{Float64}(0.444768,0.44288,0.440782)\n RGB{Float64}(0.443517,0.441628,0.439537) RGB{Float64}(0.45134,0.449462,0.447326)\n RGB{Float64}(0.446455,0.444569,0.442462) RGB{Float64}(0.455465,0.453593,0.451434)\n RGB{Float64}(0.444552,0.442663,0.440567) … RGB{Float64}(0.453032,0.451157,0.449011)\n RGB{Float64}(0.448338,0.446455,0.444337) RGB{Float64}(0.449802,0.447921,0.445794)\n RGB{Float64}(0.450719,0.44884,0.446708) RGB{Float64}(0.449924,0.448043,0.445916)", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAIAAABJgmMcAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAAChZJREFUeAHtwVuOHOd5x+Hf+9Vb1dXTPTM9I4oH8eAQsCFDEhBHiVcRbybemrOAGOAKKEMXFg3YsiSEFCly+txV3yFX/5Iu+iqoy3oe/+Mf/4tzDOMsY1BKQUouSC6Zc0IIiLtzjmFIzhkppTAwBqUUxDDOMWNQ+IXCoFAQM0MM45xCYVAYOJNROZNROZNRuZlxjmEMjLPMDLHKGBQGFgwJISClFKSUgpRcGBiDUgpSckFyzohhiAVDAgGxYAyMQSAgZgxKYVAoDApnOZNROZNROZNRuZkhhjEwBmaGlJyRUhiEEBALhlRecU6KiYExKFaQFBOSc0ZyzkgpBQkWkEBAcshIUzVIVVWIYUjOGYkxIqUUpJTCOc5kVM5kVM5kVJ5zRnLOSE4ZMTPEgiFVVSHujuSSkb7rkBgTcupOSNM0SIoJmbUzpK5rpOs6JPYR6foOsWRI0zQMCoOcMuLuSEoJiSlyjplxjjMZlTMZlTMZleeckRgjklJCzAxp6gbxyhF3R06nE3LqOuRwOCLd6YRsyxYJISC7/Q4JFpC+75HKKyT2Ebl//z7Sti3itSPr9Ro5no5IKYVzQgiImXGOMxmVMxmVMxmVUxgYhnjliAVDKq8YGIOu75Cu7xDDkNodaeoaORwPiFlAckrI9rBFnjx5wsAY1HWNzJoZ8uHuA2IYklJCZu0MyTkjIQQkpYS0sxY5nU6IMxmVMxmVMxmVmxlSVRVSecU5VaiQkjOSckFKKUiMEen7HokpIikmJOXEoDBYLpfI9fU1slgukK7rkOPhiKzv1khMEZnP55xjZpxTNzUSU2RgDJzJqJzJqJzJqNzMEHdHvHYkp4xcXl1yzsV8jmw2W+RwOCBd1yG7/R5ZXNRIM2uQvuuRUAXk+voaWd2skP1uh3y32SKVV8huv0c2mw3Sti1ye3uLpJiQlBLSti2SU0acyaicyaicyaicX8glIzFGJFhA3B3JKSPrzQaJfUQ2mw1yPB2Rex/dQ7x2JMWELBYLxN2R7W6LuDtyPB4RC4bc3Nwgp+MJca+Qvu+RruuQFBNSVRVyPB0RrxxxJqNyJqNyJqNyjIFhSM4ZOZwOSB97xDBksVggDx89RO4/uI98+PABeff2HXK8OyLL5RK5WFwg3alDlsslcupOSIwRef78OfLmzRtks94gzaxBdtsdMmtmDBoGXd8h83bOOc5kVM5kVM5kVF5VFRIsIKUU5GJ+gWx3W2Q+nyMpJeTd23fIer1G3n94j/R9jxyPR6SpG+T9T++Rq+srzulOHbLf75Hvvv+Ocx4/eYzUdY30fY/s93uk73qkmTWI8bOUM+JMRuVMRuVMRuUUBoWC5JyRnDOyWCyQxWKBHPYHpK5rZLfbIa9e/Q3puw558vQJ8vrNa+T29hYJVUDuPtwhm80Gubu7Q3b7PfLo4UNk3s6R5XKJrDdrpJ21yCmfkNhHxGvnHGcyKmcyKmcyKk8pIYXCObPZDDEMiTEi+8Meefv2LeLuyOPHj5G/vXqFrK5XyP6wR17/72vkV//yK+SbV6+Q66sr5Pnz58jl1SWSc0b+8fd/IHfrO2S5XCIxRsTMkLquka7rkLqpEWcyKmcyKmcyKs8lI6UUBoXB6XTinDrXyGq1Qp48eYKsrlfIZ59/hvzlq78gn/72U+TBg/vIn/7038hvfv1rZLFYIM+ePUP6vkc+vP+A7PY75OrqEvnp/Xtkt90h7o6klJDFYoE0TYNUoUKcyaicyaicyajcMKSUgpgZEiwgXjvSzlpks90g9z76CHn85DHyu9/9K/Lll/+GXFxcIJv1Bnn08CHyH7//PfLmzY/IfD5HzAyZzRqk71vOWS575O7DHVJKQWbtDJnP50gIASmlIM5kVM5kVM5kVG5mSKgC0jQNYmbIrGmQru+Q1fUKef/+PfLm9Rvk5cuvkFffvEK+/PcvkdPphNzc3CIvX75Enj57inzz178i7jVS1w1S+RHJJSM5Z+Tq6gqpmxpp6gaxYEioAhL7iDiTUTmTUTmTUbmZIe6OtLMWSSkhddMgm+0WiTFyztdff40sL5dI13fIfr9HHj9+jLx48QK5uLhAPv30U+T2o4+Q0/GIvH33DtlsNsjhcECqqkJyzsh8Pkd2ux1iGFLXNVJKQZzJqJzJqJzJqBxjYGZI13VIH3vEgiF93yOHwwE5Ho/IbDZDXrx4gazv1sif/+fPiAVDPr73MfLo0SPk22+/RVKMyHfff48c9gfkp59+Qt6+fYu0bYvc3t4i+90eqesaSTEhZoaEEBBnMipnMipnMiovpSAxRs4xM2S73SLr9Rp5++NbJOWEvHn9Brm7u0Pe/PgjstlskM8//4xBYfCHP/wn8s2rb5B21iKH4wE5HA5IO2uRBw8eIJVXyMX8AikUpKkbJDcZKaVwjjMZlTMZlTMZlfNLhUEpBYkpIqfuhBiGfPLJJ0jXd8jFxQWy2+0YmCEPHzxAri6vkJvbG+Tly6+Q0+mEXF5eIqEKiFeOXC4vkdVqhTRNg2y2G6SqKqSuaySmiMQ+co4zGZUzGZUzGZUHC4iZIX3skVIKYmZIH3uknbdIPmakaRpkuVgim80GWS6WSCmFc65XK2R9d4d0XYecug5Zra6Rq6srpG1nSOFnl5eXSM6Zc0opSMqJQWHgTEblTEblTEblpRQkpogcDgck9hFxd6SqKuR0PCG73Q7pY0TKRUHu3fsYKTkjr1+/Rq5X18jl5RKZty3nlFKQqqqQvu+RmBqk9hopVpCUEpJSQmKMSMmFgTFwJqNyJqNyJqPyXDKSYkJKLgyMQaEg280WSTkhKSak8LNHDx8h2+0WKRTEgiGzZobcrG6Q9XqN1E2NXF9dITkXZHWzYlAYxBiR7W6LnI4nJMbIwBgEC5zjTEblTEblTEbl/IIFQ9wdCTkg7o6sblZIigm5vr5G7j+4j7Rti2zWG+Sf3/0T+eKLL5AffvgBaectst/vkbZtkb6PSFVVyHazRU7dCUkpIbvdjkFh4LUjVaiQUAXOcSajciajciajcsOQqqo4J1hAQhWQru+Qp0+fItdX1wyMwXKxRNZ3a+Ty8pJBYfDs6TNkfbdGLi4ukJQSki0j+8MeKaUgsY9IjBFJOSFNM0NCCIhXjlgwpJSCOJNROZNROZNReaEgwQJS1RU/KwzMkOViiVQhIJvtBrm5uUF2+x3Sxx4JISApJ2S/3iEWAtLHHum7HglVQFJMnGPBkLqpkZnNEK8dcXcGxnmlIM5kVM5kVM5kVG5mnGUMglVIoSAWDOn7HuljRPq+R0opSCkFSTEhXemQnAviwZBQBcQaQ0IISAgBKbkgFgypqgoxM84ppSAlF6RQOMeZjMqZjMqZjMoNY2AMSilIoSBmhgQLyKnrkP1uj7g7Ujc1g8KgaRrOsWBI7CNSlQoJITAwBlWokGIFMTPEMAaFQSmF/y9nMipnMipnMqr/AybhGoxKcg0rAAAAAElFTkSuQmCC", "text/html": [ - "" + "" ] }, "metadata": {}, @@ -146,10 +146,10 @@ { "output_type": "execute_result", "data": { - "text/plain": "28×28 Array{RGB{Float64},2} with eltype ColorTypes.RGB{Float64}:\n RGB{Float64}(0.448391,0.446508,0.444389) … RGB{Float64}(0.450115,0.448235,0.446107)\n RGB{Float64}(0.449726,0.447845,0.445719) RGB{Float64}(0.450081,0.448201,0.446072)\n RGB{Float64}(0.453987,0.452112,0.449961) RGB{Float64}(0.448844,0.446962,0.444841)\n RGB{Float64}(0.449859,0.447978,0.445851) RGB{Float64}(0.447947,0.446064,0.443948)\n RGB{Float64}(0.441563,0.43967,0.437591) RGB{Float64}(0.450465,0.448586,0.446455)\n RGB{Float64}(0.439193,0.437297,0.435231) … RGB{Float64}(0.452883,0.451007,0.448863)\n RGB{Float64}(0.441229,0.439336,0.437258) RGB{Float64}(0.453821,0.451946,0.449796)\n RGB{Float64}(0.445588,0.443701,0.441598) RGB{Float64}(0.447974,0.446091,0.443975)\n RGB{Float64}(0.444873,0.442986,0.440887) RGB{Float64}(0.440706,0.438812,0.436737)\n RGB{Float64}(0.449014,0.447133,0.44501) RGB{Float64}(0.439847,0.437952,0.435883)\n ⋮ ⋱ \n RGB{Float64}(0.449264,0.447382,0.445259) RGB{Float64}(0.449911,0.44803,0.445903)\n RGB{Float64}(0.455663,0.453791,0.45163) … RGB{Float64}(0.445926,0.44404,0.441935)\n RGB{Float64}(0.450382,0.448502,0.446372) RGB{Float64}(0.445039,0.443152,0.441052)\n RGB{Float64}(0.442325,0.440434,0.43835) RGB{Float64}(0.446556,0.44467,0.442562)\n RGB{Float64}(0.440944,0.439051,0.436975) RGB{Float64}(0.451545,0.449667,0.44753)\n RGB{Float64}(0.444722,0.442834,0.440737) RGB{Float64}(0.453988,0.452114,0.449963)\n RGB{Float64}(0.443903,0.442013,0.439921) … RGB{Float64}(0.452315,0.450438,0.448297)\n RGB{Float64}(0.44863,0.446748,0.444627) RGB{Float64}(0.449708,0.447828,0.445701)\n RGB{Float64}(0.450083,0.448203,0.446074) RGB{Float64}(0.449769,0.447889,0.445762)", - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAIAAABJgmMcAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAACgtJREFUeAHtwVuuHNd5huH3X/VXV/fubrI3N80DuBkyBixdSAYEGfYg4tHEQ9MQIk1AcoAYom5kSjEpkiL7XKe1Vq6+oi8aMBDUZT2P/+Uv/8m/YhgfZSRnBjlnJKbIJSEExN2REAKDzCDlhOScGWQGmcwlhjEwBoYhOWckkxHDEDPjkkxmkBk4k1E5k1E5k1G5YVxixkdmiBG4yBg4joQQkKIoEOOjlDOSYkIsG5JSQnLOSMqJS0IIiGGIBUMMQ4IFBsZHmUEmM0hc5ExG5UxG5UxG5WaGmBkDY2BmSEqJQWYQQkBCCEgRCiTnjHR9j1gwJOeEpJyQlBKSUkJSTkiwgCQSUoQCCSEgRSgQC4bklJEYeyQnBpnMJc5kVM5kVM5kVJ5zRmKMSEqJi4xBEQrES2eQM9I0DVLXNdLHHim9RHLOyNXVAqmWFdJ1HdK1HdJ1HYPIIFQBMQzJZKQIBdJ0DdL3PYPMwMwYGANnMipnMipnMipPKSFd3yEpJgbGoPQScXfE3ZHYRyTGFum6Dmm7FjnsD0jhBVLXNZf0fY8URYH0fY88ePAAWSwWiJeO7HY7pK5rJKWEmBliZkgIgUFm4ExG5UxG5UxG5ZmMGIaEEBgYA3dnYAxiH5G+7xEzQ8pZiVTzCjkejogFQ1JMyOl8Qm5vbxHDkLIskVk1Q7YftogFQ2IfkWpeISklpAgFElNEqlmF1E2NOJNROZNROZNRuWGIuyMhBC5xdwbGIKWEpJyQruuQvuuRruuQGCPS1R2Sc0ZWyxVy9+5dZL1aIU3bIvW5Rra7LdL3EVleXTEwBoYhPT1SliXS9z1iGOJMRuVMRuVMRuVmhoQiICEEBpnBer1mYAyWV1fIbr9Hjscj4q1zyeJqgVSzCmm7FilCgWw2G+T6+ho5HU/Iy8NLxAtHTqcTctjvkfl8jty7uYfEPiJ97JHFfIGknBBnMipnMipnMirPZAaZiywY4qUjKSVku90iXdcjh8MBaeoG+c2D3yClO9L1PbJarRB3R46HI1J6idR1jQQz5PreNdI0DeKFI13XIU3TIDFG5MqvkLqpkaIoEGcyKmcyKmcyKjczxMy45Hw+I33fM8gMrpZXyKNHj5AHDx8gHz58QN69fYe0bYusVitkebVEmqZBVusV0jQ10vUd8u+//S3yy+tfkP1uj8yqGXI8HJF5NWdgDNqmReaLOZc4k1E5k1E5k1F5EQrE3ZGYIlJVFXI6nZCqqpAUE/Lu13fIbrdDPrz/gLRdizR1g5Rlibz79R1y9+5dBplB07XI6XRCfvrpJy55cvsEmc1mSNd2yOl0Qrq+Q6pZxSUxRcSZjMqZjMqZjMozGYkpIjFGJOeMXF1dIcvlEqnPNVJSIsfjEXnxww9I17bI06dPkVevXyE3NzdIURTIdrtF9rs9st1ukePxiDx6/AiZz+fInfUa2e32SFVVSOwj0nUdUpYllziTUTmTUTmTUXmKCYl9RDIZmVdzLum7HjmejsibN28QLx25vX2C/PDiB2Sz2SCn8wl59eoV8vzZc+T7F98jd9Z3kOfPnyPrO2skpYT8+OOPyG67Q1brFRJjREIRkJmVSN00yKycIc5kVM5kVM5kVB5TRHLOXNI0DQNjkFJCNpsNcnt7i2w2G+Szzz5D/vuvf0U++fRT5OHDh8hXX32FfPK73yFXyyvk2bNnSNd1yPv375Hj8Yis12vk/a/vkf1hj3jhSIwRWa1WyGw2Q4oiIM5kVM5kVM5kVG4Yl1gwJBQBKYoCqaoKOR6OyM3NDfLkyRPkiy++QL788ktksZgj+/0eefzoEfLHP/0RefPmLbKYzxHjo6qqkLZpuaRbd8ivv75HkidkPp8ji6sFUhQFklJGnMmonMmonMmoPISAmBkyq2YMckaq+RxpmxbZXG+Q9+/fI7+8fo189+23yPcvXiB/+MOXSNt2yM39+8i3336H3D69RV58/wIpigKZlTPEvUDOdUL6rkfu3rmDlGWJVPMKCSFwSc4ZcSajciajciajcguGFEWBlGWJ5JSQ0h3Z7/ZIH3skWED+529/Q1brNdI0DXI6nZEnT54gX//X18hyuUQ+/fQT5N7NPaRpGuTtm7fI4XBEzucz4u4MjMF6vUaOxyPSW4+Us5JLnMmonMmonMmonH8SLCBd1yF93yMhFEjXdcjxdESaukEWiwXyzdffIIfDAfnmm2+QYAG5uX+DPH78GPn7y5dI7CPy888/I/X5jHzYbpG3b94gVVUh927uIafTCfHSGWQGhiFmhjiTUTmTUTmTUXlOGen7HslkJISAHA4HZLfbIW/evkH6rkdevX6N7Pd75O3bt8hut0M+/+wzBsbgz//xZ+TF9y+QqqqQc31G6nONVPMKefDwAeLuyGK+QCwYUs0qpOs7BplBTBFxJqNyJqNyJqNyM0NSTkiMEYkxIn3XI2aGPHz4EGmbFlksFsj5fEbMDHn48CGyvrNG7l3fQ7777jukbmpkvVojIQSkcEdWqxWy2WyQWTVDjscj4sERM0MMQ/rYIzlnxJmMypmMypmMyi0YYhiSYkLMDLFgSEoJmc/mSNu0SFmWyGq9Qvb7PbJcrfgoc8n19TXyYfsB6WNEmtMRuXPnLrJar5ByVnLJ8mqJxBiRTEZSSkhKiUucyaicyaicyag8p4zEFJG6rpG+7xF3RywYEmNE6rpG+hiR0Abk/v37SMoJefXqNXJ9vUGWqyVSzSvEzJCcMlIUBdK2LbKYL5BQBCTnzCAx6NoOiTEimcwlzmRUzmRUzmRUnlJCYoxIzhlJOSExRmT7YYvEGJEYIwNj8PzZM+RwOCCZjAQLSFVVyPX1NbLf75GyLJHNZoN0XYdcX19zSdu0yOl8Quq6RmKMSLCAmBmXOJNROZNROZNReSYjZoYURcElZVki1/eukb7rkTt37yCPHz1G5os5stvukJc/vUQ+//3nyD/+9x/IYrFATqcTUs0qpG1apCgCcj6fkaZukLqpkdPphBiGeOlI4QUSQkAyGXEmo3Imo3Imo3IzQ4pQIFYaUpYlEoqAlF2J3N7eIpu7GwbGYLVcItsPW2S9WnPJ0397imy3W2QxXyApJSSTkeOpQYrTCWmbFmm7FkkpIWU5Q0IISAgBsWAMEgNnMipnMipnMirnn1gwxAvnEjNDlqsl4u7I4XhArjcb5Hg6IV3fIaEISIwROR6PSLCA9H2P9H3PJX3skdhHLpnNZohhSOEF4oVzSU4ZyWTEmYzKmYzKmYzKzQwxDDEz/hUzQ7q2Rfo+Im3bIjllJKWE9H3PIDPIKSPmhhRFgYQQGBiD0AUuCUVAQgiImSE5ZwaZQc4ZyWTEMMSZjMqZjMqZjMoNY2AMcsqIBUPMjEuatkVOpxNSFAUym824pJpVXGLBkL7rkZwzYsGQQEDcHUk5IWaGGMb/l2Fc4kxG5UxG5UxG9X+LTP+jm4nrPgAAAABJRU5ErkJggg==", + "text/plain": "28×28 Array{RGB{Float64},2} with eltype ColorTypes.RGB{Float64}:\n RGB{Float64}(0.45016,0.44828,0.446151) … RGB{Float64}(0.45165,0.449772,0.447635)\n RGB{Float64}(0.451072,0.449193,0.447059) RGB{Float64}(0.451798,0.44992,0.447782)\n RGB{Float64}(0.455397,0.453525,0.451366) RGB{Float64}(0.449553,0.447672,0.445546)\n RGB{Float64}(0.449617,0.447737,0.445611) RGB{Float64}(0.449116,0.447234,0.445111)\n RGB{Float64}(0.442343,0.440452,0.438368) RGB{Float64}(0.452817,0.450941,0.448796)\n RGB{Float64}(0.436503,0.434604,0.432553) … RGB{Float64}(0.454642,0.452769,0.450614)\n RGB{Float64}(0.441503,0.43961,0.437531) RGB{Float64}(0.455868,0.453997,0.451835)\n RGB{Float64}(0.44569,0.443803,0.4417) RGB{Float64}(0.446804,0.444919,0.442809)\n RGB{Float64}(0.446602,0.444717,0.442608) RGB{Float64}(0.442395,0.440504,0.43842)\n RGB{Float64}(0.450043,0.448163,0.446035) RGB{Float64}(0.443575,0.441685,0.439594)\n ⋮ ⋱ \n RGB{Float64}(0.447711,0.445827,0.443712) RGB{Float64}(0.458446,0.456579,0.454401)\n RGB{Float64}(0.456223,0.454353,0.452188) … RGB{Float64}(0.448222,0.446339,0.444221)\n RGB{Float64}(0.457378,0.45551,0.453339) RGB{Float64}(0.446933,0.445049,0.442938)\n RGB{Float64}(0.446997,0.445112,0.443001) RGB{Float64}(0.447016,0.445131,0.44302)\n RGB{Float64}(0.440324,0.438429,0.436357) RGB{Float64}(0.453963,0.452089,0.449938)\n RGB{Float64}(0.440753,0.43886,0.436785) RGB{Float64}(0.457179,0.45531,0.45314)\n RGB{Float64}(0.44663,0.444745,0.442636) … RGB{Float64}(0.454107,0.452233,0.450081)\n RGB{Float64}(0.450243,0.448364,0.446234) RGB{Float64}(0.450868,0.44899,0.446856)\n RGB{Float64}(0.452037,0.45016,0.44802) RGB{Float64}(0.451367,0.449489,0.447353)", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAIAAABJgmMcAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAACiFJREFUeAHtwcGOHNd5huH3P+evru6Znpkmh5RJcWjGjmAFUBzIApybcK7GuTXdQAzEQLyyaXuT4SKyKJsMSc1MT3d1d1Wdc7L6ilp0EASoZT2P/+uvf83/yfiocFQpBck5MTBDggUkekQM45hSClJKQQqFQeEj4yjDGBiDkgtHGQMzY1AYFArHOJNROZNROZNROcZRZoYYxiBwlGFILgEJZkgIESkUJISAlFKQkgvS9z2SS0ZyzohhiAVDQghIsMAgMAgWEDNDSilIoTAoDAoFcSajciajciajcjNDzAwxjIExyDkzKAyCB8TNkRACUkphUBjEGJH20CJd3yE5Z6SUguSckRACEkpASi5I9IiEGBAzQ3LOSOoTUkpBSikc40xG5UxG5UxG5SUXJJWE5JQ5yhjEGJFIREopSNd1SNd1SOoTx6SckOXpEqmqCmm7Fum6Dum6Dkl9Qmb1jGNKLkjwgKQ+IV3fcYyZIYYhzmRUzmRUzmRUnnNG+tQjqU9ICAFxd8TdkaqqkJQSklJC9vs90jQN0vcJmc0qpG1bPjKk7zskRkf6rkM++eQTZL6YI5VXyN36Dsn7jOScOcaCISEEjnEmo3Imo3Imo/JcMoPCIISAWDDE3ZFgAUkpIX3XI33fI4YhFxcXyHa7Rdwd6fse2TU75OrqioExqKoKqesaub25RcwMSSkh9bxGSi5ICAHJOSOzeoYc9gfEmYzKmYzKmYzKgwUkeEBCCEgIAYkxImaGlFyQXDLSti2Sc0aabYOklJDD/oAUCnK6PEUuVhfIcrlE2kOL7Pc7ZL1eI33fIycnJxwTQuAYrxzp+56BMXAmo3Imo3Imo3IzQ2KMSIgBKbkgy7MlYmbIYrFA1us14u5I13bIbrdD5mGO1HWN9H2PhBiQ1WqFPHjwAGm2DfLttxskekSapkHu7++R+XyOXD66RPq+Rzw5Mq/nSE4ZcSajciajciajcv4XxkcWA1JVFZJzRu7X90hKCdlutshuv0MuLy+RyiskpYScnZ8h7o5sNhukckf2+wNiZsjDhw+Rw+GARI9I3/VIe2iRlBISY0QOhwMSY0ScyaicyaicyajczBAzBjkXZLdrkL7rkVwyslwukSdPniA/+uRHyM3tDfL+/XtkvV4jp6enyKmfIofDATlbLpH94YB0fYf89O9/irx9+xZZr9dIXdfIZrNBZrMZx7RdiyzmCwbGwJmMypmMypmMymOMSIwRSTkhi8UC2TYNMq9rJKWEvH/3Dlnf3yO3t7dI27bIfr9HKq+Q7z98j1xcXCCFj9rDAdk1O+T1t6855urZFVLNKqTrOqRpGqRtW6RONQNjkFNGnMmonMmonMmovFCQlBOSUkJKKcjpyQlycnqC7Jod4u7IdrNFrv/zGmnbFnn+/Dny9u1b5PLyEokxIrd3t8j9+h65vbtDttst8vTpE2Q+nyNnZ2fIer1G6nmN5JyRru+QyiuOcSajciajciaj8pQSg8KglILU85pBYdD3PbLb7ZD3798jVVUhV1dXyPX1NbJarZBm1yBv3rxBXrx4gVy/ukbOz86Rn/zk75CL8wsk5YR881/fIOv1Glkul0jqExIsIJVXSNu2SDWrEGcyKmcyKmcyKk8pIaUUjjnsDwyMQSkFWT1YIVfPr5DVaoV88cUXyB9f/hH5/B8+Rx4/fox8/fXXyGeffYacLk+RFz9+gXR9h9zc3CC7zQ45Oz9Dbr6/QTabDeLuSEoJOV2eIrNqhsQQEWcyKmcyKmcyKjeMYywYEmJAPDpS1zWy3WyRy4eXyNWzZ8iXX36JfPWLXyDzxRzZ3G+QJ0+eIP/8y18i7969Q+aLOYM9g9lshszqGVIoyHK5RG5vb5FcCrKYz5GTxQkSYkBKLogzGZUzGZUzGZWHGBAzQ2bVDCkUZF7XSNt1yOrBCrm5vUHevn2L/OH3v0euX71CvvrqK6TrOuTRo0fIH16+RJ5fXSHX16+QGCNSeYW4O7Lf7ZGUE3J+cY7Mqhkym80QC4YYhvS5R5zJqJzJqJzJqDxYQNwjUs0qJKeMzOoaub/fIKlPSCkF+dOf/oycnJwg+90O2Ww2yPOrK+Q3//YbZHm2RD7/2c+Qh5cPkcN+j7z/8AHZ3G+Q3W6HxBA5ZnGyQLabLcfUdY2UUhBnMipnMipnMirnh8yQru2QPvUMjMF+v0f61COHwwFZLBbIf/zud8jd3Rr599/+FjEz5NHlI+Tp0yfIN3/5C5JSQl5/9x2ya3bIh+8/IO/evUPm8zny+NFjZNfskGpWISUXxMwQC4Y4k1E5k1E5k1F5Lhnp+55BYWBmSNM0yGa7QT58+B4xPnrz5g1yt75D3v33O+Tm9hb5xy++4Jhf/cuvkOtXr5C6rpHdbofsdjukrmvk6ZOniFeOLBYLBoVBNauQ1CeOMgbOZFTOZFTOZFTOD5RcOKbrO6TrOiRYQJ49e4bsdzukrmtks90gKWXk8ePHyNnZGfLw4UPk5cuXyOFwQM7PL5AYAxKjI+dn58jqwQqpqgrZ3G+Q6BGJMTIoDPrUI1YMcSajciajciaj8hACYhiSckJyygwKg8PhgFRVhaSckBAC8vjRY2R9t0ZOl0skmCFmhqzOV8jd+g5pDwek7VpkdbFCzi/OkXk9RwoFWS6XSM6ZYxIJyTlzjDMZlTMZlTMZlRuGpJSQZtcgXdchMUYkxoj0fY/smh3SdR1iwZDLR5cMCoO/vfkb8uDBA+TTs0+RxWLBMaUUpKoqpOs6pKs6JMaImBliZkjXd0jKCSmlIIYhzmRUzmRUzmRUnnJCur5DUp8YFAYlF2S73SIpJySnjJRSkE+fforcb+45xsyQuq6R1YMVsl6vEXdHVqsVknNGVqsVYmZI3/XIttkih8MB6dqOYywYA2PgTEblTEblTEblOWcGhUH0iIQSEI+OrB6skK7tkPPzc+Tpp0+RxWKBrNdr5PXr18jP/+nnyF+/+ytyslggTdMgJ4sTpO97xMyQ7XaLtIcW6foOaZoGKaUglVeIuyMhBI5xJqNyJqNyJqNyfiB6REIMSCkF8ehI13fI86vnyMXFBcecLk+Ru9s7ZHm65Jgfv/gxcnN7iyzmC6RQkL7vkV2zQ2KMSNd3SNd2SEoJmdUzJISAhBiQYAEppSDOZFTOZFTOZFQeLCAhGB8ZA2MQQkDOlmeIuyPb7Ra5WF0gTdMgXdchIQSk73pku9kiMUakTz3StR0SQkAKBUk5IWaGVLMKmdkMiTEiMUaOKRSklII4k1E5k1E5k1G5mfGRIRYMMTOOCTEgXdchbdsibdsipRSklIKklJC2a5FcMhJKQNwdCSEgwQISuoAUChJCQGKIDIyPCoNCQUopSMmFY5zJqJzJqJzJqBxjUCgMMh8FBmbGoDDYH/ZI0zSIuyOzasYxs3rGMcEC0vUdUkpBLBhSKEiMESmlIBaMgTEwDMlk/l+MgTMZlTMZlTMZ1f8AT24V2VTpui8AAAAASUVORK5CYII=", "text/html": [ - "" + "" ] }, "metadata": {}, @@ -176,10 +176,10 @@ { "output_type": "execute_result", "data": { - "text/plain": "28×28 Array{RGB{Float64},2} with eltype ColorTypes.RGB{Float64}:\n RGB{Float64}(0.44758,0.445696,0.443582) … RGB{Float64}(0.449233,0.447352,0.445228)\n RGB{Float64}(0.448893,0.447011,0.444889) RGB{Float64}(0.448745,0.446862,0.444742)\n RGB{Float64}(0.452783,0.450907,0.448762) RGB{Float64}(0.447791,0.445907,0.443792)\n RGB{Float64}(0.446748,0.444863,0.442754) RGB{Float64}(0.446789,0.444904,0.442795)\n RGB{Float64}(0.440573,0.438679,0.436605) RGB{Float64}(0.449866,0.447986,0.445858)\n RGB{Float64}(0.437677,0.435779,0.433722) … RGB{Float64}(0.451577,0.4497,0.447562)\n RGB{Float64}(0.440341,0.438446,0.436374) RGB{Float64}(0.453785,0.45191,0.44976)\n RGB{Float64}(0.443839,0.44195,0.439857) RGB{Float64}(0.446276,0.44439,0.442284)\n RGB{Float64}(0.444977,0.443089,0.44099) RGB{Float64}(0.437706,0.435808,0.433751)\n RGB{Float64}(0.450367,0.448487,0.446357) RGB{Float64}(0.440461,0.438567,0.436494)\n ⋮ ⋱ \n RGB{Float64}(0.44645,0.444565,0.442457) RGB{Float64}(0.45228,0.450403,0.448261)\n RGB{Float64}(0.453222,0.451347,0.4492) … RGB{Float64}(0.44584,0.443954,0.44185)\n RGB{Float64}(0.447859,0.445975,0.44386) RGB{Float64}(0.444286,0.442397,0.440302)\n RGB{Float64}(0.441062,0.439169,0.437092) RGB{Float64}(0.444814,0.442926,0.440828)\n RGB{Float64}(0.439746,0.437851,0.435782) RGB{Float64}(0.451182,0.449304,0.447169)\n RGB{Float64}(0.442743,0.440852,0.438765) RGB{Float64}(0.453861,0.451987,0.449836)\n RGB{Float64}(0.44324,0.44135,0.439261) … RGB{Float64}(0.451777,0.449899,0.447761)\n RGB{Float64}(0.44704,0.445155,0.443044) RGB{Float64}(0.448531,0.446649,0.444529)\n RGB{Float64}(0.449331,0.447449,0.445325) RGB{Float64}(0.448941,0.447059,0.444937)", - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAIAAABJgmMcAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAACh5JREFUeAHtwc1yHNd5x+Hfe/rtbsxghpgBKYIyQVtSleyF5CpZKecmnJuJL825gnghby1HK1EbKZRkEaSIGWA++uOck9W/ocVUpZLqZT+P//nP/84g88D4P8kpIyklTrFgiBfOwBiYGZJTQnLOPDAkkznFMAbGg8wg58xJxiBYQDKZQeYkZzIqZzIqZzIq5xfMjIExMIyTjAfGIBQBKUKBWDAkhIDklJGUE5LNkJwyknNCUkqcYmZICAEJRUAMQwxDzAzJZAaJQSYjmYw4k1E5k1E5k1G5mSHBAgNjYGZIzhnJKSPuzsAYeOFIygmJMSI5ZSSlhMQYkZwzElNEcsqIBUMKKzjFC0dCCAyMQU4Z6fseyWQkkznFmYzKmYzKmYzKc8pIlzskpcQgMwghIBYMKUOJ5JyRpmmQruuQpm2QqqyQGCMym82QsiqRtmmRvu+RruuQGCNSVzWSc0ZSikjhjnSxQ/q+5xQz4xRnMipnMipnMipPOSGxj0gfIxLMkFAEpKoqpCorpGkapO1a5HA4IM2xQXZ5h1gwZH/YI8EC0vc9UniBdF2HPH36FJmdzZCyLJHNdoMcjw2SUmJgDIpQIGbGKc5kVM5kVM5kVE5mkMlICIYEC4i7c0rbtUjXd4hhSFmWSF3VyP5wQIwHMUXkfnePXL+4RgxDyrJEqqpEbjcbxMyQGCNyVp8hKSUkFAFJMSFVXSFN0yDOZFTOZFTOZFSOMfDCEQuGmBkSLDDIGUkpITlnpO97pO97pO97JMaIdG2H5JyRxWKBXFxcIIvFAmmbFjkej8jd9g7p+h6Zz2eIYQyMgfWGlGWJxD4ihiHOZFTOZFTOZFRuZkjhBVIUBZJSQh5dPOKU8/NzZLvZIvv9Dmm7DjkcDsh8PkfqqkaapkFCEZDVaoWs12tkv9sh//3qFVIUBXK/2yF3d3fIbHaGXF5eIn3fI33skdnZDEkpIc5kVM5kVM5kVM4vZQY5JaQIBVJ6ifSxR27f3SJd3yHb7R3SNA3y3tP3kLIskb7vkfn5HCm9RHb3O6R0R47HI2JmyPpyjRybI1K6I23XIW3TIn3fI144cjwekaIoEGcyKmcyKmcyKg8WOCWljOwP90jTNohhyPn5OfL+s/eRq6srZHO7Qd68eYPcNrfIcrlE5vM50jQNslgukGPTIF3fIx999BHy+qefkO12i9RVjdzv7pGqrpCqrpC2bZHZbMbAGDiTUTmTUTmTUXkIAQkhICknZDabIYf9AanPaiSmiPz881tks90it+9ukbZrkePxiJRVifRve+Ti4oJBzkjbdsj+sEdevXrFKdfPr5GyKpGu65D9fo+0bYvUdY0YhsQUEWcyKmcyKmcyKs9kTsoMUkrIbD5DzufnyOF4QDIPdrsd8vKbb5C2aZAXv36B/PTTa+Tx40ukuC+QzWaDbO+2yGazQXa7HfLs2fvIWX2GLJdLZHu3Rc7qMySlhHRdh5RlySnOZFTOZFTOZFSeYkJSTEjKCanrmlO6vkP2+z1yc3ODlF4i19fPkW9efoOs12tkv98j//zxn8gHH3yAfP3ya+TR8hHy4QcfIo8uHiExRuTbb79FttstslgukD72SAgBqcoKadsWKasScSajciajciaj8pQSElPklOPxiJgZklJCVhcr5Pr6GlldrJBPPv0E+eq/vkJ+97vfIk+vrpD/+MtfkI8//i0yn8+RDz/6EOm6Dnn79i1y2B+QR8tHyM8//4zc3d8h7iUS+x5ZLBZIVVdICAFxJqNyJqNyJqPyTEbMDAkWkMILpCgK5OzsDLm/u0cuH18iz6+fI5999hny+ed/QGazGXJ3d49cPXuG/PFf/4i8fv0amc1mSM4ZqasK6WdnDIzBcrlE3t2+Q3JmcFbXyGw+Q4pQIDlnxJmMypmMypmMyoMFxApDqqrilLquka7tkNV6hWxuN8jrn14jX/7978jLl98gn//LH5C+j8h7T54g//jyS+T6xTXy8uVLxN2Rqq6R4/GIpJSQPvbIcrlEqrJCqqpCghliZkiKCXEmo3Imo3Imo/IQAuJeIFVVITFGpK5rZL/fI/E+cspXX32FzM/nyOF4QPa7PXL94gXy1//8K7JYLpCPP/4YefL4CdK0DfLm5g2yvbtDDvsDUoSCUxaLc+R+t0OsN6SqKiSnjDiTUTmTUTmTUTnGAzOkaRqk73sGmcHufof0sUcOhwMym82QL774Arnb3iF/++JvSAgBefz4Enn2/vvId999h8QUkVevXiH73R559+4dcnPzBjk7q5Enj58gh8MRqcoKSSlxSigC4kxG5UxG5UxG5TlnJPYRSTkhZobsdjtkt98hNzc3SIwJubm5QTabDfL2zRvkdrNBPv30EyRnBv/2pz8hL79+idRnNXI8HJHD4YDUdY1cXT1F3EtkPp/zICNVVSN97JGcM5JzRpzJqJzJqJzJqJzMIOWEZDLS9z3SHBskWECeP3+OtE2LLJcLZLvZImaGPH36FFkulsj6co18+Y8vkebYIMvlEglFQNwdWS6XyGp1gXhZIrvdDvHCkRACYsmQGCOnOJNROZNROZNReQiBgTGIMSIpJsSCIU3TIGVZIn3skRAK5OrqCtlut8hyuUTMDDEMWa/WyO3mFmnbFmnbFllfXiKPLh4hVV1zyvn5OZJiYmA8yAxSSohhiDMZlTMZlTMZlaeckBwzcjwekaZpkFAExMyQpm2R/W6PdF3HIGfk6tkVkmJCfvjxR2S9XiGLxQI5m50hhiEpJSQUAWmODeLuSFEUiJkhOWckdhFJKTHIDLJlxJmMypmMypmMylNKSIwRiSkiOWcGmcFut0NijEjKiQeG/Or5r5Dd/Q7JOTMwQ+q6QtbrNbK92yJVVSEXFxdI3/fIerXmlL7vkf1hj7Rti/Sx5xTDOMWZjMqZjMqZjMr5BcOQ0ktOqcoKWa/XSB8jsl6tkGfPrpD67AzZbDbI999/j/z+958iP3z/AzKbzZDdfofUVY00xwbx0pH9fo8cjgck9hHZ7/dIJiPujrg7EiwwMAbOZFTOZFTOZFRuGBJCQMwMCUVA3B0p2xJ58eIFslqtkJQTcj4/Rza3t8j5/JxTfv2bXyO3t7fIbDZDcs5Iygk53h0ZGIPm2CApJyT2ESnLEgllQIIFJBQBySkjzmRUzmRUzmRUjjEIFhAz4xQLhiyWC6TwArnf3SOr1QrZH/ZI23ZIKALS9z2y2+2QoiiQ2Eek6zrEgiGxj5xSFAUSckCKokBKL5GiKBgYg5wzksmIMxmVMxmVMxmVmxmnWDDEMAbGIFhAurZDur5D2qZFUk5IygmJfUQ6OiSnjOSQEXdHQgiImSF96JGUE+JFgYRQMDAGOWcGmQeZQcqJU5zJqJzJqJzJqNwwTsk5MzAGhiEWDGnbFtnv94i7I2VVckpVV5xiwZCu6xhkBhaMU0IREMvGA+P/K5P53ziTUTmTUTmTUf0PZA4ebFFK5u4AAAAASUVORK5CYII=", + "text/plain": "28×28 Array{RGB{Float64},2} with eltype ColorTypes.RGB{Float64}:\n RGB{Float64}(0.446786,0.444901,0.442792) … RGB{Float64}(0.448556,0.446674,0.444554)\n RGB{Float64}(0.448043,0.44616,0.444043) RGB{Float64}(0.447719,0.445836,0.443721)\n RGB{Float64}(0.451045,0.449167,0.447032) RGB{Float64}(0.447009,0.445124,0.443013)\n RGB{Float64}(0.447983,0.446099,0.443983) RGB{Float64}(0.446539,0.444654,0.442546)\n RGB{Float64}(0.439622,0.437727,0.435658) RGB{Float64}(0.449511,0.44763,0.445504)\n RGB{Float64}(0.433626,0.431723,0.429689) … RGB{Float64}(0.450113,0.448233,0.446104)\n RGB{Float64}(0.434694,0.432792,0.430752) RGB{Float64}(0.45161,0.449732,0.447594)\n RGB{Float64}(0.441834,0.439942,0.437861) RGB{Float64}(0.445532,0.443645,0.441542)\n RGB{Float64}(0.444303,0.442415,0.440319) RGB{Float64}(0.438834,0.436938,0.434874)\n RGB{Float64}(0.44692,0.445035,0.442925) RGB{Float64}(0.439113,0.437217,0.435151)\n ⋮ ⋱ \n RGB{Float64}(0.445789,0.443903,0.441799) RGB{Float64}(0.453586,0.451712,0.449563)\n RGB{Float64}(0.454182,0.452308,0.450156) … RGB{Float64}(0.444063,0.442174,0.440081)\n RGB{Float64}(0.450403,0.448524,0.446393) RGB{Float64}(0.444185,0.442296,0.440202)\n RGB{Float64}(0.442814,0.440923,0.438836) RGB{Float64}(0.443925,0.442036,0.439943)\n RGB{Float64}(0.440224,0.43833,0.436258) RGB{Float64}(0.451355,0.449476,0.44734)\n RGB{Float64}(0.444143,0.442254,0.440159) RGB{Float64}(0.454062,0.452188,0.450036)\n RGB{Float64}(0.443519,0.441629,0.439538) … RGB{Float64}(0.45086,0.448981,0.446848)\n RGB{Float64}(0.447006,0.445122,0.443011) RGB{Float64}(0.447652,0.445769,0.443654)\n RGB{Float64}(0.448624,0.446742,0.444622) RGB{Float64}(0.448089,0.446206,0.444089)", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAIAAABJgmMcAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAACh9JREFUeAHtwcuuHNd5huH3X2tVdffu7n2kyL0jUkIUBzIcAnIc6Cp0Nc6l6RKoeWDLE0fUILJkiBRJ7UOf6rAOGX1FDxpgAtSwnif85x//yFHGhxUGpRQk58zAGDhziA8eMTOOKbkgpRSkUBgU3jOOMoxjSikcZQzMjKMKg0JBApNRBSajCkxGFTAGhnGUMTAzxDCOKaUg5gxxziFmxjEpJQbGIOeMlFKQkgvHmDPEOYc4c4g5QwxDzAwpFKSUgpRSOCYwGVVgMqrAZFTBzBAzQwxjYAxKKUgpBfHBI2aGODPeMySXjDjnkJwzklJCcs5IzhkpuSDmDHHFIaUUxAePeO85ppSCxBiRkgtSSmFgDAKTUQUmowpMRhVKKUhOGUk58SHeecR7j+SckT4npO97JKWEeOeRmCKyWq6QxXyOdF2P9H2PdH2HpJSQWT1DCgVJOSEhBKRrOyTFhBQK4swYFEMCk1EFJqMKTEYVcs5IjBFJKSGGISEEpKoCUtc1EmNEUk5I27ZIc2iQlBISqoC0bYuYGdL3PeK9R2KMyOPHj5HFYoFUVYXc398jbdMiKSfEmUPMGWLOMSgMApNRBSajCkxGFUouSCkFMTPEOYdUVYWYGRJjRPq+R1JMiHceOT07Rfb7PWJmSEoJOewPyNOnTxkYg7qqkbqukfuHewaFQUoJmc/nSC4Zcc4hOWekrmukbVokMBlVYDKqwGRUgX9QhQpx3iHOHBJCYGAMSilIoSBd1yExRuRwOCAxJaTvO94zZLlaImfnZ8h6tULarkOaQ4M83D8gXd8hy5MlYs74kKqqkBQTA2MQmIwqMBlVYDKq4JxDfPBI8B7JuSDr9RoppSAnJyfI/cM94p1H+tgju+0OWZwskNlshsQ+Is475Pz8HLm8uER2+x3y4+ZHxHuPdLsO2W63yHw+R64ur5AYI5JiQuaLOVJyQQKTUQUmowpMRhUwBqUUJOfCwBj44JFSCrLZbJCUErLb75Dm0CBXV1dICAGJKSLr1RoJVUB22y1SVRXSNA1izpDLq0ukaRskhID0fY90XYfEGJEQAtI0DRJ8QAKTUQUmowpMRhUMQwxDcslIc2iQFBNSKMhqtUJurm+QJ0+eILe3t8i7t++Q+4d7ZLlcImEVkK7tkPVqhTRNg/R9j/zLZ58hr3/5Bbm/u0dmsxmy2+2Q2WyG1LMa6doOWSwWHBOYjCowGVVgMqrgvUe890hKCZnP58h+t0fmizkSY0TevH2DbDYb5PbXW6TrOqRpGqQKFfLrr78iZ6dnSOG9vm2R/eGA/NePP3HM02dPkbqqka7vkP1+j/Rdj8zqGQNjkFNGApNRBSajCkxGFQoFSTkhOWck5YScLE+Q5ckSaZoGKaEg280Wefn9S6RtO+TZs6fIq9evkEdXjxDvPHJ3f4dsHjbI3f0dstvtkJvrG2Q+nyPr0zXycP+AzOdzpMkN0sceqaqKYwKTUQUmowpMRhVSSgwKR83nc8QwJOWE7Pd75M3bN0hVVcjTp8+Qly+/Qy4uLpHDfo+8evUK+fTTT5GXL18ip2enyGf//BlyenqKpJyQH/7nB+T+4R5Zr9ZIjBFxziFVVSFd2yFVXSGByagCk1EFJqMKOWU+pG1ajqmqCjm/OEeePnuKnJ+fI8+fP0e+/fZb5Leff448fvIE+frrr5F//c1vkOVqiXz6ySdIHyNyd3uHbLdbZH26Rm5/vUV22x0SQkBSSshytUTqukaC90hgMqrAZFSByaiCmSGlFMTMEOcdEnxA6lmNbLdb5OrqCvn444+RL774Avn33/8eWZwskM1mg1xfXyNffvkl8subN8hicYJY2yB1XSOz+Yxj+nWP3N3dIblkZD6bI4vFAvHOIzkXJDAZVWAyqsBkVMGcIc4cUlUVx8xmM6TveuTi/AK5vb1Ffnn9Gvnzn/6EfP/998gf/uMPSNf1yKNHj5Bv//IX5JNnz5DvvnuJ+OCRqgpI8AE5lAOSc0bOzs6QuqqRqq4Q5xxizpAUExKYjCowGVVgMqpgZoj3HqnrGimlILN6huz3eyTvMmLOkL/+9b+R9ekp0jQtstlskWfPniEvXrxAlssl8tvPP0euri6RpmmQt+/eIZvNBjkcDohzjkFhsDhZINvNFjEzpJ7VSMkFCUxGFZiMKjAZVTAMMTOk6zok9hEppSBt2yJ91yNt2yLzxRz55sU3yMPmAXnxzQvEmUMeffQIubm5QX7429+QGBPy09//jhz2B+Ttu3fI2zdvkNlshlw9ukJ2ux1SVRWSc+YY5x0SmIwqMBlVYDKqUEpBYowcYxjSHBpks9kgr1//gvRdh7x5+xbZPDwgb9+9Qx4eHpB/+93vGBiDr776Cnn58iWymC+QpmmQw+GA1LMaub6+RkIVkJOTE8TMkKqqkJwSUgqDQkECk1EFJqMKTEYV+D9IKSFd1yFmhvzTzQ3Sti2yXq+R7W6HmBny5MkT5PT0FLm4vEC+/fO3SNu1yOnpKWJmSPABWa6WyOXFJVJVFbLdbpEQAhJCQBKGxBgZFAaByagCk1EFJqMKzjmOSTkhMUYk5YR0bYcsV0skNwUxZ8jjjz5CtpsNslqvea8ghiHnF+fIw/0D0rYd0nUtcnF+gZydnSGz2YyBMVitVkjKCTEMKaUgMUYGxiAwGVVgMqrAZFShlIKknJC2aZGu75AQAlLVFRJjRJrDAYkxIqlKyOPHj5FcMvLzzz8j5+fnyM3qBlnMF0ihIKUwCMEjh8MB8c4joQqIYYiZIV3XIbGPSKEghiGByagCk1EFJqMKOWckxYTEFJGcMpJIyN3dHRJjRFJKiJkh19fXyHa7RUopHDObzZDLi0vk/uEeqesaOT87Q7q+Ry4uLhDvPNLHHmkODbLf7ZE+9ogzx8A4KjAZVWAyqsBkVKFQGBiDKlSIM4f44JHzs3MkpYRcXFwgT66fILPZDHm4f0B+/OlH5Pnz58irn18hi8UC2e/3yKyeIV3XI845pG1bpGs7pO1aZL/bIyklpKorxAWHeO85JjAZVWAyqsBkVMHMEO884iqH+OCR4AMS+4jc3NwgZ+dnSCkFWS6XyN3dHbJerxHDkE8+/QS5u71D5os5knNGSixI27XI/rBH2rZF+r5HSi6IDx7x3iPee8Q5h5RckMBkVIHJqAKTUQX+gXMOMTMGxsAwZL1eId57ZLvdIudnZ8h+v0dijIhzDokpIru7HeK8Q2IfkRgjYmZIjBHJOSOGIXVVI2aGOO8Q7z2DwqCUgpRSkMBkVIHJqAKTUQUz40PMDDEzpGRDuq5DYoxI13VIyQUppSApJqSjQ0ouDByDEALivWdgDLz3SC4ZceYQHzxiZgwKg0JhUBjknDkmMBlVYDKqwGRUgf+nUgpiGNJ1HbLb75AqVEhVVRxTz2qOceaQPvYMPANzhhiGOOcQK4aYGYPCe8agUBgUBoXChwQmowpMRhWYjOp/AZyEIZ6WGjKCAAAAAElFTkSuQmCC", "text/html": [ - "" + "" ] }, "metadata": {}, @@ -208,10 +208,10 @@ { "output_type": "execute_result", "data": { - "text/plain": "28×28 Array{RGB{Float64},2} with eltype ColorTypes.RGB{Float64}:\n RGB{Float64}(0.393745,0.391795,0.389989) … RGB{Float64}(0.398502,0.396556,0.394723)\n RGB{Float64}(0.393514,0.391564,0.389759) RGB{Float64}(0.398277,0.396331,0.3945)\n RGB{Float64}(0.392243,0.390292,0.388495) RGB{Float64}(0.397181,0.395234,0.393409)\n RGB{Float64}(0.387817,0.385862,0.38409) RGB{Float64}(0.392133,0.390182,0.388385)\n RGB{Float64}(0.388182,0.386227,0.384453) RGB{Float64}(0.395797,0.393849,0.392032)\n RGB{Float64}(0.391306,0.389354,0.387562) … RGB{Float64}(0.408658,0.406723,0.404833)\n RGB{Float64}(0.390903,0.388951,0.387161) RGB{Float64}(0.4053,0.403362,0.40149)\n RGB{Float64}(0.391884,0.389932,0.388137) RGB{Float64}(0.39008,0.388127,0.386341)\n RGB{Float64}(0.386918,0.384962,0.383195) RGB{Float64}(0.374358,0.372392,0.370695)\n RGB{Float64}(0.386182,0.384225,0.382462) RGB{Float64}(0.389279,0.387325,0.385544)\n ⋮ ⋱ \n RGB{Float64}(0.397011,0.395064,0.39324) RGB{Float64}(0.389742,0.387788,0.386005)\n RGB{Float64}(0.400715,0.398772,0.396927) … RGB{Float64}(0.386969,0.385013,0.383246)\n RGB{Float64}(0.394853,0.392904,0.391092) RGB{Float64}(0.389283,0.387329,0.385548)\n RGB{Float64}(0.391859,0.389907,0.388112) RGB{Float64}(0.397595,0.395649,0.393821)\n RGB{Float64}(0.392543,0.390592,0.388793) RGB{Float64}(0.388837,0.386883,0.385105)\n RGB{Float64}(0.389707,0.387753,0.38597) RGB{Float64}(0.39158,0.389628,0.387834)\n RGB{Float64}(0.391255,0.389303,0.387511) … RGB{Float64}(0.386068,0.384111,0.382349)\n RGB{Float64}(0.392127,0.390176,0.388379) RGB{Float64}(0.386904,0.384948,0.383181)\n RGB{Float64}(0.394199,0.39225,0.390441) RGB{Float64}(0.392075,0.390123,0.388327)", - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAIAAABJgmMcAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAACmxJREFUeAHtwd2OHMd9xuFfVf17ej53uVyaEpcBLMqxZMABFCOGgBjxFTinzg3EtxhfhE4UCRAQI6JgEd5dguTucnd6pqc/qipHb4sHAxgB+rCfx/70p//kqMxRmYykmJCUEpJS4hgzQ5x3SPCBgeMDDsk5ISkmJOWEWDAkhIA45xDnHJLJDDKDnDOSc+b/w5iMypiMypiMyviAwyEuOAaZnzh+khl0XYe0XYuklBALhhSzAunokXlZIilHpO97JKeM9LFHUkhIURSIFYaYGX9PiglJKSGZjOScGWQGxmRUxmRUxmRUxgcyGXE4Bo6Bc46B4yjvPZJiYuAYOBxSFAVSzAokpYS0TYvEGJE+9kjX9UhKCXHOIY8ePeIYh0MOzQGp65qjMoNMRozJqIzJqIzJqCynjGQyEvuIZDJSWIHklJH5fI50XYe0bYs0hwap9zWyXC2Rvu+RtmsR5xzStC3SHA5IsICU5Qzx3iMpJuTQHJAYI9J3PVLXNeK8QworEOccYkxGZUxGZUxGZXwg54x0XccxIQRkuVoiq+UKyTkjs9kMWS6XSFVVSN/3SEoJadsO8d4jXdsih+aALNwCebh/QHbVDrl7f4c8PGyRWVEgRVEgwQKymC8Q7z3HGJNRGZNRGZNRmXMOcdkhOWck5YR45xGHQ5xzSEwRWa1XyGa9QbquQ7bVFsk5I2/fvkVSjEiMEdmcbJAQApJzRrq+Q9quRUIIyHK1QgozxMwQKwwJITBwDIzJqIzJqIzJqIwP+OCRcl4i3nkGjkF9qJGmbTim6zrk+uoauby6Qm5vb5CimCGHwwH52c+eIL/4x18gT548QcwM2e/3SLWtkGpXIW3TInW9Z7BYIDFGJISA+JlHYoyIMRmVMRmVMRmV8YHCCqSYFUhhBZJzRvb1nr+nqnbI1eUl8uOrH5GcMvLLzz5DXrx4gTx/foH86vNfIedPzpH1aoVcXV0h725ukJffv0R+fPUKaZoGqaoKcTgkWECcc0jmJ8ZkVMZkVMZkVJZzRmKKiOsdst/vEQuG9LFH+r5HqmqHpBSRzWaD/PZffoucPjpFLi4ukNPTU6Tve8QKQ9arFeJDQFarFXJ7e4csFgvk2bOPkcu/XTJwDOp9jex3e8ThEB8CYkxGZUxGZUxGZZmfpJSQ/X6P1HWNrFdrZHOyQR6fPUY2Jxvk9OQUWa6WSN/1yHa7RYrCkL/+9UfkYfuA3N/fI19//TVy8ewCOTQHZLFYMHAM1qs18vj8MXJ7e4ucnJ4g3nukKArEeYcYk1EZk1EZk1GZc45j6rpG7u7ukP1uj9SHGjk7O0PKskSKwpCmaZBdtUOur6+Rw+GAXF5dIbc3N8i+rhELASnLEnl/f4/85jf/jDx//hyZl3Pk2cUz5OryCum6Dtlut0gIAen6DjEmozImozImozLvHeKcQ9brNZJTRnzwyOnJKWJmyN3tHXJ5eYnsd3vk/f17pDk0yM3tDbKrdsj3L18ir1+/RkIISIwRWSwWyNnZI2Q2myGr1Qrp2o5j9vUecd4hKSfEOYcYk1EZk1EZk1EZHwghIOWsRAorkL7vkYftFvn2m2+RbbVFVssVUu0q5Jv//gb5+Sc/R6pqh1xcPEM+++yXyO9+96/Il19+iZSzEtls1kjOGfmfv/wFeffuHeK9R25vbxEzQ5bLJeKcQ4IPiDEZlTEZlTEZleWUGWQGIQRkVs6Qtm2Ruq6RsiyR58+fI6vVEvn888+Rf//DH5CUM/Lu3Tvk+voa6fuI/NOvf4388T/+iNR1jQTvka+++gq5uLhAvvvuO6SqKqTaVkhZlkhKCZnP50hRFIgxGZUxGZUxGZWFEBAfPDKfzxHnHZJSQs6fnCM5Z+TFpy+QL774Avn97/8Nuby8RL755lvEe49UVYVUVYU0bYO8fPkSyTkjTdMg19evkUenp8jpySmy2+2Quq6R+4cH5KOnTxGH4xhjMipjMipjMirz3iMWDCnnJeKcQxyOY7quQ2KMyGazQV6+/AH5/n+/R9q2RV79+Aq5/Nslcv36NfLw8IDc3t4iHz39CLm/v0dOTk+Q29s75M3bN0jXdkhRFMjmZIOYGcc4HGJMRmVMRmVMRmU5ZwaOQc4ZSTEidV0jNzc3SFEUSDErkD//15+RTz99gfQxIu/evUNC8MihOSDL5RJZzBfIfr9HXr16hdze3SFt2yC7aodUVYVYUSDL5QKxYEgxKxALhjjvEWMyKmMyKmMyKstkxDmH1HWN7Hd7ZF/vkTdv3yDbbYVcXV8hZ4/OkOvraySmiPzw8gfkzds3SEoJWa/WyM3NDbJaLZG+65EYI8eU8zmyWCyQs7MzpCxLJOeMLBYLpO1axDkGxmRUxmRUxmRURmYQ+4g455Cu75BDfUBijEjTNMjtTYe8vn6NbDYb5Pr1a6Tve2SzXiOHpkHWqzVyd3uL3L9/j8SUkLZtkI8/+hgpUkTK+RzJOSOL5QIJISA5ZQYdg5wyYkxGZUxGZUxGZSknJMaI9LFHmqZF6rpGdrsdErxHvPfIer1G2q5DiqJAzs7OkBgjcn5+jqxWK+Ty6gpZLZdIfzgg54/PkdPTU+Tx48fI06dPkZOTE45p2gZpmgYJPiApJ8SYjMqYjMqYjMqcc0hMEUkxMcgZscKQ1WqFLOYLxMyQtmuRel8js6JANicbpGs75Pz8HMk5IxcXF0hRGNJ1PTIrCuT5PzxHNpsNcnZ2hiyXS6Te75HdrkdyykjKmUFmYExGZUxGZUxGZTllJOWE9H2PHJoD4pxDzAqO6boOub9/QHJKyGKxYJAZlGXJwDFYLpfIs2cfI+WsRDYnG8TMkOViiTw6e4R475FdtUMyGZnP50jTHJC29UhMETEmozImozImo7KcMxJTRMwMmc1myH6/RywEpI8ROdQ1EmNETk9OkJQTMitLxEJAlssl8sknnyBt2yLOOWRezpHdfofs93vkzZs3yGKxQIqiQNarNfLw8IB4HxArDMldRozJqIzJqIzJqAzHIPiAxBQR7z2ymC+QGCMS24Ss12tkvVkj69UaSSkhu/0eWa2WiJkhzjmkbVpkuVoiTdMgdV0jTdMgzjmkbVsk+IBU2wrp2g6JKSJd2yEpJ8SYjMqYjMqYjMpyzhyTU+YYHzwyK2ZIUcyQk5MNMpvNkJwz0rUdEswQh0NSysj79++REAJSVRXivUfqukZ2ux2SUkLm8znycP+ALFdLpO96JMbIwDFw2SHGZFTGZFTGZFTmcEgmI957Bo5B3/fIfD5HilmBlGWJFGZICIY89A9I17ZIOS+Rrm2R7XaLxD4iVhiSUkLapkWapkFiH5G+65GyLJFqWyEhBI7JZCTlhBiTURmTURmTURkfcDgGnqNCCIjDIWaGpJiQQ98gIfRIzhlZrpZI8AHxwSM5ZySTkaZpEOcc0seI5JyR2WyGWGEc45xDYowMHEc5HGJMRmVMRmVMRmXOOSSTGSR+4hh47xGHQ3LKSEwRCSEgOWdkVsyQ6COSyUjf90jOGXE4JJMR7z0ymxVIURjH5JwR7zzHOOcQ5xwDx08yA2MyKmMyKmMyqv8DkxxUUp5yvhcAAAAASUVORK5CYII=", + "text/plain": "28×28 Array{RGB{Float64},2} with eltype ColorTypes.RGB{Float64}:\n RGB{Float64}(0.37343,0.371463,0.369772) … RGB{Float64}(0.374107,0.372141,0.370446)\n RGB{Float64}(0.371234,0.369266,0.367587) RGB{Float64}(0.3726,0.370633,0.368946)\n RGB{Float64}(0.374165,0.372199,0.370504) RGB{Float64}(0.371492,0.369524,0.367844)\n RGB{Float64}(0.36935,0.36738,0.365712) RGB{Float64}(0.370127,0.368158,0.366485)\n RGB{Float64}(0.373324,0.371358,0.369667) RGB{Float64}(0.376335,0.37437,0.372663)\n RGB{Float64}(0.375408,0.373443,0.37174) … RGB{Float64}(0.381851,0.379891,0.378152)\n RGB{Float64}(0.372697,0.37073,0.369043) RGB{Float64}(0.375581,0.373616,0.371913)\n RGB{Float64}(0.369287,0.367318,0.36565) RGB{Float64}(0.369729,0.36776,0.36609)\n RGB{Float64}(0.371845,0.369877,0.368195) RGB{Float64}(0.355707,0.35373,0.352138)\n RGB{Float64}(0.370929,0.36896,0.367283) RGB{Float64}(0.355765,0.353788,0.352195)\n ⋮ ⋱ \n RGB{Float64}(0.381908,0.379948,0.378209) RGB{Float64}(0.370624,0.368656,0.36698)\n RGB{Float64}(0.383097,0.381137,0.379392) … RGB{Float64}(0.367911,0.365941,0.36428)\n RGB{Float64}(0.377367,0.375403,0.37369) RGB{Float64}(0.370527,0.368558,0.366883)\n RGB{Float64}(0.375377,0.373412,0.37171) RGB{Float64}(0.374331,0.372365,0.370669)\n RGB{Float64}(0.375946,0.373981,0.372276) RGB{Float64}(0.369621,0.367652,0.365982)\n RGB{Float64}(0.369526,0.367557,0.365888) RGB{Float64}(0.368147,0.366177,0.364516)\n RGB{Float64}(0.370186,0.368217,0.366544) … RGB{Float64}(0.370786,0.368818,0.367142)\n RGB{Float64}(0.370592,0.368624,0.366948) RGB{Float64}(0.371356,0.369388,0.367708)\n RGB{Float64}(0.371014,0.369045,0.367368) RGB{Float64}(0.371707,0.369739,0.368058)", + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHAAAABwCAIAAABJgmMcAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAACmFJREFUeAHtwc1yHFWexuHfOXnyo1RSSbJsYxuwvQHGBBFM9GyhL2X6QgluoImeHpqeiBls4w6LlozRZ31mVuY5/1m9KRbVwSaX+TzhT3/6T3pGzzB6xk4xJSTFiJgZYtwJWYY475DMZ4j3HjGMXcwMSSkhWZYh3nvEO48475AYIz2jF1NEHI7fYxgSGA0qMBpUYDSogLGT9x4xM3pGzywhXdchMUbEMCSFgIQQEO89kiwhMUYkdhFp2xYxDKmqCsnzHMlChnjv+T0+eSSmyC6WjF0Co0EFRoMKjAYVcPScc4j3nl3MjF5Lr+s6xDCk6zqkLEokyzKkyAskCwHZbNZI0zVIs20QS4aYGZJiQvyeRw4PDxGfeXZp6gZZrVdIjBExM3YJjAYVGA0qMBpUMDMkJUPabcsuIQQkpYSEEJC2bZGUErJcrbhjyHRvijjvkRQjYtwxo7dtWyRZQtpti5RViSRLyLbesst6vUbquka88/Sc444hgdGgAqNBBUaDCs45JMWItG2LOO+Qyd4EqSYVMqkmSEwR2W63iCVDVqsVPUcvZAHZ1BukazukaRqkrmukKkvEcnq/vv8Vubm+QeaLORKygOR5jkwmE6QoC6QsSsTMkMBoUIHRoAKjQQWHQ7z3SAiBnqNX1w1SFgXi9hxS5iXivUeqskL29/eR09NTpHUt8urVKyQLAfHOIZPJhF1WqyWSUkIynyEhBCQvcqQsS8R7j+QhR3zmkZQSEhgNKjAaVGA0qGAYu5RlicQYkRgjUjc10vy6RQ7295FNvUF+unyDnP3zn8jbt2+R+WKBHB8dIS9evEA+/PBD5P6D+8hkMkGurq6Q+e0cWSwWiPce2W63SJ1qZG+6h5gZPaNnyZDAaFCB0aACo0EFM0OykCHOOXqRXhc7pCxL5ODgAMnzHGm7Fnn18iXy+qefEOcc8uWXXyL//uWXyOeff458+uknyNHxMfL48WPkhx9+QK6urpDvvvsOOT09RbrYIdbRS5aQEAI7OXqB0aACo0EFRoMKDoc4HBK7iCyWC8SM3rbZItvtFknJkBQjsjedIn/849fIRx99jHz22afI8+fPkaZpkJgScnx8jGw2G6QsCsRSQmazGXJ0eISELCAxRmTbtkjXdUiWZfSMXmA0qMBoUIHRoAL/wna7Req6RoqiQMpqgjx69Ah59uwZMpvNEO88MplMkOuba6QqS+T07VtksVgir169Qr799lvkyZMnyGQyQbquQ5bLJXJ4dIiYGbJer5EsZIiZ0TN2CowGFRgNKjAaVDAzpOs6pG1bZLOpkXpTI+v1GinLArl3fIyURYlsNhtktVoh79+/R27nc+Ty8gJp6gZZLBbI48ePkbOzM+Qfb/6BfPHFF8jh4SFycHCAPHv6DDk7O0MWywXSNA2SZRkSY0QCo0EFRoMKjAYVnHOI9x45OjpCyrJENvUGmR3MkKqqkLdvT5H//b8fEUsJubq+RrxzyMXFBbLZbJDXr18jbdch/vvvkbZtETNDiqJAXnz+AsmLHIk3EUkp0TN6zjkkpYRkWYYERoMKjAYVGA0qOO+QoiiQoiiQEAIymUyQzWaDXF5dInXdIL++f4+cnZ8jFxcXyGeffkrPOeTjjz9Cnjx5gjx48AD5w3/8AVksFkjbtsjx8TFyfnaO1HWNXF/fIJeXF4j3GTKpKsR7j2Q+QwKjQQVGgwqMBhXAIckS4pxD9vb2EDNDirJApvtT5PDwEIldRJ4+e4osF0ukmlTIL+9+QeaLOVKWJXLv+B7y9ddfI8455Pz8HHnz0xtkOp0iP/74I1I3NXJ5eYnkeY4cHMyQSVUhzjkkMBpUYDSowGhQwTvHLjFFeh07ZVnGLp988gny1VdfIS9e/Bvy97//D/LmzRukbVvk5uYGSTEhXeyQq6srZLVcIqvVGrm+vkbm8znSti1yc3OLLBZLpCgKpCorpCgKJKaIBEaDCowGFRgNKmRZhpRFiVSTCtk2W2S9WSNt2yJFXiDL5RIxM+T8/B1yfnaOvHv3Drm9vUViisjr16+RBw8eIH8u/ozcu3cP+fnnn5F6UyPzxRw5OztD6qZG8jxH9vb2kKqqEO8cYsmQwGhQgdGgAqNBBTNjlxQT0mwbZL1eI7e3c+TgYB+5vrpGvvnmG+T4+Bg5uXcPmc1myC/vfkFevnyJTKdTZH9/H1mv1sjZ2Rk9o3d1dYVcXFwgt7e3yL2TE+T4+BipqhIJISApJXYJjAYVGA0qMBpUMAyJMSJd1yGWjDsOWa9XyPv375G6rpFHjx4hjx49Qv72/d+Qv/zXX5DXr39Cuq5DHn3wAbJYLpCTk/vIzc0NUhQFcnNzg+QhII8fP0aOjo+Q2WyGeO8R7z1S1zViRi8wGlRgNKjAaFAhpYQYd5xzSIwRaZoaqesaWa9WyPXVNTKfz5G//vW/kdPTU+T29hY5OjpCJlWFxJSQy8srZLlcIdumQQxD8rxAyrJEZrMZsjfZQ7z3SMgCElNEUkyIzzwSGA0qMBpUYDSowG90XcsdQ1arFbJarpDYReRgdoAkS4i1hsQYka7rkKdPnyJFUSBVWSLT6RS5vLxEnHNIlmXIdDpF7p2cIAcH+8jB/gEy3Z8iZVkiXdshbdsiKSUkdhEJjAYVGA0qMBpUcDgkxoSkuEW6GOk5etP9KTKZ7CFFniO38zlysL+PPH/+HJlUFZIsIQ8fPESKokCKskBub+fIBwcHyHQ6RU7unyB5niMnJyfI4ewQWS6XyHwxR6w1xMwQM0MCo0EFRoMKjAYVkiV6Ri+ZIV3bItttizTNlp7Ra7IM2TYNkoWAZFmGrNZr5OHDh8jDDx4i90/uI9fX14j3HtnUG3pGbzabIUeHR8jscIZ475G6qZE8z5HNesNOziGB0aACo0EFRoMKGD3jTrKEZFmGhBDYpW4apChyxHmHHOzvI23XInuTPcQ5h1xfXyMP7j9ADg8PkWQJqaoKqesa6doOSZaQ5XKJ5HmOdF2HdG2HZCFDilQgdWqQwGhQgdGgAqNBBX7LDDEzJM9zZH9/ipRFgaSUkLKqkCzzyKSaIJvNBpkvFkiR50jaJOT65hrx3iP1pmYnR6/dtshqtUK894iZIXme03P0UkpI0zSIWUICo0EFRoMKjAYVkiXEkiEpGbuUZYlUZYU455CyKhEzQ4q8QLz3SNM0iPMOmVQTZLFYIJaMXZx3SNt2SNe1SNM0yHbbIoezGbI33UMcDjHupJTYJTAaVGA0qMBoUIHfcN4hHodkWY5YMnqOXpEXiPcZ4r1DYopI0zRIyHOkKkvEe4+kmJC2a5GQBcSiIZYSslwskZgSUpYF0nYt0jRbJPMecd7Rc/QcDgmMBhUYDSowGlRwOHZxznHH0XP8hiExRaSLHeKdR3zmEeccUlUlkoccSSmxS1mUiJkhMUbEZx6ZTqeImSFFUbCL406yRK+j57xjl8BoUIHRoAKjQQX+BYdjJ8dvOMTM6Bk9n3nEO49MJhMkxsgu3nvEzJAudoh3HvGZR8zoFWWBOBw9x+8yM8Q7jzgcuwRGgwqMBhUYDer/AR23UREPHg3qAAAAAElFTkSuQmCC", "text/html": [ - "" + "" ] }, "metadata": {}, diff --git a/dev/generated/augmentations/index.html b/dev/generated/augmentations/index.html index e83482b..1e93b31 100644 --- a/dev/generated/augmentations/index.html +++ b/dev/generated/augmentations/index.html @@ -20,15 +20,15 @@ convert2image(MNIST, x)

Noise augmentation

The NoiseAugmentation wrapper computes explanations averaged over noisy inputs. Let's demonstrate this on the Gradient analyzer. First, we compute the heatmap of an explanation without augmentation:

analyzer = Gradient(model)
 heatmap(input, analyzer)

Now we wrap the analyzer in a NoiseAugmentation with 10 samples of noise. By default, the noise is sampled from a Gaussian distribution with mean 0 and standard deviation 1.

analyzer = NoiseAugmentation(Gradient(model), 50)
-heatmap(input, analyzer)

Note that a higher sample size is desired, as it will lead to a smoother heatmap. However, this comes at the cost of a longer computation time.

We can also set the standard deviation of the Gaussian distribution:

analyzer = NoiseAugmentation(Gradient(model), 50, 0.1)
-heatmap(input, analyzer)

When used with a Gradient analyzer, this is equivalent to SmoothGrad:

analyzer = SmoothGrad(model, 50)
-heatmap(input, analyzer)

We can also use any distribution from Distributions.jl, for example Poisson noise with rate $\lambda=0.5$:

using Distributions
+heatmap(input, analyzer)

Note that a higher sample size is desired, as it will lead to a smoother heatmap. However, this comes at the cost of a longer computation time.

We can also set the standard deviation of the Gaussian distribution:

analyzer = NoiseAugmentation(Gradient(model), 50, 0.1)
+heatmap(input, analyzer)

When used with a Gradient analyzer, this is equivalent to SmoothGrad:

analyzer = SmoothGrad(model, 50)
+heatmap(input, analyzer)

We can also use any distribution from Distributions.jl, for example Poisson noise with rate $\lambda=0.5$:

using Distributions
 
 analyzer = NoiseAugmentation(Gradient(model), 50, Poisson(0.5))
-heatmap(input, analyzer)

Is is also possible to define your own distributions or mixture distributions.

NoiseAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.

Integration augmentation

The InterpolationAugmentation wrapper computes explanations averaged over n steps of linear interpolation between the input and a reference input, which is set to zero(input) by default:

analyzer = InterpolationAugmentation(Gradient(model), 50)
+heatmap(input, analyzer)

Is is also possible to define your own distributions or mixture distributions.

NoiseAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.

Integration augmentation

The InterpolationAugmentation wrapper computes explanations averaged over n steps of linear interpolation between the input and a reference input, which is set to zero(input) by default:

analyzer = InterpolationAugmentation(Gradient(model), 50)
 heatmap(input, analyzer)

When used with a Gradient analyzer, this is equivalent to IntegratedGradients:

analyzer = IntegratedGradients(model, 50)
 heatmap(input, analyzer)

To select a different reference input, pass it to the analyze function using the keyword argument input_ref. Note that this is an arbitrary example for the sake of demonstration.

matrix_of_ones = ones(Float32, size(input))
 
 analyzer = InterpolationAugmentation(Gradient(model), 50)
 expl = analyzer(input; input_ref=matrix_of_ones)
-heatmap(expl)

Once again, InterpolationAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.

This page was generated using Literate.jl.

+heatmap(expl)

Once again, InterpolationAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.

This page was generated using Literate.jl.

diff --git a/dev/generated/example/index.html b/dev/generated/example/index.html index 116979d..491af8a 100644 --- a/dev/generated/example/index.html +++ b/dev/generated/example/index.html @@ -43,4 +43,4 @@ heatmap(expl)

This heatmap shows us that the "upper loop" of the hand-drawn 9 has negative relevance with respect to the output neuron corresponding to digit 4!

Note

The output neuron can also be specified when calling heatmap:

heatmap(input, analyzer, 5)

Analyzing batches

ExplainableAI also supports explanations of input batches:

batchsize = 20
 xs, _ = MNIST(Float32, :test)[1:batchsize]
 batch = reshape(xs, 28, 28, 1, :) # reshape to WHCN format
-expl = analyze(batch, analyzer);

This will return a single Explanation expl for the entire batch. Calling heatmap on expl will detect the batch dimension and return a vector of heatmaps.

heatmap(expl)
(a vector displayed as a row to save space)

For more information on heatmapping batches, refer to the heatmapping documentation.

This page was generated using Literate.jl.

+expl = analyze(batch, analyzer);

This will return a single Explanation expl for the entire batch. Calling heatmap on expl will detect the batch dimension and return a vector of heatmaps.

heatmap(expl)
(a vector displayed as a row to save space)

For more information on heatmapping batches, refer to the heatmapping documentation.

This page was generated using Literate.jl.

diff --git a/dev/generated/heatmapping/index.html b/dev/generated/heatmapping/index.html index cdf8de6..3371ab3 100644 --- a/dev/generated/heatmapping/index.html +++ b/dev/generated/heatmapping/index.html @@ -31,4 +31,4 @@ mosaic(heatmaps; nrow=10)Example block output
Output type consistency

To obtain a singleton Vector containing a single heatmap for non-batched inputs, use the heatmap keyword argument unpack_singleton=false.

Processing heatmaps

Heatmapping makes use of the Julia-based image processing ecosystem Images.jl.

If you want to further process heatmaps, you may benefit from reading about some fundamental conventions that the ecosystem utilizes that are different from how images are typically represented in OpenCV, MATLAB, ImageJ or Python.

Saving heatmaps

Since heatmaps are regular Images.jl images, they can be saved as such:

using FileIO
 
 img = heatmap(input, analyzer)
-save("heatmap.png", img)

This page was generated using Literate.jl.

+save("heatmap.png", img)

This page was generated using Literate.jl.

diff --git a/dev/index.html b/dev/index.html index 5f323f8..de0e9dd 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -Home · ExplainableAI.jl
GitHub

ExplainableAI.jl

Explainable AI in Julia.

Installation

To install this package and its dependencies, open the Julia REPL and run 

julia> ]add ExplainableAI

Manual

General usage

API reference

+Home · ExplainableAI.jl
GitHub

ExplainableAI.jl

Explainable AI in Julia.

Installation

To install this package and its dependencies, open the Julia REPL and run 

julia> ]add ExplainableAI

Manual

General usage

API reference

diff --git a/dev/search_index.js b/dev/search_index.js index d40eefc..6ba952f 100644 --- a/dev/search_index.js +++ b/dev/search_index.js @@ -1,3 +1,3 @@ var documenterSearchIndex = {"docs": -[{"location":"api/#Basic-API","page":"API Reference","title":"Basic API","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"All methods in ExplainableAI.jl work by calling analyze on an input and an analyzer:","category":"page"},{"location":"api/","page":"API Reference","title":"API Reference","text":"analyze\nExplanation\nheatmap","category":"page"},{"location":"api/#XAIBase.analyze","page":"API Reference","title":"XAIBase.analyze","text":"analyze(input, method)\nanalyze(input, method, neuron_selection)\n\nApply the analyzer method for the given input, returning an Explanation. If neuron_selection is specified, the explanation will be calculated for that neuron. Otherwise, the output neuron with the highest activation is automatically chosen.\n\nSee also Explanation and heatmap.\n\nKeyword arguments\n\nadd_batch_dim: add batch dimension to the input without allocating. Default is false.\n\n\n\n\n\n","category":"function"},{"location":"api/#XAIBase.Explanation","page":"API Reference","title":"XAIBase.Explanation","text":"Return type of analyzers when calling analyze.\n\nFields\n\nval: numerical output of the analyzer, e.g. an attribution or gradient\noutput: model output for the given analyzer input\noutput_selection: index of the output used for the explanation\nanalyzer: symbol corresponding the used analyzer, e.g. :Gradient or :LRP\nheatmap: symbol indicating a preset heatmapping style, e.g. :attibution, :sensitivity or :cam\nextras: optional named tuple that can be used by analyzers to return additional information.\n\n\n\n\n\n","category":"type"},{"location":"api/#XAIBase.heatmap","page":"API Reference","title":"XAIBase.heatmap","text":"heatmap(explanation)\n\nVisualize Explanation from XAIBase as a vision heatmap. Assumes WHCN convention (width, height, channels, batchsize) for explanation.val.\n\nKeyword arguments\n\ncolorscheme::Union{ColorScheme,Symbol}: color scheme from ColorSchemes.jl. Defaults to :seismic.\nreduce::Symbol: Selects how color channels are reduced to a single number to apply a color scheme. The following methods can be selected, which are then applied over the color channels for each \"pixel\" in the array:\n:sum: sum up color channels\n:norm: compute 2-norm over the color channels\n:maxabs: compute maximum(abs, x) over the color channels\nDefaults to :sum.\nrangescale::Symbol: Selects how the color channel reduced heatmap is normalized before the color scheme is applied. Can be either :extrema or :centered. Defaults to :centered.\nprocess_batch::Bool: When heatmapping a batch, setting process_batch=true will apply the rangescale normalization to the entire batch instead of computing it individually for each sample in the batch. Defaults to false.\npermute::Bool: Whether to flip W&H input channels. Default is true.\nunpack_singleton::Bool: If false, heatmap will always return a vector of images. When heatmapping a batch with a single sample, setting unpack_singleton=true will unpack the singleton vector and directly return the image. Defaults to true.\n\n\n\n\n\nheatmap(input, analyzer)\n\nCompute an Explanation for a given input using the method analyzer and visualize it as a vision heatmap.\n\nAny additional arguments and keyword arguments are passed to the analyzer. Refer to analyze for more information on available keyword arguments.\n\nTo customize the heatmapping style, first compute an explanation using analyze and then call heatmap on the explanation.\n\n\n\n\n\nheatmap(explanation, text)\n\nVisualize Explanation from XAIBase as text heatmap. Text should be a vector containing vectors of strings, one for each input in the batched explanation.\n\nKeyword arguments\n\ncolorscheme::Union{ColorScheme,Symbol}: color scheme from ColorSchemes.jl. Defaults to :seismic.\nrangescale::Symbol: selects how the color channel reduced heatmap is normalized before the color scheme is applied. Can be either :extrema or :centered. Defaults to :centered for use with the default color scheme :seismic.\n\n\n\n\n\n","category":"function"},{"location":"api/#Analyzers","page":"API Reference","title":"Analyzers","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"Gradient\nInputTimesGradient\nSmoothGrad\nIntegratedGradients","category":"page"},{"location":"api/#ExplainableAI.Gradient","page":"API Reference","title":"ExplainableAI.Gradient","text":"Gradient(model)\n\nAnalyze model by calculating the gradient of a neuron activation with respect to the input.\n\n\n\n\n\n","category":"type"},{"location":"api/#ExplainableAI.InputTimesGradient","page":"API Reference","title":"ExplainableAI.InputTimesGradient","text":"InputTimesGradient(model)\n\nAnalyze model by calculating the gradient of a neuron activation with respect to the input. This gradient is then multiplied element-wise with the input.\n\n\n\n\n\n","category":"type"},{"location":"api/#ExplainableAI.SmoothGrad","page":"API Reference","title":"ExplainableAI.SmoothGrad","text":"SmoothGrad(analyzer, [n=50, std=0.1, rng=GLOBAL_RNG])\nSmoothGrad(analyzer, [n=50, distribution=Normal(0, σ²=0.01), rng=GLOBAL_RNG])\n\nAnalyze model by calculating a smoothed sensitivity map. This is done by averaging sensitivity maps of a Gradient analyzer over random samples in a neighborhood of the input, typically by adding Gaussian noise with mean 0.\n\nReferences\n\nSmilkov et al., SmoothGrad: removing noise by adding noise\n\n\n\n\n\n","category":"function"},{"location":"api/#ExplainableAI.IntegratedGradients","page":"API Reference","title":"ExplainableAI.IntegratedGradients","text":"IntegratedGradients(analyzer, [n=50])\nIntegratedGradients(analyzer, [n=50])\n\nAnalyze model by using the Integrated Gradients method.\n\nReferences\n\nSundararajan et al., Axiomatic Attribution for Deep Networks\n\n\n\n\n\n","category":"function"},{"location":"api/#Input-augmentations","page":"API Reference","title":"Input augmentations","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"SmoothGrad and IntegratedGradients are special cases of the input augmentations NoiseAugmentation and InterpolationAugmentation, which can be applied as a wrapper to any analyzer:","category":"page"},{"location":"api/","page":"API Reference","title":"API Reference","text":"NoiseAugmentation\nInterpolationAugmentation","category":"page"},{"location":"api/#ExplainableAI.NoiseAugmentation","page":"API Reference","title":"ExplainableAI.NoiseAugmentation","text":"NoiseAugmentation(analyzer, n, [std=1, rng=GLOBAL_RNG])\nNoiseAugmentation(analyzer, n, distribution, [rng=GLOBAL_RNG])\n\nA wrapper around analyzers that augments the input with n samples of additive noise sampled from distribution. This input augmentation is then averaged to return an Explanation.\n\n\n\n\n\n","category":"type"},{"location":"api/#ExplainableAI.InterpolationAugmentation","page":"API Reference","title":"ExplainableAI.InterpolationAugmentation","text":"InterpolationAugmentation(model, [n=50])\n\nA wrapper around analyzers that augments the input with n steps of linear interpolation between the input and a reference input (typically zero(input)). The gradients w.r.t. this augmented input are then averaged and multiplied with the difference between the input and the reference input.\n\n\n\n\n\n","category":"type"},{"location":"api/#Input-preprocessing","page":"API Reference","title":"Input preprocessing","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"preprocess_imagenet","category":"page"},{"location":"api/#ExplainableAI.preprocess_imagenet","page":"API Reference","title":"ExplainableAI.preprocess_imagenet","text":"preprocess_imagenet(img)\n\nPreprocess an image for use with Metalhead.jl's ImageNet models using PyTorch weights. Uses PyTorch's normalization constants.\n\n\n\n\n\n","category":"function"},{"location":"api/#Index","page":"API Reference","title":"Index","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"EditURL = \"../literate/heatmapping.jl\"","category":"page"},{"location":"generated/heatmapping/#docs-heatmapping","page":"Heatmapping","title":"Heatmapping","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Since numerical explanations are not very informative at first sight, we can visualize them by computing a heatmap. This page showcases different options and preset for heatmapping, building on the basics shown in the Getting started section.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"We start out by loading the same pre-trained LeNet5 model and MNIST input data:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"using ExplainableAI\nusing Flux\n\nusing BSON # hide\nmodel = BSON.load(\"../model.bson\", @__MODULE__)[:model] # hide\nmodel","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"using MLDatasets\nusing ImageCore, ImageIO, ImageShow\n\nindex = 10\nx, y = MNIST(Float32, :test)[10]\ninput = reshape(x, 28, 28, 1, :)\n\nconvert2image(MNIST, x)","category":"page"},{"location":"generated/heatmapping/#Automatic-heatmap-presets","page":"Heatmapping","title":"Automatic heatmap presets","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"The function heatmap automatically applies common presets for each method.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Since InputTimesGradient computes attributions, heatmaps are shown in a blue-white-red color scheme. Gradient methods however are typically shown in grayscale:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"analyzer = Gradient(model)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"analyzer = InputTimesGradient(model)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/heatmapping/#Custom-heatmap-settings","page":"Heatmapping","title":"Custom heatmap settings","text":"","category":"section"},{"location":"generated/heatmapping/#Color-schemes","page":"Heatmapping","title":"Color schemes","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"We can partially or fully override presets by passing keyword arguments to heatmap. For example, we can use a custom color scheme from ColorSchemes.jl using the keyword argument colorscheme:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"using ColorSchemes\n\nexpl = analyze(input, analyzer)\nheatmap(expl; colorscheme=:jet)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; colorscheme=:inferno)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Refer to the ColorSchemes.jl catalogue for a gallery of available color schemes.","category":"page"},{"location":"generated/heatmapping/#docs-heatmap-reduce","page":"Heatmapping","title":"Color channel reduction","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Explanations have the same dimensionality as the inputs to the classifier. For images with multiple color channels, this means that the explanation also has a \"color channel\" dimension.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"The keyword argument reduce can be used to reduce this dimension to a single scalar value for each pixel. The following presets are available:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":":sum: sum up color channels (default setting)\n:norm: compute 2-norm over the color channels\n:maxabs: compute maximum(abs, x) over the color channels","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; reduce=:sum)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; reduce=:norm)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; reduce=:maxabs)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"In this example, the heatmaps look identical. Since MNIST only has a single color channel, there is no need for color channel reduction.","category":"page"},{"location":"generated/heatmapping/#docs-heatmap-rangescale","page":"Heatmapping","title":"Mapping explanations onto the color scheme","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"To map a color-channel-reduced explanation onto a color scheme, we first need to normalize all values to the range 0 1.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"For this purpose, two presets are available through the rangescale keyword argument:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":":extrema: normalize to the minimum and maximum value of the explanation\n:centered: normalize to the maximum absolute value of the explanation. Values of zero will be mapped to the center of the color scheme.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Depending on the color scheme, one of these presets may be more suitable than the other. The default color scheme for InputTimesGradient, seismic, is centered around zero, making :centered a good choice:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; rangescale=:centered)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; rangescale=:extrema)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"However, for the inferno color scheme, which is not centered around zero, :extrema leads to a heatmap with higher contrast.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; rangescale=:centered, colorscheme=:inferno)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; rangescale=:extrema, colorscheme=:inferno)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"For the full list of heatmap keyword arguments, refer to the heatmap documentation.","category":"page"},{"location":"generated/heatmapping/#docs-heatmapping-batches","page":"Heatmapping","title":"Heatmapping batches","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Heatmapping also works with input batches. Let's demonstrate this by using a batch of 100 images from the MNIST dataset:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"xs, ys = MNIST(Float32, :test)[1:100]\nbatch = reshape(xs, 28, 28, 1, :); # reshape to WHCN format\nnothing #hide","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"The heatmap function automatically recognizes that the explanation is batched and returns a Vector of images:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmaps = heatmap(batch, analyzer)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Image.jl's mosaic function can used to display them in a grid:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"mosaic(heatmaps; nrow=10)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"When heatmapping batches, the mapping to the color scheme is applied per sample. For example, rangescale=:extrema will normalize each heatmap to the minimum and maximum value of each sample in the batch. This ensures that heatmaps don't depend on other samples in the batch.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"If this bevahior is not desired, heatmap can be called with the keyword-argument process_batch=true:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"expl = analyze(batch, analyzer)\nheatmaps = heatmap(expl; process_batch=true)\nmosaic(heatmaps; nrow=10)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"This can be useful when comparing heatmaps for fixed output neurons:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"expl = analyze(batch, analyzer, 7) # explain digit \"6\"\nheatmaps = heatmap(expl; process_batch=true)\nmosaic(heatmaps; nrow=10)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"note: Output type consistency\nTo obtain a singleton Vector containing a single heatmap for non-batched inputs, use the heatmap keyword argument unpack_singleton=false.","category":"page"},{"location":"generated/heatmapping/#Processing-heatmaps","page":"Heatmapping","title":"Processing heatmaps","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Heatmapping makes use of the Julia-based image processing ecosystem Images.jl.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"If you want to further process heatmaps, you may benefit from reading about some fundamental conventions that the ecosystem utilizes that are different from how images are typically represented in OpenCV, MATLAB, ImageJ or Python.","category":"page"},{"location":"generated/heatmapping/#Saving-heatmaps","page":"Heatmapping","title":"Saving heatmaps","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Since heatmaps are regular Images.jl images, they can be saved as such:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"using FileIO\n\nimg = heatmap(input, analyzer)\nsave(\"heatmap.png\", img)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"This page was generated using Literate.jl.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"EditURL = \"../literate/augmentations.jl\"","category":"page"},{"location":"generated/augmentations/#docs-augmentations","page":"Input augmentations","title":"Analyzer augmentations","text":"","category":"section"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"All analyzers implemented in ExplainableAI.jl can be augmented by two types of augmentations: NoiseAugmentations and InterpolationAugmentations. These augmentations are wrappers around analyzers that modify the input before passing it to the analyzer.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"We build on the basics shown in the Getting started section and start out by loading the same pre-trained LeNet5 model and MNIST input data:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"using ExplainableAI\nusing Flux\n\nusing BSON # hide\nmodel = BSON.load(\"../model.bson\", @__MODULE__)[:model] # hide\nmodel","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"using MLDatasets\nusing ImageCore, ImageIO, ImageShow\n\nindex = 10\nx, y = MNIST(Float32, :test)[10]\ninput = reshape(x, 28, 28, 1, :)\n\nconvert2image(MNIST, x)","category":"page"},{"location":"generated/augmentations/#Noise-augmentation","page":"Input augmentations","title":"Noise augmentation","text":"","category":"section"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"The NoiseAugmentation wrapper computes explanations averaged over noisy inputs. Let's demonstrate this on the Gradient analyzer. First, we compute the heatmap of an explanation without augmentation:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = Gradient(model)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"Now we wrap the analyzer in a NoiseAugmentation with 10 samples of noise. By default, the noise is sampled from a Gaussian distribution with mean 0 and standard deviation 1.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = NoiseAugmentation(Gradient(model), 50)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"Note that a higher sample size is desired, as it will lead to a smoother heatmap. However, this comes at the cost of a longer computation time.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"We can also set the standard deviation of the Gaussian distribution:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = NoiseAugmentation(Gradient(model), 50, 0.1)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"When used with a Gradient analyzer, this is equivalent to SmoothGrad:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = SmoothGrad(model, 50)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"We can also use any distribution from Distributions.jl, for example Poisson noise with rate lambda=05:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"using Distributions\n\nanalyzer = NoiseAugmentation(Gradient(model), 50, Poisson(0.5))\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"Is is also possible to define your own distributions or mixture distributions.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"NoiseAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.","category":"page"},{"location":"generated/augmentations/#Integration-augmentation","page":"Input augmentations","title":"Integration augmentation","text":"","category":"section"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"The InterpolationAugmentation wrapper computes explanations averaged over n steps of linear interpolation between the input and a reference input, which is set to zero(input) by default:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = InterpolationAugmentation(Gradient(model), 50)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"When used with a Gradient analyzer, this is equivalent to IntegratedGradients:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = IntegratedGradients(model, 50)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"To select a different reference input, pass it to the analyze function using the keyword argument input_ref. Note that this is an arbitrary example for the sake of demonstration.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"matrix_of_ones = ones(Float32, size(input))\n\nanalyzer = InterpolationAugmentation(Gradient(model), 50)\nexpl = analyzer(input; input_ref=matrix_of_ones)\nheatmap(expl)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"Once again, InterpolationAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"This page was generated using Literate.jl.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"EditURL = \"../literate/example.jl\"","category":"page"},{"location":"generated/example/#docs-getting-started","page":"Getting started","title":"Getting started","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"For this first example, we already have loaded a pre-trained LeNet5 model to look at explanations on the MNIST dataset.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"using ExplainableAI\nusing Flux\n\nusing BSON # hide\nmodel = BSON.load(\"../model.bson\", @__MODULE__)[:model] # hide\nmodel","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"note: Supported models\nExplainableAI.jl can be used on any differentiable classifier.","category":"page"},{"location":"generated/example/#Preparing-the-input-data","page":"Getting started","title":"Preparing the input data","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"We use MLDatasets to load a single image from the MNIST dataset:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"using MLDatasets\nusing ImageCore, ImageIO, ImageShow\n\nindex = 10\nx, y = MNIST(Float32, :test)[10]\n\nconvert2image(MNIST, x)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"By convention in Flux.jl, this input needs to be resized to WHCN format by adding a color channel and batch dimensions.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"input = reshape(x, 28, 28, 1, :);\nnothing #hide","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"note: Input format\nFor any explanation of a model, ExplainableAI.jl assumes the batch dimension to come last in the input.For the purpose of heatmapping, the input is assumed to be in WHCN order (width, height, channels, batch), which is Flux.jl's convention.","category":"page"},{"location":"generated/example/#Explanations","page":"Getting started","title":"Explanations","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"We can now select an analyzer of our choice and call analyze to get an Explanation:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"analyzer = InputTimesGradient(model)\nexpl = analyze(input, analyzer);\nnothing #hide","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"The return value expl is of type Explanation and bundles the following data:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"expl.val: numerical output of the analyzer, e.g. an attribution or gradient\nexpl.output: model output for the given analyzer input\nexpl.output_selection: index of the output used for the explanation\nexpl.analyzer: symbol corresponding the used analyzer, e.g. :Gradient or :LRP\nexpl.heatmap: symbol indicating a preset heatmapping style, e.g. :attibution, :sensitivity or :cam\nexpl.extras: optional named tuple that can be used by analyzers to return additional information.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"We used InputTimesGradient, so expl.analyzer is :InputTimesGradient.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"expl.analyzer","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"By default, the explanation is computed for the maximally activated output neuron. Since our digit is a 9 and Julia's indexing is 1-based, the output neuron at index 10 of our trained model is maximally activated.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"Finally, we obtain the result of the analyzer in form of an array.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"expl.val","category":"page"},{"location":"generated/example/#Heatmapping-basics","page":"Getting started","title":"Heatmapping basics","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"Since the array expl.val is not very informative at first sight, we can visualize Explanations by computing a heatmap:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"heatmap(expl)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"If we are only interested in the heatmap, we can combine analysis and heatmapping into a single function call:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"heatmap(input, analyzer)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"For a more detailed explanation of the heatmap function, refer to the heatmapping section.","category":"page"},{"location":"generated/example/#docs-analyzers-list","page":"Getting started","title":"List of analyzers","text":"","category":"section"},{"location":"generated/example/#Neuron-selection","page":"Getting started","title":"Neuron selection","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"By passing an additional index to our call to analyze, we can compute an explanation with respect to a specific output neuron. Let's see why the output wasn't interpreted as a 4 (output neuron at index 5)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"expl = analyze(input, analyzer, 5)\nheatmap(expl)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"This heatmap shows us that the \"upper loop\" of the hand-drawn 9 has negative relevance with respect to the output neuron corresponding to digit 4!","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"note: Note\nThe output neuron can also be specified when calling heatmap:heatmap(input, analyzer, 5)","category":"page"},{"location":"generated/example/#Analyzing-batches","page":"Getting started","title":"Analyzing batches","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"ExplainableAI also supports explanations of input batches:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"batchsize = 20\nxs, _ = MNIST(Float32, :test)[1:batchsize]\nbatch = reshape(xs, 28, 28, 1, :) # reshape to WHCN format\nexpl = analyze(batch, analyzer);\nnothing #hide","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"This will return a single Explanation expl for the entire batch. Calling heatmap on expl will detect the batch dimension and return a vector of heatmaps.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"heatmap(expl)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"For more information on heatmapping batches, refer to the heatmapping documentation.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"This page was generated using Literate.jl.","category":"page"},{"location":"","page":"Home","title":"Home","text":"CurrentModule = ExplainableAI","category":"page"},{"location":"#ExplainableAI.jl","page":"Home","title":"ExplainableAI.jl","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Explainable AI in Julia.","category":"page"},{"location":"#Installation","page":"Home","title":"Installation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"To install this package and its dependencies, open the Julia REPL and run ","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> ]add ExplainableAI","category":"page"},{"location":"#Manual","page":"Home","title":"Manual","text":"","category":"section"},{"location":"#General-usage","page":"Home","title":"General usage","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Pages = [\n \"generated/example.md\",\n \"generated/heatmapping.md\",\n \"generated/augmentations.md\",\n]\nDepth = 3","category":"page"},{"location":"#API-reference","page":"Home","title":"API reference","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Pages = [\"api.md\"]\nDepth = 2","category":"page"}] +[{"location":"api/#Basic-API","page":"API Reference","title":"Basic API","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"All methods in ExplainableAI.jl work by calling analyze on an input and an analyzer:","category":"page"},{"location":"api/","page":"API Reference","title":"API Reference","text":"analyze\nExplanation\nheatmap","category":"page"},{"location":"api/#XAIBase.analyze","page":"API Reference","title":"XAIBase.analyze","text":"analyze(input, method)\nanalyze(input, method, neuron_selection)\n\nApply the analyzer method for the given input, returning an Explanation. If neuron_selection is specified, the explanation will be calculated for that neuron. Otherwise, the output neuron with the highest activation is automatically chosen.\n\nSee also Explanation and heatmap.\n\nKeyword arguments\n\nadd_batch_dim: add batch dimension to the input without allocating. Default is false.\n\n\n\n\n\n","category":"function"},{"location":"api/#XAIBase.Explanation","page":"API Reference","title":"XAIBase.Explanation","text":"Return type of analyzers when calling analyze.\n\nFields\n\nval: numerical output of the analyzer, e.g. an attribution or gradient\noutput: model output for the given analyzer input\noutput_selection: index of the output used for the explanation\nanalyzer: symbol corresponding the used analyzer, e.g. :Gradient or :LRP\nheatmap: symbol indicating a preset heatmapping style, e.g. :attibution, :sensitivity or :cam\nextras: optional named tuple that can be used by analyzers to return additional information.\n\n\n\n\n\n","category":"type"},{"location":"api/#XAIBase.heatmap","page":"API Reference","title":"XAIBase.heatmap","text":"heatmap(explanation)\n\nVisualize Explanation from XAIBase as a vision heatmap. Assumes WHCN convention (width, height, channels, batchsize) for explanation.val.\n\nKeyword arguments\n\ncolorscheme::Union{ColorScheme,Symbol}: color scheme from ColorSchemes.jl. Defaults to :seismic.\nreduce::Symbol: Selects how color channels are reduced to a single number to apply a color scheme. The following methods can be selected, which are then applied over the color channels for each \"pixel\" in the array:\n:sum: sum up color channels\n:norm: compute 2-norm over the color channels\n:maxabs: compute maximum(abs, x) over the color channels\nDefaults to :sum.\nrangescale::Symbol: Selects how the color channel reduced heatmap is normalized before the color scheme is applied. Can be either :extrema or :centered. Defaults to :centered.\nprocess_batch::Bool: When heatmapping a batch, setting process_batch=true will apply the rangescale normalization to the entire batch instead of computing it individually for each sample in the batch. Defaults to false.\npermute::Bool: Whether to flip W&H input channels. Default is true.\nunpack_singleton::Bool: If false, heatmap will always return a vector of images. When heatmapping a batch with a single sample, setting unpack_singleton=true will unpack the singleton vector and directly return the image. Defaults to true.\n\n\n\n\n\nheatmap(input, analyzer)\n\nCompute an Explanation for a given input using the method analyzer and visualize it as a vision heatmap.\n\nAny additional arguments and keyword arguments are passed to the analyzer. Refer to analyze for more information on available keyword arguments.\n\nTo customize the heatmapping style, first compute an explanation using analyze and then call heatmap on the explanation.\n\n\n\n\n\nheatmap(explanation, text)\n\nVisualize Explanation from XAIBase as text heatmap. Text should be a vector containing vectors of strings, one for each input in the batched explanation.\n\nKeyword arguments\n\ncolorscheme::Union{ColorScheme,Symbol}: color scheme from ColorSchemes.jl. Defaults to :seismic.\nrangescale::Symbol: selects how the color channel reduced heatmap is normalized before the color scheme is applied. Can be either :extrema or :centered. Defaults to :centered for use with the default color scheme :seismic.\n\n\n\n\n\n","category":"function"},{"location":"api/#Analyzers","page":"API Reference","title":"Analyzers","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"Gradient\nInputTimesGradient\nSmoothGrad\nIntegratedGradients","category":"page"},{"location":"api/#ExplainableAI.Gradient","page":"API Reference","title":"ExplainableAI.Gradient","text":"Gradient(model)\n\nAnalyze model by calculating the gradient of a neuron activation with respect to the input.\n\n\n\n\n\n","category":"type"},{"location":"api/#ExplainableAI.InputTimesGradient","page":"API Reference","title":"ExplainableAI.InputTimesGradient","text":"InputTimesGradient(model)\n\nAnalyze model by calculating the gradient of a neuron activation with respect to the input. This gradient is then multiplied element-wise with the input.\n\n\n\n\n\n","category":"type"},{"location":"api/#ExplainableAI.SmoothGrad","page":"API Reference","title":"ExplainableAI.SmoothGrad","text":"SmoothGrad(analyzer, [n=50, std=0.1, rng=GLOBAL_RNG])\nSmoothGrad(analyzer, [n=50, distribution=Normal(0, σ²=0.01), rng=GLOBAL_RNG])\n\nAnalyze model by calculating a smoothed sensitivity map. This is done by averaging sensitivity maps of a Gradient analyzer over random samples in a neighborhood of the input, typically by adding Gaussian noise with mean 0.\n\nReferences\n\nSmilkov et al., SmoothGrad: removing noise by adding noise\n\n\n\n\n\n","category":"function"},{"location":"api/#ExplainableAI.IntegratedGradients","page":"API Reference","title":"ExplainableAI.IntegratedGradients","text":"IntegratedGradients(analyzer, [n=50])\nIntegratedGradients(analyzer, [n=50])\n\nAnalyze model by using the Integrated Gradients method.\n\nReferences\n\nSundararajan et al., Axiomatic Attribution for Deep Networks\n\n\n\n\n\n","category":"function"},{"location":"api/#Input-augmentations","page":"API Reference","title":"Input augmentations","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"SmoothGrad and IntegratedGradients are special cases of the input augmentations NoiseAugmentation and InterpolationAugmentation, which can be applied as a wrapper to any analyzer:","category":"page"},{"location":"api/","page":"API Reference","title":"API Reference","text":"NoiseAugmentation\nInterpolationAugmentation","category":"page"},{"location":"api/#ExplainableAI.NoiseAugmentation","page":"API Reference","title":"ExplainableAI.NoiseAugmentation","text":"NoiseAugmentation(analyzer, n, [std=1, rng=GLOBAL_RNG])\nNoiseAugmentation(analyzer, n, distribution, [rng=GLOBAL_RNG])\n\nA wrapper around analyzers that augments the input with n samples of additive noise sampled from distribution. This input augmentation is then averaged to return an Explanation.\n\n\n\n\n\n","category":"type"},{"location":"api/#ExplainableAI.InterpolationAugmentation","page":"API Reference","title":"ExplainableAI.InterpolationAugmentation","text":"InterpolationAugmentation(model, [n=50])\n\nA wrapper around analyzers that augments the input with n steps of linear interpolation between the input and a reference input (typically zero(input)). The gradients w.r.t. this augmented input are then averaged and multiplied with the difference between the input and the reference input.\n\n\n\n\n\n","category":"type"},{"location":"api/#Index","page":"API Reference","title":"Index","text":"","category":"section"},{"location":"api/","page":"API Reference","title":"API Reference","text":"","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"EditURL = \"../literate/heatmapping.jl\"","category":"page"},{"location":"generated/heatmapping/#docs-heatmapping","page":"Heatmapping","title":"Heatmapping","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Since numerical explanations are not very informative at first sight, we can visualize them by computing a heatmap. This page showcases different options and preset for heatmapping, building on the basics shown in the Getting started section.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"We start out by loading the same pre-trained LeNet5 model and MNIST input data:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"using ExplainableAI\nusing Flux\n\nusing BSON # hide\nmodel = BSON.load(\"../model.bson\", @__MODULE__)[:model] # hide\nmodel","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"using MLDatasets\nusing ImageCore, ImageIO, ImageShow\n\nindex = 10\nx, y = MNIST(Float32, :test)[10]\ninput = reshape(x, 28, 28, 1, :)\n\nconvert2image(MNIST, x)","category":"page"},{"location":"generated/heatmapping/#Automatic-heatmap-presets","page":"Heatmapping","title":"Automatic heatmap presets","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"The function heatmap automatically applies common presets for each method.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Since InputTimesGradient computes attributions, heatmaps are shown in a blue-white-red color scheme. Gradient methods however are typically shown in grayscale:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"analyzer = Gradient(model)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"analyzer = InputTimesGradient(model)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/heatmapping/#Custom-heatmap-settings","page":"Heatmapping","title":"Custom heatmap settings","text":"","category":"section"},{"location":"generated/heatmapping/#Color-schemes","page":"Heatmapping","title":"Color schemes","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"We can partially or fully override presets by passing keyword arguments to heatmap. For example, we can use a custom color scheme from ColorSchemes.jl using the keyword argument colorscheme:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"using ColorSchemes\n\nexpl = analyze(input, analyzer)\nheatmap(expl; colorscheme=:jet)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; colorscheme=:inferno)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Refer to the ColorSchemes.jl catalogue for a gallery of available color schemes.","category":"page"},{"location":"generated/heatmapping/#docs-heatmap-reduce","page":"Heatmapping","title":"Color channel reduction","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Explanations have the same dimensionality as the inputs to the classifier. For images with multiple color channels, this means that the explanation also has a \"color channel\" dimension.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"The keyword argument reduce can be used to reduce this dimension to a single scalar value for each pixel. The following presets are available:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":":sum: sum up color channels (default setting)\n:norm: compute 2-norm over the color channels\n:maxabs: compute maximum(abs, x) over the color channels","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; reduce=:sum)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; reduce=:norm)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; reduce=:maxabs)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"In this example, the heatmaps look identical. Since MNIST only has a single color channel, there is no need for color channel reduction.","category":"page"},{"location":"generated/heatmapping/#docs-heatmap-rangescale","page":"Heatmapping","title":"Mapping explanations onto the color scheme","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"To map a color-channel-reduced explanation onto a color scheme, we first need to normalize all values to the range 0 1.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"For this purpose, two presets are available through the rangescale keyword argument:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":":extrema: normalize to the minimum and maximum value of the explanation\n:centered: normalize to the maximum absolute value of the explanation. Values of zero will be mapped to the center of the color scheme.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Depending on the color scheme, one of these presets may be more suitable than the other. The default color scheme for InputTimesGradient, seismic, is centered around zero, making :centered a good choice:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; rangescale=:centered)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; rangescale=:extrema)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"However, for the inferno color scheme, which is not centered around zero, :extrema leads to a heatmap with higher contrast.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; rangescale=:centered, colorscheme=:inferno)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmap(expl; rangescale=:extrema, colorscheme=:inferno)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"For the full list of heatmap keyword arguments, refer to the heatmap documentation.","category":"page"},{"location":"generated/heatmapping/#docs-heatmapping-batches","page":"Heatmapping","title":"Heatmapping batches","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Heatmapping also works with input batches. Let's demonstrate this by using a batch of 100 images from the MNIST dataset:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"xs, ys = MNIST(Float32, :test)[1:100]\nbatch = reshape(xs, 28, 28, 1, :); # reshape to WHCN format\nnothing #hide","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"The heatmap function automatically recognizes that the explanation is batched and returns a Vector of images:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"heatmaps = heatmap(batch, analyzer)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Image.jl's mosaic function can used to display them in a grid:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"mosaic(heatmaps; nrow=10)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"When heatmapping batches, the mapping to the color scheme is applied per sample. For example, rangescale=:extrema will normalize each heatmap to the minimum and maximum value of each sample in the batch. This ensures that heatmaps don't depend on other samples in the batch.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"If this bevahior is not desired, heatmap can be called with the keyword-argument process_batch=true:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"expl = analyze(batch, analyzer)\nheatmaps = heatmap(expl; process_batch=true)\nmosaic(heatmaps; nrow=10)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"This can be useful when comparing heatmaps for fixed output neurons:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"expl = analyze(batch, analyzer, 7) # explain digit \"6\"\nheatmaps = heatmap(expl; process_batch=true)\nmosaic(heatmaps; nrow=10)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"note: Output type consistency\nTo obtain a singleton Vector containing a single heatmap for non-batched inputs, use the heatmap keyword argument unpack_singleton=false.","category":"page"},{"location":"generated/heatmapping/#Processing-heatmaps","page":"Heatmapping","title":"Processing heatmaps","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Heatmapping makes use of the Julia-based image processing ecosystem Images.jl.","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"If you want to further process heatmaps, you may benefit from reading about some fundamental conventions that the ecosystem utilizes that are different from how images are typically represented in OpenCV, MATLAB, ImageJ or Python.","category":"page"},{"location":"generated/heatmapping/#Saving-heatmaps","page":"Heatmapping","title":"Saving heatmaps","text":"","category":"section"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"Since heatmaps are regular Images.jl images, they can be saved as such:","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"using FileIO\n\nimg = heatmap(input, analyzer)\nsave(\"heatmap.png\", img)","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"","category":"page"},{"location":"generated/heatmapping/","page":"Heatmapping","title":"Heatmapping","text":"This page was generated using Literate.jl.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"EditURL = \"../literate/augmentations.jl\"","category":"page"},{"location":"generated/augmentations/#docs-augmentations","page":"Input augmentations","title":"Analyzer augmentations","text":"","category":"section"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"All analyzers implemented in ExplainableAI.jl can be augmented by two types of augmentations: NoiseAugmentations and InterpolationAugmentations. These augmentations are wrappers around analyzers that modify the input before passing it to the analyzer.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"We build on the basics shown in the Getting started section and start out by loading the same pre-trained LeNet5 model and MNIST input data:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"using ExplainableAI\nusing Flux\n\nusing BSON # hide\nmodel = BSON.load(\"../model.bson\", @__MODULE__)[:model] # hide\nmodel","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"using MLDatasets\nusing ImageCore, ImageIO, ImageShow\n\nindex = 10\nx, y = MNIST(Float32, :test)[10]\ninput = reshape(x, 28, 28, 1, :)\n\nconvert2image(MNIST, x)","category":"page"},{"location":"generated/augmentations/#Noise-augmentation","page":"Input augmentations","title":"Noise augmentation","text":"","category":"section"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"The NoiseAugmentation wrapper computes explanations averaged over noisy inputs. Let's demonstrate this on the Gradient analyzer. First, we compute the heatmap of an explanation without augmentation:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = Gradient(model)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"Now we wrap the analyzer in a NoiseAugmentation with 10 samples of noise. By default, the noise is sampled from a Gaussian distribution with mean 0 and standard deviation 1.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = NoiseAugmentation(Gradient(model), 50)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"Note that a higher sample size is desired, as it will lead to a smoother heatmap. However, this comes at the cost of a longer computation time.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"We can also set the standard deviation of the Gaussian distribution:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = NoiseAugmentation(Gradient(model), 50, 0.1)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"When used with a Gradient analyzer, this is equivalent to SmoothGrad:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = SmoothGrad(model, 50)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"We can also use any distribution from Distributions.jl, for example Poisson noise with rate lambda=05:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"using Distributions\n\nanalyzer = NoiseAugmentation(Gradient(model), 50, Poisson(0.5))\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"Is is also possible to define your own distributions or mixture distributions.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"NoiseAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.","category":"page"},{"location":"generated/augmentations/#Integration-augmentation","page":"Input augmentations","title":"Integration augmentation","text":"","category":"section"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"The InterpolationAugmentation wrapper computes explanations averaged over n steps of linear interpolation between the input and a reference input, which is set to zero(input) by default:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = InterpolationAugmentation(Gradient(model), 50)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"When used with a Gradient analyzer, this is equivalent to IntegratedGradients:","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"analyzer = IntegratedGradients(model, 50)\nheatmap(input, analyzer)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"To select a different reference input, pass it to the analyze function using the keyword argument input_ref. Note that this is an arbitrary example for the sake of demonstration.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"matrix_of_ones = ones(Float32, size(input))\n\nanalyzer = InterpolationAugmentation(Gradient(model), 50)\nexpl = analyzer(input; input_ref=matrix_of_ones)\nheatmap(expl)","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"Once again, InterpolationAugmentation can be combined with any analyzer type from the Julia-XAI ecosystem, for example LRP from RelevancePropagation.jl.","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"","category":"page"},{"location":"generated/augmentations/","page":"Input augmentations","title":"Input augmentations","text":"This page was generated using Literate.jl.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"EditURL = \"../literate/example.jl\"","category":"page"},{"location":"generated/example/#docs-getting-started","page":"Getting started","title":"Getting started","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"For this first example, we already have loaded a pre-trained LeNet5 model to look at explanations on the MNIST dataset.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"using ExplainableAI\nusing Flux\n\nusing BSON # hide\nmodel = BSON.load(\"../model.bson\", @__MODULE__)[:model] # hide\nmodel","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"note: Supported models\nExplainableAI.jl can be used on any differentiable classifier.","category":"page"},{"location":"generated/example/#Preparing-the-input-data","page":"Getting started","title":"Preparing the input data","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"We use MLDatasets to load a single image from the MNIST dataset:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"using MLDatasets\nusing ImageCore, ImageIO, ImageShow\n\nindex = 10\nx, y = MNIST(Float32, :test)[10]\n\nconvert2image(MNIST, x)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"By convention in Flux.jl, this input needs to be resized to WHCN format by adding a color channel and batch dimensions.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"input = reshape(x, 28, 28, 1, :);\nnothing #hide","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"note: Input format\nFor any explanation of a model, ExplainableAI.jl assumes the batch dimension to come last in the input.For the purpose of heatmapping, the input is assumed to be in WHCN order (width, height, channels, batch), which is Flux.jl's convention.","category":"page"},{"location":"generated/example/#Explanations","page":"Getting started","title":"Explanations","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"We can now select an analyzer of our choice and call analyze to get an Explanation:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"analyzer = InputTimesGradient(model)\nexpl = analyze(input, analyzer);\nnothing #hide","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"The return value expl is of type Explanation and bundles the following data:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"expl.val: numerical output of the analyzer, e.g. an attribution or gradient\nexpl.output: model output for the given analyzer input\nexpl.output_selection: index of the output used for the explanation\nexpl.analyzer: symbol corresponding the used analyzer, e.g. :Gradient or :LRP\nexpl.heatmap: symbol indicating a preset heatmapping style, e.g. :attibution, :sensitivity or :cam\nexpl.extras: optional named tuple that can be used by analyzers to return additional information.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"We used InputTimesGradient, so expl.analyzer is :InputTimesGradient.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"expl.analyzer","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"By default, the explanation is computed for the maximally activated output neuron. Since our digit is a 9 and Julia's indexing is 1-based, the output neuron at index 10 of our trained model is maximally activated.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"Finally, we obtain the result of the analyzer in form of an array.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"expl.val","category":"page"},{"location":"generated/example/#Heatmapping-basics","page":"Getting started","title":"Heatmapping basics","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"Since the array expl.val is not very informative at first sight, we can visualize Explanations by computing a heatmap:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"heatmap(expl)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"If we are only interested in the heatmap, we can combine analysis and heatmapping into a single function call:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"heatmap(input, analyzer)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"For a more detailed explanation of the heatmap function, refer to the heatmapping section.","category":"page"},{"location":"generated/example/#docs-analyzers-list","page":"Getting started","title":"List of analyzers","text":"","category":"section"},{"location":"generated/example/#Neuron-selection","page":"Getting started","title":"Neuron selection","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"By passing an additional index to our call to analyze, we can compute an explanation with respect to a specific output neuron. Let's see why the output wasn't interpreted as a 4 (output neuron at index 5)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"expl = analyze(input, analyzer, 5)\nheatmap(expl)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"This heatmap shows us that the \"upper loop\" of the hand-drawn 9 has negative relevance with respect to the output neuron corresponding to digit 4!","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"note: Note\nThe output neuron can also be specified when calling heatmap:heatmap(input, analyzer, 5)","category":"page"},{"location":"generated/example/#Analyzing-batches","page":"Getting started","title":"Analyzing batches","text":"","category":"section"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"ExplainableAI also supports explanations of input batches:","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"batchsize = 20\nxs, _ = MNIST(Float32, :test)[1:batchsize]\nbatch = reshape(xs, 28, 28, 1, :) # reshape to WHCN format\nexpl = analyze(batch, analyzer);\nnothing #hide","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"This will return a single Explanation expl for the entire batch. Calling heatmap on expl will detect the batch dimension and return a vector of heatmaps.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"heatmap(expl)","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"For more information on heatmapping batches, refer to the heatmapping documentation.","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"","category":"page"},{"location":"generated/example/","page":"Getting started","title":"Getting started","text":"This page was generated using Literate.jl.","category":"page"},{"location":"","page":"Home","title":"Home","text":"CurrentModule = ExplainableAI","category":"page"},{"location":"#ExplainableAI.jl","page":"Home","title":"ExplainableAI.jl","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Explainable AI in Julia.","category":"page"},{"location":"#Installation","page":"Home","title":"Installation","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"To install this package and its dependencies, open the Julia REPL and run ","category":"page"},{"location":"","page":"Home","title":"Home","text":"julia> ]add ExplainableAI","category":"page"},{"location":"#Manual","page":"Home","title":"Manual","text":"","category":"section"},{"location":"#General-usage","page":"Home","title":"General usage","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Pages = [\n \"generated/example.md\",\n \"generated/heatmapping.md\",\n \"generated/augmentations.md\",\n]\nDepth = 3","category":"page"},{"location":"#API-reference","page":"Home","title":"API reference","text":"","category":"section"},{"location":"","page":"Home","title":"Home","text":"Pages = [\"api.md\"]\nDepth = 2","category":"page"}] }