-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathvisualize-img-disc.html
159 lines (152 loc) · 6.36 KB
/
visualize-img-disc.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.1.1/css/bootstrap.min.css" integrity="sha384-WskhaSGFgHYWDcbwN70/dfYBj47jz9qbsMId/iRN3ewGhXQFZCSftd1LZCfmhktB" crossorigin="anonymous">
<link rel="stylesheet" type="text/css" href="assets/vid/css/vid.css">
</head>
<body>
<div class="container">
<div class="row">
<div class="col-12">
<br>
<h1 class="text-center">Visualizing Image Content to Explain Novel Image Discovery</h1>
</div>
</div>
<br>
<div class="row">
<div class="col-lg-2"></div>
<div class="col-lg-4">
<table class="table table-borderless table-sm text-center">
<tbody>
<tr>
<td style="font-size:22px;"><a href="http://www.jakehlee.com/">
Jake Lee
</a></td>
</tr>
<tr>
<td style="font-size:22px;">
Columbia University
</td>
</tr>
</tbody>
</table>
</div>
<div class="col-lg-4">
<table class="table table-borderless table-sm text-center">
<tbody>
<tr>
<td style="font-size:22px;"><a href="http://www.wkiri.com/">
Kiri Wagstaff
</a></td>
</tr>
<tr>
<td style="font-size:22px;">
Jet Propulsion Laboratory, <br>California Institute of Technology
</td>
</tr>
</tbody>
</table>
</div>
<div class="col-lg-2"></div>
</div>
<div class="row" style="font-size:22px;">
<div class="col-lg-12 text-center">
Published in Data Mining and Knowledge Discovery, 34, 1777-1804(2020)
<br>
<a href="https://link.springer.com/article/10.1007/s10618-020-00700-0">[Link to Springer]</a>
<br>
Accepted Manuscript: <a href="assets/vid/pdf/dmkd-novel-image.pdf">[PDF]</a>
</div>
</div>
<hr>
<div class="row">
<div class="col-md-1"></div>
<div class="col-md-10">
<img src="assets/vid/img/disc.png" class="img-fluid" alt="Diversity collection">
</div>
<div class="col-md-1"></div>
</div>
<hr>
<div class="row">
<div class="col-md-6">
<img src="assets/vid/img/teaser1.png" class="img-fluid" alt="Class discovery explanations">
</div>
<div class="col-md-6">
<img src="assets/vid/img/teaser2.png" class="img-fluid" alt="In-class discovery explanations">
</div>
</div>
<hr>
<div>
<h1 class="text-center">Abstract</h1>
<br>
<p class="text-left">The initial analysis of any large data set can be divided into two phases: (1) the identification of common trends or patterns and (2) the identification of anomalies or outliers that deviate from those trends. We focus on the goal of detecting observations with novel content, which can alert us to artifacts in the data set or, potentially, the discovery of previously unknown phenomena. To aid in interpreting and diagnosing the novel aspect of these selected observations, we recommend the use of novelty detection methods that generate explanations. In the context of large image data sets, these explanations should highlight what aspect of a given image is new (color, shape, texture, content) in a human-comprehensible form. We propose DEMUD-VIS, the first method for providing visual explanations of novel image content by employing a convolutional neural network (CNN) to extract image features, a method that uses reconstruction error to detect novel content, and an up-convolutional network to convert CNN feature representations back into image space. We demonstrate this approach on diverse images from ImageNet, freshwater streams, and the surface of Mars.</p>
</div>
<hr>
<div class="text-center">
<h1>Code & Data</h1>
<br>
<img src="assets/vid/img/diagram.png" class="img-fluid" alt="Interpretable image discovery diagram">
</div>
<br>
<dl class="row" style="font-size:22px;">
<dt class="col-sm-6 text-switch">DEMUD</dt>
<dd class="col-sm-6"><a href="https://github.com/wkiri/DEMUD">[GitHub repository]</a></dd>
<dt class="col-sm-6 text-switch">Supplemental scripts and data sets</dt>
<dd class="col-sm-6"><a href="https://github.com/jakehlee/dmkd-vis-image-disc">[GitHub repository]</a></dd>
</dl>
<hr>
<div>
<h1 class="text-center">Paper</h1>
<br>
<div class="row">
<div class="col-lg-2"></div>
<div class="col-lg-4">
<div class="img-stack">
<a href="assets/vid/pdf/dmkd-novel-image.pdf"><img src="assets/vid/pages/1.png" class="stack-1"></a>
<img src="assets/vid/pages/2.png" class="stack-2">
<img src="assets/vid/pages/3.png" class="stack-3">
</div>
</div>
<div class="col-lg-4" style="display: flex; align-items: center;">
<div class="text-left">
Jake Lee, Kiri Wagstaff
<br>
<strong>Visualizing Image Content to Explain Novel Image Discovery</strong>
<br>
Data Mining and Knowledge Discovery
<br>
<a href="https://arxiv.org/abs/1908.05006">[pre-print on arXiv]</a>
<br>
<a href="assets/vid/pdf/dmkd-novel-image.pdf">[accepted manuscript PDF]</a>
<br>
<pre><code>
@article{lee2020visualizing,
title={Visualizing Image Content to Explain Novel Image Discovery},
author={Lee, Jake H and Wagstaff, Kiri L},
journal={Data Mining and Knowledge Discovery},
volume={34},
number={6},
pages={1777--1804},
year={2020},
publisher={Springer}
}
</code></pre>
</div>
</div>
<div class="col-lg-2"></div>
</div>
</div>
<hr>
<div>
<h1 class="text-center">Acknowledgements</h1>
<br>
<p>We thank the Planetary Data System Imaging Node for funding this project. Part of this research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration.</p>
</div>
</div>
<script src="https://code.jquery.com/jquery-3.3.1.slim.min.js" integrity="sha384-q8i/X+965DzO0rT7abK41JStQIAqVgRVzpbzo5smXKp4YfRvH+8abtTE1Pi6jizo" crossorigin="anonymous"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.3/umd/popper.min.js" integrity="sha384-ZMP7rVo3mIykV+2+9J3UJ46jBk0WLaUAdn689aCwoqbBJiSnjAK/l8WvCWPIPm49" crossorigin="anonymous"></script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.1.1/js/bootstrap.min.js" integrity="sha384-smHYKdLADwkXOn1EmN1qk/HfnUcbVRZyYmZ4qpPea6sjB/pTJ0euyQp0Mk8ck+5T" crossorigin="anonymous"></script>
</body>
</html>