-
Notifications
You must be signed in to change notification settings - Fork 13
/
publications.html
1291 lines (1268 loc) · 85.1 KB
/
publications.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="">
<meta name="author" content="">
<title>Information, Inference and Machine Learning Group at University College London</title>
<!-- Bootstrap core CSS -->
<link href="vendor/bootstrap/css/bootstrap.min.css" rel="stylesheet">
<!-- Custom styles for this template -->
<link href="css/business-frontpage.css" rel="stylesheet">
<script src="https://kit.fontawesome.com/f071da7bc5.js"></script>
<link rel='icon' href='imgs/favicon.ico' type='image/x-icon' sizes="16x16" />
<script src="https://code.jquery.com/jquery-3.3.1.js" integrity="sha256-2Kok7MbOyxpgUVvAk/HJ2jigOSYS2auK4Pfzbm7uH60="
crossorigin="anonymous">
</script>
<script src="vendor/bootstrap/js/bootstrap.bundle.min.js"></script>
</head>
<script>
$(function () {
$("#footer").load("footer.html");
});
</script>
<body>
<!-- Navigation -->
<nav class="navbar navbar-expand-lg navbar-dark bg-dark fixed-top">
<div class="container">
<a class="navbar-brand" href="index.html">I<sup>2</sup>|ML at UCL</a>
<button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarResponsive"
aria-controls="navbarResponsive" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<div class="collapse navbar-collapse" id="navbarResponsive">
<ul class="navbar-nav ml-auto">
<li class="nav-item">
<a class="nav-link" href="index.html">Home
</a>
</li>
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="members/members.html" id="navbarDropdownMenuLink"
data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
Members
</a>
<div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
<a class="dropdown-item" href="members/members.html#principal-investigator">Head of Lab</a>
<a class="dropdown-item" href="members/members.html#postdocs">Postdocs</a>
<a class="dropdown-item" href="members/members.html#phds">PhD Students</a>
<a class="dropdown-item" href="members/members.html#alumni">Alumni</a>
</div>
</li>
<li class="nav-item">
<a class="nav-link" href="projects/projects.html">Projects</a>
</li>
<li class="nav-item dropdown">
<a class="nav-link active dropdown-toggle" href="publications.html" id="navbarDropdownMenuLink"
data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
Publications
</a>
<div class="dropdown-menu" aria-labelledby="navbarDropdownMenuLink">
<a class="dropdown-item" href="publications.html#2020">2021</a>
<a class="dropdown-item" href="publications.html#2020">2020</a>
<a class="dropdown-item" href="publications.html#2019">2019</a>
<a class="dropdown-item" href="publications.html#2018">2018</a>
<a class="dropdown-item" href="publications.html#2017">2017</a>
<a class="dropdown-item" href="publications.html#2016">2016</a>
<a class="dropdown-item" href="publications.html#Older">Older</a>
</div>
</li>
<li class="nav-item">
<a class="nav-link" target="_blank" href="https://github.com/iiml-ucl">Software</a>
</li>
<li class="nav-item">
<a class="nav-link" href="press.html">Press</a>
</li>
<li class="nav-item">
<a class="nav-link" href="openings.html">Openings</a>
</li>
<li class="nav-item">
<a class="nav-link" href="location.html">Location</a>
</li>
</ul>
</div>
</div>
</nav>
<!-- Header -->
<header class="bg-primary py-5 mb-5">
<div class="container h-50">
<div class="row h-100 align-items-center">
<div class="col-lg-12">
<h1 class="display-4 text-white mt-2 mb-2">Publications</h1>
</div>
</div>
</div>
</header>
<!-- Page Content -->
<div class="container">
<div class="row">
<div id="sidebar-container" class="sidebar-expanded col- d-1 d-md-block" style="position:sticky;">
<ul class="nobull list-group sticky-top sticky-offset">
<li class="nav-item">
<a class="nav-link" href="#2021">
<h4>2021</h4>
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#2020">
<h4>2020</h4>
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#2019">
<h4>2019</h4>
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#2018">
<h4>2018</h4>
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#2017">
<h4>2017</h4>
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#2016">
<h4>2016</h4>
</a>
</li>
<li class="nav-item">
<a class="nav-link" href="#older">
<h4>Older</h4>
</a>
</li>
</ul>
</div>
<div class="col offset-1" id="main">
<div class="row">
<div id="2021" class="col-md-12 mb-5">
<h2>2021</h2>
<hr>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="zhou2021ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/abstract/document/9654204?casa_token=gVqKmA2R_4UAAAAA:EdSLWXF397G3pgLgRJWk0XNHdVq4dpQdJ8QgqpRtduLNXmbTf5VGjdg3b-qeRFbyDCOusQiuJQ"
target="_blank"><img class="img-fluid" src="publications/2021/zhou2021ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> ADMM-Based Hyperspectral Unmixing Networks for Abundance and Endmember
Estimation <br>
<strong>Authors:</strong> C. Zhou and M. R.D. Rodrigues<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Geoscience and Remote Sensing<br>
<p> <strong>Abstract:</strong> Hyperspectral image (HSI) unmixing is an increasingly studied problem in
various areas, including remote sensing. It has been tackled using both physical model-based
approaches and more recently machine learning-based ones. In this paper, we propose a new HSI unmixing
algorithm combining both model- and learning-based techniques, based on algorithm unrolling
approaches, delivering improved unmixing performance. Our approach unrolls the Alternating Direction
Method of Multipliers (ADMM) solver of a constrained sparse regression problem underlying a linear
mixture model. We then propose a neural network structure for abundance estimation that can be trained
using supervised learning techniques based on a new composite loss function. We also propose another
neural network structure for blind unmixing that can be trained using unsupervised learning
techniques. Our proposed networks are also shown to possess a lighter and richer structure containing
less learnable parameters and more skip connections compared with other competing architectures.
Extensive experiments show that the proposed methods can achieve much faster convergence and better
performance even with a very small training dataset size when compared with other unmixing methods
such as MNN-AE&BU, UnDIP and EGU-Net.</p>
<a href="https://ieeexplore.ieee.org/abstract/document/9654204?casa_token=gVqKmA2R_4UAAAAA:EdSLWXF397G3pgLgRJWk0XNHdVq4dpQdJ8QgqpRtduLNXmbTf5VGjdg3b-qeRFbyDCOusQiuJQ"
target="_blank"><strong><i class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="xu2021ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/abstract/document/9438648?casa_token=O-3sI7AEVh8AAAAA:p3TUjmsDCR9sBbAtd8Xd0WGKeb43ROQuyEDJzQDcD4yn11SYoHaYnjZd1IdzTQZnMXQFFcPTPQ"
target="_blank"><img class="img-fluid" src="publications/2021/xu2021ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Wireless Image Transmission Using Deep Source Channel Coding With Attention
Modules <br>
<strong>Authors:</strong> J. Xu, B. Ai, W. Chen, A. Yang, P. Sun, and M. R. D. Rodrigues<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Circuits and Systems for Video
Technology<br>
<p> <strong>Abstract:</strong> Recent research on joint source channel coding (JSCC) for wireless
communications has achieved great success owing to the employment of deep learning (DL). However,
the existing work on DL based JSCC usually trains the designed network to operate under a specific
signal-to-noise ratio (SNR) regime, without taking into account that the SNR level during the
deployment stage may differ from that during the training stage. A number of networks are required
to cover the scenario with a broad range of SNRs, which is computational inefficiency (in the
training stage) and requires large storage. To overcome these drawbacks our paper proposes a novel
method called Attention DL based JSCC (ADJSCC) that can successfully operate with different SNR
levels during transmission. This design is inspired by the resource assignment strategy in
traditional JSCC, which dynamically adjusts the compression ratio in source coding and the channel
coding rate according to the channel SNR. This is achieved by resorting to attention mechanisms
because these are able to allocate computing resources to more critical tasks. Instead of applying
the resource allocation strategy in traditional JSCC, the ADJSCC uses the channel-wise soft
attention to scaling features according to SNR conditions. We compare the ADJSCC method with the
state-of-the-art DL based JSCC method through extensive experiments to demonstrate its adaptability,
robustness and versatility. Compared with the existing methods, the proposed method takes less
storage and is more robust in the presence of channel mismatch.</p>
<a href="https://ieeexplore.ieee.org/abstract/document/9438648?casa_token=O-3sI7AEVh8AAAAA:p3TUjmsDCR9sBbAtd8Xd0WGKeb43ROQuyEDJzQDcD4yn11SYoHaYnjZd1IdzTQZnMXQFFcPTPQ"
target="_blank"><strong><i class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="amjad2021ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/abstract/document/9609667?casa_token=VNywa9qyJY4AAAAA:CA_i0Muwp6Gq7fQDa60HJLfvOyI_vqokhvX64NyBUKecp2unJBqgCwUPZQz-zzOGf0klNqdM0A"
target="_blank"><img class="img-fluid" src="publications/2021/amjad2021ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Deep Learning Model-Aware Regulatization With Applications to Inverse
Problems <br>
<strong>Authors:</strong> J. Amjad, Z. Lyu, and M. R. D. Rodrigues<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Signal Processing<br>
<p> <strong>Abstract:</strong> There are various inverse problems – including reconstruction problems
arising in medical imaging - where one is often aware of the forward operator that maps variables
of interest to the observations. It is therefore natural to ask whether such knowledge of the
forward operator can be exploited in deep learning approaches increasingly used to solve inverse
problems. In this paper, we provide one such way via an analysis of the generalisation error of
deep learning approaches to inverse problems. In particular, by building on the algorithmic
robustness framework, we offer a generalisation error bound that encapsulates key ingredients
associated with the learning problem such as the complexity of the data space, the size of the
training set, the Jacobian of the deep neural network and the Jacobian of the composition of the
forward operator with the neural network. We then propose a ‘plug-and-play’ regulariser that
leverages the knowledge of the forward map to improve the generalization of the network. We
likewise also use a new method allowing us to tightly upper bound the Jacobians of the relevant
operators that is much more computationally efficient than existing ones. We demonstrate the
efficacy of our model-aware regularised deep learning algorithms against other state-of-the-art
approaches on inverse problems involving various sub-sampling operators such as those used in
classical compressed sensing tasks, image super-resolution problems and accelerated Magnetic
Resonance Imaging (MRI) setups.</p>
<a href="https://ieeexplore.ieee.org/abstract/document/9609667?casa_token=VNywa9qyJY4AAAAA:CA_i0Muwp6Gq7fQDa60HJLfvOyI_vqokhvX64NyBUKecp2unJBqgCwUPZQz-zzOGf0klNqdM0A"
target="_blank"><strong><i class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="aminian2021neurips" class="row publication">
<div class="col-md-4">
<a href="https://proceedings.neurips.cc/paper/2021/hash/445e24b5f22cacb9d51a837c10e91a3f-Abstract.html"
target="_blank"><img class="img-fluid" src="publications/2021/aminian2021neurips.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> An Exact Characterization of the Generalization Error for the Gibbs Algorithm
<br>
<strong>Authors:</strong> G. Aminian, Y. Bu, L. Toni, M. R. D. Rodrigues, and G. Wornell<br>
<strong>Journal/Conference:</strong> Neural Information Processing Systems (NeurIPS) <br>
<p> <strong>Abstract:</strong> Recent research on joint source channel coding (JSCC) for wireless
communications has achieved great success owing to the employment of deep learning (DL). However,
the existing work on DL based JSCC usually trains the designed network to operate under a specific
signal-to-noise ratio (SNR) regime, without taking into account that the SNR level during the
deployment stage may differ from that during the training stage. A number of networks are required
to cover the scenario with a broad range of SNRs, which is computational inefficiency (in the
training stage) and requires large storage. To overcome these drawbacks our paper proposes a novel
method called Attention DL based JSCC (ADJSCC) that can successfully operate with different SNR
levels during transmission. This design is inspired by the resource assignment strategy in
traditional JSCC, which dynamically adjusts the compression ratio in source coding and the channel
coding rate according to the channel SNR. This is achieved by resorting to attention mechanisms
because these are able to allocate computing resources to more critical tasks. Instead of applying
the resource allocation strategy in traditional JSCC, the ADJSCC uses the channel-wise soft
attention to scaling features according to SNR conditions. We compare the ADJSCC method with the
state-of-the-art DL based JSCC method through extensive experiments to demonstrate its adaptability,
robustness and versatility. Compared with the existing methods, the proposed method takes less
storage and is more robust in the presence of channel mismatch.</p>
<a href="https://proceedings.neurips.cc/paper/2021/hash/445e24b5f22cacb9d51a837c10e91a3f-Abstract.html"
target="_blank"><strong><i class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="martinez2021icml" class="row publication">
<div class="col-md-4">
<a href="https://proceedings.mlr.press/v139/martinez21a.html" target="_blank"><img class="img-fluid"
src="publications/2021/martinez2021icml.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Blind Pareto Fairness
<br>
<strong>Authors:</strong> N. Martinez, M. A. Bertran, A. Papadaki, M. R. D. Rodrigues, and G. Sapiro<br>
<strong>Journal/Conference:</strong> International Conference on Machine Learning (ICML) <br>
<p> <strong>Abstract:</strong> Much of the work in the field of group fairness addresses disparities
between predefined groups based on protected features such as gender, age, and race, which need to be
available at train, and often also at test, time. These approaches are static and retrospective, since
algorithms designed to protect groups identified a priori cannot anticipate and protect the needs of
different at-risk groups in the future. In this work we analyze the space of solutions for worst-case
fairness beyond demographics, and propose Blind Pareto Fairness (BPF), a method that leverages
no-regret dynamics to recover a fair minimax classifier that reduces worst-case risk of any potential
subgroup of sufficient size, and guarantees that the remaining population receives the best possible
level of service. BPF addresses fairness beyond demographics, that is, it does not rely on predefined
notions of at-risk groups, neither at train nor at test time. Our experimental results show that the
proposed framework improves worst-case risk in multiple standard datasets, while simultaneously
providing better levels of service for the remaining population.</p>
<a href="https://proceedings.mlr.press/v139/martinez21a.html" target="_blank"><strong><i
class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="ferianc2021uai" class="row publication">
<div class="col-md-4">
<a href="https://proceedings.mlr.press/v161/ferianc21a.html" target="_blank"><img class="img-fluid"
src="publications/2021/ferianc2021uai.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> On the effects of quantisation on model uncertainty in Bayesian neural networks
<br>
<strong>Authors:</strong> M. Ferianc, P. Maji, M. Mattina, and M. R. D. Rodrigues<br>
<strong>Journal/Conference:</strong> Conference on Uncertainty in Artificial Intelligence (UAI) <br>
<p> <strong>Abstract:</strong> Bayesian neural networks (BNNs) are making significant progress in many
research areas where decision-making needs to be accompanied by uncertainty estimation. Being able to
quantify uncertainty while making decisions is essential for understanding when the model is
over-/under-confident, and hence BNNs are attracting interest in safety-critical applications, such as
autonomous driving, healthcare, and robotics. Nevertheless, BNNs have not been as widely used in
industrial practice, mainly because of their increased memory and compute costs. In this work, we
investigate quantisation of BNNs by compressing 32-bit floating-point weights and activations to their
integer counterparts, that has already been successful in reducing the compute demand in standard
pointwise neural networks. We study three types of quantised BNNs, we evaluate them under a wide range
of different settings, and we empirically demonstrate that a uniform quantisation scheme applied to
BNNs does not substantially decrease their quality of uncertainty estimation.</p>
<a href="https://proceedings.mlr.press/v161/ferianc21a.html" target="_blank"><strong><i
class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div id="2020" class="col-md-12 mb-5">
<h2>2020</h2>
<hr>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="sokolic2020ieee" class="row publication">
<div class="col-md-4">
<a href="https://www.ncbi.nlm.nih.gov/pubmed/32031948" target="_blank"><img class="img-fluid"
src="publications/2020/sokolic2020ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Accurate, Very Low Computational Complexity Spike Sorting Using Unsupervised
Matched Subspace Learning<br>
<strong>Authors:</strong> M. Zamani, J. Sokolic, D. Jiang, F. Renna, M. R. D. Rodrigues, and A.
Demosthenous<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Biomedical Circuits and Systems<br>
<p> <strong>Abstract:</strong> This paper presents an adaptable dictionary-based feature extraction
approach for spike sorting offering high accuracy and low computational complexity for implantable
applications. It extracts and learns identifiable features from evolving subspaces through matched
unsupervised subspace filtering. To provide compatibility with the strict constraints in implantable
devices such as the chip area and power budget, the dictionary contains arrays of { −1,0 and 1 and
the
algorithm need only process addition and subtraction operations. Three types of such dictionary were
considered. To quantify and compare the performance of the resulting three feature extractors with
existing systems, a neural signal simulator based on several different libraries was developed. For
noise levels σN between 0.05 and 0.3 and groups of 3 to 6 clusters, all three feature extractors
provide robust high performance with average classification errors of less than 8% over five
iterations, each consisting of 100 generated data segments. To our knowledge, the proposed adaptive
feature extractors are the first able to classify reliably 6 clusters for implantable applications.
An
ASIC implementation of the best performing dictionary-based feature extractor was synthesized in a
65-nm CMOS process. It occupies an area of 0.09 mm2 and dissipates up to about 10.48 μW from a 1 V
supply voltage, when operating with 8-bit resolution at 30 kHz operating frequency.</p>
<a href="https://www.ncbi.nlm.nih.gov/pubmed/32031948" target="_blank"><strong><i
class="fas fa-link"></i>
Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="zah2020ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/document/8950399" target="_blank"><img class="img-fluid"
src="publications/2020/zah2020ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Source Separation with Side Information Based on Gaussian Mixture Models With
Application in Art Investigation<br>
<strong>Authors:</strong> Z. Sabetsarvestani, F. Renna, F. Kiraly and M. R. D. Rodrigues<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Signal Processing<br>
<p> <strong>Abstract:</strong> n this paper, we propose an algorithm for source separation with side
information where one observes the linear superposition of two source signals plus two additional
signals that are correlated with the mixed ones. Our algorithm is based on two ingredients: first,
we
learn a Gaussian mixture model (GMM) for the joint distribution of a source signal and the
corresponding correlated side information signal; second, we separate the signals using standard
computationally efficient conditional mean estimators. The paper also puts forth new recovery
guarantees for this source separation algorithm. In particular, under the assumption that the
signals
can be perfectly described by a GMM model, we characterize necessary and sufficient conditions for
reliable source separation in the asymptotic regime of low-noise as a function of the geometry of
the
underlying signals and their interaction. It is shown that if the subspaces spanned by the
innovation
components of the source signals with respect to the side information signals have zero
intersection,
provided that we observe a certain number of linear measurements from the mixture, then we can
reliably separate the sources; otherwise we cannot. Our proposed framework – which provides a new
way
to incorporate side information to aid the solution of source separation problems where the decoder
has access to linear projections of superimposed sources and side information – is also employed in
a
real-world art investigation application involving the separation of mixtures of X-ray images. The
simulation results showcase the superiority of our algorithm against other state-of-the-art
algorithms.</p>
<a href="https://ieeexplore.ieee.org/document/8950399" target="_blank"><strong><i
class="fas fa-link"></i>
Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="mrd2020" class="row publication">
<div class="col-md-4">
<a href="https://www.amazon.com/Information-Theoretic-Methods-Science-Miguel-Rodrigues/dp/1108427138"
target="_blank"><img class="img-fluid" src="publications/2020/mrd2020.jpg" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Information-Theoretic Methods in Data Science<br>
<strong>Authors:</strong> Rodrigues, M. R. D. and Eldar, Y. C.<br>
<strong>Journal/Conference:</strong> Cambridge University Press<br>
<p> <strong>Abstract:</strong> Learn about the state-of-the-art at the interface between information
theory and data science with
this first unified treatment of the subject. Written by leading experts in a clear, tutorial style,
and using consistent notation and definitions throughout, it shows how information-theoretic methods
are being used in data acquisition, data representation, data analysis, and statistics and machine
learning. Coverage is broad, with chapters on signal acquisition, data compression, compressive
sensing, data communication, representation learning, emerging topics in statistics, and much more.
Each chapter includes a topic overview, definition of the key problems, emerging and open problems,
and an extensive reference list, allowing readers to develop in-depth knowledge and understanding.
Providing a thorough survey of the current research area and cutting-edge trends, this is essential
reading for graduate students and researchers working in information theory, signal processing,
machine learning, and statistics.</p>
<a href="https://www.amazon.com/Information-Theoretic-Methods-Science-Miguel-Rodrigues/dp/1108427138"
target="_blank"><strong><i class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div id="2019" class="col-md-12 mb-5" style="padding-top: 20px">
<h2>2019</h2>
<hr>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="z2019sci" class="row publication">
<div class="col-md-4">
<a href="https://advances.sciencemag.org/content/5/8/eaaw7416" target="_blank"><img class="img-fluid"
src="publications/2019/z2019sci.jpg" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Artificial intelligence for art investigation: Meeting the challenge of
separating x-ray images of the Ghent Altarpiece<br>
<strong>Authors:</strong> Z. Sabetsarvestani, B. Sober, C. Higgitt, I. Daubechies, M. R. D.
Rodrigues<br>
<strong>Journal/Conference:</strong> Science Advances<br>
<p> <strong>Abstract:</strong> X-ray images of polyptych wings, or other artworks painted on both
sides
of their support, contain in one image content from both paintings, making them difficult for
experts
to “read.” To improve the utility of these x-ray images in studying these artworks, it is desirable
to
separate the content into two images, each pertaining to only one side. This is a difficult task for
which previous approaches have been only partially successful. Deep neural network algorithms have
recently achieved remarkable progress in a wide range of image analysis and other challenging tasks.
We, therefore, propose a new self-supervised approach to this x-ray separation, leveraging an
available convolutional neural network architecture; results obtained for details from the Adam and
Eve panels of the Ghent Altarpiece spectacularly improve on previous attempts.</p>
<a href="https://advances.sciencemag.org/content/5/8/eaaw7416" target="_blank"><strong><i
class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="song2019ieee1" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/abstract/document/8715417" target="_blank"><img class="img-fluid"
src="publications/2019/song2019ieee1.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Multimodal Image Super-resolution via Joint Sparse Representations induced by
Coupled Dictionaries<br>
<strong>Authors:</strong> P. Song, X. Deng, J. F. C. Mota, N. Deligiannis, P.-L. Dragotti, and M. R.
D.
Rodrigues<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Computational Imaging<br>
<p> <strong>Abstract:</strong> Real-world data processing problems often involve various image
modalities associated with a certain scene, including RGB images, infrared images or multi-spectral
images. The fact that different image modalities often share certain attributes, such as edges,
textures and other structure primitives, represents an opportunity to enhance various image
processing
tasks. This paper proposes a new approach to construct a high-resolution (HR) version of a
low-resolution (LR) image given another HR image modality as guidance, based on joint sparse
representations induced by coupled dictionaries. The proposed approach captures complex dependency
correlations, including similarities and disparities, between different image modalities in a
learned
sparse feature domain in lieu of the original image domain. It consists of two phases: coupled
dictionary learning phase and coupled super-resolution phase. The learning phase learns a set of
dictionaries from the training dataset to couple different image modalities together in the sparse
feature domain. In turn, the super-resolution phase leverages such dictionaries to construct a HR
version of the LR target image with another related image modality for guidance. In the advanced
version of our approach, multi-stage strategy and neighbourhood regression concept are introduced to
further improve the model capacity and performance. Extensive guided image super-resolution
experiments on real multimodal images demonstrate that the proposed approach admits distinctive
advantages with respect to the state-of-the-art approaches, for example, overcoming the texture
copying artifacts commonly resulting from inconsistency between the guidance and target images. Of
particular relevance, the proposed model demonstrates much better robustness than competing deep
models in a range of noisy scenarios.</p>
<a href="https://ieeexplore.ieee.org/abstract/document/8715417" target="_blank"><strong><i
class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="deng2019ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/abstract/document/8741062" target="_blank"><img class="img-fluid"
src="publications/2019/deng2019ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> RADAR: Robust Algorithm for Depth Image Super Resolution Based on FRI Theory
and
Multimodal Dictionary Learning<br>
<strong>Authors:</strong> X. Deng, P. Song, M. R. D. Rodrigues, and P.-L. Dragotti<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Circuits and Systems for Video
Technology<br>
<p> <strong>Abstract:</strong> Depth image super-resolution is a challenging problem, since normally
high upscaling factors are required (e.g., 16×), and depth images are often noisy. In order to
achieve
large upscaling factors and resilience to noise, we propose a Robust Algorithm for Depth imAge super
Resolution (RADAR) that combines the power of finite rate of innovation (FRI) theory with multimodal
dictionary learning. Given a low-resolution (LR) depth image, we first model its rows and columns as
piece-wise polynomials and propose a FRI-based depth upscaling (FDU) algorithm to super-resolve the
image. Then, the upscaled moderate quality (MQ) depth image is further enhanced with the guidance of
a
registered high-resolution (HR) intensity image. This is achieved by learning multimodal mappings
from
the joint MQ depth and HR intensity pairs to the HR depth, through a recently proposed triple
dictionary learning (TDL) algorithm. Moreover, to speed up the super-resolution process, we
introduce
a new projection-based rapid upscaling (PRU) technique that pre-calculates the projections from the
joint MQ depth and HR intensity pairs to the HR depth. Compared with state-of-the-art deep learning
based methods, our approach has two distinct advantages: we need a fraction of training data but can
achieve the best performance, and we are resilient to mismatches between training and testing
datasets. Extensive numerical results show that the proposed method outperforms other
state-of-the-art
methods on either noise-free or noisy datasets with large upscaling factors up to 16× and can handle
unknown blurring kernels well.</p>
<a href="https://ieeexplore.ieee.org/abstract/document/8741062" target="_blank"><strong><i
class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="shlezinger2019ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/document/8736805/" target="_blank"><img class="img-fluid"
src="publications/2019/shlezinger2019ieee1.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Asymptotic Task-Based Quantization with Application to Massive MIMO<br>
<strong>Authors:</strong> N. Shlezinger, Y. C. Eldar, and M. R. D. Rodrigues<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Signal Processing<br>
<p> <strong>Abstract:</strong> Quantizers take part in nearly every digital signal
processing system which operates on physical signals. They are
commonly designed to accurately represent the underlying signal,
regardless of the specific task to be performed on the quantized
data. In systems working with high-dimensional signals, such
as massive multiple-input multiple-output (MIMO) systems, it is
beneficial to utilize low-resolution quantizers, due to cost, power,
and memory constraints. In this work we study quantization of
high-dimensional inputs, aiming at improving performance under
resolution constraints by accounting for the system task in the
quantizers design. We focus on the task of recovering a desired signal statistically related to the
high-dimensional input, and analyze
two quantization approaches: We first consider vector quantization, which is typically
computationally
infeasible, and characterize
the optimal performance achievable with this approach. Next, we
focus on practical systems which utilize hardware-limited scalar
uniform analog-to-digital converters (ADCs), and design a taskbased quantizer under this model. The
resulting system accounts
for the task by linearly combining the observed signal into a lower
dimension prior to quantization. We then apply our proposed
technique to channel estimation in massive MIMO networks. Our
results demonstrate that a system utilizing low-resolution scalar
ADCs can approach the optimal channel estimation performance
by properly accounting for the task in the system design.</p>
<a href="https://ieeexplore.ieee.org/document/8736805/" target="_blank"><strong><i
class="fas fa-link"></i>
Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="shlezinger2019ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/document/8815422" target="_blank"><img class="img-fluid"
src="publications/2019/shlezinger2019ieee2.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Hardware-Limited Task-Based Quantization<br>
<strong>Authors:</strong> N. Shlezinger, Y. C. Eldar, and M. R. D. Rodrigues<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Signal Processing<br>
<p> <strong>Abstract:</strong> Quantization plays a critical role in digital signal processing
systems.
Quantizers are typically
designed to obtain an accurate digital representation of the input signal, operating independently
of
the system task, and are commonly implemented using serial scalar analog-to-digital converters
(ADCs).
In this work, we study hardware-limited task-based quantization, where a system utilizing a serial
scalar
ADC is designed to provide a suitable representation in order to allow the recovery of a parameter
vector
underlying the input signal. We propose hardware-limited task-based quantization systems for a fixed
and
finite quantization resolution, and characterize their achievable distortion. We then apply the
analysis to
the practical setups of channel estimation and eigen-spectrum recovery from quantized measurements.
Our
results illustrate that properly designed hardware-limited systems can approach the optimal
performance
achievable with vector quantizers, and that by taking the underlying task into account, the
quantization
error can be made negligible with a relatively small number of bits.</p>
<a href="https://ieeexplore.ieee.org/document/8815422" target="_blank"><strong><i
class="fas fa-link"></i>
Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="song2019ieee2" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/abstract/document/8786180" target="_blank"><img class="img-fluid"
src="publications/2019/song2019ieee2.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Coupled Dictionary Learning for Multi-contrast MRI Reconstruction<br>
<strong>Authors:</strong> P. Song, L. Weizmann, J. M. C. Mota, Y. Eldar, and M. R. D. Rodrigues<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Medical Imaging<br>
<p> <strong>Abstract:</strong> Magnetic resonance (MR) imaging tasks often involve multiple contrasts,
such as T1-weighted, T2-weighted and Fluid-attenuated inversion recovery (FLAIR) data. These
contrasts
capture information associated with the same underlying anatomy and thus exhibit similarities in
either structure level or gray level. In this paper, we propose a Coupled Dictionary Learning based
multi-contrast MRI reconstruction (CDLMRI) approach to leverage the dependency correlation between
different contrasts for guided or joint reconstruction from their under-sampled k-space data. Our
approach iterates between three stages: coupled dictionary learning, coupled sparse denoising, and
enforcing k-space consistency. The first stage learns a set of dictionaries that not only are
adaptive
to the contrasts, but also capture correlations among multiple contrasts in a sparse transform
domain.
By capitalizing on the learned dictionaries, the second stage performs coupled sparse coding to
remove
the aliasing and noise in the corrupted contrasts. The third stage enforces consistency between the
denoised contrasts and the measurements in the k-space domain. Numerical experiments, consisting of
retrospective under-sampling of various MRI contrasts with a variety of sampling schemes,
demonstrate
that CDLMRI is capable of capturing structural dependencies between different contrasts. The learned
priors indicate notable advantages in multi-contrast MR imaging and promising applications in
quantitative MR imaging such as MR fingerprinting.</p>
<a href="https://ieeexplore.ieee.org/abstract/document/8786180" target="_blank"><strong><i
class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="song2019med" class="row publication">
<div class="col-md-4">
<a href="https://www.ncbi.nlm.nih.gov/pubmed/31329307" target="_blank"><img class="img-fluid"
src="publications/2019/song2019med.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> HYDRA: Hybrid Deep Magnetic Resonance Fingerprinting<br>
<strong>Authors:</strong> P. Song, G. Mazor, Y. C. Eldar, and M. R. D. Rodrigues<br>
<strong>Journal/Conference:</strong> Medical Physics<br>
<p> <strong>Abstract:</strong> PURPOSE:
Magnetic resonance fingerprinting (MRF) methods typically rely on dictionary matching to map the
temporal MRF signals to quantitative tissue parameters. Such approaches suffer from inherent
discretization errors, as well as high computational complexity as the dictionary size grows. To
alleviate these issues, we propose a HYbrid Deep magnetic ResonAnce fingerprinting (HYDRA) approach,
referred to as HYDRA.
<br>
METHODS:
HYDRA involves two stages: a model-based signature restoration phase and a learning-based parameter
restoration phase. Signal restoration is implemented using low-rank based de-aliasing techniques
while
parameter restoration is performed using a deep nonlocal residual convolutional neural network. The
designed network is trained on synthesized MRF data simulated with the Bloch equations and fast
imaging with steady-state precession (FISP) sequences. In test mode, it takes a temporal MRF signal
as
input and produces the corresponding tissue parameters.
<br>
RESULTS:
We validated our approach on both synthetic data and anatomical data generated from a healthy
subject.
The results demonstrate that, in contrast to conventional dictionary matching-based MRF techniques,
our approach significantly improves inference speed by eliminating the time-consuming dictionary
matching operation, and alleviates discretization errors by outputting continuous-valued parameters.
We further avoid the need to store a large dictionary, thus reducing memory requirements.
<br>
CONCLUSIONS:
Our approach demonstrates advantages in terms of inference speed, accuracy, and storage requirements
over competing MRF methods.</p>
<a href="https://www.ncbi.nlm.nih.gov/pubmed/31329307" target="_blank"><strong><i
class="fas fa-link"></i>
Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="bertran2019icml" class="row publication">
<div class="col-md-4">
<a href="http://proceedings.mlr.press/v97/bertran19a.html" target="_blank"><img class="img-fluid"
src="publications/2019/bertran2019icml.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Adversarially Learned Representations for Information Obfuscation and
Inference<br>
<strong>Authors:</strong> M. A. Bertran, N. Martinez, A. Papadaki, Q. Qiu, M. R. D. Rodrigues, G.
Reeves, and G. Sapiro.<br>
<strong>Journal/Conference:</strong> International Conference on Machine Learning (ICML)<br>
<p> <strong>Abstract:</strong> Data collection and sharing are pervasive aspects of modern society.
This
process can either be voluntary, as in the case of a person taking a facial image to unlock his/her
phone, or incidental, such as traffic cameras collecting videos on pedestrians. An undesirable side
effect of these processes is that shared data can carry information about attributes that users
might
consider as sensitive, even when such information is of limited use for the task. It is therefore
desirable for both data collectors and users to design procedures that minimize sensitive
information
leakage. Balancing the competing objectives of providing meaningful individualized service levels
and
inference while obfuscating sensitive information is still an open problem. In this work, we take an
information theoretic approach that is implemented as an unconstrained adversarial game between Deep
Neural Networks in a principled, data-driven manner. This approach enables us to learn
domain-preserving stochastic transformations that maintain performance on existing algorithms while
minimizing sensitive information leakage.</p>
<a href="http://proceedings.mlr.press/v97/bertran19a.html" target="_blank"><strong><i
class="fas fa-link"></i>
Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div id="2018" class="col-md-12 mb-5">
<h2>2018</h2>
<hr>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="mrd2018" class="row publication">
<div class="col-md-12 publication-text">
<strong>Title:</strong> Introduction to the issue on information-theoretic methods in data
acquisition,
analysis, and processing<br>
<strong>Authors:</strong> Rodrigues, M., Bolcskei, H., Draper, S., Eldar, Y., & Tan, V<br>
<strong>Journal/Conference:</strong> IEEE Journal on Selected Topics in Signal Processing<br>
<p> <strong>Abstract:</strong> The twenty papers that are included in this special section explore
applications of information theoretic methods to emerging data science problems. In particular, the
papers cover a wide range of topics that can broadly be organized into four themes: (1) data
acquisition, (2) data analysis and processing, (3) statistics and machine learning, and (4) privacy
and fairness.</p>
<a href="https://ieeexplore.ieee.org/document/8476530" target="_blank"><strong><i
class="fas fa-link"></i>
Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="chen2018ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/document/8301528" target="_blank"><img class="img-fluid"
src="publications/2018/chen2018ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Compressive Sensing With Side Information: How to Optimally Capture This Extra
Information for GMM Signals?<br>
<strong>Authors:</strong> Chen, M., Renna, F., & Rodrigues, M. R. D<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Signal Processing<br>
<p> <strong>Abstract:</strong> This paper studies how to optimally capture side information to aid in
the reconstruction of high-dimensional signals from low-dimensional random linear and noisy
measurements, by assuming that both the signal of interest and the side information signal are drawn
from a joint Gaussian mixture model. In particular, we derive sufficient and (occasionally)
necessary
conditions on the number of linear measurements for the signal reconstruction minimum mean squared
error (MMSE) to approach zero in the low-noise regime; moreover, we also derive closed-form linear
side information measurement designs for the reconstruction MMSE to approach zero in the low-noise
regime. Our designs suggest that a linear projection kernel that optimally captures side information
is such that it measures the attributes of side information that are maximally correlated with the
signal of interest. A number of experiments both with synthetic and real data confirm that our
theoretical results are well aligned with numerical ones. Finally, we offer a case study associated
with a panchromatic sharpening (pan sharpening) application in the presence of compressive
hyperspectral data that demonstrates that our proposed linear side information measurement designs
can
lead to reconstruction peak signal-to-noise ratio (PSNR) gains in excess of 2 dB over other
approaches
in this practical application.</p>
<a href="https://ieeexplore.ieee.org/document/8301528" target="_blank"><strong><i
class="fas fa-link"></i>
Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div id="2017" class="col-md-12 mb-5">
<h2>2017</h2>
<hr>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="deligiannis2017ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/abstract/document/8017563" target="_blank"><img class="img-fluid"
src="publications/2017/deligiannis2017ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Heterogeneous Networked Data Recovery from Compressive Measurements Using a
Copula Prior<br>
<strong>Authors:</strong> Deligiannis, N., Mota, J. F. C., Zimos, E., & Rodrigues, M. R. D<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Communications<br>
<p> <strong>Abstract:</strong> Large-scale data collection by means of wireless sensor network and
Internet-of-Things technology poses various challenges in view of the limitations in transmission,
computation, and energy resources of the associated wireless devices. Compressive data gathering
based
on compressed sensing has been proven a well-suited solution to the problem. Existing designs
exploit
the spatiotemporal correlations among data collected by a specific sensing modality. However, many
applications, such as environmental monitoring, involve collecting heterogeneous data that are
intrinsically correlated. In this paper, we propose to leverage the correlation from multiple
heterogeneous signals when recovering the data from compressive measurements. To this end, we
propose
a novel recovery algorithm-built upon belief-propagation principles-that leverages correlated
information from multiple heterogeneous signals. To efficiently capture the statistical dependencies
among diverse sensor data, the proposed algorithm uses the statistical model of copula functions.
Experiments with heterogeneous air-pollution sensor measurements show that the proposed design
provides significant performance improvements against the state-of-the-art compressive data
gathering
and recovery schemes that use classical compressed sensing, compressed sensing with side
information,
and distributed compressed sensing.</p>
<a href="https://ieeexplore.ieee.org/abstract/document/8017563" target="_blank"><strong><i
class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="sokolic2017ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/document/7934087" target="_blank"><img class="img-fluid"
src="publications/2017/sokolic2017ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Robust Large Margin Deep Neural Networks<br>
<strong>Authors:</strong> Sokolic, J., Giryes, R., Sapiro, G., & Rodrigues, M. R. D<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Signal Processing<br>
<p> <strong>Abstract:</strong> The generalization error of deep neural networks via their
classification
margin is studied in this work. Our approach is based on the Jacobian matrix of a deep neural
network
and can be applied to networks with arbitrary non-linearities and pooling layers, and to networks
with
different architectures such as feed forward networks and residual networks. Our analysis leads to
the
conclusion that a bounded spectral norm of the network's Jacobian matrix in the neighbourhood of the
training samples is crucial for a deep neural network of arbitrary depth and width to generalize
well.
This is a significant improvement over the current bounds in the literature, which imply that the
generalization error grows with either the width or the depth of the network. Moreover, it shows
that
the recently proposed batch normalization and weight normalization re-parametrizations enjoy good
generalization properties, and leads to a novel network regularizer based on the network's Jacobian
matrix. The analysis is supported with experimental results on the MNIST, CIFAR-10, LaRED and
ImageNet
datasets.</p>
<a href="https://ieeexplore.ieee.org/document/7934087" target="_blank"><strong><i
class="fas fa-link"></i>
Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="mota2017ieee" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/abstract/document/7904593" target="_blank"><img class="img-fluid"
src="publications/2017/mota2017ieee.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Compressed Sensing With Prior Information: Strategies, Geometry, and
Bounds<br>
<strong>Authors:</strong> Mota, J. F. C., Deligiannis, N., & Rodrigues, M. R. D<br>
<strong>Journal/Conference:</strong> IEEE Transactions on Information Theory<br>
<p> <strong>Abstract:</strong> We address the problem of compressed sensing (CS) with prior
information: reconstruct a target CS signal with the aid of a similar signal that is known
beforehand,
our prior information. We integrate the additional knowledge of the similar signal into CS via l 1
-l
1 and l 1 -l 2 minimization. We then establish bounds on the number of measurements required by
these
problems to successfully reconstruct the original signal. Our bounds and geometrical interpretations
reveal that if the prior information has good enough quality, l 1 -l 1 minimization improves the
performance of CS dramatically. In contrast, l 1 -l 2 minimization has a performance very similar to
classical CS, and brings no significant benefits. In addition, we use the insight provided by our
bounds to design practical schemes to improve prior information. All our findings are illustrated
with
experimental results.</p>
<a href="https://ieeexplore.ieee.org/abstract/document/7904593" target="_blank"><strong><i
class="fas fa-link"></i> Link</strong></a>
</div>
</div>
</div>
</div>
<div class="row">
<div class="col-md-12">
<div id="sokolic2017aistats" class="row publication">
<div class="col-md-4">
<a href="https://ieeexplore.ieee.org/document/8024476" target="_blank"><img class="img-fluid"
src="publications/2017/sokolic2017aistats.png" alt=""></a>
</div>
<div class="col-md-8 publication-text">
<strong>Title:</strong> Generalization Error of Invariant Classifiers<br>
<strong>Authors:</strong> J. Sokolic, R. Gyries, G. Sapiro, and M. R. D. Rodrigue<br>
<strong>Journal/Conference:</strong> Proceedings of the 20th International Conference on Artificial
Intelligence and Statistics (AISTATS)<br>
<p> <strong>Abstract:</strong> This paper studies the generalization error of invariant classifiers.
In
particular, we consider the common scenario where the classification task is invariant to certain
transformations of the input, and that the classifier is constructed (or learned) to be invariant to
these transformations. Our approach relies on factoring the input space into a product of a base
space
and a set of transformations. We show that whereas the generalization error of a non-invariant
classifier is proportional to the complexity of the input space, the generalization error of an
invariant classifier is proportional to the complexity of the base space. We also derive a set of
sufficient conditions on the geometry of the base space and the set of transformations that ensure
that the complexity of the base space is much smaller than the complexity of the input space. Our
analysis applies to general classifiers such as convolutional neural networks. We demonstrate the
implications of the developed theory for such classifiers with experiments on the MNIST and CIFAR-10
datasets.</p>