Merge branch 'main' of https://github.com/heatingma/pygmtools

README.md

@@ -95,6 +95,8 @@ This library offers user-friendly API for the following solvers:
       based neural graph matching solver for matching individual graphs.
     * [Neural graph matching solver (NGM)](https://pygmtools.readthedocs.io/en/latest/api/_autosummary/pygmtools.neural_solvers.ngm.html) [12]
       for the general quadratic assignment formulation.
+    * [Graph edit neural network A-star (GENN-A*)](https://pygmtools.readthedocs.io/en/latest/api/_autosummary/pygmtools.neural_solvers.genn_astar.html) [13] 
+      for the graph edit distance problem.
 
 ## Available Backends
 This library is designed to support multiple backends with the same set of API. 
@@ -122,7 +124,7 @@ The following backends are available:
 
 <img src="https://pygmtools.readthedocs.io/en/latest/_images/tensorflow_logo.png" alt="tensorflow logo" width="200"/>
 
-### Development status (0.4.2)
+### Development status
 
 |                     | Numpy | PyTorch | Jittor | PaddlePaddle | Tensorflow | MindSpore |
 | ------------------- | ----- | ------- | ------ | ------------ | ---------- | --------- |
@@ -180,26 +182,28 @@ Shanghai Jiao Tong University.
 ## References
 <!--MLA style references-->
 
-[1] Sinkhorn, Richard, and Paul Knopp. "Concerning nonnegative matrices and doubly stochastic matrices." Pacific Journal of Mathematics 21.2 (1967): 343-348.
+[1] Sinkhorn, Richard, and Paul Knopp. "Concerning nonnegative matrices and doubly stochastic matrices." _Pacific Journal of Mathematics_ 21.2 (1967): 343-348.
 
-[2] Munkres, James. "Algorithms for the assignment and transportation problems." Journal of the society for industrial and applied mathematics 5.1 (1957): 32-38.
+[2] Munkres, James. "Algorithms for the assignment and transportation problems." _Journal of the Society for Industrial and Applied Mathematics_ 5.1 (1957): 32-38.
 
-[3] Leordeanu, Marius, and Martial Hebert. "A spectral technique for correspondence problems using pairwise constraints." International Conference on Computer Vision (2005).
+[3] Leordeanu, Marius, and Martial Hebert. "A spectral technique for correspondence problems using pairwise constraints." _International Conference on Computer Vision_ (2005).
 
-[4] Cho, Minsu, Jungmin Lee, and Kyoung Mu Lee. "Reweighted random walks for graph matching." European conference on Computer vision. Springer, Berlin, Heidelberg, 2010.
+[4] Cho, Minsu, Jungmin Lee, and Kyoung Mu Lee. "Reweighted random walks for graph matching." _European conference on Computer Vision_ (2010).
 
-[5] Leordeanu, Marius, Martial Hebert, and Rahul Sukthankar. "An integer projected fixed point method for graph matching and map inference." Advances in neural information processing systems 22 (2009).
+[5] Leordeanu, Marius, Martial Hebert, and Rahul Sukthankar. "An integer projected fixed point method for graph matching and map inference." _Advances in Neural Information Processing Systems_ 22 (2009).
 
-[6] Yan, Junchi, et al. "Multi-graph matching via affinity optimization with graduated consistency regularization." IEEE transactions on pattern analysis and machine intelligence 38.6 (2015): 1228-1242.
+[6] Yan, Junchi, et al. "Multi-graph matching via affinity optimization with graduated consistency regularization." _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 38.6 (2015): 1228-1242.
 
-[7] Jiang, Zetian, Tianzhe Wang, and Junchi Yan. "Unifying offline and online multi-graph matching via finding shortest paths on supergraph." IEEE transactions on pattern analysis and machine intelligence 43.10 (2020): 3648-3663.
+[7] Jiang, Zetian, Tianzhe Wang, and Junchi Yan. "Unifying offline and online multi-graph matching via finding shortest paths on supergraph." _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 43.10 (2020): 3648-3663.
 
-[8] Solé-Ribalta, Albert, and Francesc Serratosa. "Graduated assignment algorithm for multiple graph matching based on a common labeling." International Journal of Pattern Recognition and Artificial Intelligence 27.01 (2013): 1350001.
+[8] Solé-Ribalta, Albert, and Francesc Serratosa. "Graduated assignment algorithm for multiple graph matching based on a common labeling." _International Journal of Pattern Recognition and Artificial Intelligence_ 27.01 (2013): 1350001.
 
-[9] Wang, Runzhong, Junchi Yan, and Xiaokang Yang. "Graduated assignment for joint multi-graph matching and clustering with application to unsupervised graph matching network learning." Advances in Neural Information Processing Systems 33 (2020): 19908-19919.
+[9] Wang, Runzhong, Junchi Yan, and Xiaokang Yang. "Unsupervised Learning of Graph Matching with Mixture of Modes via Discrepancy Minimization." _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 45.8 (2023): 10500-10518.
 
-[10] Wang, Runzhong, Junchi Yan, and Xiaokang Yang. "Combinatorial learning of robust deep graph matching: an embedding based approach." IEEE Transactions on Pattern Analysis and Machine Intelligence (2020).
+[10] Wang, Runzhong, Junchi Yan, and Xiaokang Yang. "Combinatorial learning of robust deep graph matching: an embedding based approach." _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 45.6 (2023): 6984-7000.
 
-[11] Yu, Tianshu, et al. "Learning deep graph matching with channel-independent embedding and hungarian attention." International conference on learning representations. 2019.
+[11] Yu, Tianshu, et al. "Learning deep graph matching with channel-independent embedding and hungarian attention." _International Conference on Learning Representations_. 2019.
 
-[12] Wang, Runzhong, Junchi Yan, and Xiaokang Yang. "Neural graph matching network: Learning lawler’s quadratic assignment problem with extension to hypergraph and multiple-graph matching." IEEE Transactions on Pattern Analysis and Machine Intelligence (2021).
+[12] Wang, Runzhong, Junchi Yan, and Xiaokang Yang. "Neural graph matching network: Learning lawler’s quadratic assignment problem with extension to hypergraph and multiple-graph matching." _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 44.9 (2022): 5261-5279.
+
+[13] Wang, Runzhong, Junchi Yan, and Xiaokang Yang. "Combinatorial Learning of Graph Edit Distance via Dynamic Embedding." _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ (2021): 5241-5250.

docs/guide/benchmark.rst

@@ -185,13 +185,29 @@ Overall Comparison
 ^^^^^^^^^^^^^^^^^^^
 Charts below illustrate the results of our experimental investigation into the efficiency of some ``pygmtools`` solvers, comparing execution time among different backends and against previous packages (``ZAC_GM`` for classic solvers and ``Multiway`` for multigraph solvers).
 
-Experiments of ``pygmtools`` were conducted on both CPU and GPU to explore the acceleration of CUDA for graph matching problems, and the existing packages were executed by both Matlab and Octave. Also examined are the variance of computation time with different input graph sizes and the dissimilar trends in time increments on different devices and backends. These information combined provide rich indication in hope that you can select a preferable backend and determine the necessity of enabling CUDA for specific problem scales.
+.. Note ::
+  Input affinity matrices are randomly generated with a fixed batchsize of 64 and the solving time of multigraph algorithms are evaluated upon matching 10 graphs. The execution times have been averaged across 50 runs, with the first run of each test configuration excluded to mitigate initialization biases.
+
+Solving time on CPU
+++++++++++++++++++++++
+The following results were conducted on consistent CPU hardware to ensure fair comparison. To highlight, our CPU implementations of GM solvers are on-par and usually more
+efficient than the open-source Octave implementation. 
+
+.. image:: ../images/ori_time_classic_cpu.png
+.. image:: ../images/ori_time_multi_cpu.png
+
+Solving time on GPU
+++++++++++++++++++++++
+The following results were conducted on consistent GPU hardware to explore the acceleration of CUDA for graph matching problems. The speed-up brought by GPU
+becomes more significant when the number of nodes grows, because the calling of GPU introduces non-negligible overhead with a smaller number of nodes. Note that log-scaled times are used for visual clarity and the dashed line labeled ``Pytorch-CPU`` acts as a baseline for 
+your direct reference.
 
-.. image:: ../images/logtime.png
+.. image:: ../images/log_time_classic_gpu.png
+.. image:: ../images/log_time_multi_gpu.png
 
 Original Results
 ^^^^^^^^^^^^^^^^^
-We provide the original data of our time tests here. Input affinity matrices are randomly generated with a fixed batchsize of 64 and the execution times have been averaged across 50 runs, with the first run of each test configuration excluded to mitigate initialization biases.
+Original data of our time tests are presented below.
 
 .. Note ::
   All experiments were performed on a consistent platform of Linux Ubuntu 20.04 with Python 3.9.17 and the latest compatible versions of the numerical backends listed as follows. Runtime discrepancy shall occur due to different platform, package version, CUDA version, hardware configuration, etc.

pygmtools/__init__.py

@@ -18,7 +18,7 @@
 set_backend = utils.set_backend
 
 BACKEND = 'numpy'
-__version__ = '0.4.2'
+__version__ = '0.4.3'
 __author__ = 'ThinkLab at SJTU'
 
 SUPPORTED_BACKENDS = [

pygmtools/multi_graph_solvers.py

@@ -1,6 +1,20 @@
 r"""
 Classic (learning-free) **multi-graph matching** solvers. These multi-graph matching solvers are recommended to solve
 the joint matching problem of multiple graphs.
+
+.. note::
+
+    Multi-graph matching means jointly matching :math:`\geq` 3 graphs. It is different from two-graph matching because
+    multi-graph matching is a more challenging problem and requires different solvers.
+
+    What makes multi-graph matching more challenging is the so-called **cycle-consistency constraint**: a multi-graph
+    matching solution has to be **cycle-consistent**, meaning that any pairwise matching result should be the same as
+    the result propagated from a third graph. For example, denote :math:`\mathbf{X}_{i,j}` as the matching matrix
+    between ``graph i`` and ``graph j``, it requires
+    :math:`\forall i,j,k, \mathbf{X}_{i,j} = \mathbf{X}_{i,k} \mathbf{X}_{k,j}`.
+    Such an extra constraint offers an extra source of information, but makes the problem itself more challenging
+    to solve.
+
 """
 
 # Copyright (c) 2022 Thinklab@SJTU