赛题十五：PIRBN 子目录 1D_sine_function #536

co63oc · 2023-09-12T09:13:51Z

PR types

Others

PR changes

Others

Describe

PaddlePaddle/Paddle#55663

赛题十五：PaddleScience 领域前沿论文复现
实现子目录 1D_nonlinear_spring

测试精度a_g

-	Tensorflow	Paddle
a_g	1.0004051	1.0005087

Tensorflow 运行

Paddle运行

paddle-bot · 2023-09-12T09:14:10Z

Thanks for your contribution!

wangguan1995

NTK plot needs to be implemented.

wangguan1995 · 2023-09-12T12:46:29Z

jointContribution/PIRBN/1D_nonlinear_spring/OPT.py

+paddle.framework.core.set_prim_eager_enabled(True)
+
+
+class Adam:


能否尝试使用 paddle adma优化器
https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/optimizer/Adam_cn.html

已修改Adam

wangguan1995 · 2023-09-13T03:53:09Z

jointContribution/PIRBN/1D_nonlinear_spring/Cal_jac.py

+import paddle
+
+
+def cal_adapt(pirbn, x):


参考：
源论文Page4，Theorem 3.1.4
论文Page8 https://www.sciencedirect.com/science/article/pii/S002199912100663X

此函数用于计算NTK矩阵，对应论文变量名称为Kg，请补充下相关函数注释（引用论文公式xx，页数xx）

对关键变量如 lambda_g lambda_b1 lambda_b2 进行注释说明（特征值，计算雅可比矩阵）

对NTK矩阵进行可视化，尝试复现论文Fig. 1，可以参考https://github.com/PredictiveIntelligenceLab/PINNsNTK/blob/master/PINNsNTK_Poisson1D.ipynb

修改为 pirbn.cal_ntk ，变量为 Kg
已增加引用公式页
可视化为 analytical_solution.output_fig subplot(2, 3, 4)

使用原tensorflow代码https://github.com/JinshuaiBai/PIRBN/tree/main/1D_sine_function，显示图和论文图Fig.1不同，设置参数mu=4, b=10.0

### Define mu mu = 4 ### Define the number of sample points ns = 51 ### Define the sample points' interval dx = 1./(ns-1) ### Initialise sample points' coordinates xy = np.zeros((ns, 1)).astype(np.float32) for i in range(0, ns): xy[i, 0] = i * dx xy_b = np.array([[0.], [1.]]) x = [xy, xy_b] y = [-4*mu**2*np.pi**2*np.sin(2*mu*np.pi*xy)] ### Set up raidal basis network n_in = 1 n_out = 1 n_neu = 61 b=10. c = [-0.1, 1.1]

原论文图

wangguan1995 · 2023-09-13T03:54:38Z

jointContribution/PIRBN/1D_nonlinear_spring/Main.py

+y = rbn(paddle.to_tensor(xy))
+y = y.numpy()
+plt.plot(xy, y)
+plt.plot(xy, xy * np.sin(xy))


解析解请单独命名为analytical_solution, 并进行注释

wangguan1995 · 2023-09-13T03:55:46Z

jointContribution/PIRBN/1D_nonlinear_spring/Main.py

+target_dir = os.path.join(os.path.dirname(__file__), "../target")
+if not os.path.exists(target_dir):
+    os.mkdir(target_dir)
+plt.savefig(os.path.join(target_dir, "1D_nonlinear_spring.png"))


请加一张absolute error计算结果的图片

analytical_solution 子图 plt.subplot(2, 3, 2) 计算absolute error

wangguan1995 · 2023-09-13T03:56:21Z

jointContribution/PIRBN/1D_nonlinear_spring/OPT.py

+paddle.framework.core.set_prim_eager_enabled(True)
+
+
+class Adam:


class 可以命名为Trainer更恰当

wangguan1995 · 2023-09-13T04:00:22Z

jointContribution/PIRBN/1D_nonlinear_spring/rbn_net.py

+            default_initializer=paddle.nn.initializer.Normal(mean=0.0, std=0.05),
+        )
+
+    def forward(self, inputs):  # Defines the computation from inputs to outputs


此处应该是用了类似论文Page13的激活函数
尝试单独抽象出激活函数 def activate（）

设置激活函数 RBF_layer1.rbf_activate

wangguan1995 · 2023-09-13T04:01:32Z

jointContribution/PIRBN/requirements.txt

@@ -0,0 +1,7 @@
+numpy


这个requirements在PaddleScience主目录下已经有了
如果有新的库需要添加，请加到主目录下的requirement.txt

wangguan1995

当前复现代码的公式有点问题，需要复现的论文指标为：Fig 1 Fig 2 Fig 3 Fig 6，具体内容见review

wangguan1995 · 2023-09-18T08:43:52Z

jointContribution/PIRBN/README.md

+
+ - 1D sine funtion (**Eq. 1** in the manuscript)
+
+      **PDE**: $\frac{\partial^2 }{\partial x^2}u(x)-4\mu^2\pi^2 sin(2\mu\pi(x))=0, x\in[0,1]$


之前的修改都很好，都是符合要求的，但是图片结果上还有一些不清楚的地方，特此说明下：
论文图片指标：

Fig 1 ：单层PINN方法（不是PIRBN），公式（13），mu = 4, 【tanh激活函数】，论文显示勉强收敛

Fig. 2 ：单层PINN方法（不是PIRBN），公式（13），改变 mu = 8, 【tanh激活函数】，病态了，论文的图显示结果很差

Fig. 3 ：单层PINN方法（不是PIRBN），改变公式（15）， mu = 4, 【tanh激活函数】，病态了，论文的图显示结果很差

Fig. 6：单层PIRBN（源代码实现），改变公式（15），改变 mu = 8, 改变【高斯激活函数】，结果非常优秀，说明PIRBN很有用

另外，此处公式好像和源代码对不上

Fig1, Fig2, Fig3 使用tanh和论文不一致

Fig6改变激活函数和论文一致
Fig1

Fig2

Fig3

Fig6

公式和源码是一致

不同激活函数设置

def gaussian_function(self, temp_x): x0 = ( paddle.reshape( paddle.arange(self.n_neu, dtype=paddle.get_default_dtype()), (1, self.n_neu), ) * (self.c[1] - self.c[0]) / (self.n_neu - 1) + self.c[0] ) x_new = temp_x - x0 s = self.b * self.b return paddle.exp(-(x_new * x_new) * s) def tanh_function(self, temp_x): return paddle.tanh(temp_x)

可能源代码有点问题Fig 1我用了这篇文章的Reference源码做了一下图
PredictiveIntelligenceLab/PINNsNTK@18ef519

可能jacobian那个函数有点问题，看看和这个仓库对齐一下

wangguan1995 · 2023-09-18T08:45:53Z

jointContribution/PIRBN/pirbn.py

+import paddle
+
+
+class Dif(paddle.nn.Layer):


这个类有些多余，请去掉并合并

wangguan1995 · 2023-09-18T08:47:55Z

jointContribution/PIRBN/pirbn.py

+        return [u_xx, u_b]
+
+    def cal_ntk(self, x):
+        # Formula (4), Page5, \gamma variable


是指这个公式吗？

https://arxiv.org/ftp/arxiv/papers/2304/2304.06234.pdf

这个论文是旧版的，作者8月3号更新了论文新地址和版本，辛苦更新一下页码

wangguan1995 · 2023-09-18T08:48:50Z

jointContribution/PIRBN/pirbn.py

+                gamma_g = gamma_g + paddle.sum(j**2) / n1
+            temp = paddle.concat((l1t[0], l1t[1].reshape((1, n_neu))), axis=1)
+            if i == 0:
+                # Fig.1, Page8, Kg variable


页码好像是5？

https://arxiv.org/ftp/arxiv/papers/2304/2304.06234.pdf

wangguan1995 · 2023-09-18T08:53:18Z

jointContribution/PIRBN/train.py

+            loss.backward()
+            self.optimizer.step()
+            self.optimizer.clear_grad()
+        return loss, [self.his_l1, self.his_l2]


返回值没用上, 可以去掉了

wangguan1995 · 2023-09-18T08:59:02Z

jointContribution/PIRBN/pirbn.py

+    def cal_ntk(self, x):
+        # Formula (4), Page5, \gamma variable
+        gamma_g = 0.0
+        gamma_b = 0.0


wangguan1995 · 2023-09-18T09:03:11Z

jointContribution/PIRBN/train.py

+        ]
+        self.y_train = paddle.to_tensor(y_train, dtype=paddle.get_default_dtype())
+        self.maxiter = maxiter
+        self.his_l1 = []


改为loss_b 并注释是boundary loss

wangguan1995 · 2023-09-18T09:05:02Z

jointContribution/PIRBN/train.py

+        self.y_train = paddle.to_tensor(y_train, dtype=paddle.get_default_dtype())
+        self.maxiter = maxiter
+        self.his_l1 = []
+        self.his_l2 = []


改为loss_g 并注释是eq loss

wangguan1995 · 2023-09-18T09:05:28Z

jointContribution/PIRBN/train.py

+
+    def evaluate(self):
+        # compute loss
+        loss, l1, l2 = self.Loss(self.x_train, self.y_train, self.a_g, self.a_b)


同上，增强可读性

wangguan1995 · 2023-09-18T09:08:14Z

jointContribution/PIRBN/rbn_net.py

+            + self.c[0]
+        )
+        x_new = temp_x - x0
+        return self.rbf_activate(x_new)


如果和PINN对比，可能会出现两种激活函数 tanh 和这个rbf_activate在论文里名称应该叫Gaussian function，改一下激活函数名

注释 #Formula (19), Page7

https://arxiv.org/ftp/arxiv/papers/2304/2304.06234.pdf
是在Page10

wangguan1995

need debug for fig 1

wangguan1995 · 2023-09-19T10:05:52Z

jointContribution/PIRBN/README.md

+
+ - 1D sine funtion (**Eq. 1** in the manuscript)
+
+      **PDE**: $\frac{\partial^2 }{\partial x^2}u(x)-4\mu^2\pi^2 sin(2\mu\pi(x))=0, x\in[0,1]$


可能源代码有点问题Fig 1我用了这篇文章的Reference源码做了一下图
PredictiveIntelligenceLab/PINNsNTK@18ef519

可能jacobian那个函数有点问题，看看和这个仓库对齐一下

wangguan1995 · 2023-09-19T10:08:47Z

jointContribution/PIRBN/pirbn.py

+        return [u_xx, u_b]
+
+    def cal_ntk(self, x):
+        # Formula (4), Page5, \gamma variable


这个论文是旧版的，作者8月3号更新了论文新地址和版本，辛苦更新一下页码

wangguan1995 · 2023-09-19T10:09:05Z

jointContribution/PIRBN/rbn_net.py

+            + self.c[0]
+        )
+        x_new = temp_x - x0
+        return self.rbf_activate(x_new)


wangguan1995 · 2023-09-19T11:16:33Z

jointContribution/PIRBN/train.py

@@ -0,0 +1,67 @@
+import paddle
+
+# Used to calculate the second-order derivatives


已增加 PINN 目录运行PINN.py, Kg结果图

Page页码已修改

注释已删除

wangguan1995

need to fix

jointContribution/PIRBN/rbn_net.py

docs/zh/examples/pirbn.md

jointContribution/PIRBN/main.py

docs/zh/examples/pirbn.md

jointContribution/PIRBN/main.py

jointContribution/PIRBN/rbn_net.py

wangguan1995

need fix

docs/zh/examples/pirbn.md

wangguan1995

Pass

jointContribution/PIRBN/jacobian_test.py

jointContribution/PIRBN/rbn_net.py

jointContribution/PIRBN/train.py

jointContribution/PIRBN/README.md

HydrogenSulfate

LGTM

* Add PIRBN * Fix * Fix * Fix * Fix * Fix * Fix * Fix * Fix * Fix * Fix * Fix * Fix * Fix * Fix * Fix * Fix

Add PIRBN

4bd73b2

paddle-bot bot added the contributor label Sep 12, 2023

wangguan1995 suggested changes Sep 13, 2023

View reviewed changes

luotao1 added the HappyOpenSource Pro 进阶版快乐开源活动，更具挑战性的任务 label Sep 13, 2023

luotao1 self-assigned this Sep 13, 2023

luotao1 mentioned this pull request Sep 13, 2023

中国软件开源创新大赛：飞桨框架任务挑战赛（下） PaddlePaddle/Paddle#55663

Closed

Fix

913dd05

co63oc force-pushed the pirbn branch from 9c08c32 to 913dd05 Compare September 13, 2023 23:44

co63oc changed the title ~~赛题十五：PIRBN 子目录 1D_nonlinear_spring~~ 赛题十五：PIRBN 子目录 1D_sine_function Sep 13, 2023

wangguan1995 reviewed Sep 18, 2023

View reviewed changes

co63oc added 2 commits September 19, 2023 08:31

Fix

b3c6919

Fix

c46baa4

co63oc force-pushed the pirbn branch from 6b72ad3 to c46baa4 Compare September 19, 2023 01:06

wangguan1995 suggested changes Sep 19, 2023

View reviewed changes

co63oc added 2 commits September 20, 2023 11:16

Fix

fe2a39d

Fix

a4e5982

co63oc force-pushed the pirbn branch from be7cfc2 to a4e5982 Compare September 26, 2023 08:53

luotao1 assigned wangguan1995 Oct 10, 2023

co63oc added 4 commits October 16, 2023 14:56

Fix

c64b40d

Fix

2b1ece3

Merge branch 'develop' into pirbn

4fc5a79

Fix

4626875

wangguan1995 reviewed Oct 16, 2023

View reviewed changes

co63oc added 4 commits October 16, 2023 19:52

Fix

fbe1ae8

Fix

c279d45

Fix

e4fed6f

Fix

0a59ed4

wangguan1995 reviewed Oct 17, 2023

View reviewed changes

docs/zh/examples/pirbn.md Outdated Show resolved Hide resolved

Fix

3672ad5

Fix

4244384

wangguan1995 approved these changes Oct 17, 2023

View reviewed changes

HydrogenSulfate requested changes Oct 17, 2023

View reviewed changes

Fix

55d416c

co63oc requested a review from HydrogenSulfate October 17, 2023 13:39

Fix

0c68825

HydrogenSulfate approved these changes Oct 20, 2023

View reviewed changes

HydrogenSulfate merged commit eab263b into PaddlePaddle:develop Oct 20, 2023

co63oc deleted the pirbn branch October 31, 2023 06:50

		paddle.framework.core.set_prim_eager_enabled(True)


		class Adam:


		- 1D sine funtion (Eq. 1 in the manuscript)

		PDE: $\frac{\partial^2 }{\partial x^2}u(x)-4\mu^2\pi^2 sin(2\mu\pi(x))=0, x\in[0,1]$

		@@ -0,0 +1,67 @@
		import paddle

		# Used to calculate the second-order derivatives

赛题十五：PIRBN 子目录 1D_sine_function #536

赛题十五：PIRBN 子目录 1D_sine_function #536

Conversation

co63oc commented Sep 12, 2023 • edited Loading

PR types

PR changes

Describe

paddle-bot bot commented Sep 12, 2023

wangguan1995 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangguan1995 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangguan1995 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangguan1995 left a comment

Choose a reason for hiding this comment

wangguan1995 left a comment

Choose a reason for hiding this comment

wangguan1995 left a comment

Choose a reason for hiding this comment

HydrogenSulfate left a comment

Choose a reason for hiding this comment

co63oc commented Sep 12, 2023 •

edited

Loading