赛题八：paddleviz能力扩展——Tensor信息展示 #56837

qiuwenbogdut · 2023-08-31T04:08:33Z

PR types

New features

PR changes

Others

Description

赛题简介：
飞桨深度学习框架提供了动态图编程的模式来开发深度学习模型（方便开发与调试），但动态图的反向图调试能力仍存在不足。@qiuwenbogdut 和 @Tomoko-hjf 两位社区开发者为Paddle开发了反向图可视化工具 paddleviz，很大程度上提升了反向图的调试能力。但我们还希望，在现有 paddleviz 基础上，扩展 paddleviz 的能力，进一步提升反向图的调试能力。

赛题要求：
深入学习Paddle动态图机制，掌握动态图反向图的基础数据结构之间的关系
学习了解 paddleviz 的原理
探索如何在 paddleviz 基础上，扩展反向图边（Tensor）的调试信息，包括不限于Tensor name、shape、dtype、place、stop_gradient 等。
探索以上数据可以通过框架的哪些数据结构中获得，何时、如何获得
探索如何将这些数据在 paddleviz 中展示

赛题八任务描述
 赛题八 tracking issue

当前这个PR计划完成的工作:

将反向节点的指针值作为一个grad_node的属性暴露给python端, 在python能获取到反向节点的指针值
打印反向节点的输入和输出值, 用于后面构建反向图中的边的信息 (反向图和边的信息可视化可用工具来构建显示: paddleviz) 打印的信息示例如下:

paddle-bot · 2023-08-31T04:08:38Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

wanghuancoder

third_party/flashattn的修改不要提交

wanghuancoder · 2023-09-11T02:57:18Z

paddle/fluid/eager/accumulation/accumulation_node.cc

+  std::stringstream ss;
+  ss << this;
+  std::string this_pointer = ss.str();


这段代码是没有使用的吧。删掉吧

wanghuancoder · 2023-09-11T02:57:33Z

paddle/fluid/eager/accumulation/accumulation_node.cc

  VLOG(3) << "Finish AD API Grad: GradNodeAccumulation";
-  if (VLOG_IS_ON(4)) {
+  VLOG(6) << "gradnode_ptr = " << this;
+  if (VLOG_IS_ON(6)) {


这个VLog的级别是不能改的。必须是4

wanghuancoder · 2023-09-11T02:58:22Z

paddle/fluid/eager/accumulation/accumulation_node.cc

    const char* TENSOR_X_GRAD_TEMPLATE = "(grad_out, [%s]), ";
    std::string output_x_grad_str = paddle::string::Sprintf(
        TENSOR_X_GRAD_TEMPLATE, egr::EagerUtils::TensorStr(grad_out));
    output_str += output_x_grad_str;
-    VLOG(4) << paddle::string::Sprintf(
+    VLOG(6) << paddle::string::Sprintf(


这个VLog的级别是不能改的。必须是4

wanghuancoder · 2023-09-11T02:58:33Z

paddle/fluid/eager/api/manual/eager_manual/nodes/add_n_node.cc

@@ -72,6 +72,30 @@ AddNGradNodeFinal::operator()(
    egr::CheckTensorHasNanOrInf("add_n_grad", returns);
  }

+  VLOG(6) << "gradnode_ptr = " << this;
+  if (VLOG_IS_ON(6)) {


这个VLog的级别是不能改的。必须是4

wanghuancoder · 2023-09-11T02:58:55Z

paddle/fluid/eager/api/manual/eager_manual/nodes/conv2d_nodes.cc

@@ -162,6 +163,41 @@ Conv2dGradNodeFinal::operator()(
    // Set TensorWrappers for Forward Outputs if needed
  }

+    VLOG(6) << "gradnode_ptr = " << this;
+    if (VLOG_IS_ON(6)) {


这个VLog的级别是不能改的。必须是4

wanghuancoder · 2023-09-11T03:01:09Z

paddle/fluid/eager/api/manual/eager_manual/nodes/sync_batch_norm_node.cc

@@ -457,7 +458,7 @@ SyncBatchNormGradNode::operator()(
    std::string output_bias_grad_str = paddle::string::Sprintf(
        TENSOR_BIAS_GRAD_TEMPLATE, egr::EagerUtils::TensorStr(bias_grad));
    output_str += output_bias_grad_str;
-    VLOG(4) << paddle::string::Sprintf(
+    VLOG(6) << paddle::string::Sprintf(


这个VLog的级别是不能改的。必须是4

wanghuancoder · 2023-09-11T03:01:22Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

+  //std::cout << "The pointer to the current object: " << this << std::endl;
+  //std::stringstream ss;
+  //ss << this;
+  //std::string this_pointer = ss.str();
+


这段代码删掉吧

wanghuancoder · 2023-09-11T03:01:35Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

@@ -259,10 +266,10 @@ class {} : public egr::GradNodeBase {{
 """

 AFTER_LOG_PRINT_TEMPLATE = """
-  if(VLOG_IS_ON(4)){{
+  if(VLOG_IS_ON(6)){{


这个VLog的级别是不能改的。必须是4

wanghuancoder · 2023-09-11T03:01:38Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

      const char* INPUT_PRINT_TEMPLATE = \"{{ Input: [%s],  \\n Output: [%s] }} \";
      {}
-      VLOG(4) << paddle::string::Sprintf(INPUT_PRINT_TEMPLATE, input_str, output_str);
+      VLOG(6) << paddle::string::Sprintf(INPUT_PRINT_TEMPLATE, input_str, output_str);


这个VLog的级别是不能改的。必须是4

wanghuancoder · 2023-09-11T03:02:44Z

paddle/fluid/eager/grad_node_info.cc

+      //打印一下 next_node 对象的指针值
+      std::cout << "next_node: " << next_node << std::endl;


这个日志是要保留的吗？如果要保留的话需要是英文注释，而且不能用std::cout,要用Vlog。如果不需要保留就删掉吧

wanghuancoder

LGTM

luotao1 · 2023-09-15T02:35:28Z

PR-CI-Codestyle-Check 需要解决格式问题

qiuwenbogdut · 2023-09-15T12:02:58Z

PR-CI-Codestyle-Check 需要解决格式问题

好的这边处理一下

wanghuancoder

LGTM

zyfncg · 2023-09-19T06:12:20Z

paddle/fluid/eager/auto_code_generator/generator/eager_gen.py

      VLOG(4) << paddle::string::Sprintf(INPUT_PRINT_TEMPLATE, input_str, output_str);
+      VLOG(6) << paddle::string::Sprintf(INPUT_PRINT_TEMPLATE, input_str, output_str);


这两行看上去一样，新加一个VLOG(6)的作用是什么呢？

原始的日志等级是4 等级不够高有些需要的信息没有打印出来,所以就增加了一个等级为6的日志打印

这里的信息打印重复了吧？

是的, 确实存在重复打印的问题, 这边修改一下

zyfncg · 2023-09-19T06:22:03Z

可以在描述里再补充下这个PR具体做的工作，以及这些LOG的运行截图吗？

qiuwenbogdut · 2023-09-19T08:07:05Z

可以在描述里再补充下这个PR具体做的工作，以及这些LOG的运行截图吗？

已经增加了这个PR的具体工作和日志打印图

wanghuancoder

LGTM

zyfncg · 2023-09-20T07:02:04Z

paddle/fluid/pybind/pybind.cc

@@ -870,6 +870,8 @@ PYBIND11_MODULE(libpaddle, m) {
          [](const std::shared_ptr<egr::GradNodeBase> &self) {
            return self->NextFunctions();
          })
+
+      .def("node_this_ptr", &egr::GradNodeBase::GetThisPtr)


node_this_ptr这么命名感觉有些奇怪，可以再想想有没有更好的命名

@wanghuancoder 欢哥命名这块有没有什么建议?

要不改成.def("node_ptr", &egr::GradNodeBase::GetPtr)吧
这个PR不要改了。新提一个PR修改。这个PR先合入。

for paddleviz

qiuwenbogdut added 18 commits June 6, 2023 18:11

[尝试] 给tensor增加一个属性, 这个属性是一个定值 1

260a4f7

暴露gradnode 并构建gradnode新的方法(用来测试)进行暴露给python python端可以访问

922f79f

开发grad_fn、next_functions两个API 并暴露到python端- 做一些规范化处理

4887161

增加一个单元测试

f54aff8

Merge branch 'develop' into dev_qiu

09e86f5

优化 code-style

fa9384c

Merge branch 'develop' into dev_qiu

bef4cc6

将单侧文件迁到正确的位置

1c3ad39

优化 code-style

d7925f6

删除无用注释

38a8645

解决 __main__ has no attribute

1059ae5

修改单侧文件

bcb5730

修改单侧脚本-temp

05e8edf

解决 grad_fn next_functions api 接口导致内存异常的问题

ba77d9a

修改单测内容

b5ebfaa

Merge branch 'develop' into dev_qiu

72d32fa

解决 code-style 问题

fd0c655

初版本-增加节点指针值获取接口, 增加打印构建边需要的信息

a442c1c

paddle-bot bot added the contributor External developers label Aug 31, 2023

qiuwenbogdut mentioned this pull request Aug 31, 2023

赛题八：paddleviz能力扩展——Tensor信息展示 Tracking Issue #56479

Closed

4 tasks

luotao1 assigned luotao1 and wanghuancoder Sep 1, 2023

增加eager_manual 中反向节点的打印信息

ca6a064

qiuwenbogdut force-pushed the dev_qiu branch from ca6a064 to fd0c655 Compare September 10, 2023 12:37

wanghuancoder reviewed Sep 11, 2023

View reviewed changes

不修改原有的日志等级

657412a

qiuwenbogdut force-pushed the dev_qiu branch from fb84c53 to fd0c655 Compare September 14, 2023 02:11

删除掉不必要的打印信息

c86c61c

qiuwenbogdut added 2 commits September 15, 2023 01:30

不修改之前的日志等级2

0fd9e8f

增加反向节点地址信息获取, 增加打印边的信息, 不修改原有日志等级

e6edf3d

qiuwenbogdut force-pushed the dev_qiu branch from e54e7b1 to e6edf3d Compare September 15, 2023 01:33

qiuwenbogdut requested a review from wanghuancoder September 15, 2023 01:46

wanghuancoder previously approved these changes Sep 15, 2023

View reviewed changes

Merge branch 'develop' into dev_qiu

3160ca4

qiuwenbogdut dismissed wanghuancoder’s stale review via 3160ca4 September 15, 2023 02:18

qiuwenbogdut added 2 commits September 15, 2023 11:34

优化代码格式

9c015dd

优化代码格式

daa5381

修改格式问题2

7b26f4a

wanghuancoder previously approved these changes Sep 19, 2023

View reviewed changes

zyfncg reviewed Sep 19, 2023

View reviewed changes

qiuwenbogdut requested a review from zyfncg September 19, 2023 08:07

解决一下日志重复输出的问题

988dece

qiuwenbogdut dismissed wanghuancoder’s stale review via 988dece September 19, 2023 11:58

qiuwenbogdut requested a review from wanghuancoder September 20, 2023 01:43

wanghuancoder approved these changes Sep 20, 2023

View reviewed changes

zyfncg approved these changes Sep 20, 2023

View reviewed changes

zyfncg reviewed Sep 20, 2023

View reviewed changes

wanghuancoder merged commit d132cec into PaddlePaddle:develop Sep 20, 2023

iosmers pushed a commit to iosmers/Paddle that referenced this pull request Sep 21, 2023

赛题八：paddleviz能力扩展——Tensor信息展示 (PaddlePaddle#56837)

29c89b5

for paddleviz

Frida-a pushed a commit to Frida-a/Paddle that referenced this pull request Oct 14, 2023

赛题八：paddleviz能力扩展——Tensor信息展示 (PaddlePaddle#56837)

6057eab

for paddleviz

danleifeng pushed a commit to danleifeng/Paddle that referenced this pull request Nov 14, 2023

赛题八：paddleviz能力扩展——Tensor信息展示 (PaddlePaddle#56837)

f87546b

for paddleviz

qiuwenbogdut mentioned this pull request Dec 5, 2023

WAVE SUMMIT+2023下半年飞桨开源之星评选-信息征集 PaddlePaddle/community#765

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

赛题八：paddleviz能力扩展——Tensor信息展示 #56837

赛题八：paddleviz能力扩展——Tensor信息展示 #56837

qiuwenbogdut commented Aug 31, 2023 •

edited

Loading

paddle-bot bot commented Aug 31, 2023

wanghuancoder left a comment

wanghuancoder Sep 11, 2023

wanghuancoder Sep 11, 2023

wanghuancoder Sep 11, 2023

wanghuancoder Sep 11, 2023

wanghuancoder Sep 11, 2023

wanghuancoder Sep 11, 2023

wanghuancoder Sep 11, 2023

wanghuancoder Sep 11, 2023

wanghuancoder Sep 11, 2023

wanghuancoder Sep 11, 2023

wanghuancoder left a comment

luotao1 commented Sep 15, 2023

qiuwenbogdut commented Sep 15, 2023

wanghuancoder left a comment

zyfncg Sep 19, 2023

qiuwenbogdut Sep 19, 2023

zyfncg Sep 19, 2023 •

edited

Loading

qiuwenbogdut Sep 19, 2023

zyfncg commented Sep 19, 2023

qiuwenbogdut commented Sep 19, 2023

wanghuancoder left a comment

zyfncg Sep 20, 2023

qiuwenbogdut Sep 20, 2023

wanghuancoder Sep 20, 2023

		//打印一下 next_node 对象的指针值
		std::cout << "next_node: " << next_node << std::endl;

		VLOG(4) << paddle::string::Sprintf(INPUT_PRINT_TEMPLATE, input_str, output_str);
		VLOG(6) << paddle::string::Sprintf(INPUT_PRINT_TEMPLATE, input_str, output_str);

赛题八：paddleviz能力扩展——Tensor信息展示 #56837

赛题八：paddleviz能力扩展——Tensor信息展示 #56837

Conversation

qiuwenbogdut commented Aug 31, 2023 • edited Loading

PR types

PR changes

Description

当前这个PR计划完成的工作:

paddle-bot bot commented Aug 31, 2023

wanghuancoder left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wanghuancoder left a comment

Choose a reason for hiding this comment

luotao1 commented Sep 15, 2023

qiuwenbogdut commented Sep 15, 2023

wanghuancoder left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyfncg Sep 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zyfncg commented Sep 19, 2023

qiuwenbogdut commented Sep 19, 2023

wanghuancoder left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

qiuwenbogdut commented Aug 31, 2023 •

edited

Loading

zyfncg Sep 19, 2023 •

edited

Loading