Application Layer Dashboard #523

Dhruv-J · 2023-10-04T23:18:36Z

This PR adds an Application Layer Dashboard to include visualizations for layer
seven Network Policies and HTTP values within the flows table. The changes
include the following:

A new dashboard named Application Layer Visibility, which contains two
panels, a Chord Panel to illustrate layer seven Network Policies, and a table
panel that shows the HTTP values for each flow.
Since the default grafana table plugin isn't able to handle nested JSON
values and aggregate them in a clean manner, this PR introduces a new plugin
called http-table-plugin that leverages the material-ui library to create a
simple table that includes the source and destination IPs and ports, with
expandable rows that show multiple HTTP value sets if a flow contains them.
This plugin is created using the updated @grafana/create-plugin tool, which
contains a .config directory for all plugin build and run configuration files.
Testing for the application layer dashboard and unit testing for the
http-table-plugin.

This PR builds upon #417, which adds additional fields for L7 visibility.

Dhruv-J · 2023-10-04T23:19:22Z

will update docs with following images:

codecov · 2023-10-04T23:22:51Z

Codecov Report

Attention: Patch coverage is 98.14815% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 70.87%. Comparing base (5ea4756) to head (67ae723).
Report is 27 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #523      +/-   ##
==========================================
+ Coverage   70.51%   70.87%   +0.35%     
==========================================
  Files          40       48       +8     
  Lines        5253     5565     +312     
  Branches        0       45      +45     
==========================================
+ Hits         3704     3944     +240     
- Misses       1372     1437      +65     
- Partials      177      184       +7

Flag	Coverage Δ		*Carryforward flag
kind-e2e-tests	`62.48% <ø> (ø)`		Carriedforward from 2090b55
kind-multi-cluster-e2e-tests	`43.17% <ø> (ø)`
python-coverage	`55.75% <ø> (ø)`		Carriedforward from 2090b55
unit-tests	`70.42% <ø> (+0.27%)`	⬆️

*This pull request uses carry forward flags. Click here to find out more.

Files	Coverage Δ
...ns/grafana-http-table-plugin/.config/jest-setup.js	`100.00% <100.00%> (ø)`
...ugins/grafana-http-table-plugin/src/TablePanel.tsx	`100.00% <100.00%> (ø)`
...able-plugin/.config/jest/mocks/react-inlinesvg.tsx	`83.33% <83.33%> (ø)`

... and 5 files with indirect coverage changes

Dhruv-J · 2023-10-16T18:25:04Z

tushartathgur · 2023-10-17T18:21:25Z

build/charts/theia/provisioning/dashboards/application_layer_dashboard.json

+  "uid": "tCTkERMSk",
+  "version": 1,
+  "weekStart": ""
+}


add a blank line

tushartathgur · 2023-10-17T18:22:34Z

build/charts/theia/provisioning/dashboards/homepage.json

@@ -576,7 +576,7 @@
            },
            "id": 27,
            "options": {
-                "content": "The following seven dashboards are pre-built and are recommended for network flow\nvisualization.\n\n<span style=\"color:rgb(184, 119, 217)\">Flow Records Dashboard</span> displays the flow records being captured in the selected\ntime range.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-Pod Flows Dashboard</span> shows cumulative bytes and throughput of Pod-to-Pod\ntraffic.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-External Flows Dashboard</span> shows cumulative bytes and throughput of\ntraffic from in-cluster to out-of-cluster. The destination of such a traffic flow\nis represented by the destination IP address.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-Service Flows Dashboard</span> shows cumulative bytes and throughput of\nPod-to-Service flows. The destination of such a traffic is represented by the\ndestination Service metadata.\n\n<span style=\"color:rgb(184, 119, 217)\">Node-to-Node Flows Dashboard</span> visualizes the Node-to-Node traffic, including intra-Node\nand inter-Node flows.\n\n<span style=\"color:rgb(184, 119, 217)\">Network-Policy Flows Dashboard</span> visualizes both the traffic with NetworkPolicies enforced,\nand unprotected traffic.\n\n<span style=\"color:rgb(184, 119, 217)\">Network Topology Dashboard</span> visualizes both Pod-to-Pod and Pod-to-Service traffic via a graph.",
+                "content": "The following eight dashboards are pre-built and are recommended for network flow\nvisualization.\n\n<span style=\"color:rgb(184, 119, 217)\">Flow Records Dashboard</span> displays the flow records being captured in the selected\ntime range.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-Pod Flows Dashboard</span> shows cumulative bytes and throughput of Pod-to-Pod\ntraffic.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-External Flows Dashboard</span> shows cumulative bytes and throughput of\ntraffic from in-cluster to out-of-cluster. The destination of such a traffic flow\nis represented by the destination IP address.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-Service Flows Dashboard</span> shows cumulative bytes and throughput of\nPod-to-Service flows. The destination of such a traffic is represented by the\ndestination Service metadata.\n\n<span style=\"color:rgb(184, 119, 217)\">Node-to-Node Flows Dashboard</span> visualizes the Node-to-Node traffic, including intra-Node\nand inter-Node flows.\n\n<span style=\"color:rgb(184, 119, 217)\">Network-Policy Flows Dashboard</span> visualizes both the traffic with NetworkPolicies enforced,\nand unprotected traffic.\n\n<span style=\"color:rgb(184, 119, 217)\">Network Topology Dashboard</span> visualizes both Pod-to-Pod and Pod-to-Service traffic via a graph.\n\n<span style=\"color:rgb(184, 119, 217)\">Application Visibility Dashboard</span> shows cumulative bytes and throughput for L7 NetworkPolicies\nas well as corresponding HTTP values.",


Application Visibility Dashboard displays cumulative bytes and throughput for L7 NetworkPolicies\nas well as corresponding HTTP data.

tushartathgur · 2023-10-17T18:23:41Z

build/charts/theia/provisioning/dashboards/network_topology_dashboard.json

+            }
+          },
+          "queryType": "sql",
+          "rawSql": "SELECT sourcePodName, sourcePodLabels, sourcePodNamespace, sourceNodeName, destinationPodName, destinationPodLabels, destinationNodeName, destinationServicePortName, octetDeltaCount FROM flows\nWHERE sourcePodNamespace NOT IN ('kube-system', 'flow-visibility', 'flow-aggregator')\nAND destinationPodNamespace NOT IN ('kube-system', 'flow-visibility', 'flow-aggregator')\nAND ( destinationPodName != '' OR sourcePodName != '' )\nAND octetDeltaCount != 0\nAND httpVals == ''\nAND $__timeFilter(flowEndSeconds)\nORDER BY flowEndSeconds DESC",


Shouldn't it be based on the l7ProtocolName instead of httpvals?

tushartathgur · 2023-10-17T18:24:00Z

build/charts/theia/provisioning/dashboards/network_topology_dashboard.json

@@ -60,11 +101,11 @@
            }
          },
          "queryType": "sql",
-          "rawSql": "SELECT sourcePodName, sourcePodLabels, sourcePodNamespace, sourceNodeName, destinationPodName, destinationPodLabels, destinationNodeName, destinationServicePortName, octetDeltaCount FROM flows\nWHERE sourcePodNamespace NOT IN ('kube-system', 'flow-visibility', 'flow-aggregator')\nAND destinationPodNamespace NOT IN ('kube-system', 'flow-visibility', 'flow-aggregator')\nAND destinationPodName != ''\nAND sourcePodName != ''\nAND octetDeltaCount != 0\nAND $__timeFilter(flowEndSeconds)\nORDER BY flowEndSeconds DESC",
+          "rawSql": "SELECT sourcePodName, sourcePodLabels, sourcePodNamespace, sourceNodeName, sourceIP, sourceTransportPort, destinationPodName, destinationPodLabels, destinationNodeName, destinationServicePortName, destinationIP, destinationTransportPort, octetDeltaCount, httpVals FROM flows\nWHERE sourcePodNamespace NOT IN ('kube-system', 'flow-visibility', 'flow-aggregator')\nAND destinationPodNamespace NOT IN ('kube-system', 'flow-visibility', 'flow-aggregator')\nAND ( destinationPodName != '' OR sourcePodName != '' )\nAND octetDeltaCount != 0\nAND httpVals != ''\nAND $__timeFilter(flowEndSeconds)\nORDER BY flowEndSeconds DESC",


tushartathgur · 2023-10-17T18:24:14Z

build/charts/theia/provisioning/dashboards/network_topology_dashboard.json

  "weekStart": ""
-}
+}


tushartathgur · 2023-10-17T18:25:09Z

build/yamls/flow-visibility.yml

@@ -2770,7 +2928,7 @@ data:
                },
                "id": 27,
                "options": {
-                    "content": "The following seven dashboards are pre-built and are recommended for network flow\nvisualization.\n\n<span style=\"color:rgb(184, 119, 217)\">Flow Records Dashboard</span> displays the flow records being captured in the selected\ntime range.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-Pod Flows Dashboard</span> shows cumulative bytes and throughput of Pod-to-Pod\ntraffic.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-External Flows Dashboard</span> shows cumulative bytes and throughput of\ntraffic from in-cluster to out-of-cluster. The destination of such a traffic flow\nis represented by the destination IP address.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-Service Flows Dashboard</span> shows cumulative bytes and throughput of\nPod-to-Service flows. The destination of such a traffic is represented by the\ndestination Service metadata.\n\n<span style=\"color:rgb(184, 119, 217)\">Node-to-Node Flows Dashboard</span> visualizes the Node-to-Node traffic, including intra-Node\nand inter-Node flows.\n\n<span style=\"color:rgb(184, 119, 217)\">Network-Policy Flows Dashboard</span> visualizes both the traffic with NetworkPolicies enforced,\nand unprotected traffic.\n\n<span style=\"color:rgb(184, 119, 217)\">Network Topology Dashboard</span> visualizes both Pod-to-Pod and Pod-to-Service traffic via a graph.",
+                    "content": "The following eight dashboards are pre-built and are recommended for network flow\nvisualization.\n\n<span style=\"color:rgb(184, 119, 217)\">Flow Records Dashboard</span> displays the flow records being captured in the selected\ntime range.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-Pod Flows Dashboard</span> shows cumulative bytes and throughput of Pod-to-Pod\ntraffic.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-External Flows Dashboard</span> shows cumulative bytes and throughput of\ntraffic from in-cluster to out-of-cluster. The destination of such a traffic flow\nis represented by the destination IP address.\n\n<span style=\"color:rgb(184, 119, 217)\">Pod-to-Service Flows Dashboard</span> shows cumulative bytes and throughput of\nPod-to-Service flows. The destination of such a traffic is represented by the\ndestination Service metadata.\n\n<span style=\"color:rgb(184, 119, 217)\">Node-to-Node Flows Dashboard</span> visualizes the Node-to-Node traffic, including intra-Node\nand inter-Node flows.\n\n<span style=\"color:rgb(184, 119, 217)\">Network-Policy Flows Dashboard</span> visualizes both the traffic with NetworkPolicies enforced,\nand unprotected traffic.\n\n<span style=\"color:rgb(184, 119, 217)\">Network Topology Dashboard</span> visualizes both Pod-to-Pod and Pod-to-Service traffic via a graph.\n\n<span style=\"color:rgb(184, 119, 217)\">Application Visibility Dashboard</span> shows cumulative bytes and throughput for L7 NetworkPolicies\nas well as corresponding HTTP values.",


same as above

tushartathgur · 2023-10-17T18:29:07Z

plugins/grafana-custom-plugins/grafana-dependency-plugin/.config/Dockerfile

@@ -0,0 +1,16 @@
+ARG grafana_version=latest


why use latest and not a verified version?

will update to 9.1.3

yuntanghsu · 2023-10-19T06:09:07Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/tsconfig.json

@@ -0,0 +1,3 @@
+{
+  "extends": "./.config/tsconfig.json"
+}


There are lots of generated file missing empty line. Do we have a method to avoid that? Or we should add it manually?

I don't think we do, I will add them manually

missing empty line

yuntanghsu · 2023-10-19T06:11:08Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/components/TablePanel.tsx

@@ -0,0 +1,133 @@
+import React, { useMemo } from 'react';


Maybe we can add unit test for this file to verify if the data are in our component correctly?

yes will do, working on getting tests to run locally

yuntanghsu · 2023-10-19T06:14:27Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/package.json

+  "scripts": {
+    "build": "webpack -c ./.config/webpack/webpack.config.ts --env production",
+    "dev": "webpack -w -c ./.config/webpack/webpack.config.ts --env development",
+    "test": "jest --watch --onlyChanged",


Remove watch, otherwise it won't stop when we do the typescript unit test in the workflow.

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/components/TablePanel.test.tsx

This PR adds an Application Layer Dashboard to include visualizations for layer seven Network Policies and HTTP values within the flows table. Signed-off-by: Dhruv-J <[email protected]>

Signed-off-by: Dhruv-J <[email protected]>

yuntanghsu · 2023-11-20T20:22:46Z

build/charts/theia/provisioning/dashboards/network_topology_dashboard.json

+            }
+          },
+          "queryType": "sql",
+          "rawSql": "SELECT sourcePodName, sourcePodLabels, sourcePodNamespace, sourceNodeName, destinationPodName, destinationPodLabels, destinationNodeName, destinationServicePortName, octetDeltaCount FROM flows\nWHERE sourcePodNamespace NOT IN ('kube-system', 'flow-visibility', 'flow-aggregator')\nAND destinationPodNamespace NOT IN ('kube-system', 'flow-visibility', 'flow-aggregator')\nAND ( destinationPodName != '' OR sourcePodName != '' )\nAND octetDeltaCount != 0\nAND httpVals == ''\nAND $__timeFilter(flowEndSeconds)\nORDER BY flowEndSeconds DESC",


Do we want to exclude the ns local-path-storage, which is created in the kind cluster by default?

I've already added that into #488, i'll remove it from this PR, don't know why it's showing up as a change

yuntanghsu · 2023-11-20T20:23:23Z

build/charts/theia/provisioning/dashboards/network_topology_dashboard.json

  "weekStart": ""
-}
+}


yuntanghsu · 2023-11-20T20:24:33Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/.config/.prettierrc.js

+  "singleQuote": true,
+  "useTabs": false,
+  "tabWidth": 2
+};


plugins/grafana-custom-plugins/grafana-http-table-plugin/.config/Dockerfile

plugins/grafana-custom-plugins/grafana-http-table-plugin/.eslintrc

plugins/grafana-custom-plugins/grafana-http-table-plugin/.nvmrc

yuntanghsu · 2023-11-20T20:25:53Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/.prettierrc.js

+module.exports = {
+  // Prettier configuration provided by Grafana scaffolding
+  ...require("./.config/.prettierrc.js")
+};


plugins/grafana-custom-plugins/grafana-http-table-plugin/package.json

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/components/TablePanel.test.tsx

Signed-off-by: Dhruv-J <[email protected]>

yuntanghsu · 2023-11-22T22:27:18Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/cypress/integration/01-smoke.spec.ts

@@ -0,0 +1,10 @@
+import { e2e } from '@grafana/e2e';
+
+e2e.scenario({


Do we need this e2e test for Grafana?
cc @heanlan

Can we run it with automation? e.g. will github action trigger it?

I can remove this, I don't think it's testing anything important, nor is it run during yarn test. @yuntanghsu thoughts?

Signed-off-by: Dhruv-J <[email protected]>

heanlan

If we would like to deploy and test the dashboard by ourselves, how should we do it? Where can we get the sample http data? Could you give some instructions?

heanlan · 2023-12-04T20:30:24Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/README.md

@@ -0,0 +1,114 @@
+<!-- This README file is going to be the one displayed on the Grafana.com website for your plugin. Uncomment and replace the content here before publishing.
+
+Remove any remaining comments before publishing as these may be displayed on Grafana.com -->


remove the first two lines of comment?

heanlan · 2023-12-04T20:33:00Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/README.md

+```sql
+SELECT httpVals
+FROM flows
+WHERE l7ProtocolName!=''


I think we can remove WHERE l7ProtocolName!='' in this query example for better clarity to external users.

heanlan · 2023-12-04T20:33:56Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/README.md

+Use the grafana-cli tool to install chord-panel-plugin from the commandline:
+
+```shell
+grafana-cli --pluginUrl https://github.com/Dhruv-J/grafana-http-table-plugin/archive/refs/tags/v2.zip plugins install theia-grafana-dependency-plugin


Suggested change

grafana-cli --pluginUrl https://github.com/Dhruv-J/grafana-http-table-plugin/archive/refs/tags/v2.zip plugins install theia-grafana-dependency-plugin

grafana-cli --pluginUrl https://github.com/Dhruv-J/grafana-http-table-plugin/archive/refs/tags/v2.zip plugins install theia-grafana-http-table-plugin

heanlan · 2023-12-04T20:35:19Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/img/logo.svg

this file can be removed as we are using antrea logo

heanlan · 2023-12-04T20:35:37Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/README.md

shouldn't this file be moved to plugins/grafana-custom-plugins/grafana-http-table-plugin/ ?

will do, I'll replace the README in plugins/grafana-custom-plugins/grafana-http-table-plugin/ with the one in /src

heanlan · 2023-12-04T21:10:37Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/components/TablePanel.test.tsx

+    props.options = {};
+    let component = shallow(<TablePanel {...props} />);
+    React.useLayoutEffect = React.useEffect;
+    console.log('component text: '+component.render().text());


heanlan · 2023-12-04T21:15:28Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/components/TablePanel.test.tsx

+    let component = shallow(<TablePanel {...props} />);
+    React.useLayoutEffect = React.useEffect;
+    console.log('component text: '+component.render().text());
+    let renderedHtmlString = component.render().text();


I'm not familiar whether we can test it in another way, rather than converting to text. Maybe @yuntanghsu can give more constructive feedback.

But if we are just checking the text content, instead of using includes expression checking a series of substring ,can we just do expect(textContent).toEqual(expectedContent); to check the entire string?

I think we should use `getAllByText` or `expect(getByText('10.10.1.4:42162')).toBeInTheDocument()` to check the data in the component. For example, in your test and the picture, we can see there are 3 transaction IDs for a certain combination of source and destination. We want to make sure your component can generate exactly 3 elements, which should be based on the number of HttpVals you have. And we also want to make sure the contents are correct in these 3 elements. I'll suggest you change the vals in each transaction ID and verify the context for all of them. Otherwise the result can be a false positive case as you only use "includes" to check the data.
I also found there are lots of warning when using component.render().text():

at div at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at TableContainer (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/TableContainer/TableContainer.js:40:44) at MRT_TableContainer (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/table/MRT_TableContainer.tsx:13:38) at div at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at Paper (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/Paper/Paper.js:69:44) at MRT_TablePaper (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/table/MRT_TablePaper.tsx:11:34) at useRef (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/table/MRT_TableRoot.tsx:50:28) at MaterialReactTable (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/MaterialReactTable.tsx:14:3) at ThemeProvider (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/private-theming/node/ThemeProvider/ThemeProvider.js:39:5) at ThemeProvider (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/system/ThemeProvider/ThemeProvider.js:50:5) at ThemeProvider (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/styles/ThemeProvider.js:21:14) at div 33 | React.useLayoutEffect = React.useEffect; 34 | // console.log('component text: '+component.render().text()); > 35 | let renderedHtmlString = component.render().text(); | ^ 36 | expect(renderedHtmlString.includes('10.10.1.4:42162')).toEqual(true); 37 | expect(renderedHtmlString.includes('93.184.216.34:80')).toEqual(true); 38 | expect(renderedHtmlString.includes('example.com')).toEqual(true); at printWarning (node_modules/react-dom/cjs/react-dom-server.node.development.js:82:30) at error (node_modules/react-dom/cjs/react-dom-server.node.development.js:58:5) at Object.useLayoutEffect (node_modules/react-dom/cjs/react-dom-server.node.development.js:1617:5) at useLayoutEffect (node_modules/react/cjs/react.development.js:1524:21) at useEventCallback (node_modules/@mui/utils/useEventCallback/useEventCallback.js:20:34) at useRippleHandler (node_modules/@mui/material/node/ButtonBase/ButtonBase.js:160:42) at Object.ButtonBase [as render] (node_modules/@mui/material/node/ButtonBase/ButtonBase.js:186:22) at ReactDOMServerRenderer.render (node_modules/react-dom/cjs/react-dom-server.node.development.js:3872:44) at ReactDOMServerRenderer.read (node_modules/react-dom/cjs/react-dom-server.node.development.js:3690:29) at Object.renderToStaticMarkup (node_modules/react-dom/cjs/react-dom-server.node.development.js:4314:27) at Object.renderToStaticMarkup [as render] (node_modules/enzyme-adapter-react-16/src/ReactSixteenAdapter.js:835:31) at ShallowWrapper.render (node_modules/enzyme/src/ShallowWrapper.js:1108:23) at ShallowWrapper.call [as single] (node_modules/enzyme/src/ShallowWrapper.js:1654:21) at ShallowWrapper.single [as html] (node_modules/enzyme/src/ShallowWrapper.js:1104:17) at ShallowWrapper.html [as render] (node_modules/enzyme/src/ShallowWrapper.js:1120:23) at Object.render (src/components/TablePanel.test.tsx:35:40) console.error Warning: useLayoutEffect does nothing on the server, because its effect cannot be encoded into the server renderer's output format. This will lead to a mismatch between the initial, non-hydrated UI and the intended UI. To avoid this, useLayoutEffect should only be used in components that render exclusively on the client. See https://reactjs.org/link/uselayouteffect-ssr for common fixes. at ButtonBase (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/ButtonBase/ButtonBase.js:95:44) at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at TableSortLabel (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/TableSortLabel/TableSortLabel.js:102:44) at span at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at Badge (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/Badge/Badge.js:169:44) at Tooltip (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/Tooltip/Tooltip.js:226:44) at MRT_TableHeadCellSortLabel (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/head/MRT_TableHeadCellSortLabel.tsx:14:3) at div at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at Box (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/system/createBox.js:31:41) at div at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at Box (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/system/createBox.js:31:41) at th at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at TableCell (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/TableCell/TableCell.js:108:44) at MRT_TableHeadCell (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/head/MRT_TableHeadCell.tsx:20:37) at tr at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at TableRow (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/TableRow/TableRow.js:70:44) at MRT_TableHeadRow (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/head/MRT_TableHeadRow.tsx:20:3) at thead at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at TableHead (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/TableHead/TableHead.js:44:44) at MRT_TableHead (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/head/MRT_TableHead.tsx:14:3) at table at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at Table (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/Table/Table.js:61:44) at MRT_Table (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/table/MRT_Table.tsx:19:29) at div at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at TableContainer (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/TableContainer/TableContainer.js:40:44) at MRT_TableContainer (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/table/MRT_TableContainer.tsx:13:38) at div at /Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@emotion/react/dist/emotion-element-b63ca7c6.cjs.dev.js:43:23 at Paper (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/Paper/Paper.js:69:44) at MRT_TablePaper (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/table/MRT_TablePaper.tsx:11:34) at useRef (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/table/MRT_TableRoot.tsx:50:28) at MaterialReactTable (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/material-react-table/src/MaterialReactTable.tsx:14:3) at ThemeProvider (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/private-theming/node/ThemeProvider/ThemeProvider.js:39:5) at ThemeProvider (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/system/ThemeProvider/ThemeProvider.js:50:5) at ThemeProvider (/Users/hsuy/Desktop/theia/plugins/grafana-custom-plugins/grafana-http-table-plugin/node_modules/@mui/material/node/styles/ThemeProvider.js:21:14) at div

@Dhruv-J Do you have any idea what happened?

Yeah, the material-ui library recommends running unit testing without the use of material-ui components in order to keep code immune to breaking due to changes within material-ui internal structure. So I can't directly check for including the table itself, so I check for the string being rendered within the html itself. @yuntanghsu I can attempt using those again, but I remember when I tried using similar solutions that false would be returned even though the text was supposedly being rendered. The warnings happen because the render() function is being called in a unit test, and so the result isn't actually visible anywhere. I looked into this issue specifically with material-ui and found that it is quite common, so people largely suppress the warning or haven't found a workaround. I'll try the clause that you mentioned and update the results within this thread.

heanlan · 2023-12-04T21:19:12Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/tsconfig.json

@@ -0,0 +1,3 @@
+{
+  "extends": "./.config/tsconfig.json"
+}


missing empty line

heanlan · 2023-12-04T21:21:16Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/plugin.json

+      "name": "Antrea io"
+    },
+    "logos": {
+      "small": "img/logo.svg",


please follow https://github.com/antrea-io/theia/blob/main/plugins/grafana-custom-plugins/grafana-chord-plugin/src/plugin.json#L14

not solved yet

heanlan · 2023-12-04T21:25:51Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/types.ts

@@ -0,0 +1,7 @@
+type SeriesSize = 'sm' | 'md' | 'lg';


Are these options being used anywhere? I see they are passed as parameter into TablePanel, but I don't see where users can configure them, and where do we explicitly handle their values in the source code? The name "SimpleOptions" also looks a bit too general for me to understand its usage.

those are default options, I'll remove them

Dhruv-J · 2023-12-11T17:57:02Z

Instructions for those wanting to deploy the solution for themselves @heanlan:

pull Antrea PR #5218
pull Theia PR Adding new Clickhouse fields for NP visibility #417 and this PR
execute the included script for build_antrea.sh.zip
deploy flow-visibility.yml from the new theia PR
apply the following: pods.yaml.zip
annotate the pods with the following command: kubectl pod annotate <pod-name> -n <namespace> [visibility.antrea.io/enable-l7=both](http://visibility.antrea.io/enable-l7=both)
kubectl exec into the applied admin pod and run the command curl example.com to get one row, or curl example.com example.com example.com to get multiple rows (it'll take about 2 minutes to populate clickhouse)
open the application layer dashboard to view http values

Signed-off-by: Dhruv-J <[email protected]>

heanlan

LGTM overall

Same with #488 , let's hold this PR up and take another look until all the PRs it depends on are merged.

heanlan · 2023-12-15T16:38:52Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/README.md


-1. Install dependencies
+- Clickhouse


heanlan · 2023-12-15T16:42:39Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/plugin.json

+      "name": "Antrea io"
+    },
+    "logos": {
+      "small": "img/logo.svg",


not solved yet

heanlan · 2023-12-15T16:42:41Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/plugin.json

+    },
+    "logos": {
+      "small": "img/logo.svg",
+      "large": "img/logo.svg"


heanlan · 2023-12-15T16:44:38Z

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/types.ts

@@ -0,0 +1,2 @@
+export interface SimpleOptions {


rename to TableOptions?

Signed-off-by: Dhruv-J <[email protected]>

github-actions · 2024-03-18T00:05:46Z

This PR is stale because it has been open 90 days with no activity. Remove stale label or comment, or this will be closed in 90 days

Dhruv-J force-pushed the new-dashboard branch from 01c2052 to c01e183 Compare October 16, 2023 18:44

tushartathgur reviewed Oct 17, 2023

View reviewed changes

tushartathgur requested a review from heanlan October 17, 2023 18:49

yuntanghsu reviewed Oct 19, 2023

View reviewed changes

Dhruv-J force-pushed the new-dashboard branch from 710c2eb to ff92ef9 Compare October 31, 2023 16:47

github-advanced-security bot found potential problems Oct 31, 2023

View reviewed changes

plugins/grafana-custom-plugins/grafana-http-table-plugin/src/components/TablePanel.test.tsx Fixed Show fixed Hide fixed

Application Layer Dashboard

abe5d67

This PR adds an Application Layer Dashboard to include visualizations for layer seven Network Policies and HTTP values within the flows table. Signed-off-by: Dhruv-J <[email protected]>

Dhruv-J force-pushed the new-dashboard branch from 103c0e8 to abe5d67 Compare November 6, 2023 19:39

linter fix dead links and addressed comments

6c85299

Signed-off-by: Dhruv-J <[email protected]>

Dhruv-J requested a review from yuntanghsu November 20, 2023 17:41

yuntanghsu reviewed Nov 20, 2023

View reviewed changes

fixes and feedback

ad4a97e

Signed-off-by: Dhruv-J <[email protected]>

yuntanghsu reviewed Nov 22, 2023

View reviewed changes

remove e2e default tests

c253971

Signed-off-by: Dhruv-J <[email protected]>

heanlan reviewed Dec 4, 2023

View reviewed changes

Dhruv-J added 2 commits December 11, 2023 10:02

feedback addressed, unit tests and documentation fix

e7d941f

Signed-off-by: Dhruv-J <[email protected]>

test and panel file updated

2090b55

Signed-off-by: Dhruv-J <[email protected]>

heanlan reviewed Dec 15, 2023

View reviewed changes

final feedback addressed

67ae723

Signed-off-by: Dhruv-J <[email protected]>

github-actions bot added the lifecycle/stale label Mar 18, 2024

github-actions bot closed this Jun 16, 2024

		@@ -0,0 +1,10 @@
		import { e2e } from '@grafana/e2e';

		e2e.scenario({

		@@ -0,0 +1,114 @@
		<!-- This README file is going to be the one displayed on the Grafana.com website for your plugin. Uncomment and replace the content here before publishing.

		Remove any remaining comments before publishing as these may be displayed on Grafana.com -->

	grafana-cli --pluginUrl https://github.com/Dhruv-J/grafana-http-table-plugin/archive/refs/tags/v2.zip plugins install theia-grafana-dependency-plugin
	grafana-cli --pluginUrl https://github.com/Dhruv-J/grafana-http-table-plugin/archive/refs/tags/v2.zip plugins install theia-grafana-http-table-plugin

Application Layer Dashboard #523

Application Layer Dashboard #523

Conversation

Dhruv-J commented Oct 4, 2023 • edited Loading

Dhruv-J commented Oct 4, 2023

codecov bot commented Oct 4, 2023 • edited Loading

Codecov Report

Dhruv-J commented Oct 16, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Dhruv-J Nov 27, 2023 • edited Loading

Choose a reason for hiding this comment

heanlan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Dhruv-J commented Dec 11, 2023 • edited Loading

heanlan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Mar 18, 2024

Dhruv-J commented Oct 4, 2023 •

edited

Loading

codecov bot commented Oct 4, 2023 •

edited

Loading

Dhruv-J Nov 27, 2023 •

edited

Loading

Dhruv-J commented Dec 11, 2023 •

edited

Loading