Skip to content

Latest commit

 

History

History
729 lines (678 loc) · 23.1 KB

0.17.0.md

File metadata and controls

729 lines (678 loc) · 23.1 KB

v0.17.0

修复 Dubbo 协议解析 attachment 异常的问题

config

{
  "servers": [
    {
      "default_log_path": "stdout",
      "default_log_level": "INFO",
      "listeners": [
        {
          "name": "egress_dubbo",
          "address": "0.0.0.0:20881",
          "bind_port": true,
          "log_path": "stdout",
          "filter_chains": [
            {
              "tls_context": {},
              "filters": [
                {
                  "type": "proxy",
                  "config": {
                    "downstream_protocol": "X",
                    "upstream_protocol": "X",
                    "router_config_name": "20882",
                    "extend_config": {
                      "sub_protocol": "dubbo"
                    }
                  }
                }
              ]
            }
          ],
          "stream_filters": [
            {
              "type": "dubbo_stream",
              "config": {}
            }
          ]
        }
      ],
      "routers": [
        {
          "router_config_name": "20882",
          "virtual_hosts": [
            {
              "name": "benchmark",
              "domains": ["*"],
              "routers": [
                {
                  "match": {
                    "prefix": "/"
                  },
                  "route": {
                    "cluster_name": "outbound|20882||mosn.io.dubbo.DemoService",
                    "timeout": "3s",
                    "retry_policy": {
                      "retry_on": true,
                      "retry_timeout": "3s",
                      "num_retries": 2
                    }
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "cluster_manager": {
    "clusters": [
      {
        "name": "outbound|20882||mosn.io.dubbo.DemoService",
        "type": "SIMPLE",
        "lb_type": "LB_RANDOM",
        "max_request_per_conn": 1024,
        "conn_buffer_limit_bytes": 32768,
        "hosts": [
          {
            "address": "127.0.0.1:20880"
          }
        ]
      }
    ]
  },
  "admin": {
    "address": {
      "socket_address": {
        "address": "0.0.0.0",
        "port_value": 34901
      }
    }
  }
}

result

MOSN

2020-09-29 19:20:50,260 [INFO] [admin store] [start service] start service Mosn Admin Server on [::]:34901
2020-09-29 19:20:51,261 [INFO] [server] [reconfigure] reconfigureHandler start
2020-09-29 19:21:08,608 [ERROR] [2,-] [stream] [xprotocol] conn 2, 10.12.215.34:51262 decode error: [protocol][dubbo] Decode Error, type = unknown cmd type , err = decode arguments error :illegal class index @idx 1
goroutine 68 [running]:
runtime/debug.Stack(0xc0005bd3d0, 0x509c2e0, 0x551a620)
	/usr/local/go/src/runtime/debug/stack.go:24 +0x9f
mosn.io/mosn/pkg/protocol/xprotocol/dubbo.getServiceAwareMeta.func1(0xc0005bd738)

consumer

[29/09/20 19:20:52:670 CST] main  INFO logger.LoggerFactory: using logger: org.apache.dubbo.common.logger.log4j.Log4jLoggerAdapter
[29/09/20 19:20:58:036 CST] main  WARN config.AbstractConfig:  [DUBBO] There's no valid metadata config found, if you are using the simplified mode of registry url, please make sure you have a metadata address configured properly., dubbo version: 2.7.3, current host: 10.12.215.34
[29/09/20 19:21:08:457 CST] main  INFO transport.AbstractClient:  [DUBBO] Succeed connect to server /127.0.0.1:1080 from NettyClient 10.12.215.34 using dubbo version 2.7.3, channel is NettyChannel [channel=[id: 0x637b5cff, L:/127.0.0.1:51261 - R:/127.0.0.1:1080]], dubbo version: 2.7.3, current host: 10.12.215.34
[29/09/20 19:21:08:457 CST] main  INFO transport.AbstractClient:  [DUBBO] Start NettyClient champly-deiMac.local/10.12.215.34 connect to the server /127.0.0.1:1080, dubbo version: 2.7.3, current host: 10.12.215.34
[29/09/20 19:21:08:511 CST] main  INFO config.AbstractConfig:  [DUBBO] Refer dubbo service mosn.io.dubbo.DemoService from url dubbo://127.0.0.1:20881/mosn.io.dubbo.DemoService?application=dubbo-examples-consumer&generic=false&interface=mosn.io.dubbo.DemoService&lazy=false&pid=95143&qos.enable=false&register.ip=10.12.215.34&remote.application=&side=consumer&sticky=false&timeout=2000, dubbo version: 2.7.3, current host: 10.12.215.34
^C[29/09/20 19:21:10:419 CST] DubboShutdownHook  INFO config.DubboShutdownHook:  [DUBBO] Run shutdown hook now., dubbo version: 2.7.3, current host: 10.12.215.34
[29/09/20 19:21:10:421 CST] DubboShutdownHook  INFO support.AbstractRegistryFactory:  [DUBBO] Close all registries [], dubbo version: 2.7.3, current host: 10.12.215.34
[29/09/20 19:21:10:421 CST] DubboShutdownHook  INFO dubbo.DubboProtocol:  [DUBBO] Close dubbo connect: 10.12.215.34:0-->/127.0.0.1:20881, dubbo version: 2.7.3, current host: 10.12.215.34

添加在 Dubbo 协议下通过 xDS HTTP 配置进行转换的过滤器

build

build with Makefile

$ make proxyv2
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o ./build/mosn -ldflags "-X mosn.io/mosn/pkg/types.IstioVersion=1.6.10" ../../../../../cmd/mosn/main
docker build -t symcn.tencentcloudcr.com/symcn/proxyv2:1.6.10 -f ./build/Dockerfile .
Sending build context to Docker daemon  48.71MB
Step 1/3 : FROM istio/proxyv2:1.7.2
 ---> d8fb7dcfd395
Step 2/3 : COPY build/mosn /usr/local/bin/mosn
 ---> 4f3a770f3f08
Step 3/3 : COPY build/mosn_config_dubbo_xds.json /etc/istio/mosn/mosn_config_dubbo_xds.json
 ---> 58b750d452da
Successfully built 58b750d452da
Successfully tagged symcn.tencentcloudcr.com/symcn/proxyv2:1.6.10

env

install with README.md

result

MOSN

request log

2020-09-29 19:35:06,142 [INFO] [server] [reconfigure] reconfigureHandler start
2020-09-29 19:35:19,72 [INFO] remote addr: 127.0.0.1:20880, network: tcp
[2020/09/29 19:35:19.072] 1.254515ms 62.718472ms 71 167 dubbo 0 62.799002ms false 0 127.0.0.1:20880 10.12.215.34:20881 10.12.215.34:51681  0x0 outbound|20882||mosn.io.dubbo.DemoService
[2020/09/29 19:35:24.148] 55.339µs 54.927034ms 71 167 dubbo 0 54.994238ms false 0 127.0.0.1:20880 10.12.215.34:20881 10.12.215.34:51681  0x0 outbound|20882||mosn.io.dubbo.DemoService
[2020/09/29 19:35:29.208] 51.966µs 53.777186ms 71 167 dubbo 0 53.840647ms false 0 127.0.0.1:20880 10.12.215.34:20881 10.12.215.34:51681  0x0 outbound|20882||mosn.io.dubbo.DemoService

支持 Proxy 层级的并发

config

{
  "servers": [
    {
      "default_log_path": "stdout",
      "default_log_level": "INFO",
      "listeners": [
        {
          "name": "egress_dubbo",
          "address": "0.0.0.0:20881",
          "bind_port": true,
          "log_path": "stdout",
          "filter_chains": [
            {
              "tls_context": {},
              "filters": [
                {
                  "type": "proxy",
                  "config": {
                    "downstream_protocol": "X",
                    "upstream_protocol": "X",
                    "router_config_name": "20882",
                    "extend_config": {
                      "sub_protocol": "dubbo"
                    },
                    "concurrency_num": 5
                  }
                }
              ]
            }
          ],
          "stream_filters": [
            {
              "type": "dubbo_stream",
              "config": {}
            }
          ]
        }
      ],
      "routers": [
        {
          "router_config_name": "20882",
          "virtual_hosts": [
            {
              "name": "benchmark",
              "domains": ["*"],
              "routers": [
                {
                  "match": {
                    "prefix": "/"
                  },
                  "route": {
                    "cluster_name": "outbound|20882||mosn.io.dubbo.DemoService",
                    "timeout": "3s",
                    "retry_policy": {
                      "retry_on": true,
                      "retry_timeout": "3s",
                      "num_retries": 2
                    }
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "cluster_manager": {
    "clusters": [
      {
        "name": "outbound|20882||mosn.io.dubbo.DemoService",
        "type": "SIMPLE",
        "lb_type": "LB_RANDOM",
        "max_request_per_conn": 1024,
        "conn_buffer_limit_bytes": 32768,
        "hosts": [
          {
            "address": "127.0.0.1:20880"
          }
        ]
      }
    ]
  },
  "admin": {
    "address": {
      "socket_address": {
        "address": "0.0.0.0",
        "port_value": 34901
      }
    }
  }
}

result

add log print

2020-09-29 19:45:31,73 [INFO] mosn start server
2020-09-29 19:45:31,73 [INFO] [admin store] [start service] start service Mosn Admin Server on [::]:34901
2020-09-29 19:45:32,73 [INFO] [server] [reconfigure] reconfigureHandler start
proxy workerpool:5
2020-09-29 19:45:50,350 [INFO] remote addr: 127.0.0.1:20880, network: tcp

新增最大 Header 大小限制的配置选项

config

{
  "servers": [
    {
      "default_log_path": "stdout",
      "default_log_level": "TRACE",
      "routers": [
        {
          "router_config_name": "server_router",
          "virtual_hosts": [
            {
              "name": "serverHost",
              "domains": ["*"],
              "routers": [
                {
                  "match": { "prefix": "/" },
                  "route": { "cluster_name": "serverCluster" }
                }
              ]
            }
          ] },
        {
          "router_config_name": "client_router",
          "virtual_hosts": [
            {
              "name": "clientHost",
              "domains": ["*"],
              "routers": [
                {
                  "match": { "prefix": "/" },
                  "route": { "cluster_name": "clientCluster" }
                }
              ]
            }
          ]
        }
      ],
      "listeners": [
        {
          "name": "serverListener",
          "address": "127.0.0.1:2046",
          "bind_port": true,
          "filter_chains": [
            {
              "filters": [
                {
                  "type": "proxy",
                  "config": {
                    "downstream_protocol": "Http1",
                    "upstream_protocol": "Http1",
                    "router_config_name": "server_router",
                    "extend_config": {
                      "max_header_size": 100
                    }
                  }
                }
              ]
            }
          ]
        },
        {
          "name": "clientListener",
          "address": "127.0.0.1:2045",
          "bind_port": true,
          "filter_chains": [
            {
              "filters": [
                {
                  "type": "proxy",
                  "config": {
                    "downstream_protocol": "Http1",
                    "upstream_protocol": "Http1",
                    "router_config_name": "client_router"
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "cluster_manager": {
    "clusters": [
      {
        "name": "serverCluster",
        "type": "SIMPLE",
        "lb_type": "LB_RANDOM",
        "max_request_per_conn": 1024,
        "conn_buffer_limit_bytes": 32768,
        "hosts": [{ "address": "127.0.0.1:8080" }]
      },
      {
        "name": "clientCluster",
        "type": "SIMPLE",
        "lb_type": "LB_RANDOM",
        "max_request_per_conn": 1024,
        "conn_buffer_limit_bytes": 32768,
        "hosts": [{ "address": "127.0.0.1:2046" }]
      }
    ]
  },
  "admin": {
    "address": {
      "socket_address": {
        "address": "0.0.0.0",
        "port_value": 34901
      }
    }
  }
}

result

failed log

2020-09-30 13:41:44,730 [TRACE] [proxy] extend config = map[max_header_size:100]
2020-09-30 13:41:44,730 [TRACE] [proxy] extend config proxyGeneralExtendConfig = {false 0 100}
2020-09-30 13:41:44,730 [DEBUG] [server] [listener] accept connection from 127.0.0.1:2046, condId= 11, remote addr:127.0.0.1:56757
2020-09-30 13:41:44,730 [DEBUG] [network] [check use writeloop] Connection = 11, Local Address = 127.0.0.1:2046, Remote Address = 127.0.0.1:56757
2020-09-30 13:41:44,731 [DEBUG] update listener write bytes: 28
2020-09-30 13:41:44,731 [DEBUG] [network] [write loop] Error on write. Connection = 11, Remote Address = 127.0.0.1:56757, err = EOF, conn = 0xc0002e4a00
2020-09-30 13:41:44,731 [DEBUG] [network] [close connection] Close TCP Conn, Remote Address is = 127.0.0.1:56757, eventType is = LocalClose
2020-09-30 13:41:44,731 [DEBUG] [network] [close connection] Close connection 11, event LocalClose, type NoFlush
2020-09-30 13:41:44,731 [ERROR] [stream] [http] connection has closed. Connection = 11, Local Address = 127.0.0.1:2046, Remote Address = 127.0.0.1:56757, err = send on closed channel

client log

$ curl -H "key1:value1" -H "key2:v" -I http://localhost:2046
HTTP/1.1 200 OK
Date: Wed, 30 Sep 2020 05:41:36 GMT

$ curl -H "key1:value1" -H "key2:v2" -I http://localhost:2046
HTTP/1.1 400 Bad Request

优化 XProtocol 多路复用模式，支持单机 Host 连接数可配置，默认是 1

config

{
  "servers": [
    {
      "default_log_path": "stdout",
      "default_log_level": "INFO",
      "listeners": [
        {
          "name": "egress_dubbo",
          "address": "0.0.0.0:20881",
          "bind_port": true,
          "log_path": "stdout",
          "filter_chains": [
            {
              "tls_context": {},
              "filters": [
                {
                  "type": "proxy",
                  "config": {
                    "downstream_protocol": "X",
                    "upstream_protocol": "X",
                    "router_config_name": "20882",
                    "extend_config": {
                      "sub_protocol": "dubbo"
                    }
                  }
                }
              ]
            }
          ],
          "stream_filters": [
            {
              "type": "dubbo_stream",
              "config": {}
            }
          ]
        }
      ],
      "routers": [
        {
          "router_config_name": "20882",
          "virtual_hosts": [
            {
              "name": "benchmark",
              "domains": ["*"],
              "routers": [
                {
                  "match": {
                    "prefix": "/"
                  },
                  "route": {
                    "cluster_name": "outbound|20882||mosn.io.dubbo.DemoService",
                    "timeout": "3s",
                    "retry_policy": {
                      "retry_on": true,
                      "retry_timeout": "3s",
                      "num_retries": 2
                    }
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "cluster_manager": {
    "clusters": [
      {
        "name": "outbound|20882||mosn.io.dubbo.DemoService",
        "type": "SIMPLE",
        "lb_type": "LB_RANDOM",
        "max_request_per_conn": 1024,
        "conn_buffer_limit_bytes": 32768,
        "circuit_breakers": [
          {
            "max_connections": 5
          }
        ],
        "hosts": [
          {
            "address": "127.0.0.1:20880"
          }
        ]
      }
    ]
  },
  "admin": {
    "address": {
      "socket_address": {
        "address": "0.0.0.0",
        "port_value": 34901
      }
    }
  }
}

result

mosn

2020-09-30 09:53:21,33 [INFO] [admin store] [start service] start service Mosn Admin Server on [::]:34901
2020-09-30 09:53:22,34 [INFO] [server] [reconfigure] reconfigureHandler start
2020-09-30 09:53:51,167 [INFO] remote addr: 127.0.0.1:20880, network: tcp
2020-09-30 09:53:56,350 [INFO] remote addr: 127.0.0.1:20880, network: tcp
2020-09-30 09:54:01,412 [INFO] remote addr: 127.0.0.1:20880, network: tcp
2020-09-30 09:54:06,333 [INFO] [network] [read loop] do read err: read tcp 127.0.0.1:50009->127.0.0.1:20880: i/o timeout
2020-09-30 09:54:06,501 [INFO] remote addr: 127.0.0.1:20880, network: tcp
2020-09-30 09:54:11,410 [INFO] [network] [read loop] do read err: read tcp 127.0.0.1:50013->127.0.0.1:20880: i/o timeout
2020-09-30 09:54:11,561 [INFO] remote addr: 127.0.0.1:20880, network: tcp
2020-09-30 09:54:16,495 [INFO] [network] [read loop] do read err: read tcp 127.0.0.1:50014->127.0.0.1:20880: i/o timeout

tcp conn

$ lsof -i tcp:20880
COMMAND  PID       USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
java    5614   champly    47u  IPv6 0x76fbddceb6414031      0t0  TCP *:20880 (LISTEN)
java    5614   champly    48u  IPv6 0x76fbddced72ccf31      0t0  TCP localhost:20880->localhost:50009 (ESTABLISHED)
java    5614   champly    49u  IPv6 0x76fbddced72cc2f1      0t0  TCP localhost:20880->localhost:50013 (ESTABLISHED)
java    5614   champly    50u  IPv6 0x76fbddced72d0651      0t0  TCP localhost:20880->localhost:50014 (ESTABLISHED)
java    5614   champly    51u  IPv6 0x76fbddced72cbcd1      0t0  TCP localhost:20880->localhost:50015 (ESTABLISHED)
java    5614   champly    52u  IPv6 0x76fbddced71fc7b1      0t0  TCP localhost:20880->localhost:50016 (ESTABLISHED)
main    5661   champly    14u  IPv4 0x76fbddced53810a1      0t0  TCP localhost:50009->localhost:20880 (ESTABLISHED)
main    5661   champly    15u  IPv4 0x76fbddced76f55c1      0t0  TCP localhost:50013->localhost:20880 (ESTABLISHED)
main    5661   champly    16u  IPv4 0x76fbddced7259be1      0t0  TCP localhost:50014->localhost:20880 (ESTABLISHED)
main    5661   champly    17u  IPv4 0x76fbddced7257461      0t0  TCP localhost:50015->localhost:20880 (ESTABLISHED)
main    5661   champly    18u  IPv4 0x76fbddced72800a1      0t0  TCP localhost:50016->localhost:20880 (ESTABLISHED)

Listener 配置新增对 UDS 的支持

client config

{
  "close_graceful": true,
  "servers": [
    {
      "default_log_path": "stdout",
      "default_log_level": "DEBUG",
      "listeners": [
        {
          "name": "clientListener",
          "address": "/tmp/client-proxy.sock",
          "bind_port": true,
          "log_path": "stdout",
          "network": "unix",
          "filter_chains": [
            {
              "tls_context": {},
              "filters": [
                {
                  "type": "tcp_proxy",
                  "config": {
                    "cluster": "clientCluster",
                    "routes": [
                      {
                        "cluster": "clientCluster"
                      }
                    ]
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "cluster_manager": {
    "clusters": [
      {
        "Name": "clientCluster",
        "type": "SIMPLE",
        "lb_type": "LB_RANDOM",
        "max_request_per_conn": 1024,
        "conn_buffer_limit_bytes": 32768,
        "hosts": [
          {
            "address": "127.0.0.1:2046"
          }
        ]
      }
    ]
  },
  "tracing": {
    "enable": true,
    "driver": "SOFATracer"
  }
}

server config

{
  "close_graceful": true,
  "servers": [
    {
      "default_log_path": "stdout",
      "default_log_level": "DEBUG",
      "listeners": [
        {
          "name": "serverListener",
          "address": "127.0.0.1:2046",
          "bind_port": true,
          "log_path": "stdout",
          "filter_chains": [
            {
              "tls_context": {},
              "filters": [
                {
                  "type": "tcp_proxy",
                  "config": {
                    "cluster": "serverCluster",
                    "routes": [
                      {
                        "cluster": "serverCluster"
                      }
                    ]
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "cluster_manager": {
    "clusters": [
      {
        "Name": "serverCluster",
        "type": "SIMPLE",
        "lb_type": "LB_RANDOM",
        "max_request_per_conn": 1024,
        "conn_buffer_limit_bytes": 32768,
        "hosts": [
          {
            "address": "/tmp/server-proxy.sock"
          }
        ]
      }
    ]
  },
  "tracing": {
    "enable": true,
    "driver": "SOFATracer"
  }
}

MOSN client

2020-09-30 13:09:09,579 [DEBUG] [tcp proxy] [new conn] accept new connection
2020-09-30 13:09:09,579 [INFO] remote addr: 127.0.0.1:2046, network: tcp
2020-09-30 13:09:09,579 [DEBUG] [network] [check use writeloop] Connection = 3, Local Address = 127.0.0.1:54991, Remote Address = 127.0.0.1:2046
2020-09-30 13:09:09,579 [DEBUG] [network] [client connection connect] connect raw tcp, remote address = 127.0.0.1:2046 ,event = ConnectedFlag, error = <nil>
2020-09-30 13:09:09,579 [DEBUG] new upstream connection 3 created
2020-09-30 13:09:09,579 [DEBUG] [server] [listener] accept connection from /tmp/client-proxy.sock, condId= 2, remote addr:
2020-09-30 13:09:09,579 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:09,580 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:10,582 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:10,583 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:11,585 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:11,585 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:12,590 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:12,591 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:13,593 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:13,593 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:14,595 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:14,595 [DEBUG] update listener write bytes: 17

MOSN server

2020-09-30 13:09:09,579 [DEBUG] [network] [client connection connect] connect raw unix, remote address = /tmp/server-proxy.sock ,event = ConnectedFlag, error = <nil>
2020-09-30 13:09:09,579 [DEBUG] new upstream connection 3 created
2020-09-30 13:09:09,579 [DEBUG] [server] [listener] accept connection from 127.0.0.1:2046, condId= 2, remote addr:127.0.0.1:54991
2020-09-30 13:09:09,580 [DEBUG] [network] [check use writeloop] Connection = 2, Local Address = 127.0.0.1:2046, Remote Address = 127.0.0.1:54991
2020-09-30 13:09:09,580 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:09,580 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:10,582 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:10,583 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:11,585 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:11,585 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:12,591 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:12,591 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:13,593 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:13,593 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:14,595 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:14,595 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:15,595 [DEBUG] [tcp proxy] [ondata] read data , len = 11
2020-09-30 13:09:15,596 [DEBUG] update listener write bytes: 17
2020-09-30 13:09:16,597 [DEBUG] [tcp proxy] [ondata] read data , len = 11