Skip to content

Commit

Permalink
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add more document
Browse files Browse the repository at this point in the history
123liuziming committed Aug 10, 2024
1 parent 7dd97aa commit f0e3df9
Showing 11 changed files with 245 additions and 29 deletions.
43 changes: 43 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -55,6 +55,46 @@ at [GitHub Issues](https://github.com/alibaba/opentelemetry-go-auto-instrumentat
to help us enhance this project.

Lastly, we've provided some examples in `example` direcory, you can try the project through those examples.

### Compatibility

OpenTelemetry-Go Contrib ensures compatibility with the current supported
versions of
the [Go language](https://golang.org/doc/devel/release#policy):

> Each major Go release is supported until there are two newer major releases.
> For example, Go 1.5 was supported until the Go 1.7 release, and Go 1.6 was supported until the Go 1.8 release.
For versions of Go that are no longer supported upstream, opentelemetry-go-contrib will
stop ensuring compatibility with these versions in the following manner:

- A minor release of opentelemetry-go-contrib will be made to add support for the new
supported release of Go.
- The following minor release of opentelemetry-go-contrib will remove compatibility
testing for the oldest (now archived upstream) version of Go. This, and
future, releases of opentelemetry-go-contrib may include features only supported by
the currently supported versions of Go.

This project is tested on the following systems.

| OS | Go Version | Architecture |
| -------- | ---------- | ------------ |
| Ubuntu | 1.22 | amd64 |
| Ubuntu | 1.21 | amd64 |
| Ubuntu | 1.22 | 386 |
| Ubuntu | 1.21 | 386 |
| macOS 13 | 1.22 | amd64 |
| macOS 13 | 1.21 | amd64 |
| macOS | 1.22 | arm64 |
| macOS | 1.21 | arm64 |
| Windows | 1.22 | amd64 |
| Windows | 1.21 | amd64 |
| Windows | 1.22 | 386 |
| Windows | 1.21 | 386 |

While this project should work for other systems, no compatibility guarantees
are made for those systems currently.

# Community

We are looking forward to your feedback and suggestions. Please feel free to join our [DingTalk group](https://qr.dingtalk.com/action/joingroup?code=v1,k1,GyDX5fUTYnJ0En8MrVbHBYTGUcPXJ/NdsmLODGibd0w=&_dt_no_comment=1&origin=11? ) to communicate with us.
@@ -65,5 +105,8 @@ Also there are several documents that you may find useful:

- [How it works](./docs/how-it-works.md)
- [How to add a new rule](./docs/how-to-add-a-new-rule.md)
- [How to debug](./docs/how-to-debug.md)
- [How to write tests for plugins](./docs/how-to-write-tests-for-plugins.md)
- [Supported Libraries](./docs/supported-libraries.md)
- [Discussion on this topic at OpenTelemetry community](https://github.com/open-telemetry/community/issues/1961)
- [面向OpenTelemetry的Golang应用无侵入插桩技术](https://mp.weixin.qq.com/s/FKCwzRB5Ujhe1stOH2ibXg)
26 changes: 26 additions & 0 deletions docs/how-to-debug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
## How to debug

`opentelemetry-go-auto-instrumentation` provides some convenient ways for users to debug the instrumented program.

## 1. Do the instrumentation with -debug options

```bash
# go build
$ ./otel-go-auto-instrumentation -debug
```

After alerting the `-debug` option, `otel-go-auto-instrumentation` will build an unoptimized binary file with more
debugging information.

## 2. Check `.otel-build` directory

After doing the instrumentation, users can check the debugging files in `instrument` directory and `preprocess`
directory. The debugging files
in `preprocess` directory show all the rule files that are matched, and the debugging files in `instrument` directory
show the instrumented plugin code.
![otebuild.png](otebuild.png)

## 3. Use delve to debug binary

No optimization will be taken with the `-debug` option during the hybrid compilation. Users can
use [delve](https://github.com/go-delve/delve) to debug the binary file easily.
136 changes: 136 additions & 0 deletions docs/how-to-write-tests-for-plugins.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
## How to write tests for plugins

Once you've added a new instrumentation rule according
to [how-to-add-a-new-rule.md](https://github.com/alibaba/opentelemetry-go-auto-instrumentation/blob/main/docs/how-to-add-a-new-rule.md),
you need to add tests to verify your rules. `opentelemetry-go-auto-instrumentation` provides a convenient way to verify
your rules.

## Add a general plugin test case

Change directory to `/test`, and create a new directory for the plugin you want to test, for example `redis`. In
the `redis` directory, there are some subdirectories and the name of each subdirectory represents the lowest supported
version of the plugin. If you want to add tests for redis, you should do the following things:

### 1. Add lowest-version dependency for your rule

For example, if you add a rule that support redis from `v9.0.5` to the latest version, you should verify the lowest
redis
version, which is `v9.0.5` first. You may create a subdirectory named `v9.0.5` and add the following `go.mod`:

```
module redis/v9.0.5
go 1.21
replace github.com/alibaba/opentelemetry-go-auto-instrumentation => ../../../../opentelemetry-go-auto-instrumentation
require (
// import this dependency to use verifier
github.com/alibaba/opentelemetry-go-auto-instrumentation v0.0.0-00010101000000-000000000000
github.com/redis/go-redis/v9 v9.0.5
go.opentelemetry.io/otel v1.28.0
go.opentelemetry.io/otel/sdk v1.28.0
)
```

### 2. Write business logic based on the plugin

Then you need to write some business logic based on the `redis` plugin, for example, `test_executing_commands.go` does
the basic get and set operation in redis. Your tests should cover all the usage scenarios of this plugin as much as
possible.

### 3. Write verification code

Some telemetry data(like span) will be produced if the business code you've written is matched to the rule. For example,
there should be two spans in `test_executing_commands.go`, one represents for the `set` redis operation and the other
represents for the `get` redis operation. You should use the `verifier` to verify the correctness:

```go
"github.com/alibaba/opentelemetry-go-auto-instrumentation/pkg/verifier"

verifier.WaitAndAssertTraces(func (stubs []tracetest.SpanStubs) {
verifier.VerifyDbAttributes(stubs[0][0], "set", "", "redis", "", "localhost", "set a b ex 5: ", "set")
verifier.VerifyDbAttributes(stubs[1][0], "get", "", "redis", "", "localhost", "get a: ", "get")
})
```

The `WaitAndAssertTraces` of the verifier accept a callback function, which provides all the traces that are produced.
You should verify the attribute, the parent context and all other key information of every span in all the traces.

### 4. Register your tests

Finally, you should register the test. You should write a `_tests.go` file in `test` directory to do the registration:

```go
const redis_dependency_name = "github.com/redis/go-redis/v9"
const redis_module_name = "redis"

func init() {
TestCases = append(TestCases, NewGeneralTestCase("redis-9.0.5-executing-commands-test", redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestExecutingCommands)
}

func TestExecutingCommands(t *testing.T, env ...string) {
redisC, redisPort := initRedisContainer()
defer clearRedisContainer(redisC)
UseApp("redis/v9.0.5")
RunInstrument(t, "-debuglog", "--", "test_executing_commands.go")
env = append(env, "REDIS_PORT="+redisPort.Port())
RunApp(t, "test_executing_commands", env...)
}
```

In `init` function, you need to wrap your test case with `NewGeneralTestCase`, `NewGeneralTestCase` receives the
following arguments:

testName, moduleName, minVersion, maxVersion, minGoVersion, maxGoVersion string, testFunc func(t *testing.T, env
...string)

1. testName: name of the test case.
2. moduleName: the subdirectory name in `test` directory.
3. minVersion: the lowest supported version of the plugin.
4. maxVersion: the highest supported version of the plugin
5. minGoVersion: the lowest supported Go version of the plugin.
6. maxGoVersion: the highest supported Go version of the plugin.
7. testFunc: test function to be executed.

You should build the test case with the `opentelemetry-go-auto-instrumentation` to make your test case able to produce
telemetry data. Firstly you should call `UseApp` method to change directory to the directory of your test cases, and
then call `RunInstrument` to do hybrid compilation. Finally use the `RunApp` to run the instrumented test-case binary to
verify the telemetry data.

```go
func TestExecutingUnsupporetedCommands(t *testing.T, env ...string) {
redisC, redisPort := initRedisContainer()
defer clearRedisContainer(redisC)
UseApp("redis/v9.0.5")
RunInstrument(t, "-debuglog", "--", "test_executing_unsupported_commands.go")
env = append(env, "REDIS_PORT="+redisPort.Port())
RunApp(t, "test_executing_unsupported_commands", env...)
}
```

## Add a muzzle check case

Muzzle check is inspired
by [safety-mechanisms.md](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/safety-mechanisms.md).
It is impossible for us to run general plugin test for every version because it's going to take a lot of time.
So `opentelemetry-go-auto-instrumentation` will pick some random version to do the hybrid compilation in order to verify
the API compatibility between different versions. If the muzzle check finds that some APIs are changed in some version,
community will create a new rule to adapt it.

Users can add a muzzle check case by calling `NewMuzzleTestCase`, the arguments taken by `NewMuzzleTestCase` are almost
the same as `NewGeneralTestCase`. You need to additionally specify the dependency name of the plugin and the list of
classes that need to do the muzzle check.

## Add a latest-depth check case

Instrumentation tests are generally run against the lowest version of a library that we support to ensure a baseline
against users with old dependency versions. Due to the nature of the agent and locations where we instrument private
APIs, the agent may fail on a newly released version of the library. We run instrumentation tests additionally against
the latest version of the library, as fetched from Maven, as part of a nightly build. If a new version of a library will
not work with the agent, we find out through this build and can address it by the next release of the agent.

Users can add a muzzle check case by calling `NewLatestDepthTestCase`, the arguments taken by `NewLatestDepthTestCase`
are almost
the same as `NewGeneralTestCase`. You need to additionally specify the dependency name of the plugin and the list of
classes that need to do the latest-depth check.
Binary file added docs/otebuild.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 9 additions & 0 deletions docs/supported-libraries.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
## Supported libraries

| Plugin Name | Repository Url | Min Supported Version | Max Supported Version |
|--------------|--------------------------------------------|-----------------------|-----------------------|
| net/http | https://pkg.go.dev/net/http | - | - |
| database/sql | https://pkg.go.dev/database/sql | - | - |
| go-redis | https://github.com/redis/go-redis | v9.0.5 | v9.5.1 |
| mongodb | https://github.com/mongodb/mongo-go-driver | v1.11.1 | v1.15.2 | |

26 changes: 13 additions & 13 deletions test/databasesql_tests.go
Original file line number Diff line number Diff line change
@@ -12,19 +12,19 @@ import (

func init() {
TestCases = append(TestCases,
NewGeneralTestCase("databasesql-mysql-8-access-database-test", "", "databasesql", "", "", "1.18", "", TestMySql8xAccessDatabase),
NewGeneralTestCase("databasesql-mysql-8-fetching-database-test", "", "databasesql", "", "", "1.18", "", TestMySql8xFetchingDatabase),
NewGeneralTestCase("databasesql-mysql-8-modify-data-test", "", "databasesql", "", "", "1.18", "", TestMySql8xModifyData),
NewGeneralTestCase("databasesql-mysql-8-prepared-statement-test", "", "databasesql", "", "", "1.18", "", TestPreparedStatement),
NewGeneralTestCase("databasesql-mysql-8-single-row-query-test", "", "databasesql", "", "", "1.18", "", TestSingleRowQuery),
NewGeneralTestCase("databasesql-mysql-8-single-transaction-test", "", "databasesql", "", "", "1.18", "", TestTransaction),

NewGeneralTestCase("databasesql-mysql-5-access-database-test", "", "databasesql", "", "", "1.18", "", TestMySql5xAccessDatabase),
NewGeneralTestCase("databasesql-mysql-5-fetching-database-test", "", "databasesql", "", "", "1.18", "", TestMySql5xFetchingDatabase),
NewGeneralTestCase("databasesql-mysql-5-modify-data-test", "", "databasesql", "", "", "1.18", "", TestMySql5xModifyData),
NewGeneralTestCase("databasesql-mysql-5-prepared-statement-test", "", "databasesql", "", "", "1.18", "", TestMySql5xSingleRowQuery),
NewGeneralTestCase("databasesql-mysql-5-single-row-query-test", "", "databasesql", "", "", "1.18", "", TestMySql5xPreparedStatement),
NewGeneralTestCase("databasesql-mysql-5-single-transaction-test", "", "databasesql", "", "", "1.18", "", TestMySql5xTransaction),
NewGeneralTestCase("databasesql-mysql-8-access-database-test", "databasesql", "", "", "1.18", "", TestMySql8xAccessDatabase),
NewGeneralTestCase("databasesql-mysql-8-fetching-database-test", "databasesql", "", "", "1.18", "", TestMySql8xFetchingDatabase),
NewGeneralTestCase("databasesql-mysql-8-modify-data-test", "databasesql", "", "", "1.18", "", TestMySql8xModifyData),
NewGeneralTestCase("databasesql-mysql-8-prepared-statement-test", "databasesql", "", "", "1.18", "", TestPreparedStatement),
NewGeneralTestCase("databasesql-mysql-8-single-row-query-test", "databasesql", "", "", "1.18", "", TestSingleRowQuery),
NewGeneralTestCase("databasesql-mysql-8-single-transaction-test", "databasesql", "", "", "1.18", "", TestTransaction),

NewGeneralTestCase("databasesql-mysql-5-access-database-test", "databasesql", "", "", "1.18", "", TestMySql5xAccessDatabase),
NewGeneralTestCase("databasesql-mysql-5-fetching-database-test", "databasesql", "", "", "1.18", "", TestMySql5xFetchingDatabase),
NewGeneralTestCase("databasesql-mysql-5-modify-data-test", "databasesql", "", "", "1.18", "", TestMySql5xModifyData),
NewGeneralTestCase("databasesql-mysql-5-prepared-statement-test", "databasesql", "", "", "1.18", "", TestMySql5xSingleRowQuery),
NewGeneralTestCase("databasesql-mysql-5-single-row-query-test", "databasesql", "", "", "1.18", "", TestMySql5xPreparedStatement),
NewGeneralTestCase("databasesql-mysql-5-single-transaction-test", "databasesql", "", "", "1.18", "", TestMySql5xTransaction),
)
}

6 changes: 3 additions & 3 deletions test/mongo_tests.go
Original file line number Diff line number Diff line change
@@ -13,9 +13,9 @@ const mongo_dependency_name = "go.mongodb.org/mongo-driver"
const mongo_module_name = "mongo"

func init() {
TestCases = append(TestCases, NewGeneralTestCase("mongo-1.11.1-crud-test", mongo_dependency_name, mongo_module_name, "v1.11.1", "v1.15.1", "1.18", "", TestCrudMongo),
NewGeneralTestCase("mongo-1.11.1-cursor-test", mongo_dependency_name, mongo_module_name, "v1.11.1", "v1.15.1", "1.18", "", TestCursor),
NewGeneralTestCase("mongo-1.11.1-batch-test", mongo_dependency_name, mongo_module_name, "v1.11.1", "v1.15.1", "1.18", "", TestBatch),
TestCases = append(TestCases, NewGeneralTestCase("mongo-1.11.1-crud-test", mongo_module_name, "v1.11.1", "v1.15.1", "1.18", "", TestCrudMongo),
NewGeneralTestCase("mongo-1.11.1-cursor-test", mongo_module_name, "v1.11.1", "v1.15.1", "1.18", "", TestCursor),
NewGeneralTestCase("mongo-1.11.1-batch-test", mongo_module_name, "v1.11.1", "v1.15.1", "1.18", "", TestBatch),
NewMuzzleTestCase("mongo-1.11.1-crud-muzzle", mongo_dependency_name, mongo_module_name, "v1.11.1", "v1.15.1", "1.18", "", []string{"test_crud_mongo.go", "dsn.go"}),
NewMuzzleTestCase("mongo-1.11.1-cursor-muzzle", mongo_dependency_name, mongo_module_name, "v1.11.1", "v1.15.1", "1.18", "", []string{"test_batch.go", "dsn.go"}),
NewMuzzleTestCase("mongo-1.11.1-batch-muzzle", mongo_dependency_name, mongo_module_name, "v1.11.1", "v1.15.1", "1.18", "", []string{"test_cursor.go", "dsn.go"}),
6 changes: 3 additions & 3 deletions test/net_http_tests.go
Original file line number Diff line number Diff line change
@@ -4,9 +4,9 @@ import "testing"

func init() {
TestCases = append(TestCases,
NewGeneralTestCase("nethttp-basic-test", "", "nethttp", "", "", "1.18", "", TestBasicNetHttp),
NewGeneralTestCase("nethttp-http-2-test", "", "nethttp", "", "", "1.18", "", TestHttp2),
NewGeneralTestCase("nethttp-https-test", "", "nethttp", "", "", "1.18", "", TestHttps),
NewGeneralTestCase("nethttp-basic-test", "nethttp", "", "", "1.18", "", TestBasicNetHttp),
NewGeneralTestCase("nethttp-http-2-test", "nethttp", "", "", "1.18", "", TestHttp2),
NewGeneralTestCase("nethttp-https-test", "nethttp", "", "", "1.18", "", TestHttps),
)
}

1 change: 1 addition & 0 deletions test/redis/v9.0.5/go.mod
Original file line number Diff line number Diff line change
@@ -5,6 +5,7 @@ go 1.21
replace github.com/alibaba/opentelemetry-go-auto-instrumentation => ../../../../opentelemetry-go-auto-instrumentation

require (
// import this dependency to use verifier
github.com/alibaba/opentelemetry-go-auto-instrumentation v0.0.0-00010101000000-000000000000
github.com/redis/go-redis/v9 v9.0.5
go.opentelemetry.io/otel v1.28.0
12 changes: 6 additions & 6 deletions test/redis_tests.go
Original file line number Diff line number Diff line change
@@ -13,12 +13,12 @@ const redis_dependency_name = "github.com/redis/go-redis/v9"
const redis_module_name = "redis"

func init() {
TestCases = append(TestCases, NewGeneralTestCase("redis-9.0.5-executing-commands-test", redis_dependency_name, redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestExecutingCommands),
NewGeneralTestCase("redis-9.0.5-executing-unsupported-commands-test", redis_dependency_name, redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestExecutingUnsupporetedCommands),
NewGeneralTestCase("redis-9.0.5-redis-conn-test", redis_dependency_name, redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestRedisConn),
NewGeneralTestCase("redis-9.0.5-ring-test", redis_dependency_name, redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestRedisRing),
NewGeneralTestCase("redis-9.0.5-transactions-test", redis_dependency_name, redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestRedisTransactions),
NewGeneralTestCase("redis-9.0.5-universal-test", redis_dependency_name, redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestRedisUniversal),
TestCases = append(TestCases, NewGeneralTestCase("redis-9.0.5-executing-commands-test", redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestExecutingCommands),
NewGeneralTestCase("redis-9.0.5-executing-unsupported-commands-test", redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestExecutingUnsupporetedCommands),
NewGeneralTestCase("redis-9.0.5-redis-conn-test", redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestRedisConn),
NewGeneralTestCase("redis-9.0.5-ring-test", redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestRedisRing),
NewGeneralTestCase("redis-9.0.5-transactions-test", redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestRedisTransactions),
NewGeneralTestCase("redis-9.0.5-universal-test", redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestRedisUniversal),
NewMuzzleTestCase("redis-9.0.5-muzzle", redis_dependency_name, redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", []string{"test_executing_commands.go"}),
NewLatestDepthTestCase("redis-9.0.5-executing-commands-latestDepth", redis_dependency_name, redis_module_name, "v9.0.5", "v9.5.1", "1.18", "", TestExecutingCommands))
}
Loading

0 comments on commit f0e3df9

Please sign in to comment.