-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
12 changed files
with
670 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| 2.5.3 | ||
| 2.5.4 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,91 @@ | ||
| # Redis plugin | ||
|
|
||
| Plugin to query Redis (https://redis.io) as a data source. | ||
|
|
||
|
|
||
| Compile with: | ||
| ```sh | ||
| go build -buildmode=plugin -ldflags="-w" -o redis.so ./*.go | ||
| ``` | ||
|
|
||
| **Warning** | ||
|
|
||
| Redis does NOT accept complex queries, like SQL databases do. | ||
|
|
||
| The easiest workaround is to exclude Redis DB from the `global` namespace | ||
| and query it independently, to execute needed queries one by one. | ||
|
|
||
|
|
||
| # Access details | ||
|
|
||
| Source YAML definition's `access` fields: | ||
| - **addr**: HOST:PORT database's access point, for example - `localhost:6379` | ||
| - **user**: username to connect to the database | ||
| - **password**: user's password | ||
| - **db**: database number to use | ||
| - **field**: Redis key will be used as this field name | ||
|
|
||
|
|
||
| # Usage | ||
|
|
||
| Simple example of a new Redis data source. Insert test data: | ||
| ```sh | ||
| redis-cli -u redis://localhost:6379/8 | ||
| ACL SETUSER graphoscope on >password allkeys +hset +hget +hgetall +select +ping | ||
| ACL SAVE # Or 'CONFIG REWRITE' | ||
| AUTH graphoscope password | ||
|
|
||
| HSET '[email protected]' username 'a' fqdn 'example.com' count 13 seen '18-02-2023T15:34:00.000000Z' | ||
| HSET '[email protected]' username 'b' fqdn 'example.com' count 13 seen '19-02-2023T15:34:00.000000Z' | ||
| HSET '[email protected]' username 'c' fqdn 'example.com' count 13 seen '20-02-2023T15:34:00.000000Z' | ||
| HSET '[email protected]' username 'd' fqdn 'example.com' count 13 seen '21-02-2023T15:34:00.000000Z' | ||
| HSET '[email protected]' username 'e' fqdn 'example.com' count 13 seen '22-02-2023T15:34:00.000000Z' | ||
| ``` | ||
|
|
||
| Access data will be used by the source's YAML definition. Example: | ||
| ```yaml | ||
| name: retest | ||
| label: RETest | ||
| icon: database | ||
|
|
||
| plugin: redis | ||
| inGlobal: false | ||
| includeDatetime: false | ||
| supportsSQL: false | ||
|
|
||
| access: | ||
| addr: 127.0.0.1:6379 | ||
| user: graphoscope | ||
| password: password | ||
| db: 8 | ||
| field: email | ||
|
|
||
| queryFields: | ||
|
|
||
| replaceFields: | ||
| datetime: seen | ||
| domain: fqdn | ||
|
|
||
|
|
||
| relations: | ||
| - | ||
| from: | ||
| id: email | ||
| group: email | ||
| search: email | ||
| attributes: [ "seen", "fqdn" ] | ||
|
|
||
| to: | ||
| id: username | ||
| group: username | ||
| search: username | ||
|
|
||
| edge: | ||
| attributes: [ "count" ] | ||
| ``` | ||
| Test with a query: | ||
| ```sh | ||
| curl -XGET 'https://localhost:443/api?uuid=auth-key&sql=FROM+retest+WHERE+email=%[email protected]%27' | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| /* | ||
| * SQL to the field/value list convertor | ||
| */ | ||
|
|
||
| package main | ||
|
|
||
| import ( | ||
| "github.com/blastrain/vitess-sqlparser/sqlparser" | ||
| ) | ||
|
|
||
| /* | ||
| * Convert SQL query to the list of [field,value] | ||
| */ | ||
| func (p *plugin) convert(sel *sqlparser.Select) (string, error) { | ||
|
|
||
| // Handle WHERE. | ||
| // Top level node pass in an empty interface | ||
| // to tell the children this is root. | ||
| // Is there any better way? | ||
| var rootParent sqlparser.Expr | ||
|
|
||
| // List of requested fields & values | ||
| fields, err := handleSelectWhere(&sel.Where.Expr, true, &rootParent) | ||
| if err != nil { | ||
| return "", err | ||
| } | ||
|
|
||
| if len(fields) == 1 && fields[0][0] == p.source.Access["field"] { | ||
| return fields[0][1], nil | ||
| } | ||
|
|
||
| return "", nil | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "github.com/cert-lv/graphoscope/pdk" | ||
| "github.com/redis/go-redis/v9" | ||
| ) | ||
|
|
||
| /* | ||
| * Export symbols | ||
| */ | ||
| var ( | ||
| Name = "redis" | ||
| Version = "1.0.0" | ||
| Plugin plugin | ||
| ) | ||
|
|
||
| /* | ||
| * Structure to be imported by the core as a plugin | ||
| */ | ||
| type plugin struct { | ||
|
|
||
| // Inherit default configuration fields | ||
| source *pdk.Source | ||
|
|
||
| // Custom fields | ||
| client *redis.Client | ||
| limit int | ||
| } |
Oops, something went wrong.