Skip to content

Commit

Permalink
Merge pull request #232 from guregu/v2-dev
Browse files Browse the repository at this point in the history 
dynamo v2: featuring aws-sdk-go-v2
  • Loading branch information
@guregu
guregu committed Jun 15, 2024
2 parents 929601d + 756ca4a commit ab7d057
Show file tree
Hide file tree
Showing 52 changed files with 1,874 additions and 1,841 deletions.
101 changes: 59 additions & 42 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,21 +1,28 @@
## dynamo [![GoDoc](https://godoc.org/github.com/guregu/dynamo?status.svg)](https://godoc.org/github.com/guregu/dynamo)
`import "github.com/guregu/dynamo"`
## dynamo [![GoDoc](https://godoc.org/github.com/guregu/dynamo/v2?status.svg)](https://godoc.org/github.com/guregu/dynamo/v2)
`import "github.com/guregu/dynamo/v2"`

dynamo is an expressive [DynamoDB](https://aws.amazon.com/dynamodb/) client for Go, with an easy but powerful API. dynamo integrates with the official [AWS SDK](https://github.com/aws/aws-sdk-go/).
dynamo is an expressive [DynamoDB](https://aws.amazon.com/dynamodb/) client for Go, with an easy but powerful API. dynamo integrates with the official [AWS SDK v2](https://github.com/aws/aws-sdk-go-v2/).

This library is stable and versioned with Go modules.

> [!TIP]
> dynamo v2 is finally released! See [**v2 Migration**](#migrating-from-v1) for tips on migrating from dynamo v1.
>
> For dynamo v1, which uses [aws-sdk-go v1](https://github.com/aws/aws-sdk-go/), see: [**dynamo v1 documentation**](https://pkg.go.dev/github.com/guregu/dynamo).
### Example

```go
package dynamo

import (
"time"
"context"
"log"

"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/session"
"github.com/guregu/dynamo"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/guregu/dynamo/v2"
)

// Use struct tags much like the standard JSON library,
Expand All @@ -34,27 +41,30 @@ type widget struct {


func main() {
sess := session.Must(session.NewSession())
db := dynamo.New(sess, &aws.Config{Region: aws.String("us-west-2")})
cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"))
if err != nil {
log.Fatalf("unable to load SDK config, %v", err)
}
db := dynamo.New(cfg)
table := db.Table("Widgets")

// put item
w := widget{UserID: 613, Time: time.Now(), Msg: "hello"}
err := table.Put(w).Run()
err = table.Put(w).Run(ctx)

// get the same item
var result widget
err = table.Get("UserID", w.UserID).
Range("Time", dynamo.Equal, w.Time).
One(&result)
One(ctx, &result)

// get all items
var results []widget
err = table.Scan().All(&results)
err = table.Scan().All(ctx, &results)

// use placeholders in filter expressions (see Expressions section below)
var filtered []widget
err = table.Scan().Filter("'Count' > ?", 10).All(&filtered)
err = table.Scan().Filter("'Count' > ?", 10).All(ctx, &filtered)
}
```

Expand All @@ -71,14 +81,14 @@ Please see the [DynamoDB reference on expressions](http://docs.aws.amazon.com/am
```go
// Using single quotes to escape a reserved word, and a question mark as a value placeholder.
// Finds all items whose date is greater than or equal to lastUpdate.
table.Scan().Filter("'Date' >= ?", lastUpdate).All(&results)
table.Scan().Filter("'Date' >= ?", lastUpdate).All(ctx, &results)

// Using dollar signs as a placeholder for attribute names.
// Deletes the item with an ID of 42 if its score is at or below the cutoff, and its name starts with G.
table.Delete("ID", 42).If("Score <= ? AND begins_with($, ?)", cutoff, "Name", "G").Run()
table.Delete("ID", 42).If("Score <= ? AND begins_with($, ?)", cutoff, "Name", "G").Run(ctx)

// Put a new item, only if it doesn't already exist.
table.Put(item{ID: 42}).If("attribute_not_exists(ID)").Run()
table.Put(item{ID: 42}).If("attribute_not_exists(ID)").Run(ctx)
```

### Encoding support
Expand Down Expand Up @@ -177,42 +187,38 @@ This creates a table with the primary hash key ID and range key Time. It creates

### Retrying

Requests that fail with certain errors (e.g. `ThrottlingException`) are [automatically retried](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Programming.Errors.html#Programming.Errors.RetryAndBackoff).
Methods that take a `context.Context` will retry until the context is canceled.
Methods without a context will use the `RetryTimeout` global variable, which can be changed; using context is recommended instead.

#### Limiting or disabling retrying
As of v2, dynamo relies on the AWS SDK for retrying. See: [**Retries and Timeouts documentation**](https://aws.github.io/aws-sdk-go-v2/docs/configuring-sdk/retries-timeouts/) for information about how to configure its behavior.

The maximum number of retries can be configured via the `MaxRetries` field in the `*aws.Config` passed to `dynamo.New()`. A value of `0` will disable retrying. A value of `-1` means unlimited and is the default (however, context or `RetryTimeout` will still apply).
By default, canceled transactions (i.e. errors from conflicting transactions) will not be retried. To get automatic retrying behavior like in v1, use [`dynamo.RetryTxConflicts`](https://godoc.org/github.com/guregu/dynamo/v2#RetryTxConflicts).

```go
db := dynamo.New(session, &aws.Config{
MaxRetries: aws.Int(0), // disables automatic retrying
})
```

#### Custom retrying logic

If a custom [`request.Retryer`](https://pkg.go.dev/github.com/aws/aws-sdk-go/aws/request#Retryer) is set via the `Retryer` field in `*aws.Config`, dynamo will delegate retrying entirely to it, taking precedence over other retrying settings. This allows you to have full control over all aspects of retrying.
import (
"context"
"log"

Example using [`client.DefaultRetryer`](https://pkg.go.dev/github.com/aws/aws-sdk-go/aws/client#DefaultRetryer):
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/aws/retry"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/guregu/dynamo/v2"
)

```go
retryer := client.DefaultRetryer{
NumMaxRetries: 10,
MinThrottleDelay: 500 * time.Millisecond,
MaxThrottleDelay: 30 * time.Second,
func main() {
cfg, err := config.LoadDefaultConfig(context.Background(), config.WithRetryer(func() aws.Retryer {
return retry.NewStandard(dynamo.RetryTxConflicts)
}))
if err != nil {
log.Fatal(err)
}
db := dynamo.New(cfg)
// use db
}
db := dynamo.New(session, &aws.Config{
Retryer: retryer,
})
```

### Compatibility with the official AWS library

dynamo has been in development before the official AWS libraries were stable. We use a different encoder and decoder than the [dynamodbattribute](https://godoc.org/github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute) package. dynamo uses the `dynamo` struct tag instead of the `dynamodbav` struct tag, and we also prefer to automatically omit invalid values such as empty strings, whereas the dynamodbattribute package substitutes null values for them. Items that satisfy the [`dynamodbattribute.(Un)marshaler`](https://godoc.org/github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute#Marshaler) interfaces are compatibile with both libraries.
dynamo has been in development before the official AWS libraries were stable. We use a different encoder and decoder than the [dynamodbattribute](https://pkg.go.dev/github.com/jviney/aws-sdk-go-v2/service/dynamodb/dynamodbattribute) package. dynamo uses the `dynamo` struct tag instead of the `dynamodbav` struct tag, and we also prefer to automatically omit invalid values such as empty strings, whereas the dynamodbattribute package substitutes null values for them. Items that satisfy the [`dynamodbattribute.(Un)marshaler`](https://pkg.go.dev/github.com/jviney/aws-sdk-go-v2/service/dynamodb/dynamodbattribute#Marshaler) interfaces are compatibile with both libraries.

In order to use dynamodbattribute's encoding facilities, you must wrap objects passed to dynamo with [`dynamo.AWSEncoding`](https://godoc.org/github.com/guregu/dynamo#AWSEncoding). Here is a quick example:
In order to use dynamodbattribute's encoding facilities, you must wrap objects passed to dynamo with [`dynamo.AWSEncoding`](https://godoc.org/github.com/guregu/dynamo/v2#AWSEncoding). Here is a quick example:

```go
// Notice the use of the dynamodbav struct tag
Expand All @@ -224,12 +230,23 @@ type book struct {
err := db.Table("Books").Put(dynamo.AWSEncoding(book{
ID: 42,
Title: "Principia Discordia",
})).Run()
})).Run(ctx)
// When getting an item you MUST pass a pointer to AWSEncoding!
var someBook book
err := db.Table("Books").Get("ID", 555).One(dynamo.AWSEncoding(&someBook))
err := db.Table("Books").Get("ID", 555).One(ctx, dynamo.AWSEncoding(&someBook))
```

### Migrating from v1

The API hasn't changed much from v1 to v2. Here are some migration tips:

- All request methods now take a [context](https://go.dev/blog/context) as their first argument.
- Retrying relies on the AWS SDK configuration, see: [Retrying](#retrying).
- Transactions won't retry TransactionCanceled responses by default anymore, make sure you configure that if you need it.
- Arguments that took `int64` (such as in `Query.Limit`) now take `int` instead.
- [Compatibility with the official AWS library](#compatibility-with-the-official-aws-library) uses v2 interfaces instead of v1.
- `KMSMasterKeyArn` renamed to `KMSMasterKeyARN`.

### Integration tests

By default, tests are run in offline mode. In order to run the integration tests, some environment variables need to be set.
Expand Down
108 changes: 52 additions & 56 deletions attr.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ import (
"fmt"
"strconv"

"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)

// Item is a type alias for the raw DynamoDB item type.
type Item = map[string]*dynamodb.AttributeValue
type Item = map[string]types.AttributeValue

type shapeKey byte

Expand All @@ -31,114 +31,110 @@ const (
shapeInvalid shapeKey = 0
)

func shapeOf(av *dynamodb.AttributeValue) shapeKey {
func shapeOf(av types.AttributeValue) shapeKey {
if av == nil {
return shapeInvalid
}
switch {
case av.B != nil:
switch av.(type) {
case *types.AttributeValueMemberB:
return shapeB
case av.BS != nil:
case *types.AttributeValueMemberBS:
return shapeBS
case av.BOOL != nil:
case *types.AttributeValueMemberBOOL:
return shapeBOOL
case av.N != nil:
case *types.AttributeValueMemberN:
return shapeN
case av.S != nil:
case *types.AttributeValueMemberS:
return shapeS
case av.L != nil:
case *types.AttributeValueMemberL:
return shapeL
case av.NS != nil:
case *types.AttributeValueMemberNS:
return shapeNS
case av.SS != nil:
case *types.AttributeValueMemberSS:
return shapeSS
case av.M != nil:
case *types.AttributeValueMemberM:
return shapeM
case av.NULL != nil:
case *types.AttributeValueMemberNULL:
return shapeNULL
}
return shapeAny
}

// av2iface converts an av into interface{}.
func av2iface(av *dynamodb.AttributeValue) (interface{}, error) {
switch {
case av.B != nil:
return av.B, nil
case av.BS != nil:
return av.BS, nil
case av.BOOL != nil:
return *av.BOOL, nil
case av.N != nil:
return strconv.ParseFloat(*av.N, 64)
case av.S != nil:
return *av.S, nil
case av.L != nil:
list := make([]interface{}, 0, len(av.L))
for _, item := range av.L {
func av2iface(av types.AttributeValue) (interface{}, error) {
switch v := av.(type) {
case *types.AttributeValueMemberB:
return v.Value, nil
case *types.AttributeValueMemberBS:
return v.Value, nil
case *types.AttributeValueMemberBOOL:
return v.Value, nil
case *types.AttributeValueMemberN:
return strconv.ParseFloat(v.Value, 64)
case *types.AttributeValueMemberS:
return v.Value, nil
case *types.AttributeValueMemberL:
list := make([]interface{}, 0, len(v.Value))
for _, item := range v.Value {
iface, err := av2iface(item)
if err != nil {
return nil, err
}
list = append(list, iface)
}
return list, nil
case av.NS != nil:
set := make([]float64, 0, len(av.NS))
for _, n := range av.NS {
f, err := strconv.ParseFloat(*n, 64)
case *types.AttributeValueMemberNS:
set := make([]float64, 0, len(v.Value))
for _, n := range v.Value {
f, err := strconv.ParseFloat(n, 64)
if err != nil {
return nil, err
}
set = append(set, f)
}
return set, nil
case av.SS != nil:
set := make([]string, 0, len(av.SS))
for _, s := range av.SS {
set = append(set, *s)
}
return set, nil
case av.M != nil:
m := make(map[string]interface{}, len(av.M))
for k, v := range av.M {
case *types.AttributeValueMemberSS:
return v.Value, nil
case *types.AttributeValueMemberM:
m := make(map[string]interface{}, len(v.Value))
for k, v := range v.Value {
iface, err := av2iface(v)
if err != nil {
return nil, err
}
m[k] = iface
}
return m, nil
case av.NULL != nil:
case *types.AttributeValueMemberNULL:
return nil, nil
}
return nil, fmt.Errorf("dynamo: unsupported AV: %#v", *av)
return nil, fmt.Errorf("dynamo: unsupported AV: %#v", av)
}

func avTypeName(av *dynamodb.AttributeValue) string {
func avTypeName(av types.AttributeValue) string {
if av == nil {
return "<nil>"
}
switch {
case av.B != nil:
switch av.(type) {
case *types.AttributeValueMemberB:
return "binary"
case av.BS != nil:
case *types.AttributeValueMemberBS:
return "binary set"
case av.BOOL != nil:
case *types.AttributeValueMemberBOOL:
return "boolean"
case av.N != nil:
case *types.AttributeValueMemberN:
return "number"
case av.S != nil:
case *types.AttributeValueMemberS:
return "string"
case av.L != nil:
case *types.AttributeValueMemberL:
return "list"
case av.NS != nil:
case *types.AttributeValueMemberNS:
return "number set"
case av.SS != nil:
case *types.AttributeValueMemberSS:
return "string set"
case av.M != nil:
case *types.AttributeValueMemberM:
return "map"
case av.NULL != nil:
case *types.AttributeValueMemberNULL:
return "null"
}
return "<empty>"
Expand Down
Loading

0 comments on commit ab7d057

Please sign in to comment.