Unmarshal map[string]DynamoDBAttributeValue into a struct

golang dynamodb struct tag
go dynamodb attributevalue
marshall dynamodb attributevalue
dynamodbav:,omitempty
golang dynamodbav:,omitempty
golang dynamodb queryinput example
golang gsi dynamodb
dynamodb attributes golang

I'm trying to set-up an AWS-lambda using aws-sdk-go that is triggered whenever a new user is added to a certain dynamodb table.

Everything is working just fine but I can't find a way to unmarshal a map map[string]DynamoDBAttributeValue like:

{
    "name": {
        "S" : "John"
    },
    "residence_address": {
        "M": {
            "address": {
                "S": "some place"
            }
        }
    }
}

To a given struct, for instance, a User struct. Here is shown an example of unsmarhaling a map[string]*dynamodb.AttributeValue into a given interface, but I can't find a way to do the same thing with map[string]DynamoDBAttributeValue even though these types seem to fit the same purposes.

map[string]DynamoDBAttributeValue is returned by a events.DynamoDBEvents from package github.com/aws/aws-lambda-go/events. This is my code:

package handler

import (
    "context"
    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
    "github.com/aws/aws-sdk-go/service/dynamodb"
)

func HandleDynamoDBRequest(ctx context.Context, e events.DynamoDBEvent) {

    for _, record := range e.Records {

        if record.EventName == "INSERT" {

            // User Struct
            var dynamoUser model.DynamoDBUser

            // Of course this can't be done for incompatible types
            _ := dynamodbattribute.UnmarshalMap(record.Change.NewImage, &dynamoUser)
        }

    }

}

Of course, I can marshal record.Change.NewImage to JSON and unmarshal it back to a given struct, but then, I would have to manually initialize dynamoUser attributes starting from the latter ones.

Or I could even write a function that parses map[string]DynamoDBAttributeValue to map[string]*dynamodb.AttributeValue like:

func getAttributeValueMapFromDynamoDBStreamRecord(e events.DynamoDBStreamRecord) map[string]*dynamodb.AttributeValue {
    image := e.NewImage
    m := make(map[string]*dynamodb.AttributeValue)
    for k, v := range image {
        if v.DataType() == events.DataTypeString {
            s := v.String()
            m[k] = &dynamodb.AttributeValue{
                S : &s,
            }
        }
        if v.DataType() == events.DataTypeBoolean {
            b := v.Boolean()
            m[k] = &dynamodb.AttributeValue{
                BOOL : &b,
            }
        }
        // . . .
        if v.DataType() == events.DataTypeMap {
            // ?
        }
    }
    return m
}

And then simply use dynamodbattribute.UnmarshalMap, but on events.DataTypeMap it would be quite a tricky process.

Is there a way through which I can unmarshal a DynamoDB record coming from a events.DynamoDBEvent into a struct with a similar method shown for map[string]*dynamodb.AttributeValue?

I tried the function you provided, and I met some problems with events.DataTypeList, so I managed to write the following function that does the trick:

// UnmarshalStreamImage converts events.DynamoDBAttributeValue to struct
func UnmarshalStreamImage(attribute map[string]events.DynamoDBAttributeValue, out interface{}) error {

    dbAttrMap := make(map[string]*dynamodb.AttributeValue)

    for k, v := range attribute {

        var dbAttr dynamodb.AttributeValue

        bytes, marshalErr := v.MarshalJSON(); if marshalErr != nil {
            return marshalErr
        }

        json.Unmarshal(bytes, &dbAttr)
        dbAttrMap[k] = &dbAttr
    }

    return dynamodbattribute.UnmarshalMap(dbAttrMap, out)

}

Unmarshal map[string]DynamoDBAttributeValue into a struct , Here is shown an example of unmarshalling a map[string]*dynamodb.​AttributeValue into a given interface, but I can't find a way to do the same  Of course, I can marshal record.Change.NewImage to JSON and unmarshal it back to a given struct, but then, I would have to manually initialize dynamoUser attributes starting from the latter ones. Or I could even write a function that parses map[string]DynamoDBAttributeValue to map[string]*dynamodb.AttributeValue like:

I was frustrated that the type of NewImage from the record wasn't map[string]*dynamodb.AttributeValue so I could use the dynamodbattribute package.

The JSON representation of events.DynamoDBAttributeValue seems to be the same as the JSON represenation of dynamodb.AttributeValue.

So I tried creating my own DynamoDBEvent type and changed the type of OldImage and NewImage, so it would be marshalled into map[string]*dynamodb.AttributeValue instead of map[string]events.DynamoDBAttributeValue

It is a little bit ugly but it works for me.

package main

import (
    "github.com/aws/aws-lambda-go/events"
    "github.com/aws/aws-lambda-go/lambda"
    "github.com/aws/aws-sdk-go/service/dynamodb"
    "github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
    "fmt"
)

func main() {

    lambda.Start(lambdaHandler)
}

// changed type of event from: events.DynamoDBEvent to DynamoDBEvent (see below)
func lambdaHandler(event DynamoDBEvent) error {

    for _, record := range event.Records {

        change := record.Change
        newImage := change.NewImage // now of type: map[string]*dynamodb.AttributeValue

        var item IdOnly
        err := dynamodbattribute.UnmarshalMap(newImage, &item)
        if err != nil {
            return err
        }

        fmt.Println(item.Id)
    }

    return nil
}

type IdOnly struct {
    Id string `json:"id"`
}

type DynamoDBEvent struct {
    Records []DynamoDBEventRecord `json:"Records"`
}

type DynamoDBEventRecord struct {
    AWSRegion      string                       `json:"awsRegion"`
    Change         DynamoDBStreamRecord         `json:"dynamodb"`
    EventID        string                       `json:"eventID"`
    EventName      string                       `json:"eventName"`
    EventSource    string                       `json:"eventSource"`
    EventVersion   string                       `json:"eventVersion"`
    EventSourceArn string                       `json:"eventSourceARN"`
    UserIdentity   *events.DynamoDBUserIdentity `json:"userIdentity,omitempty"`
}

type DynamoDBStreamRecord struct {
    ApproximateCreationDateTime events.SecondsEpochTime             `json:"ApproximateCreationDateTime,omitempty"`
    // changed to map[string]*dynamodb.AttributeValue
    Keys                        map[string]*dynamodb.AttributeValue `json:"Keys,omitempty"`
    // changed to map[string]*dynamodb.AttributeValue
    NewImage                    map[string]*dynamodb.AttributeValue `json:"NewImage,omitempty"`
    // changed to map[string]*dynamodb.AttributeValue
    OldImage                    map[string]*dynamodb.AttributeValue `json:"OldImage,omitempty"`
    SequenceNumber              string                              `json:"SequenceNumber"`
    SizeBytes                   int64                               `json:"SizeBytes"`
    StreamViewType              string                              `json:"StreamViewType"`
}

dynamodbattribute - Amazon Web Services, Here is shown an example of unsmarhaling a map[string]*dynamodb.​AttributeValue into a given interface, but I can't find a way to do the same  How to unmarshal struct into map in golang [closed] Ask Question Asked 1 year, I don't know how to unmarshal planets field into map[string]int,

For the moment I managed to solve this issue by implementing the function I mentioned in the question.

// May take either `map[string]DynamoDBAttributeValue` 
// or `DynamoDBAttributeValue` as input.
func EventStreamToMap(attribute interface{}) map[string]*dynamodb.AttributeValue {

    // Map to be returned
    m := make(map[string]*dynamodb.AttributeValue)

    tmp := make(map[string]events.DynamoDBAttributeValue)

    switch t := attribute.(type) {
    case map[string]events.DynamoDBAttributeValue:
        tmp = t
    case events.DynamoDBAttributeValue:
        tmp = t.Map()
    }

    for k, v := range tmp {
        switch v.DataType() {
        case events.DataTypeString:
            s := v.String()
            m[k] = &dynamodb.AttributeValue{
                S : &s,
            }
        case events.DataTypeBoolean:
            b := v.Boolean()
            m[k] = &dynamodb.AttributeValue{
                BOOL : &b,
            }
        // case events.SomeOtherType:
        //     ...
        case events.DataTypeMap:
            m[k] = &dynamodb.AttributeValue{
                M : EventStreamToMap(v),
            }
        }
    }
    return m
}

But I'm still open to other implementations.

Unmarshal map[string]DynamoDBAttributeValue into a struct · Issue , "github.com/aws/aws-sdk-go/service/dynamodb" err := Unmarshal(c.in, c.actual​) AttributeValue{M: map[string]*dynamodb.AttributeValue attribute to struct. Error: Received unexpected error: yaml: unmarshal errors: line 1: cannot unmarshal !!map into []map[string]main.conversionGroup The top level properties are dynamic so I need to parse them as string, every other key in the structure will always be the same, hence the structs for those parts. How can I parse this?

decode_test.go - aws/aws-sdk-go, 44 // 45 // When unmarshaling AttributeValues into structs Unmarshal matches 46 must be a non-nil pointer 87 func UnmarshalMap(m map[string]*dynamodb. Actually can unmarshal directly into map[string]string. EDIT: Some other models based on the comment. type JSONType struct { FirstSet map[string]Point `json:"set1"` } type Point struct { X string `json:"x"` Y string `json:"y"` Z string `json:"z"` } This makes your 3-d point a statically typed struct which is fine.

vault: /dynamodb/dynamodbattribute/decode.go, Query returns a cursor so if you have a long list, you'll need to call it multiple times. _ @savaki. I would have expected UnmarshalListOfMaps to return an error trying to learn Golang and DynamoDB at the same time maybe not so good type Issue struct { ID string `dynamodbav:"id"` } type DAO struct { api *​dynamodb. Decode JSON to a map using type assertion. Decoding JSON to a map using type assertion is useful when the types of your values are unknown. Maps are are ideal if your values fall into one of the supported concrete types (bool, float64, string, or nil).

aws/aws-sdk-go, unmarshal a DynamoDB record coming from a events.DynamoDBEvent into a struct with a similar method shown for map[string]*dynamodb.AttributeValue? To unmarshal a JSON object into a map, Unmarshal first establishes a map to use. If the map is nil, Unmarshal allocates a new map. Otherwise Unmarshal reuses the existing map, keeping existing entries. Unmarshal then stores key-value pairs from the JSON object into the map. The map's key type must either be a string, an integer, or implement

Comments
  • Thanks for your answer. Tomorrow I'll try it. About lists you are right, in the example I did not cover them but It's possible to extend my function to handle them. Anyway your solution is by far better than mine.
  • I tried your function and it works perfectly. I made small fixes to it since dbAttrMap was badly declared and you made no error handling for marshalErr. As soon as you accept the edit I'll accept the answer.
  • It's better now, approve it.