Connect to Postgres database using AWS Secrets Manager

aws secrets manager lambda example
aws secrets manager rds
aws secrets manager rotation lambda example
aws secrets manager rotate access keys
aws secrets manager client
aws secret store
aws secrets manager rds cloudformation
aws secrets manager tutorial

Wanted to use AWS Secrets manager to login to postgres without using username and password as a plain text. i am not sure if this is doable, please forgive me if not. Currently this is what i am using to login to postgres using psycopg2:

 import psycopg2

conn = psycopg2.connect(host="hostname",port='5432',database="db", user="admin", password="12345")

i've already stored the username and password in secrets manager but not sure how to use it here. Please help


You can store your credentials (username/password) in SecretsManager using the console. You can store them as key value pairs, for example -

{ "username": "admin", "password": "12345" }

To use this in your Python script, you could do something like this -

session = boto3.session.Session()
client = session.client(
    service_name='secretsmanager',
    region=< region_name >
)
secret = client.get_secret_value(
         SecretId=secret_name
)
secret_dict = json.loads(secret['SecretString'])

username = secret_dict['username']
passw = secret_dict['password']

conn = psycopg2.connect(host="hostname",port='5432',database="db", user=username, password=passw)

Note that this is a simplified example without error handling. You also need to fill in the right region in place of < region_name > in the example.

Rotate Amazon RDS database credentials automatically with AWS , Secrets Manager offers built-in integrations for MySQL, PostgreSQL, and You can control access to your secrets by using fine-grained AWS  Secrets Manager – AWS Secrets Manager helps you protect secrets for accessing your applications, services, and IT resources. You can easily rotate, manage, and retrieve database credentials, API keys, and other secrets throughout their lifecycle.


You should use the below process:

  1. connect to AWS secrets manager.
  2. Retrivee the username and password. You need to map it the way you have stored in secrets manager.
  3. Store that in a variable and pass it to connection string.

Below is the sample python script provided by amazon:

import boto3
import base64
from botocore.exceptions import ClientError


def get_secret():

    secret_name = "<<{{MySecretName}}>>"
    region_name = "<<{{MyRegionName}}>>"

    # Create a Secrets Manager client
    session = boto3.session.Session()
    client = session.client(
        service_name='secretsmanager',
        region_name=region_name
    )

    # In this sample we only handle the specific exceptions for the 'GetSecretValue' API.
    # See https://docs.aws.amazon.com/secretsmanager/latest/apireference/API_GetSecretValue.html
    # We rethrow the exception by default.

    try:
        get_secret_value_response = client.get_secret_value(
            SecretId=secret_name
        )
    except ClientError as e:
        if e.response['Error']['Code'] == 'DecryptionFailureException':
            # Secrets Manager can't decrypt the protected secret text using the provided KMS key.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InternalServiceErrorException':
            # An error occurred on the server side.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidParameterException':
            # You provided an invalid value for a parameter.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'InvalidRequestException':
            # You provided a parameter value that is not valid for the current state of the resource.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
        elif e.response['Error']['Code'] == 'ResourceNotFoundException':
            # We can't find the resource that you asked for.
            # Deal with the exception here, and/or rethrow at your discretion.
            raise e
    else:
        # Decrypts secret using the associated KMS CMK.
        # Depending on whether the secret is a string or binary, one of these fields will be populated.
        if 'SecretString' in get_secret_value_response:
            secret = get_secret_value_response['SecretString']
        else:
            decoded_binary_secret = base64.b64decode(get_secret_value_response['SecretBinary'])

    # Your code goes here.  

Select which RDS database this secret will access, Secrets Manager supports native rotation for MySQL, PostgreSQL, Oracle, SQL Server, MariaDB, Aurora MySQL, and Aurora PostgreSQL databases hosted on  Using Secrets Manager, you can store the credentials and then use AWS Identity and Access Management (IAM) to allow only certain IAM users and roles to read the credentials. For the steps involved in this, see Creating and Managing Secrets with AWS Secrets Manager in the AWS Secrets Manager User Guide.


I created an open source library called pysecret, it makes the API very clean and straightforward to use. Here's the documentation of AWS Secret Manager integration: https://github.com/MacHu-GWU/pysecret-project#aws-key-management-service-and-secret-manager-integration

  1. manually put your secret value in json or create one with pysecret.
from pysecret import AWSSecret

aws_profile = "my_aws_profile"
aws = AWSSecret(profile_name=aws_profile)

secret_id = "my-example-secret"
secret_data = {
    "host": "www.example.com",
    "port": 1234,
    "database": "mydatabase",
    "username": "admin",
    "password": "mypassword",
}
aws.deploy_secret(name=secret_id, secret_data=secret_data) # or you can pass kms_key_id if you created a custom kms key

Then you should be able to see the secret been created in your aws console.

  1. read your secret value in lambda function or in any of your python code.
aws = AWSSecret(profile_name=aws_profile) # in lambda code, don't need ``profile_name=aws_profile``
username = aws.get_secret_value(secret_id="my-example-secret", key="password") # admin
password = aws.get_secret_value(secret_id="my-example-secret", key="password") # mypassword

If you are writing code for Lambda Function you can take a look at my other answer here Using AWS Secrets Manager with Python (Lambda Console)

Hope this answers your question.

How to use AWS Secrets Manager to rotate credentials for all , Secrets Manager offers built-in integrations for rotating credentials for all Amazon RDS databases (MySQL, PostgreSQL, Oracle, Microsoft SQL  Phase 1: Store a secret in Secrets Manager. Open the Secrets Manager console and select Store a new secret . I select Credentials for RDS database because I’m storing credentials for a MySQL database hosted on Amazon RDS. For this example, I store the Next, I review the encryption setting and


Connecting to Your DB Instance Using IAM Authentication from the , You can connect from the command line to an Amazon RDS for PostgreSQL DB instance with the AWS CLI and psql command line tool as described following. Hi Daniel thanks for the response. I already was able to access a postgresSQL external database using The PostgreSQL Database Connector (by ardoRic), the challenge, since the credentials to the database will change periodically due to the secrets rotation, is to be able to connect to the AWS Secrets Manager, retrieve the secrets information and change the connection URL with the new


Manage credentials using AWS Secrets Manager, An AWS account with access to Secrets Manager Secrets can be database credentials, passwords, third-party API keys, and secretName=postgres-local  The AWS Secrets Manager allows you to securely store your database passwords (or any other secrets such as API keys) inside AWS itself. When your application needs a secret, it requests it from the AWS Secrets Manager and responds with the secret if you have the correct IAM permissions .


Managing PostgreSQL users and roles, Amazon Web Services (AWS) provides two managed PostgreSQL options: You can connect to the RDS endpoint for your PostgreSQL database using with AWS Secrets Manager in the AWS Secrets Manager User Guide. The AWS Secrets Manager JDBC Library enables Java developers to easily connect to SQL databases using secrets stored in AWS Secrets Manager. This library is licensed under the Apache 2.0 License. We provide database drivers that intercept calls to real database drivers and swap out secret IDs for