AWS Secrets Manager: Complete Password and Credentials Guide

Eliminate hardcoded credentials with centralized secrets storage, automatic rotation, and fine-grained access control.

What is Secrets Manager

Purpose: Securely store and manage sensitive information (passwords, API keys, certificates, tokens)

Key features:

• Automatic rotation for RDS, Redshift, DocumentDB
• Encryption at rest using KMS
• Fine-grained IAM permissions
• Integration with AWS services (Lambda, ECS, RDS, etc.)
• Audit trail via CloudTrail
• Cross-region replication

vs. Systems Manager Parameter Store:

• Secrets Manager: Automatic rotation, built-in RDS integration, higher cost ($0.40/secret/month)
• Parameter Store: Manual rotation, free tier (10,000 params), basic features

Creating Secrets

Database Credentials

# Store RDS credentials
aws secretsmanager create-secret \
  --name production/db/postgresql \
  --description "Production PostgreSQL database credentials" \
  --secret-string '{
    "username": "dbadmin",
    "password": "SuperSecurePassword123!",
    "engine": "postgres",
    "host": "mydb.cluster-xxxxx.us-east-1.rds.amazonaws.com",
    "port": 5432,
    "dbname": "production"
  }' \
  --kms-key-id alias/secrets-encryption

API Keys

# Store third-party API credentials
aws secretsmanager create-secret \
  --name production/api/stripe \
  --secret-string '{
    "api_key": "sk_live_xxxxx",
    "publishable_key": "pk_live_xxxxx",
    "webhook_secret": "whsec_xxxxx"
  }'

OAuth Tokens

# Store OAuth credentials
aws secretsmanager create-secret \
  --name production/oauth/github \
  --secret-string '{
    "client_id": "Iv1.xxxxx",
    "client_secret": "xxxxx",
    "access_token": "gho_xxxxx",
    "refresh_token": "ghr_xxxxx"
  }'

SSH Keys / Certificates

# Store PEM-encoded private key
aws secretsmanager create-secret \
  --name production/ssh/deploy-key \
  --secret-string "$(cat ~/.ssh/deploy_key)"

# Store SSL certificate
aws secretsmanager create-secret \
  --name production/ssl/api-certificate \
  --secret-binary fileb://certificate.pfx

Retrieving Secrets

AWS CLI

# Get secret value
aws secretsmanager get-secret-value \
  --secret-id production/db/postgresql \
  --query SecretString \
  --output text

# Parse JSON secret
aws secretsmanager get-secret-value \
  --secret-id production/db/postgresql \
  --query SecretString \
  --output text | jq -r '.password'

Python (boto3)

import boto3
import json

def get_secret(secret_name, region_name="us-east-1"):
    client = boto3.client('secretsmanager', region_name=region_name)

    try:
        response = client.get_secret_value(SecretId=secret_name)

        if 'SecretString' in response:
            secret = json.loads(response['SecretString'])
            return secret
        else:
            # Binary secret
            return response['SecretBinary']

    except Exception as e:
        print(f"Error retrieving secret: {e}")
        raise

# Usage
db_creds = get_secret('production/db/postgresql')
print(f"Host: {db_creds['host']}")
print(f"Password: {db_creds['password']}")

Node.js

const AWS = require('aws-sdk');
const client = new AWS.SecretsManager({ region: 'us-east-1' });

async function getSecret(secretName) {
  try {
    const data = await client.getSecretValue({ SecretId: secretName }).promise();

    if ('SecretString' in data) {
      return JSON.parse(data.SecretString);
    } else {
      // Binary secret
      return Buffer.from(data.SecretBinary, 'base64');
    }
  } catch (err) {
    console.error('Error retrieving secret:', err);
    throw err;
  }
}

// Usage
(async () => {
  const dbCreds = await getSecret('production/db/postgresql');
  console.log(`Host: ${dbCreds.host}`);
})();

Lambda Environment Variables (Not Recommended)

# BAD: Hardcoded in environment variable
import os
password = os.environ['DB_PASSWORD']  # Visible in console, logs

# GOOD: Retrieved from Secrets Manager
import boto3
import json

def lambda_handler(event, context):
    client = boto3.client('secretsmanager')
    response = client.get_secret_value(SecretId='production/db/postgresql')
    secret = json.loads(response['SecretString'])

    # Use secret['password'] securely
    return connect_to_database(secret)

Automatic Rotation

RDS PostgreSQL Rotation

Enable automatic rotation:

aws secretsmanager rotate-secret \
  --secret-id production/db/postgresql \
  --rotation-lambda-arn arn:aws:lambda:us-east-1:123456789012:function:SecretsManagerRDSPostgreSQLRotationSingleUser \
  --rotation-rules AutomaticallyAfterDays=30

How it works:

1. Lambda function retrieves current secret
2. Creates new password in RDS
3. Tests connection with new password
4. Updates secret in Secrets Manager
5. Marks old version as deprecated

Rotation strategies:

Single user:

• Rotates password for existing database user
• Brief downtime during password change
• Simpler configuration

Alternating users:

• Creates two database users (user_a, user_b)
• Rotates between them
• Zero downtime rotation
• Recommended for production

Custom Rotation Lambda

For third-party APIs or custom secrets:

import boto3
import json
import requests

def lambda_handler(event, context):
    service_client = boto3.client('secretsmanager')
    arn = event['SecretId']
    token = event['ClientRequestToken']
    step = event['Step']

    # Get current secret
    metadata = service_client.describe_secret(SecretId=arn)
    if not metadata['RotationEnabled']:
        raise ValueError(f"Secret {arn} is not enabled for rotation")

    versions = metadata['VersionIdsToStages']
    if token not in versions:
        raise ValueError(f"Secret version {token} has no stage for rotation")

    if step == "createSecret":
        create_secret(service_client, arn, token)
    elif step == "setSecret":
        set_secret(service_client, arn, token)
    elif step == "testSecret":
        test_secret(service_client, arn, token)
    elif step == "finishSecret":
        finish_secret(service_client, arn, token)
    else:
        raise ValueError("Invalid step parameter")

def create_secret(service_client, arn, token):
    # Generate new API key
    current_secret = service_client.get_secret_value(SecretId=arn, VersionStage="AWSCURRENT")
    current = json.loads(current_secret['SecretString'])

    # Call third-party API to generate new key
    response = requests.post(
        'https://api.example.com/keys',
        headers={'Authorization': f"Bearer {current['api_key']}"}
    )
    new_key = response.json()['key']

    # Store new secret
    new_secret = current.copy()
    new_secret['api_key'] = new_key

    service_client.put_secret_value(
        SecretId=arn,
        ClientRequestToken=token,
        SecretString=json.dumps(new_secret),
        VersionStages=['AWSPENDING']
    )

def set_secret(service_client, arn, token):
    # Update external service with new key (if needed)
    pass

def test_secret(service_client, arn, token):
    # Test new secret works
    pending_secret = service_client.get_secret_value(SecretId=arn, VersionId=token, VersionStage="AWSPENDING")
    pending = json.loads(pending_secret['SecretString'])

    # Validate new API key
    response = requests.get(
        'https://api.example.com/validate',
        headers={'Authorization': f"Bearer {pending['api_key']}"}
    )

    if response.status_code != 200:
        raise ValueError("New secret validation failed")

def finish_secret(service_client, arn, token):
    # Move AWSCURRENT to AWSPREVIOUS
    # Move AWSPENDING to AWSCURRENT
    metadata = service_client.describe_secret(SecretId=arn)
    current_version = None
    for version in metadata["VersionIdsToStages"]:
        if "AWSCURRENT" in metadata["VersionIdsToStages"][version]:
            if version == token:
                return  # Already current
            current_version = version
            break

    service_client.update_secret_version_stage(
        SecretId=arn,
        VersionStage="AWSCURRENT",
        MoveToVersionId=token,
        RemoveFromVersionId=current_version
    )

Deploy rotation Lambda:

# Create Lambda function with rotation logic
aws lambda create-function \
  --function-name CustomSecretRotation \
  --runtime python3.11 \
  --handler lambda_function.lambda_handler \
  --role arn:aws:iam::123456789012:role/LambdaSecretsRotation \
  --code S3Bucket=my-lambda-code,S3Key=rotation.zip \
  --vpc-config SubnetIds=subnet-xxxxx,SecurityGroupIds=sg-xxxxx

# Grant Secrets Manager permission to invoke Lambda
aws lambda add-permission \
  --function-name CustomSecretRotation \
  --statement-id SecretsManagerAccess \
  --action lambda:InvokeFunction \
  --principal secretsmanager.amazonaws.com

IAM Permissions

Minimal Read Access

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:us-east-1:123456789012:secret:production/db/postgresql-*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt",
        "kms:DescribeKey"
      ],
      "Resource": "arn:aws:kms:us-east-1:123456789012:key/xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
      "Condition": {
        "StringEquals": {
          "kms:ViaService": "secretsmanager.us-east-1.amazonaws.com"
        }
      }
    }
  ]
}

Admin Access (Create/Rotate/Delete)

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "secretsmanager:*"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "kms:CreateKey",
        "kms:DescribeKey",
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:GenerateDataKey"
      ],
      "Resource": "*"
    }
  ]
}

Resource-Based Policy

# Allow specific role to access secret
aws secretsmanager put-resource-policy \
  --secret-id production/api/stripe \
  --resource-policy '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Principal": {
          "AWS": "arn:aws:iam::123456789012:role/WebAppRole"
        },
        "Action": "secretsmanager:GetSecretValue",
        "Resource": "*"
      }
    ]
  }'

Integration Examples

Lambda Function

import boto3
import psycopg2
import json

# Initialize outside handler for connection reuse
secretsmanager = boto3.client('secretsmanager')
db_secret = None

def get_db_credentials():
    global db_secret
    if db_secret is None:
        response = secretsmanager.get_secret_value(SecretId='production/db/postgresql')
        db_secret = json.loads(response['SecretString'])
    return db_secret

def lambda_handler(event, context):
    creds = get_db_credentials()

    conn = psycopg2.connect(
        host=creds['host'],
        port=creds['port'],
        user=creds['username'],
        password=creds['password'],
        database=creds['dbname']
    )

    cursor = conn.cursor()
    cursor.execute("SELECT * FROM users WHERE id = %s", (event['user_id'],))
    result = cursor.fetchone()

    conn.close()
    return {'statusCode': 200, 'body': json.dumps(result)}

ECS Task Definition

{
  "family": "web-app",
  "containerDefinitions": [{
    "name": "app",
    "image": "myapp:latest",
    "secrets": [
      {
        "name": "DB_HOST",
        "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789012:secret:production/db/postgresql:host::"
      },
      {
        "name": "DB_PASSWORD",
        "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789012:secret:production/db/postgresql:password::"
      },
      {
        "name": "STRIPE_API_KEY",
        "valueFrom": "arn:aws:secretsmanager:us-east-1:123456789012:secret:production/api/stripe:api_key::"
      }
    ],
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/ecs/web-app",
        "awslogs-region": "us-east-1"
      }
    }
  }],
  "executionRoleArn": "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
  "taskRoleArn": "arn:aws:iam::123456789012:role/ecsTaskRole"
}

EC2 UserData

#!/bin/bash
# Retrieve secret and write to config file
aws secretsmanager get-secret-value \
  --secret-id production/db/postgresql \
  --region us-east-1 \
  --query SecretString \
  --output text | jq -r '.password' > /etc/myapp/db_password

chmod 600 /etc/myapp/db_password
chown myapp:myapp /etc/myapp/db_password

# Start application
systemctl start myapp

RDS Proxy Integration

# Create RDS Proxy using Secrets Manager auth
aws rds create-db-proxy \
  --db-proxy-name production-proxy \
  --engine-family POSTGRESQL \
  --auth '[{
    "AuthScheme": "SECRETS",
    "SecretArn": "arn:aws:secretsmanager:us-east-1:123456789012:secret:production/db/postgresql",
    "IAMAuth": "DISABLED"
  }]' \
  --role-arn arn:aws:iam::123456789012:role/RDSProxyRole \
  --vpc-subnet-ids subnet-xxxxx subnet-yyyyy

Versioning and Rollback

Secret versions:

• AWSCURRENT: Currently active secret
• AWSPENDING: New secret being rotated (not yet active)
• AWSPREVIOUS: Previous version (available for rollback)

Retrieve specific version:

# Get current version
aws secretsmanager get-secret-value \
  --secret-id production/db/postgresql \
  --version-stage AWSCURRENT

# Get previous version (rollback)
aws secretsmanager get-secret-value \
  --secret-id production/db/postgresql \
  --version-stage AWSPREVIOUS

# Get specific version by ID
aws secretsmanager get-secret-value \
  --secret-id production/db/postgresql \
  --version-id xxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Manual rollback:

# Promote previous version to current
aws secretsmanager update-secret-version-stage \
  --secret-id production/db/postgresql \
  --version-stage AWSCURRENT \
  --move-to-version-id <previous-version-id> \
  --remove-from-version-id <current-version-id>

Cross-Region Replication

Enable replication for disaster recovery:

aws secretsmanager replicate-secret-to-regions \
  --secret-id production/db/postgresql \
  --add-replica-regions Region=us-west-2,KmsKeyId=alias/secrets-west \
  --force-overwrite-replica-secret

Benefits:

• Automatic sync across regions
• Independent KMS keys per region
• Failover capability
• Reduced latency for global apps

Monitoring and Auditing

CloudWatch Metrics

# Secret rotation failures
aws cloudwatch put-metric-alarm \
  --alarm-name secrets-rotation-failure \
  --metric-name SecretRotationFailure \
  --namespace AWS/SecretsManager \
  --statistic Sum \
  --period 300 \
  --threshold 1 \
  --comparison-operator GreaterThanOrEqualToThreshold \
  --dimensions Name=SecretId,Value=production/db/postgresql \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:ops-alerts

CloudTrail Logging

Monitor secret access:

// CloudWatch Logs Insights query
fields @timestamp, userIdentity.principalId, eventName, requestParameters.secretId
| filter eventSource = "secretsmanager.amazonaws.com"
| filter eventName = "GetSecretValue"
| sort @timestamp desc
| limit 100

Alerts for suspicious activity:

• GetSecretValue calls from unexpected IPs
• Failed access attempts
• Secret deletion attempts
• Rotation failures

Cost Optimization

Pricing (as of 2024):

• $0.40 per secret per month
• $0.05 per 10,000 API calls

Optimization strategies:

1. Consolidate secrets: Store related values in single JSON secret

{
  "database": {
    "host": "...",
    "password": "..."
  },
  "api": {
    "stripe": "...",
    "sendgrid": "..."
  }
}

2. Cache secrets: Don’t retrieve on every Lambda invocation

# Cache secret for 5 minutes
import time

secret_cache = {}
CACHE_TTL = 300  # 5 minutes

def get_cached_secret(secret_id):
    now = time.time()
    if secret_id in secret_cache:
        cached_time, secret = secret_cache[secret_id]
        if now - cached_time < CACHE_TTL:
            return secret

    # Retrieve and cache
    response = secretsmanager.get_secret_value(SecretId=secret_id)
    secret = json.loads(response['SecretString'])
    secret_cache[secret_id] = (now, secret)
    return secret

3. Use Parameter Store for non-sensitive config: Free tier (10,000 parameters)

4. Delete unused secrets: Audit regularly

# List all secrets
aws secretsmanager list-secrets --query 'SecretList[?LastAccessedDate==`null`]'

Best Practices

1. Never hardcode credentials: Use Secrets Manager everywhere
2. Enable automatic rotation: 30-90 days for database passwords
3. Least privilege IAM: Grant access to specific secrets only
4. Use KMS customer managed keys: Better audit trail and control
5. Enable CloudTrail logging: Monitor all secret access
6. Tag secrets: For cost allocation and organization
7. Regular audits: Review who has access to what
8. Cross-region replication: For critical secrets in multi-region apps
9. Version control rotation Lambda: Test changes before deploying
10. Monitor rotation failures: Alert immediately on failure

Migration from Hardcoded Secrets

Step-by-step process:

1. Audit codebase: Find all hardcoded credentials

# Search for common patterns
grep -r "password\s*=\s*['\"]" .
grep -r "api_key" .
grep -r "secret" .

2. Create secrets in Secrets Manager: One at a time, verify

3. Update code: Replace hardcoded values with Secrets Manager calls

4. Deploy and test: Staging environment first

5. Rotate original credentials: Invalidate old hardcoded values

6. Remove from code: Delete hardcoded secrets, commit

Bottom Line

Secrets Manager eliminates hardcoded credentials, provides automatic rotation for RDS/Redshift/DocumentDB, and integrates seamlessly with AWS services. Worth the $0.40/month for production secrets. Use Parameter Store for non-sensitive config. Always enable CloudTrail logging and set up rotation alarms.