Tech

Automating Let's Encrypt with Certbot and TransIP using Docker

Want to auto update/manage your SSL certificates inside your Docker with Certbot and hosting your domain with TransIP? Then look no further!

22 days ago • 7 min read

By Remco Loup.

Managing SSL certificates for domains hosted on TransIP can be cumbersome without automation. TransIP luckily offers a robust API for DNS management, but to integrate it seamlessly with Certbot for dns-01 validation, a custom solution is necessary. Since I couldn’t find a ready-made script for this purpose, I created one that securely handles authentication, DNS updates, and certificate issuance.

You can go to the full API documentation here:

TransIP API

This guide explains the full setup using Docker, including how to generate the API key pair in TransIP’s web interface and use it for automated certificate management.

How TransIP authentication works

TransIP uses an RSA key pair for API authentication. Here’s how to set it up:

Generate the Key Pair in TransIP:
- Log in to your TransIP control panel.
- Go to API Settings under your account and enable API access.
- Generate a new API key pair.
- Download/Save the private key (transip_private_key.pem) securely and create a secret from it to use in your Docker setup.

docker secret create transip_private_key transip_private_key.pem

Keep the Private Key Secure:
- Store the private key somewhere safe after creating a secret from it.
- This private key is needed to sign requests to TransIP’s API.
How It Works:
- The script uses the private key to sign an authentication request.
- TransIP’s API responds with a short-lived token (10 minutes by default), which is then used to manage the DNS records.
- It will create a new token for convenience on every run. This made it more resilient then checking for an active token and using that which made the whole script less resilient and cumbersome.

What this setup is going to do

Automates SSL Certificates:
- Automates issuance and renewal of wildcard and domain-specific certificates.
Handles DNS Challenges:
- Manages dns-01 challenges by creating and deleting DNS TXT records via the TransIP API.
Seamless Integration:
- Encapsulates everything in a Docker container for portability and simplicity.
- You can remove or add domains in this script on the -d rule part. I put the examples in for a better explanation.

We will pipe a add or clean command to the bash script to take the necessary actions as defined in the script. So we use the --manual-auth-hook and the manual-cleanup-hook here. This makes sure that a fail in the add section still cleans up after itself for example.

You can also add a --force to this script. but your will run out of attempts to Letsencrypt quickly since they only allow you to do a few calls every 24 hours. But this will make sure it refreshes the certificates always when you need to test or verify something.

version: "3.8"

services:
  certbot:
    image: certbot/certbot:latest
    container_name: certbot-transip
    secrets:
      - transip_private_key  # Reference the secret you created earlier
    # adjust these volumes to point to the right .pem and script. right now this dockerfile expects the files to be in the same folder as the docker-certbot-transip.yml
    volumes:
      - "./certbot-transip.sh:/usr/local/bin/certbot-transip.sh:ro"
      - "./letsencrypt:/etc/letsencrypt"
    environment:
    #These Variables are going to be piped in our bash script.
      LOGIN_NAME: "your-transip-login"        # Replace with your TransIP login name
      PRIVATE_KEY_PATH: "/etc/transip_private_key.pem"
      DOMAIN: "example.com"                  # Replace with your domain
      TTL: "60"                              # Optional: Time-to-live for DNS challenge
      CERTBOT_EMAIL: "you@example.com"       # Replace with your valid email as requested by letsencrypt.
    entrypoint:
      - sh
      - -c
      - |
        apk update && apk add --no-cache bash curl jq bind-tools && \
        chmod +x /usr/local/bin/certbot-transip.sh && \
        certbot certonly --manual --preferred-challenges dns \
          --manual-auth-hook "/usr/local/bin/certbot-transip.sh add" \
          --manual-cleanup-hook "/usr/local/bin/certbot-transip.sh clean" \
          --cert-name "$DOMAIN" \
          -d "*.$DOMAIN" \
          -d "example2.$DOMAIN" \
          -d "example3.$DOMAIN" \
          --expand \
          --non-interactive --agree-tos \
          --email "$CERTBOT_EMAIL"

secrets:
  transip_private_key:
    external: true  # Secret must already exist in Docker

The Bash script

This is the "advanced" part that will do its magic and make everything work with TransIP. I tried to make it a bit more resilient then standard also. So this should hopefully help in creating a resilient and good working environment.

You can also use this TransIP script to run it locally creating certificates or in a Kubectl or Helm chart.

#!/bin/bash

# Configuration settings
LOGIN_NAME=${LOGIN_NAME}
TTL=${TTL:-60}  # Default value 60 if TTL is not set
PRIVATE_KEY_PATH=${PRIVATE_KEY_PATH}
DOMAIN=${DOMAIN}

AUTH_URL="https://api.transip.nl/v6/auth"
LOG_FILE="/var/log/transip_dyndns.log"

# Function to update a log file
#log_message() {
#    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" >> $LOG_FILE
#}

# Function to log messages to stdout
log_message() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1"
}

# Function to log an error and stop the script
handle_error() {
    log_message "ERROR: $1"
    echo "ERROR: $1" >&2
    exit 1
}

# Function to obtain a new access token with a short validity period (10 minutes)
generate_new_token() {
    log_message "Generating a new token with a validity period of 10 minutes."

    NONCE=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 16)
    TIMESTAMP=$(date +%s)
    LABEL="Certbot DNS update $TIMESTAMP"

    REQUEST_BODY=$(printf '{"login":"%s","nonce":"%s","read_only":false,"expiration_time":"10 minutes","label":"%s","global_key":true}' "$LOGIN_NAME" "$NONCE" "$LABEL")

    log_message "Request Body: $REQUEST_BODY"

    SIGNATURE=$(echo -n "$REQUEST_BODY" | openssl dgst -sha512 -sign "$PRIVATE_KEY_PATH" | base64 | tr -d '\n')

    log_message "Generated signature: $SIGNATURE"

    RESPONSE=$(curl -s -X POST "$AUTH_URL" \
        -H "Signature: $SIGNATURE" \
        -H "Content-Type: application/json" \
        -d "$REQUEST_BODY") || handle_error "Failed to connect to TransIP API"

    log_message "API-response received: $RESPONSE"

    ACCESS_TOKEN=$(echo "$RESPONSE" | jq -r '.token // empty')
    if [ -z "$ACCESS_TOKEN" ]; then
        handle_error "The received access token is empty or invalid. Check the API-response: $RESPONSE"
    fi

    log_message "Received token: $ACCESS_TOKEN"
}

# Function to add or update a TXT record
add_or_update_txt_record() {
    local full_domain="$1"
    local value="$2"

    # Extract the subdomain part from the full domain
    local name="_acme-challenge.${full_domain%%.*}"

    if [ -z "$name" ] || [ -z "$value" ]; then
        log_message "ERROR: Name or value for TXT record is missing."
        exit 1
    fi

    # Check if the DNS record already exists
    RESPONSE=$(curl -s -X GET \
        -H "Authorization: Bearer $ACCESS_TOKEN" \
        "https://api.transip.nl/v6/domains/$DOMAIN/dns")

    # Search for all existing records with the same name and type TXT
    MATCHING_RECORDS=$(echo "$RESPONSE" | jq -r --arg name "$name" '[.dnsEntries[] | select(.name==$name and .type=="TXT")]')

    # Check if there are multiple or no matching records
    MATCHING_COUNT=$(echo "$MATCHING_RECORDS" | jq 'length')

    # Ensure MATCHING_COUNT is always an integer
    if ! [[ "$MATCHING_COUNT" =~ ^[0-9]+$ ]]; then
        MATCHING_COUNT=0
    fi

    if [ "$MATCHING_COUNT" -eq 0 ]; then
        log_message "No existing TXT record found for $name. Creating a new record."
    elif [ "$MATCHING_COUNT" -ge 1 ]; then
        log_message "Multiple or one existing TXT record found for $name. Removing all existing records."

        # Remove all existing records using the full details of each record
        echo "$MATCHING_RECORDS" | jq -c '.[]' | while read -r record; do
            existing_name=$(echo "$record" | jq -r '.name')
            existing_expire=$(echo "$record" | jq -r '.expire')
            existing_content=$(echo "$record" | jq -r '.content')

            # Log the details of the record to be removed
            log_message "Remove record: name=$existing_name, expire=$existing_expire, content=$existing_content"

            # Make the DELETE call using the full DNS-entry details
            DELETE_ENTRY=$(jq -n \
            --arg name "$existing_name" \
            --arg expire "$existing_expire" \
            --arg type "TXT" \
            --arg content "$existing_content" \
            '{"dnsEntry": {"name": $name, "expire": ($expire | tonumber), "type": $type, "content": $content}}')

            DELETE_RESPONSE=$(curl -s -X DELETE \
                -H "Authorization: Bearer $ACCESS_TOKEN" \
                -H "Content-Type: application/json" \
                -d "$DELETE_ENTRY" \
                "https://api.transip.nl/v6/domains/$DOMAIN/dns")

            # Log the response of the DELETE call
            log_message "Response after removing $existing_name: $DELETE_RESPONSE"

            # Check if the removal was successful
            if [[ "$DELETE_RESPONSE" == *"error"* ]]; then
                log_message "ERROR: The record $existing_name could not be removed."
            else
                log_message "Record $existing_name successfully removed."
            fi
        done
    fi

    # Create a new record
    DNS_ENTRY=$(jq -n \
    --arg name "$name" \
    --arg expire "$TTL" \
    --arg type "TXT" \
    --arg content "$value" \
    '{"dnsEntry": {"name": $name, "expire": ($expire | tonumber), "type": $type, "content": $content}}')

    RESPONSE=$(curl -s -X POST \
        -H "Authorization: Bearer $ACCESS_TOKEN" \
        -H "Content-Type: application/json" \
        -d "$DNS_ENTRY" \
        "https://api.transip.nl/v6/domains/$DOMAIN/dns")

    log_message "API-response for TXT record creation: $RESPONSE"

    if [[ "$RESPONSE" == *"error"* ]]; then
        handle_error "The TXT record could not be created. Check the API-response: $RESPONSE"
    else
        log_message "TXT record successfully added for $name with value $value"
    fi
}

# Function to update an existing TXT record
update_txt_record() {
    local name="$1"
    local value="$2"

    DNS_ENTRY=$(jq -n \
    --arg name "$name" \
    --arg expire "$TTL" \
    --arg type "TXT" \
    --arg content "$value" \
    '{"dnsEntry": {"name": $name, "expire": ($expire | tonumber), "type": $type, "content": $content}}')

    RESPONSE=$(curl -s -X PATCH \
        -H "Authorization: Bearer $ACCESS_TOKEN" \
        -H "Content-Type: application/json" \
        -d "$DNS_ENTRY" \
        "https://api.transip.nl/v6/domains/$DOMAIN/dns")

    log_message "API-response for TXT record update: $RESPONSE"

    if [[ "$RESPONSE" == *"error"* ]]; then
        handle_error "The TXT record could not be updated. Check the API-response: $RESPONSE"
    else
        log_message "TXT record successfully updated for $name with value $value"
    fi
}

# Function to remove a TXT record for `dns-01` validation
remove_txt_record() {
    local full_domain="$1"

    # Extract the subdomain part from the full domain
    local name="_acme-challenge.${full_domain%%.*}"

    # Check if the DNS record exists
    RESPONSE=$(curl -s -X GET \
        -H "Authorization: Bearer $ACCESS_TOKEN" \
        "https://api.transip.nl/v6/domains/$DOMAIN/dns")

    # Search for all records with the same name and type TXT
    MATCHING_RECORDS=$(echo "$RESPONSE" | jq -r --arg name "$name" '[.dnsEntries[] | select(.name==$name and .type=="TXT")]')

    # Check if there are existing records that need to be removed
    MATCHING_COUNT=$(echo "$MATCHING_RECORDS" | jq 'length')

    if [ "$MATCHING_COUNT" -eq 0 ]; then
        log_message "No existing TXT record found for $name to remove."
    else
        log_message "Removing all existing TXT records for $name."

        # Loop through all found records and remove them
        echo "$MATCHING_RECORDS" | jq -c '.[]' | while read -r record; do
            existing_name=$(echo "$record" | jq -r '.name')
            existing_expire=$(echo "$record" | jq -r '.expire')
            existing_content=$(echo "$record" | jq -r '.content')

            # Make the DELETE call using the full details
            DELETE_ENTRY=$(jq -n \
            --arg name "$existing_name" \
            --arg expire "$existing_expire" \
            --arg type "TXT" \
            --arg content "$existing_content" \
            '{"dnsEntry": {"name": $name, "expire": ($expire | tonumber), "type": $type, "content": $content}}')

            DELETE_RESPONSE=$(curl -s -X DELETE \
                -H "Authorization: Bearer $ACCESS_TOKEN" \
                -H "Content-Type: application/json" \
                -d "$DELETE_ENTRY" \
                "https://api.transip.nl/v6/domains/$DOMAIN/dns")

            # Check if the removal was successful
            if [[ "$DELETE_RESPONSE" == *"error"* ]]; then
                log_message "ERROR: The TXT record $existing_name could not be removed."
            else
                log_message "TXT record $existing_name successfully removed."
            fi
        done
    fi
}

# Always generate a new token for each run
generate_new_token

# Certbot passes these values when adding and removing the record
if [ "$1" == "add" ]; then
    add_or_update_txt_record "$CERTBOT_DOMAIN" "$CERTBOT_VALIDATION"
    sleep 25  # Wait for DNS update to propagate
elif [ "$1" == "clean" ]; then
    remove_txt_record "$CERTBOT_DOMAIN"
fi

Steps to run

Download the Private Key: Save the private key file (transip_private_key.pem) from TransIP’s API settings somewhere on your device you are going to run this from.
Set Up Docker Compose: Place the docker-compose.yml and certbot-transip.sh files in the same directory. (or edit the script)
Run the Container: Start the container (cd to the path where your yml is)

docker-compose up --build

Monitor the Log output: The log output of the docker container will show the progress of DNS challenges and certificate issuance.

Thats all there is to it!

I will also post a version with a working helm chart soon. This was a little bigger culprit to make sure it all works correctly and updates all certificates through different namespaces where i needed the certificates to be available on different domains.

Automating Let's Encrypt with Certbot and TransIP using Docker

How TransIP authentication works

What this setup is going to do

The Bash script

Steps to run

Stay Connected with a dynamic DNS updater for TransIP

Supercharge Your headless (Raspberry Pi) VPN with ProtonVPN and WireGuard

Keep reading

How TransIP authentication works

What this setup is going to do

The Bash script

Steps to run

Spread the word

Stay Connected with a dynamic DNS updater for TransIP

Supercharge Your headless (Raspberry Pi) VPN with ProtonVPN and WireGuard

Keep reading

Stay Connected with a dynamic DNS updater for TransIP

Supercharge Your headless (Raspberry Pi) VPN with ProtonVPN and WireGuard

Azure + MySQL, MariaDB, PostgreSQL performance compared