VPN & Internal Tools

Production AWS resources — Aurora, Redis, ClickHouse, Bull Studio, and RedisInsight — all live in private VPC subnets with no public internet access. WireGuard VPN tunnels your traffic through the Zelly bastion EC2 into the production VPC. Once connected, every private resource is reachable by its VPC private IP — no per-port SSH tunnels needed for Redis, Aurora, or ClickHouse.

Staging Aurora is publicly accessible so VPN is not required for staging database access. For everything else in production, connect first.

What lives behind the VPN

Step 1 — Install WireGuard

Do this once per machine. Pick your OS:

brew install wireguard-tools

wg --version   # verify

sudo apt update && sudo apt install -y wireguard wireguard-tools
wg --version

sudo dnf install -y wireguard-tools
wg --version

sudo pacman -S wireguard-tools

Step 2 — Create your VPN configuration

Each developer has a unique key pair and an assigned VPN IP in the 10.10.0.0/24 subnet. You generate your own private key — it never leaves your machine.

1. Generate a key pair

   bash / macOS / Linux PowerShell / Windows

   bash

   copy

   wg genkey | tee private.key | wg pubkey > public.key
   
   echo "Private key (keep secret):" && cat private.key
   echo ""
   echo "Public key (share with DevOps):" && cat public.key

   PowerShell — use Git Bash or WSL for key generation

   copy

   # Open Git Bash (ships with Git for Windows) or WSL and run:
   wg genkey | tee private.key | wg pubkey > public.key
   
   cat private.key   # goes in your config file — never share
   cat public.key    # share this with DevOps

   !

   Never share your private key. If it leaks, revoke it immediately — see Troubleshooting for the revocation steps.
2. Get the bastion's server public key and public IP

   The bastion stores its own public key in SSM Parameter Store on first boot. Retrieve it:

   bash PowerShell

   bash — production (ap-south-1)

   copy

   # Server public key
   SERVER_PUBKEY=$(aws ssm get-parameter \
     --name /zelly/bastion/wireguard-public-key \
     --region ap-south-1 \
     --query Parameter.Value --output text)
   
   # Bastion public IP
   BASTION_IP=$(aws ec2 describe-instances \
     --filters "Name=tag:Name,Values=zelly-production-bastion" \
               "Name=instance-state-name,Values=running" \
     --region ap-south-1 \
     --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
   
   echo "Server public key: $SERVER_PUBKEY"
   echo "Bastion IP:        $BASTION_IP"

   PowerShell — production

   copy

   $SERVER_PUBKEY = aws ssm get-parameter `
     --name /zelly/bastion/wireguard-public-key `
     --region ap-south-1 `
     --query Parameter.Value --output text
   
   $BASTION_IP = aws ec2 describe-instances `
     --filters "Name=tag:Name,Values=zelly-production-bastion" `
               "Name=instance-state-name,Values=running" `
     --region ap-south-1 `
     --query "Reservations[0].Instances[0].PublicIpAddress" --output text
   
   Write-Host "Server public key: $SERVER_PUBKEY"
   Write-Host "Bastion IP:        $BASTION_IP"

3. Register your public key

   Share your public key (from public.key) and your name with a team lead. You'll be assigned a VPN IP — for example, 10.10.0.3. The bastion config is updated via SSM RunCommand (no SSH required).

4. Write your config file

   Create zelly-prod.conf (anywhere on your machine). Fill in the three placeholders:

   zelly-prod.conf

   copy

   [Interface]
   PrivateKey = <paste your private.key contents here>
   Address    = 10.10.0.<YOUR_ASSIGNED_NUMBER>/32
   
   [Peer]
   # Zelly production bastion
   PublicKey           = <server public key from Step 2>
   Endpoint            = <bastion public IP>:51820
   AllowedIPs          = 10.0.0.0/8
   PersistentKeepalive = 25

   AllowedIPs = 10.0.0.0/8 is a split tunnel — only 10.x.x.x traffic routes through the VPN. Your regular internet connection is unaffected. The VPC production CIDR is 10.0.0.0/16 and staging is 10.1.0.0/16 — both are covered by 10.0.0.0/8.

Step 3 — Connect to the VPN

# Connect (sudo required)
sudo wg-quick up ~/zelly-prod.conf

# Check handshake — confirm a recent handshake time
sudo wg show

# Test VPC connectivity
ping 10.10.0.1   # bastion VPN interface

# Disconnect when done
sudo wg-quick down ~/zelly-prod.conf

Step 4 — Access resources

Keep the VPN active for all commands in this section.

Redis (ElastiCache)

Production ElastiCache requires TLS — plain-text connections are rejected. Get credentials from Secrets Manager, then connect:

# Pull credentials
SECRET=$(aws secretsmanager get-secret-value \
  --secret-id zelly/redis/auth --region ap-south-1 \
  --query SecretString --output text)

HOST=$(echo "$SECRET" | jq -r .host)
TOKEN=$(echo "$SECRET" | jq -r .auth_token)

# Interactive CLI session
redis-cli -h "$HOST" -p 6379 --tls --no-auth-warning -a "$TOKEN"

# One-shot health check
redis-cli -h "$HOST" -p 6379 --tls -a "$TOKEN" PING

# Inspect BullMQ queues
redis-cli -h "$HOST" -p 6379 --tls -a "$TOKEN" KEYS "bull:*" | head -20
redis-cli -h "$HOST" -p 6379 --tls -a "$TOKEN" LLEN "bull:store-events:wait"
redis-cli -h "$HOST" -p 6379 --tls -a "$TOKEN" LLEN "bull:SHOPIFY_WEBHOOK:wait"

$r = aws secretsmanager get-secret-value `
  --secret-id zelly/redis/auth --region ap-south-1 `
  --query SecretString --output text | ConvertFrom-Json

# Connect with redis-cli (install via choco install redis-desktop-manager or WSL)
redis-cli -h $r.host -p 6379 --tls --no-auth-warning -a $r.auth_token

# Or display for use in a GUI client like RedisInsight
Write-Host "Host:  $($r.host)"
Write-Host "Port:  6379 (TLS on)"
Write-Host "Token: $($r.auth_token)"

Aurora MySQL (production)

Staging Aurora is publicly accessible — no VPN needed. For production, connect via VPN first.

SECRET=$(aws secretsmanager get-secret-value \
  --secret-id zelly/aurora/master --region ap-south-1 \
  --query SecretString --output text)

HOST=$(echo "$SECRET" | jq -r .host)
PASS=$(echo "$SECRET" | jq -r .password)

mysql -h "$HOST" -u zellymaster -p"$PASS" astro_primary

SECRET=$(aws secretsmanager get-secret-value \
  --secret-id zelly/aurora/master --region ap-southeast-1 \
  --query SecretString --output text)

HOST=$(echo "$SECRET" | jq -r .host)
PASS=$(echo "$SECRET" | jq -r .password)

mysql -h "$HOST" -u zellymaster -p"$PASS" astro_primary

$s = aws secretsmanager get-secret-value `
  --secret-id zelly/aurora/master --region ap-south-1 `
  --query SecretString --output text | ConvertFrom-Json

Write-Host "Host: $($s.host)"
Write-Host "User: zellymaster"
Write-Host "Pass: $($s.password)"
# Paste these into MySQL Workbench, TablePlus, or your preferred GUI client

ClickHouse

ClickHouse runs on an EC2 instance (t3.medium, Docker) in the private subnet. The host is stored in the zelly/clickhouse/env secret. ClickHouse exposes two interfaces — HTTP (port 8123) and the native TCP protocol (port 9000).

CH=$(aws secretsmanager get-secret-value \
  --secret-id zelly/clickhouse/env --region ap-south-1 \
  --query SecretString --output text)

CH_HOST=$(echo "$CH" | jq -r .host)
CH_PASS=$(echo "$CH" | jq -r .password)

# Ping the HTTP interface — should return "Ok."
curl "http://$CH_HOST:8123/ping"

# Run a query via HTTP (quickest for one-off checks)
curl "http://$CH_HOST:8123/?user=default&password=$CH_PASS" \
  --data "SELECT version()"

# Show databases
curl "http://$CH_HOST:8123/?user=default&password=$CH_PASS" \
  --data "SHOW DATABASES FORMAT Pretty"

# Interactive shell via native protocol (install clickhouse-client first)
clickhouse-client --host "$CH_HOST" --password "$CH_PASS" --port 9000

$ch = aws secretsmanager get-secret-value `
  --secret-id zelly/clickhouse/env --region ap-south-1 `
  --query SecretString --output text | ConvertFrom-Json

# Ping
Invoke-RestMethod "http://$($ch.host):8123/ping"

# Query via HTTP API
Invoke-RestMethod "http://$($ch.host):8123/?user=default&password=$($ch.password)" `
  -Method Post -Body "SELECT version()"

# Show databases
Invoke-RestMethod "http://$($ch.host):8123/?user=default&password=$($ch.password)" `
  -Method Post -Body "SHOW DATABASES"

Internal tools — direct VPN access

Bull Studio and RedisInsight are ECS Fargate tasks in private subnets. Their security group allows inbound on ports 3000 and 5540 from the bastion security group — WireGuard traffic is masqueraded through the bastion's private IP, so once the VPN is connected you can open these tools directly in a browser.

Bull Studio (BullMQ queue dashboard)

Shows all BullMQ queues — job counts, active workers, failed jobs, and retry history. Runs on port 3000.

1. Connect WireGuard (Step 3 above — must be active).
2. Look up the task's private IP

   bash PowerShell

   bash — production

   copy

   TASK=$(aws ecs list-tasks \
     --cluster zelly-production \
     --service-name bull-studio \
     --region ap-south-1 \
     --query 'taskArns[0]' --output text)
   
   aws ecs describe-tasks \
     --cluster zelly-production \
     --tasks "$TASK" \
     --region ap-south-1 \
     --query 'tasks[0].attachments[0].details[?name==`privateIPv4Address`].value' \
     --output text

   PowerShell — production

   copy

   $TASK = aws ecs list-tasks `
     --cluster zelly-production --service-name bull-studio `
     --region ap-south-1 --query "taskArns[0]" --output text
   
   aws ecs describe-tasks `
     --cluster zelly-production --tasks $TASK `
     --region ap-south-1 `
     --query "tasks[0].attachments[0].details[?name=='privateIPv4Address'].value" `
     --output text

3. Open in your browser

   Navigate to http://<task-private-ip>:3000 — you'll see all BullMQ queues with live job counts. For example: http://10.0.2.47:3000

RedisInsight (Redis GUI)

Visual key browser, stream viewer, and memory analysis for ElastiCache. Runs on port 5540. The Redis connection is pre-configured via REDIS_HOSTS — no manual setup in the UI.

1. Connect WireGuard (must be active).
2. Look up the task's private IP

   bash PowerShell

   bash — production

   copy

   TASK=$(aws ecs list-tasks \
     --cluster zelly-production \
     --service-name redis-insight \
     --region ap-south-1 \
     --query 'taskArns[0]' --output text)
   
   aws ecs describe-tasks \
     --cluster zelly-production \
     --tasks "$TASK" \
     --region ap-south-1 \
     --query 'tasks[0].attachments[0].details[?name==`privateIPv4Address`].value' \
     --output text

   PowerShell

   copy

   $TASK = aws ecs list-tasks `
     --cluster zelly-production --service-name redis-insight `
     --region ap-south-1 --query "taskArns[0]" --output text
   
   aws ecs describe-tasks `
     --cluster zelly-production --tasks $TASK `
     --region ap-south-1 `
     --query "tasks[0].attachments[0].details[?name=='privateIPv4Address'].value" `
     --output text

3. Open in your browser

   Navigate to http://<task-private-ip>:5540. The ElastiCache database will already appear in the sidebar — click it to start browsing keys.

Look up all internal tool IPs at once

for SVC in bull-studio redis-insight; do
  TASK=$(aws ecs list-tasks --cluster zelly-production --service-name $SVC \
    --region ap-south-1 --query 'taskArns[0]' --output text)
  IP=$(aws ecs describe-tasks --cluster zelly-production --tasks "$TASK" \
    --region ap-south-1 \
    --query 'tasks[0].attachments[0].details[?name==`privateIPv4Address`].value' \
    --output text)
  echo "$SVC → $IP"
done

Troubleshooting

sudo wg show   # look for "latest handshake" — should be recent

redis-cli -h "$HOST" -p 6379 --tls --no-auth-warning -a "$TOKEN"

# Ubuntu / Debian
sudo apt install redis-tools

# macOS
brew install redis   # Homebrew builds with TLS

# Verify TLS support
redis-cli --tls --version

aws ecs describe-services \
  --cluster zelly-production \
  --services bull-studio redis-insight \
  --region ap-south-1 \
  --query 'services[*].{name:serviceName,running:runningCount,desired:desiredCount}' \
  --output table