Day 06: Jenkins CI/CD Infrastructure Setup & Troubleshooting

2 minute read

Jenkins CI/CD Infrastructure Setup & Troubleshooting

Domain: merojenkins.duckdns.org
Environment: AWS EC2 (Amazon Linux/RHEL)
Stack: Nginx (Reverse Proxy), Jenkins (Core), Certbot (SSL)

1. Network & Firewall Configuration

Before services can communicate, the AWS “Virtual” firewall and OS-level firewall must be aligned.

AWS Security Group (Inbound Rules)

Protocol	Port	Source	Purpose
SSH	22	Your_IP	Secure Management
HTTP	80	0.0.0.0/0	Certbot Validation / Web Traffic
HTTPS	443	0.0.0.0/0	Secure Web Traffic

OS Firewall (Firewalld)

# Open ports for Nginx
sudo firewall-cmd --permanent --add-service=http
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload

2. DNS Resolution & Management

The site failed to resolve due to DNS Flapping and an outdated IP on the DuckDNS dashboard.

Commands to verify and fix:

# Check global resolution via Google DNS
nslookup merojenkins.duckdns.org 8.8.8.8

# Flush local DNS cache if mismatch occurs
sudo resolvectl flush-caches

# Check which IP your server actually has
curl -s http://amazonaws.com

3. Jenkins Security Hardening

To ensure Jenkins is only accessible via the Nginx proxy (and not directly on 8080), we bind it to the local loopback.

Implementation via Systemd:

# Create/Edit override file
sudo systemctl edit jenkins

# Add the following content:
[Service]
Environment="JENKINS_LISTEN_ADDRESS=127.0.0.1"

# Reload and Restart
sudo systemctl daemon-reload
sudo systemctl restart jenkins

4. Nginx Reverse Proxy Setup

Configured Nginx to handle SSL termination and forward requests to the local Jenkins instance.

Config Path: /etc/nginx/conf.d/jenkins.conf

upstream jenkins {
    keepalive 32;
    server 127.0.0.1:8080;
}

server {
    listen 80;
    server_name merojenkins.duckdns.org;

    location / {
        proxy_pass http://jenkins;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_buffering off;
    }
}

5. SSL Implementation (Certbot)

Securing traffic with Let’s Encrypt certificates.

Method A: Automated (Nginx Plugin)

sudo certbot --nginx -d merojenkins.duckdns.org

Method B: Standalone (Use if DNS/Nginx issues persist)

sudo systemctl stop nginx
sudo certbot certonly --standalone -d merojenkins.duckdns.org
sudo systemctl start nginx

6. Log Management & Rotation

To prevent the EC2 disk from filling up, we implement rotation for Jenkins logs.

Config Path: /etc/logrotate.d/jenkins

/var/log/jenkins/jenkins.log {
    weekly
    copytruncate
    rotate 10
    compress
    missingok
    notifempty
}

7. Operational Cleanup

If a terminal session crashes during configuration, clean up the swap files to prevent “Found a swap file” errors.

# Remove Nginx config swap file
rm .jenkins.conf.swp

🔑 Key Takeaways

DNS Mismatch: Always verify nslookup before running Certbot; if the IP doesn’t match the server, the challenge will fail.
Reverse Proxy Security: Binding Jenkins to 127.0.0.1 ensures that users cannot bypass your firewall/SSL by typing :8080 in their browser.
AWS Layer: Remember that AWS Security Groups act as a silent wall outside the server—OS-level changes mean nothing if the SG is closed.

Bidhan Sthapit