Bidhan Sthapit

Logging my devops journey

4 minute read

Troubleshooting Jenkins Installation Failures on Amazon EC2: Keyrings, Logs, and Plugins

Setting up Jenkins on an Amazon EC2 instance running Amazon Linux or Ubuntu should be straightforward. However, minor configuration hurdles—like missing packages, misconfigured repositories, or GPG keyring errors—can halt your installation before it even finishes.

In this post, we will walk through common Jenkins installation friction points, how to safely inspect your logs, and how to fix failing plugin installations.

1. Finding Your Initial Admin Password

When you first access the Jenkins web interface, you are prompted for an unlock key.

Standalone Installations

If you installed Jenkins directly on the EC2 host system via a package manager, your initial key is located here:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Dockerized Installations

If you are running Jenkins inside a Docker container on your instance, the key is isolated within the container volume:

/var/jenkins_home/secrets/initialAdminPassword

2. Navigating GPG Repositories & Keyrings (Amazon Linux Focus)

If you run into repository signature validation errors during setup, you need to verify where your system stores its trusted keys and how to refresh them.

Ubuntu / Debian AMIs

Modern Debian-based systems explicitly map public keys to individual files inside the APT engine:

  • Primary Path: /etc/apt/keyrings/jenkins-keyring.asc
  • Check your repository file (/etc/apt/sources.list.d/jenkins.list) to confirm it includes the matching [signed-by=...] flag pointing to that exact keyring.

Amazon Linux / RHEL AMIs (YUM & DNF)

RedHat-based operating systems handle verification differently. The jenkins.repo file inside /etc/yum.repos.d/ does not store the key locally; it points to a remote URL:

gpgkey=https://jenkins.io

Once imported, the key is integrated directly into the system’s global RPM database rather than a plaintext folder.

🛠️ Essential Keyring & Repo Commands for Amazon Linux:

  • Inspect Your Repo Configuration: Verify that your repository configuration matches the modern Jenkins layout paths (rpm-stable instead of the legacy redhat-stable layout): 
    cat /etc/yum.repos.d/jenkins.repo
  • Verify the Imported Key in the RPM Database: Query the RPM subsystem directly to ensure the Jenkins signing authority is properly stored in the system keyring: 
    rpm -q gpg-pubkey --qf '%{NAME}-%{VERSION}-%{RELEASE}\t%{SUMMARY}\n' | grep -i jenkins
  • Manually Download and Import a Fresh Keyring: If your package manager hangs or fails to fetch the key over HTTPS automatically, manually curl and ingest the signature into the trusted store: 
    sudo rpm --import https://jenkins.io
  • Clean and Force-Update Your Repository Cache: When upgrading Jenkins or updating repository metadata, system caches can stale. Force your package manager to purge local indexes and download fresh configurations: 
    # For Amazon Linux 2 (YUM)
sudo yum clean all
sudo yum makecache

# For Amazon Linux 2023 (DNF)
sudo dnf clean all
sudo dnf check-update --repo=jenkins
  • Identify Key Signatures Explicitly: To read the cryptographic properties, expiration timelines, or user ID metadata of the Jenkins signing key directly without importing it: 
    curl -s https://jenkins.io | gpg --show-keys

⚠️ Important Upgrade Note: Jenkins periodically updates its repository signing structures. Legacy endpoints pointing to redhat-stable or older keys will cause verification blocks. Always use the updated rpm-stable or debian-stable paths.

3. Filtering and Searching Logs Like a Pro

When something breaks, your first line of defense is querying systemd via journalctl.

Target Specific Services

Avoid drowning in global log noise by isolating your target processes:

# View full history for Jenkins
sudo journalctl -u jenkins

# Follow Jenkins logs in real-time
sudo journalctl -u jenkins -f

# View combined logs for both Jenkins and Docker chronologically
sudo journalctl -u jenkins -u docker

Advance Filter Combinations

If you need to quickly locate errors that involve specific overlapping topics—such as when Jenkins plugin downloads conflict with the host environment—use advanced pipeline matching.

  • Case-Insensitive OR matching (Extended Regex): Using grep -iE allows you to find either keyword without escaping syntax characters. 
    sudo journalctl -u jenkins | grep -iE 'jenkins|docker'
  • Exact Multi-Term AND matching (Chained Pipes): If you want to pinpoint lines where both conditions must exist simultaneously (e.g., looking for Git failures within the logs): 
    sudo journalctl -u jenkins | grep -i "failed" | grep -i "git"

    (Alternatively, use grep -iE 'failed.*git|git.*failed' to capture them in any order).

4. Fixing “Suggested Plugin” Installation Failures

A common headache during the initial setup wizard is seeing the GitHub Branch Source and Pipeline: GitHub Groovy Libraries plugins fail right out of the gate.

This cascading failure typically occurs because the base Amazon Linux image lacks a local Git binary. The core GitHub plugins require host-level Git access to track repositories and map pipeline components.

The 4-Step Resolution:

  1. Bypass the Screen: Click Skip or Continue at the bottom of the wizard to finish your setup. Do not get stuck retrying on a broken environment.
  2. Install Git on the EC2 Host: Run the system package manager to install the dependency: 
    sudo dnf install git -y || sudo yum install git -y
  3. Refresh the Update Center: Navigate to Manage Jenkins ➔ Plugins ➔ Advanced settings and click Check now at the bottom right to clean out broken mirror caches.
  4. Install Manually: Navigate back to Available plugins, check the box for GitHub Branch Source, and install it. Jenkins will seamlessly resolve all missing companion libraries for you.

5. Security Check: Is ec2-user Safe for Docker?

If you are integrating Docker pipelines into your Jenkins workflows, you might be tempted to add your user to the docker group (sudo usermod -aG docker ec2-user) to get rid of constant sudo prompts.

Be careful. Because the Docker daemon runs with root privileges on the host machine, adding a standard user to the docker group gives that user passwordless root-equivalent power. If that user profile is compromised, an attacker can cleanly mount your root file system onto a container (docker run -v /:/host) and gain control over your entire EC2 node.

Best Practices:

  • Mandate explicit sudo commands for your container infrastructure execution.
  • Always enforce non-root runtime environments inside your container images (USER appuser).
  • Explore Docker Rootless Mode for strict isolation requirements in highly sensitive production architectures.

You May Also Enjoy