My GitLab CI Pipeline Failed… Then Worked Without Changing Anything
When My GitLab CI Pipeline Failed Because of a Debian Mirror Issue (And Fixed Itself)
While working as a Junior DevOps Engineer, I recently faced an interesting CI/CD pipeline issue for the first time.
My GitLab pipeline failed during a Docker build with an error that looked like a package problem.
But surprisingly, rerunning the pipeline fixed it instantly.
This experience helped me understand something important about Debian package mirrors and CI environments.
Let’s break down what happened.
The Situation
My GitLab CI pipeline was building a Docker image for a Python application.
The Dockerfile had a standard dependency installation step:
FROM python:3.8-slim-bullseye
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
build-essential \
libpq-dev \
openssh-client \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
This step installs system dependencies required by the application.
Everything looked perfectly normal.
But during the pipeline execution, the build suddenly failed.
The Error I Saw
The GitLab pipeline logs showed this error:
E: Failed to fetch http://deb.debian.org/debian/pool/main/g/gcc-10/g++-10_10.2.1-6_amd64.deb
Error reading from server - read (104: Connection reset by peer)
E: Unable to fetch some archives
The job failed with:
exit code: 100
At first, it looked like:
• a Dockerfile problem
• a broken package
• or dependency conflict
But none of those were actually the issue.
Understanding the Real Cause
The key line was:
Connection reset by peer
This means the server closed the connection while the package was being downloaded.
In my case, the connection to the Debian package mirror was interrupted.
What is a Debian Mirror? (Simple Explanation)
When I first saw the term mirror, I was confused because it sounded like a technical term.
But the concept is actually very simple.
A mirror is just a copy of a software repository stored on another server.
Debian packages are stored in a repository, and instead of hosting everything on one single server, Debian keeps many identical copies of the repository across different servers around the world.
These copies are called mirrors.
All mirrors contain the same packages, but they exist in multiple locations.
A Simple Real-World Analogy
Imagine a popular bookstore.
If there is only one bookstore in the world, everyone would have to go there to buy books.
That store would become overloaded.
So instead, the company opens many branches of the same bookstore in different cities.
Each branch contains the same books, but people can go to the closest store.
Debian mirrors work exactly the same way.
Instead of downloading packages from one central server, your system downloads them from one of many mirror servers.
Example of Debian Mirrors
When we run:
apt-get update
The system connects to a repository like this:
http://deb.debian.org/debian
But this is not a single physical server.
It redirects your request to one of many mirrors such as:
mirror.us.debian.org
mirror.sg.debian.org
mirror.debian.org
mirror.in.debian.org
All these mirrors contain the same packages, but they help distribute traffic across the internet.
Why Mirrors Exist
Mirrors exist to improve:
1. Download Speed
Users download packages from the closest mirror server, reducing latency.
2. Reliability
If one server goes down, another mirror can serve the same packages.
3. Load Balancing
Millions of developers use Debian repositories daily. Mirrors distribute the traffic across multiple servers.
4. Global Access
Developers around the world can download packages from nearby servers.
Why Mirror Issues Sometimes Happen
Although mirrors improve reliability, they can still experience temporary issues.
For example:
• network interruptions
• server restarts
• overloaded mirror servers
• temporary connection resets
• bandwidth throttling
When this happens, package downloads may fail with errors like:
Failed to fetch package
Connection reset by peer
Unable to fetch archives
This is exactly what happened in my pipeline.
The Interesting Part
Instead of changing anything in the code or Dockerfile, I simply reran the pipeline.
And the build worked perfectly.
That confirmed the issue was not related to the Dockerfile or CI configuration, but rather a temporary network or mirror issue.
How to Troubleshoot This Type of Error
When a CI pipeline fails during package installation, follow a structured troubleshooting process.
Step 1 — Identify Where the Pipeline Failed
First, locate the exact step that failed.
In my pipeline it failed during:
apt-get install
This immediately indicates the failure is related to package installation.
Step 2 — Look for Network-Related Keywords
Search the logs for keywords such as:
Failed to fetch
Connection reset
Unable to fetch archives
Temporary failure resolving
These usually indicate network or mirror related issues.
Step 3 — Look at the Exit Code
APT usually fails with:
exit code: 100
This indicates a package installation failure, not a Docker or GitLab runner issue.
Step 4 — Check Which Package Failed
The logs often show the exact package that failed.
Example:
g++-10_10.2.1-6_amd64.deb
This confirms that the failure occurred during package download, not during compilation.
Step 5 — Rerun the Pipeline
If rerunning the pipeline succeeds without changes, it strongly suggests:
• a temporary mirror issue
• a network interruption
This is exactly what happened in my case.
How to Prevent This Issue
One simple improvement is using the --fix-missing flag.
Example:
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
build-essential \
libpq-dev \
openssh-client \
--fix-missing \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
This allows apt to retry missing downloads instead of failing immediately.
Another Preventive Option
Some teams add retry logic:
RUN apt-get update || apt-get update
This ensures the update command retries if the first attempt fails.
A Small Optimization Tip
If you install:
build-essential
you do not need to install gcc separately, because it is already included.
Key Lessons from This Issue
This experience taught me a few useful DevOps lessons:
Not all pipeline failures are caused by code.
External services like package repositories can fail temporarily.
Reading logs carefully helps identify the real issue.
Making pipelines resilient improves reliability.
Sometimes the simplest solution is:
Rerun the pipeline and observe the behavior.
Final Thoughts
This was the first time I encountered a Debian mirror-related issue in CI/CD.
Although the pipeline failed initially, it helped me better understand:
• how package repositories work
• why mirrors exist
• how network issues affect CI pipelines
These real-world debugging experiences gradually build stronger DevOps troubleshooting skills.
And sometimes the biggest lesson is simple:
Not every failure is caused by your code. Sometimes the infrastructure just needs another try. 🚀




