Case Study: Right-Sizing Cloud Resources

Case Study: Right-Sizing Cloud Resources by Profile

The Problem

A client was running many pipeline jobs on an initial hardware selection that was never verified. The hardware was chosen during a proof-of-concept phase and never re-evaluated against the actual workload. Time and cloud-optimization know-how were not available to the teams.

The Investigation

To avoid optimizing for build steps that did not impact overall execution times I first narrowed down to build phases that took the most time and began profiling them. The profiling quickly identified the slow processes, but since these commercial products did not provide public symbols for tracing further down, I had to draw the optimization boundary at the process level. The trace showed that processes were not affected by outside events, and their internal structure was not observable.

Root Cause

The trace clearly showed that the process was heavily utilizing a single core. It was not memory, network, or IO bound, it was simply doing heavy lifting on very few cores, despite having access to up to 16 cores on the given hardware. Since the client did not own the component and there was no trivial way of adding concurrency at these stages, the fix had to be applied at a higher level.

The Fix

The client was paying for 16 cores and substantial RAM while using almost none of it. I applied a right-sized hardware configuration to these pipelines, which improved execution time by 17% across the profiled set. A second gain was that this hardware configuration was 66% cheaper than the previous one.

Result

• Build duration: 17% faster, averaging 25 minutes saved per pipeline run
• Cost: 66% lower cloud spend per virtual machine
• Scope: 16 pipelines benefited from this time and cost improvement
• Total timeline: 3 days from engagement start to verified result