Skip to content

AWS Graviton ARM64 Migration

Moving container workloads from x86 to ARM64 without a big bang

Reference architectureImplemented & validated locally

Basis: Reference design. Relates to ARM64/Graviton ECS Fargate work shown on this site; exact production scope to be confirmed before it is presented as a case study. The published code is a reference implementation of that pattern — services are synthetic and nothing has run on real AWS.

Overview

A staged migration from AMD64 to Graviton: multi-architecture images, ARM64 build agents, mixed node groups, compatibility testing and a rollback path that is one scheduling change away.

Problem

Graviton instances are usually cheaper per unit of work, but a migration fails on details: native dependencies without ARM64 builds, base images that are single-arch, and CI agents that can only build for x86.

Solution

Build every image as a multi-arch manifest first, then move workloads one at a time by node affinity. Because the same image tag runs on both architectures, rollback is removing the ARM64 affinity, not rebuilding.

Architecture

  1. Build

    • Jenkins

      amd64 + arm64 agents

    • docker buildx

      linux/amd64,linux/arm64

  2. Registry

    • ECR

      manifest list per tag

  3. Cluster

    • x86 node group

      m6i / c6i

    • Graviton node group

      m7g / c7g

  4. Validation

    • Benchmarks

      same load, both archs

    • Canary

      affinity-based rollout

One multi-arch tag, two node pools, scheduling decides the architecture.

Technology stack

  • Graviton (m7g/c7g)
  • Docker Buildx
  • Jenkins ARM64 agents
  • Amazon EKS
  • Terraform
  • ECR
  • Helm
  • kubeconform

Implementation

  • Terraform's `eks-arch-node-group` module: `architecture` (amd64/arm64) drives `ami_type` and the default instance family (m6i/c6i vs m7g/c7g); `desired_size` is deliberately left for a runbook or Cluster Autoscaler to own, not Terraform.
  • A Jenkinsfile builds native per-arch agents in parallel and merges them into one manifest-list tag with `docker buildx imagetools create` — the per-arch tags are build-time intermediates, never referenced by a Deployment.
  • A compatibility-manifest linter (8 rules, GC001-GC008) over a per-service YAML manifest: both platforms present, base image pinned by digest per platform, a recorded scan per platform, and promotion past `canary` blocked without a passing scan on both.
  • A Helm chart schedules a stable Deployment and an optional small canary Deployment by `nodeSelector`/`affinity` on `kubernetes.io/arch`, sharing one Service with no `pool` label on its selector.

Security

  • Image scanning runs per architecture — the compatibility linter requires a recorded, consistent scan result per platform and blocks promotion past canary without a pass on both.
  • Base images pinned by digest per platform, not by tag — a manifest-list tag's two platform manifests can drift independently.
  • ECR tags are immutable, which is what makes the scheduling-based rollback trustworthy: the tag a rolled-back Deployment points at can't have silently changed shape.

Reliability

  • Rollback is reverting the scheduling constraint (`helm upgrade --set canary.enabled=false` or `--set architecture=amd64`) — no rebuild, same image tag throughout.
  • Both architectures keep capacity until a service's rollout reaches `complete`; shrinking the old node group is a deliberately separate, slower lever from the scheduling-side rollback.
  • A PodDisruptionBudget's selector matches both pools' shared labels, so draining a node in either pool during the migration can't take minAvailable pods below the floor.

Cost

  • Illustrative only: Graviton3 instances list at roughly 15% below the matching Intel/AMD instance size before any per-workload throughput difference; AWS's wider "up to ~40%" figure depends on the workload and isn't assumed here.
  • During an active migration both pools exist simultaneously, so cost is additive until the old pool is scaled down at the `complete` rollout stage.

Results

No quantified results are published for this project. Figures from client work are confidential, and this portfolio only shows numbers that can be verified or reproduced.

Code & documentation

Lessons learned

  • Decouple the registry decision from the scheduling decision: build multi-arch once, up front, so every later step is a Kubernetes change, never a CI rebuild.
  • Emulation hides exactly the failures a migration exists to find — QEMU lets a native-module build "succeed" while producing something that only breaks on real silicon, so the compatibility gate treats it as an error.
  • The canary here is a replica-count split, not a weighted one; a service mesh would be the natural next step for precise traffic shaping.