Netflix Bridges AWS IAM and Internal Identity with Workload Attestation
InfoQ, Wednesday, October 21st, 2026
Netflix lets Spark jobs on EMR swap their AWS IAM identity for short-lived internal mTLS certificates, with no injected secrets.
Netflix built a workload attestation mechanism so Apache Spark jobs on Amazon EMR can exchange their AWS execution identity for a short-lived certificate from its internal Metatron PKI.
Each Data Project maps to a dedicated IAM role spread across a pool of AWS accounts.
The Spark driver sends a pre-signed STS GetCallerIdentity request, which the identity service checks against signed metadata from the control plane before issuing a certificate.
The driver attests once and passes credentials to executors over encrypted Spark RPC, then re-attests periodically for long-running jobs.