CSIRO Uses Serverless Architecture to Query 100x More Genomic Variants on AWS
InfoQ, Friday, October 23rd, 2026
CSIRO's serverless sBeacon queries genomic data in place on S3 using Lambda, Athena and DynamoDB, at fractions of a cent per query.
CSIRO built Serverless Beacon (sBeacon), an implementation of the GA4GH Beacon standard that searches genomic variant data without copying it into a central database.
VCF files stay in Amazon S3 while metadata is indexed separately, and Lambda functions use bcftools with HTTP byte-range requests to read only the genomic regions needed.
In testing on chromosome 1 from the 1000 Genomes Project, ingesting a 1.1 GB compressed file took 18 seconds and cost about $0.0005, and a 10,000-base-pair query averaged 1.52 seconds.
It is a useful case study in event-driven, query-in-place data design.