Context
Vast datasets, and a mission that can't wait.
The agency employs thousands of people, including civilians, military members and contractors, and its analysts work with vast datasets inside the agency's Commercial Cloud Enterprise (C2E) environment.
It needed a big data tool that could process that data at scale, and let analysts create complex analyses, visualize them and improve their datasets, without waiting on engineering for every change.
The constraint
Accredited, at scale, inside C2E.
- An Authority to OperateAny tool had to hold an ATO to run in the agency's environment.
- Inside the agency's cloudProcessing had to happen in C2E, on the agency's own cloud services, not in an outside platform.
- Massive, mixed workloadsBillions of records in batch and streaming, across data engineering, data science and analytics.
- Costs hard to seeCloud processing costs needed to be visible and controlled, job by job.