Summary
Weaviate operates isolated, dedicated clusters for its customers. Each environment needs enough storage headroom for changing workloads, but unused capacity can accumulate and EBS volumes cannot natively shrink. Datafy now automatically rightsizes storage across this infrastructure, reducing wasted EBS spend by 50% while leaving Weaviate's storage architecture intact.
The challenge: excess capacity that could not shrink
Storage requirements vary across customer environments and change over time. Teams provision headroom to avoid running out of space, but AWS does not support shrinking EBS volumes natively. Capacity added for growth can therefore remain allocated long after it is needed. Before Datafy, average storage utilization across the relevant Weaviate infrastructure was approximately 35%.
Manually reviewing and adjusting storage across dedicated clusters did not scale. And because those clusters run live customer AI services, any solution had to protect production reliability while avoiding additional operational burden.
The solution: automated, bi-directional rightsizing
Weaviate introduced Datafy gradually. Datafy Sensor first gave the team visibility into storage utilization and improvement opportunities. After infrastructure integration, QA, and a proof of concept, Weaviate deployed Datafy EBS Auto-Scaler to automatically grow and shrink volumes based on storage requirements.
The deployment required no CSI driver replacement, filesystem modification, or additional storage layer. In the first three months, the team also tested enhanced EBS performance capabilities and gained a unified view of volumes across approximately 100 AWS accounts.
“Plug-and-play is what got the contract over the finish line. Every other vendor we looked at wanted us to change how we do storage. We can't afford a support relationship that puts our customers' production environments at risk, and Datafy hasn't.”
Brave Okafor, Senior Platform Engineer, Weaviate
Operational impact
Storage capacity can now track actual workload requirements through automatic grow and shrink operations. This reduces unnecessary EBS capacity and the manual work involved in managing it, while preserving the existing storage setup for customer environments.