WekaIO (WEKA) announced a new solution in collaboration with its customer Stability AI, an open-source generative artificial intelligence (GenAI) company. The WEKA Data Platform’s Converged Mode for Cloud solution will enhance Stability AI’s ability to train multiple AI models in the cloud, including its Stable Diffusion model, and extend efficiency, cost and sustainability benefits to its customers.
AI processes such as training and inference require energy and computing resources. Few companies, those pioneering the next frontier of GenAI solutions, can invest the time or expense needed to acquire and deploy the necessary horsepower on-premises, and training and inference of AI models in the cloud is often prohibitively expensive.
WEKA has developed a way to improve this equation for cloud-first GenAI customers. The WEKA Data Platform’s Converged Mode solution is the scale-out storage on deep learning capable instances available to users running workloads in the cloud. It uses ephemeral local storage and memory in cloud AI instances to yield exponential cost savings and performance improvements for large-scale GenAI resources compared to traditional data architectures.
Stability AI is seeking to use existing cloud resources. The company began working with WEKA to develop a “converged cloud” approach with the WEKA data platform and recently concluded a trial of the solution. Its converged architecture provides a single, high-performing set of resources to support the application and data platform simultaneously. The result is more optimal training of Stability AI’s models and better utility for its research teams.
“As the only independent, open, and multimodal GenAI company, we understand the limitations of running converged workloads on-premises,” says Tom Mason, CTO at Stability AI. “WEKA gives our customers more options by enabling training on converged mode in the public cloud. Our hope is that this partnership will increase our hardware utilisation and scale to thousands of instances and essentially, do more with less, more sustainably.”
Once deployed, Stability AI hopes to be able to extend the benefit of performance gains cloud cost-savings from WEKA’s data platform, all while lowering carbon emissions and energy consumption. The company expects to use the WEKA solution to underpin its forthcoming sustainability initiatives as its deployment matures and grows.
“The WEKA Data Platform can increase GPU-stack storage efficiency by 10-50x, driving a significant cost advantage for GenAI companies by helping to maximise resources they’ve already paid for,” says WEKA co-founder and CEO Liran Zvibel. “At the same time, WEKA makes it simple and affordable to run AI model training and inference in the public cloud, helping to shrink their data infrastructure and energy and carbon footprint, delivering a double-bottom line benefit for customers that want to harness the power of AI without compromising their corporate sustainability goals.”
WEKA’s Converged Mode for Cloud solution will be released as a public preview this year, initially on AWS, with other clouds to follow. Interested customers should contact their WEKA sales rep to join the waitlist.
Follow us and Comment on Twitter @TheEE_io