Cerebras Systems, Cirrascale Cloud Services debuts AI model studio to train GPT-class models with 8x accuracy - The EE

Cerebras Systems, Cirrascale Cloud Services debuts AI model studio to train GPT-class models with 8x accuracy

Cerebras Systems, the specialist in accelerating artificial intelligence (AI) compute, and Cirrascale Cloud Services, a provider of deep learning infrastructure solutions for autonomous vehicle, NLP, and computer vision workflows, today announced the availability of the Cerebras AI Model Studio. Hosted on the Cerebras Cloud @ Cirrascale, this new offering enables customers to train generative Transformer (GPT)-class models, including GPT-J, GPT-3 and GPT-NeoX, on Cerebras Wafer-Scale Clusters, including the newly announced Andromeda AI supercomputer.

Traditional cloud providers struggle with large language models as they are unable to guarantee latency between large numbers of GPUs. Variable latency produces complex and time-consuming challenges in distributing a large AI model amongst GPUs and large swings in time to train. The Cerebras AI Model Studio overcomes these challenges. Set up is quick and easy; clusters of dedicated CS-2s guarantee deterministic latency; and because the clusters rely solely on data parallelisation, there is zero distributed compute work required.

Training Large Language Models (LLMs) is challenging and expensive multi-billion parameter models require months to train on clusters of GPUs and a team of engineers experienced in distributed programming and hybrid data-model parallelism. It is a multi-million dollar investment that many organisations simply cannot afford.

ModelParameters (B)Tokens to train to Chinchilla point (B)Cerebras AI Model Studio days to trainCerebras AI Model Studio price to train
GPT-3 XL1.3260.4$2,500
GPT-J61208$45,000
GPT-3 6.7B6.713411$40,000
T-5 11B1134*9$60,000
GPT-3 13B1326039$150,000
GPT-NeoX2040047$525,000
GPT 70B701,40085$2,500,000
GPT 175B1753,500Call for quoteCall for quote

* T5 tokens to train from the original T5 paper. Chinchilla scaling laws not applicable.

The Cerebras AI Model Studio offers users the ability to train GPT-class models at half the cost of traditional cloud providers and requires only a few lines of code to get going. Users can choose from state-of-the-art GPT-class models, ranging from 1.3 billion parameters up to 175 billion parameters, and complete training with 8x faster time to accuracy than on an A100.

“The new Cerebras AI Model Studio expands our partnership with Cirrascale and further democratises AI by providing customers with access to multi-billion parameter NLP models on our powerful CS-2 clusters, with predictable, competitive model-as-a-service pricing,” says Andrew Feldman, CEO and co-founder of Cerebras Systems. “Our mission at Cerebras is to broaden access to deep learning and rapidly accelerate the performance of AI workloads. The Cerebras AI Model Studio makes this easy and dead simple just load your dataset and run a script.”

The Cerebras AI Model Studio offers users cloud access to the Cerebras Wafer-Scale Cluster, which enables GPU-impossible work with first-of-its-kind near-perfect linear scale performance. Users can access up to a 16-node Cerebras Wafer-Scale Cluster and train models using longer sequence lengths of up to 50,000 tokens a capability only available to Cerebras users opening up new opportunities for exciting research.

“We are really excited to offer our enterprise, research and academic customers easy, affordable access to the CS-2 accelerator to train GPT-class models in less than one day,” says PJ Go, CEO, Cirrascale Cloud Services. “We’ve made the process extremely simple eliminating the need for dev-ops and distributed programming with push-button model scaling, from 1 to 20 billion parameters.”

With every component optimised for AI work, the Cerebras Cloud @ Cirrascale delivers more compute performance at less space and less power than any other solution. Depending on workload, from AI to HPC, it delivers hundreds or thousands of times more performance than legacy alternatives, but uses only a fraction of the space and power. Cerebras Cloud is designed to enable fast, flexible training and low-latency datacentre inference, thanks to greater compute density, faster memory, and higher bandwidth interconnect than any other datacentre AI solution.

The Cerebras AI Model Studio is available now. For a limited time, users can sign up for free 2-day trial evaluation run.

Customers can begin using the Cerebras AI Model Studio by visiting here.

For more information, please visit here.

Follow us and Comment on Twitter @TheEE_io

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close