As artificial intelligence capabilities continue advancing at an unprecedented pace, organisations across all industries are exploring innovative ways to leverage Generative AI technologies to transform their businesses. However, the energy-intensive nature of training large AI language models means that Generative AI has significant data centre implications that must be considered.
Let’s cover some essential insights into how Generative AI will likely impact various aspects of your organisation’s data centre operations and infrastructure planning over the coming years. I will discuss the necessary changes required to data centre design and location strategy. I will also share perspectives on opportunities for organisations to build specialised training models within their private data centres using internal sensitive data sets while still considering proximity needs for user-facing AI applications.
Power Demands Will Drive Transformational Changes
The computational resources required to train advanced Generative AI models far exceed those of traditional workloads. In fact, research shows that model training can consume over ten times more energy than simply running inferences against a trained model. This energy-intensive nature means data centres supporting Generative AI development will need to be purpose-built with infrastructure optimised for handling significantly higher power densities.
Location Strategy Is a Multifaceted Consideration
When determining optimal data centre placement to support your Generative AI initiatives, a key distinction needs to be made between model training and application deployment stages due to their differing performance requirements:
- Model training has relatively loose latency constraints and can take advantage of locations with lower energy costs or favourable climates for free cooling. Centralised training-optimised facilities make strategic sense.
- However, Generative AI applications interacting with end users and customers require data centre proximity to minimise latency sensitivity, limiting location choices.
- A balanced strategy considers establishing centralised training facilities alongside a network of distributed edge sites positioned strategically close to target markets and populations for low-latency deployment of user-facing AI solutions.
Considering these factors, a multifaceted location approach can optimise costs and performance across the entire Generative AI workflow, from model building to real-world applications. Regional clustering of related capabilities makes maintenance and support more efficient, too.
Opportunities to Train Models Within Private Data Centers
While internet giants and specialised AI labs currently dominate foundational model development due to vast resources, many organisations can benefit from training customised models internally to be optimised for their specific industry use cases and internal data sets.
Some compelling advantages of building training capabilities within private data centres include:
- Stronger data privacy assurances than public clouds for sensitive enterprise data used in model development. Many regulated sectors cannot risk such information in third-party environments.
- Potential long-term cost control versus spiralling cloud expenses for heavy model training workloads involving terabytes of proprietary data.
- Ability to fine-tune generative models based directly on internal data sources inaccessible to external parties, unlocking more customised solutions.
- Leveraging existing on-premises infrastructure capacity that may be underutilised for traditional applications represents an efficient use of existing assets.
Of course, private data centres must be equipped with high-performance infrastructure optimised for Generative AI’s power-hungry model training workloads. However, a blended cloud-and-edge strategy presents compelling options.
Final thoughts
Generative AI’s heavy data centre requirements necessitate strategic planning around infrastructure, location, and cost management. A balanced approach evaluates options for centralised training facilities while also building specialised models locally using sensitive enterprise data. With proactive design, placement and operations changes, organisations can successfully navigate the balance between building public and private AI solutions to ensure the best outcome.
It will be interesting to see how organisations approach the infrastructure investments relating to AI over the next few years, especially when eventually moving from internal proof of concepts to production solutions.