Architecture
Structural and design patterns that reduce energy consumption and carbon emissions at the system level.
17 patternsBuilding an ML model takes significant computing resources that need to be optimized for efficient utilization.
- ai
- machine-learning
- serverless
- size:small
From an energy-efficiency perspective, it's better to shorten the distance a network packet travels so that less energy is required to transmit it. Similarly, from an embodied-carbon perspective, when a network packet traverses through less computing equipment, we are more efficient with hardware.
- networking
- size:small
Containerizing workloads enables better resource utilisation and bin packing, reducing unnecessary compute allocation and embodied carbon compared to running full virtual machines.
- cloud
- compute
- kubernetes
- role:cloud-engineer
- size:medium
Applications are built with a software architecture that best fits the business need they are serving. Cloud providers make it easy to evaluate other CPU types
- cloud
- size:small
Service state refers to the in-memory or on-disk data required by a service to function. State includes the data structures and member variables that the service reads and writes. Depending on how the service is architected, the state might also include files or other resources stored on the disk. Applications that consume large memory or on-disk data require larger VM sizes, especially for cloud computing where you would need larger VM SKUs to support high RAM capacity and multiple data disks.
- cloud
- compute
- kubernetes
- role:software-engineer
- size:medium
Large-scale AI/ML models require significant storage space and take more resources to run as compared to optimized models.
- ai
- machine-learning
- size:small
All systems have periods of peak and low load. From a hardware-efficiency perspective, we are more efficient with hardware if we minimise the impact of request spikes with an implementation that allows an even utilization of components. From an energy-efficiency perspective, we are more efficient with energy if we ensure that idle resources are kept to a minimum.
- cloud
- size:small
Placing VMs in the same region or availability zone minimises the physical distance data must travel between instances, reducing the energy consumed by network traversal.
- cloud
- compute
- kubernetes
- network
- role:cloud-engineer
- size:medium
Data computation for ML workloads and ML inference is a significant contributor to the carbon footprint of the ML application. Also, if the ML model is running on the cloud, the data needs to be transferred and processed on the cloud to the required format that can be used by the ML model for inference.
- ai
- machine-learning
- size:small
Decomposing applications into independently scalable microservices allows each component to be right-sized for its own demand, reducing overall compute resource consumption and embodied carbon.
- cloud
- compute
- kubernetes
- role:cloud-engineer
- role:software-engineer
- size:medium
Training an AI model implies a significant carbon footprint. The underlying framework used for the development, training, and deployment of AI/ML needs to be evaluated and considered to ensure the process is as energy efficient as possible.
- ai
- machine-learning
- size:small
Selecting the right hardware/VM instance types for training is one of the choices you should make as part of your energy-efficient AI/ML process.
- ai
- machine-learning
- size:small
Service meshes add overhead through additional containers and increased network traffic, so they should only be deployed for applications that genuinely require the capabilities they provide.
- cloud
- kubernetes
- network
- role:cloud-engineer
- security
- size:medium
Cloud VMs built on energy-efficient processors, such as ARM-based cloud-native chips, can run scale-out workloads with significantly lower energy consumption and embodied carbon than traditional alternatives.
- cloud
- compute
- kubernetes
- role:cloud-engineer
- role:software-engineer
- size:medium
Efficient storage of the model becomes extremely important to manage the data used for ML model development.
- ai
- machine-learning
- size:small
Evaluate and use alternative, more energy efficient, models that provide similar functionality.
- ai
- machine-learning
- size:small
Serverless cloud services scale dynamically with demand and share infrastructure across many applications, reducing idle resource consumption and lowering embodied carbon emissions.
- cloud
- serverless
- size:small