Skip to content

Latest commit

 

History

History
144 lines (101 loc) · 7.68 KB

methodology.md

File metadata and controls

144 lines (101 loc) · 7.68 KB

Methodology

We are largely using the work of Carbon Footprint Calculator and Esty Cloud Jewels for Energy and Carbon Emission estimations. Those are just "estimations" and will probably differ from the actual energy used and carbon emissions of your infrastructure.

In summary, for each resource, Carbonifer calculate an Energy Estimate (Watt per Hour) used by it, and multiply it by the Carbon Intensity of the underlying data center.

This tool currently estimates usage emissions, not embodied emissions (manufacturing, transport, recycling...). It is not a full LCA (Life Cycle Assessment) tool.

Estimated Carbon Emissions (gCO2eq/h) = Energy Estimate (Wh) x Carbon Intensity (gCO2eq/Wh)

Energy Estimate

Providers don't really communicate the Energy used by their resources. So we need to come up with an estimation based on different benchmarks and empirical tests. Also, because we do this estimation before deploying the infrastructure, at "plan" phase, we need to assume the average use of those resources.

CPU

To calculate an average use of a CPU, we need the minimum and max power of this CPU and use the following formula:

Average Watts = Number of vCPU * (Min Watts + Avg vCPU Utilization * (Max Watts - Min Watts))
  • Average Watts result in Watt Hour
  • Number of vCPU : depends on the machine type chosen
  • Min Watt and Max Watts depend on CPU architecture
  • Avg vCPU Utilization because we do this estimation at "plan" time, there is no way to pick a relevant value. However, to be able to plan and compare different CPUs or regions we need to set this constant. This is read from (by descending priority order)
    • user's config file in $HOME/.carbonifer/config.yml), variable avg_cpu_use
    • targeted folder config file in $TERRAFORM_PROJECT/.carbonifer/config.yml), variable avg_cpu_use
    • The default is 0.5 (50%)

Memory

Using the same methodology of Carbon Footprint Calculator we also pick the Energy Coefficient of 0.392 Watt Hour / Gigabyte and we use the following formula:

Watt hours = Memory usage (GB) x Memory Energy Coefficient

Disk Storage

We are using the same Storage Energy Coefficient as Carbon Footprint Calculator in energy coefficients file. This coefficient is different for SSD and HDD, so disk type is important.

Watt hours = Disk Size (TB) x Storage Energy Coefficient x Replication Factor

``

Replication Factor: most cloud provider offers to the customer data replication to minimize the risk of data loss:

  • Regular Disk will have a replication factor of 1
  • GCP regional disk: user sets the list of zones to replicate data, so the Replication Factor will be equal to the number of zones picked

Unless set by the user in terraform file, the default size can be hard to find:

  • GCP :
    • if an image data resource exists, if user-provided credentials, Carbonifer lets terraform get the size of the disk image
    • if no data resource has been declared, a warning is printed out and as default, we use the following values (coming from empirical testing)
      • boot disk : 10 Gb, HDD
      • persistent disk: 500 Gb, HDD

GPU

Similarily to CPU, GPU energy consumption is calculated from the GPU type from min/max Watt described in Carbon Footprint Calculator, we use min/max watt from constant file GPU Watt per GPU Type and apply same formula as CPU.

Average GPU Utilization is also read from:

  • user's config file in $HOME/.carbonifer/config.yml), variable avg_gpu_use
  • targeted folder config file in $TERRAFORM_PROJECT/.carbonifer/config.yml), variable avg_gpu_use
  • The default is 0.5 (50%)

Instance Group size and autoscaler

For group of instances, like GCP managed instance group or AWS autoscaling group, estimations will be displayed by instance and a count value will appear:

 --------------------------------------- ------------------ ------- ------------------------ 
  resource type                           name               count   emissions per instance  
 --------------------------------------- ------------------ ------- ------------------------ 
  google_compute_instance_group_manager   my-group-manager   3        0.5568 gCO2eq/h        
 --------------------------------------- ------------------ ------- ------------------------ 
                                          Total              3        1.6704 gCO2eq/h        

So that, we can have a sense of what the footprint of each instance and infer easily what would be the estimation if we change the size of the group.

However, if a resource have a terraform count attribute, they will appear individually:

resource "google_compute_instance" "foo" {
  count = 2
  ...
}

will produce :

 --------------------------------------- ------------------ ------- ------------------------ 
  resource type                           name               count   emissions per instance  
 --------------------------------------- ------------------ ------- ------------------------ 
  google_compute_instance                 foo[0]             1        0.5568 gCO2eq/h        
  google_compute_instance                 foo[1]             1        0.5568 gCO2eq/h        
  google_compute_instance                 foo[2]             1        0.5568 gCO2eq/h        
 --------------------------------------- ------------------ ------- ------------------------ 
                                          Total              3        1.6704 gCO2eq/h        

Autoscaling groups have a min and max size, and we cannot know in advance what the average size! User can define an average percentage in global config:

provider:
  gcp:
    avg_autoscaler_size_percent: 0.5

For example if min size is 1 and max size is 5, average will be 0.5 * (5-1) = 2

Carbon Intensity

This is the Carbon Emissions per Power per Time, in gCO2eq/Wh.

There are many models to get this Carbon Intensity:

  • Declared by Cloud Provider
  • From local Grid operator

Google claims net carbon emissions of all their regions are Zero, basically they compensate Carbon emissions of electricity used by investing in carbon offset. We decided to disregard this claim as electricity still comes from local Grid and are generated by actual Power plants running on Fossil fuels or renewable energy. The less electricity an infrastructure uses, the less it needs to be offset.

Currently, Carbonifer focuses on yearly average Grid carbon intensity, and we are using the following sources: