MLCOMPUTE-1155 | fix cost factor default while reading spark costs fr…

…om srv configs (#138) Co-authored-by: Sameer Sharma <[email protected]>
Yelp · Feb 5, 2024 · 768dda5 · 768dda5
1 parent 0d199d9
commit 768dda5
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/service_configuration_lib/spark_config.py b/service_configuration_lib/spark_config.py
@@ -980,7 +980,7 @@ def compute_approx_hourly_cost_dollars(
                 spark_conf.get('spark.dynamicAllocation.minExecutors', min_cores),
             ))
 
-        cost_factor = self.spark_costs.get(paasta_cluster, dict())[paasta_pool]
+        cost_factor = self.spark_costs.get(paasta_cluster, dict()).get(paasta_pool, 0)
 
         min_dollars = round(min_cores * cost_factor, 5)
         max_dollars = round(max_cores * cost_factor, 5)