Squashed commit of the following:

cr https://code.amazon.com/reviews/CR-50314382

commit 2eb3872b0d3bfde09719497440019eeb7061dd7b
Merge: fa1ac709 a3877ba
Author: Khushboo Rajput <[email protected]>
Date:   Tue May 4 21:08:59 2021 -0700

    Merge branch 'main' into aes_7.10_nightly

commit fa1ac70933ae5c1939c4a0fa30f50f81925a07ab
Merge: fac4f8b8 c70b19d
Author: Khushboo Rajput <[email protected]>
Date:   Tue May 4 21:04:35 2021 -0700

    Merge branch 'aes_7.10_gh' into aes_7.10_nightly

commit a3877ba
Author: Sruti Parthiban <[email protected]>
Date:   Thu Apr 15 12:21:34 2021 -0700

    Fix format error (#309)

commit 7e0d2f8
Author: Sruti Parthiban <[email protected]>
Date:   Thu Apr 15 11:30:34 2021 -0700

    Fix failing election term tests (#305)

commit ea643fe
Author: meetshah777 <[email protected]>
Date:   Thu Apr 1 02:29:45 2021 +0530

    Changed master total PendingQueueSize to PendingQueueSize per task type (#296)

    * Changed master total PendingQueueSize to PendingQueueSize per task type

    * Addressed comments

    Co-authored-by: Meet Shah <[email protected]>

commit bf6a4cc
Author: meetshah777 <[email protected]>
Date:   Thu Apr 1 02:27:24 2021 +0530

    Added Election Term Metric (#297)

    * Changed master total PendingQueueSize to PendingQueueSize per task type

    * Added Election Term Metric

    * Formmated files

    * Merge conflict resolved

    * Addressed comments

    Co-authored-by: Meet Shah <[email protected]>

commit 130a494
Author: Mital Awachat <[email protected]>
Date:   Thu Mar 25 23:35:49 2021 +0530

    AdmissionControl: Clear metricQueue before running test (#300)

    Co-authored-by: Mital Awachat <[email protected]>

commit 3e6b8ec
Author: Joshua Tokle <[email protected]>
Date:   Wed Mar 24 19:07:06 2021 -0700

    Configure spotless plugin and enable code formatting (#276)

    * Configure spotless plugin and enable code formatting

    * Remove checkstyle and @ignore flaky tests

commit fac4f8b835e793e3db5240f1e4d1448fd9d16baa
Merge: 253229a2 f7b430b
Author: Khushboo Rajput <[email protected]>
Date:   Tue Mar 23 17:14:37 2021 -0700

    Merge branch 'main' into aes_7.10_nightly

commit f7b430b
Author: Dharmesh <[email protected]>
Date:   Wed Mar 24 05:25:20 2021 +0530

    [Build Fix]Removing the typecasting to IndexingPressure class to fix build failure while backporting (#293)

commit d49a701
Author: amathur1893 <[email protected]>
Date:   Thu Mar 18 02:51:11 2021 +0530

    Cluster state applier stats (#282)

    * Publish latency and failure metrics for cluster state applier thread

    * Publish latency and failure metrics for cluster state applier thread

    * Extract MetricsTracker

    * Add Javadoc for Metrics Tracker

    * Add ClusterStateApplierStats to collect latency and failure metrics

    * Add ClusterStateApplierStats to collect latency and failure metrics

    * Added UTs for ClusterApplierServiceStatsCollector

    * Added UTs for ClusterApplierServiceStatsCollector

    * Java doc changes

    Co-authored-by: Arpita Mathur <[email protected]>

commit 253229a24f49b5479333d8ffda7167441e39162a
Merge: 7c5cb8e6 52e5cc4
Author: Khushboo Rajput <[email protected]>
Date:   Mon Mar 8 22:06:07 2021 -0800

    Merge branch 'main' into aes_7.10_nightly

commit 52e5cc4
Author: Mital Awachat <[email protected]>
Date:   Tue Mar 9 10:49:11 2021 +0530

    Track admission control metrics (#290)

    * Track admission control metrics

    Co-authored-by: Mital Awachat <[email protected]>

commit c280915
Author: Dharmesh <[email protected]>
Date:   Thu Mar 4 23:12:37 2021 +0530

    [Bug Fix] Incorrect metric value was returned from the getter of current bytes (#287)

commit 7c5cb8e63cde9023084ab57075bbdf554df9748c
Merge: 28ee8302 a997189
Author: Khushboo Rajput <[email protected]>
Date:   Thu Mar 4 09:39:54 2021 -0800

    Merge branch 'main' into aes_7.10_nightly

commit a997189
Author: Dharmesh <[email protected]>
Date:   Thu Mar 4 10:55:36 2021 +0530

    Added metrics publishing logic (#285)

commit 28ee8302529c279dfadc47bf9b0d4f6b6d91ac75
Author: Dharmesh <[email protected]>
Date:   Tue Mar 2 23:49:47 2021 +0530

    Adding a additional check to just process first 50 objects of hot store (#284)

commit 0d2566a61bab1531400eaae486c3734bcea386fe
Author: Dharmesh <[email protected]>
Date:   Tue Mar 2 11:33:35 2021 +0530

    Changes to collect shard indexing pressure metrics and push to shared location (#278)

    * Changes to collect shard indexing pressure metrics and push to shared location

    * Code changes to remove pushing the cold store objects and removing isActiveShard flag as it becomes irrelevant now

    * Addressing comment to change NodeRole to IndexingStage as node role already has a predefined meaning

    * Checking if class is present while adding the collector in scheduled executor so that the collector will never be invoked if class in not present

    * Addressing comments.

commit 819d485
Author: Dharmesh <[email protected]>
Date:   Tue Mar 2 23:49:47 2021 +0530

    Adding a additional check to just process first 50 objects of hot store (#284)

commit 2f3df02
Author: Dharmesh <[email protected]>
Date:   Tue Mar 2 11:33:35 2021 +0530

    Changes to collect shard indexing pressure metrics and push to shared location (#278)

    * Changes to collect shard indexing pressure metrics and push to shared location

    * Code changes to remove pushing the cold store objects and removing isActiveShard flag as it becomes irrelevant now

    * Addressing comment to change NodeRole to IndexingStage as node role already has a predefined meaning

    * Checking if class is present while adding the collector in scheduled executor so that the collector will never be invoked if class in not present

    * Addressing comments.

commit 9f83059
Author: Yu Sun <[email protected]>
Date:   Fri Feb 26 14:36:02 2021 -0800

    Log error once only (#280)

commit 3377c2f
Author: Karthik Kumarguru <[email protected]>
Date:   Mon Feb 22 15:11:45 2021 -0800

    Increase the max heap limit to check against in the integ tests (#277)

commit 641b403f941596f543e95a01b9a5795b73d7fd65
Author: Khushboo Rajput <[email protected]>
Date:   Sat Feb 20 12:45:57 2021 -0800

    Creating new brand for ES 7.10 version

licenses/performanceanalyzer-rca-1.13.jar.sha1

@@ -1 +1 @@
-0ca252df95b9c0743d7b29452c7637504d5ba50f
+0d0d568122db55b2a11cf5d0f562d754e2bcf898

src/main/java/com/amazon/opendistro/elasticsearch/performanceanalyzer/PerformanceAnalyzerPlugin.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ * Copyright 2019-2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
  *
  * Licensed under the Apache License, Version 2.0 (the "License").
  * You may not use this file except in compliance with the License.
@@ -26,7 +26,6 @@
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors.FaultDetectionMetricsCollector;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors.GCInfoCollector;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors.HeapMetricsCollector;
-import com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors.ShardIndexingPressureMetricsCollector;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors.MasterServiceEventMetrics;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors.MasterServiceMetrics;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors.MasterThrottlingMetricsCollector;
@@ -214,13 +213,7 @@ public PerformanceAnalyzerPlugin(final Settings settings, final java.nio.file.Pa
         scheduledMetricCollectorsExecutor.addScheduledMetricCollector(new ClusterApplierServiceStatsCollector(
                 performanceAnalyzerController,configOverridesWrapper));
         scheduledMetricCollectorsExecutor.addScheduledMetricCollector(new AdmissionControlMetricsCollector());
-        try {
-            Class.forName(ShardIndexingPressureMetricsCollector.SHARD_INDEXING_PRESSURE_CLASS_NAME);
-            scheduledMetricCollectorsExecutor.addScheduledMetricCollector(new ShardIndexingPressureMetricsCollector(
-                performanceAnalyzerController,configOverridesWrapper));
-        } catch (ClassNotFoundException e) {
-            LOG.info("Shard IndexingPressure not present in this ES version. Skipping ShardIndexingPressureMetricsCollector");
-        }
+
         try {
             Class.forName(MasterClusterStateUpdateStatsCollector.MASTER_CLUSTER_UPDATE_STATS_CLASS_NAME);
             scheduledMetricCollectorsExecutor.addScheduledMetricCollector(new MasterClusterStateUpdateStatsCollector(

src/main/java/com/amazon/opendistro/elasticsearch/performanceanalyzer/collectors/AdmissionControlMetricsCollector.java

@@ -1,3 +1,18 @@
+/*
+ * Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License").
+ * You may not use this file except in compliance with the License.
+ * A copy of the License is located at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * or in the "license" file accompanying this file. This file is distributed
+ * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+ * express or implied. See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
 package com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors;
 
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.PerformanceAnalyzerApp;
@@ -6,6 +21,7 @@
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.metrics.MetricsProcessor;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.metrics.PerformanceAnalyzerMetrics;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.rca.framework.metrics.ExceptionsAndErrors;
+import com.amazon.opendistro.elasticsearch.performanceanalyzer.rca.framework.metrics.WriterMetrics;
 import com.fasterxml.jackson.annotation.JsonProperty;
 import org.apache.logging.log4j.LogManager;
 import org.apache.logging.log4j.Logger;
@@ -14,7 +30,7 @@
 import java.util.Objects;
 
 /**
- * AdmissionControlMetricsCollector collects `UsedQuota`, `TotalQuota`, RejectionCount from ElasticSearch-Jetty - AdmissionController
+ * AdmissionControlMetricsCollector collects `UsedQuota`, `TotalQuota`, RejectionCount
  */
 public class AdmissionControlMetricsCollector extends PerformanceAnalyzerMetricsCollector implements MetricsProcessor {
 
@@ -41,7 +57,15 @@ public AdmissionControlMetricsCollector() {
     }
 
     @Override
+    @SuppressWarnings("unchecked")
     public void collectMetrics(long startTime) {
+        if(!isAdmissionControlFeatureAvailable()) {
+            LOG.debug("AdmissionControl is not available for this domain");
+            PerformanceAnalyzerApp.WRITER_METRICS_AGGREGATOR.updateStat(
+                    WriterMetrics.ADMISSION_CONTROL_COLLECTOR_NOT_AVAILABLE, "", 1);
+            return;
+        }
+
         long startTimeMillis = System.currentTimeMillis();
         try {
             Class admissionController = Class.forName(ADMISSION_CONTROLLER);
@@ -87,11 +111,17 @@ public void collectMetrics(long startTime) {
             }
 
             saveMetricValues(value.toString(), startTime);
+
+            PerformanceAnalyzerApp.WRITER_METRICS_AGGREGATOR.updateStat(
+                    WriterMetrics.ADMISSION_CONTROL_COLLECTOR_EXECUTION_TIME, "",
+                    System.currentTimeMillis() - startTimeMillis);
+
         } catch(Exception ex) {
             PerformanceAnalyzerApp.ERRORS_AND_EXCEPTIONS_AGGREGATOR.updateStat(
                     ExceptionsAndErrors.ADMISSION_CONTROL_COLLECTOR_ERROR, getCollectorName(),
                     System.currentTimeMillis() - startTimeMillis);
-            LOG.debug("Exception in collecting AdmissionControl Metrics: {} for startTime {}", ex::toString, () -> startTime);
+            LOG.debug("Exception in collecting AdmissionControl Metrics: {} for startTime {}",
+                    ex::toString, () -> startTime);
         }
     }
 
@@ -144,4 +174,13 @@ public long getRejectionCount() {
         }
     }
 
+    private boolean isAdmissionControlFeatureAvailable() {
+        try {
+            Class.forName(ADMISSION_CONTROLLER);
+            Class.forName(ADMISSION_CONTROL_SERVICE);
+        } catch (ClassNotFoundException e) {
+            return false;
+        }
+        return true;
+    }
 }

src/main/java/com/amazon/opendistro/elasticsearch/performanceanalyzer/collectors/ClusterApplierServiceStatsCollector.java

@@ -194,4 +194,4 @@ public double getClusterApplierServiceFailed() {
             return clusterStateAppliedFailedCount;
         }
     }
-}
+}

src/main/java/com/amazon/opendistro/elasticsearch/performanceanalyzer/collectors/ElectionTermCollector.java

@@ -103,4 +103,4 @@ public long getElectionTerm() {
             return electionTerm;
         }
     }
-}
+}

src/main/java/com/amazon/opendistro/elasticsearch/performanceanalyzer/collectors/MasterServiceMetrics.java

@@ -23,13 +23,15 @@
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.ESResources;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.metrics.AllMetrics.MasterPendingTaskDimension;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.metrics.AllMetrics.MasterPendingValue;
+import com.amazon.opendistro.elasticsearch.performanceanalyzer.metrics.AllMetrics.MasterPendingTaskDimension;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.metrics.MetricsConfiguration;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.metrics.MetricsProcessor;
 import com.fasterxml.jackson.annotation.JsonProperty;
 import org.elasticsearch.cluster.service.PendingClusterTask;
 import java.util.HashMap;
 import java.util.List;
 
+
 @SuppressWarnings("unchecked")
 public class MasterServiceMetrics extends PerformanceAnalyzerMetricsCollector implements MetricsProcessor {
     public static final int SAMPLING_TIME_INTERVAL = MetricsConfiguration.CONFIG_MAP.get(MasterServiceMetrics.class).samplingInterval;

src/main/java/com/amazon/opendistro/elasticsearch/performanceanalyzer/util/Utils.java

@@ -64,9 +64,10 @@ public static void configureMetrics() {
         MetricsConfiguration.CONFIG_MAP.put(FaultDetectionMetricsCollector.class, cdefault);
         MetricsConfiguration.CONFIG_MAP.put(ShardStateCollector.class, cdefault);
         MetricsConfiguration.CONFIG_MAP.put(MasterThrottlingMetricsCollector.class, cdefault);
-        MetricsConfiguration.CONFIG_MAP.put(ShardIndexingPressureMetricsCollector.class, cdefault);
         MetricsConfiguration.CONFIG_MAP.put(MasterClusterStateUpdateStatsCollector.class, cdefault);
         MetricsConfiguration.CONFIG_MAP.put(ClusterApplierServiceStatsCollector.class, cdefault);
+        MetricsConfiguration.CONFIG_MAP.put(ElectionTermCollector.class, cdefault);
+        MetricsConfiguration.CONFIG_MAP.put(ShardIndexingPressureMetricsCollector.class, cdefault);
     }
 
     // These methods are utility functions for the Node Stat Metrics Collectors. These methods are used by both the all

src/test/java/com/amazon/opendistro/elasticsearch/performanceanalyzer/collectors/AdmissionControlMetricsCollectorTests.java

@@ -1,28 +1,52 @@
+/*
+ * Copyright 2021 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License").
+ * You may not use this file except in compliance with the License.
+ * A copy of the License is located at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * or in the "license" file accompanying this file. This file is distributed
+ * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
+ * express or implied. See the License for the specific language governing
+ * permissions and limitations under the License.
+ */
+
 package com.amazon.opendistro.elasticsearch.performanceanalyzer.collectors;
 
+import static org.junit.Assert.assertEquals;
+
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.CustomMetricsLocationTestBase;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.metrics.MetricsConfiguration;
-import com.amazon.opendistro.elasticsearch.performanceanalyzer.metrics.PerformanceAnalyzerMetrics;
 import com.amazon.opendistro.elasticsearch.performanceanalyzer.reader_writer_shared.Event;
-import org.junit.Test;
-
-import java.util.ArrayList;
+import com.amazon.opendistro.elasticsearch.performanceanalyzer.util.TestUtil;
 import java.util.List;
-
-import static org.junit.Assert.assertEquals;
+import org.junit.Before;
+import org.junit.Test;
 
 public class AdmissionControlMetricsCollectorTests extends CustomMetricsLocationTestBase {
 
+    @Override
+    @Before
+    public void setUp() throws Exception {
+        super.setUp();
+        // clean metricQueue before running every test
+        TestUtil.readEvents();
+        System.setProperty("performanceanalyzer.metrics.log.enabled", "False");
+    }
+
     @Test
     public void admissionControlMetricsCollector() {
-        MetricsConfiguration.CONFIG_MAP.put(AdmissionControlMetricsCollector.class, MetricsConfiguration.cdefault);
-        AdmissionControlMetricsCollector admissionControlMetricsCollector = new AdmissionControlMetricsCollector();
+        MetricsConfiguration.CONFIG_MAP.put(
+                AdmissionControlMetricsCollector.class, MetricsConfiguration.cdefault);
+        AdmissionControlMetricsCollector admissionControlMetricsCollector =
+                new AdmissionControlMetricsCollector();
 
         long startTimeInMills = System.currentTimeMillis();
         admissionControlMetricsCollector.saveMetricValues("testMetric", startTimeInMills);
 
-        List<Event> metrics =  new ArrayList<>();
-        PerformanceAnalyzerMetrics.metricQueue.drainTo(metrics);
+        List<Event> metrics = TestUtil.readEvents();
         assertEquals(1, metrics.size());
         assertEquals("testMetric", metrics.get(0).value);
     }