[JENKINS-74983] Add support for authenticated Webhooks registered in Bitbucket

Verify in the webhooks processor when the signature is present is matches the configured
Add configuration in the global settings to setup HMAC credentials. The secret is not customisable by single project for the following reasons:
* events should contains a duplicate of the payload to be verified only in BitbucketSCMSource.retrieve method.
* that means spend a lot of resources just to ignore the payload. Multiple fake requests, would overload Jenkins that have to process events to lookup the right project.
* could not response to Bitbucket that the payload is invalid because events are managed async.
* each event could serve multiple projects that potentially could be configured with a different secret.

Author: nfalco79

docs/USER_GUIDE.adoc

@@ -45,20 +45,20 @@ Follow these steps to create a multi-branch project with Bitbucket as a source:
 . Create the multi-branch project. This step depends on which multi-branch plugin is installed.
 For example, "Multibranch Pipeline" should be available as a project type if Pipeline Multibranch plugin is installed.
 +
-image::images/screenshot-1.png[scaledwidth=90%]
+image::images/screenshot-1.png
 
 . Select "Bitbucket" as _Branch Source_
 +
-image::images/screenshot-2.png[scaledwidth=90%]
+image::images/screenshot-2.png
 
 . Set credentials to access Bitbucket API and checkout sources (see "Credentials configuration" section below).
 . Set the repository owner and name that will be monitored for branches and pull requests.
 +
-image::images/screenshot-4.png[scaledwidth=90%]
+image::images/screenshot-4.png
 
 . Finally, save the project. The initial indexing process will run and create projects for branches and pull requests.
 +
-image::images/screenshot-5.png[scaledwidth=90%]
+image::images/screenshot-5.png
 
 [id=bitbucket-scm-navigator]
 == Organization folders
@@ -76,20 +76,20 @@ image::images/screenshot-6.png[scaledwidth=70%]
 .. A Bitbucket Data Center Project ID: all repositories in the project are imported as Multibranch projects. *Note that the project ID needs to be used instead of the project name*.
 .. A regular username: all repositories which the username is owner of are imported.
 +
-image::images/screenshot-8.png[scaledwidth=90%]
+image::images/screenshot-8.png
 
 . Save the configuration. The initial indexing process starts. Once it finishes, a Multibranch
 project is created for each repository.
 +
-image::images/screenshot-9.png[scaledwidth=90%]
+image::images/screenshot-9.png
 
 [id=bitbucket-avatar]
 == Avatar
 
 This plugin have customized icon designed for the "Organization Folder" image:/src/main/webapp/images/bitbucket-logo.svg[icon,20,20], for "Multibranch Pipeline", for Single Repository image:/src/main/webapp/images/bitbucket-repository-git.svg[icon,20,20] and Folder image:/src/main/webapp/images/bitbucket-scmnavigator.svg[icon,20,20] project type. This is the default behaviour of the plugin starting from version 935.
 It is possible associate the Bitbucket avatar to the project item selecting the "Show Bitbucket avatar images" behaviour in the project configuration.
 
-image::images/screenshot-19.png[scaledwidth=90%]
+image::images/screenshot-19.png
 
 The supported avatars are:
 
@@ -121,26 +121,38 @@ For Bitbucket Data Center only it is possible chose which webhooks implementatio
 
 - Plugin implementation relies on the configuration available via specific APIs provided by the https://marketplace.atlassian.com/apps/1215474/post-webhooks-for-bitbucket?tab=overview&hosting=datacenter[Post Webhooks for Bitbucket] plugin itself. To get it worked plugin must be already pre-installed on the server instance. This provider allows custom settings managed by the _ignore committers_ trait. _Note: This specific implementation will be moved to an individual repository as soon as https://issues.jenkins.io/browse/JENKINS-74913[JENKINS-74913] is implemented._
 
-image::images/screenshot-14.png[scaledwidth=90%]
+image::images/screenshot-14.png
 
 For both Bitbucket _Multibranch Pipelines_ and _Organization folders_ there is an "Override hook management" behavior
 to opt out or adjust system-wide settings.
 
-image::images/screenshot-18.png[scaledwidth=90%]
+image::images/screenshot-18.png
 
 IMPORTANT: In order to have the auto-registering process working fine the Jenkins base URL must be
 properly configured in _Manage Jenkins_ » _System_
 
+=== Webhooks signature
+
+Once Jenkins is configured to receive payloads, it will listen for any delivery that's sent to the endpoint you configured. For security reasons, you should only process deliveries from Bitbucket.
+To ensure your self-hosted server only processes deliveries from Bitbucket, you need to:
+* Create a secret token for a webhook
+* Enable hooks signature verification for the chosen Bitbucket Endpoints
+* Select the secret token create at point 1, only _String credentials_ are taken into account.
+
+Any incoming webhook payloads from that given endpoint will be validated against the configured token, to verify they are coming from the configured Bitbucket endpoint URL.
+
+image::images/screenshot-20.png
+
 [id=bitbucket-creds-config]
 == Credentials configuration
 
 The plugin (for both _Bitbucket multibranch pipelines_ and _Bitbucket Workspace/Project organization folders_) requires a credential to be configured to scan branches. It will also be the default credential to use when checking out sources.
 
-image::images/screenshot-3.png[scaledwidth=90%]
+image::images/screenshot-3.png
 
 As the `Checkout Credential` configuration was removed in commit (link:https://github.com/jenkinsci/bitbucket-branch-source-plugin/commit/a4c6bf39b83168ff62fc622bd4084ef90cf810c0[a4c6bf3]), you can alternatively add a `Checkout over SSH` behavior in the configuration of Behaviours, so that to configure a seperate SSH credential for checking out sources.
 
-image::images/screenshot-7.png[scaledwidth=90%]
+image::images/screenshot-7.png
 
 === Access Token
 
@@ -154,13 +166,13 @@ First, create a new _access token_ in Bitbucket as instructed in one of the foll
 
 At least allow _read_ access for repositories. If you want the plugin to install the webhooks, allow _Read and write_ access for Webhooks.
 
-image::images/screenshot-16.png[scaledwidth=90%]
+image::images/screenshot-16.png
 
 Then create a new _Secret text_ credential in Jenkins, enter the Bitbucket token generated in the previous steps in the _Secret_ field.
 
 If you want be able to perform git push operation from CLI than you have to setup _write_ access for repositories. Than configure the _Custom user name/e-mail address_ trait with the Repository Access Token email generated when you created the Repository Access Token (for example, [email protected]).
 
-image::images/screenshot-17.png[scaledwidth=90%]
+image::images/screenshot-17.png
 
 === Personal Access Token
 
@@ -190,13 +202,13 @@ The plugin can make use of OAuth credentials (Bitbucket Cloud only) instead of t
 First create a new _OAuth consumer_ in Bitbucket as instructed in the https://confluence.atlassian.com/bitbucket/oauth-on-bitbucket-cloud-238027431.html[Bitbucket OAuth Documentation].
 Don't forget to check _This is a private consumer_ and at least allow _read_ access for repositories and pull requests. If you want the plugin to install the webhooks, also allow _read_ and _write_ access for webhooks.
 
-image::images/screenshot-10.png[scaledwidth=90%]
+image::images/screenshot-10.png
 
 Then create new _Username with password credentials_ in Jenkins, enter the Bitbucket OAuth consumer key in the _Username_ field and the Bitbucket OAuth consumer secret in the _Password_ field.
 
-image::images/screenshot-11.png[scaledwidth=90%]
+image::images/screenshot-11.png
 
-image::images/screenshot-12.png[scaledwidth=90%]
+image::images/screenshot-12.png
 
 [id=bitbucket-mirror-support]
 == Mirror support
@@ -213,14 +225,14 @@ Cloning from the mirror can only be used with native web-hooks since plugin web-
 
 For branches and tags, the mirror sync event is used. Thus, at cloning time, the mirror is already synchronized. However, in the case of a pull request event, there is no such guarantee. The plugin optimistically assumes that the mirror is synced and the required commit hashes exist in the mirrored repository at cloning time. If the plugin can't find the required hashes, it falls back to the primary repository.
 
-image::images/screenshot-13.png[scaledwidth=90%]
+image::images/screenshot-13.png
 
 [id=bitbucket-build-status]
 == Bitbucket build status
 
 When a new job build starts, the plugin send notifications to Bitbucket about the build status. An "In progress" notification is sent after complete the git checkout, another notification is sent at the end of the build, the sent value depends by the build result and the configuration given by the trait.
 
-image::images/screenshot-15.png[scaledwidth=90%]
+image::images/screenshot-15.png
 
 Follow a summary of all possible values:
 

docs/images/screenshot-20.png

@@ -35,8 +35,6 @@
         <!-- Jenkins.MANAGE is still in Beta -->
         <useBeta>true</useBeta>
         <tagNameFormat>@{project.version}</tagNameFormat>
-        <!-- because eclipse generate a JUnit5 run configuration with placeholder not replaced in the jvm argument section! -->
-        <surefire.forkNumber>1</surefire.forkNumber>
     </properties>
 
     <developers>
@@ -259,4 +257,19 @@
         </plugins>
     </build>
 
+    <profiles>
+        <profile>
+            <id>eclipse</id>
+            <activation>
+                <property>
+                    <name>m2e.version</name>
+                </property>
+            </activation>
+            <properties>
+                <!-- because eclipse generate a JUnit5 run configuration with placeholder not replaced in the jvm argument section! -->
+                <surefire.forkNumber>1</surefire.forkNumber>
+                <jenkins.javaAgent />
+            </properties>
+        </profile>
+    </profiles>
 </project>

pom.xml

@@ -221,7 +221,7 @@ public String getServerUrl() {
 
     @DataBoundSetter
     public void setServerUrl(@CheckForNull String serverUrl) {
-        serverUrl = BitbucketEndpointConfiguration.normalizeServerUrl(serverUrl);
+        serverUrl = BitbucketEndpointConfiguration.normalizeServerURL(serverUrl);
         if (serverUrl != null && !StringUtils.equals(this.serverUrl, serverUrl)) {
             this.serverUrl = serverUrl;
             resetId();

src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketSCMNavigator.java

@@ -287,7 +287,7 @@ public String getServerUrl() {
 
     @DataBoundSetter
     public void setServerUrl(@CheckForNull String serverUrl) {
-        String url = BitbucketEndpointConfiguration.normalizeServerUrl(serverUrl);
+        String url = BitbucketEndpointConfiguration.normalizeServerURL(serverUrl);
         if (url == null) {
             url = BitbucketEndpointConfiguration.get().getDefaultEndpoint().getServerUrl();
         }

src/main/java/com/cloudbees/jenkins/plugins/bitbucket/BitbucketSCMSource.java

@@ -214,10 +214,9 @@ boolean checkPathExists(@NonNull String branchOrHash, @NonNull String path)
      *
      * @return the list of webhooks registered in the repository.
      * @throws IOException if there was a network communications error.
-     * @throws InterruptedException if interrupted while waiting on remote communications.
      */
     @NonNull
-    List<? extends BitbucketWebHook> getWebHooks() throws IOException, InterruptedException;
+    List<? extends BitbucketWebHook> getWebHooks() throws IOException;
 
     /**
      * Returns the team of the current owner or {@code null} if the current owner is not a team.

src/main/java/com/cloudbees/jenkins/plugins/bitbucket/api/BitbucketApi.java

@@ -23,6 +23,7 @@
  */
 package com.cloudbees.jenkins.plugins.bitbucket.api;
 
+import edu.umd.cs.findbugs.annotations.Nullable;
 import java.util.List;
 
 /**
@@ -55,4 +56,15 @@ public interface BitbucketWebHook {
      */
     String getUuid();
 
+    /**
+     * Returns the secret used as the key to generate a HMAC digest value sent
+     * in the X-Hub-Signature header at delivery time.
+     *
+     * @return a secret
+     */
+    @Nullable
+    default String getSecret() {
+        return null;
+    }
+
 }

src/main/java/com/cloudbees/jenkins/plugins/bitbucket/api/BitbucketWebHook.java

@@ -34,6 +34,8 @@ public class BitbucketRepositoryHook implements BitbucketWebHook {
 
     private String url;
 
+    private String secret;
+
     private boolean active;
 
     private List<String> events;
@@ -83,4 +85,12 @@ public void setUuid(String uuid) {
         this.uuid = uuid;
     }
 
+    public String getSecret() {
+        return secret;
+    }
+
+    public void setSecret(String secret) {
+        this.secret = secret;
+    }
+
 }

src/main/java/com/cloudbees/jenkins/plugins/bitbucket/client/repository/BitbucketRepositoryHook.java

@@ -24,22 +24,22 @@
 package com.cloudbees.jenkins.plugins.bitbucket.endpoints;
 
 import com.cloudbees.jenkins.plugins.bitbucket.api.BitbucketAuthenticator;
-import com.cloudbees.plugins.credentials.CredentialsMatchers;
-import com.cloudbees.plugins.credentials.CredentialsProvider;
+import com.cloudbees.jenkins.plugins.bitbucket.impl.util.BitbucketCredentials;
 import com.cloudbees.plugins.credentials.common.StandardCredentials;
 import com.cloudbees.plugins.credentials.common.StandardUsernamePasswordCredentials;
-import com.cloudbees.plugins.credentials.domains.URIRequirementBuilder;
 import edu.umd.cs.findbugs.annotations.CheckForNull;
 import edu.umd.cs.findbugs.annotations.NonNull;
 import hudson.Util;
 import hudson.model.AbstractDescribableImpl;
-import hudson.security.ACL;
 import jenkins.authentication.tokens.api.AuthenticationTokens;
 import jenkins.model.Jenkins;
 import org.apache.commons.lang3.StringUtils;
 import org.jenkinsci.plugins.displayurlapi.ClassicDisplayURLProvider;
+import org.jenkinsci.plugins.plaincredentials.StringCredentials;
 import org.kohsuke.stapler.DataBoundSetter;
 
+import static hudson.Util.fixEmptyAndTrim;
+
 /**
  * Represents a {@link BitbucketCloudEndpoint} or a {@link BitbucketServerEndpoint}.
  *
@@ -58,6 +58,17 @@ public abstract class AbstractBitbucketEndpoint extends AbstractDescribableImpl<
     @CheckForNull
     private final String credentialsId;
 
+    /**
+     * {@code true} if and only if Jenkins have to verify the signature of all incoming hooks.
+     */
+    private boolean enableHookSignature;
+
+    /**
+     * The {@link StringCredentials#getId()} of the credentials to use to verify the signature of hooks.
+     */
+    @CheckForNull
+    private String hookSignatureCredentialsId;
+
     /**
      * Jenkins Server Root URL to be used by that Bitbucket endpoint.
      * The global setting from Jenkins.get().getRootUrl()
@@ -76,7 +87,7 @@ public abstract class AbstractBitbucketEndpoint extends AbstractDescribableImpl<
      */
     AbstractBitbucketEndpoint(boolean manageHooks, @CheckForNull String credentialsId) {
         this.manageHooks = manageHooks && StringUtils.isNotBlank(credentialsId);
-        this.credentialsId = manageHooks ? credentialsId : null;
+        this.credentialsId = manageHooks ? fixEmptyAndTrim(credentialsId) : null;
     }
 
     /**
@@ -106,7 +117,7 @@ static String normalizeJenkinsRootUrl(String rootUrl) {
         // This routine is not really BitbucketEndpointConfiguration
         // specific, it just works on strings with some defaults:
         return Util.ensureEndsWith(
-            BitbucketEndpointConfiguration.normalizeServerUrl(rootUrl),"/");
+            BitbucketEndpointConfiguration.normalizeServerURL(rootUrl),"/");
     }
 
     /**
@@ -124,7 +135,7 @@ public String getBitbucketJenkinsRootUrl() {
     @DataBoundSetter
     public void setBitbucketJenkinsRootUrl(String bitbucketJenkinsRootUrl) {
         if (manageHooks) {
-            this.bitbucketJenkinsRootUrl = Util.fixEmptyAndTrim(bitbucketJenkinsRootUrl);
+            this.bitbucketJenkinsRootUrl = fixEmptyAndTrim(bitbucketJenkinsRootUrl);
             if (this.bitbucketJenkinsRootUrl != null) {
                 this.bitbucketJenkinsRootUrl = normalizeJenkinsRootUrl(this.bitbucketJenkinsRootUrl);
             }
@@ -133,6 +144,29 @@ public void setBitbucketJenkinsRootUrl(String bitbucketJenkinsRootUrl) {
         }
     }
 
+    @CheckForNull
+    public String getHookSignatureCredentialsId() {
+        return hookSignatureCredentialsId;
+    }
+
+    @DataBoundSetter
+    public void setHookSignatureCredentialsId(String hookSignatureCredentialsId) {
+        if (enableHookSignature) {
+            this.hookSignatureCredentialsId = fixEmptyAndTrim(hookSignatureCredentialsId);
+        } else {
+            this.hookSignatureCredentialsId = null;
+        }
+    }
+
+    public boolean isEnableHookSignature() {
+        return enableHookSignature;
+    }
+
+    @DataBoundSetter
+    public void setEnableHookSignature(boolean enableHookSignature) {
+        this.enableHookSignature = enableHookSignature;
+    }
+
     /**
      * Jenkins Server Root URL to be used by this Bitbucket endpoint.
      * The global setting from Jenkins.get().getRootUrl()
@@ -220,18 +254,17 @@ public final String getCredentialsId() {
      */
     @CheckForNull
     public StandardCredentials credentials() {
-        return StringUtils.isBlank(credentialsId) ? null : CredentialsMatchers.firstOrNull(
-                CredentialsProvider.lookupCredentialsInItemGroup(
-                        StandardCredentials.class,
-                        Jenkins.get(),
-                        ACL.SYSTEM2 ,
-                        URIRequirementBuilder.fromUri(getServerUrl()).build()
-                ),
-                CredentialsMatchers.allOf(
-                        CredentialsMatchers.withId(credentialsId),
-                        AuthenticationTokens.matcher(BitbucketAuthenticator.authenticationContext(getServerUrl()))
-                )
-        );
+        return BitbucketCredentials.lookupCredentials(getServerUrl(), Jenkins.get(), credentialsId, StandardCredentials.class);
+    }
+
+    /**
+     * Looks up the {@link StringCredentials} to use to verify the signature of hooks.
+     *
+     * @return the credentials or {@code null}.
+     */
+    @CheckForNull
+    public StringCredentials hookSignatureCredentials() {
+        return BitbucketCredentials.lookupCredentials(getServerUrl(), Jenkins.get(), hookSignatureCredentialsId, StringCredentials.class);
     }
 
     /**