Support validation via a web service, like a function.

[davidasnider]

Assuming I want to validate a particular yaml or json file against a schema. If I have a webservice that accepts a payload with the yaml/json, the web service returns a 2XX for succes and 4xx for failure, Sourcehawk could record the result appropriately. This would open many doors for possible scans/validations.

[brianwyka]

I'll begin working on this @davidasnider. The configuration will look something like this:

file-protocols:
    - name: HTTP Function Validated
      repository-path: path/to/file.txt
      enforcers:
        - enforcer: .http.ContentHttpValidated
          url: http://function.in.cloud/validate
          connect-timeout: 200
          read-timeout: 2000

It will send the entire contents of the file as the request body via method POST to the configured url.

It will be sending the request body as Content-Type = text/plain and also expecting response body as Accept = text/plain.

Anything in 2xx range will be considered successful, 4xx failed validation, and 5xx another problem

Sound good?

[ctoestreich]

@brianwyka time to make a new protocol type? ie. file-protocols web-protocols etc. I don't know that it matters, but open for discussion if this adds value or just makes more code for no value.

[brianwyka]

Yes, something like that @ctoestreich. It's still a file-protocol since it is validating a file.

I've got something cooking already:

package com.optum.sourcehawk.enforcer.file.http;

import com.fasterxml.jackson.databind.annotation.JsonDeserialize;
import com.optum.sourcehawk.enforcer.EnforcerResult;
import com.optum.sourcehawk.enforcer.file.AbstractFileEnforcer;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.NonNull;
import lombok.val;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.stream.Collectors;

/**
 * A file enforcer which makes an HTTP request to the configured URL
 * to validate the file contents
 *
 * @author brianwyka
 */
@Builder(builderClassName = "Builder")
@JsonDeserialize(builder = ContentHttpValidated.Builder.class)
@AllArgsConstructor(staticName = "validated")
public class ContentHttpValidated extends AbstractFileEnforcer {

    private static final int DEFAULT_BUFFER_SIZE = 8192;
    private static final String HTTP_RESPONSE_VALIDATION_MESSAGE_TEMPLATE = "Validation failed with reason: %s";
    private static final String HTTP_RESPONSE_ERROR_MESSAGE_TEMPLATE = "HTTP [%s] error performing validation: %s";

    /**
     * The URL in which to make a POST request, to validate the file contents
     */
    private final String url;

    /**
     * The amount of time allowed to connect to URL before timing out
     */
    private final int connectTimeout;

    /**
     * The amount of time allowed to read response from URL before timing out
     */
    private final int readTimeout;

    /** {@inheritDoc} */
    @Override
    protected EnforcerResult enforceInternal(final @NonNull InputStream actualFileInputStream) throws IOException {
        val httpUrlConnection = (HttpURLConnection) new URL(url).openConnection();
        httpUrlConnection.setDoOutput(true);
        httpUrlConnection.setRequestMethod("POST");
        httpUrlConnection.setRequestProperty("Content-Type", "text/plain");
        httpUrlConnection.setRequestProperty("Accept", "text/plain");
        httpUrlConnection.setConnectTimeout(connectTimeout == 0 ? 500 : connectTimeout);
        httpUrlConnection.setReadTimeout(readTimeout == 0 ? 500 : readTimeout);
        try (val outputStream = httpUrlConnection.getOutputStream()) {
            val buffer = new byte[DEFAULT_BUFFER_SIZE];
            int read;
            while ((read = actualFileInputStream.read(buffer, 0, DEFAULT_BUFFER_SIZE)) >= 0) {
                outputStream.write(buffer, 0, read);
            }
        }
        val responseCode = String.valueOf(httpUrlConnection.getResponseCode());
        if (responseCode.startsWith("2")) {
            return EnforcerResult.passed();
        }
        try (val errorStream = httpUrlConnection.getErrorStream();
             val bufferedReader = new BufferedReader(new InputStreamReader(errorStream))) {
            val responseMessage = bufferedReader.lines()
                .collect(Collectors.joining());
            if (responseCode.startsWith("4")) {
                return EnforcerResult.failed(String.format(HTTP_RESPONSE_VALIDATION_MESSAGE_TEMPLATE, responseMessage));
            } else {
                return EnforcerResult.failed(String.format(HTTP_RESPONSE_ERROR_MESSAGE_TEMPLATE, responseCode, responseMessage));
            }
        }
    }

}