Release 0.5.1

gchq · Aug 4, 2021 · 6875429 · 6875429
2 parents 416123c + 771d77a
commit 6875429
Show file tree

Hide file tree

Showing 61 changed files with 651 additions and 670 deletions.
diff --git a/NOTICES.md b/NOTICES.md
@@ -22,9 +22,9 @@ List of third-party dependencies grouped by their license type
 * spring-boot-starter-data-r2dbc ([org.springframework.boot:spring-boot-starter-data-r2dbc:2.3.1.RELEASE](https://spring.io/projects/spring-boot))
 * spring-boot-starter-test ([org.springframework.boot:spring-boot-starter-test:2.3.1.RELEASE](https://spring.io/projects/spring-boot))
 * Spring Shell Starter ([org.springframework.shell:spring-shell-starter:2.0.0.RELEASE](http://projects.spring.io/spring-boot/spring-shell-parent/spring-shell-starter/))
-* GCHQ Palisade - Akka Client ([uk.gov.gchq.palisade:client-akka:0.5.0-RELEASE](https://github.com/gchq/Palisade-clients/tree/develop/client-akka))
-* GCHQ Palisade - Java Client ([uk.gov.gchq.palisade:client-java:0.5.0-RELEASE](https://github.com/gchq/Palisade-clients/tree/develop/client-java))
-* GCHQ Palisade Common Library ([uk.gov.gchq.palisade:common:0.5.0-RELEASE](https://github.com/gchq/Palisade-common))
+* GCHQ Palisade - Akka Client ([uk.gov.gchq.palisade:client-akka:0.5.1-RELEASE](https://github.com/gchq/Palisade-clients/tree/develop/client-akka))
+* GCHQ Palisade - Java Client ([uk.gov.gchq.palisade:client-java:0.5.1-RELEASE](https://github.com/gchq/Palisade-clients/tree/develop/client-java))
+* GCHQ Palisade Common Library ([uk.gov.gchq.palisade:common:0.5.1-RELEASE](https://github.com/gchq/Palisade-common))
 
 ### [Eclipse Public License 1.0](./licenses/eclipse_public_license_1.0.html):
 * Logback Classic Module ([ch.qos.logback:logback-classic:1.2.3](http://logback.qos.ch/logback-classic))

diff --git a/README.md b/README.md
@@ -22,29 +22,40 @@
 
 # Palisade Clients
 
-## Client Implementations
+## Status
+<span style="color:red">Palisade is no longer under active development.</span>
 
-The job of the client code is to send the request for data into Palisade and to interpret the result as required for the data processing technology it is written for.
-The responsibility for implementations of the client code is to provide users with a way to request data from Palisade in a way that the user has to make minimal changes to how they would normally use that processing technology.
-Implementations of this component will usually require deep understanding of the data processing technology in order to best hook into that technology, without needing to fork the code for that technology.
+Windows is not an explicitly supported environment, although where possible Palisade has been made compatible.  
+For Windows developer environments, we recommend setting up [WSL](https://docs.microsoft.com/en-us/windows/wsl/).
 
-To find out more information on our technology specific clients, please see the following modules:
-* [Akka Client](client-akka/README.md)
-* [Fuse Client](client-fuse/README.md)
+For an overview of Palisade, start with the Palisade introduction and the accompanying guides: QuickStart Guide; and Developer Guide which are found in the [Palisade README](https://github.com/gchq/Palisade/README.md).
+
+## Client Implementations
+The Client implementations will need to send requests into Palisade and process the response.  
+The workflow for this is expected to follow the pattern of first sending in a request to register access for data.  
+This will return a response consisting of a token (a unique id) that can then be used in a second request to gather the resources.
+With the resources gathered, the client is then in a position to read the data.
+An implementation of a Client will need to conform to this workflow, but the technology that is used will be best suited to the intended use of the application.
+
+The following examples demonstrate the use of different kinds of clients that can operate with the Palisade service:
 * [Java Client](client-java/README.md)
+  Java based RESTFul client using an API similar to the JDBC  
 * [Shell Client](client-shell/README.md)
-
+  Command Line Interface(CLI) client which provides interactive operations.
+* [Fuse Client](client-fuse/README.md)
+  Filesystem in Userspace (FUSE) client which uses a CLI for interactive operations and stores the results in a FUSE filesystem.
+  This client is an extension of the Java Client.
+* [Akka Client](client-akka/README.md)
+  Akka client which provides access to an interface using both Java and Akka types.
+  This client is an extension of the Java Client.
+* [S3 Client](client-s3/README.md)
+  Akka Microservice which provides an endpoint for accessing resources stored in an Amazon Web Service (AWS) Simple Cloud Storage (S3) bucket.  
+  The example uses [Apache Spark](https://spark.apache.org)for RESTful queries with the microservice.
 
 ### Prerequisites
 1. [Git](https://git-scm.com/)
 2. [Maven](https://maven.apache.org/)
 
-The examples may have additional prerequisites
-
-<span style="color:red">
-We do not currently support Windows as a build environment, If you are running on Windows then you will need this: Microsoft Visual C++ 2010 SP1 Redistributable Package
-</span>
-
 
 ## Getting started
 
@@ -65,18 +76,12 @@ You should see the following modules:
 ```
 Now you can finally build the repository by running: 
 ```bash
-mvn install
+mvn clean install
 ```
 
-
 ## License
 
 Palisade-clients is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0) and is covered by [Crown Copyright](https://www.nationalarchives.gov.uk/information-management/re-using-public-sector-information/copyright-and-re-use/crown-copyright/).
 
 ## Contributing
 We welcome contributions to the project. Detailed information on our ways of working can be found [here](https://gchq.github.io/Palisade/doc/other/ways_of_working.html).
-
-## FAQ
-
-Q: What versions of Java are supported?  
-A: We are currently using Java 11.
diff --git a/client-akka/mvn_dependency_tree.txt b/client-akka/mvn_dependency_tree.txt
@@ -1,15 +1,8 @@
-uk.gov.gchq.palisade:client-akka:jar:0.5.0-RELEASE
-+- uk.gov.gchq.palisade:common:jar:0.5.0-RELEASE:compile
+uk.gov.gchq.palisade:client-akka:jar:0.5.1-RELEASE
++- uk.gov.gchq.palisade:common:jar:0.5.1-RELEASE:compile
 |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.11.0:compile
 |  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.11.0:compile
-|  \- org.apache.avro:avro:jar:1.8.2:compile
-|     +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
-|     +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
-|     +- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
-|     +- org.xerial.snappy:snappy-java:jar:1.1.1.3:compile
-|     +- org.apache.commons:commons-compress:jar:1.8.1:compile
-|     +- org.tukaani:xz:jar:1.5:compile
-|     \- org.slf4j:slf4j-api:jar:1.7.30:compile
+|  \- org.slf4j:slf4j-api:jar:1.7.30:compile
 +- com.typesafe.akka:akka-stream_2.13:jar:2.6.10:compile
 |  +- org.scala-lang:scala-library:jar:2.13.3:compile
 |  +- com.typesafe.akka:akka-actor_2.13:jar:2.6.10:compile

diff --git a/client-akka/pom.xml b/client-akka/pom.xml
@@ -22,7 +22,7 @@
     <parent>
         <groupId>uk.gov.gchq.palisade</groupId>
         <artifactId>clients</artifactId>
-        <version>0.5.0-${revision}</version>
+        <version>0.5.1-${revision}</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
     <modelVersion>4.0.0</modelVersion>
@@ -67,7 +67,7 @@
             <!-- Common types (User, Resource, Context) -->
             <groupId>uk.gov.gchq.palisade</groupId>
             <artifactId>common</artifactId>
-            <version>0.5.0-${common.revision}</version>
+            <version>0.5.1-${common.revision}</version>
         </dependency>
         <!--
         ~ Akka and Alpakka (Kafka)

diff --git a/client-fuse/mvn_dependency_tree.txt b/client-fuse/mvn_dependency_tree.txt
@@ -1,4 +1,4 @@
-uk.gov.gchq.palisade:client-fuse:jar:0.5.0-RELEASE
+uk.gov.gchq.palisade:client-fuse:jar:0.5.1-RELEASE
 +- com.github.serceman:jnr-fuse:jar:0.5.5:compile
 |  +- com.github.jnr:jnr-ffi:jar:2.1.12:compile
 |  |  +- com.github.jnr:jffi:jar:1.2.23:compile
@@ -12,22 +12,17 @@ uk.gov.gchq.palisade:client-fuse:jar:0.5.0-RELEASE
 |  |  \- com.github.jnr:jnr-x86asm:jar:1.0.2:compile
 |  +- com.github.jnr:jnr-posix:jar:3.0.54:compile
 |  \- com.github.jnr:jnr-constants:jar:0.9.15:compile
-+- uk.gov.gchq.palisade:client-java:jar:0.5.0-RELEASE:compile
-|  +- uk.gov.gchq.palisade:common:jar:0.5.0-RELEASE:compile
-|  |  \- org.apache.avro:avro:jar:1.8.2:compile
-|  |     +- org.codehaus.jackson:jackson-core-asl:jar:1.9.13:compile
-|  |     +- org.codehaus.jackson:jackson-mapper-asl:jar:1.9.13:compile
-|  |     +- com.thoughtworks.paranamer:paranamer:jar:2.7:compile
-|  |     +- org.xerial.snappy:snappy-java:jar:1.1.1.3:compile
-|  |     +- org.apache.commons:commons-compress:jar:1.8.1:compile
-|  |     \- org.tukaani:xz:jar:1.5:compile
++- uk.gov.gchq.palisade:client-java:jar:0.5.1-RELEASE:compile
+|  +- uk.gov.gchq.palisade:common:jar:0.5.1-RELEASE:compile
 |  +- com.fasterxml.jackson.core:jackson-databind:jar:2.11.0:compile
 |  |  +- com.fasterxml.jackson.core:jackson-annotations:jar:2.11.0:compile
 |  |  \- com.fasterxml.jackson.core:jackson-core:jar:2.11.0:compile
 |  +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.11.0:compile
 |  \- io.reactivex.rxjava3:rxjava:jar:3.0.8:compile
 |     \- org.reactivestreams:reactive-streams:jar:1.0.3:compile
 +- org.slf4j:slf4j-api:jar:1.7.26:compile
++- ch.qos.logback:logback-classic:jar:1.2.3:compile
+|  \- ch.qos.logback:logback-core:jar:1.2.3:compile
 +- org.junit.jupiter:junit-jupiter:jar:5.7.0:test
 |  +- org.junit.jupiter:junit-jupiter-api:jar:5.6.2:test
 |  |  +- org.apiguardian:apiguardian-api:jar:1.1.0:test

diff --git a/client-fuse/pom.xml b/client-fuse/pom.xml
@@ -22,7 +22,7 @@
     <parent>
         <groupId>uk.gov.gchq.palisade</groupId>
         <artifactId>clients</artifactId>
-        <version>0.5.0-${revision}</version>
+        <version>0.5.1-${revision}</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
     <modelVersion>4.0.0</modelVersion>
@@ -59,6 +59,7 @@
 
         <jnr-fuse.version>0.5.5</jnr-fuse.version>
         <slf4j.version>1.7.26</slf4j.version>
+        <logback.version>1.2.3</logback.version>
 
         <junit.jupiter.version>5.7.0</junit.jupiter.version>
         <assertj.version>3.19.0</assertj.version>
@@ -80,6 +81,11 @@
             <artifactId>slf4j-api</artifactId>
             <version>${slf4j.version}</version>
         </dependency>
+        <dependency>
+            <groupId>ch.qos.logback</groupId>
+            <artifactId>logback-classic</artifactId>
+            <version>${logback.version}</version>
+        </dependency>
 
         <dependency>
             <groupId>org.junit.jupiter</groupId>
@@ -151,6 +157,30 @@
                     </execution>
                 </executions>
             </plugin>
+            <plugin>
+                <groupId>org.apache.maven.plugins</groupId>
+                <artifactId>maven-assembly-plugin</artifactId>
+                <executions>
+                    <execution>
+                        <id>make-assembly</id>
+                        <phase>package</phase>
+                        <goals>
+                            <goal>single</goal>
+                        </goals>
+                    </execution>
+                </executions>
+                <configuration>
+                    <archive>
+                        <manifest>
+                            <addClasspath>true</addClasspath>
+                            <mainClass>uk.gov.gchq.palisade.client.fuse.FuseClient</mainClass>
+                        </manifest>
+                    </archive>
+                    <descriptorRefs>
+                        <descriptorRef>jar-with-dependencies</descriptorRef>
+                    </descriptorRefs>
+                </configuration>
+            </plugin>
         </plugins>
     </build>
 

diff --git a/client-fuse/src/main/java/uk/gov/gchq/palisade/client/fuse/FuseClient.java b/client-fuse/src/main/java/uk/gov/gchq/palisade/client/fuse/FuseClient.java
@@ -26,6 +26,7 @@
 import uk.gov.gchq.palisade.client.java.internal.dft.DefaultClient;
 
 import java.io.InputStream;
+import java.net.URI;
 import java.nio.file.Path;
 import java.nio.file.Paths;
 import java.util.Arrays;
@@ -70,6 +71,7 @@ public FuseClient(final String clientUri) {
      */
     public static void main(final String... args) {
         String jarName = args[0];
+
         if (args.length >= MIN_ARGS_LEN) {
             // Parse command-line args
             Map<String, String> context = new HashMap<>();
@@ -108,7 +110,7 @@ public void mount(final String resourceId, final String mountDir, final Map<Stri
 
         ResourceTreeWithContext tree = client.register(resourceId, context);
         Function<LeafResourceNode, InputStream> reader = node -> client.read(tree.getToken(), node);
-        ResourceTreeFS fuseFs = new ResourceTreeFS(tree, reader);
+        ResourceTreeFS fuseFs = new ResourceTreeFS(tree, reader, URI.create(resourceId).getScheme());
 
         try {
             LOGGER.info("Mounted at {}, press <Ctrl-C> to unmount and exit", mountPath);

diff --git a/...ent/fuse/client/OnNextStubSubscriber.java → .../client/AbstractOnNextStubSubscriber.java b/...ent/fuse/client/OnNextStubSubscriber.java → .../client/AbstractOnNextStubSubscriber.java
@@ -18,20 +18,43 @@
 
 import java.util.concurrent.Flow.Subscriber;
 import java.util.concurrent.Flow.Subscription;
+import java.util.function.Consumer;
 
 /**
  * Stub the Subscriber interface such that it only requires an {@link Subscriber#onNext(Object)}
  * method.
  *
  * @param <T> type of elements emitted by the subscriber
  */
-public interface OnNextStubSubscriber<T> extends Subscriber<T> {
-    default void onSubscribe(final Subscription subscription) {
+public abstract class AbstractOnNextStubSubscriber<T> implements Subscriber<T> {
+    Subscription subscription;
+
+    @Override
+    public void onSubscribe(final Subscription subscription) {
+        this.subscription = subscription;
+        this.subscription.request(1);
+    }
+
+    @Override
+    public void onNext(final T t) {
+        this.subscription.request(1);
+    }
+
+    @Override
+    public void onError(final Throwable throwable) {
     }
 
-    default void onError(final Throwable throwable) {
+    @Override
+    public void onComplete() {
     }
 
-    default void onComplete() {
+    static <T> AbstractOnNextStubSubscriber<T> fromOnNextMethod(final Consumer<T> onNext) {
+        return new AbstractOnNextStubSubscriber<T>() {
+            @Override
+            public void onNext(final T t) {
+                onNext.accept(t);
+                super.onNext(t);
+            }
+        };
     }
 }