Gh-380: Update user guide querying section (#393)

* Added in new user guide section * Added new development guide structure * Added in new reference structure * Added new administration guide structure * Added new change notes structure * Added example deployment to developer docs * Added Endpoints guide * add pages on filtering and operations * add section on transformation * add aggregation section * tidy the import/export page * add examples for java and python operations --------- Co-authored-by: l46978 <[email protected]> Co-authored-by: cn337131 <[email protected]> Co-authored-by: GCHQDeveloper314 <[email protected]>
gchq · Sep 21, 2023 · a515d2f · a515d2f
1 parent c295e33
commit a515d2f
Show file tree

Hide file tree

Showing 19 changed files with 1,032 additions and 234 deletions.
diff --git a/docs/change-notes/changelist/v2-changes.md b/docs/change-notes/changelist/v2-changes.md
@@ -1,37 +1,37 @@
-# Gaffer 2 Changelist  
+# Gaffer 2 Changelist
 
-Below is a summary of changes that have been made in Gaffer version 2.   
+Below is a summary of changes that have been made in Gaffer version 2.
 
 ### Accumulo 2 Support
 The Accumulo store now supports Accumulo 2 and Hadoop 3 by default, with support for Accumulo 1 and Hadoop 2 retained. See the [Accumulo Migration page](../migrating-from-v1-to-v2/accumulo-migration.md) for more information about this change.
 
 ### Federated Store Improvements
-The Federated Operation was added to greatly improve flexibility of using a Federated Store.  
+The Federated Operation was added to greatly improve flexibility of using a Federated Store.
 !!! danger "Breaking change"
     To migrate, please see the [Federated Store Changes page](../migrating-from-v1-to-v2/federation-changes.md).
 
 ### Cache Improvements and fixes
 All "caches" within Gaffer received a lot of bug fixes which should make them significantly more stable and consistent over time. This should improve usability of FederatedStores, NamedOperations and NamedViews.
 !!! danger "Breaking change"
-    The cache will need to be reloaded, as the new internal cache interface has changed.  To do this, export all of the contents of your cache, upgrade, then re-add everything manually.  
+    The cache will need to be reloaded, as the new internal cache interface has changed.  To do this, export all of the contents of your cache, upgrade, then re-add everything manually.
 
 ### Removal of Deprecated code
-All of Gaffer 1's deprecated code has been removed.  
+All of Gaffer 1's deprecated code has been removed.
 !!! danger "Breaking change"
     To migrate, please see the [deprecations](../migrating-from-v1-to-v2/deprecations.md) page.
 
 ### Dependency Upgrades
-Dependencies have been updated, where possible to the latest version, removing vulnerabilities.  
+Dependencies have been updated, where possible to the latest version, removing vulnerabilities.
 !!! danger "Breaking change"
     You will need to migrate your dependencies to be compatible with Gaffer 2's new dependency versions. Please see the [dependencies](../migrating-from-v1-to-v2/dependencies.md) page for full details.
 
 ### Federated and Proxy store fixes
-A lot of bugs have been fixed that should facilitate FederatedStores with ProxyStores in them.  
+A lot of bugs have been fixed that should facilitate FederatedStores with ProxyStores in them.
 !!! danger "Breaking change"
     The unique store trait `DYNAMIC_SCHEMA` has been removed from Gaffer. Simply removing it from custom FederatedStore implementations should be an adequate fix.
 
 ### Removal of CloseableIterable
-The `CloseableIterable` class has been removed so Operations like `GetAllElements` now return an `Iterable` instead, but the result still implements `Closeable`.   
+The `CloseableIterable` class has been removed so Operations like `GetAllElements` now return an `Iterable` instead, but the result still implements `Closeable`.
 !!! danger "Breaking change"
     Everywhere `CloseableIterable` was used in client code should be replaced with an `Iterable`:
     ```java
@@ -42,9 +42,9 @@ The `CloseableIterable` class has been removed so Operations like `GetAllElement
     ```
 
 ### Removal of HBase and Parquet stores
-The HBase and Parquet stores have been removed from Gaffer in version 2. We made posts for both the [HBase](https://github.com/gchq/Gaffer/issues/2367) and [Parquet](https://github.com/gchq/Gaffer/discussions/2557) stores to understand the levels of usage. It was then decided to remove both stores as this would make introducing various improvements easier in the long term. HBase and Parquet remain available in Gaffer version 1. In the future, they could be reimplemented for Gaffer 2, though we do not plan to currently.  
+The HBase and Parquet stores have been removed from Gaffer in version 2. We made posts for both the [HBase](https://github.com/gchq/Gaffer/issues/2367) and [Parquet](https://github.com/gchq/Gaffer/discussions/2557) stores to understand the levels of usage. It was then decided to remove both stores as this would make introducing various improvements easier in the long term. HBase and Parquet remain available in Gaffer version 1. In the future, they could be reimplemented for Gaffer 2, though we do not plan to currently.
 !!! danger "Breaking change"
-    We would recommend instead using an Accumulo Store. If you would like these store implementations in Gaffer 2, or any other potential store for that matter, please [make an issue](https://github.com/gchq/Gaffer/issues/new?assignees=&labels=enhancement&projects=&template=feature_request.md&title=) on GitHub.  
+    We would recommend instead using an Accumulo Store. If you would like these store implementations in Gaffer 2, or any other potential store for that matter, please [make an issue](https://github.com/gchq/Gaffer/issues/new?assignees=&labels=enhancement&projects=&template=feature_request.md&title=) on GitHub.
 
 ### Gaffer now builds with Java 8 and Java 11
 There is now a maven profile that will swap dependency versions so you can build Gaffer with Java 11. The code has also been updated to build with both Java versions.
@@ -53,7 +53,7 @@ There is now a maven profile that will swap dependency versions so you can build
 The Accumulo store now supports authenticating to Accumulo and HDFS using Kerberos, in addition to username/password. For more information, see the [Kerberos support page](../migrating-from-v1-to-v2/accumulo-kerberos.md).
 
 ### CSV Import and Export
-Basic support for importing and exporting [CSVs](../../user-guide/query/api-querying/import-export-data.md) has been added.
+Basic support for importing and exporting [CSVs](../../user-guide/query/gaffer-syntax/import-export/csv.md) has been added.
 
 ### All operations can now be used within NamedOperations
 Previously, `GetElementsBetweenSets` could not be used within a NamedOperation as it used `inputB`. `GetElementsBetweenSets` and `inputB` have both been deprecated and instead you should use `GetElementsBetweenSetsPairs`.
@@ -127,4 +127,4 @@ This will mean subgraphs added to FederatedStores can have additional operation
     ```
 
     1. Schema left empty for brevity
-    2. This example operation enables file import. Read more in the [CSV](../../user-guide/query/api-querying/import-export-data.md) docs.
+    2. This example operation enables file import. Read more in the [CSV](../../user-guide/query/gaffer-syntax/import-export/csv.md) docs.
diff --git a/docs/change-notes/migrating-from-v1-to-v2/deprecations.md b/docs/change-notes/migrating-from-v1-to-v2/deprecations.md
@@ -464,7 +464,7 @@ Now set in `serialiser`:
                     .library(myGraphLibrary)
                     .view(myView)
                     .addHook(customHook)
-                    .build()) 
+                    .build())
             .addSchema(mySchema) // (1)!
             .storeProperties(storeProperties) // (2)!
             .build();

diff --git a/docs/development-guide/example-deployment/project-setup.md b/docs/development-guide/example-deployment/project-setup.md
@@ -43,7 +43,7 @@ the graph in [Neo4j syntax](https://neo4j.com/labs/apoc/4.4/export/csv/#export-d
 !!! note ""
     Please note that Gaffer often requires additional information about the data such as,
     `:String` on the column headers to help with typing of the values. This is demonstrated below
-    in the raw file. There's more detail on this in the [OpenCypher documentation](../../user-guide/query/api-querying/import-export-data.md#opencypher-formats).
+    in the raw file. There's more detail on this in the [OpenCypher documentation](../../user-guide/query/gaffer-syntax/import-export/csv.md#opencypher-formats).
 
 === "Table"
     | _id | name  | age | lang | _labels  | _start | _end | _type   | weight |

diff --git a/docs/development-guide/example-deployment/using-the-api.md b/docs/development-guide/example-deployment/using-the-api.md
@@ -1,8 +1,9 @@
 # Using the API
 
-As covered in the [Getting Started API page](../../user-guide/api.md) the main POST request used in the API is
-`/graph/operations/execute`. This part of the guide will cover the general usage of this part of the
-API and walk through some general operations you might want to use to load data or query.
+As covered in the [Getting Started API page](../../user-guide/apis/rest-api.md) the main POST
+request used in the API is `/graph/operations/execute`. This part of the guide will cover the
+general usage of this part of the API and walk through some general operations you might want to
+use to load data or query.
 
 !!! note
     The [Gaffer operations reference guide](../../reference/operations-guide/operations.md)
@@ -61,7 +62,7 @@ from the neo4j formatted CSV file.
 
 The operation chain above essentially mirrors the stages in the previous diagram. In the example
 chain we first ingest the data via the `ImportFromLocalFile` class (an additional operation we added
-via the `operationsDeclarations.json`, which
+via the `operationsDeclarations.json`), which
 streams the data from the CSV file into the next `GenerateElements` operation.
 
 For the generator we have selected the built in `Neo4jCsvElementGenerator` class, this is already

diff --git a/docs/reference/operations-guide/misc.md b/docs/reference/operations-guide/misc.md
@@ -29,7 +29,7 @@ Gets data from an endpoint. [Javadoc](https://gchq.github.io/Gaffer/uk/gov/gchq/
     ```
 
 === "JSON"
-    
+
     ``` json
     {
     "class" : "GetFromEndpoint",
@@ -38,10 +38,10 @@ Gets data from an endpoint. [Javadoc](https://gchq.github.io/Gaffer/uk/gov/gchq/
     ```
 
 === "Python"
-    
+
     ``` python
-    g.GetFromEndpoint( 
-    endpoint="https://mydata.io:8443/mydata" 
+    g.GetFromEndpoint(
+    endpoint="https://mydata.io:8443/mydata"
     )
     ```
 
@@ -68,7 +68,7 @@ This operation requires Store configuration to be set before it can be used. See
     ```
 
 === "JSON"
-    
+
     ``` json
     {
     "class" : "ScoreOperationChain",
@@ -89,34 +89,34 @@ This operation requires Store configuration to be set before it can be used. See
     ```
 
 === "Python"
-    
+
     ``` python
-    g.ScoreOperationChain( 
-    operation_chain=g.OperationChain( 
-        operations=[ 
-        g.GetElements(), 
-        g.NamedOperation( 
-            operation_name="namedOp" 
-        ), 
-        g.Limit( 
-            result_limit=3, 
-            truncate=True 
-        ) 
-        ] 
-    ) 
+    g.ScoreOperationChain(
+    operation_chain=g.OperationChain(
+        operations=[
+        g.GetElements(),
+        g.NamedOperation(
+            operation_name="namedOp"
+        ),
+        g.Limit(
+            result_limit=3,
+            truncate=True
+        )
+        ]
+    )
     )
     ```
 
 Results:
 
 === "Java"
-    
+
     ``` java
     5
     ```
 
 === "JSON"
-    
+
     ``` json
     5
     ```
@@ -140,7 +140,7 @@ Here we have added a NamedOperation to the NamedOperationCache, with a custom sc
     ```
 
 === "JSON"
-    
+
     ``` json
     {
     "class" : "ScoreOperationChain",
@@ -164,38 +164,38 @@ Here we have added a NamedOperation to the NamedOperationCache, with a custom sc
     ```
 
 === "Python"
-    
+
     ``` python
-    g.ScoreOperationChain( 
-    operation_chain=g.OperationChain( 
-        operations=[ 
-        g.NamedOperation( 
-            input=[ 
-            g.EntitySeed( 
-                vertex=1 
-            ) 
-            ], 
-            operation_name="1-hop" 
-        ), 
-        g.Limit( 
-            result_limit=3, 
-            truncate=True 
-        ) 
-        ] 
-    ) 
+    g.ScoreOperationChain(
+    operation_chain=g.OperationChain(
+        operations=[
+        g.NamedOperation(
+            input=[
+            g.EntitySeed(
+                vertex=1
+            )
+            ],
+            operation_name="1-hop"
+        ),
+        g.Limit(
+            result_limit=3,
+            truncate=True
+        )
+        ]
+    )
     )
     ```
 
 Results:
 
 === "Java"
-    
+
     ``` java
     5
     ```
 
 === "JSON"
-    
+
     ``` json
     5
     ```
diff --git a/docs/user-guide/api.md b/docs/user-guide/api.md