Add spent addresses export test #1561

DyrellC · 2019-08-04T18:38:40Z

Description

Adds a regression test for exporting and importing spentAddresses.txt files using the export-spent and merge-spent iri-extension tools. The tests export the spent addresses of one node into a file and checks to ensure that they have been exported correctly. It then imports the same addresses into an empty node and checks that the addresses were added as spent addresses.

**Note: This test currently only tests with one spent address db to test and is an incomplete PR until further notice. It should not be merged until the additional test has been added, but needs to be tested in the build pipeline to ensure it is working correctly.

Fixes #1468

Type of change

Enhancement (a non-breaking change which adds functionality)

How Has This Been Tested?

Locally tested prior to change in file location
Testing on buildkite pipeline to ensure pod file locations are correct **

Checklist:

My code follows the style guidelines for this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas

GalRogozinski · 2020-01-08T06:07:18Z

python-regression/IXI/export-spent/index.js

+    return Response.create({
+        amount: hashes.length,
+        fileName: fileName,
+        sizeInMb: new java.lang.Integer(file.length() / (1024 * 1024)),


Issue found: 'java' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:19Z

python-regression/IXI/merge-spent/index.js

+    }
+
+    return {
+        amount: amount


Issue found: Expected property shorthand. (object-shorthand)

GalRogozinski · 2020-01-08T06:07:20Z

python-regression/tests/features/steps/local_snapshots_steps.py

+    :param node: The node that the spent addresses will be merged into
+    """
+    args = step.hashes
+    for x in range(len(args)):


Issue found: Consider using enumerate instead of iterating with range and len

GalRogozinski · 2020-01-08T06:07:21Z

python-regression/IXI/export-spent/index.js

+        writer.write(hashes.length.toString() + separator);
+
+        // Write spent addresses
+        for (var i=0; i<hashes.length; i++){


Issue found: 'i' is already defined. (no-redeclare)

GalRogozinski · 2020-01-08T06:07:22Z

python-regression/IXI/export-spent/index.js

+        var digest = java.security.MessageDigest.getInstance("SHA-256");
+        for (var i=0; i<hashes.length; i++){
+            // Update with hash bytes
+            digest.update(hashes[i].bytes());


Issue found: Generic Object Injection Sink (security/detect-object-injection)

GalRogozinski · 2020-01-08T06:07:24Z

python-regression/IXI/merge-spent/index.js

+
+var spentAddressProvider = IOTA.spentAddressesProvider
+
+var iri = com.iota.iri;


Issue found: 'com' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:25Z

python-regression/IXI/export-spent/index.js

+    });
+}
+
+API.put("generateSpentAddressesFile", new Callable({ call: generate }))


Issue found: Missing semicolon. (semi)

GalRogozinski · 2020-01-08T06:07:26Z

python-regression/IXI/export-spent/index.js

+
+        // Write spent addresses
+        for (var i=0; i<hashes.length; i++){
+            var hash = hashes[i].toString();


Issue found: Generic Object Injection Sink (security/detect-object-injection)

GalRogozinski · 2020-01-08T06:07:27Z

python-regression/IXI/merge-spent/index.js

@@ -0,0 +1,147 @@
+var System = java.lang.System;


Issue found: 'java' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:28Z

python-regression/IXI/merge-spent/index.js

+                }
+            } else if (line.length() !== 64) {
+                // This must be the amount
+                amount = Number(line);


Issue found: 'line' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:29Z

python-regression/IXI/merge-spent/index.js

+                }
+            }
+        } else {
+            log.error("File {} did not exist on disk.. ignoring!", fileName)


Issue found: Missing semicolon. (semi)

GalRogozinski · 2020-01-08T06:07:30Z

python-regression/IXI/export-spent/index.js

+    });
+}
+
+API.put("generateSpentAddressesFile", new Callable({ call: generate }))


Issue found: 'API' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:31Z

python-regression/IXI/merge-spent/index.js

+
+        log.error("Caught error during read: {}", error);
+        return {
+            error: error


Issue found: Expected property shorthand. (object-shorthand)

GalRogozinski · 2020-01-08T06:07:32Z

python-regression/IXI/export-spent/index.js

+    var checksum;
+    try {
+        // False for always overwriting
+        writer = new java.io.FileWriter(file, false); 


Issue found: 'java' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:33Z

python-regression/IXI/merge-spent/index.js

+    });
+}
+
+API.put("mergeSpentAddresses", new Callable({ call: generate }))


Issue found: Missing semicolon. (semi)

GalRogozinski · 2020-01-08T06:07:34Z

python-regression/IXI/merge-spent/index.js

+var defaultSpentFileName = "spentAddresses.txt";
+
+// Log using logger of the ixi class
+var log = org.slf4j.LoggerFactory.getLogger(iri.IXI.class);


Issue found: 'org' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:35Z

python-regression/IXI/merge-spent/index.js

@@ -0,0 +1,147 @@
+var System = java.lang.System;
+
+var spentAddressProvider = IOTA.spentAddressesProvider


Issue found: 'IOTA' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:37Z

python-regression/IXI/merge-spent/index.js

+        return false;
+    }
+
+    return expectedChecksum !== checksumFromBytes(digest.digest());


Issue found: 'checksumFromBytes' was used before it was defined. (no-use-before-define)

GalRogozinski · 2020-01-08T06:07:38Z

python-regression/IXI/export-spent/index.js

@@ -0,0 +1,79 @@
+var System = java.lang.System;


Issue found: 'java' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:39Z

python-regression/IXI/export-spent/index.js

+        amount: hashes.length,
+        fileName: fileName,
+        sizeInMb: new java.lang.Integer(file.length() / (1024 * 1024)),
+        checksum: checksum


Issue found: Expected property shorthand. (object-shorthand)

GalRogozinski · 2020-01-08T06:07:40Z

python-regression/IXI/merge-spent/index.js

+        while (line = targetReader.readLine()){
+            if (firstLine){
+                firstLine = false;
+            } else if (line.length() === 64){


Issue found: 'line' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:41Z

python-regression/IXI/merge-spent/index.js

+    for (var i=0; i<files.length; i++){
+        var fileName = files[i];
+
+        var file = new java.io.File(fileName);


Issue found: 'java' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:42Z

python-regression/IXI/export-spent/index.js

@@ -0,0 +1,79 @@
+var System = java.lang.System;
+
+var spentAddressProvider = IOTA.spentAddressesProvider


Issue found: 'IOTA' is not defined. (no-undef)

GalRogozinski · 2020-01-08T06:07:42Z

python-regression/IXI/export-spent/index.js

+        var digest = java.security.MessageDigest.getInstance("SHA-256");
+        for (var i=0; i<hashes.length; i++){
+            // Update with hash bytes
+            digest.update(hashes[i].bytes());


Issue found: 'i' used outside of binding context. (block-scoped-var)

GalRogozinski · 2020-01-08T06:07:43Z

python-regression/IXI/export-spent/index.js

@@ -0,0 +1,79 @@
+var System = java.lang.System;
+
+var spentAddressProvider = IOTA.spentAddressesProvider


Issue found: Missing semicolon. (semi)

DyrellC added 13 commits January 14, 2019 19:36

Merge branch 'dev' of http://github.com/iotaledger/iri into dev

e37d3aa

Merge branch 'dev' of http://github.com/iotaledger/iri into dev

664da99

Merge branch 'dev' of http://github.com/iotaledger/iri into dev

1c7c1c2

Merge branch 'dev' of http://github.com/iotaledger/iri into dev

f486e4e

Merge branch 'dev' of https://github.com/DyrellC/iri into dev

806dd8e

Merge branch 'dev' of http://github.com/iotaledger/iri into dev

0b3d0e7

Merge branch 'dev' of https://github.com/iotaledger/iri into dev

d631628

Merge branch 'dev' of http://github.com/iotaledger/iri into dev

f9c824b

Merge branch 'dev' of http://github.com/iotaledger/iri into dev

9a454a2

Export spent addresses test initial commit

2c7db76

Added ixi modules for spent export/merge

3476040

Add spentAddresses.txt and modify location for tiab

d887852

Add ixis to config

05d59bf

iotaledger deleted a comment Aug 4, 2019

DyrellC added 5 commits January 7, 2020 16:54

Add log for working directory in kubernetes

cb0a1e5

Further logs for working directory debug in kubernetes

d7f4a6d

print directory contents

5feb1b9

Search for spentAddresses file

bf590c8

Revert back to kubernetes

6bb15f1

GalRogozinski reviewed Jan 8, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add spent addresses export test #1561

Add spent addresses export test #1561

DyrellC commented Aug 4, 2019

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020

GalRogozinski Jan 8, 2020


		var spentAddressProvider = IOTA.spentAddressesProvider

		var iri = com.iota.iri;

		@@ -0,0 +1,147 @@
		var System = java.lang.System;

		var spentAddressProvider = IOTA.spentAddressesProvider

		@@ -0,0 +1,79 @@
		var System = java.lang.System;

		var spentAddressProvider = IOTA.spentAddressesProvider

Add spent addresses export test #1561

Are you sure you want to change the base?

Add spent addresses export test #1561

Conversation

DyrellC commented Aug 4, 2019

Description

Type of change

How Has This Been Tested?

Checklist:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment