Result Handling

Handling Results with ResultTable

In JISA, data can be recorded in structures each known as a ResultTable. In essence, a ResultTable object represents a "table of results". They are created with a fixed number of columns, each with its own title and unit to which data are added, row by row. They can then be used to quickly output your recorded data in CSV format or be given to various GUI elements to graphically live-display your data.

The name ResultTable is an umbrella term that covers all the different types of objects that behave in this way. Current in JISA there are two types of ResultTable that you can create and use:

• ResultList - Holds data in memory
• ResultStream - Writes data directly to a CSV file without holding in memory

Since they both ResultTable objects, they seem to work in exactly the same way from an "outside" perspective, despite that internally the way they store the data is completely different.

Contents

Creating a ResultTable

Top ↑

Regardless of which implementation of ResultTable we are using, we first must start by defining the columns of the table. This is done by creating Column<T> objects, where T is the data type of the column. JISA has the following pre-defined Column<T> types:

// Floating-point number column
Column<Double>  numerical = Column.ofDoubles("Name", "Units");
                          = Column.ofDecimals("Name", "Units"); // Alias

// Integer number column
Column<Integer> integer   = Column.ofIntegers("Name", "Units");

// Long integer number column
Column<Long>    longInt   = Column.ofLongs("Name", "Units");

// Boolean (true/false) column
Column<Boolean> bool      = Column.ofBooleans("Name", "Units");

// Text column
Column<String>  string    = Column.ofStrings("Name", "Units");
                          = Column.ofText("Name", "Units");    // Alias

For instance, if we wanted three numerical columns to hold "Voltage", "Current" and "Frequency" values respectively we could write:

Java

Column<Double> voltage   = Column.ofDecimals("Voltage", "V");
Column<Double> current   = Column.ofDecimals("Current", "A");
Column<Double> frequency = Column.ofDecimals("Frequency", "Hz");

Kotlin

val voltage   = Column.ofDecimals("Voltage", "V")
val current   = Column.ofDecimals("Current", "A")
val frequency = Column.ofDecimals("Frequency", "Hz")

Java

We can then create a ResultList by passing it these columns as arguments:

ResultTable rTable = new ResultList(voltage, current, frequeny);

Kotlin

val rTable = ResultList(voltage, current, frequency)

In effect, we have just created the following blank table (in memory), ready for adding data to:

Voltage [V]	Current [A]	Frequency [Hz]
...	...	...

However, if we are expecting to be dealing with a large amount of data (ie so much that we might run out of memory), or if we want every data-point to be committed to permanent storage as soon as it is taken, then we will want to create a ResultStream instead. This is done almost the same way except that you need to specify the name of the file you want to write to:

Java

ResultTable rStream = new ResultStream("path/to/file.csv", voltage, current, frequency);

Kotlin

val rStream = ResultStream("path/to/file.csv", voltage, current, frequency)

To support legacy code which uses the old version of ResultTable, you can create tables of entirely numerical columns by just specifying the column names:

ResultTable table = new RestultList("Voltage", "Current", "Frequency");

Adding Data

Top ↑

After creating a ResultTable, you can add data to it in one of two ways. Let's take the following example:

val time      = Column.ofLongs("Time", "ms")
val voltage   = Column.ofDecimals("Voltage", "V")
val current   = Column.ofDecimals("Current", "A")
val frequency = Column.ofDecimals("Frequency", "Hz")

val rList = ResultList(time, voltage, current, frequency)

To add a new row, call addRow(...), providing it with a lambda function which takes the new row object and sets its values by calling .set(column, value) like so:

Java

rList.addRow(row -> {
    row.set(time, 100);
    row.set(voltage, 1.0);
    row.set(current, 50e-3);
    row.set(frequency, 5.5);
});

Kotlin

rList.addRow { row ->
    row[time]      = 100
    row[voltage]   = 1.0
    row[current]   = 50e-3
    row[frequency] = 5.5
}

This will add a new row like so:

Time [ms]	Voltage [V]	Current [A]	Frequency [Hz]
100	1.0	0.05	5.5

You do not need to call each set(...) in any particular order - any values you do not specify will simply be filled with a null value.

Alterniatvely, you can specify each value in column-order in one go by use of addData(...) like so:

rList.addData(100, 1.0, 50e-3, 5.5)

which would do the same.

As en example, starting with an empty ResultList, the following code:

rList.addRow(row -> {
    row.set(time, 100);
    row.set(voltage, 1.0);
    row.set(current, 50e-3);
    row.set(frequency, 5.5);
});

rList.addRow(row -> {
    row.set(time, 200);
    row.set(voltage, 2.0);
    row.set(current, 100e-3);
    row.set(frequency, 4.5);
});

rList.startRow(row -> {
    row.set(time, 300);
    row.set(voltage, 5.0);
    row.set(current, 120e-3);
    row.set(frequency, 2.5);
});

or written the alternative way:

rList.addData(100, 1.0, 50e-3, 5.5);
rList.addData(200, 2.0, 100e-3, 4.5);
rList.addData(300, 5.0, 120e-3, 2.5)

would both produce the following table:

Time [ms]	Voltage [V]	Current [A]	Frequency [Hz]
100	1.0	0.05	5.5
200	2.0	0.10	4.5
300	5.0	0.12	2.5

As another example let's say we're sweeping voltage on an SMU:

Java

Column<Double> voltage = Column.ofDecimals("Voltage", "V");
Column<Double> current = Column.ofDecimals("Current", "A");
ResultTable    rList   = new ResultList(voltage, current);

...

for (double v : voltages) {

  smu.setVoltage(v);

  rList.addRow(row -> {
       row.set(voltage, smu.getVoltage());
       row.set(current, smu.getCurrent());
  });

}

Kotlin

val voltage = DoubleColumn("Voltage", "V")
val current = DoubleColumn("Current", "A")
val rList   = ResultList(voltage, current)

...

for (v in voltages) {

  smu.voltage = v

  rList.addRow {
       it[voltage] = smu.voltage
       it[current] = smu.current
  }

}

In the above example, if each voltage value that we set the SMU to source, we record a voltage and current measurement in rList.

Accessing Data

Top ↑

All ResultTable objects are iterable, which means you can loop over them. On each iteration, they will yield a Row object that represents a single row of data. You can get the value of each column from a Row by use of get(...) or [...] in kotlin and passing it the Column<> object of the column you want the value of:

Java

final Column<Double> V = Column.ofDecimals("Voltage", "V");
final Column<Double> I = Column.ofDecimals("Current", "A");
final Column<Double> T = Column.ofDecimals("Temperature", "K");

ResultTable results = new ResultList(V, I, T);

...

for (Row row : results) {

    double voltage     = row.get(V);
    double current     = row.get(I);
    double temperature = row.get(T);

}

Kotlin

val V = Column.ofDecimals("Voltage", "V")
val I = Column.ofDecimals("Current", "A")
val T = Column.ofDecimals("Temperature", "K")

val results = ResultList(V, I, T)

...

for (row in results) {

    val voltage     = row[V]
    val current     = row[I]
    val temperature = row[T]

}

Attributes

Any ResultTable can have values stored along-side it to provide supplementary information. For instance, it's quite common that one would want to save experimental parameters along with the result data from a measurement. To facilitate this, there are two methods: setAttribute(key, value) and getAttribute(key).

Each attribute must have its own unique (string) key to identify it. To add an attribute to a ResultTable use setAttribute(). For instance, if we want to save the dimensions of a device we're measuring the conductivity of, we could do:

resultTable.setAttribute("Length", "400 um");   // L = 400 um
resultTable.setAttribute("Width", "200 um");    // W = 200 um
resultTable.setAttribute("Thickness", "40 nm"); // D = 40 nm

Then, when dealing with resultList later, these values can be recalled by use of getAttribute(...):

String length    = resultTable.getAttribute("Length");
String width     = resultTable.getAttribute("Width");
String thickness = resultTable.getAttribute("Thickness");

As we will see in the next section, these attributes will be saved with the data automatically when output as a CSV file.

Outputting and Saving Data

You can output the data in a ResultTable to both the terminal, visually (by use of a Table GUI element) or to a file.

For the purposes of example in this section, let's say we have a table called resultTable with three columns: two Double columns holding voltage and current values respectively and a String column holding notes about the measurement.

GUI Element

You can create a Table GUI element to show the contents of a ResultTable in real time. This is explained further on the Tables page. In short:

Table table = new Table("Title", resultTable);
table.show();

ASCII Tables

To display the ResultTable in the terminal as an ASCII table, use the outputTable() method like so:

resultTable.outputTable();

it will result in something looking like this being printed to the terminal (standard out stream):

========================================
| Voltage [V] | Current [A] | Notes    |
========================================
| 1.0         | 0.001       | None     |
+-------------+-------------+----------+
| 2.0         | 0.002       | None     |
+-------------+-------------+----------+
| 3.0         | 0.003       | None     |
+-------------+-------------+----------+
| 4.0         | 0.004       | None     |
+-------------+-------------+----------+
| 5.0         | 0.005       | None     |
+-------------+-------------+----------+
| 6.0         | 0.006       | None     |
+-------------+-------------+----------+

To output it to a file instead, just specify the path as an argument:

// Linux/Mac
resultTable.outputTable("/path/to/file.txt");

// Windows
resultTable.outputTable("C:\\path\\to\\file.txt");

CSV Format

To output the data in CSV format we use the output() method like so:

resultTable.output();

this will result in the following being printed to the terminal:

% ATTRIBUTES: {}
"Voltage [V]" {Double}, "Current [A]" {Double}, "Notes" {String}
1.0, 0.001, "None"
2.0, 0.002, "None"
3.0, 0.003, "None",
4.0, 0.004, "None",
5.0, 0.005, "None",
6.0, 0.006, "None"

To output this to a file instead, just specify the path as an argument:

// Linux/Mac
resultTable.output("/path/to/file.csv");

// Windows
resultTable.output("C:\\path\\to\\file.csv");

You may have noticed the "% Attributes: {}" line. This is where any attributes set on the ResultTable are written when output as CSV. For instance, if before calling output() we have done the following:

resultTable.setAttribute("Length", "400 um");   // L = 400 um
resultTable.setAttribute("Width", "200 um");    // W = 200 um
resultTable.setAttribute("Thickness", "40 nm"); // D = 40 nm

then the CSV output would instead look like this:

% ATTRIBUTES: {"Length": "400 um", "Width": "200 um", "Thickness": "40 nm"}
"Voltage [V]" {Double}, "Current [A]" {Double}, "Notes" {String}
1.0, 0.001, "None"
2.0, 0.002, "None"
3.0, 0.003, "None",
4.0, 0.004, "None",
5.0, 0.005, "None",
6.0, 0.006, "None"

Loading from CSV Files

You can load data (and attributes) previously written to a CSV file back into either a ResultList or ResultStream object by using ResultList.loadFile(...) or ResultStream.loadFile(...) respectively.

Loading as a ResultList will load the data back into memory whereas loading as a ResultStream will simply create a ResultStream object that uses the specified file as its backing file -- meaning any changes made to the ResultStream will be written directly to the file.

Java

ResultTable list   = ResultList.loadFile("/path/to/file.csv");
ResultTable stream = ResultStream.loadFile("/path/to/file.csv");

Kotlin

val list   = ResultList.loadFile("/path/to/file.csv")
val stream = ResultStream.loadFile("/path/to/file.csv")

As mentioned, this will also load in any attributes that were saved with the files, allowing you to retrieve them with getAttribute(...) calls.

Finding Columns

Often you may find yourself in a situation where you have a ResultTable object but not the Column<> objects that were used to create it. For instance, you may have loaded in some data from a CSV file using ResultList.loadFile(...). For these scenarios, ResultTable object provide the findColumn(...) method which lets you retrieve the correct Column<> by specifying some criteria.

For instance, if you know the name and type of the column you can use it like so:

Java

Column<Type> column = resultTable.findColumn("Name", Type.class);

// Or find a column of any type and cast it afterwards
Column<Type> column = (Column<Type>) resultTable.findColumn("Name");

Kotlin

val column = resultTable.findColumn("Name", Type::class.java)

// Or find a column of any type and cast it afterwards
val column = resultTable.findColumn("Name") as Column<Type>

Alternatively, if you have a Column<> object to hand with the same name, units and type you can simply use that - ResultTable objects and their Row objects are smart enough to match them.

However, to save it from having to search each time, you can use findColumn(...) to find the matching column in the ResultTable like so:

Column<Type> column = Column.ofX(...);

column = resultTable.findColumn(column);

If you know the integer index of the column (i.e. starting from 0) then you can use getColumn(...). However, because there's no way for Java/Kotlin to know the data type of the column your are retrieving, you will need to cast it to the correct type like so:

Java

Column<Type> column = (Column<Type>) resultTable.getColumn(index);

Kotlin

val column = resultTable.getColumn(index) as Column<Type>

Extracting Lists

You can extract any column, or any value determined by an operation performed on each row, as a List<> object by use of toList(...). For instance, let's say we have a ResultTable and we want to extract one of its columns as a list of doubles:

Java

List<Double> values = resultTable.toList(column);

Kotlin

val values = resultTable.toList(column)

If, for example, we instead wanted to return a list that is the product of two columns we could supply a (Row) → Double lambda expression instead like so:

Java

List<Double> values = resultTable.toList(row -> row.get(column1) * row.get(column2));

Kotlin

val values = resultTable.toList { row -> row[column1] * row[column2] }

/* == or by using "it" == */

val values = resultTable.toList { it[column1] * it[column2] }

In fact, you can think about our first example of retrieving the values of a single column as just being shorthand for:

Java

List<Double> values = resultList.toList(row -> row.get(column));

Kotlin

val values = resultTable.toList { it[column] }

Extracting Matrices

You can extract any combination of numerical columns as a RealMatrix object by using toMatrix(...) and specifying what you want to be in each column of the matrix. For instance, if we had the following ResultTable:

Column<String> timestamp = Column.ofText("Timestamp");
Column<Double> voltage   = Column.ofDecimals("Voltage", "V");
Column<Double> current   = Column.ofDecimals("Current", "A");
ResultTable    table     = new ResultList(timestamp, voltage, current);

for (...) {

  table.addRow(row -> {
       row.set(timestamp, Util.getCurrentTimeString());
       row.set(voltage, smu.getVoltage());
       row.set(current, smu.getCurrent());
  });

}

Then we can extract both the voltage and current columns (since they're both numbers) as a RealMatrix like so:

Java

RealMatrix matrix = table.toMatrix(voltage, current);

Kotlin

val matrix = table.toMatrix(voltage, current)

The result is a matrix with two columns, the first being the values of voltage and the second being those of current. Just as before, we can instead supply expressions for each matrix column. For instance, we could achieve the same result by writing:

Java

RealMatrix matrix = table.toMatrix(r -> r.get(voltage), r -> r.get(current));

Kotlin

val matrix = table.toMatrix({ r -> r[voltage] }, { r -> r[current] })

// == or using "it" ==

val matrix = table.toMatrix({ it[voltage] }, { it[current] })

This lets us retrieve values that are derived from those in the table. For instance, if we wanted a single column matrix of power (i.e. voltage * current):

Java

RealMatrix matrix = table.toMatrix(r -> r.get(voltage) * r.get(current));

Kotlin

val matrix = table.toMatrix { it[voltage] * it[current] }

Filtering

You can create a new coy of a ResultTable that only contains rows matching a given criterion by using the filter(...) method. This takes a lambda expression (specifically a Predicate<Row> object) that should return true or false to determine whether a given row should be retained or not.

For instance, if we have a table with three columns: "Time [s]", "Voltage [V]" and "Current [A]", we could filter these data to give us only the rows where time > 50 seconds like so:

Java

ResultTable after50 = table.filter(r -> r.get(time) > 50)

Kotlin

val after50 = table.filter { r -> r[time] > 50 }

// == or, by using "it" ==

val after50 = table.filter { it[time] > 50 }

Sorting

You can get a ResultTable to return a sorted copy of itself by using the sorted(...) method. This method requires you to either specify an expression as a lambda function to return a sortable valuable based on values in each row, or a Column<> of some sortable type:

Java

Column<Double> number = Column.ofDecimals(...);
Column<String> string = Column.ofText(...);

// By specifying column
ResultTable sortedByNumber = table.sorted(number);
ResultTable sortedByString = table.sorted(string);

// By specifying exression
ResultTable sortedByAbs = table.sorted(r -> Math.abs(r.get(number)));

Kotlin

val number = Column.ofDecimals(...)
val string = Column.ofText(...)

// By specifying column
val sortedByNumber = table.sorted(number)
val sortedByString = table.sorted(string)

// By specifying exression
val sortedByAbs = table.sorted { abs(it[number]) }

For example, if we had some temperature-dependent data but it was not in any particular order:

Column<Double> temperature = Column.ofDecimals("Temperature", "K");
Column<Double> resistance  = Column.ofDecimals("Resistance", "Ohm");
ResultTable    table       = new ResultList(temperature, resistance);

// Data gets added to table here but at random temperatures

then we can get a copy that is in ascending order of temperature like so:

ResultTable sortedByTemperature = table.sorted(temperature);

Splitting

Often times, when analysing data, you will find it useful to be able to split a table of data into multiple tables based on the value in a given column (or an expression). ResultTable objects let you do this by use of split(...). Just as with other examples, you either supply it with a Column<> or a lambda expression. It will then return a map of each unique value in that column/expression to filtered copies of the ResultTable containing only rows with that value.

Map<Type, ResultTable> split = table.split(Column<Type> column);
Map<Type, ResultTable> split = table.split(((Row) -> Type) expression);

For instance, if we had output curve data (SD Voltage, SG Voltage and SD Current) in the columns drain, gate and current respectively, then it's likely we will want to split the data by gate like so:

Java

Map<Double, ResultTable> split = table.split(gate);

split.forEach((gate, data) -> {
  // gate is the gate voltage and data all the rows with that gate voltage
});

Kotlin

val split = table.split(gate)

for ((gate, data) in split) {
  // gate is the gate voltage and data all the rows with that gate voltage
}

You can also split a ResultTable in smaller ones based on the direction of change of a given numerical column/expression by use of directionalSplit(...). For instance, if drain is swept up then down, this will split the data into two sets, one for when drain is increasing and one of when drain is decreasing:

List<ResultTable> split = table.directionalSplit(drain);

if drain sweeps up and down multiple times, the list will contain a ResultTable for each up-sweep and each down-sweep in alternating order.