Datum Capture Part 1

SolarNode Datum Capture Plug-in Example

This guide explains how to create a SolarNode plug-in to capture data from a fictional solar inverter, using the Eclipse IDE. This is the first part in a series of lessons. This part will guide you through:

Creating a SolarNode plug-in project in Eclipse
Implementing code for capturing power data (e.g. Wh generated) from a fake Foobar brand solar inverter
Creating an associated JUnit unit test project with some unit tests

The code for this example is available as well.

Eclipse Setup

If you have not already set up your SolarNetwork development environment, either go through the SolarNode Developement Guide or the Developer VM guide first, and then return here.

Create new Plug-in project for Foobar Power

The SolarNode runtime is based on OSGi and when you build a SolarNode plug-in you are actually building an OSGi bundle, which can be thought of as a normal Java JAR archive with some additional metadata stored in the MANIFEST.MF file included in the JAR.

Eclipse refers to OSGi bundles as "plug-ins" and its OSGi development tools are collectively known as the Plug-in Development Environment, or PDE. Create a new Plug-in project by selecting File > New > Project.... From the dialog window that appears, select Plug-in Project and click Next.

In the next screen, fill in some details for the new project. We will be following SolarNode naming conventions to create a bundle that collects power generation data from a fictional Foobar solar inverter. The project name will mirror our OSGi bundle name: net.solarnetwork.node.example.datum-capture. Fill in the following details:

Project name - net.solarnetwork.node.example.datum-capture
Output folder - set to build/eclipse so we can support standard SolarNode Ant-based builds later on.
Target Platform - set to a standard OSGi framework

Click the Next > button and on the next screen, fill in the OSGi bundle information:

ID - the same as the project name
Version - can start as 1.0.0
Name - something descriptive, e.g. Example Datum Capture
Vendor - an appropriate vendor, or SolarNetwork for a SolarNetwork-sponsored plug-in
Execution Environment - set to JavaSE-1.8

That's all you need to configure to get started, so click Finish now.

SolarNode datum

SolarNode collects samples of data from sensors, meters, and so on, and refers to each collection of data at a single timestamp from a single source as a datum. SolarNode provides a basic API all datum are required to implement, net.solarnetwork.node.domain.datum.NodeDatum, which looks like this (simplified here):

import net.solarnetwork.domain.datum.Datum;

public interface NodeDatum extends Datum, Cloneable {

	/**
	 * Get the object kind.
	 *
	 * @return the object kind
	 */
	ObjectDatumKind getKind();

	/**
	 * Get a domain-specific ID related to the object kind.
	 *
	 * @return the object ID, or {@literal null}
	 */
	Long getObjectId();

	/**
	 * Get the date this datum is associated with, which is often equal to
	 * either the date it was persisted or the date the associated data in this
	 * object was captured.
	 *
	 * @return the timestamp
	 */
	Instant getTimestamp();

	/**
	 * Get a unique source ID for this datum.
	 *
	 * <p>
	 * A single datum type may collect data from many different sources.
	 * </p>
	 *
	 * @return the source ID
	 */
	String getSourceId();

	/**
	 * Get a map of all available data sampled or collected on this datum.
	 *
	 * @return a map with all available sample data
	 */
	Map<String, ?> getSampleData();

	/**
	 * Get a general accessor for the sample data.
	 *
	 * @return the operations instance, never {@literal null}
	 */
	DatumSamplesOperations asSampleOperations();

	/**
	 * Get the date this object was uploaded to SolarNet.
	 *
	 * @return the upload date
	 */
	Instant getUploaded();

}

We want to collect samples of power generation our Foobar inverter, but notice that there are no properties specific to power generation on the NodeDatum interface, such as instantaneous watts or an accumulated watt-hour reading value. SolarNode provides an extension of that interface called net.solarnetwork.node.domain.datum.AcDcEnergyDatum that does include those properties, however. Here is a simplified view of that API:

public interface AcDcEnergyDatum extends AcEnergyDatum, DcEnergyDatum,
		net.solarnetwork.domain.datum.AcDcEnergyDatum {

	/**
	 * Get a watt-hour reading.
	 *
	 * <p>
	 * Generally this is an accumulating value and represents the overall energy
	 * used or produced since some reference date. Often the reference date if
	 * fixed, but it could also shift, for example shift to the last time a
	 * reading was taken.
	 * </p>
	 *
	 * @return the watt hour reading, or {@literal null} if not available
	 */
	default Long getWattHourReading() {
		return asSampleOperations().getSampleLong(Accumulating, WATT_HOUR_READING_KEY);
	}

	/**
	 * Set a watt-hour reading.
	 *
	 * @param value
	 *        the watt hour reading
	 */
	default void setWattHourReading(Long value) {
		asMutableSampleOperations().putSampleValue(Accumulating, WATT_HOUR_READING_KEY, value);
	}

	/**
	 * Get the instantaneous watts.
	 *
	 * @return watts, or {@literal null} if not available
	 */
	default Integer getWatts() {
		return asSampleOperations().getSampleInteger(Instantaneous, WATTS_KEY);
	}

	/**
	 * Get the instantaneous watts.
	 *
	 * @param value
	 *        the watts
	 */
	default void setWatts(Integer value) {
		asMutableSampleOperations().putSampleValue(Instantaneous, WATTS_KEY, value);
	}

	// more properties here...

}

SolarNode then provides a class that implements this API called net.solarnetwork.node.domain.datum.SimpleAcDcEnergyDatum that we can use.

Now we know we want to read samples from our Foobar inverter and translate the samples into SimpleAcDcEnergyDatum instances. Next, we look at the API that handles the production of those NodeDatum samples.

SolarNode datum data source

SolarNode defines two APIs for classes that can sample data and produce NodeDatum instances, referred to as data sources. Those APIs are defined by these interfaces:

net.solarnetwork.node.service.DatumDataSource
net.solarnetwork.node.service.MultiDatumDataSource

These APIs are very simple: they expose the type of NodeDatum they can produce and they return new datum instances of that type when asked. The general idea is that SolarNode will periodically query the data source for new data, so all we need to do is provide a NodeDatum when asked. To keep things simple, we will implement just net.solarnetwork.node.service.DatumDataSource, which looks like this:

public interface DatumDataSource extends Identifiable, DeviceInfoProvider {

	/**
	 * Get the class supported by this DataSource.
	 *
	 * @return class
	 */
	Class<? extends NodeDatum> getDatumType();

	/**
	 * Read the current value from the data source, returning as an unpersisted
	 * {@link NodeDatum} object.
	 *
	 * @return Datum
	 */
	NodeDatum readCurrentDatum();

}

You will notice that DatumDataSource extends net.solarnetwork.service.Identifiable, which is another pretty simple API used by the SolarNode framework to support selecting specific service instances for specific tasks. We will not worry much about this API for the purpose of this guide, but the API looks like this:

public interface Identifiable {

	/**
	 * Get a unique identifier for this service.
	 *
	 * <p>
	 * This should be meaningful to the service implementation, and should be
	 * minimally unique between instances of the same service interface.
	 * </p>
	 *
	 * @return unique identifier (should never be {@literal null})
	 */
	String getUid();

	/**
	 * Get a grouping identifier for this service.
	 *
	 * <p>
	 * This should be meaningful to the service implementation.
	 * </p>
	 *
	 * @return a group identifier, or {@literal null} if not part of any group
	 */
	String getGroupUid();

	/**
	 * Get a friendly display name for this service.
	 *
	 * @return a display name
	 */
	String getDisplayName();

}

The DatumDataSource also extends net.solarnetwork.node.service.DeviceInfoProvider which defines and API for the datum data source to provide standard device metadata. We will ignore that API for the purposes of this guide.

Implementation

To recap what we have learned, we know we want to:

write a class that implements net.solarnetwork.node.service.DatumDataSource, that performs the work of reading sample data from our Foobar inverter
this class should create instances of net.solarnetwork.node.domain.datum.SimpleAcDcEnergyDatum from the sampled data

OSGi package imports

This guide will not go into details on OSGi, but for our bundle to have access to to the DatumDataSource interface, the SimpleAcDcEnergyDatum class, and so on, our bundle must declare its desire to use the packages they are defined in. Open the META-INF/MANIFEST.MF file in Eclipse. By default this opens in Eclipse's Manifest Editor. Click on the Dependencies tab (located at the bottom of the editor).

Click on the Add... button, and use the search field to add the following packages:

net.solarnetwork.domain.datum;
net.solarnetwork.node
net.solarnetwork.node.domain.datum
net.solarnetwork.node.service
net.solarnetwork.node.service.support
net.solarnetwork.service
net.solarnetwork.service.support
org.slf4j

Eclipse will also default to using the version of the packages available in your workspace as the minimum required package version, which is generally fine. If you click on the MANIFEST.MF tab at the bottom of the manifest editor you can see the raw source of the MANIFEST.MF file, which should look like this:

☝️ Note that the actual version numbers may differ from shown; that is OK.

You will notice that all the values you filled in when you first created the project show up here, like the plug-in name and version.

Create `DatumDataSource` class

Now we are ready to create our DatumDataSource implementation. We will follow SolarNode conventions and create our class within a package named after our bundle ID (and Eclipse project name): net.solarnetwork.node.example.datum_capture.FoobarDatumDataSource. Right-click on the project in Eclipse and select New > Class. Fill in the appropriate package and name, set the superclass to net.solarnetwork.node.service.support.DatumDataSourceSupport, and add the net.solarnetwork.node.service.DatumDataSource interface, like this:

Eclipse will create stubs for the methods you are required to implement. Many methods are already implemented for you by the DatumDataSourceSupport class. The getDatumType() should simply return the SimpleAcDcEnergyDatum class, so after writing that line the class should look like this:

/**
 * Implementation of {@link DatumDataSource} for Foobar inverter power.
 *
 * @author matt
 * @version 1.0
 */
public class FoobarDatumDataSource extends DatumDataSourceSupport implements DatumDataSource {

	@Override
	public Class<? extends AcDcEnergyDatum> getDatumType() {
		return SimpleAcDcEnergyDatum.class;
	}

	@Override
	public AcDcEnergyDatum readCurrentDatum() {
		// TODO
		return null;
	}

}

Implement logic to "sample" data

So far we have just been implementing SolarNode scaffolding. Now it is time to implement the actual code to read samples from the Foobar inverter and return the data as a SimpleAcDcEnergyDatum instance. Our implementation is fictional, so we will simply return fake instantaneous watt readings and fake accumulating watt-hour readings. First, to provide data that models a real-world inverter, we will add a counter to keep track of the accumulated watt-hours generated, and a configurable property to assign a sourceId value to the returned datum instances:

	/** The {@code sourceId} property default value. */
	public static final String DEFAULT_SOURCE_ID = "Inverter1";

	private final AtomicLong wattHourReading = new AtomicLong(0);

	private String sourceId = DEFAULT_SOURCE_ID;

	public String getSourceId() {
		return sourceId;
	}

	public void setSourceId(String sourceId) {
		this.sourceId = sourceId;
	}

Then, let us implement the readCurrentDatum() method to return some data modeled after a 1kW PV system, returning randomized data:

	@Override
	public AcDcEnergyDatum readCurrentDatum() {
		// our inverter is a 1kW system, let's produce a random value between 0-1000
		int watts = (int) Math.round(Math.random() * 1000.0);

		// we will increment our Wh reading by a random amount between 0-15, with
		// the assumption we will read samples once per minute
		long wattHours = wattHourReading.addAndGet(Math.round(Math.random() * 15.0));

		SimpleAcDcEnergyDatum datum = new SimpleAcDcEnergyDatum(sourceId, Instant.now(),
				new DatumSamples());
		datum.setWatts(watts);
		datum.setWattHourReading(wattHours);
		return datum;
	}

Unit testing

We have got our DatumDataSource implemented now, let us write some unit tests. A nice way to write unit tests is to create a new OSGi fragment bundle project. A fragment bundle attaches to a host bundle, inheriting all the package imports of the host, but can add additional classes and imports. This cleanly separates the unit test code from the real code, and the unit test bundles need not be deployed on your SolarNode, minimizing the footprint on the node.

Start by creating a new fragment project in Eclipse by selecting File > New > Project....

Eclipse create fragment plug-in project dialog

From the dialog window that appears, select Fragment Plug-in Project and click Next. In the next screen, fill in the following details:

Project name - net.solarnetwork.node.example.datum-capture.test (our host project name with .test appended)
Output folder - set to build/eclipse so we can support standard SolarNode Ant-based builds later on.
Target Platform - set to a standard OSGi framework

Click the Next > button, and on the next screen fill in the OSGi bundle information. The most important field is the Host Plug-in ID which should be set to net.solarnetwork.node.example.datum-capture:

Eclipse dialogs for creating new test fragment plug-in project

Finally click the Finish button and Eclipse will create the project and open the project's manifest editor. We will be unit testing with JUnit 4 and making use of the net.solarnetwork.node.test project, and as such need to add the following package imports:

net.solarnetwork.node.test
org.junit
org.junit.runner

Save the changes. If you view the manifest source, it will look like this:

☝️ Note that the actual version numbers may differ from shown; that is OK.

Right-click on the project in Eclipse and select New > Class to create our unit test for the FoobarDatumDataSource class. Fill in the appropriate package and name the class FoobarDatumDataSourceTests:

For the implementation, we will add a setup method to configure an instance of the FoobarDatumDataSource class, and a method that verifies the data source produces data as expected. We will use standard JUnit 4 annotations @Before and @Test:

public class FoobarDatumDataSourceTests {

	private static final String TEST_SOURCE_ID = "Test";

	private FoobarDatumDataSource service;

	private final Logger log = LoggerFactory.getLogger(getClass());

	@Before
	public void setup() {
		service = new FoobarDatumDataSource();
		service.setSourceId(TEST_SOURCE_ID);
	}

	@Test
	public void readOneDatum() {
		AcDcEnergyDatum d = service.readCurrentDatum();
		log.trace("Got datum: {}", d);

		// datum should not be null
		Assert.assertNotNull("Current datum", d);

		// the source ID should be what we configured in setup()
		Assert.assertEquals("Source ID", TEST_SOURCE_ID, d.getSourceId());

		// the watts and watt hours values should not be null, but we do not know
		// exactly what they will be because they produce random data
		Assert.assertNotNull("Watts", d.getWatts());
		Assert.assertNotNull("Watt hours", d.getWattHourReading());
		Assert.assertTrue("Watt range", d.getWatts() >= 0 && d.getWatts() <= 1000);
		Assert.assertTrue("Watt hour range",
				d.getWattHourReading() >= 0L && d.getWattHourReading() <= 15L);
	}

}

Now you can right-click on the class in Eclipse and select Run As > JUnit Test. The test should run and complete without errors.

Now let's add another unit test method that verifies that the watt hour reading does not decrease each time we call readCurrentDatum(), to give us confidence the class is modeling a real inverter sensibly. We will also add a debug log statement, to see how logging can be used. Add this following method:

@Test
public void readSeveralDatum() {
	long lastWattHourReading = 0;
	for ( int i = 0; i < 10; i++ ) {
		AcDcEnergyDatum d = service.readCurrentDatum();
		log.trace("Got datum: {}", d);

		Assert.assertNotNull("Current datum", d);
		Assert.assertEquals("Source ID", TEST_SOURCE_ID, d.getSourceId());
		Assert.assertNotNull("Watts", d.getWatts());
		Assert.assertNotNull("Watt hours", d.getWattHourReading());
		Assert.assertTrue("Watt range", d.getWatts() >= 0 && d.getWatts() <= 1000);
		log.debug("Got Wh reading: {}", d.getWattHourReading());
		Assert.assertTrue("Watt hour range", d.getWattHourReading() >= lastWattHourReading
				&& d.getWattHourReading() <= lastWattHourReading + 15L);
		lastWattHourReading = d.getWattHourReading();
	}
}

Re-run the unit test class in Eclipse, and both tests should pass. Click on the Console tab in Eclipse to see the generated logging output. If you do not see any log output, you may not have configured the necessary log4j2-test.xml file for logging to work properly. In the net.solarnetwork.node.test project, copy the environment/example/log4j2-test.xml file into the environment/local directory. This will enable TRACE level logging for SolarNode classes by default. Re-run the unit tests, and you should see output similar to this:

14:15:18.926 [                main] DEBUG ure.test.FoobarDatumDataSourceTests - Got Wh reading: 49
14:15:18.926 [                main] TRACE ure.test.FoobarDatumDataSourceTests - Got datum: Datum{kind=Node,sourceId=Test,ts=2023-04-21T02:15:18.926Z,data={i={watts=841}, a={wattHours=60}}}
14:15:18.927 [                main] DEBUG ure.test.FoobarDatumDataSourceTests - Got Wh reading: 60
14:15:18.927 [                main] TRACE ure.test.FoobarDatumDataSourceTests - Got datum: Datum{kind=Node,sourceId=Test,ts=2023-04-21T02:15:18.927Z,data={i={watts=454}, a={wattHours=68}}}
14:15:18.927 [                main] DEBUG ure.test.FoobarDatumDataSourceTests - Got Wh reading: 68
14:15:18.927 [                main] TRACE ure.test.FoobarDatumDataSourceTests - Got datum: Datum{kind=Node,sourceId=Test,ts=2023-04-21T02:15:18.927Z,data={i={watts=930}, a={wattHours=69}}}
14:15:18.927 [                main] DEBUG ure.test.FoobarDatumDataSourceTests - Got Wh reading: 69
14:15:18.927 [                main] TRACE ure.test.FoobarDatumDataSourceTests - Got datum: Datum{kind=Node,sourceId=Test,ts=2023-04-21T02:15:18.927Z,data={i={watts=298}, a={wattHours=2}}}

Continue...

You have got some working code now, but it still is not integrated into SolarNode. Continue to part two to see how that works.

SolarNode Datum Capture Plug-in
- Part 2
- Part 3
- Part 4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Datum Capture Part 1

SolarNode Datum Capture Plug-in Example

Eclipse Setup

Create new Plug-in project for Foobar Power

SolarNode datum

SolarNode datum data source

Implementation

OSGi package imports

Create `DatumDataSource` class

Implement logic to "sample" data

Unit testing

Continue...

Clone this wiki locally

Datum Capture Part 1

SolarNode Datum Capture Plug-in Example

Eclipse Setup

Create new Plug-in project for Foobar Power

SolarNode datum

SolarNode datum data source

Implementation

OSGi package imports

Create DatumDataSource class

Implement logic to "sample" data

Unit testing

Continue...

Clone this wiki locally

Create `DatumDataSource` class