Creating server side components (node modules)

Introduction

Many things can be done client side: however, if you wish to access a database, or connect to a message bus, you also need to expand the server side functionality. In case you need a very specific purpose, you can simply add your extra functionality to (a fork of) csMap as an internal module. That's the most easy way to go. However, if you want to offer this functionality too others too, please read on.

As we are dealing with the node server, it makes sense to offer this additional server side functionality as components that are delivered as node modules, so a developer can pick and choose the components that he needs. What follows is a description of the development of one such node module, and publish it as a node package.

The example is based on the development of the cs-offline-search node package. It's purpose is to create server side an index file of the currently available layers, which can be used by the csWeb.offlineSearch directive on the client.

Creating an Internal Module

To start, create a new folder OfflineSearch in the csMap project. In this folder, I've created the OfflineSearchManager and OfflineSearcher classes, including several helper classes and interfaces. As you may know, an internal module in node uses the name of the file to create a module (the file name is considered a kind of namespace). So instead of starting each file with module ModuleName { ... }, as we did on the client side, here we don't need them. Unfortunately. Why unfortunately, you may ask? Well, because most of the helper interfaces and classes that I needed to create are copies of the ones that are used server side, but without the module part. This is quite a pain, since I cannot (or, better put, do not know how to) reuse the files that I created for the client. And any change on the client side may actually break my module. For example, I need to parse the solution file (still named projects.json), and if its layout is changed on the client side, I cannot parse it anymore.

Anyways, after some meddling I finished a working internal module of the offline search. As we are not interested in this particular functionality, let's move on to the next step.

Convert an Internal Module to an External Module

Basically, I just copied everything I had to csServerComp. At first, I thought I could put the helper classes and interfaces in a main folder, and have one tsconfig.json compile many different modules, but that bird didn't fly. Instead, I needed to copy every file I needed to a new folder, adequately named OfflineSearch, and add the tsconfig file there too. The folder structure is, therefore, as follows:

- csServerComp
  - OfflineSearch
    - Helpers\
    - Scripts\node
    - public\data\projects\...
    ts files

In addition to the existing files, I also needed to add four new files:

• tsconfig.json, so I can specify what to compile.
• package.json, which specifies how my node module is published
• Scripts/typings/node/node.d.ts so I can reference basic node functionality (like the file system or path)
• readme.md, a required description of your package

Minor side-note: also in this case, we cannot use these helpers across node modules, and any change client side is going to impact every node module that we are developing!

Configuring TypeScript

My tsconfig.json is quite similar to most others you have come across, I guess. See here or here for more details. I've just specified the output to the dist folder, so the generated JavaScript files don't end up between my TypeScript files. If everything went OK, you can press CTRL-SHIFT-B (in Atom using the atom-typescript package) to compile it.

{
    "compilerOptions": {
        "target": "es5",
        "module": "commonjs",
        "declaration": true,
        "noImplicitAny": false,
        "removeComments": false,
        "noLib": false,
        "outDir": "dist"
    },
    "filesGlob": [
        "./**/*.ts",
        "!./node_modules/**/*.ts"
    ]
}

Creating a Node Package Specification

As this was my first node package, I've read this tutorial and this one about how to proceed. Please do so too. A brief walk-through:

• Before creating node packages, I've added some default information about myself:
  • npm set init.author.name "Your Name"
  • npm set init.author.email "[email protected]"
  • npm set init.author.url "http://www.your.website"
  • npm adduser <your username>
  • npm view cs-your-package-name: by convention, all our packages should start with a cs, so presumably, your package name should be unique. Also note that your package name should be in lower case in order for the node package repository to accept it.
• npm init: which will walk you through a list of questions to generate the package.json description. Mine looked as follows:

{
  "name": "cs-offline-search",
  "version": "0.0.4",
  "description": "Create an index file of the locally stored geojson files.",
  "main": "./dist/OfflineSearch/index.js",
  "scripts": {
    "test": "node ./dist/OfflineSearch/test.js"
  },
  "repository": {
    "type": "git",
    "url": "https://github.com/TNOCS/csWeb"
  },
  "keywords": [
    "offline",
    "search"
  ],
  "author": "Erik Vullings <erik dot vullings at gmail dot com> (https://nl.linkedin.com/in/erikvullings)",
  "license": "MIT",
  "readmeFilename": "readme.md",
  "bugs": {
    "url": "https://github.com/TNOCS/csWeb/issues"
  }
}

Testing your Node Module

Besides the main functionality, I've also added a public folder with test data, and a very simple script to test my new package, test.ts (which is also referenced in the package.json below the scripts tag).

import OfflineSearchManager  = require('./index');
import IOfflineSearchOptions = require('./IOfflineSearchOptions');

var offlineSearchOptions: IOfflineSearchOptions = {
    propertyNames: ['Name', 'GeoAddress'],
    stopWords: ['de', 'het', 'een', 'en', 'van', 'aan']
};

var offlineSearchManager = new OfflineSearchManager('public/data/projects/projects.json', offlineSearchOptions);

Enter npm test to test the functionality.

In addition, there are two other ways you can test your package before publishing"

• Use node in the command line to test your functionality. When you enter node, a new command line appears, in which you can enter valid JavaScript.
• In addition, you can use npm link to create a symbolic link before publishing your package.

cd OfflineSearch
npm link
cs YourApp
npm link cs-offline-search

See 'Using your External Module' below to proceed.

Using private modules

Assume you wish to extend the server-side functionality with some very specific code, e.g. for connecting it to an internal service. In that case, it does not make sense to create a public node package for it. However, you can still use the current approach to create a private node package, which are going to be available on NPM in 2015. Alternatively, instead of publishing it to GitHub, publish the code to your private repository, and use the symbolic link functionality to use it.

cd ~/projects/your-private-module  # go into the package directory
npm link                           # creates global link
cd ~/projects/your-project         # go into some other package directory.
npm link your-private-module       # link-install the package

Now, any changes to ~/projects/your-private-module will be reflected in ~/projects/your-project/node_modules/your-private-module/.

Publish your node module

When everything works as expected, you can finally publish your package, assuming you have added everything to your git repository already.

git tag 0.0.1
git push origin master --tags 
npm version 0.0.1
npm pack
npm publish

About the npm version number: the first number indicates the major number, especially reserved for breaking changes with previous versions. The second number is for minor versions, for example when you add a new feature. And finally the latter number is for patches. Luckily, npm also includes a shortcut instead of editing this manually in the package.json file using the npm version command:

`npm version patch' updates the last number, and patch can also be replaced by major or minor, or an explicit version number like 1.0.0.

Using your External Node Module

Now that you've published your node module, you can easily use it in your own project too. Just install is as a regular node module using npm install cs-offline-search --save (the --save option automatically adds the package to your package.json).

Next, add the following code to server.ts to invoke it.

import offlineSearch = require('cs-offline-search');

/**
 * Create a search index file which can be loaded statically.
 */
var offlineSearchManager = new offlineSearch('public/data/projects/projects.json', {
    propertyNames: ['Name', ''postcode', 'Postcode'],
    stopWords    : ['de', 'het', 'een', 'en', 'van', 'aan']
});

Unfortunately, TypeScript will show a number of errors when you do this, since it does not know your package. You still need to create a definition file. Unfortunately, I could not use the definitions that were created automatically, as they didn't use the correct module name, so I had to create one myself (see also Steve Fenton's tips about how to do this, a recent stackoverflow discussion, or have a look at an alternative approach). Note the most important thing at the end export = OfflineSearchManager, which is the magic word in order to use it in server.ts. Basically, it exposes this module as the OfflineSearchManager class, so we can use it to create a new instance. I've also added this file to the dist/Scripts/typings/cs-offline-search folder, so others can use it too.

declare module "cs-offline-search" {
	interface IProjectLocation {
	  	title: string;
	  	url:   string;
	}

	/**
	 * Specify the offline search options.
	 */
	interface IOfflineSearchOptions {
	    /**
	     * Words you wish to exclude from the index.
	     * @type {string[]}
	     */
	    stopWords: string[];
	    /**
	     * The property types that you wish to use for generating the index.
	     * @type {osr.PropertyType}
	     */
	    propertyNames: string[];
	}

	/**
	* Offline Search reads the solution file, and creates an OfflineSearchResult for each project it contains.
	*/
	class OfflineSearchManager {
	    private solutionsFile;
	    private options;
	    constructor(solutionsFile: string, options: IOfflineSearchOptions);
	    private openSolution();
	    private processProject(project: IProjectLocation): void;
	}

	export = OfflineSearchManager;
}