This is a set of functions specialized in comparing and merging arrays of elements signed by unique set of IDs. This is typically used in scenario in which client-side data-sets with changes coming from the an SQL server (server > client replication).
Simply open up a terminal, go to your project directory and run:
npm i --save array-merge-by-id
You can try out all the examples (listed below) in your browser at RunKit.
Comparing, merging, linking elements of two arrays:
mergeA
- merges new/changed elements into an existing arraycompareA
- compares elements of two arrays and returns an object containing common elements and differenceslinkA
- extends each child array element with a reference to it's parent elementeachPair
- calls a callback method for each matched elements of provided arrays
Searching & filtering:
filterByKeys
- extract all the array elements which match the given key values (or are indicated by a comparer function)findFirstById
- returns the first matched element of the given typeindexOf
- returns index of first matching element in the given arrayuniqueA
- copies unique elements from source to a new array, which is then returned
Modifying, sorting, adding, removing elements of a single array:
concat
- does an in-place concatination of elements of the source array at the end of target arrayclear
- removes all the elements of the given arraypurgeA
- removes elements indicated by a hit list from the provided arrayoverwrite
- removes all the elements of target and replaces them with elements from sourcesortOn
- sorts the given array based on the given key name array (or comparer function)
Other helper functions:
compileC
- compiles and returns a function which compares two data elements and detects which comes before which in an orderd liststringifyIDs
- creates CSV containing values of all the keys ending with "ID". It's usefull for debugging
This lib was written in TypeScript, which is good news for people using TypeScript - all the typing information is included in the build (don't worry - it can still be used in vanilla JavaScript ... etc).
- compareA(leftA, rightA, key_columns, config) ⇒
ArrayDiffResult
Compares elements of two arrays and returns an object containing common elements and differences (see comparisson results object).
- filterByKeys(aSearch, key_columns, key_values, findFirstOnly) ⇒
Array
Extract all the array elements which match the given key values (or are indicated by a comparer function)
- findFirstById(aSearch, key_columns, key_values) ⇒
Object
Returns the first matched element of the given type
- indexOf(aSearch, key_columns, key_values) ⇒
number
Returns index of first matching element in the given array
- linkA(parentA, childA, key_columns, linkName, config) ⇒
ArrayLinkResult
Extends each child array element with a reference to it's parent element (in
parentA
array). Optionally it can extend parane element with a map of it's children.- mergeA(currData, newData, key_columns, config) ⇒
ArrayDiffResult
Merges new/changed elements into an existing array
- purgeA(aTarget, aHitList, key_columns, config) ⇒
Array
Removes elements indicated by a hit list from the provided array
- uniqueA(source, key_columns, config) ⇒
Array
Copies unique elements from
source
to a new array, which is then returned- compileC(key_columns) ⇒
function
Compiles and returns a function which compares two data elements and detects which comes before which in an orderd list. The compiled function expects the compared values to be numeric
- eachPair(leftA, rightA, key_columns, callbackFn, config) ⇒
ArrayDiffResult
Calls a callback method for each matched elements of provided arrays
- sortOn(source, key_columns) ⇒
Array
Sorts the given array based on the given key name array (or comparer function)
- clear(target) ⇒
Array
Removes all the elements of the given array
- overwrite(target, source) ⇒
Array
removes all the elements of
target
and replaces them with elements fromsource
- concat(target, source) ⇒
Array
Does an in-place concatination of elements of the
source
array at the end oftarget
array- stringifyIDs(data) ⇒
string
Creates CSV containing values of all the keys ending with "ID". It's usefull for debugging
Compares elements of two arrays and returns an object containing common elements and differences (see comparisson results object).
Kind: global function
Returns: ArrayDiffResult
- comparisson results object (see ArrayDiffResult)
Param | Type | Description |
---|---|---|
leftA | Array |
first array be compared |
rightA | Array |
second array be compared |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
config | ArrayDiffConfig |
(optional) additional config parameters (see ArrayDiffConfig) |
Example
let leftA = [
{cityID:1, cityName:'New York', weather:"windy"},
{cityID:2, cityName:'London', weather:"raining"}
];
let rightA = [
{cityID:2, cityName:'London', weather:"thunderstorm"},
{cityID:3, cityName:'Moscow', weather:"snowing"}
];
let diff = compareA(leftA, rightA, ["cityID"]);
// statment below will print:
// {
// leftDiff: [{cityID:1, cityName:'New York', weather:"windy"}], <= New York doesn't exist in the `rightA`
// leftCommon: [{cityID:2, cityName:'London', weather:"raining"}], <= London does exist in `rightA`
// rightDiff: [{cityID:3, cityName:'Moscow', weather:"snowing"}], <= Moscow exists in `leftA`
// rightCommon: [{cityID:2, cityName:'London', weather:"thunderstorm"}] <= London exists in `leftA`
// }
console.dir(diff);
Extract all the array elements which match the given key values (or are indicated by a comparer function)
Kind: global function
Returns: Array
- array of matched elements
Param | Type | Description |
---|---|---|
aSearch | Array |
array to be searched |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
key_values | Map |
(optional) an object which should be matched with an element from aSearch array - it's optional because key_columns param can contain a function which doesn't need it |
findFirstOnly | boolean |
(optional) should only the first matched element be returned (defaults to false ) |
Example
let streets = [{cityID:22, streetID:1}, {cityID:44, streetID:2}, {cityID:22, streetID:3}];
let myCityStreets = filterByKeys(streets, ['cityID'], {cityID:22});
console.dir(myCityStreets); // will output [{cityID:22, streetID:1}, {cityID:22, streetID:3}]
Returns the first matched element of the given type
Kind: global function
Returns: Object
- matched array element
Param | Type | Description |
---|---|---|
aSearch | Array |
array to be searched |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
key_values | Map |
(optional) an object which should be matched with an element from aSearch array - it's optional because key_columns param can contain a function which doesn't need it |
Example
let streets = [{cityID:22, streetID:1}, {cityID:44, streetID:2}, {cityID:22, streetID:3}];
let myStreet = findFirstById(streets, ['cityID','streetID'], {cityID:44, streetID:2});
console.dir(myStreet); // will output {cityID:44, streetID:2}
Returns index of first matching element in the given array
Kind: global function
Returns: number
- element index
Param | Type | Description |
---|---|---|
aSearch | Array |
array to be searched |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
key_values | Map |
(optional) an object which should be matched with an element from aSearch array - it's optional because key_columns param can contain a function which doesn't need it |
Example
let streets = [{cityID:22, streetID:1}, {cityID:44, streetID:2}, {cityID:22, streetID:3}];
let streetIndex = findFirstById(streets, ['cityID','streetID'], {cityID:44, streetID:2});
console.log(streetIndex); // will print 1
Extends each child array element with a reference to it's parent element (in parentA
array). Optionally it can extend parane element with a map of it's children.
Kind: global function
Returns: ArrayLinkResult
- linking results object(see ArrayLinkResult)
Param | Type | Default | Description |
---|---|---|---|
parentA | Array |
array containting parent elements | |
childA | Array |
array containing child elements | |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) | |
linkName | string |
"parent" |
(optional) name of the property which should be assigned a reference to parent element (defaults to 'parent') |
config | ArrayDiffConfig |
(optional) additional config parameters (see ArrayDiffConfig) |
Example
let cities = [ {cityID:22, cityName:'New York'}, {cityID:44, cityName:'London'} ];
let streets = [{cityID:22, streetID:1}, {cityID:44, streetID:2}, {cityID:22, streetID:3}];
let result = linkA(cities, streets, ['cityID'], 'city')
console.dir(result); // will output [ {cityID:22,streetID:1,city:<reference to New York>},
// {cityID:44,streetID:2,city:<reference to London>},
// {cityID:22,streetID:3,city:<reference to New York>}]
Merges new/changed elements into an existing array
Kind: global function
Returns: ArrayDiffResult
- comparisson results object (see ArrayDiffResult)
Param | Type | Description |
---|---|---|
currData | Array |
an array of "current" data elements |
newData | Array |
an array of changes and new data elements |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
config | ArrayDiffConfig |
(optional) additional config parameters (see ArrayDiffConfig) |
Example
let currData = [
{cityID:1, cityName:'New York'},
{cityID:2, cityName:'Londonnnnn'}
];
let newData = [
{cityID:2, cityName:'London'}, // London is fixed
{cityID:3, cityName:'Rome' } // Rome is added
];
// function which applies changes to an existing element
let mergeFn = (element, changes) => { element.cityName = changes.cityName; };
let result = mergeA(currData, newData, ['cityID'], { callbackFn: mergeFn });
console.dir(currData); // will print [ {cityID:1, cityName:'New York'},
// {cityID:2, cityName:'London'},
// {cityID:3, cityName:'Rome' } ];
Removes elements indicated by a hit list from the provided array
Kind: global function
Returns: Array
- an array of removed elements
Param | Type | Description |
---|---|---|
aTarget | Array |
array to be purged |
aHitList | Array |
hit list - indicates which element from aTarget should be removed |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
config | ArrayPurgeConfig |
(optional) additional config parameters (see ArrayPurgeConfig) |
Example
let targetA = [
{cityID:1, cityName:'New York'},
{cityID:2, cityName:'London'},
{cityID:3, cityName:'Rome' }
];
let hitList = [ { cityID: 1 }, { cityID: 3 } ];
let elFreq = [];
let result = purgeA(targetA, hitList, ['cityID'] { elFreq: elFreq });
console.dir(targetA); // will print [ {cityID:1, cityName:'New York'} ];
console.dir(elFreq); // will print [1, 2] - it means that the first array
// element was found once, while the second twice
Copies unique elements from source
to a new array, which is then returned
Kind: global function
Returns: Array
- array of unique elements
Param | Type | Description |
---|---|---|
source | Array |
source array - which may contain duplicates |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
config | ArrayUniqueConfig |
(optional) additional config parameters (see ArrayUniqueConfig) |
Example
let source = [ {cityID:1, cityName:'New York'}, {cityID:2, cityName:'London'}, {cityID:2, cityName:'London'} ];
let result = uniqueA(source, ["cityID"]);
console.dir(result); // will print [ {cityID:1, cityName:'New York'}, {cityID:2, cityName:'London'} ]
Compiles and returns a function which compares two data elements and detects which comes before which in an orderd list. The compiled function expects the compared values to be numeric
Kind: global function
Returns: function
- compiled function
Param | Type | Description |
---|---|---|
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
Example
// function for an ASCENDING order
let ascFn = compileC('cityID','streetID');
// will return 0
ascFn({cityID:1, streetID:1}, {cityID:1, streetID:1});
// will return 1
ascFn({cityID:1, streetID:1}, {cityID:1, streetID:2});
// will return -1
ascFn({cityID:2, streetID:1}, {cityID:1, streetID:2});
// function for an DESCENDING order
let descFn = compileC('cityID:desc','streetID:desc');
// will return 0
descFn({cityID:1, streetID:1}, {cityID:1, streetID:1});
// will return -1
descFn({cityID:1, streetID:1}, {cityID:1, streetID:2});
// will return 1
descFn({cityID:2, streetID:1}, {cityID:1, streetID:2});
Calls a callback method for each matched elements of provided arrays
Kind: global function
Returns: ArrayDiffResult
- comparisson results object
Param | Type | Description |
---|---|---|
leftA | Array |
first array of elements |
rightA | Array |
second array of elements |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
callbackFn | function |
function to be called for each of mathced element pairs |
config | ArrayDiffConfig |
(optional) additional config parameters (see ArrayDiffConfig) |
Example
let cities = [ {cityID:22, name:'New York'}, {cityID:44, name:'London'} ];
let streets = [
{cityID:22, streetID:1, name:'Elm Street'},
{cityID:22, streetID:3, name:'Wall St'},
{cityID:44, streetID:2, name:'Downing St'}
];
let callbackFn = (city, street) => {
console.log(street.name + ' ' + city.name)
}
eachPair(cities, streets, ["cityID"], callbackFn);
// The following will be printed to console:
// Elm Street in New York
// Wall St in New York
// Downing St in London
Sorts the given array based on the given key name array (or comparer function)
Kind: global function
Returns: Array
- array passed via source
param
Param | Type | Description |
---|---|---|
source | Array |
array to be sorted |
key_columns | CompareBy |
definition on how elements of two arrays should be compared (see CompareBy) |
Example
let source = [{cityID:2},{cityID:3},{cityID:1}];
// sorting in ASCENDING order
sortOn(source, ["cityID"]);
console.log(source); // will print [{cityID:1},{cityID:2},{cityID:3}];
// sorting in DESCENDING order
sortOn(source, ["cityID:desc"]);
let streets = [
{cityID:22, streetID:1, name:'Elm Street'},
{cityID:44, streetID:2, name:'Downing St'},
{cityID:22, streetID:3, name:'Wall St'}
];
// using multiple keys
sortOn(source, ["cityID", "streetID:desc"]);
Removes all the elements of the given array
Kind: global function
Returns: Array
- array passed via target
param
Param | Type | Description |
---|---|---|
target | Array |
array to be cleared |
Example
let source = [{cityID:2},{cityID:3},{cityID:1}];
clear(source);
console.dir(source); // will print []
removes all the elements of target
and replaces them with elements from source
Kind: global function
Returns: Array
- array passed via target
param
Param | Type | Description |
---|---|---|
target | Array |
array to be overwritten |
source | Array |
array who's elements are to be places in target |
Example
let target = [{cityID:100},{cityID:200}];
let source = [{cityID:2},{cityID:3},{cityID:1}];
overwrite(target, source);
console.dir(target); // will print [{cityID:2},{cityID:3},{cityID:1}]
Does an in-place concatination of elements of the source
array at the end of target
array
Kind: global function
Returns: Array
- array passed via target
param
Param | Type | Description |
---|---|---|
target | Array |
array which should receive new elements |
source | Array |
array of elements which should be added at the end of the target array |
Example
let target = [{cityID:100},{cityID:200}];
let source = [{cityID:2},{cityID:3},{cityID:1}];
concat(target, source);
console.dir(target); // will print [{cityID:100},{cityID:200},{cityID:2},{cityID:3},{cityID:1}]
Creates CSV containing values of all the keys ending with "ID". It's usefull for debugging
Kind: global function
Returns: string
- CSV of key values
Param | Type | Description |
---|---|---|
data | Object |
object or array to be converted to CSV |
Example
let obj = { cityID: 11, streetID: 22, streetName: 'Elm Street' };
let result = stringifyIDs(obj);
console.log(result); // will print "cityID:11,streetID:22"
Functions which compare array elements need to be instructed how two elements can be compared. This can be done in two ways:
- by passing an array of ID property names, which should be compared to determin the relation of the two objects
- by passing a comparer function, which receives tow elements and returns a numeric value indicating the relation of the objects
If an array of property names is passed, a comparer function will be compiled automatically (via compileC
) function.
The following snippet shows how comparisson can be defined via a comparer function:
let leftA = [
{cityID:1, cityName:'New York', weather:"windy"},
{cityID:2, cityName:'London', weather:"raining"}
];
let rightA = [
{cityID:2, cityName:'London', weather:"thunderstorm"},
{cityID:3, cityName:'Moscow', weather:"snowing"}
];
const comparerFn = (leftEl, rightEl) => {
if(leftEl.cityID===rightEl.cityID) {
return(0);
}
if(leftEl.cityID<rightEl.cityID) {
return(1);
}
return(0);
}
let diff = compareA(leftA, rightA, comparerFn);
The previous example can be re-written to define comparrison via property name array instead of comparer function:
let leftA = [
{cityID:1, cityName:'New York', weather:"windy"},
{cityID:2, cityName:'London', weather:"raining"}
];
let rightA = [
{cityID:2, cityName:'London', weather:"thunderstorm"},
{cityID:3, cityName:'Moscow', weather:"snowing"}
];
let diff = compareA(leftA, rightA, ["cityID"]);
If a ID name array is passed as key_columns
params, the compiled function will compare elements in ascending order.
We can change this behaviour by appending :desc
to a ID name ... like so: ['cityID:desc','streetID:desc']
See an example given for the sortOn
function.
We can modify the way the functions work, by providing an config object. All the options in the config
object are optional.
Here's a list of available options:
sortLeftBy
- how should theleftA
array be sortedsortRightBy
- how should therightA
be sortedskipSort
- set it totrue
if arrays are not to be sortedunique
- set it totrue
if are all the array elements unique - it speeds up the algorithmlinkName
- property assigned to therightA
elements, which should be pointing to the matching element in theleftA
mapName
- property assigned to theleftA
elements, containig array of all the matched elemens from therightA
callbackFn
- a callback function, which should be called for each of the matched element pairs
Description: defines how the arrays passed to the function should be sorted
Defaults to: value passed as key_columns
param
Expected value: we can pass a function or an array of ID param names (see CompareBy)
In order to be more efficient, functions which rely on comparing array elements will sort both of the given arrays.
By default the functions use key_columns
parameter to sort the arrays.
This can be overriden by specifying a dedicated sorting order for each of the two arrays:
config.sortLeftBy
= defines how the left array should be sorted (passed as the first param)config.sortRightBy
= defines how the right array should be sorted (passed as the second param)
Sorting of an array can be disabled by assigning null
to corresponding sort config param:
{
sortLeftBy: null // don't sort the left array
}
This config si very similar to ArrayDiffConfig
param type. The following params are the same as in ArrayDiffConfig
:
sortLeftBy
sortRightBy
skipSort
The following params are unique to this param type:
sortBy
= how should both arrays be sortedmapRemoved
- flag indicating should removed elements be mapped and returnedmatchMulti
- can an element from the hit list array be matched with multiple elements from the target array (defaults tofalse
)
skipSort
- set it totrue
if arrays are not to be sortedelFreq
- output param - an array in which element frequency is to be recorded (see the example given in theuniqueA
method description)
ArrayDiffResult
contains results of comparing two arrays. It has the following structure:
{
// an array of elements from left array (`leftA` param), which have not
// been matched with any of the elements of the right array (`rightA` param)
leftDiff:Array,
// an array of elements from left array (`leftA` param),
// which have been matched with at leas one element of the right array (`rightA` param)
leftCommon:Array,
// an array of elements from right array (`rightA` param), which have not
// been matched with any of the elements of the left array (`leftA` param)
rightCommon:Array,
// an array of elements from right array (`rightA` param),
// which have been matched with at leas one element of the left array (`leftA` param)
rightDiff:Array
}
ArrayLinkResult
contains results of linking two arrays. It has the following structure:
{
// parent elements with no matched child elements
childless:Array,
// parent elements with matching children
parents:Array,
// child elements with matching parent
children:Array,
// child elements without a matching parent
orphans:Array
}
MIT License, http://www.opensource.org/licenses/MIT