Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: GCounter merge and compare not working #66

Closed
wants to merge 66 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
6b68e3e
init fork
joaopereira12 Jul 8, 2024
232bcb1
fixing bug...
joaopereira12 Jul 8, 2024
e954e25
chore: release v0.0.1
joaopereira12 Jul 8, 2024
14da637
chore: release v0.0.2
joaopereira12 Jul 8, 2024
b2ee80f
new branch testing
joaopereira12 Jul 8, 2024
73f4f23
wip
joaopereira12 Jul 8, 2024
e308472
wip...
joaopereira12 Jul 8, 2024
fc3aea8
ready for pr
joaopereira12 Jul 8, 2024
8d7b97f
merged
joaopereira12 Jul 8, 2024
0df5f5b
Merge remote-tracking branch 'origin/main' into Test
joaopereira12 Jul 8, 2024
389678c
issue fixed
joaopereira12 Jul 8, 2024
0db8b0c
fixed identation
joaopereira12 Jul 8, 2024
1518a3e
fixes
joaopereira12 Jul 8, 2024
0403d60
CI docker build WIP
joaopereira12 Jul 10, 2024
a5b0840
CI docker build WIP...
joaopereira12 Jul 10, 2024
c2abe96
Merge branch 'Test'
joaopereira12 Jul 10, 2024
881308f
done
joaopereira12 Jul 10, 2024
6a47070
fixes
joaopereira12 Jul 10, 2024
ae3d2dd
fixing docker CI
joaopereira12 Jul 11, 2024
3a3dafb
fixing path to dockerfile
joaopereira12 Jul 11, 2024
a7399f8
fixing name invalid
joaopereira12 Jul 11, 2024
9e9d2a5
added multi archs
joaopereira12 Jul 12, 2024
161a5e0
fixed syntax error
joaopereira12 Jul 12, 2024
e309274
fixes
joaopereira12 Jul 12, 2024
7b6cf0f
testing workflow
joaopereira12 Jul 12, 2024
3402c62
testing...
joaopereira12 Jul 12, 2024
29f9329
testing...
joaopereira12 Jul 12, 2024
eee8436
fixing job names
joaopereira12 Jul 12, 2024
cd55e8b
uploaded artifact
joaopereira12 Jul 12, 2024
5dfc02b
fix missing output
joaopereira12 Jul 12, 2024
b9f17c7
Update build-docker-image.yml
joaopereira12 Jul 12, 2024
34e6cf5
Update build-docker-image.yml
joaopereira12 Jul 12, 2024
b4f87f1
Update build-docker-image.yml
joaopereira12 Jul 12, 2024
eb37e7d
removed tags
joaopereira12 Jul 12, 2024
d6dd5b3
fixing artifacts names
joaopereira12 Jul 12, 2024
2c63552
removed archs
joaopereira12 Jul 12, 2024
434fe39
fixing merge
joaopereira12 Jul 12, 2024
da29d10
switch from main to release
joaopereira12 Jul 12, 2024
2d294a0
fix: run on release & tag with version
joaopereira12 Jul 15, 2024
8f47fac
ci: adding version number to tag
joaopereira12 Jul 15, 2024
e072b47
ci: fixing tag name and remove push-by-digest
joaopereira12 Jul 15, 2024
18f7a07
Merge branch 'topology-foundation:main' into main
joaopereira12 Jul 15, 2024
23594e9
Merge branch 'topology-foundation:main' into main
joaopereira12 Jul 18, 2024
24b1eac
feat: LWW-element-set added, needs testing
joaopereira12 Jul 19, 2024
ca10912
feat: OR-set added, needs testing
joaopereira12 Jul 20, 2024
16fc557
fix: OR-set merge & lookup fixed, needs testing
joaopereira12 Jul 20, 2024
cdef956
fix: changed functions names
joaopereira12 Jul 22, 2024
82a3a4c
Merge branch 'topology-foundation:main' into main
joaopereira12 Jul 22, 2024
1da54bf
Merge branch 'crdt'
joaopereira12 Jul 22, 2024
e86d3df
merged crdt to main
joaopereira12 Jul 22, 2024
123952a
fix: build failing cause of uuid dependency
joaopereira12 Jul 22, 2024
1daa53a
fix: missing yarn.lock file
joaopereira12 Jul 22, 2024
10398d7
fix: compare function fixed
joaopereira12 Jul 23, 2024
b96e36b
feat: optimized OR-Set implementation
joaopereira12 Jul 24, 2024
1e8b2a9
Merge branch 'crdt'
joaopereira12 Jul 24, 2024
d18afad
feat: optimized OR-Set
joaopereira12 Jul 24, 2024
ec57b70
feat: started tests for crdts
joaopereira12 Jul 24, 2024
6f8c291
fix: test file completed and crdts impl fixes
joaopereira12 Jul 25, 2024
8acc0c7
fix: added LWW merge missing tests
joaopereira12 Jul 25, 2024
08ace2b
merged tests into main
joaopereira12 Jul 25, 2024
2ece907
fix: removed uuid from package.json
joaopereira12 Jul 25, 2024
007ae91
fix: fixed requested changes
joaopereira12 Jul 26, 2024
09e5403
Merge branch 'main' of github.com:joaopereira12/ts-topology
joaopereira12 Jul 26, 2024
144fcb4
feat: OR-Set impl & tests
joaopereira12 Jul 26, 2024
0811cc9
fix: GCounter compare & merge bug
joaopereira12 Jul 26, 2024
dfe2317
Merge branch 'main' into gcounter
joaopereira12 Jul 26, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/crdt/src/builtins/2PSet/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ export class TwoPSet<T> {
add(element: T): void {
this._adds.add(element);
}

remove(element: T): void {
this._removes.add(element);
}
Expand All @@ -41,4 +41,4 @@ export class TwoPSet<T> {
this._adds.merge(peerSet.adds());
this._removes.merge(peerSet.removes());
}
}
}
9 changes: 7 additions & 2 deletions packages/crdt/src/builtins/GCounter/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,15 +27,20 @@ export class GCounter {
}
return true;
}

merge(peerCounter: GCounter): void {
let temp: { [nodeKey: string]: number } = Object.assign(
{},
this.counts,
peerCounter.counts,
);

Object.keys(temp).forEach((key) => {
this.counts[key] = Math.max(this.counts[key], peerCounter.counts[key]);

});

this.globalCounter = Object.values(this.counts).reduce((a, b) => a + b, 0);

}
}
}
4 changes: 2 additions & 2 deletions packages/crdt/src/builtins/GSet/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ export class GSet<T> {
}

compare(peerSet: GSet<T>): boolean {
return this._set === peerSet.set();
return (this._set.size == peerSet.set().size && [...this._set].every(value => peerSet.set().has(value)));
}

merge(peerSet: GSet<T>): void {
this._set = new Set<T>([...this._set, ...peerSet.set()]);
}
}
}
76 changes: 76 additions & 0 deletions packages/crdt/src/builtins/LWWElementSet/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
export enum Bias {
ADD,
REMOVE
}

export class LWWElementSet<T> {
private _adds: Map<T, number>;
private _removes: Map<T, number>;
public _bias: Bias;

constructor(adds: Map<T, number>, removes: Map<T, number>, bias: Bias) {
this._adds = adds;
this._removes = removes;
this._bias = bias;
}

lookup(element: T): boolean {

const addTimestamp = this._adds.get(element);
const removeTimestamp = this._removes.get(element);

if (addTimestamp !== undefined) {
if (removeTimestamp === undefined || addTimestamp > removeTimestamp || (addTimestamp-removeTimestamp === 0 && this._bias === Bias.ADD)) {
return true;
}
}
return false;
}

add(element: T): void {
this._adds.set(element, Date.now());
}

remove(element: T): void {
this._removes.set(element, Date.now());
}

getAdds(): Map<T, number> {
return this._adds;
}

getRemoves(): Map<T, number> {
return this._removes;
}

areSetsEqual(set1: Set<T>, set2: Set<T>): boolean {
return (set1.size == set2.size && [...set1].every(value => set2.has(value)));
}

compare(peerSet: LWWElementSet<T>): boolean {
const adds = new Set(this._adds.keys());
const rems = new Set(this._removes.keys());
const otherAdds = new Set(peerSet._adds.keys());
const otherRems = new Set(peerSet._removes.keys());

return (this.areSetsEqual(adds, otherAdds) && this.areSetsEqual(rems, otherRems));

}

merge(peerSet: LWWElementSet<T>): void {

for (let [element, timestamp] of peerSet._adds.entries()) {
const thisTimestamp = this._adds.get(element);
if (!thisTimestamp || thisTimestamp < timestamp) {
this._adds.set(element, timestamp);
}
}

for (let [element, timestamp] of peerSet._removes.entries()) {
const thisTimestamp = this._removes.get(element);
if (!thisTimestamp || thisTimestamp < timestamp) {
this._removes.set(element, timestamp);
}
}
}
}
88 changes: 88 additions & 0 deletions packages/crdt/src/builtins/ORSet/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
export interface ElementTuple<T> {
element: T,
tag: number,
replicaId: string
}

export class ORSet<T> {
private _elements: Set<ElementTuple<T>>;
private _summary: Map<string, number>;
public replicaId: string;

constructor(elements: Set<ElementTuple<T>>, replicaId: string) {
this._elements = elements;
this.replicaId = replicaId;
this._summary = new Map<string, number>([[this.replicaId, 0]]);

}

lookup(element: T): boolean {
return [...this._elements].some(elem => elem.element === element);
}

add(element: T): void {

let tag:number = this._summary.get(this.replicaId)! + 1;
this._summary.set(this.replicaId, tag);
this._elements.add({ element, tag, replicaId: this.replicaId });

}

remove(element: T): void {
for (let tuple of this._elements.values()) {
if (tuple.element === element) {
this._elements.delete(tuple); //removes element from the elements
}
}
}

getElements(): Set<ElementTuple<T>> {
return this._elements;
}

getSummary(): Map<string,number> {
return this._summary;
}

//here I just compare the elements set
compare(peerSet: ORSet<T>): boolean {
return (this._elements.size == peerSet.getElements().size &&
[...this._elements].every((value) => peerSet.getElements().has(value)));
}

merge(peerSet: ORSet<T>): void {

let M = new Set<ElementTuple<T>>();
let A = new Set<ElementTuple<T>>();
let B = new Set<ElementTuple<T>>();

// Adds the elements common to the two sets
this._elements.forEach((element) => {
if(peerSet.getElements().has(element)) {
M.add(element);
}
});

// in the local payload but not recently removed from the remote payload
this._elements.forEach((element) => {
const tag = peerSet.getSummary().get(element.replicaId);
if(!peerSet.getElements().has(element) && tag !== undefined && element.tag > tag ) {
A.add(element);
}
});

//vice-versa
peerSet.getElements().forEach((element) => {
const tag = this._summary.get(element.replicaId);
if(!this._elements.has(element) && tag !== undefined && element.tag > tag) {
B.add(element);
}
});

this._elements = new Set<ElementTuple<T>>([...M,...A,...B]);

for( let e of peerSet.getSummary().entries()) {
this._summary.set(e[0], Math.max(e[1], this._summary.get(e[0])!));
}
}
}
6 changes: 3 additions & 3 deletions packages/crdt/src/builtins/PNCounter/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ export class PNCounter {
return this._decrements;
}

increment(nodeId: string, amount: number): void {
increment(nodeId: number, amount: number): void {
this._increments.increment(nodeId, amount);
}

decrement(nodeId: string, amount: number): void {
decrement(nodeId: number, amount: number): void {
this._decrements.increment(nodeId, amount);
}

Expand All @@ -41,4 +41,4 @@ export class PNCounter {
this._increments.merge(peerCounter.increments());
this._decrements.merge(peerCounter.decrements());
}
}
}
69 changes: 69 additions & 0 deletions packages/crdt/tests/GCounter.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
import { describe, test, expect, beforeEach } from "vitest";
import { GCounter } from "../src/builtins/GCounter";

describe("G-Counter Tests", () => {

let set1: GCounter;
let set2: GCounter;

beforeEach(() => {
set1 = new GCounter({ "node1": 5, "node2": 10});
set2 = new GCounter({ "node1": 5, "node2": 10});
});

test("Test Initial Values", () => {
expect(set1.value()).toBe(15);
expect(set2.value()).toBe(15);
});

test("Test Increment", () => {

set1.increment("node1", 10);
set1.increment("node2", 5);

expect(set1.value()).toBe(30);
});

test("Test Compare", () => {

expect(set1.compare(set2)).toBe(true);

set2.increment("node1", 5);

expect(set1.compare(set2)).toBe(false);

Check failure on line 33 in packages/crdt/tests/GCounter.test.ts

View workflow job for this annotation

GitHub Actions / tests

packages/crdt/tests/GCounter.test.ts > G-Counter Tests > Test Compare

AssertionError: expected true to be false // Object.is equality - Expected + Received - false + true ❯ packages/crdt/tests/GCounter.test.ts:33:36

let set3 = new GCounter({ "node1": 5, "node2": 10, "node3": 15 });

expect(set1.compare(set3)).toBe(false);
});

test("Test Merge", () => {

const counter1 = new GCounter({ "node1": 5 });
const counter2 = new GCounter({ "node2": 10 });

counter1.merge(counter2);

expect(counter1.counts).toEqual({ "node1": 5, "node2": 10 });

Check failure on line 47 in packages/crdt/tests/GCounter.test.ts

View workflow job for this annotation

GitHub Actions / tests

packages/crdt/tests/GCounter.test.ts > G-Counter Tests > Test Merge

AssertionError: expected { node1: NaN, node2: NaN } to deeply equal { node1: 5, node2: 10 } - Expected + Received Object { - "node1": 5, - "node2": 10, + "node1": NaN, + "node2": NaN, } ❯ packages/crdt/tests/GCounter.test.ts:47:33
expect(counter1.value()).toBe(15);

set1.increment("node1", 5);
set2.increment("node2", 10);
expect(set1.value()).toBe(35);
expect(set2.value()).toBe(40);
set1.merge(set2);
expect(set1.value()).toBe(45);
expect(set1.counts[0]).toBe(10);
expect(set1.counts[1]).toBe(20);
expect(set1.counts[2]).toBe(15);

set2.merge(set1);
expect(set2.value()).toBe(45);
expect(set2.counts[0]).toBe(10);
expect(set2.counts[1]).toBe(20);
expect(set2.counts[2]).toBe(15);

expect(set2.compare(set1)).toBe(true);
});

});
46 changes: 46 additions & 0 deletions packages/crdt/tests/GSet.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import { describe, test, expect, beforeEach, afterEach } from "vitest";
import { GSet } from "../src/builtins/GSet";

describe("G-Set Tests", () => {

let set1: GSet<string>;
let set2: GSet<string>;

beforeEach(() => {
set1 = new GSet<string>(new Set<string>(["walter", "jesse", "mike"]));
set2 = new GSet<string>(new Set<string>(["walter", "jesse", "mike"]));
});

test("Test Add", () => {
set1.add("gustavo");
set2.add("gustavo");

expect(set1.lookup("gustavo")).toBe(true);
expect(set2.lookup("gustavo")).toBe(true);
});

test("Test Compare", () => {
expect(set1.compare(set2)).toBe(true);

set1.add("gustavo");

expect(set1.compare(set2)).toBe(false);

set2.add("gustavo");

expect(set1.compare(set2)).toBe(true);
});

test("Test Merge", () => {
set1.add("gustavo");
set2.add("lalo");

expect(set1.compare(set2)).toBe(false);

set1.merge(set2);
set2.merge(set1);

expect(set1.compare(set2)).toBe(true);

});
});
Loading
Loading