Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add batchwrite functionality #144

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .github/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,75 @@ firestore.deleteDocument("FirstCollection/FirstDocument");
```
**Note:** This cannot handle deleting collections or subcollections, *only* individual documents.

##### Batch Write
To do multiple operations in one request, you can utilize `WritePatch` api. It supports two modes of executing the writes, `atomic` and `non-atomic`. In `atomic` mode, it behaves like a transaction, it fails if any of the requested writes fails and changes are rolled-back. In `non-atomic` mode, the changes can happen out of order and each write would fail independently of the remaining writes.

The api contains two restrictions:
- You cannot write more than once to the same document.
- You can have at most 500 write operations.

###### To create a batch write:
```javascript
const batch = firestore.batch();
```

###### To add a document:
At the moment of writing, the WriteBatch does not contain an `add` operation. The reason for that is to ensure the same functionality as Firestore JS SDK. However, we may add it later for ease of use.
To circumvent this limitation, you would use `set` operation with a unique document id.
```javascript
const doc = `collectionName/${firestore.newId()}`;
batch.set(doc, docData);
```
However, this approach will **NOT** fail if the document already exists. This limitation would be an incentive to adding a `create` method that utilizes `preconditions.exists` to ensure failure if document already exist.

###### To set a document:
```javascript
batch.set(doc, docData); // this would overwrite the document if exists, create if not
batch.set(doc, docData, {merge: true}); // this would update/merge the document if exists, create if not
batch.set(doc, docData, {mergeFields: ['field1', 'field2']}); // this allows to pass a write mask, only these fields would be set. If a field exists in the mask but not in data, it would be deleted from the document
```
`merge` and `mergeFields` are mutually exclusive, if both are provided, `merge` takes precedence.


###### To update a document:
```javascript
batch.update(doc, docData);
```
Another option for update:
```javascript
batch.update(doc, 'field1', field1Data);
batch.update(doc, 'field1', field1Data, 'field2', field2Data, ...);
```
This variant is added for mere compatibility with the Firestore JS SDK.
The update operation **fails**, if the document does not exit.

###### To delete a document:
```javascript
batch.delete(doc);
```
This operation does not fail if document does not exist.

###### To execute a batch write atomically:
```javascript
batch.commit(true);
```
If the commit operation fails, it throws an exception.

###### To execute a batch write independently, either pass `false` or `undefined`:
```javascript
const results = batch.commit();
```
or
```javascript
const results = batch.commit(false);
```
The result of the commit operation is an array with either `true` or error message per each write operation. The order of the result array is guaranteed to have the same order as the operations in the WriteBatch.

You cannot commit a WriteBatch more than once. The commit method will throw an exception if there are no write operations. You can use `isEmpty` getter, to check if the batch contains any writes.
```javascript
const isEmpty = batch.isEmpty;
```

##### Getting Documents
You can retrieve documents by calling the `getDocument` function:

Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
.clasp.json
.vs
.idea
gapps.config.json
node_modules
17 changes: 17 additions & 0 deletions Firestore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,23 @@ class Firestore implements FirestoreRead, FirestoreWrite, FirestoreDelete {
return this.query_(path, request);
}
query_ = FirestoreRead.prototype.query_;

/**
* Create a batch update
*/
batch(): WriteBatch {
return new WriteBatch(this);
}

/**
* Generates a new unique id
* Useful for WriteBatch create document
*
* @return {string} unique doc id
*/
newId(): string {
return Util_.newId();
}
abkarino marked this conversation as resolved.
Show resolved Hide resolved
}

type Version = 'v1' | 'v1beta1' | 'v1beta2';
Expand Down
4 changes: 3 additions & 1 deletion Request.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,9 @@ class Request {
this.queryString = '';
this.authToken = authToken || '';

if (!this.authToken) options = options || {};
if (!this.authToken) {
options = options || {};
}
// Set default header options if none are passed in
this.options = options || {
headers: {
Expand Down
32 changes: 32 additions & 0 deletions Util.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,4 +148,36 @@ class Util_ {
.map(([k, v]) => `${process(k)}=${process(v)}`)
.join('&');
}

/**
* Create a new unique document id
*
* @return {string} a new unique document id
* @link https://github.com/firebase/firebase-js-sdk/blob/34ad43cc2a9863f7ac326c314d9539fcbc1f0913/packages/firestore/src/util/misc.ts#L28
*/
static newId(): string {
// Alphanumeric characters
const chars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
// The largest byte value that is a multiple of `char.length`.
const maxMultiple = Math.floor(256 / chars.length) * chars.length;

let autoId = '';
const targetLength = 20;
while (autoId.length < targetLength) {
const nBytes = 40;
const bytes = new Uint8Array(nBytes);
for (let i = 0; i < nBytes; i++) {
bytes[i] = Math.floor(Math.random() * 256);
}
for (let i = 0; i < bytes.length; ++i) {
// Only accept values that are [0, maxMultiple), this ensures they can
// be evenly mapped to indices of `chars` via a modulo operation.
if (autoId.length < targetLength && bytes[i] < maxMultiple) {
autoId += chars.charAt(bytes[i] % chars.length);
}
}
}

return autoId;
}
}
174 changes: 174 additions & 0 deletions WriteBatch.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
/**
* Write multiple documents in one request.
* Can be atomic or not.
*/
class WriteBatch {
#mutations: FirestoreAPI.Write[] = [];
#committed = false;

/**
* Getter to check if a WriteBatch has pending write
*/
public get isEmpty() {
return this.#mutations.length === 0;
}

/**
* A container for multiple writes
* @param {Firestore} _firestore the parent instance
* @param {Boolean} _atomic the REST api supports batch writes in a non-atomic way
*/
constructor(private readonly _firestore: Firestore) {}

/**
* Writes to the document referred to by the provided path.
* If the document does not exist yet, it will be created.
*
* @param {string} path - A path to the document to be set.
* @param data - An object of the fields and values for the document.
* @returns This `WriteBatch` instance. Used for chaining method calls.
*/
set(path: string, fields: Record<string, any>): WriteBatch;
/**
* Writes to the document referred to by the provided path.
* If the document does not exist yet, it will be created.
* If you provide `merge` or `mergeFields`, the provided data can be merged
* into an existing document.
*
* @param {string} path - A path to the document to be set.
* @param data - An object of the fields and values for the document.
* @param options - An object to configure the set behavior.
* @throws Error - If the provided input is not a valid Firestore document.
* @returns This `WriteBatch` instance. Used for chaining method calls.
*/
set(path: string, fields: Record<string, any>, options: Record<string, any>): WriteBatch;
set(
path: string,
fields: Record<string, any>,
options?: Record<string, any> // FirestoreAPI.SetOptions
): WriteBatch {
this.verifyNotCommitted_();

const isMerge = options && (options.merge || options.mergeFields);
const updateMask: FirestoreAPI.DocumentMask | undefined = isMerge
? { fieldPaths: options.merge ? Object.keys(fields) : options.mergeFields }
: undefined;
const update: FirestoreAPI.Document = new Document(fields, `${this._firestore.basePath}${path}`);

const mutation: FirestoreAPI.Write = {
updateMask: updateMask,
update: update,
};
this.#mutations.push(mutation);
return this;
}

/**
* Updates fields in the document referred to by the provided path.
* The update will fail if applied to a document that does
* not exist.
*
* @param {string} path - A path to the document to be updated.
* @param data - An object containing the fields and values with which to
* update the document. Fields can contain dots to reference nested fields
* within the document.
* @throws Error - If the provided input is not valid Firestore data.
* @returns This `WriteBatch` instance. Used for chaining method calls.
*/
update(path: string, data: Record<string, any>): WriteBatch;
/**
* Updates fields in the document referred to by the provided path.
* The update will fail if applied to a document that does
* not exist.
*
* Nested fields can be update by providing dot-separated field path strings.
*
* @param {string} path - A path to the document to be updated.
* @param field - The first field to update.
* @param value - The first value.
* @param moreFieldsAndValues - Additional key value pairs.
* @throws Error - If the provided input is not valid Firestore data.
* @returns This `WriteBatch` instance. Used for chaining method calls.
*/
update(path: string, field: string, value: unknown, ...moreFieldsAndValues: unknown[]): WriteBatch;
update(
path: string,
fieldOrUpdateData: string | Record<string, any>,
value?: unknown,
...moreFieldsAndValues: unknown[]
): WriteBatch {
this.verifyNotCommitted_();

let fields;
if (typeof fieldOrUpdateData === 'string') {
fields = {
[fieldOrUpdateData]: value,
};
for (let i = 0; i < moreFieldsAndValues.length; i += 2) {
fields[moreFieldsAndValues[i] as string] = moreFieldsAndValues[i + 1];
}
} else {
fields = fieldOrUpdateData;
}
const updateMask: FirestoreAPI.DocumentMask = { fieldPaths: Object.keys(fields) };
const update: FirestoreAPI.Document = new Document(fields, `${this._firestore.basePath}${path}`);

const mutation: FirestoreAPI.Write = {
updateMask: updateMask,
update: update,
currentDocument: {
exists: true,
},
};
this.#mutations.push(mutation);
return this;
}

/**
* Deletes the document referred to by the provided path.
*
* @param {string} path - A path to the document to be deleted.
* @returns This `WriteBatch` instance. Used for chaining method calls.
*/
delete(path: string): WriteBatch {
this.verifyNotCommitted_();
// const ref = validateReference(documentRef, this._firestore);
// new DeleteMutation(ref._key, Precondition.none())
const mutation: FirestoreAPI.Write = {
delete: `${this._firestore.basePath}${path}`,
};
this.#mutations.push(mutation);
return this;
}

/**
* Issue the write request.
* If atomic, a true value is returned on success or throws an error on failure.
* If not atomic, an array of either true or error message is returned.
*/
commit(atomic = false): boolean | Array<true | string | undefined> {
this.verifyNotCommitted_();
this.#committed = true;
if (!this.#mutations.length) {
throw new Error('A write batch cannot commit with no changes requested.');
}

const request = new Request(this._firestore.baseUrl, this._firestore.authToken);
request.route(atomic ? 'commit' : 'batchWrite');

const payload: FirestoreAPI.CommitRequest | FirestoreAPI.BatchWriteRequest = { writes: this.#mutations };
const responseObj = request.post<FirestoreAPI.CommitResponse | FirestoreAPI.BatchWriteResponse>(undefined, payload);

if (atomic) {
return true;
} else {
return ((responseObj as FirestoreAPI.BatchWriteResponse).status || []).map((s) => !s.code || s.message);
}
}

private verifyNotCommitted_(): void {
if (this.#committed) {
throw new Error('A write batch can no longer be used after commit() ' + 'has been called.');
}
}
}
Loading