Skip to content

Commit

Permalink
improve code and add retry
Browse files Browse the repository at this point in the history
  • Loading branch information
Rod Lewis committed Jul 17, 2022
1 parent 1065f9c commit 38abc8a
Show file tree
Hide file tree
Showing 10 changed files with 111 additions and 71 deletions.
79 changes: 44 additions & 35 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,63 +21,72 @@ or

### import

import { doHardwork, fetchTool, createWorkerPromise } from "worker-mate"
import { fnWorker, fetchWorker } from "worker-mate"


## doHardwork
## fnWorker
Perform a long runnning or expensive task in a worker with a simple promise interface.

### example

const contrivedFn = (arrayOfNumbers) => arrayOfNumbers.map(n => n ** n).filter(n => n > 9999).sort()[0]
const contrivedArray = [420, 10, 225, 50,100,1000]
let largestSquare = await doHardwork(contrivedFn, contrivedArray).then(n => n)
let largestSquare = await fnWorker(contrivedFn, contrivedArray).then(n => n)

> **Tip:** Each **doHardwork()** opens in a new web worker
> **Tip:** Each **fnWorker()** opens in a new web worker


### doHardwork function arguments
doHardwork requires two arguments. The first should be a pure function, that takes the second, your unprocessed data does some expensive computation and returns the result. Both arguments are required. Side effects are not recommended.
### props
fnWorker requires two arguments. The first should be a pure function, that takes the second, your unprocessed data does some expensive computation and returns the result. Both arguments are required. Side effects are not recommended.
- fn - required
- rawData- required

## fetchTool
## fetchWorker
Fetch with middleware in a worker. Offload expensive data transformations onto their own thread. Need to mutate the body of a request? No dramas, we've got you covered.
### example
fetchTool<Record<string, any>>({ url: 'https://swapi.dev/api/starships/9', responseMiddleware: (d) => ({
name: d?.name ?? '',
model: d?.model ?? '',
manufacturer: d?.manufacturer ?? '',
})})
.then((d: { name: string; model: string; manufacturer: string }) => { setStar(d) })
.catch((err) => console.log(err));
}, []);
## createWorkerPromise
This is the function we created to create fetchTool and doHardwork. If we aren't covering your use case, create your own.
### example
import { serializeFunction } from '.';
import { createWorkerPromise } from './createWorkerPromise';
import FetchWorker from '../worker/fetch_worker.ts?worker&inline';

export interface FetchToolProps {
fetchWorker<Record<string, any>>({
url: 'https://swapi.dev/api/starships/9',
responseMiddleware: (d) => ({
name: d?.name ?? '',
model: d?.model ?? '',
manufacturer: d?.manufacturer ?? ''
}),
retry: {
attempts: 2,
delay: 1000
}
})
.then((d: { name: string; model: string; manufacturer: string }) => { setStar(d) })
.catch((err) => console.log(err));

### props
- fetchProps: FetchToolProps

interface FetchToolProps {
url: string;
body?: any;
options?: RequestInit;
requestMiddleware?: Function;
responseMiddleware?: Function;
retry?: {
attempts: number;
delay: number;
};
}

export const fetchTool = <T>({ url, body, requestMiddleware, responseMiddleware, options }: FetchToolProps) => {
return createWorkerPromise<T>(FetchWorker, {
url,
body,
options,
requestMiddleware: requestMiddleware && serializeFunction(requestMiddleware),
responseMiddleware: responseMiddleware && serializeFunction(responseMiddleware)
})
}

#### url and options
url is the same as the first option in the fetch api and options is exactly the same as the second option. Please refer to [fetch API MDN](https://developer.mozilla.org/en-US/docs/Web/API/fetch) for details. url is equivalant to resource and options is options.

#### requestMiddleware
requestMiddleware is an optional function you can provide to process data before you send a POST or PUT request. Add your unprocessed data to the body of options using JSON.stringify and your function will be executed on the data in the web worker.

#### responseMiddleware
responseMiddleware is an optional function you can provide to process data from a response. This will run in the web worker.

#### retry
retry is an object with two entries := attempts is the amount of times you want to attempt a request and delay is the time you want to wait between attempts.



## Why Worker Mate?

Worker mate is just Typescript with no dependencies. It makes offloading expensive computations to web workers simple . This allows you to keep the main thread clear and your site responsive. It's a super simple, easy to use function that returns a promise. Each instantiation creates a new web worker thread, which terminates itself once the request is complete.
Expand Down
15 changes: 9 additions & 6 deletions src/lib/utils/fetchTool.ts → src/lib/fetchWorker/index.ts
Original file line number Diff line number Diff line change
@@ -1,21 +1,24 @@
import { serializeFunction } from '.';
import { createWorkerPromise } from './createWorkerPromise';
import { serializeFunction } from '../utils';
import { createWorkerPromise } from '../utils/createWorkerPromise';
import FetchWorker from '../worker/fetch_worker.ts?worker&inline';

export interface FetchToolProps {
url: string;
body?: any;
options?: RequestInit;
requestMiddleware?: Function;
responseMiddleware?: Function;
retry?: {
attempts: number;
delay: number;
};
}

export const fetchTool = <T>({ url, body, requestMiddleware, responseMiddleware, options }: FetchToolProps) => {
export const fetchWorker = <T>({ url, requestMiddleware, responseMiddleware, options, retry }: FetchToolProps) => {
return createWorkerPromise<T>(FetchWorker, {
url,
body,
options,
requestMiddleware: requestMiddleware && serializeFunction(requestMiddleware),
responseMiddleware: responseMiddleware && serializeFunction(responseMiddleware)
responseMiddleware: responseMiddleware && serializeFunction(responseMiddleware),
retry
})
}
7 changes: 7 additions & 0 deletions src/lib/fnWorker/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import { createWorkerPromise } from '../utils/createWorkerPromise';
import DoWork from '../worker/worker.ts?worker&inline';
import { serializeFunction } from '../utils';

export const fnWorker = <T>(fn: Function, rawData: any) => {
return createWorkerPromise<T>(DoWork, {fn: serializeFunction(fn), rawData})
}
5 changes: 2 additions & 3 deletions src/lib/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,2 @@
export { doHardwork } from "./utils/doHardwork";
export { fetchTool } from "./utils/fetchTool";
export { createWorkerPromise } from "./utils/createWorkerPromise";
export { fnWorker } from './fnWorker';
export { fetchWorker } from './fetchWorker';
3 changes: 2 additions & 1 deletion src/lib/types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ export interface WorkerResponseType<T> {
isTrusted: boolean;
data: {
type: 'error' | 'data';
data: T;
data?: T;
error?: Error;
};
}
2 changes: 1 addition & 1 deletion src/lib/utils/createWorkerPromise.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ export const createWorkerPromise = <T>(Worker: new () => Worker, postMessage: Re
if (worker) {
worker.onmessage = (e: WorkerResponseType<T>) => {
if(e.isTrusted) {
e.data?.type === 'error' ? reject(e.data.data) : resolve(e.data.data);
e.data?.type === 'error' ? reject(e.data.error) : resolve(e.data?.data as T);
}
worker?.terminate();
}
Expand Down
7 changes: 0 additions & 7 deletions src/lib/utils/doHardwork.ts

This file was deleted.

6 changes: 6 additions & 0 deletions src/lib/utils/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,9 @@ export const deserializeFunction = (s: string) =>
new Function(`return ${decodeURI(s)}`)();
export const methodType = (options: RequestInit | undefined) =>
options?.method?.toUpperCase() ?? "GET";

export function* generateRetryDelay(retry: { attempts: number; delay: number }) {
for (let i = 0; i < retry.attempts; i++) {
yield new Promise(resolve => i === 0 ? resolve(void 0) : setTimeout(resolve, retry.delay));
}
}
54 changes: 38 additions & 16 deletions src/lib/worker/fetch_worker.ts
Original file line number Diff line number Diff line change
@@ -1,25 +1,47 @@
import { deserializeFunction, isFunction, methodType } from '../utils';
import { deserializeFunction, generateRetryDelay, isFunction, methodType } from '../utils';

self.addEventListener('message', async (event) => {
if(!event.isTrusted) return;
let { data } = event;
let { url, body, options } = data;
let { url, options, retry } = data;
let method = methodType(options);
if((method === 'POST' || method === 'PUT') && body) {
let requestMiddleware = data?.requestMiddleware && deserializeFunction(data.requestMiddleware);
options = { ...options, body: isFunction(requestMiddleware) ? JSON.stringify(requestMiddleware(body)) : JSON.stringify(body) };
if((method === 'POST' || method === 'PUT') && data?.requestMiddleware) {
let requestMiddleware = deserializeFunction(data.requestMiddleware);
let body = options.body;
if(body && isFunction(requestMiddleware)) {
options = { ...options, body: JSON.stringify(requestMiddleware(JSON.parse(body)))};
}
}
const fetchData = async () => {
try {
let response = await fetch(url, options);
if(response.ok && response.status !== 404 && response.status !== 403) {
let responseData = await response.json();
let responseMiddleware = data?.responseMiddleware && deserializeFunction(data.responseMiddleware);
let responseMiddlewareData = isFunction(responseMiddleware) ? responseMiddleware(responseData) : responseData;
return responseMiddlewareData;
} else {
throw new Error(`${response.status ?? response.statusText}`);
}
} catch (error) {
throw error;
}
}
try {
let response = await fetch(url, options);
if(response.ok && response.status !== 404 && response.status !== 403) {
let responseData = await response.json();
let responseMiddleware = data?.responseMiddleware && deserializeFunction(data.responseMiddleware);
let responseMiddlewareData = isFunction(responseMiddleware) ? responseMiddleware(responseData) : responseData;
self.postMessage({ type: 'data', data: responseMiddlewareData });
} else {
self.postMessage({ type: 'error', data: `Error: ${response.status}` });
if(!retry) {retry = { attempts: 1, delay: 0 }}
let retryDelay = generateRetryDelay(retry);
let response;
while(!response) {
try {
response = await fetchData();
self.postMessage({ type: 'data', data: response });
} catch (err) {
let delay = retryDelay.next();
if(!delay.done) {
await delay.value;
} else {
response = true;
self.postMessage({ type: 'error', error: err });
}
}
} catch (error) {
self.postMessage({ type: 'error', data: (error as Error)?.message ?? error });
}
});
4 changes: 2 additions & 2 deletions src/lib/worker/worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ self.addEventListener('message', async (event) => {
if (!isFunction(fn)) {
self.postMessage({
type: 'error',
data: 'no function provided',
error: 'no function provided',
});
} else if (!data?.rawData) {
self.postMessage({ type: 'error', data: 'no data provided' });
self.postMessage({ type: 'error', error: 'no data provided' });
} else {
self.postMessage({ type: 'data', data: fn(data.rawData) });
}
Expand Down

0 comments on commit 38abc8a

Please sign in to comment.