Skip to content

Commit

Permalink
docs: add documentation on minimizing lambda bundle (#167)
Browse files Browse the repository at this point in the history
* Add e2e test instructions; Add documentation on bundle size
  • Loading branch information
khuezy authored Oct 12, 2023
1 parent 6efbc25 commit d291e6a
Show file tree
Hide file tree
Showing 3 changed files with 82 additions and 5 deletions.
8 changes: 7 additions & 1 deletion docs/pages/common_issues.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,10 @@ If you are using sentry, API routes returns empty body. You could try configurin

#### My ISR page has this cache-control header `s-maxage=2, stale-while-revalidate=2592000`

Given how ISR works, while waiting for the revalidation to happen, the page will be served using this cache control header. This prevent your server from being overloaded by a lot of requests while the revalidation is done. You can read more about it [here](/inner_workings/isr).
Given how ISR works, while waiting for the revalidation to happen, the page will be served using this cache control header. This prevent your server from being overloaded by a lot of requests while the revalidation is done. You can read more about it [here](/inner_workings/isr).

#### Unzipped size must be smaller than 262144000 bytes

AWS Lambda has an unzipped size limit of 250MB. If your app is over this limit, then it is most likely using a node_module library that is too large for serverless or there is a large dev dependency getting bundled.
For example, `pdfjs` has `canvas` optional dependency which takes up 180MB. For more details, [read me](/common_issues/bundle_size).
Note: a large bundle size will increase cold start significantly.
55 changes: 55 additions & 0 deletions docs/pages/common_issues/bundle_size.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import {Callout} from 'nextra/components'


#### Reducing Bundle Size

Next will incorrectly trace dev dependencies to be included in the output `node_modules`, which will significantly increase the lambda bundle. For example, the @swc/core-\* binary is ~33MB!

This comment has been minimized.

Copy link
@feedthejim

feedthejim Oct 31, 2023

hey @khuezy, someone sent me this link. I'm fairly confident this is not correct, are you not using the NFT files to get the files needed for the lambda?

This comment has been minimized.

Copy link
@feedthejim

feedthejim Oct 31, 2023

I double-checked on my end, and a hello world lambda is around 800KB gzipped. If you have a scenario where a lambda is more than 30MB, I'd be happy to look at it.

This comment has been minimized.

Copy link
@khuezy

khuezy Oct 31, 2023

Author Contributor

Hi @feedthejim! Thanks for sharing your knowledge. We aren't using NFT to get the files - that's something we should look into.
What we do is run the user's next build script.

We don't know all the inner workings of Nextjs, so we're liking missing things and doing wrong/bad things.

I reran build and now I can't find the "@swc/core" lib anymore... this is no longer an issue, but I believe this was a problem somewhere in 13.x

This comment has been minimized.

Copy link
@khuezy

khuezy Oct 31, 2023

Author Contributor

While I have you here @feedthejim , can you PTAL at vercel/next.js#57622 when you get the chance, it's blocking me and some other folks from jumping on the Next 14 train.

This comment has been minimized.

Copy link
@feedthejim

feedthejim Oct 31, 2023

I see; thanks for looking! Can you perhaps correct the documentation?

This comment has been minimized.

Copy link
@feedthejim

feedthejim Oct 31, 2023

I'll take a look @khuezy thanks for reporting!


Add this to your next.config.js to help minimize the lambda bundle size:

```typescript
outputFileTracingExcludes: {
'*': [
'@swc/core',
'esbuild',
'uglify-js',
'watchpack',
'webassemblyjs',
'sharp'
],
},
```

<Callout type="warning" emoji="⚠️">
NextJS currently doesn't expose `outputFileTracingExcludes` as an environmental variable so `open-next`cannot programmatically set this like it does for`output`and`outputFileTracingRoot`.
Currently, next uses `webpack` to trace server actions, so you shouldn't add `webpack` to the excludes list, otherwise it will break server actions.
</Callout>

#### Unzipped size must be smaller than 262144000 bytes

To identify the module that's taking up too much space (and isn't serverless friendly):

```bash
du -hs .open-next/server-function/node_modules/* | sort -rh
```

If your app requires the offending library, then consider moving your business logic of the `api` to its own lambda, eg: `/api/v2` => `Api Lambda`

<Callout type="info" emoji="ℹ️">
There is a [PR](https://github.com/sst/open-next/pull/242) to remove some dev dependency from the output node_modules but that requires more testing before it can merge.
</Callout>

#### Common issues

##### Sharp

`sharp` is not needed outside of the `Image Optimization` function so you should not have it as a dependency. But if you are depending on `sharp`, be sure to install it with the correct flags for your lambda.
eg: `--arch=arm64 --platform=linux --target=18 --libc=glibc`

##### pdfjs

If you need to use pdfjs, you should install it with `npm i pdfjs-dist--no-optional` because the optional dep: `canvas` takes about 180MB.

##### Others

Please open an issue or let us know on discord if there are any other modules that are causing issues.
24 changes: 20 additions & 4 deletions packages/tests-e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@ The 3 permutations are:

Their respective `tests/` folders are:

1. [appDirOnly](./tests/appDirOnly)
2. [pagesOnly](./tests/pagesOnly)
3. [appDirAndPages](./tests//appDirAndPages)
1. [appRouter](./tests/appDirOnly)
2. [pagesRouter](./tests/pagesOnly)
3. [appPagesRouter](./tests//appDirAndPages)

Their respective `packages/` are located at:

Expand All @@ -22,9 +22,25 @@ Their respective `packages/` are located at:

The GitHub actions will trigger the [e2e test](/.github/workflows//e2e.yml), which deploys the app in the [Example](/example/) folder. The deploy command is:

```
### Running the tests against the deployed app

1. Deploy the app
```bash
cd examples/sst
npx sst deploy --stage e2e
```
2. Export the URLS
```bash
export APP_ROUTER_URL=$(jq -r '.["e2e-example-AppRouter"].url' .sst/outputs.json)
export PAGES_ROUTER_URL=$(jq -r '.["e2e-example-PagesRouter"].url' .sst/outputs.json)
export APP_PAGES_ROUTER_URL=$(jq -r '.["e2e-example-AppPagesRouter"].url' .sst/outputs.json)
```
3. Run the test
```bash
cd ../packages/tests-e2e
npm run e2e:dev
```


## Gotchas

Expand Down

1 comment on commit d291e6a

@vercel
Copy link

@vercel vercel bot commented on d291e6a Oct 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Successfully deployed to the following URLs:

open-next – ./

open-next-git-main-sst-dev.vercel.app
open-next.vercel.app
open-next-sst-dev.vercel.app

Please sign in to comment.