PDF generation in local dev environment is broken

[Cruikshanks]

We've spotted if you try to send a paper form notification the UI will tell you all is well but nothing gets sent to Notify and we see an error in the logs

5|service-background  | 2023-10-18T08:42:45.221Z - error: Error sending batch message stack=Error: Could not find Chromium (rev. 1108766). This can occur if either
5|service-background  |  1. you did not perform an installation before running the script (e.g. `npm install`) or
5|service-background  |  2. your cache path is incorrectly configured (which is: /root/.cache/puppeteer).
5|service-background  | For (2), check out our guide on configuring puppeteer at https://pptr.dev/guides/configuration.
5|service-background  |     at ChromeLauncher.resolveExecutablePath (/home/repos/water-abstraction-service/node_modules/puppeteer-core/lib/cjs/puppeteer/node/ProductLauncher.js:127:27)
5|service-background  |     at ChromeLauncher.executablePath (/home/repos/water-abstraction-service/node_modules/puppeteer-core/lib/cjs/puppeteer/node/ChromeLauncher.js:207:25)
5|service-background  |     at ChromeLauncher.launch (/home/repos/water-abstraction-service/node_modules/puppeteer-core/lib/cjs/puppeteer/node/ChromeLauncher.js:93:37)
5|service-background  |     at async createPdf (/home/repos/water-abstraction-service/src/lib/services/pdf-generation/pdf.js:15:19)
5|service-background  |     at async Object.sendPDF [as pdf] (/home/repos/water-abstraction-service/src/modules/batch-notifications/lib/notify-connector.js:48:15)
5|service-background  |     at async handleSendMessage (/home/repos/water-abstraction-service/src/modules/batch-notifications/lib/jobs/send-message.js:39:28)
5|service-background  |     at async Promise.all (index 0)
5|service-background  |     at async Worker.handler [as processFn] (/home/repos/water-abstraction-service/src/modules/batch-notifications/lib/jobs/send-message.js:52:5)
5|service-background  |     at async Worker.processJob (/home/repos/water-abstraction-service/node_modules/bullmq/dist/cjs/classes/worker.js:333:28)
5|service-background  |     at async Worker.retryIfFailed (/home/repos/water-abstraction-service/node_modules/bullmq/dist/cjs/classes/worker.js:455:24)
5|service-background  |     at async Worker.run (/home/repos/water-abstraction-service/node_modules/bullmq/dist/cjs/classes/worker.js:140:46), component=/modules/batch-notifications/lib/jobs/send-message.js:42:12, messageId=5df786ca-867c-4920-97e9-a8b95e16bf54

We think it's because of changes in the latest version of chromium which we install as part of the Docker image we build and use locally. This is the output when connected to the running docker dev container and we try and just check the version of Chromium.

root@218cea6a0b7a:/home# chromium -v
find: '/root/.config/chromium/Crash Reports/pending/': No such file or directory
[834:834:1018/084414.447708:ERROR:zygote_host_impl_linux.cc(100)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

root@218cea6a0b7a:/home# chromium -v --no-sandbox
[855:882:1018/084427.366408:ERROR:bus.cc(406)] Failed to connect to the bus: Failed to connect to socket /run/dbus/system_bus_socket: No such file or directory
[855:855:1018/084427.423336:ERROR:ozone_platform_x11.cc(240)] Missing X server or $DISPLAY
[855:855:1018/084427.423395:ERROR:env.cc(255)] The platform failed to initialize.  Exiting.
root@218cea6a0b7a:/home#

Initial thoughts were a change in chromium which means we must pass the --no-sandbox flag when we call it from puppeteer.

// src/lib/services/pdf-generation/pdf.js
const createPdf = async html => {
  const browser = await puppeteer.launch({
    args: ['--no-proxy-server']
  })
  const page = await browser.newPage()

  await page.setContent(html)

  const buffer = await page.pdf({ format: 'A4' })
  await browser.close()
  return buffer
}

We tried that though and got the same error.

So, this is going to need a bit more digging. We've prided ourselves on creating a complete working environment locally taking no shortcuts like disabling ClamAV. But this means we can no longer say that until we get this working again.

[Cruikshanks]

Links to things that might be of use

• https://pptr.dev/api/puppeteer.configuration
• https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md

[jonathangoulding]

This can be fixed by specifying the chromium path and using the --no-sandbox' arg.

Source - https://pptr.dev/guides/configuration

Proposed config change:

  const browser = await puppeteer.launch({
    executablePath: '/usr/bin/chromium',
    args: [
        '--no-proxy-server'
      , '--no-sandbox'
    ]
  })

---

Error Chromium install

Error sending batch message stack=Error: Could not find Chromium (rev. 1108766). This can occur if either
4|service-background | 1. you did not perform an installation before running the script (e.g. npm install) or
4|service-background | 2. your cache path is incorrectly configured (which is: /root/.cache/puppeteer).

Fix

Specify the path to chromium in the puppeteer config

    executablePath: '/usr/bin/chromium',

---

Error running as root

Running as root without --no-sandbox is not supported. See https://crbug.com/638180.

Fix

    args: [ '--no-sandbox' ]

We probably only need to set this for local development based on my current understanding. So we can set an env locally of in the docker container to enable this config changes.

A spike PR has been created to prove this solution works - DEFRA/water-abstraction-service#2651

[jonathangoulding]

As a result of this fix a new error occurs

Error sending batch message stack=TypeError: Cannot read properties of undefined (reading 'body')

I have assumed this is another error but out side the scope of this issue to fix.