[Bug]: PHP not installed behind corporate proxy

[jorgelloret]

Platform

Windows

Operating system version

Windows 10

System architecture

Windows

Herd Version

1.9.0

PHP Version

No response

Bug description

Being behind a corporate proxy, PHP is not installed on the first run. There is no option to configure one.

Steps to reproduce

No response

Relevant log output

%USERPROFILE%\AppData\Roaming\Herd\logs\main.log

[2024-07-17 12:05:16.314] [info]  Starting Herd...
[2024-07-17 12:05:16.321] [error] (node:8660) [DEP0128] DeprecationWarning: Invalid 'main' field in 'C:\Program Files\Herd\resources\app.asar\node_modules\ini-api\package.json' of './dist/index.js'. Please either fix that or report it to the module author
(Use `Herd --trace-deprecation ...` to show where the warning was created)
[2024-07-17 12:05:16.442] [info]  Starting API Server
[2024-07-17 12:05:17.248] [info]  Internal API listening at http://localhost:9001
[2024-07-17 12:05:33.450] [info]  Copying Nginx config...
[2024-07-17 12:05:33.455] [info]  Copied Nginx config to C:\Users\XXX\.config\herd\config\nginx\herd.conf
[2024-07-17 12:05:33.466] [info]  Copying binaries to C:\Users\XXX\.config\herd\bin
[2024-07-17 12:05:33.959] [info]  No .bash_profile found at C:\Users\XXX\.bash_profile
[2024-07-17 12:05:34.432] [info]  Downloading PHP 8.3 (8.3.8) from https://download.herdphp.com/8.3/php83-win.zip to C:\Users\216880~1\AppData\Local\Temp\php-8.3.zip
[2024-07-17 12:05:34.952] [error] Error in download process: {
  message: 'Request failed with status code 503',
  name: 'AxiosError',
  stack: 'AxiosError: Request failed with status code 503\n' +
    '    at settle (C:\\Program Files\\Herd\\resources\\app.asar\\node_modules\\axios\\dist\\node\\axios.cjs:1967:12)\n' +
    '    at RedirectableRequest.handleResponse (C:\\Program Files\\Herd\\resources\\app.asar\\node_modules\\axios\\dist\\node\\axios.cjs:3010:9)\n' +
    '    at RedirectableRequest.emit (node:events:517:28)\n' +
    '    at RedirectableRequest._processResponse (C:\\Program Files\\Herd\\resources\\app.asar\\node_modules\\follow-redirects\\index.js:398:10)\n' +
    '    at ClientRequest.<anonymous> (C:\\Program Files\\Herd\\resources\\app.asar\\node_modules\\follow-redirects\\index.js:91:12)\n' +
    '    at Object.onceWrapper (node:events:632:26)\n' +
    '    at ClientRequest.emit (node:events:517:28)\n' +
    '    at HTTPParser.parserOnIncomingClient (node:_http_client:700:27)\n' +
    '    at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)\n' +
    '    at Socket.socketOnData (node:_http_client:541:22)',
  config: {
    transitional: {
      silentJSONParsing: true,
      forcedJSONParsing: true,
      clarifyTimeoutError: false
    },
    adapter: [ 'xhr', 'http' ],
    transformRequest: [
      '[function] function transformRequest(data, headers) {\n' +
        "    const contentType = headers.getContentType() || '';\n" +
        "    const hasJSONContentType = contentType.indexOf('application/json') > -1;\n" +
        '    const isObjectPayload = utils$1.isObject(data);\n' +
        '\n' +
        '    if (isObjectPayload && utils$1.isHTMLForm(data)) {\n' +
        '      data = new FormData(data);\n' +
        '    }\n' +
        '\n' +
        '    const isFormData = utils$1.isFormData(data);\n' +
        '\n' +
        '    if (isFormData) {\n' +
        '      if (!hasJSONContentType) {\n' +
        '        return data;\n' +
        '      }\n' +
        '      return hasJSONContentType ? JSON.stringify(formDataToJSON(data)) : data;\n' +
        '    }\n' +
        '\n' +
        '    if (utils$1.isArrayBuffer(data) ||\n' +
        '      utils$1.isBuffer(data) ||\n' +
        '      utils$1.isStream(data) ||\n' +
        '      utils$1.isFile(data) ||\n' +
        '      utils$1.isBlob(data)\n' +
        '    ) {\n' +
        '      return data;\n' +
        '    }\n' +
        '    if (utils$1.isArrayBufferView(data)) {\n' +
        '      return data.buffer;\n' +
        '    }\n' +
        '    if (utils$1.isURLSearchParams(data)) {\n' +
        "      headers.setContentType('application/x-www-form-urlencoded;charset=utf-8', false);\n" +
        '      return data.toString();\n' +
        '    }\n' +
        '\n' +
        '    let isFileList;\n' +
        '\n' +
        '    if (isObjectPayload) {\n' +
        "      if (contentType.indexOf('application/x-www-form-urlencoded') > -1) {\n" +
        '        return toURLEncodedForm(data, this.formSerializer).toString();\n' +
        '      }\n' +
        '\n' +
        "      if ((isFileList = utils$1.isFileList(data)) || contentType.indexOf('multipart/form-data') > -1) {\n" +
        '        const _FormData = this.env && this.env.FormData;\n' +
        '\n' +
        '        return toFormData(\n' +
        "          isFileList ? {'files[]': data} : data,\n" +
        '          _FormData && new _FormData(),\n' +
        '          this.formSerializer\n' +
        '        );\n' +
        '      }\n' +
        '    }\n' +
        '\n' +
        '    if (isObjectPayload || hasJSONContentType ) {\n' +
        "      headers.setContentType('application/json', false);\n" +
        '      return stringifySafely(data);\n' +
        '    }\n' +
        '\n' +
        '    return data;\n' +
        '  }'
    ],
    transformResponse: [
      '[function] function transformResponse(data) {\n' +
        '    const transitional = this.transitional || defaults.transitional;\n' +
        '    const forcedJSONParsing = transitional && transitional.forcedJSONParsing;\n' +
        "    const JSONRequested = this.responseType === 'json';\n" +
        '\n' +
        '    if (data && utils$1.isString(data) && ((forcedJSONParsing && !this.responseType) || JSONRequested)) {\n' +
        '      const silentJSONParsing = transitional && transitional.silentJSONParsing;\n' +
        '      const strictJSONParsing = !silentJSONParsing && JSONRequested;\n' +
        '\n' +
        '      try {\n' +
        '        return JSON.parse(data);\n' +
        '      } catch (e) {\n' +
        '        if (strictJSONParsing) {\n' +
        "          if (e.name === 'SyntaxError') {\n" +
        '            throw AxiosError.from(e, AxiosError.ERR_BAD_RESPONSE, this, null, this.response);\n' +
        '          }\n' +
        '          throw e;\n' +
        '        }\n' +
        '      }\n' +
        '    }\n' +
        '\n' +
        '    return data;\n' +
        '  }'
    ],
    timeout: 0,
    xsrfCookieName: 'XSRF-TOKEN',
    xsrfHeaderName: 'X-XSRF-TOKEN',
    maxContentLength: -1,
    maxBodyLength: -1,
    env: {
      FormData: '[function] function FormData(options) {\n' +
        '  if (!(this instanceof FormData)) {\n' +
        '    return new FormData(options);\n' +
        '  }\n' +
        '\n' +
        '  this._overheadLength = 0;\n' +
        '  this._valueLength = 0;\n' +
        '  this._valuesToMeasure = [];\n' +
        '\n' +
        '  CombinedStream.call(this);\n' +
        '\n' +
        '  options = options || {};\n' +
        '  for (var option in options) {\n' +
        '    this[option] = options[option];\n' +
        '  }\n' +
        '}',
      Blob: '[function] class Blob {\n' +
        '  /**\n' +
        '   * @typedef {string|ArrayBuffer|ArrayBufferView|Blob} SourcePart\n' +
        '   */\n' +
        '\n' +
        '  /**\n' +
        '   * @param {SourcePart[]} [sources]\n' +
        '   * @param {{\n' +
        '   *   endings? : string,\n' +
        '   *   type? : string,\n' +
        '   * }} [options]\n' +
        '   * @constructs {Blob}\n' +
        '   */\n' +
        '  constructor(sources = [], options) {\n' +
        '    if (sources === null ||\n' +
        "        typeof sources[SymbolIterator] !== 'function' ||\n" +
        "        typeof sources === 'string') {\n" +
        "      throw new ERR_INVALID_ARG_TYPE('sources', 'a sequence', sources);\n" +
        '    }\n' +
        "    validateDictionary(options, 'options');\n" +
        '    let {\n' +
        "      type = '',\n" +
        "      endings = 'transparent',\n" +
        '    } = options ?? kEmptyObject;\n' +
        '\n' +
        '    endings = `${endings}`;\n' +
        "    if (endings !== 'transparent' && endings !== 'native')\n" +
        "      throw new ERR_INVALID_ARG_VALUE('options.endings', endings);\n" +
        '\n' +
        '    let length = 0;\n' +
        '    const sources_ = ArrayFrom(sources, (source) => {\n' +
        '      const { 0: len, 1: src } = getSource(source, endings);\n' +
        '      length += len;\n' +
        '      return src;\n' +
        '    });\n' +
        '\n' +
        '    if (length > kMaxLength)\n' +
        '      throw new ERR_BUFFER_TOO_LARGE(kMaxLength);\n' +
        '\n' +
        '    this[kHandle] = _createBlob(sources_, length);\n' +
        '    this[kLength] = length;\n' +
        '\n' +
        '    type = `${type}`;\n' +
        '    this[kType] = RegExpPrototypeExec(disallowedTypeCharacters, type) !== null ?\n' +
        "      '' : StringPrototypeToLowerCase(type);\n" +
        '\n' +
        '    // eslint-disable-next-line no-constructor-return\n' +
        '    return makeTransferable(this);\n' +
        '  }\n' +
        '\n' +
        '  [kInspect](depth, options) {\n' +
        '    if (depth < 0)\n' +
        '      return this;\n' +
        '\n' +
        '    const opts = {\n' +
        '      ...options,\n' +
        '      depth: options.depth == null ? null : options.depth - 1,\n' +
        '    };\n' +
        '\n' +
        '    return `Blob ${inspect({\n' +
        '      size: this.size,\n' +
        '      type: this.type,\n' +
        '    }, opts)}`;\n' +
        '  }\n' +
        '\n' +
        '  [kClone]() {\n' +
        '    const handle = this[kHandle];\n' +
        '    const type = this[kType];\n' +
        '    const length = this[kLength];\n' +
        '    return {\n' +
        '      data: { handle, type, length },\n' +
        "      deserializeInfo: 'internal/blob:ClonedBlob',\n" +
        '    };\n' +
        '  }\n' +
        '\n' +
        '  [kDeserialize]({ handle, type, length }) {\n' +
        '    this[kHandle] = handle;\n' +
        '    this[kType] = type;\n' +
        '    this[kLength] = length;\n' +
        '  }\n' +
        '\n' +
        '  /**\n' +
        '   * @readonly\n' +
        '   * @type {string}\n' +
        '   */\n' +
        '  get type() {\n' +
        '    if (!isBlob(this))\n' +
        "      throw new ERR_INVALID_THIS('Blob');\n" +
        '    return this[kType];\n' +
        '  }\n' +
        '\n' +
        '  /**\n' +
        '   * @readonly\n' +
        '   * @type {number}\n' +
        '   */\n' +
        '  get size() {\n' +
        '    if (!isBlob(this))\n' +
        "      throw new ERR_INVALID_THIS('Blob');\n" +
        '    return this[kLength];\n' +
        '  }\n' +
        '\n' +
        '  /**\n' +
        '   * @param {number} [start]\n' +
        '   * @param {number} [end]\n' +
        '   * @param {string} [contentType]\n' +
        '   * @returns {Blob}\n' +
        '   */\n' +
        "  slice(start = 0, end = this[kLength], contentType = '') {\n" +
        '    if (!isBlob(this))\n' +
        "      throw new ERR_INVALID_THIS('Blob');\n" +
        '    if (start < 0) {\n' +
        '      start = MathMax(this[kLength] + start, 0);\n' +
        '    } else {\n' +
        '      start = MathMin(start, this[kLength]);\n' +
        '    }\n' +
        '    start |= 0;\n' +
        '\n' +
        '    if (end < 0) {\n' +
        '      end = MathMax(this[kLength] + end, 0);\n' +
        '    } else {\n' +
        '      end = MathMin(end, this[kLength]);\n' +
        '    }\n' +
        '    end |= 0;\n' +
        '\n' +
        '    contentType = `${contentType}`;\n' +
        '    if (RegExpPrototypeExec(disallowedTypeCharacters, contentType) !== null) {\n' +
        "      contentType = '';\n" +
        '    } else {\n' +
        '      contentType = StringPrototypeToLowerCase(contentType);\n' +
        '    }\n' +
        '\n' +
        '    const span = MathMax(end - start, 0);\n' +
        '\n' +
        '    return createBlob(\n' +
        '      this[kHandle].slice(start, start + span),\n' +
        '      span,\n' +
        '      contentType);\n' +
        '  }\n' +
        '\n' +
        '  /**\n' +
        '   * @returns {Promise<ArrayBuffer>}\n' +
        '   */\n' +
        '  arrayBuffer() {\n' +
        '    if (!isBlob(this))\n' +
        "      return PromiseReject(new ERR_INVALID_THIS('Blob'));\n" +
        '\n' +
        "    // If there's already a promise in flight for the content,\n" +
        "    // reuse it, but only while it's in flight. After the cached\n" +
        '    // promise resolves it will be cleared, allowing it to be\n' +
        '    // garbage collected as soon as possible.\n' +
        '    if (this[kArrayBufferPromise])\n' +
        '      return this[kArrayBufferPromise];\n' +
        '\n' +
        '    const job = new FixedSizeBlobCopyJob(this[kHandle]);\n' +
        '\n' +
        '    const ret = job.run();\n' +
        '\n' +
        '    // If the job returns a value immediately, the ArrayBuffer\n' +
        '    // was generated synchronously and should just be returned\n' +
        '    // directly.\n' +
        '    if (ret !== undefined)\n' +
        '      return PromiseResolve(ret);\n' +
        '\n' +
        '    const {\n' +
        '      promise,\n' +
        '      resolve,\n' +
        '      reject,\n' +
        '    } = createDeferredPromise();\n' +
        '\n' +
        '    job.ondone = (err, ab) => {\n' +
        '      if (err !== undefined)\n' +
        '        return reject(new AbortError(undefined, { cause: err }));\n' +
        '      resolve(ab);\n' +
        '    };\n' +
        '    this[kArrayBufferPromise] =\n' +
        '    SafePromisePrototypeFinally(\n' +
        '      promise,\n' +
        '      () => this[kArrayBufferPromise] = undefined);\n' +
        '\n' +
        '    return this[kArrayBufferPromise];\n' +
        '  }\n' +
        '\n' +
        '  /**\n' +
        '   * @returns {Promise<string>}\n' +
        '   */\n' +
        '  async text() {\n' +
        '    if (!isBlob(this))\n' +
        "      throw new ERR_INVALID_THIS('Blob');\n" +
        '\n' +
        '    dec ??= new TextDecoder();\n' +
        '\n' +
        '    return dec.decode(await this.arrayBuffer());\n' +
        '  }\n' +
        '\n' +
        '  /**\n' +
        '   * @returns {ReadableStream}\n' +
        '   */\n' +
        '  stream() {\n' +
        '    if (!isBlob(this))\n' +
        "      throw new ERR_INVALID_THIS('Blob');\n" +
        '\n' +
        '    const self = this;\n' +
        '    return new lazyReadableStream({\n' +
        '      async start() {\n' +
        '        this[kState] = await self.arrayBuffer();\n' +
        '        this[kIndex] = 0;\n' +
        '      },\n' +
        '\n' +
        '      pull(controller) {\n' +
        '        if (this[kState].byteLength - this[kIndex] <= kMaxChunkSize) {\n' +
        '          controller.enqueue(new Uint8Array(this[kState], this[kIndex]));\n' +
        '          controller.close();\n' +
        '          this[kState] = undefined;\n' +
        '        } else {\n' +
        '          controller.enqueue(new Uint8Array(this[kState], this[kIndex], kMaxChunkSize));\n' +
        '          this[kIndex] += kMaxChunkSize;\n' +
        '        }\n' +
        '      },\n' +
        '    });\n' +
        '  }\n' +
        '}'
    },
    validateStatus: '[function] function validateStatus(status) {\n' +
      '    return status >= 200 && status < 300;\n' +
      '  }',
    headers: {
      Accept: 'application/json, text/plain, */*',
      'User-Agent': 'axios/1.6.2',
      'Accept-Encoding': 'gzip, compress, deflate, br'
    },
    url: 'https://download.herdphp.com/8.3/php83-win.zip',
    method: 'get',
    responseType: 'stream'
  },
  code: 'ERR_BAD_RESPONSE',
  status: 503
}
[2024-07-17 12:05:34.955] [error] Error occurred in handler for 'herd.install-php-version': Error: An object could not be cloned.
    at WebContents.<anonymous> (node:electron/js2c/browser_init:2:78003)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

[sschlein]

I am currently looking into proxy support for Herd but it's a new topic to me and wonder what your typical proxy setup is. When you add information about your proxy, what do you add in other apps?

[jorgelloret]

Hi @sschlein,

A corporate proxy usually consists of a .pac script that indicates which urls go direct and which urls are proxied.

Normally in other applications it is configured as an IP and port pair, for example, IP_OF_PROXY:PORT.

Composer gets it from system variables called HTTP_PROXY and HTTPS_PROXY with the following content: http://IP_OF_PROXY:PORT.

npm from a .npmrc file located in the user folder with the following content:

proxy=http://IP_OF_PROXY:PORT
https-proxy=http://IP_OF_PROXY:PORT
http-proxy=http://IP_OF_PROXY:PORT

git of a .gitconfig file located in the user folder with the following content:

[http]
	proxy = http://IP_OF_PROXY:PORT

Here is a manual for setting up axios for proxy use: https://brightdata.com/blog/how-tos/axios-proxy

[ryanlovett-au]

Also, please be aware, it is not just proxies, many many organisations now use ZScalers, which perform man-in-the-middle decrypting and re-encrypting of HTTPS traffic, and require a custom Root CA certificate to be used. NPM/Node uses the NODE_EXTRA_CA_CERTS environment variable in Windows, however this is not being recognised by the Herd instance of Axios.

I have successfully got this working in non javascript places in Herd by updating the php.ini openssl.cafile= key in the default version of PHP used by Herd, and other applications.

However, for proxies, many other applications, such as Composer on Windows, use the HTTP_PROXY and HTTPS_PROXY environment variables, so it may be an option to look for those and apply them to standardise the setup for people?

[ryanlovett-au]

Actually, it looks like Herd is respecting HTTP_PROXY.

When I have an HTTP_PROXY configured Herd is unable to connect the HerdHelper on http://127.0.0.1:5000, but when I remove the HTTP_PROXY environment variable and restart Herd it IS able to connect.

[ryanlovett-au]

Further, it does appear that Herd is respecting the NODE_TLS_REJECT_UNAUTHORIZED but not NODE_EXTRA_CA_CERTS.

electron/electron#10257

[dbaker02]

I am also interested in a fix for this as this is what is keeping me from switching to Herd from Laragon.

My company uses Netskope, which is man-in-the-middle as mentioned above. For me, I have to point to our NS .pem certificate. I had this issue with composer, previously, and I had to add the PEM certificate in the cafile value in php.ini to get it to work.

One issue with this that I would run into is that if I added the PEM certificate in the php.ini file, it did work for intercepted traffic, but for traffic that was bypassed (whitelisted), injecting the certificate caused it to fail for those resources.

For Herd, I can't use this workaround anyway since the issue that is caused is downloading PHP to begin with.

Interesting with Herd or Mac and Windows... I have the NS issue with Windows version and not Mac. I have not done logging at all, but it appears that they might not download PHP from the same location? Same network blocks traffic when downloading PHP on Windows version, but not on Mac version.

[sschlein]

I've finally implemented proxy support for Herd by using internal Electron APIs for proxy detection and it's likely coming with the next release.

[jorgelloret]

I've finally implemented proxy support for Herd by using internal Electron APIs for proxy detection and it's likely coming with the next release.

Thanks, if you need a tester to check if it works, I offer myself.

[mpociot]

This is now fixed in 1.20.2