[Bug] CI Blocker Build Pulsar docker image failure

[JooHyukKim]

Search before asking

• I searched in the issues and found nothing similar.

Version

master branch

Minimal reproduce step

Currently, CI Build Pulsar docker image action is failing on master branch. with following PRs as examples.

• [fix][sec] Suppress already covered CVE-2023-2976 in clickhouse-jdbc-0.4.6-all.jar and canal.client-1.1.5.jar #20792
  • https://github.com/apache/pulsar/actions/runs/5598232691/jobs/10242732078?pr=20792
• [improve][doc] Add language support flag for --cleanup-subscription #20836
  • https://github.com/apache/pulsar/actions/runs/5594652811/jobs/10232369328?pr=20836
• [improve][broker] Follow up #4196 use PulsarByteBufAllocator handle OOM #20837
  • https://github.com/apache/pulsar/actions/runs/5597413965/jobs/10235999625?pr=20837

What did you expect to see?

Clean CI Pass.

What did you see instead?

Error log (Check above actions link for more)

Error:  Failed to execute goal io.fabric8:docker-maven-plugin:0.42.1:build (default) on project pulsar-docker-image: Unable to build image [apachepulsar/pulsar] : "The command '/bin/sh -c pip3 install pyyaml==5.4.1' returned a non-zero code: 1" -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Error:  
Error:  After correcting the problems, you can resume the build with the command
Error:    mvn <args> -rf :pulsar-docker-image
Error: Process completed with exit code 1.

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[michaeljmarshall]

Thanks for opening this issue @JooHyukKim. In looking at the logs for one of your links, I see:

[INFO] DOCKER> Processing triggers for libc-bin (2.35-0ubuntu3.1) ...
[INFO] DOCKER> Reading package lists...
[INFO] DOCKER> 
[INFO] DOCKER> Building dependency tree...
[INFO] DOCKER> Reading state information...
[INFO] DOCKER> 
[INFO] DOCKER> Removing intermediate container 499e92166c4d
[INFO] DOCKER> ---> d9a55ff54a1a
[INFO] DOCKER> Step 21/30 : RUN pip3 install pyyaml==5.4.1
[INFO] DOCKER> 
[INFO] DOCKER> ---> Running in 3a7a0f70a2fb
[INFO] DOCKER> Collecting pyyaml==5.4.1
[INFO] DOCKER> Downloading PyYAML-5.4.1.tar.gz (175 kB)
[INFO] DOCKER> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 175.1/175.1 KB 3.2 MB/s eta 0:00:00
[INFO] DOCKER> 
[INFO] DOCKER> Installing build dependencies: started
[INFO] DOCKER> Installing build dependencies: finished with status 'done'
[INFO] DOCKER> Getting requirements to build wheel: started
[INFO] DOCKER> Getting requirements to build wheel: finished with status 'error'
[INFO] DOCKER> [91m  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [43 lines of output]
      running egg_info
      Traceback (most recent call last):
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 162, in get_requires_for_build_wheel
          return self._get_build_requires(
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 143, in _get_build_requires
          self.run_setup()
        File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 271, in <module>
          setup(
        File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 153, in setup
          return distutils.core.setup(**attrs)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 148, in setup
          return run_commands(dist)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/core.py", line 163, in run_commands
          dist.run_commands()
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line 299, in run
          self.find_sources()
        File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line 306, in find_sources
          mm.run()
        File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line 541, in run
          self.add_defaults()
        File "/usr/lib/python3/dist-packages/setuptools/command/egg_info.py", line 578, in add_defaults
          sdist.add_defaults(self)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/sdist.py", line 228, in add_defaults
          self._add_defaults_ext()
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/command/sdist.py", line 312, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
        File "setup.py", line 201, in get_source_files
          self.cython_sources(ext.sources, ext)
        File "/usr/lib/python3/dist-packages/setuptools/_distutils/cmd.py", line 103, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.

[INFO] DOCKER> [91merror: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[INFO] DOCKER> Removing intermediate container 3a7a0f70a2fb
Error:  DOCKER> Unable to build image [apachepulsar/pulsar] : "The command '/bin/sh -c pip3 install pyyaml==5.4.1' returned a non-zero code: 1"  ["The command '/bin/sh -c pip3 install pyyaml==5.4.1' returned a non-zero code: 1" ]

I'll look into this a bit more tonight--I haven't found the root cause yet.

[michaeljmarshall]

This is a known issue: yaml/pyyaml#724. Looking into the right fix now.

[JooHyukKim]

Thanks for opening this issue @JooHyukKim. In looking at the logs for one of your links, I see:

np! 👍🏻 All links will lead to the same log you shared. I got stuck there also.

I'll look into this a bit more tonight--I haven't found the root cause yet.

Thankssss.

[JooHyukKim]

This is a known issue: yaml/pyyaml#724. Looking into the right fix now.

Nice. Thank you for your time~!

[michaeljmarshall]

After reading through more of the pyyaml issues/prs, we should be able to unblock builds by going to 6.0.1. Here is the PR: #20842.