CI: add 2.14 tests, create ignore-2.15 to prevent CI failure #139

csmart · 2022-10-05T08:40:40Z

Fixes #137

Im0 · 2022-10-19T23:26:55Z

I tried to check why these failed but it appears that the old logs have been deleted. This was perhaps erroring due to #141 but unsure.

Fixes ansible-collections#137

csmart · 2022-10-19T23:53:54Z

I'll re-test and see where Ubuntu fails with fresh logs...

Im0 · 2022-10-20T01:32:59Z

@csmart I read on another forum that perhaps ssl1.1 isn't available via apt on Ubuntu 22.04. It looks like this PR introduces 22.04? Looks like it maybe v3 now... https://packages.ubuntu.com/jammy/libssl-dev

Edit: 20.04 looks like version 1.1.1 .. https://packages.ubuntu.com/focal/libssl-dev

csmart · 2022-10-21T22:25:25Z

Looks like it's failing when trying to talk to an IPv6 localhost? Maybe something changed with Docker 2.14 images..

04:32 TASK [rabbitmq_publish : RabbitMQ basic publish test] **************************
04:32 task path: /root/ansible_collections/community/rabbitmq/tests/output/.tmp/integration/rabbitmq_publish-_ir7cm5g-ÅÑŚÌβŁÈ/tests/integration/targets/rabbitmq_publish/tasks/ubuntu.yml:7
04:33 <localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
04:33 <localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
04:33 Including module_utils file ansible/module_utils/common/text/formatters.py
04:33 Including module_utils file ansible/module_utils/common/validation.py
04:33 Including module_utils file ansible/module_utils/common/warnings.py
04:33 Including module_utils file ansible/module_utils/compat/selectors.py
04:33 Including module_utils file ansible/module_utils/compat/__init__.py
04:33 Including module_utils file ansible/module_utils/compat/_selectors2.py
04:33 Including module_utils file ansible/module_utils/compat/selinux.py
04:33 Including module_utils file ansible/module_utils/distro/__init__.py
04:33 Including module_utils file ansible/module_utils/distro/_distro.py
04:33 Including module_utils file ansible/module_utils/errors.py
04:33 Including module_utils file ansible/module_utils/parsing/convert_bool.py
04:33 Including module_utils file ansible/module_utils/parsing/__init__.py
04:33 Including module_utils file ansible/module_utils/pycompat24.py
04:33 Including module_utils file ansible/module_utils/six/__init__.py
04:33 Including module_utils file ansible_collections/community/rabbitmq/plugins/module_utils/rabbitmq.py
04:33 Including module_utils file ansible_collections/__init__.py
04:33 Including module_utils file ansible_collections/community/__init__.py
04:33 Including module_utils file ansible_collections/community/rabbitmq/__init__.py
04:33 Including module_utils file ansible_collections/community/rabbitmq/plugins/__init__.py
04:33 Including module_utils file ansible_collections/community/rabbitmq/plugins/module_utils/__init__.py
04:33 Using module file /root/ansible_collections/community/rabbitmq/plugins/modules/rabbitmq_publish.py
04:33 <localhost> PUT /root/.ansible/tmp/ansible-local-904rvsc4c7b/tmpy5_q7wrd TO /root/.ansible/tmp/ansible-tmp-1666223930.149605-1025-51122755624198/AnsiballZ_rabbitmq_publish.py
04:33 <localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1666223930.149605-1025-51122755624198/ /root/.ansible/tmp/ansible-tmp-1666223930.149605-1025-51122755624198/AnsiballZ_rabbitmq_publish.py && sleep 0'
04:33 <localhost> EXEC /bin/sh -c '/usr/bin/python3.11 /root/.ansible/tmp/ansible-tmp-1666223930.149605-1025-51122755624198/AnsiballZ_rabbitmq_publish.py && sleep 0'
04:33 <localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1666223930.149605-1025-51122755624198/ > /dev/null 2>&1 && sleep 0'
04:33 The full traceback is:
04:33   File "/tmp/ansible_rabbitmq_publish_payload_bw8tknjo/ansible_rabbitmq_publish_payload.zip/ansible_collections/community/rabbitmq/plugins/module_utils/rabbitmq.py", line 141, in connect_to_rabbitmq
04:33     self.connection = pika.BlockingConnection(parameters)
04:33                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
04:33   File "/usr/local/lib/python3.11/dist-packages/pika/adapters/blocking_connection.py", line 377, in __init__
04:33     self._process_io_for_connection_setup()
04:33   File "/usr/local/lib/python3.11/dist-packages/pika/adapters/blocking_connection.py", line 416, in _process_io_for_connection_setup
04:33     self._flush_output(self._opened_result.is_ready,
04:33   File "/usr/local/lib/python3.11/dist-packages/pika/adapters/blocking_connection.py", line 471, in _flush_output
04:33     raise exceptions.ConnectionClosed(maybe_exception)
04:33 fatal: [testhost -> localhost]: FAILED! => {
04:33     "changed": false,
04:33     "invocation": {
04:33         "module_args": {
04:33             "auto_delete": false,
04:33             "body": "Hello world from ansible module rabbitmq_publish",
04:33             "cafile": null,
04:33             "certfile": null,
04:33             "content_type": "text/plain",
04:33             "durable": false,
04:33             "exchange": "",
04:33             "exclusive": false,
04:33             "headers": {},
04:33             "host": null,
04:33             "keyfile": null,
04:33             "password": null,
04:33             "port": null,
04:33             "proto": null,
04:33             "queue": "publish_test",
04:33             "routing_key": null,
04:33             "src": null,
04:33             "url": "amqp://guest:***@localhost:5672/%2F",
04:33             "username": null,
04:33             "vhost": null
04:33         }
04:33     },
04:33     "msg": "Connection issue: Connection to ::1:5672 failed: [Errno 99] Cannot assign requested address"
04:33 }

…into ci_2_15

Im0 · 2022-10-24T09:48:19Z

@csmart does this help? csmart#1

Ci 2 15

csmart · 2022-10-24T10:07:31Z

@csmart does this help? csmart#1
@Im0 thanks, let's see! 🤞

Im0 · 2022-10-24T10:38:12Z

Pipeline taking ages today! Must be working it too hard. :)

Im0 · 2022-10-24T22:39:59Z

@csmart I'm so sorry, I was adding a PR via the github web interface on your forked repo and ci_2_15 branch... github gave me a 404 error. Then out of curiosity I tried the other option fully expecting it to fail, but, it allowed me to update your forked repo! Really sorry about that... it is probably because I'm on the maintainers list?

Edit: CI is now failing due to not being able to connect to 192.168.250.1 in the lookup_rabbitmq plugin. I'm afraid to touch git right now but the tests/integration/targets/lookup_rabbitmq/tasks/ubuntu.yml file needs updating now to change 192.168.250.1 to localhost or 127.0.0.1.

csmart · 2022-10-25T01:48:16Z

@Im0 oh, no need to apologise! I don't think you did anything wrong ☺️ Definitely appreciate all the help we can get! I have been pretty smashed at work lately, I hope to find some free time to take a close look, maybe later tonight. Feel free to play with git and change whatever you think, you can just do a brand new PR so that you don't have to merge into mine, and if yours gets going then great, we can merge it.

My only concern I had before about the change that I introduced checking if connection failed, I'm not sure if that's a valid response to the request, or whether that might mask a problem where we can't talk to rabbitmq for another reason... I tried to run the tests locally to check, but I ran into other issues and they wouldn't run... And then I ran out of time...

So please feel free to try whatever you think and we'll go from there.

Thanks for the help!

Ci 2 15 x2

Im0 · 2022-10-25T20:46:35Z

CI (Docker 2.14 Ubuntu 20.04)
  TASK [rabbitmq_publish : RabbitMQ basic publish test] **************************
  "msg": "Connection issue: Connection to ::1:5672 failed: [Errno 99] Cannot assign requested address"

CI (Docker 2.14 Ubuntu 22.04)
  TASK [rabbitmq_publish : RabbitMQ basic publish test]
  "msg": "Connection issue: module 'collections' has no attribute 'Callable'"

CI (Docker devel Ubuntu 20.04) 
  TASK [rabbitmq_publish : RabbitMQ basic publish test] **************************
  "msg": "Connection issue: Connection to ::1:5672 failed: [Errno 99] Cannot assign requested address"

CI (Docker devel Ubuntu 22.04) 
  TASK [rabbitmq_publish : RabbitMQ basic publish test] **************************
  "msg": "Connection issue: module 'collections' has no attribute 'Callable'"

I'll take a look at these today.

Im0 · 2022-10-26T22:14:07Z

Test 1: "Connection issue: module 'collections' has no attribute 'Callable'"
cmd: python3.9 /usr/local/bin/ansible-test integration rabbitmq_publish --docker ubuntu2204
env: ansible 2.15 dev
result: Connection issue: module 'collections' has no attribute 'Callable'
Try # 1: install pip pika 1.3.0
Result # 1: Publish to random queue... the error changed to a new error. Use pika 1.3.0 going forward.

fatal: [testhost -> localhost]: FAILED! => {"changed": false, "msg": "Queue declare issue: BlockingChannel.queue_declare() missing 1 required positional argument: 'queue'"}

Thought # 1: This is fixed already in the exchange patch (PR #140)

Try # 2: Copy rabbitmq_publish and module_utils/rabbitmq.py from fix_138_publish_to_exchange for testing
Result # 2: Worked! 3m6.702s. Suggest merge #140 and rebase csmart's PR.

Test 2: "Connection issue: Connection to ::1:5672 failed: [Errno 99] Cannot assign requested address"
cmd: time python3.9 /usr/local/bin/ansible-test integration rabbitmq_publish --docker ubuntu2004
env: ansible 2.15 dev
Result # 1:

TASK [rabbitmq_publish : RabbitMQ basic publish test] **************************
 fatal: [testhost -> localhost]: FAILED! => {"changed": false, "msg": "Connection issue: "}
 Try # 2: time python3.9 /usr/local/bin/ansible-test integration rabbitmq_publish --docker ubuntu2004 -vvvvv
 Result # 2: "Connection issue: "
   The full traceback is:  File   "/tmp/ansible_rabbitmq_publish_payload_aqrtutqd/ansible_rabbitmq_publish_payload.zip/ansible_collections/community/rabbitmq/plugins/module_utils/rabbitmq.py", line 143, in connect_to_rabbitmq
    self.connection = pika.BlockingConnection(parameters)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/pika/adapters/blocking_connection.py", line 360, in __init__
    self._impl = self._create_connection(parameters, _impl_class)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Try # 3: python3.9 /usr/local/bin/ansible-test shell --docker ubuntu2004
env: inside the target container. ansible 2.15/python 2.9
Result # 3: Worked.
Comment: Uncertain if this is going to work in AZP or not.

Test 3: "Connection issue: Connection to ::1:5672 failed: [Errno 99] Cannot assign requested address"
cmd: time python3.9 /usr/local/bin/ansible-test integration rabbitmq_publish --docker ubuntu2004
env: ansible 2.13.5
Result # 1: Success. 3m5.064s. Fixed with #140 and or pika 1.3.0

Suggest merging PR #140 and rebasing csmart:ci_2_15

…into ci_2_15

Im0 · 2022-10-27T03:46:51Z

@csmart hopefully csmart#4 will help with some of these CI failures. Or rebase with origin/main. Fingers crossed but I think there will still be one issue.

Updating to pull in recent merge

Im0 · 2022-10-30T10:00:31Z

Well, we got a CI success! But only one check. I can't see how our change would have dropped all other checks.

.azure-pipelines/azure-pipelines.yml

Currently 20.04 is causing devel to fail. This maybe due to Ubuntu 20.04 running python 3.8, however, ansible-test requires 3.9+. This means ansible test spins up a controller and target container which is probably why rabbitmq_publish is not able to connect to rabbitmq on localhost.

Im0 · 2022-10-30T23:58:39Z

@csmart Hi mate, I've disabled 20.04 from 2.14 and devel as @felixfontein (thanks heaps) suggested that the issue maybe that Ubuntu 20.04 uses python 3.8 but ansible-test requires 3.9+. Ansible-test therefore uses a controller and target container which is probably why we can't hit it with localhost in rabbitmq_publish.

I think we have good coverage with what we have and am happy with this PR as is. I'm happy to merge if you are.

However maybe we create an issue to try and fix 20.04 on 2.14/devel? I got some container information in another CI run which maybe useful:

"ansible_locally_reachable_ips": { 
"ipv4": [ 
"127.0.0.0/8", 
"127.0.0.1", 
"172.19.0.3" 
],

I'm not sure if 172.19.0.3 will be the controller or target... I'm assuming controller as otherwise we'd be able to connect to 127.0.0.1.

felixfontein · 2022-10-31T05:39:02Z

The IP 172.19.0.3 is only valid in this specific CI run, it might be different on every other one. You probably need to dynamically query the IP of the target container (some ansible facts should help for that) and use that.

Im0 · 2022-10-31T05:43:55Z

@felixfontein I got some ansible facts out here: https://dev.azure.com/ansible/community.rabbitmq/_build/results?buildId=58759&view=logs&j=1d520954-9606-5520-b594-0b935de5aeca&t=cbeffed3-49d8-5f25-fa26-6591d04d4a87

I don't understand if this was generated by the controller or target? I presume target? In which case we could use ansible_facts['ansible_eth0']['ipv4.address'] ?

felixfontein · 2022-10-31T05:47:57Z

@Im0 that should be for the target. The debug in line 3691 shows that ansible_default_ipv4.address is probably a good choice. I've just searched around and saw that the community.docker tests use advertise_addr: "{{ansible_default_ipv4.address | default('127.0.0.1')}}" for the Docker Swarm tests, sometimes also just ansible_default_ipv4.address, so that seems to work.

Im0 · 2022-10-31T05:52:17Z

Brilliant, thanks! I'll try that.

csmart · 2022-10-31T06:10:23Z

Wow, this MR got complicated 😄 Thanks @Im0

Im0 · 2022-10-31T10:15:16Z

Thanks for starting this off @csmart ! I've created #145 to document the 20.04 issue.

CI: add 2.14 tests, create ignore-2.15 to prevent CI failure

737779c

Fixes ansible-collections#137

csmart force-pushed the ci_2_15 branch from d8032b3 to 737779c Compare October 19, 2022 23:53

John Imison added 2 commits October 24, 2022 20:39

Merge branch 'ci_2_15' of https://github.com/csmart/community.rabbitmq …

9bc1af1

…into ci_2_15

Changelog fragment and listening on IPv6 within test container.

be4eef4

Merge pull request #1 from Im0/ci_2_15

81ee7f1

Ci 2 15

Update ubuntu.yml

7602f3d

John Imison added 2 commits October 25, 2022 12:10

Changing IP address on lookup plugin tests.

9019027

Modifying test assertion string.

93a220f

Merge pull request #3 from Im0/ci_2_15_x2

d4bdcae

Ci 2 15 x2

Im0 mentioned this pull request Oct 26, 2022

Fixing publishing to exchanges #140

Merged

Merge branch 'ci_2_15' of https://github.com/csmart/community.rabbitmq …

6c3169b

…into ci_2_15

Im0 mentioned this pull request Oct 28, 2022

Release community.rabbitmq collection v1.2.3 #143

Closed

Merge pull request #4 from Im0/ci_2_15

6fa4666

Updating to pull in recent merge

Im0 closed this Oct 30, 2022

Im0 reopened this Oct 30, 2022

felixfontein reviewed Oct 30, 2022

View reviewed changes

.azure-pipelines/azure-pipelines.yml Outdated Show resolved Hide resolved

Removed duplicate ubuntu2204 entry

a457842

Im0 mentioned this pull request Oct 30, 2022

Fixing bug whereby a queue name with a slash in it resulted in a REST API error #144

Merged

Im0 mentioned this pull request Oct 31, 2022

Disabled 20.04 from 2.14/devel AZP CI testing #145

Open

Im0 merged commit ebf9d99 into ansible-collections:main Oct 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: add 2.14 tests, create ignore-2.15 to prevent CI failure #139

CI: add 2.14 tests, create ignore-2.15 to prevent CI failure #139

csmart commented Oct 5, 2022

Im0 commented Oct 19, 2022

csmart commented Oct 19, 2022

Im0 commented Oct 20, 2022 •

edited

Loading

csmart commented Oct 21, 2022 •

edited

Loading

Im0 commented Oct 24, 2022

csmart commented Oct 24, 2022

Im0 commented Oct 24, 2022

Im0 commented Oct 24, 2022 •

edited

Loading

csmart commented Oct 25, 2022

Im0 commented Oct 25, 2022

Im0 commented Oct 26, 2022 •

edited

Loading

Im0 commented Oct 27, 2022

Im0 commented Oct 30, 2022

Im0 commented Oct 30, 2022

felixfontein commented Oct 31, 2022

Im0 commented Oct 31, 2022

felixfontein commented Oct 31, 2022

Im0 commented Oct 31, 2022

csmart commented Oct 31, 2022 •

edited

Loading

Im0 commented Oct 31, 2022

CI: add 2.14 tests, create ignore-2.15 to prevent CI failure #139

CI: add 2.14 tests, create ignore-2.15 to prevent CI failure #139

Conversation

csmart commented Oct 5, 2022

Im0 commented Oct 19, 2022

csmart commented Oct 19, 2022

Im0 commented Oct 20, 2022 • edited Loading

csmart commented Oct 21, 2022 • edited Loading

Im0 commented Oct 24, 2022

csmart commented Oct 24, 2022

Im0 commented Oct 24, 2022

Im0 commented Oct 24, 2022 • edited Loading

csmart commented Oct 25, 2022

Im0 commented Oct 25, 2022

Im0 commented Oct 26, 2022 • edited Loading

Im0 commented Oct 27, 2022

Im0 commented Oct 30, 2022

Im0 commented Oct 30, 2022

felixfontein commented Oct 31, 2022

Im0 commented Oct 31, 2022

felixfontein commented Oct 31, 2022

Im0 commented Oct 31, 2022

csmart commented Oct 31, 2022 • edited Loading

Im0 commented Oct 31, 2022

Im0 commented Oct 20, 2022 •

edited

Loading

csmart commented Oct 21, 2022 •

edited

Loading

Im0 commented Oct 24, 2022 •

edited

Loading

Im0 commented Oct 26, 2022 •

edited

Loading

csmart commented Oct 31, 2022 •

edited

Loading