Support self-signed TLS certiticate on HTTP component #95

mortymacs · 2022-10-07T13:20:51Z

No description provided.

lotyp · 2022-11-10T21:09:21Z

try mkcert and it will be accepted as valid in your system
https://github.com/FiloSottile/mkcert

mortymacs · 2022-11-11T20:26:56Z

Hi @lotyp
Thanks for the suggestion, but the idea is to use the already prepared certificates.
For example, some APIs need certificate files prepared by the company and it's not public certificates. This issue is about those types of certificates.

sbaeurle · 2022-11-14T07:37:04Z

@mortymacs would an approach similar to https://forfuncsake.github.io/post/2017/08/trust-extra-ca-cert-in-go-app/ work?
maybe using SystemCertPool instead of NewCertPool?

mortymacs · 2022-11-14T08:34:30Z

@sbaeurle I was thinking the same to use SystemCertPool by supporting additional CA file, but maybe it should support these 3 type of inputs like helm install command:

--ca-file
--cert-file
--key-file

What's your idea?

sbaeurle · 2022-11-14T21:14:32Z

Probably a good approach, what kind of formats would you expect for each argument? pem, crt and cem?

mortymacs · 2022-11-15T07:59:15Z

We're looking for something like: this https://gist.github.com/michaljemala/d6f4e01c4834bf47a9c4
What's your opinion?

sbaeurle · 2022-11-15T14:43:55Z

Looks like a good approach!
The only open question on my side would be how to handle the system root store when we provide a custom certificate.
Do we just append to the "internal" root store or do we override it and only accept our root cert?

mortymacs · 2022-11-15T15:15:09Z

It should be append in the system root store by using SystemCertPool.

mortymacs · 2022-11-15T15:16:27Z

So, would you like to send a PR or I send it?

sbaeurle · 2022-11-16T16:39:10Z

I can take a look at it. I'm probably quite booked this week, but next week looks promising and I will try to adjust the certificate handling then :)

mortymacs · 2022-11-17T20:10:20Z

@sbaeurle awesome!

vbnetadmin · 2022-11-29T08:57:05Z

Today tried to use wait4x on my local domains, like https://router.docker, or https://auth.wod.docker
And I don't understand, is this problem, because I've installed custom certs with mkcert on my machine, and wait4x can't see them?

But, in browser, I can open these domains:

I'm using traefik locally, with mkcert

I have macos Ventura 13.0.1

Interesting, that if I try, shipped curl with macos, it works and can access domains:

I'm wondering, is this related to custom SSL certs ?

Currently, can't use wait4x to ping services using http

sbaeurle · 2022-11-29T09:51:54Z

@vbnetadmin please add -l debug to the wait4x command, so we can verify if the incorrect ssl validation is indeed the problem

PS. logs in text format are better than screenshots, so we can copy and compare it more easily

vbnetadmin · 2022-11-29T10:45:21Z

Output:

Wait4X with debug

➜ wait4x http https://router.docker -l debug
2022-11-29T12:43:09+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:09+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:10+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:10+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:11+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:11+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:12+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:12+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:13+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:13+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:14+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:14+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:15+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:15+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:16+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:16+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:17+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:17+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host
2022-11-29T12:43:18+02:00 INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:43:18+02:00 DBG Get "https://router.docker": dial tcp: lookup router.docker on 100.64.0.2:53: no such host

Ping:

ping router.docker
PING router.docker (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.051 ms
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.104 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.260 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.311 ms

Brew curl:

➜ curl -L router.docker
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

macOS default curl:

➜ /usr/bin/curl -L https://router.docker
<!DOCTYPE html><html><head><title>Traefik</title><meta charset=utf-8><meta name=description content="Traefik UI"><meta name=format-detection content="telephone=no"><meta name=msapplication-tap-highlight content=no><meta name=viewport content="user-scalable=no,initial-scale=1,maximum-scale=1,minimum-scale=1,width=device-width"><link rel=icon type=image/png href=statics/app-logo-128x128.png><link rel=icon type=image/png sizes=16x16 href=statics/icons/favicon-16x16.png><link rel=icon type=image/png sizes=32x32 href=statics/icons/favicon-32x32.png><link rel=icon type=image/png sizes=96x96 href=statics/icons/favicon-96x96.png><link rel=icon type=image/ico href=statics/icons/favicon.ico><link rel=apple-touch-icon href=statics/icons/apple-icon-152x152.png><link rel=apple-touch-icon sizes=152x152 href=statics/icons/apple-icon-152x152.png><link rel=apple-touch-icon sizes=167x167 href=statics/icons/apple-icon-167x167.png><link rel=apple-touch-icon sizes=180x180 href=statics/icons/apple-icon-180x180.png><link href=css/041f4ca8.f6e3d404.css rel=prefetch><link href=css/05759188.7a9c0db7.css rel=prefetch><link href=css/09354cb8.0e433876.css rel=prefetch><link href=css/09995f48.60caa8f0.css rel=prefetch><link href=css/0c428e32.ed2ba60f.css rel=prefetch><link href=css/18c6e965.5d8876c8.css rel=prefetch><link href=css/3e3ce03c.9b22e80f.css rel=prefetch><link href=css/46fd955e.9b22e80f.css rel=prefetch><link href=css/491024e9.9b22e80f.css rel=prefetch><link href=css/4f0a01e4.f8382310.css rel=prefetch><link href=css/52875482.9b22e80f.css rel=prefetch><link href=css/54b4dbdf.6e23d447.css rel=prefetch><link href=css/6d73c73c.9b22e80f.css rel=prefetch><link href=css/77d911b4.9b22e80f.css rel=prefetch><link href=css/797e4f23.9b22e80f.css rel=prefetch><link href=css/b34404c8.9b22e80f.css rel=prefetch><link href=js/041f4ca8.4f70d06d.js rel=prefetch><link href=js/05759188.f1376142.js rel=prefetch><link href=js/09354cb8.2e39e534.js rel=prefetch><link href=js/09995f48.a140ed0a.js rel=prefetch><link href=js/0c428e32.92dcacf8.js rel=prefetch><link href=js/18c6e965.5dc22ece.js rel=prefetch><link href=js/2d21e8fd.c00ac0e6.js rel=prefetch><link href=js/3e3ce03c.293f4729.js rel=prefetch><link href=js/46fd955e.343a8488.js rel=prefetch><link href=js/491024e9.5ba4955c.js rel=prefetch><link href=js/4f0a01e4.e6772caf.js rel=prefetch><link href=js/52875482.65e645a6.js rel=prefetch><link href=js/54b4dbdf.915c5773.js rel=prefetch><link href=js/6d73c73c.d8a27bbb.js rel=prefetch><link href=js/77d911b4.8d5002b7.js rel=prefetch><link href=js/797e4f23.771f63ae.js rel=prefetch><link href=js/b34404c8.7a4b9be8.js rel=prefetch><link href=css/app.3d2d5994.css rel=preload as=style><link href=js/app.45ca101c.js rel=preload as=script><link href=js/runtime.db7dd4a3.js rel=preload as=script><link href=js/vendor.f60bcb86.js rel=preload as=script><link href=css/app.3d2d5994.css rel=stylesheet></head><body><div id=q-app></div><script type=text/javascript src=js/app.45ca101c.js></script><script type=text/javascript src=js/runtime.db7dd4a3.js></script><script type=text/javascript src=js/vendor.f60bcb86.js></script></body></html>%

mortymacs · 2022-11-29T10:55:59Z

Hi Patrick,

This is the DNS problem. Wait4x tries to lookup the domain via 100.64.0.2 DNS server which is depends on your system configuration.
Would you please try with dig router.docker @100.64.0.2 and let us know the result?

vbnetadmin · 2022-11-29T11:02:28Z

@mortymacs

➜ dig router.docker @100.64.0.2

; <<>> DiG 9.10.6 <<>> router.docker @100.64.0.2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11785
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;router.docker. IN A

;; Query time: 35 msec
;; SERVER: 100.64.0.2#53(100.64.0.2)
;; WHEN: Tue Nov 29 13:02:07 EET 2022
;; MSG SIZE rcvd: 31

vbnetadmin · 2022-11-29T11:05:13Z

I'm using dnsmasq on my mac, and my ansible role https://github.com/wayofdev/ansible-role-dnsmasq to setup dns on mac machines.

➜ cat /etc/resolver/docker
nameserver 127.0.0.1

➜ pwd
/opt/homebrew/etc

➜ cat dnsmasq.conf
address=/docker/127.0.0.1
address=/mac/127.0.0.1

mortymacs · 2022-11-29T12:04:27Z

@mortymacs

➜ dig router.docker @100.64.0.2

; <<>> DiG 9.10.6 <<>> router.docker @100.64.0.2 ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 11785 ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION: ;router.docker. IN A

;; Query time: 35 msec ;; SERVER: 100.64.0.2#53(100.64.0.2) ;; WHEN: Tue Nov 29 13:02:07 EET 2022 ;; MSG SIZE rcvd: 31

So, as it says, it couldn't fine anything.

I'm using dnsmasq on my mac, and my ansible role https://github.com/wayofdev/ansible-role-dnsmasq to setup dns on mac machines.

➜ cat /etc/resolver/docker nameserver 127.0.0.1

➜ pwd /opt/homebrew/etc

➜ cat dnsmasq.conf address=/docker/127.0.0.1 address=/mac/127.0.0.1

Well, this issue is not related to Wait4X, but, in order to address your issue, we need to know how do you run Wait4X? by Docker or binary on your local machine?

vbnetadmin · 2022-11-29T12:09:33Z

@mortymacs tried locally using binary and through docker (in separate image)

➜ which wait4x
/usr/local/bin/wait4x

Quick result, using docker image:

docker run --rm --name='wait4x' \
    atkrad/wait4x:latest http https://router.docker -l debug
2022-11-29T12:13:32Z INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:13:32Z DBG Get "https://router.docker": dial tcp 127.0.0.1:443: connect: connection refused
2022-11-29T12:13:33Z INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:13:33Z DBG Get "https://router.docker": dial tcp 127.0.0.1:443: connect: connection refused
2022-11-29T12:13:35Z INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:13:35Z DBG Get "https://router.docker": dial tcp 127.0.0.1:443: connect: connection refused
2022-11-29T12:13:36Z INF [HTTP] Checking the https://router.docker ...
2022-11-29T12:13:36Z DBG Get "https://router.docker": dial tcp 127.0.0.1:443: connect: connection refused

But, as you can see, using docker image for wait4x, it shows proper IP address

As a workaround, temporarily I can compile wait4x inside keycloak container (where I need to check for readiness)

Update:

Ok, this looks more promising, by adding flag --net=host I get this result:

➜ docker run --rm --net=host --name='wait4x'
atkrad/wait4x:latest http http://router.docker -l debug
2022-11-29T12:19:35Z INF [HTTP] Checking the http://router.docker ...
2022-11-29T12:19:35Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority
2022-11-29T12:19:36Z INF [HTTP] Checking the http://router.docker ...
2022-11-29T12:19:36Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority
2022-11-29T12:19:37Z INF [HTTP] Checking the http://router.docker ...
2022-11-29T12:19:37Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority
2022-11-29T12:19:38Z INF [HTTP] Checking the http://router.docker ...
2022-11-29T12:19:38Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority
2022-11-29T12:19:39Z INF [HTTP] Checking the http://router.docker ...
2022-11-29T12:19:39Z DBG Get "https://router.docker/": x509: certificate signed by unknown authority

vbnetadmin · 2022-11-29T13:03:11Z

Temporary, working solution

docker-compose.yaml:

...
  keycloak:
    image: quay.io/keycloak/keycloak:20.0.1
    container_name: ${COMPOSE_PROJECT_NAME_SAFE}_keycloak
    restart: on-failure
    ports:
      - "8765:8080"
...

Makefile:

_kc-wait:
	$(DOCKER) run --rm --net=host --name='wait4x' atkrad/wait4x:latest \
		http $(shell docker inspect --format 'http://127.0.0.1:{{ (index (index .NetworkSettings.Ports "8080/tcp") 0).HostPort }}' wod_keycloak) \
		-l debug \
		-t 1m
.PHONY: _kc-wait

Result:

➜ make _kc-wait
docker run --rm --net=host --name='wait4x' atkrad/wait4x:latest
http http://127.0.0.1:8765
-l debug
-t 1m
2022-11-29T12:59:19Z INF [HTTP] Checking the http://127.0.0.1:8765 ...

Btw, wait4x does not work, if container port of target isn't statically exposed:

    ports:
      - "8080"

➜ make _kc-wait
docker run --rm --net=host --name='wait4x' atkrad/wait4x:latest
http http://127.0.0.1:64584
-l debug
-t 1m
2022-11-29T13:02:32Z INF [HTTP] Checking the http://127.0.0.1:64584 ...
2022-11-29T13:02:32Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused
2022-11-29T13:02:33Z INF [HTTP] Checking the http://127.0.0.1:64584 ...
2022-11-29T13:02:33Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused
2022-11-29T13:02:34Z INF [HTTP] Checking the http://127.0.0.1:64584 ...
2022-11-29T13:02:34Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused
2022-11-29T13:02:35Z INF [HTTP] Checking the http://127.0.0.1:64584 ...
2022-11-29T13:02:35Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused
2022-11-29T13:02:36Z INF [HTTP] Checking the http://127.0.0.1:64584 ...
2022-11-29T13:02:36Z DBG Get "http://127.0.0.1:64584": dial tcp 127.0.0.1:64584: connect: connection refused

docker compose ps
NAME COMMAND SERVICE STATUS PORTS
wod_keycloak "/opt/keycloak/bin/k…" keycloak running (healthy) 8443/tcp, 0.0.0.0:64584->8080/tcp

sbaeurle · 2022-11-30T08:06:16Z

@vbnetadmin I'm not sure if you understand networking and docker correctly (see https://docs.docker.com/network/). Your first approach using a docker container and the localhost address may never work per default (since each container usually has it's own virtual network adapter).

If instead of 127.0.0.1 you would use the actual address of the service/host you want to check (either the IP address or the compose service name) then wait4x would be able to find a route to the service and verify it.

atkrad · 2022-12-06T08:24:36Z

@vbnetadmin, If you can't fix the problem, you can use the --insecure-skip-tls-verify flag to skip TLS verification.

mortymacs added the feature New feature or request label Oct 7, 2022

mortymacs self-assigned this Oct 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support self-signed TLS certiticate on HTTP component #95

Support self-signed TLS certiticate on HTTP component #95

mortymacs commented Oct 7, 2022

lotyp commented Nov 10, 2022 •

edited

Loading

mortymacs commented Nov 11, 2022

sbaeurle commented Nov 14, 2022

mortymacs commented Nov 14, 2022

sbaeurle commented Nov 14, 2022 •

edited

Loading

mortymacs commented Nov 15, 2022

sbaeurle commented Nov 15, 2022

mortymacs commented Nov 15, 2022

mortymacs commented Nov 15, 2022

sbaeurle commented Nov 16, 2022

mortymacs commented Nov 17, 2022

vbnetadmin commented Nov 29, 2022

sbaeurle commented Nov 29, 2022 •

edited

Loading

vbnetadmin commented Nov 29, 2022 •

edited

Loading

mortymacs commented Nov 29, 2022

vbnetadmin commented Nov 29, 2022

vbnetadmin commented Nov 29, 2022 •

edited

Loading

mortymacs commented Nov 29, 2022 •

edited

Loading

vbnetadmin commented Nov 29, 2022 •

edited

Loading

vbnetadmin commented Nov 29, 2022

sbaeurle commented Nov 30, 2022 •

edited

Loading

atkrad commented Dec 6, 2022

Support self-signed TLS certiticate on HTTP component #95

Support self-signed TLS certiticate on HTTP component #95

Comments

mortymacs commented Oct 7, 2022

lotyp commented Nov 10, 2022 • edited Loading

mortymacs commented Nov 11, 2022

sbaeurle commented Nov 14, 2022

mortymacs commented Nov 14, 2022

sbaeurle commented Nov 14, 2022 • edited Loading

mortymacs commented Nov 15, 2022

sbaeurle commented Nov 15, 2022

mortymacs commented Nov 15, 2022

mortymacs commented Nov 15, 2022

sbaeurle commented Nov 16, 2022

mortymacs commented Nov 17, 2022

vbnetadmin commented Nov 29, 2022

sbaeurle commented Nov 29, 2022 • edited Loading

vbnetadmin commented Nov 29, 2022 • edited Loading

mortymacs commented Nov 29, 2022

vbnetadmin commented Nov 29, 2022

vbnetadmin commented Nov 29, 2022 • edited Loading

mortymacs commented Nov 29, 2022 • edited Loading

vbnetadmin commented Nov 29, 2022 • edited Loading

vbnetadmin commented Nov 29, 2022

sbaeurle commented Nov 30, 2022 • edited Loading

atkrad commented Dec 6, 2022

lotyp commented Nov 10, 2022 •

edited

Loading

sbaeurle commented Nov 14, 2022 •

edited

Loading

sbaeurle commented Nov 29, 2022 •

edited

Loading

vbnetadmin commented Nov 29, 2022 •

edited

Loading

vbnetadmin commented Nov 29, 2022 •

edited

Loading

mortymacs commented Nov 29, 2022 •

edited

Loading

vbnetadmin commented Nov 29, 2022 •

edited

Loading

sbaeurle commented Nov 30, 2022 •

edited

Loading