[libnetwork] cali interface not show on other host and cannot ping in the same network #188

huynhducbk95 · 2018-04-19T09:49:06Z

My deployment deploy docker container with calico network have 2 nodes (virtual machine created by KVM).

node 1: hostname = docker and ip = 192.168.122.45
node 2: hostname = database and ip = 192.168.122.13

when I done all step, I get state:

root@docker:~# calicoctl node status
Calico process is running.

IPv4 BGP status
+-----------------+-------------------+-------+----------+-------------+
|  PEER ADDRESS   |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+-----------------+-------------------+-------+----------+-------------+
| 192.168.122.13  | node-to-node mesh | up    | 08:11:11 | Established |
+-----------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

root@database:~# calicoctl node status
Calico process is running.

IPv4 BGP status
+-----------------+-------------------+-------+----------+-------------+
|  PEER ADDRESS   |     PEER TYPE     | STATE |  SINCE   |    INFO     |
+-----------------+-------------------+-------+----------+-------------+
| 192.168.122.45  | node-to-node mesh | up    | 08:11:10 | Established |
+-----------------+-------------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

I got two problems as follow:

I created a ipPool and a network and two container in this network:

root@docker:~# cat << EOF | calicoctl create -f -
> - apiVersion: v1
>   kind: ipPool
>   metadata:
>     cidr: 192.0.2.0/24
> EOF
Successfully created 1 'ipPool' resource(s)

root@docker:~# docker network create --driver calico --ipam-driver calico-ipam --subnet=192.0.2.0/24 my_net
0946dcb47a02515b80edbb09eadd170683bca7510d7a9d50656990760faab026
root@docker:~# docker run --net my_net --name my_workload --ip 192.0.2.100 -tid busybox
028462ea67ebb8ed2d8678b30cf158657cf2328b3fb39e70b4fe6be0534f5833
root@docker:~# docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' my_workload
192.0.2.100
root@docker:~#docker run --net my_net --name my_workload01 --ip 192.0.2.200 -tid busybox
bb3777ffdbc5ec02671a6e152b6a84d2fac61e90c7cb11b20589e8f1c02e0630
root@docker:~# docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' my_workload01
192.0.2.200

And then, I tried to ping from container my_workload to container my_workload01 but it's failed.

root@docker:~# docker exec my_workload ping -c 4 my_workload01.my_net
PING my_workload01.my_net (192.0.2.200): 56 data bytes

--- my_workload01.my_net ping statistics ---
4 packets transmitted, 0 packets received, 100% packet loss

2, I can see these container's IP on docker host, but not on database host:

root@docker:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.122.1   0.0.0.0         UG    0      0        0 ens3
10.10.10.0      192.168.122.13  255.255.255.192 UG    0      0        0 ens3
25.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 ham0
100.64.100.0    192.168.122.13  255.255.255.192 UG    0      0        0 tunl0
100.64.100.4    0.0.0.0         255.255.255.255 UH    0      0        0 cali7d5062fda79
100.64.100.8    0.0.0.0         255.255.255.255 UH    0      0        0 calic0d781ea16e
100.64.100.128  0.0.0.0         255.255.255.192 U     0      0        0 *
100.64.100.192  192.168.122.13  255.255.255.192 UG    0      0        0 tunl0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.0.2.64      0.0.0.0         255.255.255.192 U     0      0        0 *
192.0.2.100     0.0.0.0         255.255.255.255 UH    0      0        0 calif7149fe5d58
192.0.2.192     0.0.0.0         255.255.255.192 U     0      0        0 *
192.0.2.200     0.0.0.0         255.255.255.255 UH    0      0        0 calia50a4846148
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 ens3

root@database:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.122.1   0.0.0.0         UG    0      0        0 ens3
10.10.10.0      0.0.0.0         255.255.255.192 U     0      0        0 *
25.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 ham0
100.64.100.0    0.0.0.0         255.255.255.192 U     0      0        0 *
100.64.100.128  192.168.122.45  255.255.255.192 UG    0      0        0 tunl0
100.64.100.192  0.0.0.0         255.255.255.192 U     0      0        0 *
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.0.2.64      192.168.122.45  255.255.255.192 UG    0      0        0 ens3
192.0.2.192     192.168.122.45  255.255.255.192 UG    0      0        0 ens3
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 ens3

I aslo tried with IPIP mode, but I got the same error
can anyone help me:
why cali* interfaces not shown in other host?
why ping failed between containers in the same host and the same network, please?

My Environment

calicoctl version (e.g. calicoctl version):

root@docker:~# calicoctl version
Client Version:    v1.6.3
Build date:        2017-12-20T22:32:36+0000
Git commit:        d4cfc95c
Cluster Version:   v2.6.8
Cluster Type:      unknown

calico/node version (e.g. docker run --rm calico/node versions): v2.6.8
Orchestrator type and version (e.g. kubernetes, mesos, rkt): docker
Operating System and version: Ubuntu 16.04.2 LTS
Link to your project (optional):

The text was updated successfully, but these errors were encountered:

caseydavenport · 2018-04-19T22:13:49Z

The cali interfaces will only show up on nodes that are actually running the containers, so I think that's OK since you launched the continers on only one node.

To figure out why ping isn't working, I'd recommend using tcpdump to view the traffic and see where it gets dropped.

Also iptables-save -c | grep DROP to see if any iptables rules are dropping the traffic.

huynhducbk95 · 2018-04-20T03:17:46Z

@caseydavenport thanks for your response.
I know that when I create a container on one node (docker node), on other node that peer with docker node will create a route in the node's routing table (database node).
This route has :

Destination field is IP of the container that I created on docker node.
Gateway field is IP of the docker node that container was created.
But, I don't see that on other peer with docker node.
is this correct for this calico version?

tmjd · 2018-04-20T16:23:01Z

You most likely won't see a specific IP of the container but a subnet on the database node. That is why you see

192.0.2.64      192.168.122.45  255.255.255.192 UG    0      0        0 ens3
192.0.2.192     192.168.122.45  255.255.255.192 UG    0      0        0 ens3

Calico uses /26 subnets to reduce the number of routes to distribute low.

Above I said 'most likely' because if you have used up all the /26 subnets and a node needs another address it can use an IP from a subnet another host has claimed but will then distribute that specific route and the more specific route takes precedence.

huynhducbk95 · 2018-04-21T18:38:47Z

@tmjd thanks for your explanation and sorry for delay reply.
Now, I know why some IP of containers didn't shown in routing table on other node.
But a little sad, when I ping between two container on one node. It's failed.
by tcpdump on these container's interface, I see that the destination container got ICMP request packet, but I don't see any reply packet.

# docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' centos-ssh13
10.10.20.21

# docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' centos-ssh12
10.10.20.10

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.122.1   0.0.0.0         UG    0      0        0 ens3
10.10.10.0      192.168.122.13  255.255.255.192 UG    0      0        0 tunl0
10.10.10.10     0.0.0.0         255.255.255.255 UH    0      0        0 cali7721a9c82ab
10.10.10.64     192.168.122.13  255.255.255.192 UG    0      0        0 tunl0
10.10.10.128    0.0.0.0         255.255.255.192 U     0      0        0 *
10.10.20.0      0.0.0.0         255.255.255.192 U     0      0        0 *
10.10.20.10     0.0.0.0         255.255.255.255 UH    0      0        0 cali9027171f6c6
10.10.20.12     0.0.0.0         255.255.255.255 UH    0      0        0 cali0b54a54a96c
10.10.20.13     0.0.0.0         255.255.255.255 UH    0      0        0 cali5c2866a2deb
10.10.20.14     0.0.0.0         255.255.255.255 UH    0      0        0 calib316fafae20
10.10.20.20     192.168.122.13  255.255.255.255 UGH   0      0        0 tunl0
10.10.20.21     0.0.0.0         255.255.255.255 UH    0      0        0 cali5d7128fa54b
25.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 ham0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 ens3

# docker exec -it centos-ssh13 /bin/bash
[root@aca0a99ef00e /]# ping 10.10.20.10
PING 10.10.20.10 (10.10.20.10) 56(84) bytes of data.

and at the same time, at the interface of the container that has IP 10.10.20.10:

# tcpdump -i cali9027171f6c6
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on cali9027171f6c6, link-type EN10MB (Ethernet), capture size 262144 bytes
01:33:08.271422 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 1, length 64
01:33:09.271116 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 2, length 64
01:33:10.271090 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 3, length 64
01:33:11.271115 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 4, length 64
01:33:12.271135 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 5, length 64
01:33:13.271091 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 6, length 64
01:33:14.271134 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 7, length 64
01:33:15.271093 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 8, length 64
01:33:16.271127 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 9, length 64
01:33:17.271097 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 10, length 64
01:33:18.271084 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 11, length 64
01:33:19.271082 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 12, length 64
01:33:20.271159 IP 10.10.20.21 > 10.10.20.10: ICMP echo request, id 127, seq 13, length 64

I checked to find any packet was dropped, but no one:

:FORWARD DROP [0:0]
[0:0] -A cali-INPUT -p ipencap -m comment --comment "cali:JhfQUFFJ2v0jbipF" -m comment --comment "Drop IPIP packets from non-Calico hosts" -j DROP
[0:0] -A cali-from-wl-dispatch -m comment --comment "cali:aqepaSa3MXh7Xxam" -m comment --comment "Unknown interface" -j DROP
[0:0] -A cali-from-wl-dispatch-5 -m comment --comment "cali:1VYHDwn-XTXEuBin" -m comment --comment "Unknown interface" -j DROP
[0:0] -A cali-fw-cali0b54a54a96c -m comment --comment "cali:ICqkhxTDsNW_FCNk" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-fw-cali0b54a54a96c -m comment --comment "cali:MWiF4UrSv31dAtsb" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-fw-cali5c2866a2deb -m comment --comment "cali:V28oIxWcFdKRBKi6" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-fw-cali5c2866a2deb -m comment --comment "cali:KrqzJce7_Mghhckm" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-fw-cali5d7128fa54b -m comment --comment "cali:-aHux0npyzU7m26z" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-fw-cali5d7128fa54b -m comment --comment "cali:z7dLDSlTQalBjeUX" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-fw-cali7721a9c82ab -m comment --comment "cali:BKhV-4pkkG7idn-m" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-fw-cali7721a9c82ab -m comment --comment "cali:xPbFhkEp6A7tmzfu" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-fw-cali9027171f6c6 -m comment --comment "cali:LHXOFUUurfunNlBZ" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-fw-cali9027171f6c6 -m comment --comment "cali:f29xsWB8BN-kefLT" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-fw-calib316fafae20 -m comment --comment "cali:oEbscWabYM2x0BYh" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-fw-calib316fafae20 -m comment --comment "cali:dhJVhkipa1mlUC9C" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-to-wl-dispatch -m comment --comment "cali:ASVb8Sm9GP1T6uU5" -m comment --comment "Unknown interface" -j DROP
[0:0] -A cali-to-wl-dispatch-5 -m comment --comment "cali:x-ffYVdO9l0mNu3L" -m comment --comment "Unknown interface" -j DROP
[0:0] -A cali-tw-cali0b54a54a96c -m comment --comment "cali:XqGUdPqFy69Souqs" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-tw-cali0b54a54a96c -m comment --comment "cali:evfZPw0dOMI67mgg" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-tw-cali5c2866a2deb -m comment --comment "cali:sCcK0psrCUUtPYcr" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-tw-cali5c2866a2deb -m comment --comment "cali:pJ4fz7nU4rYe_cNK" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-tw-cali5d7128fa54b -m comment --comment "cali:XS4zL7fxjly4zB6G" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-tw-cali5d7128fa54b -m comment --comment "cali:dToPt6YwgXEBwRJk" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-tw-cali7721a9c82ab -m comment --comment "cali:PtdZjV6UYfXxesy-" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-tw-cali7721a9c82ab -m comment --comment "cali:fxn9qzLBmzTNi9Rv" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-tw-cali9027171f6c6 -m comment --comment "cali:nBSSKAQ78rkrzBU4" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-tw-cali9027171f6c6 -m comment --comment "cali:1-Jm-vPF2hD6qkDX" -m comment --comment "Drop if no profiles matched" -j DROP
[0:0] -A cali-tw-calib316fafae20 -m comment --comment "cali:gIp-YYm6od0MSODn" -m conntrack --ctstate INVALID -j DROP
[0:0] -A cali-tw-calib316fafae20 -m comment --comment "cali:DO1iA0sgWknpAi_J" -m comment --comment "Drop if no profiles matched" -j DROP

Could you please explain why this happen and how can I get ping between them success?

tmjd · 2018-04-23T14:34:50Z

If you try pinging the other way 10.10.20.10 > 10.10.20.21 do you see the ICMP echo requests if you tcpdump cali5d7128fa54b? (If you haven't done that already, I'm just trying to make sure traffic works both ways.)

huynhducbk95 · 2018-04-23T14:43:26Z

yes, I see the ICMP echo requests by tcpdump cali5d7128fa54b when I ping 10.10.20.10 > 10.10.20.21
But, both ways I don't see any reply packet :(

tmjd · 2018-04-23T15:13:27Z

Could you run ip a and ip r inside the container?
Also have you tried pinging from the host to either of the containers?

You could try downgrading to Calico v2.6.2? I'm only suggesting this because of projectcalico/calico#1512

huynhducbk95 · 2018-04-24T07:06:24Z

@tmjd I tried commands, both work fine:

root@docker01:~#  docker attach my_workload03
/ # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
30: cali0@if31: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether 86:24:00:ce:d2:5d brd ff:ff:ff:ff:ff:ff
    inet 192.0.2.103/32 brd 192.0.2.103 scope global cali0
       valid_lft forever preferred_lft forever
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
30: cali0@if31: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether 86:24:00:ce:d2:5d brd ff:ff:ff:ff:ff:ff
    inet 192.0.2.103/32 brd 192.0.2.103 scope global cali0
       valid_lft forever preferred_lft forever
/ # ip r
default via 169.254.1.1 dev cali0 
169.254.1.1 dev cali0 scope link 
/ #

I also tried ping from the host to containers, I got the same result:

root@docker01:~# ping 192.0.2.103
PING 192.0.2.103 (192.0.2.103) 56(84) bytes of data.

--- 192.0.2.103 ping statistics ---
71 packets transmitted, 0 received, 100% packet loss, time 69999ms

at the same time:

root@docker01:~# tcpdump -i calie215c05df7a
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on calie215c05df7a, link-type EN10MB (Ethernet), capture size 262144 bytes
14:01:56.517421 IP docker01 > 192.0.2.103: ICMP echo request, id 2596, seq 29, length 64
14:01:57.525470 IP docker01 > 192.0.2.103: ICMP echo request, id 2596, seq 30, length 64
14:01:58.533479 IP docker01 > 192.0.2.103: ICMP echo request, id 2596, seq 31, length 64
14:01:59.541438 IP docker01 > 192.0.2.103: ICMP echo request, id 2596, seq 32, length 64
14:02:00.549378 IP docker01 > 192.0.2.103: ICMP echo request, id 2596, seq 33, length 64
14:02:01.557432 IP docker01 > 192.0.2.103: ICMP echo request, id 2596, seq 34, length 64
14:02:02.565510 IP docker01 > 192.0.2.103: ICMP echo request, id 2596, seq 35, length 64
14:02:03.573501 IP docker01 > 192.0.2.103: ICMP echo request, id 2596, seq 36, length 64
^C
8 packets captured
8 packets received by filter
0 packets dropped by kernel

I aslo get this problem when I try downgrading to Calico v2.6.2.

huynhducbk95 · 2018-04-24T08:39:46Z

when I ping from container to the host (in data center), I get reply packet but not shown in output of ping command:

root@docker01:/home/controller# docker attach my_workload02
/ # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
40: cali0@if41: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether da:9b:a6:1d:15:c5 brd ff:ff:ff:ff:ff:ff
    inet 192.0.2.102/32 brd 192.0.2.102 scope global cali0
       valid_lft forever preferred_lft forever

# ping 192.168.122.45
PING 192.168.122.45 (192.168.122.45): 56 data bytes
^C
--- 192.168.122.45 ping statistics ---
12 packets transmitted, 0 packets received, 100% packet loss

and at vether calif73b54c4622:

root@docker01:~# tcpdump -i calif73b54c4622
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on calif73b54c4622, link-type EN10MB (Ethernet), capture size 262144 bytes
15:36:17.823026 IP 192.0.2.102 > docker01: ICMP echo request, id 3584, seq 2, length 64
15:36:17.823070 IP docker01 > 192.0.2.102: ICMP echo reply, id 3584, seq 2, length 64
15:36:18.823253 IP 192.0.2.102 > docker01: ICMP echo request, id 3584, seq 3, length 64
15:36:18.823316 IP docker01 > 192.0.2.102: ICMP echo reply, id 3584, seq 3, length 64
15:36:19.823580 IP 192.0.2.102 > docker01: ICMP echo request, id 3584, seq 4, length 64
15:36:19.823664 IP docker01 > 192.0.2.102: ICMP echo reply, id 3584, seq 4, length 64
15:36:20.823828 IP 192.0.2.102 > docker01: ICMP echo request, id 3584, seq 5, length 64
15:36:20.823901 IP docker01 > 192.0.2.102: ICMP echo reply, id 3584, seq 5, length 64
15:36:20.827264 ARP, Request who-has 169.254.1.1 tell 192.0.2.102, length 28
15:36:20.827279 ARP, Reply 169.254.1.1 is-at 32:82:ad:c5:c4:0d (oui Unknown), length 28
15:36:21.824100 IP 192.0.2.102 > docker01: ICMP echo request, id 3584, seq 6, length 64
15:36:21.824166 IP docker01 > 192.0.2.102: ICMP echo reply, id 3584, seq 6, length 64
15:36:22.824399 IP 192.0.2.102 > docker01: ICMP echo request, id 3584, seq 7, length 64
15:36:22.824458 IP docker01 > 192.0.2.102: ICMP echo reply, id 3584, seq 7, length 64
15:36:23.824561 IP 192.0.2.102 > docker01: ICMP echo request, id 3584, seq 8, length 64
15:36:23.824603 IP docker01 > 192.0.2.102: ICMP echo reply, id 3584, seq 8, length 64
15:36:24.824816 IP 192.0.2.102 > docker01: ICMP echo request, id 3584, seq 9, length 64
15:36:24.824886 IP docker01 > 192.0.2.102: ICMP echo reply, id 3584, seq 9, length 64

Could you please explain why I don't see any output from ping command? is this because of calico?

tmjd · 2018-05-01T14:41:37Z

I'm not sure why that would be. If you are seeing the traffic on the interface then it should be in the container, there is no filtering (that I'm aware of) in the container.
Could you try running an nginx container or run a container with nc/netcat and try a TCP connection instead of ICMP traffic? I don't have any reasoning to expect a difference but it may be interesting to see if there is a difference there,

huynhducbk95 · 2018-05-02T08:38:33Z

I just installed with following basic commands. It's so basic, so I don't know why I got this bug:

Install docker.
Add --cluster-store to docker service file and restart daemon.
set prefix calico interface for container created by this docker host.

etcdctl set /calico/v1/host/<docker host>/config/InterfacePrefix cali

Download and change mode for calicoctl:

sudo wget -O /usr/local/bin/calicoctl https://github.com/projectcalico/calicoctl/releases/download/v1.6.3/calicoctl
sudo chmod +x /usr/local/bin/calicoctl

Run calico node:

ETCD_ENDPOINTS=http://${NODE_IP}:2379 calicoctl node run --no-default-ippools --ip=${NODE_IP} --node-image=quay.io/calico/node:v2.6.8

When calico node run correctly.
I created one ip pool:

# cat << EOF | calicoctl create -f -
- apiVersion: v1
  kind: ipPool
  metadata:
    cidr: 192.0.2.0/24
EOF

and then, I created a network with above pool:

# docker network create --driver calico --ipam-driver calico-ipam --subnet=192.0.2.0/24 my_net

and created containers:

# docker run --net my_net --name my_workload2 --ip 192.0.2.2 -tid busybox
# docker run --net my_net --name my_workload3 --ip 192.0.2.3 -tid busybox

That's all my basic commands I did. please take a look, and tell me if any command wrong?
thanks so much.

tmjd · 2018-05-04T13:17:06Z

One thing that looks odd to me is that you set the InterfacePrefix for calico. Did you find that was needed? because it looks like what you've set should be the default. TBH I didn't know that was an option to change.

Did you try as I suggested and try netcat or nginx as your container? If the ping traffic is making it to the cali interface then it is in the container so at that point I don't know what could be wrong. You show that you're using busybox and that is not a problem I've seen before with that image.

Anyone else got ideas?

huynhducbk95 · 2018-05-04T17:10:33Z

@tmjd thanks for your help,
I found this is a bug with docker 18.x
everything will ok with docker 17.12.

tmjd · 2018-05-10T16:16:12Z

@huynhducbk95 Do you have a link to the issue?

wegel · 2018-06-18T18:37:47Z

@huynhducbk95 I would be very interested to know exactly what is that Docker 18.x issue, as I'm facing the problem. Thanks!

huynhducbk95 · 2018-06-19T01:55:14Z

@wegel I don't know exactly what version used, but I just downgrade to 17.12.1~ce-0~ubuntu

Kr1Kr1 · 2018-08-17T15:16:00Z

I was facing the exact same issue that @huynhducbk95 and the problem was solved after downgrading from docker 18.03.1-ce to 17.03.2-ce. If someone knows whats going on with Docker 18.x, that would be great!

xej520 · 2018-10-26T06:01:40Z

docker 18.06.0-ce is ok

hyponet · 2021-05-11T09:00:57Z

I was facing the exact same issue that @huynhducbk95 and the problem was solved after downgrading from docker 18.03.1-ce to 17.03.2-ce. If someone knows whats going on with Docker 18.x, that would be great!

thx, I never thought about docker, but the problem was solved after restart docker.

docker version

docker version
Client:
 Version:           18.09.9
 API version:       1.39
 Go version:        go1.11.13
 Git commit:        039a7df
 Built:             Wed Sep  4 16:54:32 2019
 OS/Arch:           linux/arm64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.9
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.11.13
  Git commit:       039a7df
  Built:            Wed Sep  4 16:25:05 2019
  OS/Arch:          linux/arm64
  Experimental:     false

caseydavenport changed the title ~~cali interface not show on other host and cannot ping in the same network~~ [libnetwork] cali interface not show on other host and cannot ping in the same network Mar 31, 2019

caseydavenport transferred this issue from projectcalico/calicoctl Apr 5, 2019

caseydavenport added the priority/backlog label Apr 17, 2019

caseydavenport added the kind/support label May 13, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[libnetwork] cali interface not show on other host and cannot ping in the same network #188

[libnetwork] cali interface not show on other host and cannot ping in the same network #188

huynhducbk95 commented Apr 19, 2018

caseydavenport commented Apr 19, 2018

huynhducbk95 commented Apr 20, 2018

tmjd commented Apr 20, 2018

huynhducbk95 commented Apr 21, 2018 •

edited

Loading

tmjd commented Apr 23, 2018

huynhducbk95 commented Apr 23, 2018

tmjd commented Apr 23, 2018

huynhducbk95 commented Apr 24, 2018 •

edited

Loading

huynhducbk95 commented Apr 24, 2018 •

edited

Loading

tmjd commented May 1, 2018

huynhducbk95 commented May 2, 2018

tmjd commented May 4, 2018

huynhducbk95 commented May 4, 2018

tmjd commented May 10, 2018

wegel commented Jun 18, 2018

huynhducbk95 commented Jun 19, 2018 •

edited

Loading

Kr1Kr1 commented Aug 17, 2018

xej520 commented Oct 26, 2018

hyponet commented May 11, 2021

[libnetwork] cali interface not show on other host and cannot ping in the same network #188

[libnetwork] cali interface not show on other host and cannot ping in the same network #188

Comments

huynhducbk95 commented Apr 19, 2018

My Environment

caseydavenport commented Apr 19, 2018

huynhducbk95 commented Apr 20, 2018

tmjd commented Apr 20, 2018

huynhducbk95 commented Apr 21, 2018 • edited Loading

tmjd commented Apr 23, 2018

huynhducbk95 commented Apr 23, 2018

tmjd commented Apr 23, 2018

huynhducbk95 commented Apr 24, 2018 • edited Loading

huynhducbk95 commented Apr 24, 2018 • edited Loading

tmjd commented May 1, 2018

huynhducbk95 commented May 2, 2018

tmjd commented May 4, 2018

huynhducbk95 commented May 4, 2018

tmjd commented May 10, 2018

wegel commented Jun 18, 2018

huynhducbk95 commented Jun 19, 2018 • edited Loading

Kr1Kr1 commented Aug 17, 2018

xej520 commented Oct 26, 2018

hyponet commented May 11, 2021

huynhducbk95 commented Apr 21, 2018 •

edited

Loading

huynhducbk95 commented Apr 24, 2018 •

edited

Loading

huynhducbk95 commented Apr 24, 2018 •

edited

Loading

huynhducbk95 commented Jun 19, 2018 •

edited

Loading