Container has no network

Discussion about Docker
User avatar
mukmuk
Posts: 25
Joined: 08 Jan 2021, 21:39

Container has no network

Post by mukmuk »

Hello,

I'm trying to get docker via ssh to work. F2-221, TOS 4.1.30, Docker-App 17.05.5. My goal is having a reverse proxy (Traefik) and some containers like nextcloud and other stuff. To achieve this I'm using this tutorial: https://www.digitalocean.com/community/ ... untu-20-04

The traefik container comes up, but is not able to use the network connection through the newly created bridge called web. I can't reach the Traefik dashboard and the container itself can't lookup any DNS names:

Code: Select all

Unable to obtain ACME certificate for domains \"removed-for-forum\": cannot get ACME client get directory at 'https://acme-v02.api.letsencrypt.org/directory': Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp: lookup acme-v02.api.letsencrypt.org on 127.0.0.11:53: read udp 127.0.0.1:47496->127.0.0.11:53: i/o timeout
I've took the tutorial and used it on a fresh debian 10 machine and it works right as it should. I'm getting crazy over this. To prevent any misconfiguration by me, I've already reinstalled the Docker-App and deleted all remaining files.

Is there any service/iptables setting or something like that, that is preventing the connection? What can I do?



PS: to "unlock" port :80 and :443 I've disabled "jumpd" in /etc/nasips.ini as stated in this post: viewtopic.php?p=4017#p4017


Edit: this happens with any bridge that is created by "docker network create $bridgename". I've made all kinds of tests with a standard alpine container etc - no success on getting network to the containers.
User avatar
matt_30
Posts: 67
Joined: 02 Jan 2021, 23:22

Re: Container has no network

Post by matt_30 »

Oddly enough I have had the same problem

I tried to telnet acme-v02.api.letsencrypt.org 443 but this failed.

I went to do the same command on my pc and this worked.

Is this a bug?
User avatar
mukmuk
Posts: 25
Joined: 08 Jan 2021, 21:39

Re: Container has no network

Post by mukmuk »

Thanks matt_30, this is more or less good (or not so good) to hear. I thought I was going crazy, because I could've made an error.

Let's wait for the support then. Unless someone else has an idea. :)
User avatar
matt_30
Posts: 67
Joined: 02 Jan 2021, 23:22

Re: Container has no network

Post by matt_30 »

I will keep you in the loop. I did try to disable jumpd but this had no effect.

Oddly when I tried to setup swag I had to redirect port 443 -> 444 then in docker compose pipe 444 - >443 (same with port 80).

Will post more when I find more out.

Matt.
User avatar
mukmuk
Posts: 25
Joined: 08 Jan 2021, 21:39

Re: Container has no network

Post by mukmuk »

I've probably found a solution. While tinkering with "route", I saw that the docker bridges get random IPs from an unknown range. The F2-221 has 2 physical NICs - I have only one of them (eth0) running. eth1 has - despite being disabled - an IP from the subnet 169.254.0.0 assigned!

Take a look here:

Code: Select all

# ip a
3: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 6c:bf:b5:01:5f:56 brd ff:ff:ff:ff:ff:ff
    inet 192.168.178.36/24 brd 192.168.178.255 scope global eth0
11: eth1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether 6c:bf:b5:01:5f:57 brd ff:ff:ff:ff:ff:ff
    inet 169.254.253.106/16 brd 169.254.255.255 scope global eth1
       valid_lft forever preferred_lft forever

# docker network create newbridge
# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
c9ad54e42073        bridge              bridge              local
83ec6ae8033b        host                host                local
7528eb0f37de        newbridge         bridge              local

# ip addr show br-7528eb0f37de
37: br-7528eb0f37de: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:f7:66:65:c0 brd ff:ff:ff:ff:ff:ff
    inet 169.254.251.231/16 brd 169.254.255.255 scope global br-7528eb0f37de
       valid_lft forever preferred_lft forever
My LAN runs on eth0 192.168.178.0. The disabled eth1 interface has an IP inside 169.0.0.0. The bridge, that docker has created, got a correct route, but a wrong ip (169.254.251.231) on its interface. Adding the docker virtual ip to the interface, the container can connect to the internet:

Code: Select all

~~~ inside debian container on newbridge ~~~
root@9ad61a932cad:/# ping 172.19.0.1
PING 172.19.0.1 (172.19.0.1) 56(84) bytes of data.
^C
--- 172.19.0.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 64ms


~~~ on the NAS ~~~
# ip addr add 172.19.0.1/24 dev br-7528eb0f37de


~~~ inside debian container again ~~~
root@9ad61a932cad:/# ping 172.19.0.1
PING 172.19.0.1 (172.19.0.1) 56(84) bytes of data.
64 bytes from 172.19.0.1: icmp_seq=1 ttl=64 time=0.209 ms
64 bytes from 172.19.0.1: icmp_seq=2 ttl=64 time=0.133 ms
^C
--- 172.19.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 18ms
rtt min/avg/max/mdev = 0.133/0.171/0.209/0.038 ms
root@9ad61a932cad:/# ping google.com
PING google.com (216.58.212.142) 56(84) bytes of data.
64 bytes from ams15s21-in-f14.1e100.net (216.58.212.142): icmp_seq=1 ttl=118 time=24.4 ms
64 bytes from ams15s21-in-f14.1e100.net (216.58.212.142): icmp_seq=2 ttl=118 time=33.9 ms
64 bytes from ams15s21-in-f14.1e100.net (216.58.212.142): icmp_seq=3 ttl=118 time=22.6 ms
^C
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 6ms
rtt min/avg/max/mdev = 22.625/26.951/33.854/4.933 ms
I guess that any part of the bridge creation takes a wrong IP or getting confused by eth1. I'm not sure on this one, as I have no knowledge about the underlying functions. But apparently the connection works now.

I'll try to setup my complete installation with Traefik etc. tomorrow and keep y'all updated!
User avatar
mukmuk
Posts: 25
Joined: 08 Jan 2021, 21:39

Re: Container has no network

Post by mukmuk »

Sadly the edit window is closed already. I just wanted to add that the route sometimes takes the wrong subnet, too. Manually adjusting the route needs to be done in that case, too.
User avatar
matt_30
Posts: 67
Joined: 02 Jan 2021, 23:22

Re: Container has no network

Post by matt_30 »

The 169.254 range is the apipa used in DHCP on windows for finding devices (https://www.webopedia.com/definitions/apipa/)

I would expect to see that.

What does your route table show you? im hoping the 169.254 range is not your default gateway.

Matt.
User avatar
mukmuk
Posts: 25
Joined: 08 Jan 2021, 21:39

Re: Container has no network

Post by mukmuk »

I'm aware of the 169-range. As soon as the container starts, the bridge gets routed over this subnet. But only when the first container starts...

Code: Select all

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.178.1   0.0.0.0         UG    0      0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.178.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0

# docker network create heymatt
acfe7629b9c70b38429a8ab972c2cc53d520c17c432130f7740e1fb5b9e1a799
# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
c9ad54e42073        bridge              bridge              local
acfe7629b9c7        heymatt             bridge              local
83ec6ae8033b        host                host                local
61ee2316d6d1        none                null                local

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.178.1   0.0.0.0         UG    0      0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-acfe7629b9c7
192.168.178.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0

--- fine up to here ---

second shell: # docker run --rm -it --name debian4 --network heymatt debian:buster

first shell: # route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.178.1   0.0.0.0         UG    0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 br-acfe7629b9c7
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 vethaf32b60
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
192.168.178.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0

# ip route list
default via 192.168.178.1 dev eth0 src 192.168.178.36 
169.254.0.0/16 dev br-acfe7629b9c7 proto kernel scope link src 169.254.251.125 
169.254.0.0/16 dev vethaf32b60 proto kernel scope link src 169.254.251.81 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 
192.168.178.0/24 dev eth0 scope link src 192.168.178.36
Modifying the route as well as the interface of br-acfe7629b9c7 to the docker assigned subnet 172.18.0.0/16 seem to work fine.
User avatar
TMS
TerraMaster Team
Posts: 508
Joined: 20 Oct 2020, 17:06

Re: Container has no network

Post by TMS »

mukmuk wrote: 09 Jan 2021, 02:47 I'm trying to get docker via ssh to work. F2-221, TOS 4.1.30, Docker-App 17.05.5. My goal is having a reverse proxy (Traefik) and some containers like nextcloud and other stuff. To achieve this I'm using this tutorial: https://www.digitalocean.com/community/ ... untu-20-04
Hi,
According to this tutorial, there is no need to add the line command --netweb web (it will affect the use of other ports). Can it run if you try again?
Attachments
photo1.png
photo1.png (17.54 KiB) Viewed 5196 times
To contact our tech team, please email to support(at)terra-master.com, remember to replace (at) with @
User avatar
mukmuk
Posts: 25
Joined: 08 Jan 2021, 21:39

Re: Container has no network

Post by mukmuk »

Thanks for your reply. The web-bridge is needed for traefik to discover further services as well as using it via docker-compose. In the tutorial it reads:
The docker provider enables Traefik to act as a proxy in front of Docker containers. You’ve configured the provider to watch for new containers on the web network, which you’ll create soon.

-snip-

The Docker network is necessary so that you can use it with applications that are run using Docker Compose.
Further down you'll find the docker-compose example with the instruction

Code: Select all

networks:
  web:
    external: true
  internal:
    external: false
Attached there is a quick sketch of the network/container design, so you can take a better look at the setup. Neither the reverse proxy traefik, nor any other services inside the web-bridge can access any network connection. No DNS, no ping etc.
Attachments
sketch.jpg
sketch.jpg (18.81 KiB) Viewed 5192 times
Locked