Introduction
This document describes how to run EdgeVPN.io nodes as endpoints in a simulated Mininet network.
Mininet allows you to create virtual networks with emulated links, where you can control link topology, latency and bandwidth. This allows great flexibility in running test environments and performance evaluations with large numbers of EdgeVPN.io nodes in a single server. This tutorial uses the Containernet extensions to Mininet, where it is possible to run Docker containers as host endpoints.
Roadmap
The setup to run an emulated network requires the following steps.
Note: these steps have been tested on an Ubuntu 18.04 LTS platform:
- Install Docker and mininet/containernet on your host
- Deploy an Openfire container to serve as a local XMPP server
- Configure and deploy a Mininet topology (using Python APIs) with two or more Docker evio-node-mininet containers
- Configure and deploy EdgeVPN.io in the above containers
Install Docker and mininet/containernet on your host
Run the following commands to configure Docker and Open vSwitch for your host:
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable"
sudo apt-get update -y
sudo apt-get install -y openvswitch-switch \
python3 python3-pip python3-venv \
apt-transport-https \
ca-certificates \
curl git \
software-properties-common \
containerd.io \
docker-ce-cli \
docker-ce
groupadd -f docker
sudo usermod -a -G docker $USER
Note: log into another shell after these commands so the user is able to run Docker commands
Follow the instructions from the Containernet manual to install Mininet and Containernet on your host
Deploy an Openfire container to serve as a local XMPP server
We have created a pre-configured Openfire Docker container that makes it easy to run an XMPP server on your local host. To get it started, run:
sudo docker run --name openfire -d -p 9090:9090 -p 5222:5222 -p 5269:5269 -p 5223:5223 -p 7443:7443 -p 7777:7777 -p 7070:7070 -p 5229:5229 -p 5275:5275 edgevpnio/openfire_edgevpn_demo
Extra Dependencies
sudo apt install net-tools
Configure and deploy a Mininet topology
We'll make use of a simple example based on the Containernet demo. Save the code below to a start-mininet.py file:
"""
Example topology with two containers (d1, d2),
two switches, and one controller:
- (c)-
| |
(d1) - (s1) - (s2) - (d2)
"""
from mininet.net import Containernet
from mininet.node import Controller
from mininet.cli import CLI
from mininet.link import TCLink
from mininet.log import info, setLogLevel
setLogLevel('info')
net = Containernet(controller=Controller)
info('*** Adding controller\n')
net.addController('c0')
info('*** Adding docker containers using evio-node-mininet images\n')
d1 = net.addDocker('d1', ip='10.0.0.251', dimage="edgevpnio/evio-node-mininet:latest")
d2 = net.addDocker('d2', ip='10.0.0.252', dimage="edgevpnio/evio-node-mininet:latest")
info('*** Adding switches\n')
s1 = net.addSwitch('s1')
s2 = net.addSwitch('s2')
info('*** Creating links\n')
net.addLink(d1, s1)
net.addLink(s1, s2, cls=TCLink, delay='100ms', bw=1)
net.addLink(s2, d2)
info('*** Starting network\n')
net.start()
info('*** Testing connectivity\n')
net.ping([d1, d2])
info('*** Running CLI\n')
CLI(net)
info('*** Stopping network')
net.stop()
Then, execute this program with:
sudo python3 start-mininet.py
This should start a simple topology with two hosts (10.0.0.251, and 10.0.0.252), and do a ping test between them. If all goes well, two containers (named mn.d1 and mn.d2) should be running on your host, and you should see the following in your terminal:
*** Adding controller
*** Adding docker containers using ubuntu:trusty images
1:
d1: kwargs {'ip': '10.0.0.251'}
d1: update resources {'cpu_quota': -1}
1:
d2: kwargs {'ip': '10.0.0.252'}
d2: update resources {'cpu_quota': -1}
*** Adding switches
*** Creating links
(1.00Mbit 100ms delay) (1.00Mbit 100ms delay) (1.00Mbit 100ms delay) (1.00Mbit 100ms delay) *** Starting network
*** Configuring hosts
d1 d2
*** Starting controller
c0
*** Starting 2 switches
s1 (1.00Mbit 100ms delay) s2 (1.00Mbit 100ms delay) ...(1.00Mbit 100ms delay) (1.00Mbit 100ms delay)
*** Testing connectivity
d1 -> d2
d2 -> d1
*** Results: 0% dropped (2/2 received)
*** Running CLI
*** Starting CLI:
containernet>
Configure and deploy EdgeVPN.io
The containers mn.d1 and mn.d2 are now running and are pingable over the 10.0.0.* interface, but they are not yet running EdgeVPN.io. The next step is to copy configuration files and run EdgeVPN.io in the two containers.
Setup the configuration file
For this demo, you need to create two Evio configuration files, config-001.json and config-002.json. Copy the template below, and replace:
- X.X.X.X with the IP address of the Openfire Docker container you just started above
- Y.Y.Y.Y with the IP address of the Evio node - for instance, 192.168.10.1 for config-001.json and 192.168.10.2 for config-002.json
{
"CFx": {
"Model": "Default",
"Overlays": [
"1234567"
]
},
"Logger": {
"LogLevel": "DEBUG",
"Device": "File",
"Directory": "/var/log/edge-vpnio/",
"CtrlLogFileName": "ctrl.log",
"TincanLogFileName": "tincan_log",
"MaxFileSize": 10000000,
"MaxArchives": 1
},
"Signal": {
"Enabled": true,
"Overlays": {
"1234567": {
"HostAddress": "X.X.X.X",
"Port": "5222",
"Username": "test1@openfire.local",
"Password": "password_test1",
"AuthenticationMethod": "PASSWORD"
}
}
},
"Topology": {
"PeerDiscoveryCoalesce": 1,
"Overlays": {
"1234567": {
"Name": "SymphonyRing",
"Description": "Scalable Symphony Ring Overlay for Bounded Flooding.",
"MaxSuccessors": 2,
"MaxOnDemandEdges": 1,
"MaxConcurrentEdgeSetup": 5,
"Role": "Switch"
}
}
},
"LinkManager": {
"Dependencies": [
"Logger",
"TincanInterface",
"Signal"
],
"Stun": [
"stun.l.google.com:19302",
"stun1.l.google.com:19302"
],
"Overlays": {
"1234567": {
"Type": "TUNNEL",
"IgnoredNetInterfaces": [ "eth1", "eth0" ],
"TapName": "tnl-"
}
}
},
"OverlayVisualizer": {
"Enabled": false,
"TimerInterval": 25,
"WebServiceAddress": "",
"NodeName": "n1"
},
"BridgeController": {
"Dependencies": [
"Logger",
"LinkManager"
],
"BoundedFlood": {
"OverlayId": "1234567",
"LogDir": "/var/log/edge-vpnio/",
"LogFilename": "bf.log",
"LogLevel": "DEBUG",
"BridgeName": "edgbr",
"DemandThreshold": "100M",
"FlowIdleTimeout": 60,
"FlowHardTimeout": 60,
"MulticastBroadcastInterval": 60,
"MaxBytes": 10000000,
"BackupCount": 0,
"ProxyListenAddress": "",
"ProxyListenPort": 5802,
"MonitorInterval": 60,
"MaxOnDemandEdges": 0
},
"Overlays": {
"1234567": {
"NetDevice": {
"AutoDelete": true,
"Type": "OVS",
"SwitchProtocol": "BF",
"NamePrefix": "edgbr",
"MTU": 1410,
"AppBridge": {
"AutoDelete": true,
"Type": "OVS",
"NamePrefix": "brl",
"IP4": "Y.Y.Y.Y",
"PrefixLen": 24,
"MTU": 1410
}
},
"SDNController": {
"ConnectionType": "tcp",
"HostName": "127.0.0.1",
"Port": "6633"
}
}
}
}
}
To copy these configuration files into the mn.d1 and mn.d2 and run EdgeVPN.io, copy and run the following Bash script on your host; save this as run-evio.sh, chmod it to 755 and run:
#!/bin/bash
docker cp config-001.json mn.d1:/etc/opt/edge-vpnio/config.json
docker cp config-002.json mn.d2:/etc/opt/edge-vpnio/config.json
docker exec -it mn.d1 /bin/bash -c "/etc/init.d/openvswitch-switch start"
docker exec -it mn.d2 /bin/bash -c "/etc/init.d/openvswitch-switch start"
docker exec -it mn.d1 /bin/bash -c "nohup /opt/edge-vpnio/evio-start"
docker exec -it mn.d2 /bin/bash -c "nohup /opt/edge-vpnio/evio-start"
You should see the following in your terminal:
* /etc/openvswitch/conf.db does not exist
* Creating empty database /etc/openvswitch/conf.db
* Starting ovsdb-server
* Configuring Open vSwitch system IDs
* Starting ovs-vswitchd
* Enabling remote OVSDB managers
* /etc/openvswitch/conf.db does not exist
* Creating empty database /etc/openvswitch/conf.db
* Starting ovsdb-server
* Configuring Open vSwitch system IDs
* Starting ovs-vswitchd
* Enabling remote OVSDB managers
nohup: ignoring input and appending output to 'nohup.out'
nohup: ignoring input and appending output to 'nohup.out'
If you do a docker ps at this point you should see:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a008fefec3bc edgevpnio/evio-node-mininet:latest "/bin/bash" 3 minutes ago Up 3 minutes mn.d2
a93fcc69b2ba edgevpnio/evio-node-mininet:latest "/bin/bash" 3 minutes ago Up 3 minutes mn.d1
37c64df491c1 edgevpnio/openfire_edgevpn_demo "/sbin/my_init" 5 minutes ago Up 5 minutes 5000-5221/tcp, 0.0.0.0:5222-5223->5222-5223/tcp, 5224-5228/tcp, 0.0.0.0:5229->5229/tcp, 5230-5268/tcp, 0.0.0.0:5269->5269/tcp, 5270-5274/tcp, 0.0.0.0:5275->5275/tcp, 0.0.0.0:7070->7070/tcp, 0.0.0.0:7443->7443/tcp, 0.0.0.0:7777->7777/tcp, 5276-6000/tcp, 9091/tcp, 5000-6000/udp, 0.0.0.0:9090->9090/tcp openfire
Now you should be able to log in to a container, and ping it over the Evio interface (example below assumes you configured Y.Y.Y.Y as 192.168.10.1 for mn.d1)
docker exec -it mn.d1 /bin/bash
ping 192.168.10.2
Here's an example output from the console within mn.d1 - note that packets sent through the Evio patch bridge brl1234567 (192.168.10.1) are tunneled through the mininet interface d1-eth0 (10.0.0.251), and that the emulated round-trip time is around 200ms:
ubuntu@ip-172-31-1-225:~$ docker exec -it mn.d1 /bin/bash
root@d1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether ee:5c:7f:9c:a5:7c brd ff:ff:ff:ff:ff:ff
3: edgbr1234567: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1410 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether a2:ec:bc:1f:e8:4e brd ff:ff:ff:ff:ff:ff
4: brl1234567: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1410 qdisc noqueue state UNKNOWN group default qlen 1000
link/ether b6:f1:a5:19:93:47 brd ff:ff:ff:ff:ff:ff
inet 192.168.10.1/24 scope global brl1234567
valid_lft forever preferred_lft forever
5: tnl-a26447b: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether d6:32:22:82:62:c1 brd ff:ff:ff:ff:ff:ff
19: eth0@if20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:ac:11:00:03 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 172.17.0.3/16 brd 172.17.255.255 scope global eth0
valid_lft forever preferred_lft forever
24: d1-eth0@if23: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 02:e5:38:3e:6a:37 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.0.0.251/8 scope global d1-eth0
valid_lft forever preferred_lft forever
root@d1:~# ping 192.168.10.2
PING 192.168.10.2 (192.168.10.2) 56(84) bytes of data.
64 bytes from 192.168.10.2: icmp_seq=1 ttl=64 time=420 ms
64 bytes from 192.168.10.2: icmp_seq=2 ttl=64 time=200 ms
64 bytes from 192.168.10.2: icmp_seq=3 ttl=64 time=200 ms
^C
--- 192.168.10.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 200.709/274.137/420.985/103.838 ms
Notes
This is the Dockerfile that has been used to create the evio-node-mininet image:
FROM evio-base:latest
WORKDIR /root/
COPY ./edge-vpnio_20.7_amd64.deb .
RUN apt-get install -y ./edge-vpnio_20.7_amd64.deb && rm -rf /var/lib/apt/lists/* && apt-get autoclean
RUN apt-get update && apt-get install -y \
net-tools \
iputils-ping \
iproute2
CMD /bin/bash