Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

bond of vlan interfaces


mcrichie
3 Posts
May 3, 2021, 6:30 am

Hello.

Hello.

I'm trying to set up a active-backup bond on top of vlan interfaces with PetaSAN for Ceph private/public networks.
The idea is to have a bonding setup that gives us better performance and redundancy without using MLAG or any other method to have inter-switch LACP.
I tried some configs but failed to get them to work.
Is a setup like this possible with PetaSAN?
Here some configs...I hope it illustrates what I mean 🙂

cheers
Richard

---------------------------------------------
| eth0.100 & eth1.100 => bond0 => primary (eth0) on switch1 (ceph private)
| eth0.101 & eth1.101 => bond1 => primary (eth1) on switch2 (ceph public)

---------------------------------------------

{
"backend_1_base_ip": "172.30.0.0",
"backend_1_eth_name": "bond0",
"backend_1_mask": "255.255.0.0",
"backend_1_vlan_id": "",
"backend_2_base_ip": ""172.29.0.0",",
"backend_2_eth_name": "bond1",
"backend_2_mask": ""255.255.0.0",",
"backend_2_vlan_id": "",
"bonds": [
{
"interfaces":"eth1.100,eth2.100",
"is_jumbo_frames": true,
"mode": "active-backup",
"name": "bond0",
"primary_interface": "eth0.100"
},
{
"interfaces":"eth1.101,eth2.101",
"is_jumbo_frames": true,
"mode": "active-backup",
"name": "bond0",
"primary_interface": "eth1.101"
},
],
"default_pool": "both",
"default_pool_pgs": "1024",
"default_pool_replicas": "3",
"eth_count": 6,
"jf_mtu_size": "",
"jumbo_frames": [],
"management_eth_name": "eth0",
"management_nodes": [
{
"backend_1_ip": "172.30.1.1",
"backend_2_ip": "172.29.1.1",
"is_backup": false,
"is_cifs": true,
"is_iscsi": false,
"is_management": true,
"is_nfs": false,
"is_storage": true,
"management_ip": "192.168.1.1",
"name": "node1"
},
{
"backend_1_ip": "172.30.1.2",
"backend_2_ip": "172.29.1.2",
"is_backup": false,
"is_cifs": true,
"is_iscsi": false,
"is_management": true,
"is_nfs": false,
"is_storage": true,
"management_ip": "192.168.1.2",
"name": "node2"
},
{
"backend_1_ip": "172.30.1.3",
"backend_2_ip": "172.29.1.3",
"is_backup": false,
"is_cifs": true,
"is_iscsi": false,
"is_management": true,
"is_nfs": false,
"is_storage": true,
"management_ip": "192.168.1.3",
"name": "node3"
},
],
"name": "petasancluster",
"storage_engine": "bluestore",
"writecache_writeback_inflight_mb": ""
}

#1

admin
2,930 Posts
May 3, 2021, 1:37 pm

i believe instead of

"backend_1_eth_name": "bond0",
"backend_1_vlan_id": "",

"bonds": [
{
"interfaces":"eth1.100,eth2.100",
"is_jumbo_frames": true,
"mode": "active-backup",
"name": "bond0",
"primary_interface": "eth0.100"
},

 

should be more
"backend_1_eth_name": "bond0",
"backend_1_vlan_id": "100",

"bonds": [
{
"interfaces":"eth1,eth2",
"is_jumbo_frames": true,
"mode": "active-backup",
"name": "bond0",
"primary_interface": "eth0"
},

best just create a dummy vm your desktop, put the correct settings that matches your cluster, then grab its cluster_info.json

 

#2

mcrichie
3 Posts
May 3, 2021, 3:32 pm

Hello,
thanks for the quick reply.

I already thought that my question might be confusing 😉

Indeed we want to use the vlans inside the bond in an active-backup configuration and not for the bond it self.
This enables the possibility to use both links of a redundant switch configuration for different networks.
As far as we could test, at the moment PetaSAN setup doesn't allow using vlan interfaces as members of a bond.
Is there some way to get it to work anyways?

thanks in advance
Richard

#3

admin
2,930 Posts
May 3, 2021, 10:13 pm

PetaSAN creates vlans on top of the bond.

Example eth1/eth2 interfaces are bonded to create bond0. If you have 2 vlans 100, 101 these will be created over the bond0 ie bond0.100 and bond0.101

This is the common way to use vlans with bonds, i am not sure if adding vlans to the lower interfaces rather than the bond is possible or not.

#4

admin
2,930 Posts
May 3, 2021, 10:58 pm

in addition, if you wish you can do any custom network setup in script

/opt/petasan/scripts/custom/post_start_network.sh

#5

mcrichie
3 Posts
May 24, 2021, 8:05 am

Using VLAN interfaces as members of a bond in an active-backup config is possible.
As far a I know it is the only way to use all interfaceses in a dual switch config that doesn't use MLAG.
In my case I used VLAN interfaces for the ceph cluster_network und regular interfaces for the public network.

------------------------------------------------------------

So setup looks like this:

eth1 & eth2 => bond0 => primary (eth1) on switch1 (ceph public)
eth1.101 & eth2.101 => bond1 => primary (eth2.101) on switch2 (ceph private)

------------------------------------------------------------

purposed cluster config would look like that:
/opt/petasan/config/cluster_info.json:

{
"backend_1_base_ip": "172.30.0.0",
"backend_1_eth_name": "bond0",
"backend_1_mask": "255.255.0.0",
"backend_1_vlan_id": "",
"backend_2_base_ip": ""172.29.0.0",",
"backend_2_eth_name": "bond1",
"backend_2_mask": ""255.255.0.0",",
"backend_2_vlan_id": "",
"bonds": [
{
"interfaces":"eth1,eth2",
"is_jumbo_frames": true,
"mode": "active-backup",
"name": "bond0",
"primary_interface": "eth1"
},
{
"interfaces":"eth1.101,eth2.101",
"is_jumbo_frames": true,
"mode": "active-backup",
"name": "bond1",
"primary_interface": "eth2.101"
},
],
"default_pool": "both",
"default_pool_pgs": "1024",
"default_pool_replicas": "3",
"eth_count": 6,
"jf_mtu_size": "",
"jumbo_frames": [],
"management_eth_name": "eth0",
"management_nodes": [
{
"backend_1_ip": "172.30.1.1",
"backend_2_ip": "172.29.1.1",
"is_backup": false,
"is_cifs": true,
"is_iscsi": false,
"is_management": true,
"is_nfs": false,
"is_storage": true,
"management_ip": "192.168.1.1",
"name": "node1"
},
...

------------------------------------------------------------

But this does not work unfortunately.
So I used the regular interfaces at the cluster config, tweaked in the required interface config to "/etc/network/interfaces" and created a "post_start_network" script to activate it.
This works for now.
However is there a more proper way to do this?

------------------------------------------------------------
/opt/petasan/config/cluster_info.json:
{
"backend_1_base_ip": "172.30.11.0",
"backend_1_eth_name": "eth1",
"backend_1_mask": "255.255.0.0",
"backend_1_vlan_id": "",
"backend_2_base_ip": "172.29.0.0",
"backend_2_eth_name": "eth2",
"backend_2_mask": "255.255.0.0",
"backend_2_vlan_id": "",
"bonds": [],
"default_pool": "both",
"default_pool_pgs": "1024",
"default_pool_replicas": "3",
"eth_count": 4,
"jf_mtu_size": "",
"jumbo_frames": [],
"management_eth_name": "eth0",
"management_nodes": [
{
"backend_1_ip": "172.30.1.1",
"backend_2_ip": "172.29.1.1",
"is_backup": false,
"is_cifs": true,
"is_iscsi": false,
"is_management": true,
"is_nfs": false,
"is_storage": true,
"management_ip": "192.168.10.1",
"name": "leg1"
},
...

------------------------------------------------------------
/etc/network/interfaces:

auto eth0
iface eth0 inet static
address 192.168.10.1
netmask 255.255.255.0
gateway 192.168.10.254
dns-nameservers 192.168.10.3

auto eth1
iface eth1 inet manual
bond-master bond0
bond-mode active-backup
bond-primary eth1

auto eth2
iface eth2 inet manual
bond-master bond0
bond-mode active-backup
bond-primary eth2

auto eth1.101
iface eth1.101 inet manual
bond-master bond1
bond-mode active-backup
bond-primary eth2.101

auto eth2.101
iface eth2.101 inet manual
bond-master bond1
bond-mode active-backup
bond-primary eth2.101

auto bond0
iface bond0 inet static
address 172.30.1.1
netmask 255.255.0.0
bond-mode active-backup
bond-miimon 100
bond-updelay 200
bond-downdelay 200
bond-slaves none

auto bond1
iface bond1 inet static
address 172.29.1.1
netmask 255.255.0.0
bond-mode active-backup
bond-miimon 100
bond-updelay 200
bond-downdelay 200
bond-slaves none

------------------------------------------------------------

/opt/petasan/scripts/custom/post_start_network.sh:

ip link set eth1 down
ifdown eth1
ip link set eth2 down
ifdown eth2
ifdown bond0
ifdown bond1

ifup eth1
ifup eth2
#ifup bond0
#ifup bond1

sleep 2

ip link set eth1 down
ifdown eth1
ip link set eth2 down
ifdown eth2
ifdown bond0
ifdown bond1

ifup eth1
ifup eth2

#6