Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

Yet another nic design request


f.cuseo
65 Posts
October 23, 2023, 8:40 pm

Hello.

We are installing a cluster with new servers that will replace an old one.

Old (3) servers are with 4 nic, ethernet for management and bond with vlan for backend and services.

New servers, have 4 x 10gbit, and we will use a single lacp bond with vlan for every service; we also would like to replace the old servers with "node replacement", with no need to destroy the cluster and create downtime.

But we can't customize network for each node, because all the node clusters will use the same network configuration (both number and names).
Can you add "per node" network configuration that should include ONLY the physical nics that belongs to bond.
So, cluster will have the same bond, same vlans, but a "per node" "bond slaves nics". It will be really useful when cluster will grow with different hardware.

PS: also software-raid for OS will be really useful, because for ceph i need to configure my raid controller in IT-Mode, so no raid is possibile.

thanks, Fabrizio

#1

admin
2,921 Posts
October 23, 2023, 11:27 pm

Can you give more detail or example of what limitation you see in network configuration.

#2

f.cuseo
65 Posts
October 24, 2023, 5:22 am

Thank you for your answer.

Yes; first 3 servers, with 4 x giga ethernet but only 2 used; is an ALB bond with several vlan (one is management, one is backend, last is s3 gateway).
New servers (I need to add 7 servers and replace first 3 servers) have 4 x 10 giga ethernet (and 4 x giga that i don't want to use), and i would like to setup with a single 4 x 10Gbit bond  with jumbo frames (802.3ad LACP) with same vlan configuration.
So, I can keep for the whole cluster the same "vlan over bond" configuration, but i want to customize the bond for every single server, so I can have it with 4x10Gbit in new servers with LACP, 2 x giga in the old servers with alb (of course, only for some time so i can replace them).

 

{
"backend_1_base_ip": "172.16.34.0",
"backend_1_eth_name": "bond0",
"backend_1_mask": "255.255.255.0",
"backend_1_vlan_id": "904",
"backend_2_base_ip": "",
"backend_2_eth_name": "",
"backend_2_mask": "",
"backend_2_vlan_id": "",
"bonds": [
{
"interfaces": "eth0,eth1",
"is_jumbo_frames": false,
"mode": "balance-alb",
"name": "bond0",
"primary_interface": "eth0"

 

 

"management_eth_name": "bond0",
"management_nodes": [
{
"backend_1_ip": "172.16.34.1",
"backend_2_ip": "",
"is_backup": false,
"is_cifs": false,
"is_iscsi": false,
"is_management": true,
"is_nfs": false,
"is_s3": true,
"is_storage": true,
"management_ip": "172.16.33.1",
"name": "petasan1",
"petasan_version": ""
},

 

It should be enought if in this configuration part I could insert

"bond_slaves" :" ethx/y/z/",
"bond_mode" : "802.3ad",
"bond_mtu" : "9000",

Thank you, Fabrizio

 

#3

f.cuseo
65 Posts
October 24, 2023, 7:54 am

I don't know if is enough to modify this line (72) of  /opt/petasan/scripts/node_start_ips.py :

for eth in bond.interfaces.split(','):

with:

for eth in bond.nodeXX_interfaces.split(','):

of course, reading bond interfaces for nodeXX, or having a "per node" node_interfaces configuration entry.

 

 

#4

admin
2,921 Posts
October 27, 2023, 2:00 pm

You can customize network per node in different ways:

You can rename/re-order interfaces using the blue menu on each node.

You can manually edit the file
/opt/petasan/config/cluster_info.json

You can customize your own network configuration (add/remove bonds, ips..)
/opt/petasan/scripts/custom/post_start_network.sh

 
For node replacement:
In installer, you need to set the same hostname, management ip and mamagement interface as old node
In deployment wizard, the system will double check the above as well double check you have the additional interface defined for backend as well as interfaces defined for the different roles the old node was assigned. New node does not have to have the sample number of interfaces as old node.

Last edited on October 27, 2023, 2:01 pm by admin · #5