Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

Mac address, bonding, vlan and v4

Pages: 1 2

f.cuseo
84 Posts
March 31, 2025, 4:45 pm

Hello.
I have a 5 nodes homelab cluster upgraded from 3.3.0 to 4.0

Now, all the 5 nodes have the same mac address on eth0, eth1, bond and vlans. So, with same mac address on all 5 nodes, it can't work. Where is taken this mac address ?

The config is:
- on /etc/network/interfaces, eth0.201 is the only interface (with management address)
- cluster_info has a 802.3ad bond with eth0 and eth1, management is on bond0.201, ceph on bond0.809, and s3 on another bond vlan

PS: for an error, i have upgraded first nodes 4 and 5 (storage servers), and to complete 3.3.0 to 4.0, i have modified the upgrade script to ignore the ceph health status. Now all is on the 4.0 release

 

#1

admin
2,969 Posts
April 1, 2025, 8:53 pm

the mac address is read from interface hardware, this is very starnge.

on any node with issue, can you post output of:

/opt/petasan/scripts/detect-interfaces.sh
ip addr

and post files:

/opt/petasan/config/cluster_info.json
/etc/network/interfaces

#2

f.cuseo
84 Posts
April 1, 2025, 9:13 pm

Same on all 5 nodes (the mac address)

 

/opt/petasan/scripts/detect-interfaces.sh

root@petasan-01:~# /opt/petasan/scripts/detect-interfaces.sh
device=eth0,mac=08:9e:01:6c:63:33,pci=02:00.0,model=Broadcom Inc. and subsidiaries NetXtreme II BCM5716 Gigabit Ethernet,path=enp2s0f0
device=eth1,mac=08:9e:01:6c:63:34,pci=02:00.1,model=Broadcom Inc. and subsidiaries NetXtreme II BCM5716 Gigabit Ethernet,path=enp2s0f1

ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 62:37:b3:63:bd:12 brd ff:ff:ff:ff:ff:ff permaddr 08:9e:01:6c:63:33
altname enp2s0f0
altname eno1
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 62:37:b3:63:bd:12 brd ff:ff:ff:ff:ff:ff permaddr 08:9e:01:6c:63:34
altname enp2s0f1
altname eno2
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 62:37:b3:63:bd:12 brd ff:ff:ff:ff:ff:ff
inet6 fe80::6037:b3ff:fe63:bd12/64 scope link
valid_lft forever preferred_lft forever
6: bond0.201@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 62:37:b3:63:bd:12 brd ff:ff:ff:ff:ff:ff
inet 10.172.1.60/24 scope global bond0.201
valid_lft forever preferred_lft forever
inet6 fe80::6037:b3ff:fe63:bd12/64 scope link
valid_lft forever preferred_lft forever
7: bond0.908@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 62:37:b3:63:bd:12 brd ff:ff:ff:ff:ff:ff
inet 10.172.40.60/24 scope global bond0.908
valid_lft forever preferred_lft forever
inet6 fe80::6037:b3ff:fe63:bd12/64 scope link
valid_lft forever preferred_lft forever

and post files:

/opt/petasan/config/cluster_info.json

cat /opt/petasan/config/cluster_info.json
{
"backend_1_base_ip": "10.172.40.0",
"backend_1_eth_name": "bond0",
"backend_1_mask": "255.255.255.0",
"backend_1_vlan_id": "908",
"backend_2_base_ip": "",
"backend_2_eth_name": "",
"backend_2_mask": "",
"backend_2_vlan_id": "",
"bonds": [
{
"interfaces": "eth0,eth1",
"is_jumbo_frames": false,
"mode": "802.3ad",
"name": "bond0",
"primary_interface": ""
}
],
"default_pool": [
"cephfs",
"radosgw",
"rbd"
],
"default_pool_pgs": "1024",
"default_pool_replicas": "3",
"eth_count": 2,
"jf_mtu_size": "",
"jumbo_frames": [],
"management_eth_name": "bond0",
"management_nodes": [
{
"backend_1_ip": "10.172.40.60",
"backend_2_ip": "",
"is_backup": false,
"is_cifs": false,
"is_iscsi": false,
"is_management": true,
"is_nfs": false,
"is_s3": false,
"is_storage": true,
"management_ip": "10.172.1.60",
"name": "petasan-01",
"petasan_version": ""
},
{
"backend_1_ip": "10.172.40.61",
"backend_2_ip": "",
"is_backup": false,
"is_cifs": false,
"is_iscsi": false,
"is_management": true,
"is_nfs": false,
"is_s3": false,
"is_storage": true,
"management_ip": "10.172.1.61",
"name": "petasan-02",
"petasan_version": ""
},
{
"backend_1_ip": "10.172.40.62",
"backend_2_ip": "",
"is_backup": false,
"is_cifs": false,
"is_iscsi": false,
"is_management": true,
"is_nfs": false,
"is_s3": false,
"is_storage": true,
"management_ip": "10.172.1.62",
"name": "petasan-03",
"petasan_version": "3.2.1"
}
],
"name": "ClusterSLIM",
"storage_engine": "bluestore",
"writecache_writeback_inflight_mb": ""

/etc/network/interfaces

cat /etc/network/interfaces
auto eth0.201
iface eth0.201 inet static
address 10.172.1.60
netmask 255.255.255.0
gateway 10.172.1.254
dns-nameservers 8.8.8.8

#3

admin
2,969 Posts
April 2, 2025, 9:19 pm

Can you run an online update:

/opt/petasan/scripts/online-updates/update.sh

then reboot and see if this fixes the issue.

#4

f.cuseo
84 Posts
April 3, 2025, 8:54 am
Quote from admin on April 2, 2025, 9:19 pm

Can you run an online update:

/opt/petasan/scripts/online-updates/update.sh

then reboot and see if this fixes the issue.

Tested on the first 2 nodes... same problem, same mac-address for both nodes on eth0/1 bond and bond.xxx

#5

admin
2,969 Posts
April 3, 2025, 11:30 pm

on each node, can you try

rm -f /etc/machine-id
systemd-machine-id-setup
sync

then reboot

#6

f.cuseo
84 Posts
April 4, 2025, 4:54 am
Quote from admin on April 3, 2025, 11:30 pm

on each node, can you try

rm -f /etc/machine-id
systemd-machine-id-setup
sync

then reboot

Same problem.

But is strange because machine-id is the same on all the nodes, and if I create again with your commands, is the same again,

 

#7

f.cuseo
84 Posts
April 4, 2025, 4:59 am
Quote from admin on April 3, 2025, 11:30 pm

on each node, can you try

rm -f /etc/machine-id
systemd-machine-id-setup
sync

then reboot

I am trying with:

rm -f /etc/machine-id
dbus-uuidgen --ensure=/etc/machine-id
rm /var/lib/dbus/machine-id
dbus-uuidgen --ensure /var/lib/dbus/machine-id
#8

f.cuseo
84 Posts
April 4, 2025, 5:11 am

Yes, fixed !
Thanks for your info...

#9

admin
2,969 Posts
April 4, 2025, 8:37 am

Excellent 🙂

Any reason you used the dbus command over the systemd ?

#10
Pages: 1 2