Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

NIC mapping is changed after reboot

Pages: 1 2

netatvision
10 Posts
April 29, 2019, 4:42 pm

Hi,

on on of our nodes the nic mapping has changed after reboot. The only thing we have done, we have increased the ram from 64gb to 128gb. We´ve done this on all nodes, only one node has lost his mapping.

failed mapping:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond-mgmt state UP group default qlen 1000
link/ether 1c:98:ec:29:dd:04 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond-mgmt state UP group default qlen 1000
link/ether 1c:98:ec:29:dd:04 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 1c:98:ec:29:dd:06 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq portid fc15b413af60 state UP group default qlen 1000                      ###### this must be eth4
link/ether fc:15:b4:13:af:60 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fe15:b4ff:fe13:af60/64 scope link
valid_lft forever preferred_lft forever
6: eth4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 9000 qdisc mq state DOWN group default qlen 1000                                                ###### this must be eth3
link/ether 1c:98:ec:29:dd:07 brd ff:ff:ff:ff:ff:ff
inet xx.xx.xx.100/24 scope global eth4
valid_lft forever preferred_lft forever
7: eth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq portid fc15b413af64 state UP group default qlen 1000
link/ether fc:15:b4:13:af:64 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fe15:b4ff:fe13:af64/64 scope link
valid_lft forever preferred_lft forever

 

correct mapping from another node:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond-mgmt state UP group default qlen 1000
link/ether 3c:a8:2a:15:91:b8 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond-mgmt state UP group default qlen 1000
link/ether 3c:a8:2a:15:91:b8 brd ff:ff:ff:ff:ff:ff
4: eth2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 3c:a8:2a:15:91:ba brd ff:ff:ff:ff:ff:ff
5: eth3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000                                                   ####### correct
link/ether 3c:a8:2a:15:91:bb brd ff:ff:ff:ff:ff:ff
6: eth4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq portid 1458d050f9c0 state UP group default qlen 1000                      #######  correct
link/ether 14:58:d0:50:f9:c0 brd ff:ff:ff:ff:ff:ff
inet6 fe80::1658:d0ff:fe50:f9c0/64 scope link
valid_lft forever preferred_lft forever
7: eth5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq portid 1458d050f9c4 state UP group default qlen 1000
link/ether 14:58:d0:50:f9:c4 brd ff:ff:ff:ff:ff:ff
inet xx.xx.xx.100/24 scope global eth5
valid_lft forever preferred_lft forever
inet6 fe80::1658:d0ff:fe50:f9c4/64 scope link
valid_lft forever preferred_lft forever

 

Eth4 and eth5 are the iscsi nics.

 

please can you help us.

Regards, Carsten

 

 

#1

admin
2,930 Posts
April 29, 2019, 6:22 pm

You can set/force mapping using the blue node console menu.  not sure what made it change.

 

#2

netatvision
10 Posts
April 30, 2019, 3:27 am

Thx for the advice. I´ve used this feature in the past and I´ve had a problem, that some of my bonds, I also use, were destroyed. Can you give me some background information about what the feature will do? Does the feature creat a udev rule /etc/udev/rules.d/70-persistent-net.rules?

Regards, Carsten

#3

admin
2,930 Posts
April 30, 2019, 8:07 am

yes it reads/writes  /etc/udev/rules.d/70-persistent-net.rules   i  recommend you use it, then double check the file  and change/correct as needed. If you do find any issues let us know.

it should not affect your bonds, these rules occur at early stage in the boot phase while interfaces are detected, all they do is assign the nic name via its mac address. bonds are assigned much later by the PetaSAN services when thety start.

not sure what caused your initial problem on this node, my guess is some interface/port did not function after the ram upgrade, maybe it was unseated. the udev file will prevent mapping issues in case this occurs.

 

Last edited on April 30, 2019, 8:07 am by admin · #4

netatvision
10 Posts
May 1, 2019, 1:20 pm

Hello,

after making the change we´ve lost the node now. The management bond is destroyed.

https://wetransfer.com/downloads/1cd301b3bb7d868ab5e6064891c7f94f20190501131757/417dbc8782dadb308a2d458e84162c0f20190501131757/b15a53/grid

https://wetransfer.com/downloads/1cd301b3bb7d868ab5e6064891c7f94f20190501131757/417dbc8782dadb308a2d458e84162c0f20190501131757/b15a53/grid

Please, can you help.

#5

netatvision
10 Posts
May 1, 2019, 1:41 pm

I could get acces to the node. I´ve renamed the 70-persistent-net.rules. Now my node is up again and also the old, original nic mapping comes back. it´s very, very mysterios. We are using a qualified hw. It´s an HP DL360Gen9, with only HP components.

My pulse comes back to normal rate now....

#6

admin
2,930 Posts
May 1, 2019, 2:00 pm

i would try to find out if the content of 70-persistent-net.rules makes sense and if so what mapping to you actually get if you reboot. You may need to be physically working on the node via console in case you "loose" the node.  You can either open the interface naming menu to check this, you can also run the script:

 /opt/petasan/scripts/detect-interfaces.sh

which will give names/mac/pci

we need to know if either the udev rule is wrong or is not honored after reboot

Also i am not sure i understand, but on this node do you see some nics missing  from the menu ? are they also missing from the output of above script ? do you see any dmesg kernel errors ? If nics are missing it could be hardware related, i do not believe bonding configuration is related here as it happen later, if the kernel cannot detect the nic it ishardware/firmware issue.

Last edited on May 1, 2019, 2:12 pm by admin · #7

netatvision
10 Posts
May 7, 2019, 4:22 am

Have you got my email? I send it to contact-us@petasan.org.

 

#8

admin
2,930 Posts
May 7, 2019, 7:26 am

no, please resend.

#9

admin
2,930 Posts
May 7, 2019, 6:43 pm

nothing received.

#10
Pages: 1 2