PetaSAN 2.8.0 installation not recognizing all network adapters
dh0253
15 Posts
September 25, 2021, 4:30 pmQuote from dh0253 on September 25, 2021, 4:30 pmDon't if it's actually a bug. Hi, I've been testing with 2.7.1, but now I want to try 2.8.0 because of object storage support. With version 2.7.1, the installation recognized all my adapters (2-HPE/SolarFlare SFN7322F and 2-HPE/Mellanox MT27520 ConnectX-3 Pro (MCX354A-FCCT)). However, with the 2.8.0 installer, the installation only recognized 1-Solarflare and 1-Mellanox. I did not move the adapters to another slot or upgraded the server BIOS. The only thing I did was removing my SanDisk 128GB Endurance SD card and format it with the SD Formatter.
As a curious mind wants to know, I decided to re-install 2.7.2, and wouldn't you know it? It recognized all four adapters. I will still install 2.8.0. I went to the bash shell and ran: lspci --vv | grep Ethernet
All four network interfaces are there.
03:00.0 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 570FLR-SFP+ Adapter (R1)
Product Name: HP Ethernet 10Gb 2-port 570FLR-SFP+ Adapter
03:00.1 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 570FLR-SFP+ Adapter (R1)
Product Name: HP Ethernet 10Gb 2-port 570FLR-SFP+ Adapter
Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Product Name: HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Product Name: HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Is there any recommended or "approved" way to turn up the other adapters?
Thank you
Don't if it's actually a bug. Hi, I've been testing with 2.7.1, but now I want to try 2.8.0 because of object storage support. With version 2.7.1, the installation recognized all my adapters (2-HPE/SolarFlare SFN7322F and 2-HPE/Mellanox MT27520 ConnectX-3 Pro (MCX354A-FCCT)). However, with the 2.8.0 installer, the installation only recognized 1-Solarflare and 1-Mellanox. I did not move the adapters to another slot or upgraded the server BIOS. The only thing I did was removing my SanDisk 128GB Endurance SD card and format it with the SD Formatter.
As a curious mind wants to know, I decided to re-install 2.7.2, and wouldn't you know it? It recognized all four adapters. I will still install 2.8.0. I went to the bash shell and ran: lspci --vv | grep Ethernet
All four network interfaces are there.
03:00.0 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 570FLR-SFP+ Adapter (R1)
Product Name: HP Ethernet 10Gb 2-port 570FLR-SFP+ Adapter
03:00.1 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 570FLR-SFP+ Adapter (R1)
Product Name: HP Ethernet 10Gb 2-port 570FLR-SFP+ Adapter
Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Product Name: HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Product Name: HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Is there any recommended or "approved" way to turn up the other adapters?
Thank you
dh0253
15 Posts
September 28, 2021, 12:22 pmQuote from dh0253 on September 28, 2021, 12:22 pmOf the three HPE DL380P G8 servers I am testing, two have dual riser cage (one for each CPU). The third server only has one riser cage, and all the cards are plugged into this riser. Once we purchased an additional riser for CPU2 and moved one of the Mellanox to the new riser, the installer was able to find all the adapters. For production, I recommend matching the server's equipment as much as possible. On with the testing...
Of the three HPE DL380P G8 servers I am testing, two have dual riser cage (one for each CPU). The third server only has one riser cage, and all the cards are plugged into this riser. Once we purchased an additional riser for CPU2 and moved one of the Mellanox to the new riser, the installer was able to find all the adapters. For production, I recommend matching the server's equipment as much as possible. On with the testing...
Shiori
86 Posts
December 17, 2021, 6:29 pmQuote from Shiori on December 17, 2021, 6:29 pmfirst are you using Infiniband or just as extra ethernet ports?
We use Infiniband and you must add the drivers after upgrade every time.
if using the Melanox card as ethernet then you must add mlx4_en to modules to get both ports. If your cards are compatible, use the mlx5_en module.
first are you using Infiniband or just as extra ethernet ports?
We use Infiniband and you must add the drivers after upgrade every time.
if using the Melanox card as ethernet then you must add mlx4_en to modules to get both ports. If your cards are compatible, use the mlx5_en module.
PetaSAN 2.8.0 installation not recognizing all network adapters
dh0253
15 Posts
Quote from dh0253 on September 25, 2021, 4:30 pmDon't if it's actually a bug. Hi, I've been testing with 2.7.1, but now I want to try 2.8.0 because of object storage support. With version 2.7.1, the installation recognized all my adapters (2-HPE/SolarFlare SFN7322F and 2-HPE/Mellanox MT27520 ConnectX-3 Pro (MCX354A-FCCT)). However, with the 2.8.0 installer, the installation only recognized 1-Solarflare and 1-Mellanox. I did not move the adapters to another slot or upgraded the server BIOS. The only thing I did was removing my SanDisk 128GB Endurance SD card and format it with the SD Formatter.
As a curious mind wants to know, I decided to re-install 2.7.2, and wouldn't you know it? It recognized all four adapters. I will still install 2.8.0. I went to the bash shell and ran: lspci --vv | grep Ethernet
All four network interfaces are there.03:00.0 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 570FLR-SFP+ Adapter (R1)
Product Name: HP Ethernet 10Gb 2-port 570FLR-SFP+ Adapter
03:00.1 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 570FLR-SFP+ Adapter (R1)
Product Name: HP Ethernet 10Gb 2-port 570FLR-SFP+ Adapter
Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Product Name: HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Product Name: HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP AdapterIs there any recommended or "approved" way to turn up the other adapters?
Thank you
Don't if it's actually a bug. Hi, I've been testing with 2.7.1, but now I want to try 2.8.0 because of object storage support. With version 2.7.1, the installation recognized all my adapters (2-HPE/SolarFlare SFN7322F and 2-HPE/Mellanox MT27520 ConnectX-3 Pro (MCX354A-FCCT)). However, with the 2.8.0 installer, the installation only recognized 1-Solarflare and 1-Mellanox. I did not move the adapters to another slot or upgraded the server BIOS. The only thing I did was removing my SanDisk 128GB Endurance SD card and format it with the SD Formatter.
As a curious mind wants to know, I decided to re-install 2.7.2, and wouldn't you know it? It recognized all four adapters. I will still install 2.8.0. I went to the bash shell and ran: lspci --vv | grep Ethernet
All four network interfaces are there.
03:00.0 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 570FLR-SFP+ Adapter (R1)
Product Name: HP Ethernet 10Gb 2-port 570FLR-SFP+ Adapter
03:00.1 Ethernet controller: Solarflare Communications SFC9020 10G Ethernet Controller
Subsystem: Hewlett-Packard Company Ethernet 10Gb 2-port 570FLR-SFP+ Adapter (R1)
Product Name: HP Ethernet 10Gb 2-port 570FLR-SFP+ Adapter
Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Product Name: HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Subsystem: Hewlett-Packard Company InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Product Name: HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter
Is there any recommended or "approved" way to turn up the other adapters?
Thank you
dh0253
15 Posts
Quote from dh0253 on September 28, 2021, 12:22 pmOf the three HPE DL380P G8 servers I am testing, two have dual riser cage (one for each CPU). The third server only has one riser cage, and all the cards are plugged into this riser. Once we purchased an additional riser for CPU2 and moved one of the Mellanox to the new riser, the installer was able to find all the adapters. For production, I recommend matching the server's equipment as much as possible. On with the testing...
Of the three HPE DL380P G8 servers I am testing, two have dual riser cage (one for each CPU). The third server only has one riser cage, and all the cards are plugged into this riser. Once we purchased an additional riser for CPU2 and moved one of the Mellanox to the new riser, the installer was able to find all the adapters. For production, I recommend matching the server's equipment as much as possible. On with the testing...
Shiori
86 Posts
Quote from Shiori on December 17, 2021, 6:29 pmfirst are you using Infiniband or just as extra ethernet ports?
We use Infiniband and you must add the drivers after upgrade every time.
if using the Melanox card as ethernet then you must add mlx4_en to modules to get both ports. If your cards are compatible, use the mlx5_en module.
first are you using Infiniband or just as extra ethernet ports?
We use Infiniband and you must add the drivers after upgrade every time.
if using the Melanox card as ethernet then you must add mlx4_en to modules to get both ports. If your cards are compatible, use the mlx5_en module.