Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

Servers with Two NICs.

Pages: 1 2 3

msalem
87 Posts
May 25, 2018, 8:06 pm

So how would you suggest building it with two NICs,

I would rather having one iscsi and one backend if thats the case, however the UI forces you to use two of each !

#11

admin
2,930 Posts
May 25, 2018, 8:11 pm

management + iscsi 1  + backend 1  on nic 1

iscsi 2 + backend 2 on nic 2

#12

msalem
87 Posts
May 25, 2018, 8:23 pm

Hello admin,

The issue is without vlan adding in the config,  I cannot even access the server after installing,  I figure its because the nic is missing the vlan tag.

With that said how does it know which network to you without the vlan tag?

#13

admin
2,930 Posts
May 25, 2018, 9:04 pm

You can assign multiple ips/subnets to a nic. the networking layer chooses the correct nic based on this alone.  Vlan tagging does provide broadcast isolation between different subnets, which is a nice to have but not essential for PetaSAN since it is a single application. If you cannot access the server this means your client machine nic is using vlan tagging, if you disable this (ie use the default vlan ) + any switches in between, things will work.

Last edited on May 25, 2018, 9:05 pm by admin · #14

msalem
87 Posts
May 25, 2018, 10:02 pm

Maybe thats the case however,  when we setup the server we only have one ip,  and we cannot access the URL, or even SSH,  with the client PC or another server on the same network.

We will need to troubleshoot this more,  I would assume your comment above will do best if everything worked.

I will test that in our POC lab and get back to you

Thanks

#15

msalem
87 Posts
May 26, 2018, 3:04 am

Hello Admin,

I have tested your idea just now, and lets assume the following: the server has only 2x10Gb nics, on the first I have 3 Vlans, as you stated:

1 - Mgmt Vlan 10 - 10.1.1.0/24

2 - ISCSI vlan 20 - 10.2.2.0/24

3 - Backend vlan 30 - 10.3.3.0/24

while installed the server, I selected the first nic (eth0) as vlan 10 to the server, the installation goes well, however the server is not accessible via any port.

After troubleshooting, I logged into the server via shell, and ran the following commands, to add vlan 20 and 30 and Add the IP's to the server. I used IP command , since vconfig and vlan are not installed by default.

# ip link add link eth0 name eth0.20 type vlan id 20
# ip link
# ip -d link show eth0.20

# ip link add link eth0 name eth0.30 type vlan id 30
# ip link
# ip -d link show eth0.30
# ip addr add 10.2.2.10/24 brd 10.2.2.255 dev eth0.20
# ip link set dev eth0.20 up

# ip addr add 10.3.3.10/24 brd 10.3.3.255 dev eth0.30
# ip link set dev eth0.30 up

and I was able to ping and access SSH, however I do not know the default root password. since Admin users is not set yet. with that said, there must be an option, to atleast have a vlan setup for the mgmt network or running multiple networks under one NIC.

the port on the switch is set into trunk mode, vlan1 is not part of the default setup, for security reasons and will have a very big STP topology which can result in problems during STP convergence.

an example will be this port:

!
interface Ethernet10
   description Server: MASS STORAGE PETASAN
   switchport trunk allowed vlan 10,20,30,80
   switchport mode trunk
!

Hope this clears.

Thanks

Last edited on May 26, 2018, 3:09 am by msalem · #16

admin
2,930 Posts
May 26, 2018, 1:10 pm

To summarize what works in PetaSAN out of the box: You need 2-5 nics /bonds, the min is 2. If you use 2 you will need to map the 5 subnets to these 2 nics/bonds. You do not need vlan trunking for this to work whether you use 2 or 5 nics/bonds, in fact if you setup vlan trunking on your switches or hypervisors, it will not work out of the box.

So your options are:

  • Do not use port trunking on your switches, treat PetaSAN as your storage layer with its dedicated network not shared with others.
  • Set vlan yourself manually, this my involve changing some of the PetaSAN scripts.

I would recommend the first options, i cannot help too much with the second. I know vlan is a feature we have in our support list but not sure of timing.

Note: prior to node deployment you can set the initial root password by using the bash shell from the node console menu (the blue node screen), then use the passwd command., then you can use ssh. This will be over-ridden after deployment with the cluster password.

Last edited on May 26, 2018, 1:22 pm by admin · #17

msalem
87 Posts
May 26, 2018, 9:35 pm

Thanks for the reply,

With the two options you have provided, I understand the trunking needs further setup on the Vmware side that need to add the vlan ID to the DVS or Standard switches. the switches are shared with other solutions, for use to deploy a dedicated 10/40GB switch is not in our plans.

 

So with the options you have provided.

1 - If we make the port in access-mode will it work, without adding vlan id's to the configs ?

2 - what are the scripts that we need to edit to review to take a better look at it - in case option one fails ?

 

Thanks

#18

admin
2,930 Posts
May 28, 2018, 12:11 pm

on your ESXi hypervisors you need 2 nics dedicated to iSCSI for connection to PetaSAN,  you should assign a static ip on each within the SCSI 1 and iSCSI  subnets, they should be within the subnet but outside the auto-assign range allocated for the target paths, you will leave the vlan as default (untagged) on them. we have a document which goes through this that  you can download. Leave your switch in access mode / untagged, everything will work out of the box.

Last edited on May 28, 2018, 12:35 pm by admin · #19

msalem
87 Posts
May 28, 2018, 3:56 pm

Hello Admin,

 

our ESXI servers are in production, so even just to add two new Dedicated 10GB ports on the server wont cut it. we use vlan tagging on Esxi since it supports it without any issues. we will be working on PetaSan and try to get it running on 3 NICs,

1 - Management.

2 - ISCSI1+Back1

3 - ISCSI2+Back2.

I will keep you posted.

 

#20
Pages: 1 2 3