Network vlans
Pages: 1 2
philip.shannon
37 Posts
June 28, 2017, 3:04 pmQuote from philip.shannon on June 28, 2017, 3:04 pmsetting up here, each esxi host only has 4 x 10G ports. Each esxi host needs 2 x iscsi and 1 x vmotion VMkernel connections. This leaves only 1 ten GB port for the vm's which will run petaSAN which makes it important that VLAN tagging can be used. I'm trying to get things working now using this setup, but the petaSAN vm's distributed ports for the 2 iscsi vlans aren't working, must be because of VLAN tagging.
setting up here, each esxi host only has 4 x 10G ports. Each esxi host needs 2 x iscsi and 1 x vmotion VMkernel connections. This leaves only 1 ten GB port for the vm's which will run petaSAN which makes it important that VLAN tagging can be used. I'm trying to get things working now using this setup, but the petaSAN vm's distributed ports for the 2 iscsi vlans aren't working, must be because of VLAN tagging.
admin
2,930 Posts
June 28, 2017, 4:19 pmQuote from admin on June 28, 2017, 4:19 pmIn ESXi there are several tagging modes, VST: Virtual Switch tagging (i believe this is the default) mode the virtual switch inserts the tags from packets coming from VM and strips the tags for packets going into the VM, the VM does not see any tag. In contrast VGT, Virtual Guest tagging mode, it is the responsibility of the VM to deal with VLAN, but from the top of my had VST is the default, so you still can use VLAN within ESXi without configuration in PetaSAN.
In bare metal, you can share different subnets on same nics and bridges without VLANs, the drawback without having VLANs is you have a single broadcast domain, a broadcast within a subnet will be picked by all others, this could be an issue if your iSCSI subnets are exposed to large number of end user machines, but for a backend deployment i think the effect would be minimum.
Of course the best is to have different physical networks for your subnets.
In ESXi there are several tagging modes, VST: Virtual Switch tagging (i believe this is the default) mode the virtual switch inserts the tags from packets coming from VM and strips the tags for packets going into the VM, the VM does not see any tag. In contrast VGT, Virtual Guest tagging mode, it is the responsibility of the VM to deal with VLAN, but from the top of my had VST is the default, so you still can use VLAN within ESXi without configuration in PetaSAN.
In bare metal, you can share different subnets on same nics and bridges without VLANs, the drawback without having VLANs is you have a single broadcast domain, a broadcast within a subnet will be picked by all others, this could be an issue if your iSCSI subnets are exposed to large number of end user machines, but for a backend deployment i think the effect would be minimum.
Of course the best is to have different physical networks for your subnets.
philip.shannon
37 Posts
June 28, 2017, 7:28 pmQuote from philip.shannon on June 28, 2017, 7:28 pmout of the 4 networks (Back Ends & iSCSi's) which two would you put on 1GB vs 10GB? Basically which is better off at 10G the iscsi or backend networks. Obviously we'd like them all at 10G but are facing challenges with the trunk ports/vlan tagging on servers with only 4 10g adapters
out of the 4 networks (Back Ends & iSCSi's) which two would you put on 1GB vs 10GB? Basically which is better off at 10G the iscsi or backend networks. Obviously we'd like them all at 10G but are facing challenges with the trunk ports/vlan tagging on servers with only 4 10g adapters
admin
2,930 Posts
June 28, 2017, 8:36 pmQuote from admin on June 28, 2017, 8:36 pmWhat i can say is that backend 1 will have the combined bandwidth of both iSCSI subnets, during writes backend 2 bandwidth will be either the same as backend 1 in case of replica count=2 or will be double in case of replica count=3.
Maybe you can put each iSCSI subnet on 1G and each backend on 10 G.
Another option is to bond both 10G NICs and map all 4 subnets on this bond.
I wish i have a more concrete recommendation to run PetaSAN hyper-converged, we have started some tests ourselves but they are not done yet...sorry.
I would be interested to know why you need VLAN tagging and not just map all subnets to the same switch/nic un-tagged but if you do need tagging the default VST tagging mode should work with PetaSAN.
What i can say is that backend 1 will have the combined bandwidth of both iSCSI subnets, during writes backend 2 bandwidth will be either the same as backend 1 in case of replica count=2 or will be double in case of replica count=3.
Maybe you can put each iSCSI subnet on 1G and each backend on 10 G.
Another option is to bond both 10G NICs and map all 4 subnets on this bond.
I wish i have a more concrete recommendation to run PetaSAN hyper-converged, we have started some tests ourselves but they are not done yet...sorry.
I would be interested to know why you need VLAN tagging and not just map all subnets to the same switch/nic un-tagged but if you do need tagging the default VST tagging mode should work with PetaSAN.
Last edited on June 28, 2017, 8:41 pm · #14
philip.shannon
37 Posts
June 29, 2017, 5:19 pmQuote from philip.shannon on June 29, 2017, 5:19 pmthe only options that I'm aware of from netops is you get 1 vlan, or else you tell them which vlans you want for 1 connection, and they configure that port as trunk allowed and it can access the vlans you specify. This uses vlan tagging, you create a distributed switch in vcenter and a port group for each vlan involved. I tried this and the iscsi networks don't work at all, the esxi host cannot ping the ip addresses for the shared disk for petasan, therefore cannot connect. The strange thing is, the esxi hosts can ping the Back End ip's for the petasan vm's, just can't ping anything on iscsi networks. I'm using vcenter 6.5 and esxi 6.5 hosts I don't see anything about VST in here but will keep looking. Maybe you are thinking of vcenter 5 & esxi 5 for VST?
the only options that I'm aware of from netops is you get 1 vlan, or else you tell them which vlans you want for 1 connection, and they configure that port as trunk allowed and it can access the vlans you specify. This uses vlan tagging, you create a distributed switch in vcenter and a port group for each vlan involved. I tried this and the iscsi networks don't work at all, the esxi host cannot ping the ip addresses for the shared disk for petasan, therefore cannot connect. The strange thing is, the esxi hosts can ping the Back End ip's for the petasan vm's, just can't ping anything on iscsi networks. I'm using vcenter 6.5 and esxi 6.5 hosts I don't see anything about VST in here but will keep looking. Maybe you are thinking of vcenter 5 & esxi 5 for VST?
Last edited on June 29, 2017, 5:21 pm · #15
admin
2,930 Posts
June 29, 2017, 6:21 pmQuote from admin on June 29, 2017, 6:21 pmThis is the 6.5 docs
https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.networking.doc/GUID-D5960C77-0D19-4669-A00C-B05D58A422F8.html
in step 3 you can either say
- None ( no VLANs )
- VLAN this is VST mode you need to specify a VLAN id from 1-4094. In this mode the virtual switch strips vlan tags for traffic going into the VMs and adds tags for traffic coming out. The VMs are unware of the tagging and need not specify any configurations themselves.
- VLAN Trunking this is the VGT mode, PetaSAN will not work in this mode. This is where the VMs need to be aware of VLANs and handle the configuration themselves.
The first 2 should work in PetaSAN.
It is also strange your backend subnets can be pinged but not the iSCSI. just double check that your iSCSI disks are started and not stopped, if stopped there will be no ips on these subnets..they are virtual ips that move from node to node.
This is the 6.5 docs
in step 3 you can either say
- None ( no VLANs )
- VLAN this is VST mode you need to specify a VLAN id from 1-4094. In this mode the virtual switch strips vlan tags for traffic going into the VMs and adds tags for traffic coming out. The VMs are unware of the tagging and need not specify any configurations themselves.
- VLAN Trunking this is the VGT mode, PetaSAN will not work in this mode. This is where the VMs need to be aware of VLANs and handle the configuration themselves.
The first 2 should work in PetaSAN.
It is also strange your backend subnets can be pinged but not the iSCSI. just double check that your iSCSI disks are started and not stopped, if stopped there will be no ips on these subnets..they are virtual ips that move from node to node.
Last edited on June 29, 2017, 6:22 pm · #16
Wido
7 Posts
June 29, 2017, 6:24 pmQuote from Wido on June 29, 2017, 6:24 pmThe net guys can giv you 3 things: all the vlans / a few vlans / just one vlan (native). In the first 2 cases, you will have to tag the vlans you want to use from the OS. In the second, you don't have to do anything.
In the case of petasan, vlan tagging is supported by the OS, but not by the web-ui. So, unless you can patch the web-ui and system to allow for vlan taggin (I have sent a patch for configuring only the backend networks , but not the iscsi networks), you won't have much success in deploying petasan.
I would suggest you to use the 1G lan for management (web interface, ssh, mailing, etc) and make a bond with all the 10G interfaces and use subnetting for the rest
The net guys can giv you 3 things: all the vlans / a few vlans / just one vlan (native). In the first 2 cases, you will have to tag the vlans you want to use from the OS. In the second, you don't have to do anything.
In the case of petasan, vlan tagging is supported by the OS, but not by the web-ui. So, unless you can patch the web-ui and system to allow for vlan taggin (I have sent a patch for configuring only the backend networks , but not the iscsi networks), you won't have much success in deploying petasan.
I would suggest you to use the 1G lan for management (web interface, ssh, mailing, etc) and make a bond with all the 10G interfaces and use subnetting for the rest
philip.shannon
37 Posts
June 29, 2017, 7:18 pmQuote from philip.shannon on June 29, 2017, 7:18 pmthanks you two! I'm using the 2nd option at step 3 VLAN "this is VST mode you need to specify a VLAN id from 1-4094" I'm specifying the specific vlan there. Will keep chipping away trying to get this working.
thanks you two! I'm using the 2nd option at step 3 VLAN "this is VST mode you need to specify a VLAN id from 1-4094" I'm specifying the specific vlan there. Will keep chipping away trying to get this working.
Pages: 1 2
Network vlans
philip.shannon
37 Posts
Quote from philip.shannon on June 28, 2017, 3:04 pmsetting up here, each esxi host only has 4 x 10G ports. Each esxi host needs 2 x iscsi and 1 x vmotion VMkernel connections. This leaves only 1 ten GB port for the vm's which will run petaSAN which makes it important that VLAN tagging can be used. I'm trying to get things working now using this setup, but the petaSAN vm's distributed ports for the 2 iscsi vlans aren't working, must be because of VLAN tagging.
setting up here, each esxi host only has 4 x 10G ports. Each esxi host needs 2 x iscsi and 1 x vmotion VMkernel connections. This leaves only 1 ten GB port for the vm's which will run petaSAN which makes it important that VLAN tagging can be used. I'm trying to get things working now using this setup, but the petaSAN vm's distributed ports for the 2 iscsi vlans aren't working, must be because of VLAN tagging.
admin
2,930 Posts
Quote from admin on June 28, 2017, 4:19 pmIn ESXi there are several tagging modes, VST: Virtual Switch tagging (i believe this is the default) mode the virtual switch inserts the tags from packets coming from VM and strips the tags for packets going into the VM, the VM does not see any tag. In contrast VGT, Virtual Guest tagging mode, it is the responsibility of the VM to deal with VLAN, but from the top of my had VST is the default, so you still can use VLAN within ESXi without configuration in PetaSAN.
In bare metal, you can share different subnets on same nics and bridges without VLANs, the drawback without having VLANs is you have a single broadcast domain, a broadcast within a subnet will be picked by all others, this could be an issue if your iSCSI subnets are exposed to large number of end user machines, but for a backend deployment i think the effect would be minimum.
Of course the best is to have different physical networks for your subnets.
In ESXi there are several tagging modes, VST: Virtual Switch tagging (i believe this is the default) mode the virtual switch inserts the tags from packets coming from VM and strips the tags for packets going into the VM, the VM does not see any tag. In contrast VGT, Virtual Guest tagging mode, it is the responsibility of the VM to deal with VLAN, but from the top of my had VST is the default, so you still can use VLAN within ESXi without configuration in PetaSAN.
In bare metal, you can share different subnets on same nics and bridges without VLANs, the drawback without having VLANs is you have a single broadcast domain, a broadcast within a subnet will be picked by all others, this could be an issue if your iSCSI subnets are exposed to large number of end user machines, but for a backend deployment i think the effect would be minimum.
Of course the best is to have different physical networks for your subnets.
philip.shannon
37 Posts
Quote from philip.shannon on June 28, 2017, 7:28 pmout of the 4 networks (Back Ends & iSCSi's) which two would you put on 1GB vs 10GB? Basically which is better off at 10G the iscsi or backend networks. Obviously we'd like them all at 10G but are facing challenges with the trunk ports/vlan tagging on servers with only 4 10g adapters
out of the 4 networks (Back Ends & iSCSi's) which two would you put on 1GB vs 10GB? Basically which is better off at 10G the iscsi or backend networks. Obviously we'd like them all at 10G but are facing challenges with the trunk ports/vlan tagging on servers with only 4 10g adapters
admin
2,930 Posts
Quote from admin on June 28, 2017, 8:36 pmWhat i can say is that backend 1 will have the combined bandwidth of both iSCSI subnets, during writes backend 2 bandwidth will be either the same as backend 1 in case of replica count=2 or will be double in case of replica count=3.
Maybe you can put each iSCSI subnet on 1G and each backend on 10 G.
Another option is to bond both 10G NICs and map all 4 subnets on this bond.
I wish i have a more concrete recommendation to run PetaSAN hyper-converged, we have started some tests ourselves but they are not done yet...sorry.
I would be interested to know why you need VLAN tagging and not just map all subnets to the same switch/nic un-tagged but if you do need tagging the default VST tagging mode should work with PetaSAN.
What i can say is that backend 1 will have the combined bandwidth of both iSCSI subnets, during writes backend 2 bandwidth will be either the same as backend 1 in case of replica count=2 or will be double in case of replica count=3.
Maybe you can put each iSCSI subnet on 1G and each backend on 10 G.
Another option is to bond both 10G NICs and map all 4 subnets on this bond.
I wish i have a more concrete recommendation to run PetaSAN hyper-converged, we have started some tests ourselves but they are not done yet...sorry.
I would be interested to know why you need VLAN tagging and not just map all subnets to the same switch/nic un-tagged but if you do need tagging the default VST tagging mode should work with PetaSAN.
philip.shannon
37 Posts
Quote from philip.shannon on June 29, 2017, 5:19 pmthe only options that I'm aware of from netops is you get 1 vlan, or else you tell them which vlans you want for 1 connection, and they configure that port as trunk allowed and it can access the vlans you specify. This uses vlan tagging, you create a distributed switch in vcenter and a port group for each vlan involved. I tried this and the iscsi networks don't work at all, the esxi host cannot ping the ip addresses for the shared disk for petasan, therefore cannot connect. The strange thing is, the esxi hosts can ping the Back End ip's for the petasan vm's, just can't ping anything on iscsi networks. I'm using vcenter 6.5 and esxi 6.5 hosts I don't see anything about VST in here but will keep looking. Maybe you are thinking of vcenter 5 & esxi 5 for VST?
the only options that I'm aware of from netops is you get 1 vlan, or else you tell them which vlans you want for 1 connection, and they configure that port as trunk allowed and it can access the vlans you specify. This uses vlan tagging, you create a distributed switch in vcenter and a port group for each vlan involved. I tried this and the iscsi networks don't work at all, the esxi host cannot ping the ip addresses for the shared disk for petasan, therefore cannot connect. The strange thing is, the esxi hosts can ping the Back End ip's for the petasan vm's, just can't ping anything on iscsi networks. I'm using vcenter 6.5 and esxi 6.5 hosts I don't see anything about VST in here but will keep looking. Maybe you are thinking of vcenter 5 & esxi 5 for VST?
admin
2,930 Posts
Quote from admin on June 29, 2017, 6:21 pmThis is the 6.5 docs
https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.networking.doc/GUID-D5960C77-0D19-4669-A00C-B05D58A422F8.html
in step 3 you can either say
- None ( no VLANs )
- VLAN this is VST mode you need to specify a VLAN id from 1-4094. In this mode the virtual switch strips vlan tags for traffic going into the VMs and adds tags for traffic coming out. The VMs are unware of the tagging and need not specify any configurations themselves.
- VLAN Trunking this is the VGT mode, PetaSAN will not work in this mode. This is where the VMs need to be aware of VLANs and handle the configuration themselves.
The first 2 should work in PetaSAN.
It is also strange your backend subnets can be pinged but not the iSCSI. just double check that your iSCSI disks are started and not stopped, if stopped there will be no ips on these subnets..they are virtual ips that move from node to node.
This is the 6.5 docs
in step 3 you can either say
- None ( no VLANs )
- VLAN this is VST mode you need to specify a VLAN id from 1-4094. In this mode the virtual switch strips vlan tags for traffic going into the VMs and adds tags for traffic coming out. The VMs are unware of the tagging and need not specify any configurations themselves.
- VLAN Trunking this is the VGT mode, PetaSAN will not work in this mode. This is where the VMs need to be aware of VLANs and handle the configuration themselves.
The first 2 should work in PetaSAN.
It is also strange your backend subnets can be pinged but not the iSCSI. just double check that your iSCSI disks are started and not stopped, if stopped there will be no ips on these subnets..they are virtual ips that move from node to node.
Wido
7 Posts
Quote from Wido on June 29, 2017, 6:24 pmThe net guys can giv you 3 things: all the vlans / a few vlans / just one vlan (native). In the first 2 cases, you will have to tag the vlans you want to use from the OS. In the second, you don't have to do anything.
In the case of petasan, vlan tagging is supported by the OS, but not by the web-ui. So, unless you can patch the web-ui and system to allow for vlan taggin (I have sent a patch for configuring only the backend networks , but not the iscsi networks), you won't have much success in deploying petasan.
I would suggest you to use the 1G lan for management (web interface, ssh, mailing, etc) and make a bond with all the 10G interfaces and use subnetting for the rest
The net guys can giv you 3 things: all the vlans / a few vlans / just one vlan (native). In the first 2 cases, you will have to tag the vlans you want to use from the OS. In the second, you don't have to do anything.
In the case of petasan, vlan tagging is supported by the OS, but not by the web-ui. So, unless you can patch the web-ui and system to allow for vlan taggin (I have sent a patch for configuring only the backend networks , but not the iscsi networks), you won't have much success in deploying petasan.
I would suggest you to use the 1G lan for management (web interface, ssh, mailing, etc) and make a bond with all the 10G interfaces and use subnetting for the rest
philip.shannon
37 Posts
Quote from philip.shannon on June 29, 2017, 7:18 pmthanks you two! I'm using the 2nd option at step 3 VLAN "this is VST mode you need to specify a VLAN id from 1-4094" I'm specifying the specific vlan there. Will keep chipping away trying to get this working.
thanks you two! I'm using the 2nd option at step 3 VLAN "this is VST mode you need to specify a VLAN id from 1-4094" I'm specifying the specific vlan there. Will keep chipping away trying to get this working.