Network Gateway for Subnets
NTmatter
8 Posts
July 17, 2020, 4:20 amQuote from NTmatter on July 17, 2020, 4:20 amI've set up a test cluster, roughly as described in the Quick Start setup guide, and I've placed Management, Backend, iSCSI 1, iSCSI 2, and CIFS into their own respective subnets. The one piece I'm missing is where I specify the network gateway for each of these subnets.
To reiterate the issue, I can't reach nodes on the SMB subnet because their gateway is unspecified.
I'm looking at manually adding a route on all SMB gateway hosts. Am I missing some intended functionality? Is the expectation that SMB lives on the Management network, as that's the only place a gateway can be specified?
I've set up a test cluster, roughly as described in the Quick Start setup guide, and I've placed Management, Backend, iSCSI 1, iSCSI 2, and CIFS into their own respective subnets. The one piece I'm missing is where I specify the network gateway for each of these subnets.
To reiterate the issue, I can't reach nodes on the SMB subnet because their gateway is unspecified.
I'm looking at manually adding a route on all SMB gateway hosts. Am I missing some intended functionality? Is the expectation that SMB lives on the Management network, as that's the only place a gateway can be specified?
admin
2,930 Posts
July 17, 2020, 11:42 amQuote from admin on July 17, 2020, 11:42 amCurrently we expect in CIFS to have all clients be on a CIFS client subnet ( similar to iSCSI clients having client subnet ). In such setup you can have the netbios name for the cluster served by round robin dns so your clients get distributed across the different PetaSAN servers and the connection is direct.
Routed access will lose benefit of scale out, you can somewhat make it work if you can put the CIFS on the same subnet and interface as your management network which is routed, but it is not an ideal solution.
Do you think it is advantageous to define a different gateway for the CIFS network will be better ?
Currently we expect in CIFS to have all clients be on a CIFS client subnet ( similar to iSCSI clients having client subnet ). In such setup you can have the netbios name for the cluster served by round robin dns so your clients get distributed across the different PetaSAN servers and the connection is direct.
Routed access will lose benefit of scale out, you can somewhat make it work if you can put the CIFS on the same subnet and interface as your management network which is routed, but it is not an ideal solution.
Do you think it is advantageous to define a different gateway for the CIFS network will be better ?
Last edited on July 17, 2020, 11:42 am by admin · #2
NTmatter
8 Posts
July 17, 2020, 1:16 pmQuote from NTmatter on July 17, 2020, 1:16 pmI'd say it's advantageous to have an optional gateway for all services. My CIFS clients are primarily end users using a mix of Ethernet and WiFi, all coming from Department-specific subnets, VPNs, and possibly other sites. These can't all be moved into a single /16.
Ceph and iSCSI clients may have a bit more flexibility in their VLAN membership, however there are cases where clients live in different subnets as well. The extra flexibility will help when it's needed, and a well-placed firewall or network ACL can manage access where necessary.
With a routed CIFS subnet, round-robin DNS should still work for scale-out. It might be less efficient than a purely packet-switched network, but each client will still get a random server IP pointing at a random backend server. The mix of IP's and source/destination ports also ensures balanced link utilization on LACP/Aggregated links at all points in the network. This is ideal, so long as the routers have sufficient bandwidth.
I'd say it's advantageous to have an optional gateway for all services. My CIFS clients are primarily end users using a mix of Ethernet and WiFi, all coming from Department-specific subnets, VPNs, and possibly other sites. These can't all be moved into a single /16.
Ceph and iSCSI clients may have a bit more flexibility in their VLAN membership, however there are cases where clients live in different subnets as well. The extra flexibility will help when it's needed, and a well-placed firewall or network ACL can manage access where necessary.
With a routed CIFS subnet, round-robin DNS should still work for scale-out. It might be less efficient than a purely packet-switched network, but each client will still get a random server IP pointing at a random backend server. The mix of IP's and source/destination ports also ensures balanced link utilization on LACP/Aggregated links at all points in the network. This is ideal, so long as the routers have sufficient bandwidth.
DividedByPi
32 Posts
November 19, 2020, 6:32 pmQuote from DividedByPi on November 19, 2020, 6:32 pmHey guys!
I am now experiencing the same limitation, and am curious if there has been any updates on this, or any work-arounds that are in place (aside from running your cifs on the same interface as your management)
I would very much appreciate it!
Also, if the only current work-around is to run your CIFS and management on the same interface, I am curious how I would go about moving my management do a different network interface, as currently I am just using the on-board 1Gbe interfaces for management, whereas I expected to use 10Gbe for my CIFS.
Thanks in advance for any help!
Hey guys!
I am now experiencing the same limitation, and am curious if there has been any updates on this, or any work-arounds that are in place (aside from running your cifs on the same interface as your management)
I would very much appreciate it!
Also, if the only current work-around is to run your CIFS and management on the same interface, I am curious how I would go about moving my management do a different network interface, as currently I am just using the on-board 1Gbe interfaces for management, whereas I expected to use 10Gbe for my CIFS.
Thanks in advance for any help!
admin
2,930 Posts
November 19, 2020, 7:02 pmQuote from admin on November 19, 2020, 7:02 pmThis is supported in upcoming version 2.7 due by end of this month
http://www.petasan.org/forums/?view=thread&id=769&part=1#postid-4380
In current version you could manually add custom routes yourself, to make it persistent on reboots you should add any network customizations to:
/opt/petasan/scripts/custom/post_start_network.sh
else wait a couple of days for 2.7
This is supported in upcoming version 2.7 due by end of this month
http://www.petasan.org/forums/?view=thread&id=769&part=1#postid-4380
In current version you could manually add custom routes yourself, to make it persistent on reboots you should add any network customizations to:
/opt/petasan/scripts/custom/post_start_network.sh
else wait a couple of days for 2.7
DividedByPi
32 Posts
November 19, 2020, 7:07 pmQuote from DividedByPi on November 19, 2020, 7:07 pmThis is incredible news my friend! Fantastic work, as well as a ton of other very great features! I have been using the PG autoscaler on our Petasan cluster for a while now as well so that is great to see it being added to the GUI !
Thanks for the quick reply! I was contemplating the addition of some custom routes myself, but I think I will wait for 2.7!
This is incredible news my friend! Fantastic work, as well as a ton of other very great features! I have been using the PG autoscaler on our Petasan cluster for a while now as well so that is great to see it being added to the GUI !
Thanks for the quick reply! I was contemplating the addition of some custom routes myself, but I think I will wait for 2.7!
Network Gateway for Subnets
NTmatter
8 Posts
Quote from NTmatter on July 17, 2020, 4:20 amI've set up a test cluster, roughly as described in the Quick Start setup guide, and I've placed Management, Backend, iSCSI 1, iSCSI 2, and CIFS into their own respective subnets. The one piece I'm missing is where I specify the network gateway for each of these subnets.
To reiterate the issue, I can't reach nodes on the SMB subnet because their gateway is unspecified.
I'm looking at manually adding a route on all SMB gateway hosts. Am I missing some intended functionality? Is the expectation that SMB lives on the Management network, as that's the only place a gateway can be specified?
I've set up a test cluster, roughly as described in the Quick Start setup guide, and I've placed Management, Backend, iSCSI 1, iSCSI 2, and CIFS into their own respective subnets. The one piece I'm missing is where I specify the network gateway for each of these subnets.
To reiterate the issue, I can't reach nodes on the SMB subnet because their gateway is unspecified.
I'm looking at manually adding a route on all SMB gateway hosts. Am I missing some intended functionality? Is the expectation that SMB lives on the Management network, as that's the only place a gateway can be specified?
admin
2,930 Posts
Quote from admin on July 17, 2020, 11:42 amCurrently we expect in CIFS to have all clients be on a CIFS client subnet ( similar to iSCSI clients having client subnet ). In such setup you can have the netbios name for the cluster served by round robin dns so your clients get distributed across the different PetaSAN servers and the connection is direct.
Routed access will lose benefit of scale out, you can somewhat make it work if you can put the CIFS on the same subnet and interface as your management network which is routed, but it is not an ideal solution.
Do you think it is advantageous to define a different gateway for the CIFS network will be better ?
Currently we expect in CIFS to have all clients be on a CIFS client subnet ( similar to iSCSI clients having client subnet ). In such setup you can have the netbios name for the cluster served by round robin dns so your clients get distributed across the different PetaSAN servers and the connection is direct.
Routed access will lose benefit of scale out, you can somewhat make it work if you can put the CIFS on the same subnet and interface as your management network which is routed, but it is not an ideal solution.
Do you think it is advantageous to define a different gateway for the CIFS network will be better ?
NTmatter
8 Posts
Quote from NTmatter on July 17, 2020, 1:16 pmI'd say it's advantageous to have an optional gateway for all services. My CIFS clients are primarily end users using a mix of Ethernet and WiFi, all coming from Department-specific subnets, VPNs, and possibly other sites. These can't all be moved into a single /16.
Ceph and iSCSI clients may have a bit more flexibility in their VLAN membership, however there are cases where clients live in different subnets as well. The extra flexibility will help when it's needed, and a well-placed firewall or network ACL can manage access where necessary.
With a routed CIFS subnet, round-robin DNS should still work for scale-out. It might be less efficient than a purely packet-switched network, but each client will still get a random server IP pointing at a random backend server. The mix of IP's and source/destination ports also ensures balanced link utilization on LACP/Aggregated links at all points in the network. This is ideal, so long as the routers have sufficient bandwidth.
I'd say it's advantageous to have an optional gateway for all services. My CIFS clients are primarily end users using a mix of Ethernet and WiFi, all coming from Department-specific subnets, VPNs, and possibly other sites. These can't all be moved into a single /16.
Ceph and iSCSI clients may have a bit more flexibility in their VLAN membership, however there are cases where clients live in different subnets as well. The extra flexibility will help when it's needed, and a well-placed firewall or network ACL can manage access where necessary.
With a routed CIFS subnet, round-robin DNS should still work for scale-out. It might be less efficient than a purely packet-switched network, but each client will still get a random server IP pointing at a random backend server. The mix of IP's and source/destination ports also ensures balanced link utilization on LACP/Aggregated links at all points in the network. This is ideal, so long as the routers have sufficient bandwidth.
DividedByPi
32 Posts
Quote from DividedByPi on November 19, 2020, 6:32 pmHey guys!
I am now experiencing the same limitation, and am curious if there has been any updates on this, or any work-arounds that are in place (aside from running your cifs on the same interface as your management)
I would very much appreciate it!
Also, if the only current work-around is to run your CIFS and management on the same interface, I am curious how I would go about moving my management do a different network interface, as currently I am just using the on-board 1Gbe interfaces for management, whereas I expected to use 10Gbe for my CIFS.
Thanks in advance for any help!
Hey guys!
I am now experiencing the same limitation, and am curious if there has been any updates on this, or any work-arounds that are in place (aside from running your cifs on the same interface as your management)
I would very much appreciate it!
Also, if the only current work-around is to run your CIFS and management on the same interface, I am curious how I would go about moving my management do a different network interface, as currently I am just using the on-board 1Gbe interfaces for management, whereas I expected to use 10Gbe for my CIFS.
Thanks in advance for any help!
admin
2,930 Posts
Quote from admin on November 19, 2020, 7:02 pmThis is supported in upcoming version 2.7 due by end of this month
http://www.petasan.org/forums/?view=thread&id=769&part=1#postid-4380In current version you could manually add custom routes yourself, to make it persistent on reboots you should add any network customizations to:
/opt/petasan/scripts/custom/post_start_network.shelse wait a couple of days for 2.7
This is supported in upcoming version 2.7 due by end of this month
http://www.petasan.org/forums/?view=thread&id=769&part=1#postid-4380
In current version you could manually add custom routes yourself, to make it persistent on reboots you should add any network customizations to:
/opt/petasan/scripts/custom/post_start_network.sh
else wait a couple of days for 2.7
DividedByPi
32 Posts
Quote from DividedByPi on November 19, 2020, 7:07 pmThis is incredible news my friend! Fantastic work, as well as a ton of other very great features! I have been using the PG autoscaler on our Petasan cluster for a while now as well so that is great to see it being added to the GUI !
Thanks for the quick reply! I was contemplating the addition of some custom routes myself, but I think I will wait for 2.7!
This is incredible news my friend! Fantastic work, as well as a ton of other very great features! I have been using the PG autoscaler on our Petasan cluster for a while now as well so that is great to see it being added to the GUI !
Thanks for the quick reply! I was contemplating the addition of some custom routes myself, but I think I will wait for 2.7!