Cluster Network Reconfiguration + VLANS
JG
26 Posts
February 9, 2022, 5:20 pmQuote from JG on February 9, 2022, 5:20 pmHello all,
We have been using PetaSAN for almost a year and a half and there is nothing else to say than good stuff, thank you a lot for all the work you put on this, this is a great great product!
We are running 2 clusters at the moment (both runing the latest version of PetaSAN), one of them hooked to our Hyper-V Cluster, all working great. Recently our business needs have been increasing and we wanted to give a shoot to the support for S3, but we found an issue related to networking, our servers have 4x 10 Gb/s NICs configured on 2 bond interfaces as follow:
Cluster #1
NIC #1 + NIC #2 = BOND0 ( CEPH Backend + CEPH Public)
NIC #3 + NIC #4 = BOND1 (iSCSI)
Cluster #2
NIC #1 + NIC #2 = BOND0 ( CEPH Backend + CEPH Public)
NIC #3 + NIC #4 = BOND1 (FREE)
Our goal here is to divide traffic on BOND1 on both clusters through VLANs, but we want to avoid (or minimize the most we can) the downtime. Is there any good practice recommendations or a guide you could share with us to accomplish this since both cluster are up and running on production ?
Please let me know if you have any question about our setup and/our goals with our configuration.
Thank you
Hello all,
We have been using PetaSAN for almost a year and a half and there is nothing else to say than good stuff, thank you a lot for all the work you put on this, this is a great great product!
We are running 2 clusters at the moment (both runing the latest version of PetaSAN), one of them hooked to our Hyper-V Cluster, all working great. Recently our business needs have been increasing and we wanted to give a shoot to the support for S3, but we found an issue related to networking, our servers have 4x 10 Gb/s NICs configured on 2 bond interfaces as follow:
Cluster #1
NIC #1 + NIC #2 = BOND0 ( CEPH Backend + CEPH Public)
NIC #3 + NIC #4 = BOND1 (iSCSI)
Cluster #2
NIC #1 + NIC #2 = BOND0 ( CEPH Backend + CEPH Public)
NIC #3 + NIC #4 = BOND1 (FREE)
Our goal here is to divide traffic on BOND1 on both clusters through VLANs, but we want to avoid (or minimize the most we can) the downtime. Is there any good practice recommendations or a guide you could share with us to accomplish this since both cluster are up and running on production ?
Please let me know if you have any question about our setup and/our goals with our configuration.
Thank you
Last edited on February 9, 2022, 6:18 pm by JG · #1
admin
2,930 Posts
February 10, 2022, 7:08 pmQuote from admin on February 10, 2022, 7:08 pmWith iSCSI, if you change network setting for vlans, it will not affect current running disks, it will take effect for new disks or current disks if you stop them and start them again, or if a path is moved from node to node. So it is possible due to multipath to perform a change then use the path assignment page to move a subset of the paths at a time so in effect the disk will be served by both old and new configuration. This is just from PetaSAN end, you will need to figure from switches and also your clients how to achieve smooth transition.
For S3, if you change the vlans, the web server on the gateways running the S3 services are restarted to listen on the new vlan interface. This will be a few seconds, S3 clients should have no problem dealing with this short pause.
With iSCSI, if you change network setting for vlans, it will not affect current running disks, it will take effect for new disks or current disks if you stop them and start them again, or if a path is moved from node to node. So it is possible due to multipath to perform a change then use the path assignment page to move a subset of the paths at a time so in effect the disk will be served by both old and new configuration. This is just from PetaSAN end, you will need to figure from switches and also your clients how to achieve smooth transition.
For S3, if you change the vlans, the web server on the gateways running the S3 services are restarted to listen on the new vlan interface. This will be a few seconds, S3 clients should have no problem dealing with this short pause.
admin
2,930 Posts
February 10, 2022, 7:08 pmQuote from admin on February 10, 2022, 7:08 pmWith iSCSI, if you change network setting for vlans, it will not affect current running disks, it will take effect for new disks or current disks if you stop them and start them again, or if a path is moved from node to node. So it is possible due to multipath to perform a change then use the path assignment page to move a subset of the paths at a time so in effect the disk will be served by both old and new configuration. This is just from PetaSAN end, you will need to figure from switches and also your clients how to achieve smooth transition.
For S3, if you change the vlans, the web server on the gateways running the S3 services are restarted to listen on the new vlan interface. This will be a few seconds, S3 clients should have no problem dealing with this short pause.
With iSCSI, if you change network setting for vlans, it will not affect current running disks, it will take effect for new disks or current disks if you stop them and start them again, or if a path is moved from node to node. So it is possible due to multipath to perform a change then use the path assignment page to move a subset of the paths at a time so in effect the disk will be served by both old and new configuration. This is just from PetaSAN end, you will need to figure from switches and also your clients how to achieve smooth transition.
For S3, if you change the vlans, the web server on the gateways running the S3 services are restarted to listen on the new vlan interface. This will be a few seconds, S3 clients should have no problem dealing with this short pause.
JG
26 Posts
February 10, 2022, 8:18 pmQuote from JG on February 10, 2022, 8:18 pmThank you for your reply. Last question on this thread, how PetaSAN handles the network config gor iSCSI, S3, etc I couldn't find config files for this, is this handled by consul?
Thank you for your reply. Last question on this thread, how PetaSAN handles the network config gor iSCSI, S3, etc I couldn't find config files for this, is this handled by consul?
admin
2,930 Posts
February 10, 2022, 10:43 pmQuote from admin on February 10, 2022, 10:43 pmPetaSAN does its own network configuration rather than use Ubuntu setup so to provide a consistent configuration to all nodes in the cluster, it is a cluster wide configuration.
Services iSCSI/NFS/CIFS/S3 have their network config stored in Consul
Management an Backend "core" networks are done via config files:
/opt/petasan/config/cluster_info.json
/opt/petasan/config/node_info.json
So if Consul is down, you still have core network up, also Consul itself needs Backend to be up for it to work.
Note: if needed, an admin can customise any network setup by adding any setup commands to script:
/opt/petasan/scripts/custom/post_start_network.sh
PetaSAN does its own network configuration rather than use Ubuntu setup so to provide a consistent configuration to all nodes in the cluster, it is a cluster wide configuration.
Services iSCSI/NFS/CIFS/S3 have their network config stored in Consul
Management an Backend "core" networks are done via config files:
/opt/petasan/config/cluster_info.json
/opt/petasan/config/node_info.json
So if Consul is down, you still have core network up, also Consul itself needs Backend to be up for it to work.
Note: if needed, an admin can customise any network setup by adding any setup commands to script:
/opt/petasan/scripts/custom/post_start_network.sh
Last edited on February 10, 2022, 10:44 pm by admin · #5
JG
26 Posts
February 22, 2022, 9:56 pmQuote from JG on February 22, 2022, 9:56 pmThank you for sharing this info. I managed to figure this outsetting the network port as a hybrid (trunk and access at the same time).
Thank you for sharing this info. I managed to figure this outsetting the network port as a hybrid (trunk and access at the same time).
Cluster Network Reconfiguration + VLANS
JG
26 Posts
Quote from JG on February 9, 2022, 5:20 pmHello all,
We have been using PetaSAN for almost a year and a half and there is nothing else to say than good stuff, thank you a lot for all the work you put on this, this is a great great product!
We are running 2 clusters at the moment (both runing the latest version of PetaSAN), one of them hooked to our Hyper-V Cluster, all working great. Recently our business needs have been increasing and we wanted to give a shoot to the support for S3, but we found an issue related to networking, our servers have 4x 10 Gb/s NICs configured on 2 bond interfaces as follow:
Cluster #1
NIC #1 + NIC #2 = BOND0 ( CEPH Backend + CEPH Public)
NIC #3 + NIC #4 = BOND1 (iSCSI)
Cluster #2
NIC #1 + NIC #2 = BOND0 ( CEPH Backend + CEPH Public)
NIC #3 + NIC #4 = BOND1 (FREE)
Our goal here is to divide traffic on BOND1 on both clusters through VLANs, but we want to avoid (or minimize the most we can) the downtime. Is there any good practice recommendations or a guide you could share with us to accomplish this since both cluster are up and running on production ?
Please let me know if you have any question about our setup and/our goals with our configuration.
Thank you
Hello all,
We have been using PetaSAN for almost a year and a half and there is nothing else to say than good stuff, thank you a lot for all the work you put on this, this is a great great product!
We are running 2 clusters at the moment (both runing the latest version of PetaSAN), one of them hooked to our Hyper-V Cluster, all working great. Recently our business needs have been increasing and we wanted to give a shoot to the support for S3, but we found an issue related to networking, our servers have 4x 10 Gb/s NICs configured on 2 bond interfaces as follow:
Cluster #1
NIC #1 + NIC #2 = BOND0 ( CEPH Backend + CEPH Public)
NIC #3 + NIC #4 = BOND1 (iSCSI)
Cluster #2
NIC #1 + NIC #2 = BOND0 ( CEPH Backend + CEPH Public)
NIC #3 + NIC #4 = BOND1 (FREE)
Our goal here is to divide traffic on BOND1 on both clusters through VLANs, but we want to avoid (or minimize the most we can) the downtime. Is there any good practice recommendations or a guide you could share with us to accomplish this since both cluster are up and running on production ?
Please let me know if you have any question about our setup and/our goals with our configuration.
Thank you
admin
2,930 Posts
Quote from admin on February 10, 2022, 7:08 pmWith iSCSI, if you change network setting for vlans, it will not affect current running disks, it will take effect for new disks or current disks if you stop them and start them again, or if a path is moved from node to node. So it is possible due to multipath to perform a change then use the path assignment page to move a subset of the paths at a time so in effect the disk will be served by both old and new configuration. This is just from PetaSAN end, you will need to figure from switches and also your clients how to achieve smooth transition.
For S3, if you change the vlans, the web server on the gateways running the S3 services are restarted to listen on the new vlan interface. This will be a few seconds, S3 clients should have no problem dealing with this short pause.
With iSCSI, if you change network setting for vlans, it will not affect current running disks, it will take effect for new disks or current disks if you stop them and start them again, or if a path is moved from node to node. So it is possible due to multipath to perform a change then use the path assignment page to move a subset of the paths at a time so in effect the disk will be served by both old and new configuration. This is just from PetaSAN end, you will need to figure from switches and also your clients how to achieve smooth transition.
For S3, if you change the vlans, the web server on the gateways running the S3 services are restarted to listen on the new vlan interface. This will be a few seconds, S3 clients should have no problem dealing with this short pause.
admin
2,930 Posts
Quote from admin on February 10, 2022, 7:08 pmWith iSCSI, if you change network setting for vlans, it will not affect current running disks, it will take effect for new disks or current disks if you stop them and start them again, or if a path is moved from node to node. So it is possible due to multipath to perform a change then use the path assignment page to move a subset of the paths at a time so in effect the disk will be served by both old and new configuration. This is just from PetaSAN end, you will need to figure from switches and also your clients how to achieve smooth transition.
For S3, if you change the vlans, the web server on the gateways running the S3 services are restarted to listen on the new vlan interface. This will be a few seconds, S3 clients should have no problem dealing with this short pause.
With iSCSI, if you change network setting for vlans, it will not affect current running disks, it will take effect for new disks or current disks if you stop them and start them again, or if a path is moved from node to node. So it is possible due to multipath to perform a change then use the path assignment page to move a subset of the paths at a time so in effect the disk will be served by both old and new configuration. This is just from PetaSAN end, you will need to figure from switches and also your clients how to achieve smooth transition.
For S3, if you change the vlans, the web server on the gateways running the S3 services are restarted to listen on the new vlan interface. This will be a few seconds, S3 clients should have no problem dealing with this short pause.
JG
26 Posts
Quote from JG on February 10, 2022, 8:18 pmThank you for your reply. Last question on this thread, how PetaSAN handles the network config gor iSCSI, S3, etc I couldn't find config files for this, is this handled by consul?
Thank you for your reply. Last question on this thread, how PetaSAN handles the network config gor iSCSI, S3, etc I couldn't find config files for this, is this handled by consul?
admin
2,930 Posts
Quote from admin on February 10, 2022, 10:43 pmPetaSAN does its own network configuration rather than use Ubuntu setup so to provide a consistent configuration to all nodes in the cluster, it is a cluster wide configuration.
Services iSCSI/NFS/CIFS/S3 have their network config stored in Consul
Management an Backend "core" networks are done via config files:/opt/petasan/config/cluster_info.json
/opt/petasan/config/node_info.jsonSo if Consul is down, you still have core network up, also Consul itself needs Backend to be up for it to work.
Note: if needed, an admin can customise any network setup by adding any setup commands to script:
/opt/petasan/scripts/custom/post_start_network.sh
PetaSAN does its own network configuration rather than use Ubuntu setup so to provide a consistent configuration to all nodes in the cluster, it is a cluster wide configuration.
Services iSCSI/NFS/CIFS/S3 have their network config stored in Consul
Management an Backend "core" networks are done via config files:
/opt/petasan/config/cluster_info.json
/opt/petasan/config/node_info.json
So if Consul is down, you still have core network up, also Consul itself needs Backend to be up for it to work.
Note: if needed, an admin can customise any network setup by adding any setup commands to script:
/opt/petasan/scripts/custom/post_start_network.sh
JG
26 Posts
Quote from JG on February 22, 2022, 9:56 pmThank you for sharing this info. I managed to figure this outsetting the network port as a hybrid (trunk and access at the same time).
Thank you for sharing this info. I managed to figure this outsetting the network port as a hybrid (trunk and access at the same time).