Upgrade Recommendations
RobertH
27 Posts
November 8, 2021, 6:23 pmQuote from RobertH on November 8, 2021, 6:23 pmWe are currently running Petasan Version 2.6.2 looking to upgrade to latest Petasan build and want to make sure there aren't any "gotchas" during the process.
Petasan cluster consists of 3 nodes providing iSCSI storage for a Hyper-V cluster (8 nodes)
PS Node Specs (in each):
= 5x 1.6 TB SAS HDD for OSDs (15x total OSDs 128 PGs in RBD pool)
= 64GB RAM
= 4x 10GB fiber NICs (2x bonded with VLANs for MGMT and BACKEND, other two NICs dedicated for iSCSI, jumbo enabled on all)
= Dual 6 core Intel Xeon CPU E5-2630L (12 total cores, HT disabled)
= 1x 250GB SATA SSD for OS drive
Have not had to do an upgrade to the cluster previously, are there any active recommendations on the process or is it just shell in as root and do apt update/upgrades (one node at a time)??
Ideally in a perfect world I would stop all iSCSI traffic to the cluster but best I can probably do is cut the IO down by about 70% on the weekend by downing a bunch of machines that are only being used Mon-Fri, so cluster needs to be upgraded "live"
Looking at the documentation I'm thinking the process should be:
- Use the iSCSI path assignment to move the iSCSI targets off of node being upgraded
- Go into cluster maintenance settings turn off fencing and set all items to OFF to prevent data from being rebalanced / drives being marked as failed
- Log into node using SSH/Console
- Perform apt update
- Perform apt upgrade
- Perform apt install petasan
- Verify latest version is installed (dpkg -s petasan | grep Version)
- Access web admin console and verify node is only and cluster status is OK
- Repeat 1-8 on remaining nodes
- Once all 3 nodes are done and status is OK change maintenance settings back to ON and re-enable fencing
- Use iSCSI path assignment to re-balance the iSCSI paths across all 3 nodes
Thanks for the input
We are currently running Petasan Version 2.6.2 looking to upgrade to latest Petasan build and want to make sure there aren't any "gotchas" during the process.
Petasan cluster consists of 3 nodes providing iSCSI storage for a Hyper-V cluster (8 nodes)
PS Node Specs (in each):
= 5x 1.6 TB SAS HDD for OSDs (15x total OSDs 128 PGs in RBD pool)
= 64GB RAM
= 4x 10GB fiber NICs (2x bonded with VLANs for MGMT and BACKEND, other two NICs dedicated for iSCSI, jumbo enabled on all)
= Dual 6 core Intel Xeon CPU E5-2630L (12 total cores, HT disabled)
= 1x 250GB SATA SSD for OS drive
Have not had to do an upgrade to the cluster previously, are there any active recommendations on the process or is it just shell in as root and do apt update/upgrades (one node at a time)??
Ideally in a perfect world I would stop all iSCSI traffic to the cluster but best I can probably do is cut the IO down by about 70% on the weekend by downing a bunch of machines that are only being used Mon-Fri, so cluster needs to be upgraded "live"
Looking at the documentation I'm thinking the process should be:
- Use the iSCSI path assignment to move the iSCSI targets off of node being upgraded
- Go into cluster maintenance settings turn off fencing and set all items to OFF to prevent data from being rebalanced / drives being marked as failed
- Log into node using SSH/Console
- Perform apt update
- Perform apt upgrade
- Perform apt install petasan
- Verify latest version is installed (dpkg -s petasan | grep Version)
- Access web admin console and verify node is only and cluster status is OK
- Repeat 1-8 on remaining nodes
- Once all 3 nodes are done and status is OK change maintenance settings back to ON and re-enable fencing
- Use iSCSI path assignment to re-balance the iSCSI paths across all 3 nodes
Thanks for the input
admin
2,930 Posts
November 9, 2021, 10:04 pmQuote from admin on November 9, 2021, 10:04 pmmake sue you read our online upgrade guide.
you can also test a test vm cluster running 2.6.2 and do an upgrade to see the ptocess.
make sue you read our online upgrade guide.
you can also test a test vm cluster running 2.6.2 and do an upgrade to see the ptocess.
Upgrade Recommendations
RobertH
27 Posts
Quote from RobertH on November 8, 2021, 6:23 pmWe are currently running Petasan Version 2.6.2 looking to upgrade to latest Petasan build and want to make sure there aren't any "gotchas" during the process.
Petasan cluster consists of 3 nodes providing iSCSI storage for a Hyper-V cluster (8 nodes)
PS Node Specs (in each):
= 5x 1.6 TB SAS HDD for OSDs (15x total OSDs 128 PGs in RBD pool)
= 64GB RAM
= 4x 10GB fiber NICs (2x bonded with VLANs for MGMT and BACKEND, other two NICs dedicated for iSCSI, jumbo enabled on all)
= Dual 6 core Intel Xeon CPU E5-2630L (12 total cores, HT disabled)
= 1x 250GB SATA SSD for OS driveHave not had to do an upgrade to the cluster previously, are there any active recommendations on the process or is it just shell in as root and do apt update/upgrades (one node at a time)??
Ideally in a perfect world I would stop all iSCSI traffic to the cluster but best I can probably do is cut the IO down by about 70% on the weekend by downing a bunch of machines that are only being used Mon-Fri, so cluster needs to be upgraded "live"
Looking at the documentation I'm thinking the process should be:
- Use the iSCSI path assignment to move the iSCSI targets off of node being upgraded
- Go into cluster maintenance settings turn off fencing and set all items to OFF to prevent data from being rebalanced / drives being marked as failed
- Log into node using SSH/Console
- Perform apt update
- Perform apt upgrade
- Perform apt install petasan
- Verify latest version is installed (dpkg -s petasan | grep Version)
- Access web admin console and verify node is only and cluster status is OK
- Repeat 1-8 on remaining nodes
- Once all 3 nodes are done and status is OK change maintenance settings back to ON and re-enable fencing
- Use iSCSI path assignment to re-balance the iSCSI paths across all 3 nodes
Thanks for the input
We are currently running Petasan Version 2.6.2 looking to upgrade to latest Petasan build and want to make sure there aren't any "gotchas" during the process.
Petasan cluster consists of 3 nodes providing iSCSI storage for a Hyper-V cluster (8 nodes)
PS Node Specs (in each):
= 5x 1.6 TB SAS HDD for OSDs (15x total OSDs 128 PGs in RBD pool)
= 64GB RAM
= 4x 10GB fiber NICs (2x bonded with VLANs for MGMT and BACKEND, other two NICs dedicated for iSCSI, jumbo enabled on all)
= Dual 6 core Intel Xeon CPU E5-2630L (12 total cores, HT disabled)
= 1x 250GB SATA SSD for OS drive
Have not had to do an upgrade to the cluster previously, are there any active recommendations on the process or is it just shell in as root and do apt update/upgrades (one node at a time)??
Ideally in a perfect world I would stop all iSCSI traffic to the cluster but best I can probably do is cut the IO down by about 70% on the weekend by downing a bunch of machines that are only being used Mon-Fri, so cluster needs to be upgraded "live"
Looking at the documentation I'm thinking the process should be:
- Use the iSCSI path assignment to move the iSCSI targets off of node being upgraded
- Go into cluster maintenance settings turn off fencing and set all items to OFF to prevent data from being rebalanced / drives being marked as failed
- Log into node using SSH/Console
- Perform apt update
- Perform apt upgrade
- Perform apt install petasan
- Verify latest version is installed (dpkg -s petasan | grep Version)
- Access web admin console and verify node is only and cluster status is OK
- Repeat 1-8 on remaining nodes
- Once all 3 nodes are done and status is OK change maintenance settings back to ON and re-enable fencing
- Use iSCSI path assignment to re-balance the iSCSI paths across all 3 nodes
Thanks for the input
admin
2,930 Posts
Quote from admin on November 9, 2021, 10:04 pmmake sue you read our online upgrade guide.
you can also test a test vm cluster running 2.6.2 and do an upgrade to see the ptocess.
make sue you read our online upgrade guide.
you can also test a test vm cluster running 2.6.2 and do an upgrade to see the ptocess.