iSCSI Disk Custom Sizes
RobertH
27 Posts
October 19, 2020, 6:19 pmQuote from RobertH on October 19, 2020, 6:19 pmIm trying to create an iSCSI disk with a custom size. Our storage pool gives (with a 3 way replication) roughly 9.2TB of usable space for the iSCSI disk, we want to try and maximize the space being used / available without potentially making the disk larger than the pool could sustain. No over provision, cause that is just asking for disaster down the road when someone drops a file on the disk and doesnt realize the underlying storage doesnt have that much space.
However when using the GUI to create a disk you have the slider which jumps from 5TB to 10TB, so we tried to type in 9 but that changed the unit size to GB instead of TB, so we punched in 9216, the slider sits at 5TB, clicked the OK button and it creates a 5TB disk....
How can we create a disk with a size that the slider doesnt allow for??
Feature request: Toggle button (checkbox, etc) to turn the slider off and allow for custom input in the text box in GB.
Additional question, since the iSCSI disks are all numbered (ie 00001, 00002, etc) if you delete all the iSCSI disks it doesnt seem to want to start numbering back at 1 again, it keeps going, is there a way to reset this back to 1 if there are no disks in the RBD pool, after playing with it and deleting things for testing the number is up at 6 and for consistency with the other cluster we build we would like to keep them identical (ie quorum target is 00001, storage target is 00002, etc)
Im trying to create an iSCSI disk with a custom size. Our storage pool gives (with a 3 way replication) roughly 9.2TB of usable space for the iSCSI disk, we want to try and maximize the space being used / available without potentially making the disk larger than the pool could sustain. No over provision, cause that is just asking for disaster down the road when someone drops a file on the disk and doesnt realize the underlying storage doesnt have that much space.
However when using the GUI to create a disk you have the slider which jumps from 5TB to 10TB, so we tried to type in 9 but that changed the unit size to GB instead of TB, so we punched in 9216, the slider sits at 5TB, clicked the OK button and it creates a 5TB disk....
How can we create a disk with a size that the slider doesnt allow for??
Feature request: Toggle button (checkbox, etc) to turn the slider off and allow for custom input in the text box in GB.
Additional question, since the iSCSI disks are all numbered (ie 00001, 00002, etc) if you delete all the iSCSI disks it doesnt seem to want to start numbering back at 1 again, it keeps going, is there a way to reset this back to 1 if there are no disks in the RBD pool, after playing with it and deleting things for testing the number is up at 6 and for consistency with the other cluster we build we would like to keep them identical (ie quorum target is 00001, storage target is 00002, etc)
admin
2,929 Posts
October 19, 2020, 8:27 pmQuote from admin on October 19, 2020, 8:27 pmYou are correct, it only allows sizes known to the slider..i will take up your request for dis-allowing the slider.
If you deleted all your images and want to start at index rather than next index, do a
consul kv delete -recurse PetaSAN/DiskPools
You are correct, it only allows sizes known to the slider..i will take up your request for dis-allowing the slider.
If you deleted all your images and want to start at index rather than next index, do a
consul kv delete -recurse PetaSAN/DiskPools
RobertH
27 Posts
October 20, 2020, 12:09 pmQuote from RobertH on October 20, 2020, 12:09 pmSo if we wanted to have a disk of say 9TB can we create a disk using the slider at 5TB and then some how resize it without using the GUI to a customized size? and if so what are the appropriate commands to do that??
The slider is really granular until you get into the TB size ranges then it seems to double in size with each step, sliders are nice from a GUI perspective but without more granularity they usually end up getting in the way of things.
So if we wanted to have a disk of say 9TB can we create a disk using the slider at 5TB and then some how resize it without using the GUI to a customized size? and if so what are the appropriate commands to do that??
The slider is really granular until you get into the TB size ranges then it seems to double in size with each step, sliders are nice from a GUI perspective but without more granularity they usually end up getting in the way of things.
admin
2,929 Posts
October 20, 2020, 3:16 pmQuote from admin on October 20, 2020, 3:16 pmWe will look into the slider ui issue.
We will look into the slider ui issue.
RobertH
27 Posts
October 20, 2020, 3:17 pmQuote from RobertH on October 20, 2020, 3:17 pmIn the mean time is there a way to change the size of an iscsi disk after the fact to a custom size without using the GUI??
In the mean time is there a way to change the size of an iscsi disk after the fact to a custom size without using the GUI??
admin
2,929 Posts
October 20, 2020, 3:30 pmQuote from admin on October 20, 2020, 3:30 pmsearch for resize in the forum it will also show you how to update the metadata
search for resize in the forum it will also show you how to update the metadata
RobertH
27 Posts
October 20, 2020, 6:46 pmQuote from RobertH on October 20, 2020, 6:46 pmHad to hunt around in about 9 different threads to find all the details, would be really helpful if there were a KB or documentation for some of the more advanced features that can be done via the command line...
In case anyone else ends up having to search to find the answer here is more concise in one spot:
- Log into the web admin and stop the iSCSI disk in question, make note of the disk's ID number (ie 00002)
- SSH into one of the management nodes using root and the cluster password
- Issue the command to resize the disk image on the RBD pool:
rbd resize image-###### -s $$$GB
where ##### is the disk ID number, and $$$ is the new size of the disk image in gigabytes
- Change to the utility scripts folder:
cd /opt/petasan/scripts/util
- Export the disks current meta data to a file:
./disk_meta.py read --image image-##### --pool rbd > /tmp/diskdata.cfg
where ##### is the disk ID number
- Edit the /tmp/diskdata.cfg with your editor of choice:
nano /tmp/diskdata.cfg
- Scroll to the bottom of the file and locate the entry labeled "size" and change its value to the size you specified in the RBD resize command above, then save the file
- Import the update disk meta data:
./disk_meta.py write --image image-##### --pool rbd --file /tmp/diskdata.cfg
- Verify the updated value on the iSCSI disk page in the web admin console
Notes:
- The iSCSI disk page does not reflect values that aren't whole numbers, if you resize the disk to say 5900GB (ie 5.9TB) the GUI will show 5TB, if you click the disk ID number in the GUI it will pop up a screen that shows the actual raw disk size in TB
- The RBD Resize command can take values for the disk size in TB (ie 5TB instead of 5120GB) however the meta data is stored in gigabytes so you will need to convert when updating the meta data
Had to hunt around in about 9 different threads to find all the details, would be really helpful if there were a KB or documentation for some of the more advanced features that can be done via the command line...
In case anyone else ends up having to search to find the answer here is more concise in one spot:
- Log into the web admin and stop the iSCSI disk in question, make note of the disk's ID number (ie 00002)
- SSH into one of the management nodes using root and the cluster password
- Issue the command to resize the disk image on the RBD pool:
rbd resize image-###### -s $$$GB
where ##### is the disk ID number, and $$$ is the new size of the disk image in gigabytes
- Change to the utility scripts folder:
cd /opt/petasan/scripts/util
- Export the disks current meta data to a file:
./disk_meta.py read --image image-##### --pool rbd > /tmp/diskdata.cfg
where ##### is the disk ID number
- Edit the /tmp/diskdata.cfg with your editor of choice:
nano /tmp/diskdata.cfg
- Scroll to the bottom of the file and locate the entry labeled "size" and change its value to the size you specified in the RBD resize command above, then save the file
- Import the update disk meta data:
./disk_meta.py write --image image-##### --pool rbd --file /tmp/diskdata.cfg
- Verify the updated value on the iSCSI disk page in the web admin console
Notes:
- The iSCSI disk page does not reflect values that aren't whole numbers, if you resize the disk to say 5900GB (ie 5.9TB) the GUI will show 5TB, if you click the disk ID number in the GUI it will pop up a screen that shows the actual raw disk size in TB
- The RBD Resize command can take values for the disk size in TB (ie 5TB instead of 5120GB) however the meta data is stored in gigabytes so you will need to convert when updating the meta data
entrigant
9 Posts
September 30, 2024, 4:46 amQuote from entrigant on September 30, 2024, 4:46 amSorry if this is an unacceptable necro. I wanted to thank OP for collecting this info, ask if the steps listed are appropriate for a 3.0 install, and bump the request that the UI allow more granular control. The 10TB to 16TB bump is _rough_.
Thanks!
Sorry if this is an unacceptable necro. I wanted to thank OP for collecting this info, ask if the steps listed are appropriate for a 3.0 install, and bump the request that the UI allow more granular control. The 10TB to 16TB bump is _rough_.
Thanks!
RobertH
27 Posts
September 30, 2024, 1:38 pmQuote from RobertH on September 30, 2024, 1:38 pmIm not sure if the steps are still needed or not, easiest way to tell would be to try creating a drive with the GUI if it still jumps then the steps above would be needed, not sure if they have changed the manual scripts or not however, as we stopped using PetaSan earlier this year and switched to a hyper-converged setup using ProxMox with CEPH.
The performance jump on the same hardware was almost double, getting all the extra overhead from Hyper-V, Windows Cluster, and iSCSI out of the way was a night and day difference in our lab setup so we moved it to production.
There was a little bit of learning curve moving to ProxMox as we have been using Hyper-V for our hosting cluster since the 2008 R2 version but overall the performance has been a lot better, CEPH is maintained and upgraded by ProxMox directly and other than some low-level CEPH commands to setup email alerting it handles everything out of the box.
Im not sure if the steps are still needed or not, easiest way to tell would be to try creating a drive with the GUI if it still jumps then the steps above would be needed, not sure if they have changed the manual scripts or not however, as we stopped using PetaSan earlier this year and switched to a hyper-converged setup using ProxMox with CEPH.
The performance jump on the same hardware was almost double, getting all the extra overhead from Hyper-V, Windows Cluster, and iSCSI out of the way was a night and day difference in our lab setup so we moved it to production.
There was a little bit of learning curve moving to ProxMox as we have been using Hyper-V for our hosting cluster since the 2008 R2 version but overall the performance has been a lot better, CEPH is maintained and upgraded by ProxMox directly and other than some low-level CEPH commands to setup email alerting it handles everything out of the box.
admin
2,929 Posts
October 23, 2024, 10:34 amQuote from admin on October 23, 2024, 10:34 amWe added support in version 4,0 to support custom sizes, with any values.
Note that current jump from 10 to 16 TB in slider, the max variance is 30%, this as pointed may seem high but realistically if you purchase physical drives or you estimate new storage for a system, you typically allow for such variances.
We added support in version 4,0 to support custom sizes, with any values.
Note that current jump from 10 to 16 TB in slider, the max variance is 30%, this as pointed may seem high but realistically if you purchase physical drives or you estimate new storage for a system, you typically allow for such variances.
iSCSI Disk Custom Sizes
RobertH
27 Posts
Quote from RobertH on October 19, 2020, 6:19 pmIm trying to create an iSCSI disk with a custom size. Our storage pool gives (with a 3 way replication) roughly 9.2TB of usable space for the iSCSI disk, we want to try and maximize the space being used / available without potentially making the disk larger than the pool could sustain. No over provision, cause that is just asking for disaster down the road when someone drops a file on the disk and doesnt realize the underlying storage doesnt have that much space.
However when using the GUI to create a disk you have the slider which jumps from 5TB to 10TB, so we tried to type in 9 but that changed the unit size to GB instead of TB, so we punched in 9216, the slider sits at 5TB, clicked the OK button and it creates a 5TB disk....
How can we create a disk with a size that the slider doesnt allow for??
Feature request: Toggle button (checkbox, etc) to turn the slider off and allow for custom input in the text box in GB.
Additional question, since the iSCSI disks are all numbered (ie 00001, 00002, etc) if you delete all the iSCSI disks it doesnt seem to want to start numbering back at 1 again, it keeps going, is there a way to reset this back to 1 if there are no disks in the RBD pool, after playing with it and deleting things for testing the number is up at 6 and for consistency with the other cluster we build we would like to keep them identical (ie quorum target is 00001, storage target is 00002, etc)
Im trying to create an iSCSI disk with a custom size. Our storage pool gives (with a 3 way replication) roughly 9.2TB of usable space for the iSCSI disk, we want to try and maximize the space being used / available without potentially making the disk larger than the pool could sustain. No over provision, cause that is just asking for disaster down the road when someone drops a file on the disk and doesnt realize the underlying storage doesnt have that much space.
However when using the GUI to create a disk you have the slider which jumps from 5TB to 10TB, so we tried to type in 9 but that changed the unit size to GB instead of TB, so we punched in 9216, the slider sits at 5TB, clicked the OK button and it creates a 5TB disk....
How can we create a disk with a size that the slider doesnt allow for??
Feature request: Toggle button (checkbox, etc) to turn the slider off and allow for custom input in the text box in GB.
Additional question, since the iSCSI disks are all numbered (ie 00001, 00002, etc) if you delete all the iSCSI disks it doesnt seem to want to start numbering back at 1 again, it keeps going, is there a way to reset this back to 1 if there are no disks in the RBD pool, after playing with it and deleting things for testing the number is up at 6 and for consistency with the other cluster we build we would like to keep them identical (ie quorum target is 00001, storage target is 00002, etc)
admin
2,929 Posts
Quote from admin on October 19, 2020, 8:27 pmYou are correct, it only allows sizes known to the slider..i will take up your request for dis-allowing the slider.
If you deleted all your images and want to start at index rather than next index, do a
consul kv delete -recurse PetaSAN/DiskPools
You are correct, it only allows sizes known to the slider..i will take up your request for dis-allowing the slider.
If you deleted all your images and want to start at index rather than next index, do a
consul kv delete -recurse PetaSAN/DiskPools
RobertH
27 Posts
Quote from RobertH on October 20, 2020, 12:09 pmSo if we wanted to have a disk of say 9TB can we create a disk using the slider at 5TB and then some how resize it without using the GUI to a customized size? and if so what are the appropriate commands to do that??
The slider is really granular until you get into the TB size ranges then it seems to double in size with each step, sliders are nice from a GUI perspective but without more granularity they usually end up getting in the way of things.
So if we wanted to have a disk of say 9TB can we create a disk using the slider at 5TB and then some how resize it without using the GUI to a customized size? and if so what are the appropriate commands to do that??
The slider is really granular until you get into the TB size ranges then it seems to double in size with each step, sliders are nice from a GUI perspective but without more granularity they usually end up getting in the way of things.
admin
2,929 Posts
Quote from admin on October 20, 2020, 3:16 pmWe will look into the slider ui issue.
We will look into the slider ui issue.
RobertH
27 Posts
Quote from RobertH on October 20, 2020, 3:17 pmIn the mean time is there a way to change the size of an iscsi disk after the fact to a custom size without using the GUI??
In the mean time is there a way to change the size of an iscsi disk after the fact to a custom size without using the GUI??
admin
2,929 Posts
Quote from admin on October 20, 2020, 3:30 pmsearch for resize in the forum it will also show you how to update the metadata
search for resize in the forum it will also show you how to update the metadata
RobertH
27 Posts
Quote from RobertH on October 20, 2020, 6:46 pmHad to hunt around in about 9 different threads to find all the details, would be really helpful if there were a KB or documentation for some of the more advanced features that can be done via the command line...
In case anyone else ends up having to search to find the answer here is more concise in one spot:
- Log into the web admin and stop the iSCSI disk in question, make note of the disk's ID number (ie 00002)
- SSH into one of the management nodes using root and the cluster password
- Issue the command to resize the disk image on the RBD pool:
rbd resize image-###### -s $$$GB
where ##### is the disk ID number, and $$$ is the new size of the disk image in gigabytes- Change to the utility scripts folder:
cd /opt/petasan/scripts/util- Export the disks current meta data to a file:
./disk_meta.py read --image image-##### --pool rbd > /tmp/diskdata.cfg
where ##### is the disk ID number- Edit the /tmp/diskdata.cfg with your editor of choice:
nano /tmp/diskdata.cfg- Scroll to the bottom of the file and locate the entry labeled "size" and change its value to the size you specified in the RBD resize command above, then save the file
- Import the update disk meta data:
./disk_meta.py write --image image-##### --pool rbd --file /tmp/diskdata.cfg- Verify the updated value on the iSCSI disk page in the web admin console
Notes:
- The iSCSI disk page does not reflect values that aren't whole numbers, if you resize the disk to say 5900GB (ie 5.9TB) the GUI will show 5TB, if you click the disk ID number in the GUI it will pop up a screen that shows the actual raw disk size in TB
- The RBD Resize command can take values for the disk size in TB (ie 5TB instead of 5120GB) however the meta data is stored in gigabytes so you will need to convert when updating the meta data
Had to hunt around in about 9 different threads to find all the details, would be really helpful if there were a KB or documentation for some of the more advanced features that can be done via the command line...
In case anyone else ends up having to search to find the answer here is more concise in one spot:
- Log into the web admin and stop the iSCSI disk in question, make note of the disk's ID number (ie 00002)
- SSH into one of the management nodes using root and the cluster password
- Issue the command to resize the disk image on the RBD pool:
rbd resize image-###### -s $$$GB
where ##### is the disk ID number, and $$$ is the new size of the disk image in gigabytes - Change to the utility scripts folder:
cd /opt/petasan/scripts/util - Export the disks current meta data to a file:
./disk_meta.py read --image image-##### --pool rbd > /tmp/diskdata.cfg
where ##### is the disk ID number - Edit the /tmp/diskdata.cfg with your editor of choice:
nano /tmp/diskdata.cfg - Scroll to the bottom of the file and locate the entry labeled "size" and change its value to the size you specified in the RBD resize command above, then save the file
- Import the update disk meta data:
./disk_meta.py write --image image-##### --pool rbd --file /tmp/diskdata.cfg - Verify the updated value on the iSCSI disk page in the web admin console
Notes:
- The iSCSI disk page does not reflect values that aren't whole numbers, if you resize the disk to say 5900GB (ie 5.9TB) the GUI will show 5TB, if you click the disk ID number in the GUI it will pop up a screen that shows the actual raw disk size in TB
- The RBD Resize command can take values for the disk size in TB (ie 5TB instead of 5120GB) however the meta data is stored in gigabytes so you will need to convert when updating the meta data
entrigant
9 Posts
Quote from entrigant on September 30, 2024, 4:46 amSorry if this is an unacceptable necro. I wanted to thank OP for collecting this info, ask if the steps listed are appropriate for a 3.0 install, and bump the request that the UI allow more granular control. The 10TB to 16TB bump is _rough_.
Thanks!
Sorry if this is an unacceptable necro. I wanted to thank OP for collecting this info, ask if the steps listed are appropriate for a 3.0 install, and bump the request that the UI allow more granular control. The 10TB to 16TB bump is _rough_.
Thanks!
RobertH
27 Posts
Quote from RobertH on September 30, 2024, 1:38 pmIm not sure if the steps are still needed or not, easiest way to tell would be to try creating a drive with the GUI if it still jumps then the steps above would be needed, not sure if they have changed the manual scripts or not however, as we stopped using PetaSan earlier this year and switched to a hyper-converged setup using ProxMox with CEPH.
The performance jump on the same hardware was almost double, getting all the extra overhead from Hyper-V, Windows Cluster, and iSCSI out of the way was a night and day difference in our lab setup so we moved it to production.
There was a little bit of learning curve moving to ProxMox as we have been using Hyper-V for our hosting cluster since the 2008 R2 version but overall the performance has been a lot better, CEPH is maintained and upgraded by ProxMox directly and other than some low-level CEPH commands to setup email alerting it handles everything out of the box.
Im not sure if the steps are still needed or not, easiest way to tell would be to try creating a drive with the GUI if it still jumps then the steps above would be needed, not sure if they have changed the manual scripts or not however, as we stopped using PetaSan earlier this year and switched to a hyper-converged setup using ProxMox with CEPH.
The performance jump on the same hardware was almost double, getting all the extra overhead from Hyper-V, Windows Cluster, and iSCSI out of the way was a night and day difference in our lab setup so we moved it to production.
There was a little bit of learning curve moving to ProxMox as we have been using Hyper-V for our hosting cluster since the 2008 R2 version but overall the performance has been a lot better, CEPH is maintained and upgraded by ProxMox directly and other than some low-level CEPH commands to setup email alerting it handles everything out of the box.
admin
2,929 Posts
Quote from admin on October 23, 2024, 10:34 amWe added support in version 4,0 to support custom sizes, with any values.
Note that current jump from 10 to 16 TB in slider, the max variance is 30%, this as pointed may seem high but realistically if you purchase physical drives or you estimate new storage for a system, you typically allow for such variances.
We added support in version 4,0 to support custom sizes, with any values.
Note that current jump from 10 to 16 TB in slider, the max variance is 30%, this as pointed may seem high but realistically if you purchase physical drives or you estimate new storage for a system, you typically allow for such variances.