Best way to add more disks
Ste
125 Posts
April 22, 2020, 11:54 amQuote from Ste on April 22, 2020, 11:54 amHi Admin, please can you give me an advice ?
I currently have a total of 5 HDD in the cluster, now I want to add other 25 HDD. The first time I installed the cluster having only 5 HDD and choosing "15-50 disks" setup, but i encountered this issue (http://www.petasan.org/forums/?view=thread&id=629), so I reinstalled choosing less than 15 disks.
Now that HDD number will grow from 5 to 30, is it better to install the cluster again choosing 15-50 disks, or I can simply hot add the disks ? Will this have an impact on PG number ?
Thanks, S.
Hi Admin, please can you give me an advice ?
I currently have a total of 5 HDD in the cluster, now I want to add other 25 HDD. The first time I installed the cluster having only 5 HDD and choosing "15-50 disks" setup, but i encountered this issue (http://www.petasan.org/forums/?view=thread&id=629), so I reinstalled choosing less than 15 disks.
Now that HDD number will grow from 5 to 30, is it better to install the cluster again choosing 15-50 disks, or I can simply hot add the disks ? Will this have an impact on PG number ?
Thanks, S.
admin
2,930 Posts
April 22, 2020, 1:19 pmQuote from admin on April 22, 2020, 1:19 pmYou do not have to re-install .
With 30 disks you should have a min of 512 PGs in total, but 1024 in total is recommended.
If you do not mind re-balance traffic on existing data, manually increase the pg count on the exiting pool(s) to reach the above total.
Else if you do not mind storing your data on several pools, you can create a new pool with required pg count and you manage specifying the desired pool each time you create a disk.
You do not have to re-install .
With 30 disks you should have a min of 512 PGs in total, but 1024 in total is recommended.
If you do not mind re-balance traffic on existing data, manually increase the pg count on the exiting pool(s) to reach the above total.
Else if you do not mind storing your data on several pools, you can create a new pool with required pg count and you manage specifying the desired pool each time you create a disk.
Ste
125 Posts
April 22, 2020, 3:25 pmQuote from Ste on April 22, 2020, 3:25 pm
Quote from admin on April 22, 2020, 1:19 pm
manually increase the pg count on the exiting pool(s)
How can I manually increase the pg count ? Il I go in "Configuration -> pools -> edit" there's no possibility to change that number. I must delete and create again the pools as I did when decreasing the pg counts ?
Thnak you, S.
Quote from admin on April 22, 2020, 1:19 pm
manually increase the pg count on the exiting pool(s)
How can I manually increase the pg count ? Il I go in "Configuration -> pools -> edit" there's no possibility to change that number. I must delete and create again the pools as I did when decreasing the pg counts ?
Thnak you, S.
admin
2,930 Posts
April 22, 2020, 5:21 pmQuote from admin on April 22, 2020, 5:21 pmWe do not allow it via ui since it can cause a lot of data rebalance, it is not something you should be doing regularly, and try to avoid.
for the commands:
https://docs.ceph.com/docs/master/rados/operations/placement-groups/
ceph osd pool set {pool_name} pg_num {value}
ceph osd pool set {pool_name} pgp_num {value}
We do not allow it via ui since it can cause a lot of data rebalance, it is not something you should be doing regularly, and try to avoid.
for the commands:
https://docs.ceph.com/docs/master/rados/operations/placement-groups/
ceph osd pool set {pool_name} pg_num {value}
ceph osd pool set {pool_name} pgp_num {value}
Ste
125 Posts
April 22, 2020, 5:59 pmQuote from Ste on April 22, 2020, 5:59 pm
Quote from admin on April 22, 2020, 5:21 pm
... it is not something you should be doing regularly, and try to avoid.
Yes. I agree, in fact this is a very particular situation.
Just las very practical question: when I have to issue the ceph command, before of after adding the 25 disks ? Thanks
Quote from admin on April 22, 2020, 5:21 pm
... it is not something you should be doing regularly, and try to avoid.
Yes. I agree, in fact this is a very particular situation.
Just las very practical question: when I have to issue the ceph command, before of after adding the 25 disks ? Thanks
admin
2,930 Posts
April 22, 2020, 6:40 pmQuote from admin on April 22, 2020, 6:40 pmi believe the command will not allow you to increase by a large amount and requires you to do it steps.
if you can afford some downtime, you can add all the disks then do the pg increase, at first you osds will report an error of too few pgs per OSDs, until the pgs are increased to the min value. Else you can add the disks in steps, in that case i would recommend you do not add disks unless the cluster is OK and all rebalance of prev step is done, do the math taking into account you PGs per OSDs (including replicas/ec chunks) should be min 30 per OSD and max 300.
i believe the command will not allow you to increase by a large amount and requires you to do it steps.
if you can afford some downtime, you can add all the disks then do the pg increase, at first you osds will report an error of too few pgs per OSDs, until the pgs are increased to the min value. Else you can add the disks in steps, in that case i would recommend you do not add disks unless the cluster is OK and all rebalance of prev step is done, do the math taking into account you PGs per OSDs (including replicas/ec chunks) should be min 30 per OSD and max 300.
Best way to add more disks
Ste
125 Posts
Quote from Ste on April 22, 2020, 11:54 amHi Admin, please can you give me an advice ?
I currently have a total of 5 HDD in the cluster, now I want to add other 25 HDD. The first time I installed the cluster having only 5 HDD and choosing "15-50 disks" setup, but i encountered this issue (http://www.petasan.org/forums/?view=thread&id=629), so I reinstalled choosing less than 15 disks.
Now that HDD number will grow from 5 to 30, is it better to install the cluster again choosing 15-50 disks, or I can simply hot add the disks ? Will this have an impact on PG number ?
Thanks, S.
Hi Admin, please can you give me an advice ?
I currently have a total of 5 HDD in the cluster, now I want to add other 25 HDD. The first time I installed the cluster having only 5 HDD and choosing "15-50 disks" setup, but i encountered this issue (http://www.petasan.org/forums/?view=thread&id=629), so I reinstalled choosing less than 15 disks.
Now that HDD number will grow from 5 to 30, is it better to install the cluster again choosing 15-50 disks, or I can simply hot add the disks ? Will this have an impact on PG number ?
Thanks, S.
admin
2,930 Posts
Quote from admin on April 22, 2020, 1:19 pmYou do not have to re-install .
With 30 disks you should have a min of 512 PGs in total, but 1024 in total is recommended.
If you do not mind re-balance traffic on existing data, manually increase the pg count on the exiting pool(s) to reach the above total.
Else if you do not mind storing your data on several pools, you can create a new pool with required pg count and you manage specifying the desired pool each time you create a disk.
You do not have to re-install .
With 30 disks you should have a min of 512 PGs in total, but 1024 in total is recommended.
If you do not mind re-balance traffic on existing data, manually increase the pg count on the exiting pool(s) to reach the above total.
Else if you do not mind storing your data on several pools, you can create a new pool with required pg count and you manage specifying the desired pool each time you create a disk.
Ste
125 Posts
Quote from Ste on April 22, 2020, 3:25 pmQuote from admin on April 22, 2020, 1:19 pmmanually increase the pg count on the exiting pool(s)
How can I manually increase the pg count ? Il I go in "Configuration -> pools -> edit" there's no possibility to change that number. I must delete and create again the pools as I did when decreasing the pg counts ?
Thnak you, S.
Quote from admin on April 22, 2020, 1:19 pmmanually increase the pg count on the exiting pool(s)
How can I manually increase the pg count ? Il I go in "Configuration -> pools -> edit" there's no possibility to change that number. I must delete and create again the pools as I did when decreasing the pg counts ?
Thnak you, S.
admin
2,930 Posts
Quote from admin on April 22, 2020, 5:21 pmWe do not allow it via ui since it can cause a lot of data rebalance, it is not something you should be doing regularly, and try to avoid.
for the commands:
https://docs.ceph.com/docs/master/rados/operations/placement-groups/
ceph osd pool set {pool_name} pg_num {value}
ceph osd pool set {pool_name} pgp_num {value}
We do not allow it via ui since it can cause a lot of data rebalance, it is not something you should be doing regularly, and try to avoid.
for the commands:
https://docs.ceph.com/docs/master/rados/operations/placement-groups/
ceph osd pool set {pool_name} pg_num {value}
ceph osd pool set {pool_name} pgp_num {value}
Ste
125 Posts
Quote from Ste on April 22, 2020, 5:59 pmQuote from admin on April 22, 2020, 5:21 pm... it is not something you should be doing regularly, and try to avoid.
Yes. I agree, in fact this is a very particular situation.
Just las very practical question: when I have to issue the ceph command, before of after adding the 25 disks ? Thanks
Quote from admin on April 22, 2020, 5:21 pm... it is not something you should be doing regularly, and try to avoid.
Yes. I agree, in fact this is a very particular situation.
Just las very practical question: when I have to issue the ceph command, before of after adding the 25 disks ? Thanks
admin
2,930 Posts
Quote from admin on April 22, 2020, 6:40 pmi believe the command will not allow you to increase by a large amount and requires you to do it steps.
if you can afford some downtime, you can add all the disks then do the pg increase, at first you osds will report an error of too few pgs per OSDs, until the pgs are increased to the min value. Else you can add the disks in steps, in that case i would recommend you do not add disks unless the cluster is OK and all rebalance of prev step is done, do the math taking into account you PGs per OSDs (including replicas/ec chunks) should be min 30 per OSD and max 300.
i believe the command will not allow you to increase by a large amount and requires you to do it steps.
if you can afford some downtime, you can add all the disks then do the pg increase, at first you osds will report an error of too few pgs per OSDs, until the pgs are increased to the min value. Else you can add the disks in steps, in that case i would recommend you do not add disks unless the cluster is OK and all rebalance of prev step is done, do the math taking into account you PGs per OSDs (including replicas/ec chunks) should be min 30 per OSD and max 300.