ratio, sizing, general considerations
niqts
12 Posts
February 18, 2022, 4:15 pmQuote from niqts on February 18, 2022, 4:15 pmHi all,
we plan a setup of 5 nodes (three management nodes).
When creating a new cluster, there are some questions.
Should cephfs/radosgw/rbd be created on each node (or only on management nodes)?
Regarding the disk usage, each node ships with 10 identical SSD disks, would it even make sense for us to use journal on dedicated disks?
If we should have journal on dedicated disks, what is the ratio having 10 disks in each node (3 management nodes and 2 additional nodes).
The tuning template can be left "default" right?
Our hardware is a bit outdated, but we have about 32 GB RAM and 10 SSD disks with 2x6 cores Xeon CPUs.
Thanks so much in advance for any help (or pointing us to some sizing documents).
Kind regards
Hi all,
we plan a setup of 5 nodes (three management nodes).
When creating a new cluster, there are some questions.
Should cephfs/radosgw/rbd be created on each node (or only on management nodes)?
Regarding the disk usage, each node ships with 10 identical SSD disks, would it even make sense for us to use journal on dedicated disks?
If we should have journal on dedicated disks, what is the ratio having 10 disks in each node (3 management nodes and 2 additional nodes).
The tuning template can be left "default" right?
Our hardware is a bit outdated, but we have about 32 GB RAM and 10 SSD disks with 2x6 cores Xeon CPUs.
Thanks so much in advance for any help (or pointing us to some sizing documents).
Kind regards
admin
2,930 Posts
February 19, 2022, 1:45 pmQuote from admin on February 19, 2022, 1:45 pmThe most important issue is ram, 32 GB is too low for 10 OSDs, it would probably lead to problems. Either increase it to 64 GB at least or add 5 OSDs per node.
No need for external journals for SSDs.
Yes use default tuning profile
It does not matter if you create default pools at deployment or after you deploy, in your case you have simple case of similar OSDs, you adding them at deploy should be ok. in more advanced cases where you want to create pools with more control you should do so after deploy. Only create default pools than you need, do not create S3 pools if you will not use them now, you can always create later after deploy if needed. You can add OSDs at deployment for nodes designated for storage, similarly you need to enable gateway services (iSCSI/NFS/SMB/S3) on the nodes assigned for this, nodes with gateway services need not be OSD storage nodes.
The most important issue is ram, 32 GB is too low for 10 OSDs, it would probably lead to problems. Either increase it to 64 GB at least or add 5 OSDs per node.
No need for external journals for SSDs.
Yes use default tuning profile
It does not matter if you create default pools at deployment or after you deploy, in your case you have simple case of similar OSDs, you adding them at deploy should be ok. in more advanced cases where you want to create pools with more control you should do so after deploy. Only create default pools than you need, do not create S3 pools if you will not use them now, you can always create later after deploy if needed. You can add OSDs at deployment for nodes designated for storage, similarly you need to enable gateway services (iSCSI/NFS/SMB/S3) on the nodes assigned for this, nodes with gateway services need not be OSD storage nodes.
niqts
12 Posts
February 21, 2022, 1:46 pmQuote from niqts on February 21, 2022, 1:46 pmwow, thanks, that is super useful information. So we will definitely try to upgrade our RAM to 64.
There is one more thing, the management network does not require high bandwidth right? So 1GBIT is enough, as it is only for "management" of the cluster as the name indicates (no public/backend traffic flowing through it), so even if the management network is not available, the cluster will still function as expected?
Thanks in advance and kind regards
wow, thanks, that is super useful information. So we will definitely try to upgrade our RAM to 64.
There is one more thing, the management network does not require high bandwidth right? So 1GBIT is enough, as it is only for "management" of the cluster as the name indicates (no public/backend traffic flowing through it), so even if the management network is not available, the cluster will still function as expected?
Thanks in advance and kind regards
admin
2,930 Posts
February 21, 2022, 11:09 pmQuote from admin on February 21, 2022, 11:09 pmyes this is correct
yes this is correct
niqts
12 Posts
February 23, 2022, 7:51 amQuote from niqts on February 23, 2022, 7:51 amok, got it, thanks again for your feedback!
ok, got it, thanks again for your feedback!
ratio, sizing, general considerations
niqts
12 Posts
Quote from niqts on February 18, 2022, 4:15 pmHi all,
we plan a setup of 5 nodes (three management nodes).
When creating a new cluster, there are some questions.
Should cephfs/radosgw/rbd be created on each node (or only on management nodes)?
Regarding the disk usage, each node ships with 10 identical SSD disks, would it even make sense for us to use journal on dedicated disks?
If we should have journal on dedicated disks, what is the ratio having 10 disks in each node (3 management nodes and 2 additional nodes).
The tuning template can be left "default" right?
Our hardware is a bit outdated, but we have about 32 GB RAM and 10 SSD disks with 2x6 cores Xeon CPUs.
Thanks so much in advance for any help (or pointing us to some sizing documents).
Kind regards
Hi all,
we plan a setup of 5 nodes (three management nodes).
When creating a new cluster, there are some questions.
Should cephfs/radosgw/rbd be created on each node (or only on management nodes)?
Regarding the disk usage, each node ships with 10 identical SSD disks, would it even make sense for us to use journal on dedicated disks?
If we should have journal on dedicated disks, what is the ratio having 10 disks in each node (3 management nodes and 2 additional nodes).
The tuning template can be left "default" right?
Our hardware is a bit outdated, but we have about 32 GB RAM and 10 SSD disks with 2x6 cores Xeon CPUs.
Thanks so much in advance for any help (or pointing us to some sizing documents).
Kind regards
admin
2,930 Posts
Quote from admin on February 19, 2022, 1:45 pmThe most important issue is ram, 32 GB is too low for 10 OSDs, it would probably lead to problems. Either increase it to 64 GB at least or add 5 OSDs per node.
No need for external journals for SSDs.
Yes use default tuning profile
It does not matter if you create default pools at deployment or after you deploy, in your case you have simple case of similar OSDs, you adding them at deploy should be ok. in more advanced cases where you want to create pools with more control you should do so after deploy. Only create default pools than you need, do not create S3 pools if you will not use them now, you can always create later after deploy if needed. You can add OSDs at deployment for nodes designated for storage, similarly you need to enable gateway services (iSCSI/NFS/SMB/S3) on the nodes assigned for this, nodes with gateway services need not be OSD storage nodes.
The most important issue is ram, 32 GB is too low for 10 OSDs, it would probably lead to problems. Either increase it to 64 GB at least or add 5 OSDs per node.
No need for external journals for SSDs.
Yes use default tuning profile
It does not matter if you create default pools at deployment or after you deploy, in your case you have simple case of similar OSDs, you adding them at deploy should be ok. in more advanced cases where you want to create pools with more control you should do so after deploy. Only create default pools than you need, do not create S3 pools if you will not use them now, you can always create later after deploy if needed. You can add OSDs at deployment for nodes designated for storage, similarly you need to enable gateway services (iSCSI/NFS/SMB/S3) on the nodes assigned for this, nodes with gateway services need not be OSD storage nodes.
niqts
12 Posts
Quote from niqts on February 21, 2022, 1:46 pmwow, thanks, that is super useful information. So we will definitely try to upgrade our RAM to 64.
There is one more thing, the management network does not require high bandwidth right? So 1GBIT is enough, as it is only for "management" of the cluster as the name indicates (no public/backend traffic flowing through it), so even if the management network is not available, the cluster will still function as expected?
Thanks in advance and kind regards
wow, thanks, that is super useful information. So we will definitely try to upgrade our RAM to 64.
There is one more thing, the management network does not require high bandwidth right? So 1GBIT is enough, as it is only for "management" of the cluster as the name indicates (no public/backend traffic flowing through it), so even if the management network is not available, the cluster will still function as expected?
Thanks in advance and kind regards
admin
2,930 Posts
Quote from admin on February 21, 2022, 11:09 pmyes this is correct
yes this is correct
niqts
12 Posts
Quote from niqts on February 23, 2022, 7:51 amok, got it, thanks again for your feedback!
ok, got it, thanks again for your feedback!