hw recommendation
red
4 Posts
July 28, 2021, 11:22 amQuote from red on July 28, 2021, 11:22 amHi,
we are thinking about building 12 node EC cluster,
our premilary hardware configuration:
2x cpu, 256g-384 ram, 2x 240-480G for os and ceph mon, 2x 1.6tb nvme for db/wal, 34x hdd, 2x 25G network, 12g hba,
1 vlan for front and 1 vlan for backend.
any opinion or recommendations?
does petasan support splitting front/back with a vlan?
is it possible to create 17x db/wal on single nvme disk?
br
Hi,
we are thinking about building 12 node EC cluster,
our premilary hardware configuration:
2x cpu, 256g-384 ram, 2x 240-480G for os and ceph mon, 2x 1.6tb nvme for db/wal, 34x hdd, 2x 25G network, 12g hba,
1 vlan for front and 1 vlan for backend.
any opinion or recommendations?
does petasan support splitting front/back with a vlan?
is it possible to create 17x db/wal on single nvme disk?
br
admin
2,930 Posts
July 28, 2021, 11:21 pmQuote from admin on July 28, 2021, 11:21 pmFor EC we recommend 4+2 profile, in your case 8+3 can also be considered, but 4+2 will give better latency
For VM and latency sensitive workloads, we recommend SSD and replicated pools. So to use EC, make sure your workload can tolerate the higher latencies. You may also consider having a mix of pools for different workloads.
We support vlans for the different types of networks.
nvme should serve 12-16 HDDs, so 17 should be acceptable if performance is not critical. One thing to consider is if the nvme is damaged, it will render all 17 OSDs useless and will need to be cleaned and re-added as new OSD, so all their data will need to be recovered over the network, could cause load on cluster.
For EC we recommend 4+2 profile, in your case 8+3 can also be considered, but 4+2 will give better latency
For VM and latency sensitive workloads, we recommend SSD and replicated pools. So to use EC, make sure your workload can tolerate the higher latencies. You may also consider having a mix of pools for different workloads.
We support vlans for the different types of networks.
nvme should serve 12-16 HDDs, so 17 should be acceptable if performance is not critical. One thing to consider is if the nvme is damaged, it will render all 17 OSDs useless and will need to be cleaned and re-added as new OSD, so all their data will need to be recovered over the network, could cause load on cluster.
Last edited on July 28, 2021, 11:22 pm by admin · #2
red
4 Posts
July 29, 2021, 5:22 amQuote from red on July 29, 2021, 5:22 amthank you, it is mostly for backup space, still you are right if needed we can create different pools for different use cases, the main one being ec 8+3,
for nvme the intel p4610 1.6t looks nice, yes that is the disadvantage using single wal/db for multiple osds.
im still little concerned about performance, i would like to add something to boost read performance, more ram should help, not so sure about flashcache as it would be too fragmented,
any idea on that?
thank you, it is mostly for backup space, still you are right if needed we can create different pools for different use cases, the main one being ec 8+3,
for nvme the intel p4610 1.6t looks nice, yes that is the disadvantage using single wal/db for multiple osds.
im still little concerned about performance, i would like to add something to boost read performance, more ram should help, not so sure about flashcache as it would be too fragmented,
any idea on that?
admin
2,930 Posts
July 29, 2021, 9:59 amQuote from admin on July 29, 2021, 9:59 am4+2 will give better latency than 8+3.
For backups, try to use application that does large block size io and high concurrency. petasan-copy we have on our download page is a good tool to try under Windows. Are you using vmware/windows/linux ? do you use a special backup tool ?
4+2 will give better latency than 8+3.
For backups, try to use application that does large block size io and high concurrency. petasan-copy we have on our download page is a good tool to try under Windows. Are you using vmware/windows/linux ? do you use a special backup tool ?
pedro6161
36 Posts
August 17, 2021, 8:02 pmQuote from pedro6161 on August 17, 2021, 8:02 pmsorry if i'm stupid, could you please help me to explain what is 4+2 and 8+3 ?
Thanks
sorry if i'm stupid, could you please help me to explain what is 4+2 and 8+3 ?
Thanks
admin
2,930 Posts
August 17, 2021, 9:56 pmQuote from admin on August 17, 2021, 9:56 pm4+2 -> EC profile k=4 (data chunks) , m=2 (parity chunks)
4+2 -> EC profile k=4 (data chunks) , m=2 (parity chunks)
hw recommendation
red
4 Posts
Quote from red on July 28, 2021, 11:22 amHi,
we are thinking about building 12 node EC cluster,
our premilary hardware configuration:
2x cpu, 256g-384 ram, 2x 240-480G for os and ceph mon, 2x 1.6tb nvme for db/wal, 34x hdd, 2x 25G network, 12g hba,
1 vlan for front and 1 vlan for backend.any opinion or recommendations?
does petasan support splitting front/back with a vlan?
is it possible to create 17x db/wal on single nvme disk?br
Hi,
we are thinking about building 12 node EC cluster,
our premilary hardware configuration:
2x cpu, 256g-384 ram, 2x 240-480G for os and ceph mon, 2x 1.6tb nvme for db/wal, 34x hdd, 2x 25G network, 12g hba,
1 vlan for front and 1 vlan for backend.
any opinion or recommendations?
does petasan support splitting front/back with a vlan?
is it possible to create 17x db/wal on single nvme disk?
br
admin
2,930 Posts
Quote from admin on July 28, 2021, 11:21 pmFor EC we recommend 4+2 profile, in your case 8+3 can also be considered, but 4+2 will give better latency
For VM and latency sensitive workloads, we recommend SSD and replicated pools. So to use EC, make sure your workload can tolerate the higher latencies. You may also consider having a mix of pools for different workloads.
We support vlans for the different types of networks.
nvme should serve 12-16 HDDs, so 17 should be acceptable if performance is not critical. One thing to consider is if the nvme is damaged, it will render all 17 OSDs useless and will need to be cleaned and re-added as new OSD, so all their data will need to be recovered over the network, could cause load on cluster.
For EC we recommend 4+2 profile, in your case 8+3 can also be considered, but 4+2 will give better latency
For VM and latency sensitive workloads, we recommend SSD and replicated pools. So to use EC, make sure your workload can tolerate the higher latencies. You may also consider having a mix of pools for different workloads.
We support vlans for the different types of networks.
nvme should serve 12-16 HDDs, so 17 should be acceptable if performance is not critical. One thing to consider is if the nvme is damaged, it will render all 17 OSDs useless and will need to be cleaned and re-added as new OSD, so all their data will need to be recovered over the network, could cause load on cluster.
red
4 Posts
Quote from red on July 29, 2021, 5:22 amthank you, it is mostly for backup space, still you are right if needed we can create different pools for different use cases, the main one being ec 8+3,
for nvme the intel p4610 1.6t looks nice, yes that is the disadvantage using single wal/db for multiple osds.
im still little concerned about performance, i would like to add something to boost read performance, more ram should help, not so sure about flashcache as it would be too fragmented,
any idea on that?
thank you, it is mostly for backup space, still you are right if needed we can create different pools for different use cases, the main one being ec 8+3,
for nvme the intel p4610 1.6t looks nice, yes that is the disadvantage using single wal/db for multiple osds.
im still little concerned about performance, i would like to add something to boost read performance, more ram should help, not so sure about flashcache as it would be too fragmented,
any idea on that?
admin
2,930 Posts
Quote from admin on July 29, 2021, 9:59 am4+2 will give better latency than 8+3.
For backups, try to use application that does large block size io and high concurrency. petasan-copy we have on our download page is a good tool to try under Windows. Are you using vmware/windows/linux ? do you use a special backup tool ?
4+2 will give better latency than 8+3.
For backups, try to use application that does large block size io and high concurrency. petasan-copy we have on our download page is a good tool to try under Windows. Are you using vmware/windows/linux ? do you use a special backup tool ?
pedro6161
36 Posts
Quote from pedro6161 on August 17, 2021, 8:02 pmsorry if i'm stupid, could you please help me to explain what is 4+2 and 8+3 ?
Thanks
sorry if i'm stupid, could you please help me to explain what is 4+2 and 8+3 ?
Thanks
admin
2,930 Posts
Quote from admin on August 17, 2021, 9:56 pm4+2 -> EC profile k=4 (data chunks) , m=2 (parity chunks)
4+2 -> EC profile k=4 (data chunks) , m=2 (parity chunks)