New PetaSAN system build question
X1M
5 Posts
March 1, 2022, 10:36 amQuote from X1M on March 1, 2022, 10:36 amWe are working on a new CEPH system, we have used CEPH since 2017 and very happy with CEPH. But now we want to move from vanilla CEPH to PetaSAN since PetaSAN now offers CephFS, which are our primary use case.
Q: Would it be a OK plan to build our new system with PetaSAN using like 8x1U servers, each with 1x250GB for OS, 4x16TB for storage and 1 or 2 NVMe for cache?
Right now, we use 2U servers with 8x8TB for storage, but I am worried about performance going down to only 4 disks for storage.
We are working on a new CEPH system, we have used CEPH since 2017 and very happy with CEPH. But now we want to move from vanilla CEPH to PetaSAN since PetaSAN now offers CephFS, which are our primary use case.
Q: Would it be a OK plan to build our new system with PetaSAN using like 8x1U servers, each with 1x250GB for OS, 4x16TB for storage and 1 or 2 NVMe for cache?
Right now, we use 2U servers with 8x8TB for storage, but I am worried about performance going down to only 4 disks for storage.
Last edited on March 1, 2022, 10:37 am by X1M · #1
admin
2,930 Posts
March 1, 2022, 10:40 pmQuote from admin on March 1, 2022, 10:40 pmHaving 4 OSDs will give less total performance than 8. Unless your workload concurrency/queue depth is small compared to the OSD count in which case the impact will be small, else in majority of cases it will make a difference.
Using an nvme as an external journal/db is a must. The other may be used as write cache, if your workload has a lot of small block size writes the cache will help, else if it is large block sizes like backups the cache may slow things as they would double writes. Best is to test yourself as also this is hardware and workload dependent.
Having 4 OSDs will give less total performance than 8. Unless your workload concurrency/queue depth is small compared to the OSD count in which case the impact will be small, else in majority of cases it will make a difference.
Using an nvme as an external journal/db is a must. The other may be used as write cache, if your workload has a lot of small block size writes the cache will help, else if it is large block sizes like backups the cache may slow things as they would double writes. Best is to test yourself as also this is hardware and workload dependent.
X1M
5 Posts
March 2, 2022, 7:52 amQuote from X1M on March 2, 2022, 7:52 amThank you for your reply, I think we in that case to be sure will still go with the 2U 8 disk system. Looking forward to take PetaSAN for at test drive. 🙂
Thank you for your reply, I think we in that case to be sure will still go with the 2U 8 disk system. Looking forward to take PetaSAN for at test drive. 🙂
New PetaSAN system build question
X1M
5 Posts
Quote from X1M on March 1, 2022, 10:36 amWe are working on a new CEPH system, we have used CEPH since 2017 and very happy with CEPH. But now we want to move from vanilla CEPH to PetaSAN since PetaSAN now offers CephFS, which are our primary use case.
Q: Would it be a OK plan to build our new system with PetaSAN using like 8x1U servers, each with 1x250GB for OS, 4x16TB for storage and 1 or 2 NVMe for cache?
Right now, we use 2U servers with 8x8TB for storage, but I am worried about performance going down to only 4 disks for storage.
We are working on a new CEPH system, we have used CEPH since 2017 and very happy with CEPH. But now we want to move from vanilla CEPH to PetaSAN since PetaSAN now offers CephFS, which are our primary use case.
Q: Would it be a OK plan to build our new system with PetaSAN using like 8x1U servers, each with 1x250GB for OS, 4x16TB for storage and 1 or 2 NVMe for cache?
Right now, we use 2U servers with 8x8TB for storage, but I am worried about performance going down to only 4 disks for storage.
admin
2,930 Posts
Quote from admin on March 1, 2022, 10:40 pmHaving 4 OSDs will give less total performance than 8. Unless your workload concurrency/queue depth is small compared to the OSD count in which case the impact will be small, else in majority of cases it will make a difference.
Using an nvme as an external journal/db is a must. The other may be used as write cache, if your workload has a lot of small block size writes the cache will help, else if it is large block sizes like backups the cache may slow things as they would double writes. Best is to test yourself as also this is hardware and workload dependent.
Having 4 OSDs will give less total performance than 8. Unless your workload concurrency/queue depth is small compared to the OSD count in which case the impact will be small, else in majority of cases it will make a difference.
Using an nvme as an external journal/db is a must. The other may be used as write cache, if your workload has a lot of small block size writes the cache will help, else if it is large block sizes like backups the cache may slow things as they would double writes. Best is to test yourself as also this is hardware and workload dependent.
X1M
5 Posts
Quote from X1M on March 2, 2022, 7:52 amThank you for your reply, I think we in that case to be sure will still go with the 2U 8 disk system. Looking forward to take PetaSAN for at test drive. 🙂
Thank you for your reply, I think we in that case to be sure will still go with the 2U 8 disk system. Looking forward to take PetaSAN for at test drive. 🙂