Sanity Check on New Build
natnick
4 Posts
June 14, 2019, 6:01 pmQuote from natnick on June 14, 2019, 6:01 pmI have been using CEPH in production for the past 3 years and am very pleased with the flexibility and Scale-Out support it provides. I stumbled upon PetaSAN when looking for a clean easy to manage interface for CEPH that my Junior sysadmin's can feel comfortable monitoring our SAN but still be able to manually edit the configs if needed for tuning.
I am looking to re-purpose some existing hardware to build a new PetaSAN array that will support future expansion. This will be serving up iSCSI to a VMWare vCloud infrastructure. Some compute nodes will have NVMe vFlash (Read Cache) available as needed for workloads and performance.
I am looking initially at 8 Nodes to start (3 Dedicated to MNG/iSCSI & 5 Dedicated to Storage)
MNG/iSCSI Nodes (3) would each have the following specs.
- Dual Intel Xeon E5-2630L v2 2.4Ghz 6C(12T) Processors
- 32GB DDR3-RDIMM ECC Memory
- 2x200GB SSD (RAID1) OS Drive
- 4x10Gbe (2x iSCSI, 2x Backend)
- 2x1Gbe LAG (Mng)
Storage Nodes (5) would each have the following specs.
- Dual Intel Xeon E5-2630L 2.0Ghz 6C(12T) Processors or Intel Xeon E5-2620 2.0Ghz 6C(12T) Processors
- 48GB DDR-RDIMM ECC Memory
- 1x80GB SSD OS Drive
- 1x400GB Intel P3600 NVMe PCIe (DB/WAL)
- 5x4TB 7.2k NLSAS Enterprise Drive
- 2x10Gbe (2x Backend)
- 2x1Gbe (Mng)
Is there any red flags with this config?
I am planning to have 2 pools initially. 3 Replicated (VM OS Drives, etc) & EC (Bulk / Low Performance VMs). I am familiar with the native CEPH RADOS performance calculations for said configuration and have read on the PetaSAN forums that I should expect about 75% of the RADOS performance in VMware. Is this still correct?
Any other tips or guidance would be appreciated.
EDIT: Forgot to add the storage nodes would be using HBAs for Drive interface. Would I see any benefit in doing a RAID controller with Battery Back Write Cache in JBOD/Individual Desk RAID 0 mode?
I have been using CEPH in production for the past 3 years and am very pleased with the flexibility and Scale-Out support it provides. I stumbled upon PetaSAN when looking for a clean easy to manage interface for CEPH that my Junior sysadmin's can feel comfortable monitoring our SAN but still be able to manually edit the configs if needed for tuning.
I am looking to re-purpose some existing hardware to build a new PetaSAN array that will support future expansion. This will be serving up iSCSI to a VMWare vCloud infrastructure. Some compute nodes will have NVMe vFlash (Read Cache) available as needed for workloads and performance.
I am looking initially at 8 Nodes to start (3 Dedicated to MNG/iSCSI & 5 Dedicated to Storage)
MNG/iSCSI Nodes (3) would each have the following specs.
- Dual Intel Xeon E5-2630L v2 2.4Ghz 6C(12T) Processors
- 32GB DDR3-RDIMM ECC Memory
- 2x200GB SSD (RAID1) OS Drive
- 4x10Gbe (2x iSCSI, 2x Backend)
- 2x1Gbe LAG (Mng)
Storage Nodes (5) would each have the following specs.
- Dual Intel Xeon E5-2630L 2.0Ghz 6C(12T) Processors or Intel Xeon E5-2620 2.0Ghz 6C(12T) Processors
- 48GB DDR-RDIMM ECC Memory
- 1x80GB SSD OS Drive
- 1x400GB Intel P3600 NVMe PCIe (DB/WAL)
- 5x4TB 7.2k NLSAS Enterprise Drive
- 2x10Gbe (2x Backend)
- 2x1Gbe (Mng)
Is there any red flags with this config?
I am planning to have 2 pools initially. 3 Replicated (VM OS Drives, etc) & EC (Bulk / Low Performance VMs). I am familiar with the native CEPH RADOS performance calculations for said configuration and have read on the PetaSAN forums that I should expect about 75% of the RADOS performance in VMware. Is this still correct?
Any other tips or guidance would be appreciated.
EDIT: Forgot to add the storage nodes would be using HBAs for Drive interface. Would I see any benefit in doing a RAID controller with Battery Back Write Cache in JBOD/Individual Desk RAID 0 mode?
Last edited on June 14, 2019, 6:08 pm by natnick · #1
admin
2,930 Posts
June 14, 2019, 7:55 pmQuote from admin on June 14, 2019, 7:55 pmYou can also refer to
http://www.petasan.org/forums/?view=thread&id=478&part=1#postid-2824
for many of the points you raise. you should get over 75% native rados performance from your vms
You can also refer to
http://www.petasan.org/forums/?view=thread&id=478&part=1#postid-2824
for many of the points you raise. you should get over 75% native rados performance from your vms
Sanity Check on New Build
natnick
4 Posts
Quote from natnick on June 14, 2019, 6:01 pmI have been using CEPH in production for the past 3 years and am very pleased with the flexibility and Scale-Out support it provides. I stumbled upon PetaSAN when looking for a clean easy to manage interface for CEPH that my Junior sysadmin's can feel comfortable monitoring our SAN but still be able to manually edit the configs if needed for tuning.
I am looking to re-purpose some existing hardware to build a new PetaSAN array that will support future expansion. This will be serving up iSCSI to a VMWare vCloud infrastructure. Some compute nodes will have NVMe vFlash (Read Cache) available as needed for workloads and performance.
I am looking initially at 8 Nodes to start (3 Dedicated to MNG/iSCSI & 5 Dedicated to Storage)
MNG/iSCSI Nodes (3) would each have the following specs.
- Dual Intel Xeon E5-2630L v2 2.4Ghz 6C(12T) Processors
- 32GB DDR3-RDIMM ECC Memory
- 2x200GB SSD (RAID1) OS Drive
- 4x10Gbe (2x iSCSI, 2x Backend)
- 2x1Gbe LAG (Mng)
Storage Nodes (5) would each have the following specs.
- Dual Intel Xeon E5-2630L 2.0Ghz 6C(12T) Processors or Intel Xeon E5-2620 2.0Ghz 6C(12T) Processors
- 48GB DDR-RDIMM ECC Memory
- 1x80GB SSD OS Drive
- 1x400GB Intel P3600 NVMe PCIe (DB/WAL)
- 5x4TB 7.2k NLSAS Enterprise Drive
- 2x10Gbe (2x Backend)
- 2x1Gbe (Mng)
Is there any red flags with this config?
I am planning to have 2 pools initially. 3 Replicated (VM OS Drives, etc) & EC (Bulk / Low Performance VMs). I am familiar with the native CEPH RADOS performance calculations for said configuration and have read on the PetaSAN forums that I should expect about 75% of the RADOS performance in VMware. Is this still correct?
Any other tips or guidance would be appreciated.
EDIT: Forgot to add the storage nodes would be using HBAs for Drive interface. Would I see any benefit in doing a RAID controller with Battery Back Write Cache in JBOD/Individual Desk RAID 0 mode?
I have been using CEPH in production for the past 3 years and am very pleased with the flexibility and Scale-Out support it provides. I stumbled upon PetaSAN when looking for a clean easy to manage interface for CEPH that my Junior sysadmin's can feel comfortable monitoring our SAN but still be able to manually edit the configs if needed for tuning.
I am looking to re-purpose some existing hardware to build a new PetaSAN array that will support future expansion. This will be serving up iSCSI to a VMWare vCloud infrastructure. Some compute nodes will have NVMe vFlash (Read Cache) available as needed for workloads and performance.
I am looking initially at 8 Nodes to start (3 Dedicated to MNG/iSCSI & 5 Dedicated to Storage)
MNG/iSCSI Nodes (3) would each have the following specs.
- Dual Intel Xeon E5-2630L v2 2.4Ghz 6C(12T) Processors
- 32GB DDR3-RDIMM ECC Memory
- 2x200GB SSD (RAID1) OS Drive
- 4x10Gbe (2x iSCSI, 2x Backend)
- 2x1Gbe LAG (Mng)
Storage Nodes (5) would each have the following specs.
- Dual Intel Xeon E5-2630L 2.0Ghz 6C(12T) Processors or Intel Xeon E5-2620 2.0Ghz 6C(12T) Processors
- 48GB DDR-RDIMM ECC Memory
- 1x80GB SSD OS Drive
- 1x400GB Intel P3600 NVMe PCIe (DB/WAL)
- 5x4TB 7.2k NLSAS Enterprise Drive
- 2x10Gbe (2x Backend)
- 2x1Gbe (Mng)
Is there any red flags with this config?
I am planning to have 2 pools initially. 3 Replicated (VM OS Drives, etc) & EC (Bulk / Low Performance VMs). I am familiar with the native CEPH RADOS performance calculations for said configuration and have read on the PetaSAN forums that I should expect about 75% of the RADOS performance in VMware. Is this still correct?
Any other tips or guidance would be appreciated.
EDIT: Forgot to add the storage nodes would be using HBAs for Drive interface. Would I see any benefit in doing a RAID controller with Battery Back Write Cache in JBOD/Individual Desk RAID 0 mode?
admin
2,930 Posts
Quote from admin on June 14, 2019, 7:55 pmYou can also refer to
http://www.petasan.org/forums/?view=thread&id=478&part=1#postid-2824
for many of the points you raise. you should get over 75% native rados performance from your vms
You can also refer to
http://www.petasan.org/forums/?view=thread&id=478&part=1#postid-2824
for many of the points you raise. you should get over 75% native rados performance from your vms