Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

Sanity Check on New Build

I have been using CEPH in production for the past 3 years and am very pleased with the flexibility and Scale-Out support it provides.  I stumbled upon PetaSAN when looking for a clean easy to manage interface for CEPH that my Junior sysadmin's can feel comfortable monitoring our SAN but still be able to manually edit the configs if needed for tuning.

I am looking to re-purpose some existing hardware to build a new PetaSAN array that will support future expansion.  This will be serving up iSCSI to a VMWare vCloud infrastructure.  Some compute nodes will have NVMe vFlash (Read Cache) available as needed for workloads and performance.

I am looking initially at 8 Nodes to start (3 Dedicated to MNG/iSCSI & 5 Dedicated to Storage)

MNG/iSCSI Nodes (3) would each have the following specs.

  • Dual Intel Xeon E5-2630L v2 2.4Ghz 6C(12T) Processors
  • 32GB DDR3-RDIMM ECC Memory
  • 2x200GB SSD (RAID1) OS Drive
  • 4x10Gbe (2x iSCSI, 2x Backend)
  • 2x1Gbe LAG (Mng)

Storage Nodes (5) would each have the following specs.

  • Dual Intel Xeon E5-2630L 2.0Ghz 6C(12T) Processors or Intel Xeon E5-2620 2.0Ghz 6C(12T) Processors
  • 48GB DDR-RDIMM ECC Memory
  • 1x80GB SSD OS Drive
  • 1x400GB Intel P3600 NVMe PCIe (DB/WAL)
  • 5x4TB 7.2k NLSAS Enterprise Drive
  • 2x10Gbe (2x Backend)
  • 2x1Gbe (Mng)

Is there any red flags with this config?

I am planning to have 2 pools initially. 3 Replicated (VM OS Drives, etc) & EC (Bulk / Low Performance VMs).  I am familiar with the native CEPH RADOS performance calculations for said configuration and have read on the PetaSAN forums that I should expect about 75% of the RADOS performance in VMware. Is this still correct?

Any other tips or guidance would be appreciated.

EDIT: Forgot to add the storage nodes would be using HBAs for Drive interface.  Would I see any benefit in doing a RAID controller with Battery Back Write Cache in JBOD/Individual Desk RAID 0 mode?

You can also refer to

http://www.petasan.org/forums/?view=thread&id=478&part=1#postid-2824

for many of the points you raise. you should get over 75% native rados performance from your vms