Cluster Design and Performance
ryan
1 Post
March 15, 2023, 8:23 pmQuote from ryan on March 15, 2023, 8:23 pmI am looking at deploying a PetaSan cluster but am struggling to find any performance metrics.
I am looking at a 12 node cluster stretched across 2 datacenters (6 per site) connected by a 100GB link. Servers would be interconnected with a pair of 100GB switches.
Each node would looks something like:
- 1x AMD Epyx 16 core CPU 3Ghz+
- 128GB RAM
- 6x 3.84TB U.2 NVME
- Dual Port 100gbe NICS
There would be a couple of distinct workloads
- MSSQL via ISCSI, this would not be replicated between sites by SAN.
- VM workload via ISCSI this would not be replicated between sites by SAN.
- Fileservice, this has a reasonable amount of IO and would be replicated by SAN.
I am wandering what sort of performance in terms of IOPs and latency I would expect?
Thanks in advance.
I am looking at deploying a PetaSan cluster but am struggling to find any performance metrics.
I am looking at a 12 node cluster stretched across 2 datacenters (6 per site) connected by a 100GB link. Servers would be interconnected with a pair of 100GB switches.
Each node would looks something like:
- 1x AMD Epyx 16 core CPU 3Ghz+
- 128GB RAM
- 6x 3.84TB U.2 NVME
- Dual Port 100gbe NICS
There would be a couple of distinct workloads
- MSSQL via ISCSI, this would not be replicated between sites by SAN.
- VM workload via ISCSI this would not be replicated between sites by SAN.
- Fileservice, this has a reasonable amount of IO and would be replicated by SAN.
I am wandering what sort of performance in terms of IOPs and latency I would expect?
Thanks in advance.
admin
2,930 Posts
March 15, 2023, 8:53 pmQuote from admin on March 15, 2023, 8:53 pmAssume you use enterprise grade nvme and using 3x replication in each site and no cross site replication..read latency around 0.3 ms, write latency approx 1 ms. iops per host machine approx 40k read, 20k write (seen by client) depends a lot on cpu speed and number of cores. throughput MB/s will saturate the network.
Assume you use enterprise grade nvme and using 3x replication in each site and no cross site replication..read latency around 0.3 ms, write latency approx 1 ms. iops per host machine approx 40k read, 20k write (seen by client) depends a lot on cpu speed and number of cores. throughput MB/s will saturate the network.
Cluster Design and Performance
ryan
1 Post
Quote from ryan on March 15, 2023, 8:23 pmI am looking at deploying a PetaSan cluster but am struggling to find any performance metrics.
I am looking at a 12 node cluster stretched across 2 datacenters (6 per site) connected by a 100GB link. Servers would be interconnected with a pair of 100GB switches.
Each node would looks something like:
- 1x AMD Epyx 16 core CPU 3Ghz+
- 128GB RAM
- 6x 3.84TB U.2 NVME
- Dual Port 100gbe NICS
There would be a couple of distinct workloads
- MSSQL via ISCSI, this would not be replicated between sites by SAN.
- VM workload via ISCSI this would not be replicated between sites by SAN.
- Fileservice, this has a reasonable amount of IO and would be replicated by SAN.
I am wandering what sort of performance in terms of IOPs and latency I would expect?
Thanks in advance.
I am looking at deploying a PetaSan cluster but am struggling to find any performance metrics.
I am looking at a 12 node cluster stretched across 2 datacenters (6 per site) connected by a 100GB link. Servers would be interconnected with a pair of 100GB switches.
Each node would looks something like:
- 1x AMD Epyx 16 core CPU 3Ghz+
- 128GB RAM
- 6x 3.84TB U.2 NVME
- Dual Port 100gbe NICS
There would be a couple of distinct workloads
- MSSQL via ISCSI, this would not be replicated between sites by SAN.
- VM workload via ISCSI this would not be replicated between sites by SAN.
- Fileservice, this has a reasonable amount of IO and would be replicated by SAN.
I am wandering what sort of performance in terms of IOPs and latency I would expect?
Thanks in advance.
admin
2,930 Posts
Quote from admin on March 15, 2023, 8:53 pmAssume you use enterprise grade nvme and using 3x replication in each site and no cross site replication..read latency around 0.3 ms, write latency approx 1 ms. iops per host machine approx 40k read, 20k write (seen by client) depends a lot on cpu speed and number of cores. throughput MB/s will saturate the network.
Assume you use enterprise grade nvme and using 3x replication in each site and no cross site replication..read latency around 0.3 ms, write latency approx 1 ms. iops per host machine approx 40k read, 20k write (seen by client) depends a lot on cpu speed and number of cores. throughput MB/s will saturate the network.