Wrong Nodes in Benchmark test
killerodin
33 Posts
August 12, 2022, 1:48 pmQuote from killerodin on August 12, 2022, 1:48 pmHello
I've got the following problem:
If I want to take a benchmarktest of our pool B I got only the nodes from our Pool A. If I take a benchmark from Pool A I got the nodes from Pool A aswell.
How can I make a benchmark test from our pool B?
Thanks a lot
Hello
I've got the following problem:
If I want to take a benchmarktest of our pool B I got only the nodes from our Pool A. If I take a benchmark from Pool A I got the nodes from Pool A aswell.
How can I make a benchmark test from our pool B?
Thanks a lot
admin
2,930 Posts
August 12, 2022, 8:52 pmQuote from admin on August 12, 2022, 8:52 pmWe show performance on all nodes except the nodes you select to act as clients.
In an ideal world, you would run your client stress machines outside your storage servers, which is something you can do in PetaSAN by adding client nodes then removing them from cluster when test is done...but in real world you do not have extra nodes just to stress the cluster, so you need to select 1 or more existing cluster nodes to test the cluster. This nodes that simulate client workload are overloaded with both server and client tasks so their load stats report will not be correct and so they get excluded from the load reports.
We show performance on all nodes except the nodes you select to act as clients.
In an ideal world, you would run your client stress machines outside your storage servers, which is something you can do in PetaSAN by adding client nodes then removing them from cluster when test is done...but in real world you do not have extra nodes just to stress the cluster, so you need to select 1 or more existing cluster nodes to test the cluster. This nodes that simulate client workload are overloaded with both server and client tasks so their load stats report will not be correct and so they get excluded from the load reports.
killerodin
33 Posts
August 15, 2022, 6:55 amQuote from killerodin on August 15, 2022, 6:55 amHello
Thanks for the answer.
Then it's ok that I can see the performance from the nodes from the other pool but what can be the reason that the test don't show me the performance from our 4 HDD nodes? I don't use them as clients.
Hello
Thanks for the answer.
Then it's ok that I can see the performance from the nodes from the other pool but what can be the reason that the test don't show me the performance from our 4 HDD nodes? I don't use them as clients.
admin
2,930 Posts
August 16, 2022, 8:29 amQuote from admin on August 16, 2022, 8:29 amThis should not happen, you should see load stats on all nodes not acting as clients.
Do you see any errors in the /opt/petasan/log/PetaSAN.log files for the HDD nodes ?
It may also be a bug we have, are the HDD nodes using a by-host-dd crush rule from presets ? or do they have other rule, can you post
ceph osd tree
Note that you can always run the bencmark tests from UI, then for load on the nodes, look at the node stats charts to identify bottlenecks. In such cases it may be better to run tests for 5 min as we sample the charts 1 per min, so to get a nice graph.
This should not happen, you should see load stats on all nodes not acting as clients.
Do you see any errors in the /opt/petasan/log/PetaSAN.log files for the HDD nodes ?
It may also be a bug we have, are the HDD nodes using a by-host-dd crush rule from presets ? or do they have other rule, can you post
ceph osd tree
Note that you can always run the bencmark tests from UI, then for load on the nodes, look at the node stats charts to identify bottlenecks. In such cases it may be better to run tests for 5 min as we sample the charts 1 per min, so to get a nice graph.
Wrong Nodes in Benchmark test
killerodin
33 Posts
Quote from killerodin on August 12, 2022, 1:48 pmHello
I've got the following problem:
If I want to take a benchmarktest of our pool B I got only the nodes from our Pool A. If I take a benchmark from Pool A I got the nodes from Pool A aswell.
How can I make a benchmark test from our pool B?
Thanks a lot
Hello
I've got the following problem:
If I want to take a benchmarktest of our pool B I got only the nodes from our Pool A. If I take a benchmark from Pool A I got the nodes from Pool A aswell.
How can I make a benchmark test from our pool B?
Thanks a lot
admin
2,930 Posts
Quote from admin on August 12, 2022, 8:52 pmWe show performance on all nodes except the nodes you select to act as clients.
In an ideal world, you would run your client stress machines outside your storage servers, which is something you can do in PetaSAN by adding client nodes then removing them from cluster when test is done...but in real world you do not have extra nodes just to stress the cluster, so you need to select 1 or more existing cluster nodes to test the cluster. This nodes that simulate client workload are overloaded with both server and client tasks so their load stats report will not be correct and so they get excluded from the load reports.
We show performance on all nodes except the nodes you select to act as clients.
In an ideal world, you would run your client stress machines outside your storage servers, which is something you can do in PetaSAN by adding client nodes then removing them from cluster when test is done...but in real world you do not have extra nodes just to stress the cluster, so you need to select 1 or more existing cluster nodes to test the cluster. This nodes that simulate client workload are overloaded with both server and client tasks so their load stats report will not be correct and so they get excluded from the load reports.
killerodin
33 Posts
Quote from killerodin on August 15, 2022, 6:55 amHello
Thanks for the answer.
Then it's ok that I can see the performance from the nodes from the other pool but what can be the reason that the test don't show me the performance from our 4 HDD nodes? I don't use them as clients.
Hello
Thanks for the answer.
Then it's ok that I can see the performance from the nodes from the other pool but what can be the reason that the test don't show me the performance from our 4 HDD nodes? I don't use them as clients.
admin
2,930 Posts
Quote from admin on August 16, 2022, 8:29 amThis should not happen, you should see load stats on all nodes not acting as clients.
Do you see any errors in the /opt/petasan/log/PetaSAN.log files for the HDD nodes ?
It may also be a bug we have, are the HDD nodes using a by-host-dd crush rule from presets ? or do they have other rule, can you post
ceph osd tree
Note that you can always run the bencmark tests from UI, then for load on the nodes, look at the node stats charts to identify bottlenecks. In such cases it may be better to run tests for 5 min as we sample the charts 1 per min, so to get a nice graph.
This should not happen, you should see load stats on all nodes not acting as clients.
Do you see any errors in the /opt/petasan/log/PetaSAN.log files for the HDD nodes ?
It may also be a bug we have, are the HDD nodes using a by-host-dd crush rule from presets ? or do they have other rule, can you post
ceph osd tree
Note that you can always run the bencmark tests from UI, then for load on the nodes, look at the node stats charts to identify bottlenecks. In such cases it may be better to run tests for 5 min as we sample the charts 1 per min, so to get a nice graph.