High disk-utilisation with non workload
sgs29690
1 Post
May 28, 2021, 7:14 pmQuote from sgs29690 on May 28, 2021, 7:14 pmHello,
I 'm running a 3 node petasan cluster with 17 NVME-OSDs inside. Each node has 4x40GBit Intel NICs,20 Cores, 128GB. I 'm only using RDB/iSCSI. The highest troughoutput I've ever seen was poor 300MB/s vs Windows 2019 MPIO.
The Dashboard shows 100% disk-utilisation the whole time, no matter the value of Cluster troughoutput. The internal benchmark shows the same thing.
Disk
tps
rkB/s
wkB/s
areq-sz
aqu-sz
await
svctm
%util
nvme1n1
578
2230
1745
6
1
0
1
99
nvme0n1
1143
24223
2587
23
1
0
0
99
nvme2n1
345
1353
1169
7
1
0
2
99
nvme5n1
608
2467
357
4
1
0
1
99
nvme4n1
644
2565
1691
6
1
0
1
99
nvme6n1
455
1806
882
5
1
0
2
99
Cluster IOPS :
- Write 23561
- Read 53515
Node
Memory Util%
CPU Util%
Network Util%
Disks Util%
Actions
Avg
Max
Avg
Max
Journals
OSDs
Avg
Max
Avg
Max
psn03
29
25
31
1
2
0
0
100
100
What can I do?
Kind regards Hartmut
Hello,
I 'm running a 3 node petasan cluster with 17 NVME-OSDs inside. Each node has 4x40GBit Intel NICs,20 Cores, 128GB. I 'm only using RDB/iSCSI. The highest troughoutput I've ever seen was poor 300MB/s vs Windows 2019 MPIO.
The Dashboard shows 100% disk-utilisation the whole time, no matter the value of Cluster troughoutput. The internal benchmark shows the same thing.
Disk
tps
rkB/s
wkB/s
areq-sz
aqu-sz
await
svctm
%util
nvme1n1
578
2230
1745
6
1
0
1
99
nvme0n1
1143
24223
2587
23
1
0
0
99
nvme2n1
345
1353
1169
7
1
0
2
99
nvme5n1
608
2467
357
4
1
0
1
99
nvme4n1
644
2565
1691
6
1
0
1
99
nvme6n1
455
1806
882
5
1
0
2
99
Cluster IOPS :
- Write 23561
- Read 53515
Node
Memory Util%
CPU Util%
Network Util%
Disks Util%
Actions
Avg
Max
Avg
Max
Journals
OSDs
Avg
Max
Avg
Max
psn03
29
25
31
1
2
0
0
100
100
What can I do?
Kind regards Hartmut
Last edited on May 28, 2021, 7:22 pm by sgs29690 · #1
admin
2,930 Posts
May 29, 2021, 3:50 pmQuote from admin on May 29, 2021, 3:50 pmThis is kernel issue
https://github.com/sysstat/sysstat/issues/187
This is kernel issue
https://github.com/sysstat/sysstat/issues/187
High disk-utilisation with non workload
sgs29690
1 Post
Quote from sgs29690 on May 28, 2021, 7:14 pmHello,
I 'm running a 3 node petasan cluster with 17 NVME-OSDs inside. Each node has 4x40GBit Intel NICs,20 Cores, 128GB. I 'm only using RDB/iSCSI. The highest troughoutput I've ever seen was poor 300MB/s vs Windows 2019 MPIO.
The Dashboard shows 100% disk-utilisation the whole time, no matter the value of Cluster troughoutput. The internal benchmark shows the same thing.
Disk tps rkB/s wkB/s areq-sz aqu-sz await svctm %util nvme1n1 578 2230 1745 6 1 0 1 99 nvme0n1 1143 24223 2587 23 1 0 0 99 nvme2n1 345 1353 1169 7 1 0 2 99 nvme5n1 608 2467 357 4 1 0 1 99 nvme4n1 644 2565 1691 6 1 0 1 99 nvme6n1 455 1806 882 5 1 0 2 99 Cluster IOPS :
- Write 23561
- Read 53515
Node Memory Util% CPU Util% Network Util% Disks Util% Actions Avg Max Avg Max Journals OSDs Avg Max Avg Max psn03 29 25 31 1 2 0 0 100 100 What can I do?
Kind regards Hartmut
Hello,
I 'm running a 3 node petasan cluster with 17 NVME-OSDs inside. Each node has 4x40GBit Intel NICs,20 Cores, 128GB. I 'm only using RDB/iSCSI. The highest troughoutput I've ever seen was poor 300MB/s vs Windows 2019 MPIO.
The Dashboard shows 100% disk-utilisation the whole time, no matter the value of Cluster troughoutput. The internal benchmark shows the same thing.
| Disk | tps | rkB/s | wkB/s | areq-sz | aqu-sz | await | svctm | %util |
|---|---|---|---|---|---|---|---|---|
| nvme1n1 | 578 | 2230 | 1745 | 6 | 1 | 0 | 1 | 99 |
| nvme0n1 | 1143 | 24223 | 2587 | 23 | 1 | 0 | 0 | 99 |
| nvme2n1 | 345 | 1353 | 1169 | 7 | 1 | 0 | 2 | 99 |
| nvme5n1 | 608 | 2467 | 357 | 4 | 1 | 0 | 1 | 99 |
| nvme4n1 | 644 | 2565 | 1691 | 6 | 1 | 0 | 1 | 99 |
| nvme6n1 | 455 | 1806 | 882 | 5 | 1 | 0 | 2 | 99 |
- Write 23561
- Read 53515
| Node | Memory Util% | CPU Util% | Network Util% | Disks Util% | Actions | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| Avg | Max | Avg | Max | Journals | OSDs | |||||
| Avg | Max | Avg | Max | |||||||
| psn03 | 29 | 25 | 31 | 1 | 2 | 0 | 0 | 100 | 100 | |
What can I do?
Kind regards Hartmut
admin
2,930 Posts
Quote from admin on May 29, 2021, 3:50 pmThis is kernel issue
https://github.com/sysstat/sysstat/issues/187
This is kernel issue
https://github.com/sysstat/sysstat/issues/187