Reconfiguration a node and general questions
Pages: 1 2
sgorla
6 Posts
August 9, 2019, 11:28 pmQuote from sgorla on August 9, 2019, 11:28 pmI really want to thank you for your assistance. It is a pleasure to work with people as dedicated as you.
I don't have solid state units. The hardware I have is from the laboratory and they are equipment that is no longer used for production, however it is much better than a conventional PC.
The disks you select for journal are 15K SAS.
I share the hardware information, all nodes are relatively similar:
root @ PDSTCL01: ~ # hwinfo --short
cpu:
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
keyboard:
/ dev / input / event0 ATEN International ATEN-COMPOSITE
mouse:
/ dev / input / mice ATEN International ATEN-COMPOSITE
graphics card:
ATI ES1000 515E
storage:
Intel 631xESB / 632xESB / 3100 Chipset SATA IDE Controller
Dell PERC 6 / i Integrated RAID Controller
network:
eth1 Broadcom NetXtreme II BCM5708 Gigabit Ethernet
eth2 Intel 82545GM Gigabit Ethernet Controller
eth0 Broadcom NetXtreme II BCM5708 Gigabit Ethernet
network interface:
eth0 Ethernet network interface
Loopback network interface
eth1 Ethernet network interface
eth2 Ethernet network interface
disk:
/ dev / sdf DELL PERC 6 / i
/ dev / sdd DELL PERC 6 / i
/ dev / sdb DELL PERC 6 / i
/ dev / sdg DELL PERC 6 / i
/ dev / sde DELL PERC 6 / i
/ dev / sdc DELL PERC 6 / i
/ dev / sda DELL PERC 6 / i
/ dev / sdh DELL PERC 6 / i
partition:
/ dev / sdf1 Partition
/ dev / sdd1 Partition
/ dev / sdd2 Partition
/ dev / sdb1 Partition
/ dev / sdb2 Partition
/ dev / sdb3 Partition
/ dev / sdg1 Partition
/ dev / sde1 Partition
/ dev / sdc1 Partition
/ dev / sda1 Partition
/ dev / sda2 Partition
/ dev / sda3 Partition
/ dev / sda4 Partition
/ dev / sda5 Partition
/ dev / sdh1 Partition
CD ROM:
/ dev / sr0 TSSTcorp DVD + -RW TS-H653F
usb controller:
Intel 631xESB / 632xESB / 3100 Chipset UHCI USB Controller # 3
Intel 631xESB / 632xESB / 3100 UHCI USB Controller Chipset # 1
Intel 631xESB / 632xESB / 3100 Chipset EHCI USB2 Controller
Intel 631xESB / 632xESB / 3100 Chipset UHCI USB Controller # 4
Intel 631xESB / 632xESB / 3100 UHCI USB Controller Chipset # 2
bios:
BIOS
bridge:
Intel 6311ESB / 6321ESB PCI Express Downstream Port E2
Intel 82801 PCI Bridge
Intel 5000X Chipset Memory Controller Hub
Intel 5000 Series PCI Express x4 Port 3 Chipset
Broadcom EPB PCI-Express to PCI-X Bridge
Intel 5000 Series Chipset FSB Registers
Intel 5000 Series Chipset FBD Registers
Intel 5000 Series PCI Express x8 Port 6-7 Chipset
Intel 6311ESB / 6321ESB PCI Express Downstream Port E1
Intel 5000 Series Chipset Reserved Registers
Intel 5000 Series PCI Express x4 Port 2 Chipset
Intel 6311ESB / 6321ESB PCI Express Upstream Port
Intel 5000 Series PCI Express x4 Port 5 Chipset
Intel 631xESB / 632xESB / 3100 Chipset PCI Express Root Port 1
Intel 5000 Series Chipset FSB Registers
Intel 631xESB / 632xESB / 3100 Chipset LPC Interface Controller
Intel 5000 Series Chipset FSB Registers
Broadcom EPB PCI-Express to PCI-X Bridge
Intel 5000 Series Chipset Reserved Registers
Intel 6311ESB / 6321ESB PCI Express to PCI-X Bridge
Intel 5000 Series PCI Express x4 Port 4 Chipset
Intel 5000 Series Chipset FBD Registers
Intel 5000 Series PCI Express x4 Port 7 Chipset
How do I enable this setting? on the server controller?
"enable caching on the client, application side"
Thank you!
I really want to thank you for your assistance. It is a pleasure to work with people as dedicated as you.
I don't have solid state units. The hardware I have is from the laboratory and they are equipment that is no longer used for production, however it is much better than a conventional PC.
The disks you select for journal are 15K SAS.
I share the hardware information, all nodes are relatively similar:
root @ PDSTCL01: ~ # hwinfo --short
cpu:
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
keyboard:
/ dev / input / event0 ATEN International ATEN-COMPOSITE
mouse:
/ dev / input / mice ATEN International ATEN-COMPOSITE
graphics card:
ATI ES1000 515E
storage:
Intel 631xESB / 632xESB / 3100 Chipset SATA IDE Controller
Dell PERC 6 / i Integrated RAID Controller
network:
eth1 Broadcom NetXtreme II BCM5708 Gigabit Ethernet
eth2 Intel 82545GM Gigabit Ethernet Controller
eth0 Broadcom NetXtreme II BCM5708 Gigabit Ethernet
network interface:
eth0 Ethernet network interface
Loopback network interface
eth1 Ethernet network interface
eth2 Ethernet network interface
disk:
/ dev / sdf DELL PERC 6 / i
/ dev / sdd DELL PERC 6 / i
/ dev / sdb DELL PERC 6 / i
/ dev / sdg DELL PERC 6 / i
/ dev / sde DELL PERC 6 / i
/ dev / sdc DELL PERC 6 / i
/ dev / sda DELL PERC 6 / i
/ dev / sdh DELL PERC 6 / i
partition:
/ dev / sdf1 Partition
/ dev / sdd1 Partition
/ dev / sdd2 Partition
/ dev / sdb1 Partition
/ dev / sdb2 Partition
/ dev / sdb3 Partition
/ dev / sdg1 Partition
/ dev / sde1 Partition
/ dev / sdc1 Partition
/ dev / sda1 Partition
/ dev / sda2 Partition
/ dev / sda3 Partition
/ dev / sda4 Partition
/ dev / sda5 Partition
/ dev / sdh1 Partition
CD ROM:
/ dev / sr0 TSSTcorp DVD + -RW TS-H653F
usb controller:
Intel 631xESB / 632xESB / 3100 Chipset UHCI USB Controller # 3
Intel 631xESB / 632xESB / 3100 UHCI USB Controller Chipset # 1
Intel 631xESB / 632xESB / 3100 Chipset EHCI USB2 Controller
Intel 631xESB / 632xESB / 3100 Chipset UHCI USB Controller # 4
Intel 631xESB / 632xESB / 3100 UHCI USB Controller Chipset # 2
bios:
BIOS
bridge:
Intel 6311ESB / 6321ESB PCI Express Downstream Port E2
Intel 82801 PCI Bridge
Intel 5000X Chipset Memory Controller Hub
Intel 5000 Series PCI Express x4 Port 3 Chipset
Broadcom EPB PCI-Express to PCI-X Bridge
Intel 5000 Series Chipset FSB Registers
Intel 5000 Series Chipset FBD Registers
Intel 5000 Series PCI Express x8 Port 6-7 Chipset
Intel 6311ESB / 6321ESB PCI Express Downstream Port E1
Intel 5000 Series Chipset Reserved Registers
Intel 5000 Series PCI Express x4 Port 2 Chipset
Intel 6311ESB / 6321ESB PCI Express Upstream Port
Intel 5000 Series PCI Express x4 Port 5 Chipset
Intel 631xESB / 632xESB / 3100 Chipset PCI Express Root Port 1
Intel 5000 Series Chipset FSB Registers
Intel 631xESB / 632xESB / 3100 Chipset LPC Interface Controller
Intel 5000 Series Chipset FSB Registers
Broadcom EPB PCI-Express to PCI-X Bridge
Intel 5000 Series Chipset Reserved Registers
Intel 6311ESB / 6321ESB PCI Express to PCI-X Bridge
Intel 5000 Series PCI Express x4 Port 4 Chipset
Intel 5000 Series Chipset FBD Registers
Intel 5000 Series PCI Express x4 Port 7 Chipset
How do I enable this setting? on the server controller?
"enable caching on the client, application side"
Thank you!
admin
2,930 Posts
August 10, 2019, 12:28 amQuote from admin on August 10, 2019, 12:28 amNo problem, i hope it helped.
The 15K SAS will not replace an ssd as a journal, in fact thay may help a small bit over raw hdd devices but may even give negative results if you try to have a single journal disk serve more than 1 hdd osd (which is typical when using ssds). I noticed under iops load, the journals were 100% busy ie they were slowing the system.
1G network is definitely a bottleneck, if you can upgrade something than this is it. Even if you bond 1G together, they will increase total io of all streams, but a single stream (like a single process dd command) will still be limited by 1Gbps. Note that with 3x replication, your available bandwidth is further reduced.
The client caching is client os specific, for linux you can have your dd command either write directly or make use of the linux page cache
No problem, i hope it helped.
The 15K SAS will not replace an ssd as a journal, in fact thay may help a small bit over raw hdd devices but may even give negative results if you try to have a single journal disk serve more than 1 hdd osd (which is typical when using ssds). I noticed under iops load, the journals were 100% busy ie they were slowing the system.
1G network is definitely a bottleneck, if you can upgrade something than this is it. Even if you bond 1G together, they will increase total io of all streams, but a single stream (like a single process dd command) will still be limited by 1Gbps. Note that with 3x replication, your available bandwidth is further reduced.
The client caching is client os specific, for linux you can have your dd command either write directly or make use of the linux page cache
Pages: 1 2
Reconfiguration a node and general questions
sgorla
6 Posts
Quote from sgorla on August 9, 2019, 11:28 pmI really want to thank you for your assistance. It is a pleasure to work with people as dedicated as you.
I don't have solid state units. The hardware I have is from the laboratory and they are equipment that is no longer used for production, however it is much better than a conventional PC.
The disks you select for journal are 15K SAS.I share the hardware information, all nodes are relatively similar:
root @ PDSTCL01: ~ # hwinfo --short
cpu:
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
keyboard:
/ dev / input / event0 ATEN International ATEN-COMPOSITE
mouse:
/ dev / input / mice ATEN International ATEN-COMPOSITE
graphics card:
ATI ES1000 515E
storage:
Intel 631xESB / 632xESB / 3100 Chipset SATA IDE Controller
Dell PERC 6 / i Integrated RAID Controller
network:
eth1 Broadcom NetXtreme II BCM5708 Gigabit Ethernet
eth2 Intel 82545GM Gigabit Ethernet Controller
eth0 Broadcom NetXtreme II BCM5708 Gigabit Ethernet
network interface:
eth0 Ethernet network interface
Loopback network interface
eth1 Ethernet network interface
eth2 Ethernet network interface
disk:
/ dev / sdf DELL PERC 6 / i
/ dev / sdd DELL PERC 6 / i
/ dev / sdb DELL PERC 6 / i
/ dev / sdg DELL PERC 6 / i
/ dev / sde DELL PERC 6 / i
/ dev / sdc DELL PERC 6 / i
/ dev / sda DELL PERC 6 / i
/ dev / sdh DELL PERC 6 / i
partition:
/ dev / sdf1 Partition
/ dev / sdd1 Partition
/ dev / sdd2 Partition
/ dev / sdb1 Partition
/ dev / sdb2 Partition
/ dev / sdb3 Partition
/ dev / sdg1 Partition
/ dev / sde1 Partition
/ dev / sdc1 Partition
/ dev / sda1 Partition
/ dev / sda2 Partition
/ dev / sda3 Partition
/ dev / sda4 Partition
/ dev / sda5 Partition
/ dev / sdh1 Partition
CD ROM:
/ dev / sr0 TSSTcorp DVD + -RW TS-H653F
usb controller:
Intel 631xESB / 632xESB / 3100 Chipset UHCI USB Controller # 3
Intel 631xESB / 632xESB / 3100 UHCI USB Controller Chipset # 1
Intel 631xESB / 632xESB / 3100 Chipset EHCI USB2 Controller
Intel 631xESB / 632xESB / 3100 Chipset UHCI USB Controller # 4
Intel 631xESB / 632xESB / 3100 UHCI USB Controller Chipset # 2
bios:
BIOS
bridge:
Intel 6311ESB / 6321ESB PCI Express Downstream Port E2
Intel 82801 PCI Bridge
Intel 5000X Chipset Memory Controller Hub
Intel 5000 Series PCI Express x4 Port 3 Chipset
Broadcom EPB PCI-Express to PCI-X Bridge
Intel 5000 Series Chipset FSB Registers
Intel 5000 Series Chipset FBD Registers
Intel 5000 Series PCI Express x8 Port 6-7 Chipset
Intel 6311ESB / 6321ESB PCI Express Downstream Port E1
Intel 5000 Series Chipset Reserved Registers
Intel 5000 Series PCI Express x4 Port 2 Chipset
Intel 6311ESB / 6321ESB PCI Express Upstream Port
Intel 5000 Series PCI Express x4 Port 5 Chipset
Intel 631xESB / 632xESB / 3100 Chipset PCI Express Root Port 1
Intel 5000 Series Chipset FSB Registers
Intel 631xESB / 632xESB / 3100 Chipset LPC Interface Controller
Intel 5000 Series Chipset FSB Registers
Broadcom EPB PCI-Express to PCI-X Bridge
Intel 5000 Series Chipset Reserved Registers
Intel 6311ESB / 6321ESB PCI Express to PCI-X Bridge
Intel 5000 Series PCI Express x4 Port 4 Chipset
Intel 5000 Series Chipset FBD Registers
Intel 5000 Series PCI Express x4 Port 7 ChipsetHow do I enable this setting? on the server controller?
"enable caching on the client, application side"
Thank you!
I really want to thank you for your assistance. It is a pleasure to work with people as dedicated as you.
I don't have solid state units. The hardware I have is from the laboratory and they are equipment that is no longer used for production, however it is much better than a conventional PC.
The disks you select for journal are 15K SAS.
I share the hardware information, all nodes are relatively similar:
root @ PDSTCL01: ~ # hwinfo --short
cpu:
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
Intel (R) Xeon (R) CPU E5405 @ 2.00GHz, 1995 MHz
keyboard:
/ dev / input / event0 ATEN International ATEN-COMPOSITE
mouse:
/ dev / input / mice ATEN International ATEN-COMPOSITE
graphics card:
ATI ES1000 515E
storage:
Intel 631xESB / 632xESB / 3100 Chipset SATA IDE Controller
Dell PERC 6 / i Integrated RAID Controller
network:
eth1 Broadcom NetXtreme II BCM5708 Gigabit Ethernet
eth2 Intel 82545GM Gigabit Ethernet Controller
eth0 Broadcom NetXtreme II BCM5708 Gigabit Ethernet
network interface:
eth0 Ethernet network interface
Loopback network interface
eth1 Ethernet network interface
eth2 Ethernet network interface
disk:
/ dev / sdf DELL PERC 6 / i
/ dev / sdd DELL PERC 6 / i
/ dev / sdb DELL PERC 6 / i
/ dev / sdg DELL PERC 6 / i
/ dev / sde DELL PERC 6 / i
/ dev / sdc DELL PERC 6 / i
/ dev / sda DELL PERC 6 / i
/ dev / sdh DELL PERC 6 / i
partition:
/ dev / sdf1 Partition
/ dev / sdd1 Partition
/ dev / sdd2 Partition
/ dev / sdb1 Partition
/ dev / sdb2 Partition
/ dev / sdb3 Partition
/ dev / sdg1 Partition
/ dev / sde1 Partition
/ dev / sdc1 Partition
/ dev / sda1 Partition
/ dev / sda2 Partition
/ dev / sda3 Partition
/ dev / sda4 Partition
/ dev / sda5 Partition
/ dev / sdh1 Partition
CD ROM:
/ dev / sr0 TSSTcorp DVD + -RW TS-H653F
usb controller:
Intel 631xESB / 632xESB / 3100 Chipset UHCI USB Controller # 3
Intel 631xESB / 632xESB / 3100 UHCI USB Controller Chipset # 1
Intel 631xESB / 632xESB / 3100 Chipset EHCI USB2 Controller
Intel 631xESB / 632xESB / 3100 Chipset UHCI USB Controller # 4
Intel 631xESB / 632xESB / 3100 UHCI USB Controller Chipset # 2
bios:
BIOS
bridge:
Intel 6311ESB / 6321ESB PCI Express Downstream Port E2
Intel 82801 PCI Bridge
Intel 5000X Chipset Memory Controller Hub
Intel 5000 Series PCI Express x4 Port 3 Chipset
Broadcom EPB PCI-Express to PCI-X Bridge
Intel 5000 Series Chipset FSB Registers
Intel 5000 Series Chipset FBD Registers
Intel 5000 Series PCI Express x8 Port 6-7 Chipset
Intel 6311ESB / 6321ESB PCI Express Downstream Port E1
Intel 5000 Series Chipset Reserved Registers
Intel 5000 Series PCI Express x4 Port 2 Chipset
Intel 6311ESB / 6321ESB PCI Express Upstream Port
Intel 5000 Series PCI Express x4 Port 5 Chipset
Intel 631xESB / 632xESB / 3100 Chipset PCI Express Root Port 1
Intel 5000 Series Chipset FSB Registers
Intel 631xESB / 632xESB / 3100 Chipset LPC Interface Controller
Intel 5000 Series Chipset FSB Registers
Broadcom EPB PCI-Express to PCI-X Bridge
Intel 5000 Series Chipset Reserved Registers
Intel 6311ESB / 6321ESB PCI Express to PCI-X Bridge
Intel 5000 Series PCI Express x4 Port 4 Chipset
Intel 5000 Series Chipset FBD Registers
Intel 5000 Series PCI Express x4 Port 7 Chipset
How do I enable this setting? on the server controller?
"enable caching on the client, application side"
Thank you!
admin
2,930 Posts
Quote from admin on August 10, 2019, 12:28 amNo problem, i hope it helped.
The 15K SAS will not replace an ssd as a journal, in fact thay may help a small bit over raw hdd devices but may even give negative results if you try to have a single journal disk serve more than 1 hdd osd (which is typical when using ssds). I noticed under iops load, the journals were 100% busy ie they were slowing the system.
1G network is definitely a bottleneck, if you can upgrade something than this is it. Even if you bond 1G together, they will increase total io of all streams, but a single stream (like a single process dd command) will still be limited by 1Gbps. Note that with 3x replication, your available bandwidth is further reduced.
The client caching is client os specific, for linux you can have your dd command either write directly or make use of the linux page cache
No problem, i hope it helped.
The 15K SAS will not replace an ssd as a journal, in fact thay may help a small bit over raw hdd devices but may even give negative results if you try to have a single journal disk serve more than 1 hdd osd (which is typical when using ssds). I noticed under iops load, the journals were 100% busy ie they were slowing the system.
1G network is definitely a bottleneck, if you can upgrade something than this is it. Even if you bond 1G together, they will increase total io of all streams, but a single stream (like a single process dd command) will still be limited by 1Gbps. Note that with 3x replication, your available bandwidth is further reduced.
The client caching is client os specific, for linux you can have your dd command either write directly or make use of the linux page cache