Advice on hardware to buy for PetaSAN nodes
Ste
125 Posts
October 15, 2018, 3:22 pmQuote from Ste on October 15, 2018, 3:22 pmHello,
I would like to assemble a brand new 3 nodes (as a starting point) PetaSAN cluster, using commercial components available on the market.
My idea is to setup each node with one tiny SATA SSD for the OS (60 GB or less), 1 NVMe SSD 1 TB for the cache, and 4 (or possibly 5) rotating HD between 6 and 8 TB each for the OSDs.
I would like to have some hints/recommendation/suggestions on the type of hardware to buy, keeping in mind not to be too tight nor wasting money in useless features. In detail:
1) Which best motherboard can support at the same time 6 SATA drives + 1 NVMe SSD ?
2) An Intel Core i7 would be appropriate ? Are there recommended frequency and number of cores ?
3) How much RAM would be a good amount ?
4) I suppose no GPU as the onboard graphic should be enough, is it correct ?
5) Networking: I'm thinking to 10 Gb ports for iSCSI and 1 Gb for backend and management, is it correct ?
6) For the Hard Drives I have in mind "Western Digital RED".
7) Which power supply would be good for 4/5 HD and 2 SSD ? 1000 W are enough or better have more ?
Please any comment will be accepted, as I have to prepare the list of goods to buy and give a budget estimate to my boss.
Thank you in advance,
Ste
Hello,
I would like to assemble a brand new 3 nodes (as a starting point) PetaSAN cluster, using commercial components available on the market.
My idea is to setup each node with one tiny SATA SSD for the OS (60 GB or less), 1 NVMe SSD 1 TB for the cache, and 4 (or possibly 5) rotating HD between 6 and 8 TB each for the OSDs.
I would like to have some hints/recommendation/suggestions on the type of hardware to buy, keeping in mind not to be too tight nor wasting money in useless features. In detail:
1) Which best motherboard can support at the same time 6 SATA drives + 1 NVMe SSD ?
2) An Intel Core i7 would be appropriate ? Are there recommended frequency and number of cores ?
3) How much RAM would be a good amount ?
4) I suppose no GPU as the onboard graphic should be enough, is it correct ?
5) Networking: I'm thinking to 10 Gb ports for iSCSI and 1 Gb for backend and management, is it correct ?
6) For the Hard Drives I have in mind "Western Digital RED".
7) Which power supply would be good for 4/5 HD and 2 SSD ? 1000 W are enough or better have more ?
Please any comment will be accepted, as I have to prepare the list of goods to buy and give a budget estimate to my boss.
Thank you in advance,
Ste
Last edited on October 15, 2018, 3:23 pm by Ste · #1
admin
2,930 Posts
October 16, 2018, 9:29 amQuote from admin on October 16, 2018, 9:29 amFor performance sensitive virtualization workloads, the recommendation is to use all SSDs. With HDDs to get decent performance you would need a controller with cache + a relatively large number of HDDs (16, 24 ) per host, this is in addition to the journal device. This is because such workloads are latency sensitive and spinning disk latency plays a large factor in Ceph or any other SDS solution.
For workloads that read/write in large block sizes such as backups and video streaming, your setup would be good. Also if your workload is virtualization but is not latency critical then your setup may work too, you may need to try it and decide this yourself.
For cpu + ram requirement: do see the hardware recommendation guide
No gpu is needed
Backend network load is the combined load of your iSCSI, so it need to be at least 10G too, management can be 1G or can be shared on a 10G nic ( example shared with iSCSI 1)
For performance sensitive virtualization workloads, the recommendation is to use all SSDs. With HDDs to get decent performance you would need a controller with cache + a relatively large number of HDDs (16, 24 ) per host, this is in addition to the journal device. This is because such workloads are latency sensitive and spinning disk latency plays a large factor in Ceph or any other SDS solution.
For workloads that read/write in large block sizes such as backups and video streaming, your setup would be good. Also if your workload is virtualization but is not latency critical then your setup may work too, you may need to try it and decide this yourself.
For cpu + ram requirement: do see the hardware recommendation guide
No gpu is needed
Backend network load is the combined load of your iSCSI, so it need to be at least 10G too, management can be 1G or can be shared on a 10G nic ( example shared with iSCSI 1)
Ste
125 Posts
October 24, 2018, 2:31 pmQuote from Ste on October 24, 2018, 2:31 pmHi,
I'm planning to use 6 disks per node (5 OSD + 1 journal), so according to the guide the CPU should have 8 cores. But I couldn't find any Intel i7 processor with more than 6 cores. The latest i9-7900X (Skylake-X) has 10 cores but it costs around 1.000 Euros. Would it be so critical to use a 6-cores CPU, 2 core less than what it is recommended ?
Bye, Ste.
Hi,
I'm planning to use 6 disks per node (5 OSD + 1 journal), so according to the guide the CPU should have 8 cores. But I couldn't find any Intel i7 processor with more than 6 cores. The latest i9-7900X (Skylake-X) has 10 cores but it costs around 1.000 Euros. Would it be so critical to use a 6-cores CPU, 2 core less than what it is recommended ?
Bye, Ste.
Last edited on October 24, 2018, 2:31 pm by Ste · #3
admin
2,930 Posts
October 24, 2018, 3:04 pmQuote from admin on October 24, 2018, 3:04 pmI think it should be OK. Generally this requirement applies more to SSD OSDs, which is what we recommend, it is because in such systems cpu will saturate if you have a couple of fast disks, so the total iops depends linearly on the number of cores. In spinning disks, iops are lower and cores are less stressed.
I think it should be OK. Generally this requirement applies more to SSD OSDs, which is what we recommend, it is because in such systems cpu will saturate if you have a couple of fast disks, so the total iops depends linearly on the number of cores. In spinning disks, iops are lower and cores are less stressed.
Last edited on October 24, 2018, 3:07 pm by admin · #4
Ste
125 Posts
October 26, 2018, 10:51 amQuote from Ste on October 26, 2018, 10:51 amHi, regarding the 10 Gb/s card, I saw the following options:
- Dell Broadcom 57810 Dual Port 10Gb SFP+
- 10Gtek per Intel X520-DA2/ X520-SR2, Intel 82599ES Controller
- Intel 10 Gigabit AF da Dual Port SFP + E10G42AFDA PCIe
- 2-Port 10G Fiber Network Card with Open SFP+ - PCIe, Intel Chip
would they all work with PetaSAN, or there are special issues to be adviced on ?
Thanks again, Ste.
Hi, regarding the 10 Gb/s card, I saw the following options:
- Dell Broadcom 57810 Dual Port 10Gb SFP+
- 10Gtek per Intel X520-DA2/ X520-SR2, Intel 82599ES Controller
- Intel 10 Gigabit AF da Dual Port SFP + E10G42AFDA PCIe
- 2-Port 10G Fiber Network Card with Open SFP+ - PCIe, Intel Chip
would they all work with PetaSAN, or there are special issues to be adviced on ?
Thanks again, Ste.
Last edited on October 26, 2018, 10:56 am by Ste · #5
admin
2,930 Posts
October 26, 2018, 7:51 pmQuote from admin on October 26, 2018, 7:51 pmGenerally you can check how well these cards work under Linux. More specifically PetaSAN v 2.2 will ship with kernel 4.12 up streamed from SUSE SLE 15, you can check the SUSE recommended hardware list.
Generally you can check how well these cards work under Linux. More specifically PetaSAN v 2.2 will ship with kernel 4.12 up streamed from SUSE SLE 15, you can check the SUSE recommended hardware list.
Advice on hardware to buy for PetaSAN nodes
Ste
125 Posts
Quote from Ste on October 15, 2018, 3:22 pmHello,
I would like to assemble a brand new 3 nodes (as a starting point) PetaSAN cluster, using commercial components available on the market.
My idea is to setup each node with one tiny SATA SSD for the OS (60 GB or less), 1 NVMe SSD 1 TB for the cache, and 4 (or possibly 5) rotating HD between 6 and 8 TB each for the OSDs.I would like to have some hints/recommendation/suggestions on the type of hardware to buy, keeping in mind not to be too tight nor wasting money in useless features. In detail:
1) Which best motherboard can support at the same time 6 SATA drives + 1 NVMe SSD ?
2) An Intel Core i7 would be appropriate ? Are there recommended frequency and number of cores ?
3) How much RAM would be a good amount ?
4) I suppose no GPU as the onboard graphic should be enough, is it correct ?
5) Networking: I'm thinking to 10 Gb ports for iSCSI and 1 Gb for backend and management, is it correct ?
6) For the Hard Drives I have in mind "Western Digital RED".
7) Which power supply would be good for 4/5 HD and 2 SSD ? 1000 W are enough or better have more ?
Please any comment will be accepted, as I have to prepare the list of goods to buy and give a budget estimate to my boss.
Thank you in advance,
Ste
Hello,
I would like to assemble a brand new 3 nodes (as a starting point) PetaSAN cluster, using commercial components available on the market.
My idea is to setup each node with one tiny SATA SSD for the OS (60 GB or less), 1 NVMe SSD 1 TB for the cache, and 4 (or possibly 5) rotating HD between 6 and 8 TB each for the OSDs.
I would like to have some hints/recommendation/suggestions on the type of hardware to buy, keeping in mind not to be too tight nor wasting money in useless features. In detail:
1) Which best motherboard can support at the same time 6 SATA drives + 1 NVMe SSD ?
2) An Intel Core i7 would be appropriate ? Are there recommended frequency and number of cores ?
3) How much RAM would be a good amount ?
4) I suppose no GPU as the onboard graphic should be enough, is it correct ?
5) Networking: I'm thinking to 10 Gb ports for iSCSI and 1 Gb for backend and management, is it correct ?
6) For the Hard Drives I have in mind "Western Digital RED".
7) Which power supply would be good for 4/5 HD and 2 SSD ? 1000 W are enough or better have more ?
Please any comment will be accepted, as I have to prepare the list of goods to buy and give a budget estimate to my boss.
Thank you in advance,
Ste
admin
2,930 Posts
Quote from admin on October 16, 2018, 9:29 amFor performance sensitive virtualization workloads, the recommendation is to use all SSDs. With HDDs to get decent performance you would need a controller with cache + a relatively large number of HDDs (16, 24 ) per host, this is in addition to the journal device. This is because such workloads are latency sensitive and spinning disk latency plays a large factor in Ceph or any other SDS solution.
For workloads that read/write in large block sizes such as backups and video streaming, your setup would be good. Also if your workload is virtualization but is not latency critical then your setup may work too, you may need to try it and decide this yourself.
For cpu + ram requirement: do see the hardware recommendation guide
No gpu is needed
Backend network load is the combined load of your iSCSI, so it need to be at least 10G too, management can be 1G or can be shared on a 10G nic ( example shared with iSCSI 1)
For performance sensitive virtualization workloads, the recommendation is to use all SSDs. With HDDs to get decent performance you would need a controller with cache + a relatively large number of HDDs (16, 24 ) per host, this is in addition to the journal device. This is because such workloads are latency sensitive and spinning disk latency plays a large factor in Ceph or any other SDS solution.
For workloads that read/write in large block sizes such as backups and video streaming, your setup would be good. Also if your workload is virtualization but is not latency critical then your setup may work too, you may need to try it and decide this yourself.
For cpu + ram requirement: do see the hardware recommendation guide
No gpu is needed
Backend network load is the combined load of your iSCSI, so it need to be at least 10G too, management can be 1G or can be shared on a 10G nic ( example shared with iSCSI 1)
Ste
125 Posts
Quote from Ste on October 24, 2018, 2:31 pmHi,
I'm planning to use 6 disks per node (5 OSD + 1 journal), so according to the guide the CPU should have 8 cores. But I couldn't find any Intel i7 processor with more than 6 cores. The latest i9-7900X (Skylake-X) has 10 cores but it costs around 1.000 Euros. Would it be so critical to use a 6-cores CPU, 2 core less than what it is recommended ?
Bye, Ste.
Hi,
I'm planning to use 6 disks per node (5 OSD + 1 journal), so according to the guide the CPU should have 8 cores. But I couldn't find any Intel i7 processor with more than 6 cores. The latest i9-7900X (Skylake-X) has 10 cores but it costs around 1.000 Euros. Would it be so critical to use a 6-cores CPU, 2 core less than what it is recommended ?
Bye, Ste.
admin
2,930 Posts
Quote from admin on October 24, 2018, 3:04 pmI think it should be OK. Generally this requirement applies more to SSD OSDs, which is what we recommend, it is because in such systems cpu will saturate if you have a couple of fast disks, so the total iops depends linearly on the number of cores. In spinning disks, iops are lower and cores are less stressed.
I think it should be OK. Generally this requirement applies more to SSD OSDs, which is what we recommend, it is because in such systems cpu will saturate if you have a couple of fast disks, so the total iops depends linearly on the number of cores. In spinning disks, iops are lower and cores are less stressed.
Ste
125 Posts
Quote from Ste on October 26, 2018, 10:51 amHi, regarding the 10 Gb/s card, I saw the following options:
- Dell Broadcom 57810 Dual Port 10Gb SFP+
- 10Gtek per Intel X520-DA2/ X520-SR2, Intel 82599ES Controller
- Intel 10 Gigabit AF da Dual Port SFP + E10G42AFDA PCIe
- 2-Port 10G Fiber Network Card with Open SFP+ - PCIe, Intel Chip
would they all work with PetaSAN, or there are special issues to be adviced on ?
Thanks again, Ste.
Hi, regarding the 10 Gb/s card, I saw the following options:
- Dell Broadcom 57810 Dual Port 10Gb SFP+
- 10Gtek per Intel X520-DA2/ X520-SR2, Intel 82599ES Controller
- Intel 10 Gigabit AF da Dual Port SFP + E10G42AFDA PCIe
- 2-Port 10G Fiber Network Card with Open SFP+ - PCIe, Intel Chip
would they all work with PetaSAN, or there are special issues to be adviced on ?
Thanks again, Ste.
admin
2,930 Posts
Quote from admin on October 26, 2018, 7:51 pmGenerally you can check how well these cards work under Linux. More specifically PetaSAN v 2.2 will ship with kernel 4.12 up streamed from SUSE SLE 15, you can check the SUSE recommended hardware list.
Generally you can check how well these cards work under Linux. More specifically PetaSAN v 2.2 will ship with kernel 4.12 up streamed from SUSE SLE 15, you can check the SUSE recommended hardware list.