Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

OSD on System Disk?


jcid
14 Posts
February 10, 2020, 10:43 pm

Hello,

I have a 3 node PetaSAN cluster with 32CPU and 256GB Ram on each host however there are only 4 disk bays per host. I currently have 2.1TB of Raid 5 (4 800GB SSD) on each host. Can i create OSD's on the system disk and use the 2Tb free on each hosts system disk for the SAN storage? This would be ideal in my situation. If this is possible can someone please list the commands to achieve this?

Thanks!!

Last edited on February 10, 2020, 10:44 pm by jcid · #1

admin
2,930 Posts
February 10, 2020, 11:14 pm

This is not possible in PetaSAN.  it is also not something you will find in Ceph production installations, it is better to have disks for storage dedicated for that purpose.

#2

jcid
14 Posts
February 10, 2020, 11:33 pm

Thanks,

Out of curiosity i can rebuild the systems with 2 mirrored Boot disks and two 3TB DATA disks each. Would the 2 3TB Data disks on each host be replicated across all three hosts then?

 

thanks.

#3

admin
2,930 Posts
February 10, 2020, 11:54 pm

Data will be replicated. The default replication factor is 3, so each data is written to 3 nodes. If you had 10 nodes, each data object/disk sector  will be written to 3 nodes, different objects/sectors will be written to different combination of 3 nodes.

#4

jcid
14 Posts
February 11, 2020, 12:25 am

ok,

so on each host I dont really need to mirror (raid1) the two disks then. I can add them to PetaSAN individually to the storage pool and they will replicate not needing the mirroring on each host then? If thats the case it will give me double the space then.

Last edited on February 11, 2020, 1:16 am by jcid · #5

admin
2,930 Posts
February 11, 2020, 8:58 am

Yes, not raid your data / osd disks, the system will handle redundancy + data healing by itself.

You can raid 1 your os disk, it is better but not a must as the system can tolerate failures

#6

jcid
14 Posts
February 11, 2020, 2:51 pm

Ok can I use 2 DATA disks or 1 DATA disk and 1 SSD for cashing since I only have 2 bays? The system also has 128Gb Ram so dont know if it uses any of that for cache.

 

Thanks for the help!!

Last edited on February 11, 2020, 2:52 pm by jcid · #7

jcid
14 Posts
February 11, 2020, 4:30 pm

So I am booting all 3 nodes now on raid1 SSD. Then I have a standalone 800G SSD and 3TB Disk.. The two standalone disks are seen at boot but not in PetaSAN.

Any ideas why this would be?

megacli shows them but there not available in the OS.

 

root@nbufsan1a:/usr/src# megacli -pdList -a0 | grep -i -E "slot|RAW Size|pd type"
Slot Number: 0
PD Type: SATA
Raw Size: 745.211 GB [0x5d26ceb0 Sectors]
Slot Number: 1
PD Type: SATA
Raw Size: 745.211 GB [0x5d26ceb0 Sectors]
Slot Number: 2
PD Type: SATA
Raw Size: 745.211 GB [0x5d26ceb0 Sectors]
Slot Number: 3
PD Type: SATA
Raw Size: 2.728 TB [0x15d50a3b0 Sectors]

 

 

Thanks!!

Last edited on February 11, 2020, 4:53 pm by jcid · #8

jcid
14 Posts
February 11, 2020, 5:17 pm

Disregard. Had to make them RAID0 with megacli to be recognized.

#9