Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

No EC Crush Scripts in 2.7.3 Distro


jgraczyk
7 Posts
August 6, 2021, 6:10 pm

I just installed a new 4 node PetaSAN Cluster.  Given that I haven't successfully tested v2.8.0 yet and had trouble with a test install booting, I installed from the v2.7.3 Distro CD. After completing the install and deleting and re-creating an rbd pool (too many PGs per OSD), I attempted to create an EC Pool from the templates included.  While the templates seemed the same as v2.7.0-2.7.2 and found that there are no Crush Rules for EC pools included in the Distro.

Can anyone tell me how I can import the rules that used to be available on the v2.7.0 Distro into a V2.7.3 cluster?

Also, I'm interested in creating a 2-2 EC Profile (4 servers, go figure) and if it's simpler, could someone just tell me what a rule that separates 2 data and 2 parity chunks by host should look like?

Thanks,

I would hope I don't have to reinstall PetaSAN from the v2.7.0 CD and upgrade my way to v2.7.3 just to get the Crush Rules....

Any help would be appreciated.

Jim Graczyk

 

#1

admin
2,930 Posts
August 6, 2021, 6:56 pm

EC templates should be there, can you check
ls /opt/petasan/config/crush/rule_templates/

If there are on nodes 2 or 3, you can log to these nodes

else you can add a template yourself if you need, just add a file named ec-by-host-hdd ( or any other names ) with the below content, it will work with different profiles including 2+2 ( if you use ssd please change class )

{
# Placement rule per PG/object in EC Pool
# Choose K+M distinct hosts
# For each, choose a single HDD OSD disk

id 0
type erasure
min_size 3
max_size 20
step set_chooseleaf_tries 5
step set_choose_tries 100
step take default class hdd
step chooseleaf indep 0 type host
step emit
}

Last edited on August 6, 2021, 6:57 pm by admin · #2

jgraczyk
7 Posts
August 6, 2021, 7:31 pm

Thanks so much for the quick response -

The files are there but they don't appear in the web app when listing rules or as choices when creating EC Pools.

root@nwsan04:~# ls -l /opt/petasan/config/crush/rule_templates/
total 36
-rw-rw-r-- 1 root root 356 May 6 2020 by-host-hdd
-rw-rw-r-- 1 root root 356 May 6 2020 by-host-ssd
-rw-rw-r-- 1 root root 356 May 6 2020 by-rack-hdd
-rw-rw-r-- 1 root root 356 May 6 2020 by-rack-ssd
-rw-rw-r-- 1 root root 282 May 6 2020 ec-by-host-hdd
-rw-rw-r-- 1 root root 282 May 6 2020 ec-by-host-ssd
-rw-rw-r-- 1 root root 282 May 6 2020 ec-by-rack-hdd
-rw-rw-r-- 1 root root 282 May 6 2020 ec-by-rack-ssd
-rw-rw-r-- 1 root root 281 May 6 2020 example-2rooms-4replicas
root@nwsan04:~#

root@nwsan04:~# cat /opt/petasan/config/crush/rule_templates/ec-by-host-hdd
{
# Placement rule per PG/object in EC Pool
# Choose K+M distinct hosts
# For each, choose a single HDD OSD disk

id 0
type erasure
min_size 3
max_size 20
step set_chooseleaf_tries 5
step set_choose_tries 100
step take default class hdd
step chooseleaf indep 0 type host
step emit
}root@nwsan04:~#

The contents of this one, at least, looks correct.  I have no idea what the import method is, but I can copy and paste into a new rule to see if  I can it for use.  Also, not that web app would have to be running as root to be able to access these files (or so it seems to me).

Jim

#3

jgraczyk
7 Posts
August 6, 2021, 7:51 pm

Perhaps this is just all my ignorance.....

When I went to create a Crush Rule, the Templates pulldown is full of choices - the template files you told me to list. So a big DUH on my part.

I was under the impression that every EC-Profile included with PetaSAN had a corresponding Crush Rule already created.  I was looking for one that matched the K=3, M=2 (ec-32-profile), thinking that in PetaSAN, K+M had to be less than or equal to the number of PetaSAN nodes in the cluster (or we'd have too many egg in one or more of the baskets, so to speak).  I was looking for an existing 3-2 Crush rule to go with the corresponding profile, as an example to make a ec-22-profile.

Sorry about my misunderstanding.

That said, is the above rule you wrote usable of a any value of K and M?  If so, I obviously just need to use the one that matches - that would be ec-by-host-hdd, which yours appears to be and create my own rule.

Is this correct?

Jim Graczyk

 

#4

admin
2,930 Posts
August 6, 2021, 10:06 pm

Yes the rule will work with all profiles. to be exact: the rule works for any profile with size from 3 to 20. ec profile size is k+m. 2+2 profile has size 4, 4+2 has 6, so all will work

#5