Please add selectable journal size during OSD provisioning.
rhamon
30 Posts
August 29, 2022, 6:29 pmQuote from rhamon on August 29, 2022, 6:29 pmSalutations
Please add selectable journal size in the GUI for OSD provisioning as for S3 workload the recommended journal size is at least 4% of the OSD which for large multi TB drives the default 60GB does not meet.
I can of course work around the issue by manually managing the OSD provisioning but would make life that much better if in the gui. 😉
Ref:Â https://docs.ceph.com/en/quincy/rados/configuration/bluestore-config-ref/#sizing
<<For RGW workloads, it is recommended that the block.db
 size isn’t smaller than 4% of block
, because RGW heavily uses it to store metadata (omap keys).>>
Cheers and thanks!
Salutations
Please add selectable journal size in the GUI for OSD provisioning as for S3 workload the recommended journal size is at least 4% of the OSD which for large multi TB drives the default 60GB does not meet.
I can of course work around the issue by manually managing the OSD provisioning but would make life that much better if in the gui. 😉
Ref:Â https://docs.ceph.com/en/quincy/rados/configuration/bluestore-config-ref/#sizing
<<For RGW workloads, it is recommended that the block.db
 size isn’t smaller than 4% of block
, because RGW heavily uses it to store metadata (omap keys).>>
Cheers and thanks!
admin
2,921 Posts
August 30, 2022, 8:09 amQuote from admin on August 30, 2022, 8:09 amThanks for the feedback, which does make things easier.
Currently we set the size to 60 GB, if you wish to change it, you can change the config variable
bluestore_block_db_size
Note that the 4% is questionable and there are several discussions on this: the rocksdb uses several levels on caching, with a fixed 10x multiplier: the sizes are 30 GB, 300 GB, 3 TB..so is you use say a partition of 200 GB it will effectively be using only 30 GB, any more will spill over to the slower HDD device, so it is wasted space. For example see
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-October/030913.html
Some other factor is that rocksdb does compaction and during this time it does require more space, the recommendation is to use 2x above sizes to accommodate extra space for compaction:Â so it is best to use 60 GB or 600 GB.
Also note that if spill over does occur, Ceph will give a warning so it is detectable. There are also commands to move the partition (to a larger one) for an existing OSD in case it is needed, we do have scripts in the /opt/petasan/scripts dir that help with this if ever needed.
Thanks for the feedback, which does make things easier.
Currently we set the size to 60 GB, if you wish to change it, you can change the config variable
bluestore_block_db_size
Note that the 4% is questionable and there are several discussions on this: the rocksdb uses several levels on caching, with a fixed 10x multiplier: the sizes are 30 GB, 300 GB, 3 TB..so is you use say a partition of 200 GB it will effectively be using only 30 GB, any more will spill over to the slower HDD device, so it is wasted space. For example see
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-October/030913.html
Some other factor is that rocksdb does compaction and during this time it does require more space, the recommendation is to use 2x above sizes to accommodate extra space for compaction:Â so it is best to use 60 GB or 600 GB.
Also note that if spill over does occur, Ceph will give a warning so it is detectable. There are also commands to move the partition (to a larger one) for an existing OSD in case it is needed, we do have scripts in the /opt/petasan/scripts dir that help with this if ever needed.
Last edited on August 30, 2022, 8:49 am by admin · #2
Please add selectable journal size during OSD provisioning.
rhamon
30 Posts
Quote from rhamon on August 29, 2022, 6:29 pmSalutations
Please add selectable journal size in the GUI for OSD provisioning as for S3 workload the recommended journal size is at least 4% of the OSD which for large multi TB drives the default 60GB does not meet.
I can of course work around the issue by manually managing the OSD provisioning but would make life that much better if in the gui. 😉
Ref:Â https://docs.ceph.com/en/quincy/rados/configuration/bluestore-config-ref/#sizing
<<For RGW workloads, it is recommended that theÂ
block.db
 size isn’t smaller than 4% ofÂblock
, because RGW heavily uses it to store metadata (omap keys).>>Cheers and thanks!
Salutations
Please add selectable journal size in the GUI for OSD provisioning as for S3 workload the recommended journal size is at least 4% of the OSD which for large multi TB drives the default 60GB does not meet.
I can of course work around the issue by manually managing the OSD provisioning but would make life that much better if in the gui. 😉
Ref:Â https://docs.ceph.com/en/quincy/rados/configuration/bluestore-config-ref/#sizing
<<For RGW workloads, it is recommended that the block.db
 size isn’t smaller than 4% of block
, because RGW heavily uses it to store metadata (omap keys).>>
Cheers and thanks!
admin
2,921 Posts
Quote from admin on August 30, 2022, 8:09 amThanks for the feedback, which does make things easier.
Currently we set the size to 60 GB, if you wish to change it, you can change the config variable
bluestore_block_db_size
Note that the 4% is questionable and there are several discussions on this: the rocksdb uses several levels on caching, with a fixed 10x multiplier: the sizes are 30 GB, 300 GB, 3 TB..so is you use say a partition of 200 GB it will effectively be using only 30 GB, any more will spill over to the slower HDD device, so it is wasted space. For example see
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-October/030913.html
Some other factor is that rocksdb does compaction and during this time it does require more space, the recommendation is to use 2x above sizes to accommodate extra space for compaction:Â so it is best to use 60 GB or 600 GB.
Also note that if spill over does occur, Ceph will give a warning so it is detectable. There are also commands to move the partition (to a larger one) for an existing OSD in case it is needed, we do have scripts in the /opt/petasan/scripts dir that help with this if ever needed.
Thanks for the feedback, which does make things easier.
Currently we set the size to 60 GB, if you wish to change it, you can change the config variable
bluestore_block_db_size
Note that the 4% is questionable and there are several discussions on this: the rocksdb uses several levels on caching, with a fixed 10x multiplier: the sizes are 30 GB, 300 GB, 3 TB..so is you use say a partition of 200 GB it will effectively be using only 30 GB, any more will spill over to the slower HDD device, so it is wasted space. For example see
http://lists.ceph.com/pipermail/ceph-users-ceph.com/2018-October/030913.html
Some other factor is that rocksdb does compaction and during this time it does require more space, the recommendation is to use 2x above sizes to accommodate extra space for compaction:Â so it is best to use 60 GB or 600 GB.
Also note that if spill over does occur, Ceph will give a warning so it is detectable. There are also commands to move the partition (to a larger one) for an existing OSD in case it is needed, we do have scripts in the /opt/petasan/scripts dir that help with this if ever needed.