Simultaneous journal and cache on single NVMe/SSD
rhamon
30 Posts
November 7, 2022, 5:45 amQuote from rhamon on November 7, 2022, 5:45 amGiven how large and fast SSD are and to limit the failure domain, I'd like to share a single SSD as both journal and write cache for some HDD.
In my particular use case I have 4 large SSD for 16 HDD per host and it would make much more sense to fully use the capacity/performance or a single SSD as journal and write cache per 4 HDD.
As it is now, I need to dedicate 2 large SSD as journal and the 2 others as write cache. This wastes capacity on the journal SSD and also make the failure domain be 8 HDD.
Thanks.
Given how large and fast SSD are and to limit the failure domain, I'd like to share a single SSD as both journal and write cache for some HDD.
In my particular use case I have 4 large SSD for 16 HDD per host and it would make much more sense to fully use the capacity/performance or a single SSD as journal and write cache per 4 HDD.
As it is now, I need to dedicate 2 large SSD as journal and the 2 others as write cache. This wastes capacity on the journal SSD and also make the failure domain be 8 HDD.
Thanks.
ghbiz
76 Posts
admin
2,921 Posts
May 27, 2023, 12:31 pmQuote from admin on May 27, 2023, 12:31 pmIt will degrade performance. A single SSD is good for 4 journals Or 4 write caches. It is a performance not a capacity issue.
In my particular use case I have 4 large SSD for 16 HDD per host and it would make much more sense to fully use the capacity/performance or a single SSD as journal and write cache per 4 HDD.
As it is now, I need to dedicate 2 large SSD as journal and the 2 others as write cache. This wastes capacity on the journal SSD and also make the failure domain be 8 HDD.
It will be too much for your 2 SSDs to serve as journal AND cache for your 16 HDDs. it will definitely affect throughput.
It will degrade performance. A single SSD is good for 4 journals Or 4 write caches. It is a performance not a capacity issue.
In my particular use case I have 4 large SSD for 16 HDD per host and it would make much more sense to fully use the capacity/performance or a single SSD as journal and write cache per 4 HDD.
As it is now, I need to dedicate 2 large SSD as journal and the 2 others as write cache. This wastes capacity on the journal SSD and also make the failure domain be 8 HDD.
It will be too much for your 2 SSDs to serve as journal AND cache for your 16 HDDs. it will definitely affect throughput.
ghbiz
76 Posts
July 17, 2023, 10:07 pmQuote from ghbiz on July 17, 2023, 10:07 pmAs a side note.. the following may work for your needs ...
We enabled auto-tiering and used a SSD pool to cache HDD pools.... something like this ....
#pool1-hdd == HDD Pool
#pool1-hdd-hot == SSD Pool to be used for auto tiering hot data ...
ceph osd tier add pool1-hdd pool1-hdd-hot
ceph osd pool set pool1-hdd-hot hit_set_type bloom
ceph osd pool set pool1-hdd-hot hit_set_count 1
ceph osd pool set pool1-hdd-hot hit_set_period 3600 # 1 hour
ceph osd pool set pool1-hdd-hot target_max_bytes 1000000000000 # 100 GB
ceph osd pool set pool1-hdd-hot min_read_recency_for_promote 1
ceph osd pool set pool1-hdd-hot min_write_recency_for_promote 1
As a side note.. the following may work for your needs ...
We enabled auto-tiering and used a SSD pool to cache HDD pools.... something like this ....
#pool1-hdd == HDD Pool
#pool1-hdd-hot == SSD Pool to be used for auto tiering hot data ...
ceph osd tier add pool1-hdd pool1-hdd-hot
ceph osd pool set pool1-hdd-hot hit_set_type bloom
ceph osd pool set pool1-hdd-hot hit_set_count 1
ceph osd pool set pool1-hdd-hot hit_set_period 3600 # 1 hour
ceph osd pool set pool1-hdd-hot target_max_bytes 1000000000000 # 100 GB
ceph osd pool set pool1-hdd-hot min_read_recency_for_promote 1
ceph osd pool set pool1-hdd-hot min_write_recency_for_promote 1
Simultaneous journal and cache on single NVMe/SSD
rhamon
30 Posts
Quote from rhamon on November 7, 2022, 5:45 amGiven how large and fast SSD are and to limit the failure domain, I'd like to share a single SSD as both journal and write cache for some HDD.
In my particular use case I have 4 large SSD for 16 HDD per host and it would make much more sense to fully use the capacity/performance or a single SSD as journal and write cache per 4 HDD.
As it is now, I need to dedicate 2 large SSD as journal and the 2 others as write cache. This wastes capacity on the journal SSD and also make the failure domain be 8 HDD.
Thanks.
Given how large and fast SSD are and to limit the failure domain, I'd like to share a single SSD as both journal and write cache for some HDD.
In my particular use case I have 4 large SSD for 16 HDD per host and it would make much more sense to fully use the capacity/performance or a single SSD as journal and write cache per 4 HDD.
As it is now, I need to dedicate 2 large SSD as journal and the 2 others as write cache. This wastes capacity on the journal SSD and also make the failure domain be 8 HDD.
Thanks.
ghbiz
76 Posts
admin
2,921 Posts
Quote from admin on May 27, 2023, 12:31 pmIt will degrade performance. A single SSD is good for 4 journals Or 4 write caches. It is a performance not a capacity issue.
In my particular use case I have 4 large SSD for 16 HDD per host and it would make much more sense to fully use the capacity/performance or a single SSD as journal and write cache per 4 HDD.
As it is now, I need to dedicate 2 large SSD as journal and the 2 others as write cache. This wastes capacity on the journal SSD and also make the failure domain be 8 HDD.
It will be too much for your 2 SSDs to serve as journal AND cache for your 16 HDDs. it will definitely affect throughput.
It will degrade performance. A single SSD is good for 4 journals Or 4 write caches. It is a performance not a capacity issue.
In my particular use case I have 4 large SSD for 16 HDD per host and it would make much more sense to fully use the capacity/performance or a single SSD as journal and write cache per 4 HDD.
As it is now, I need to dedicate 2 large SSD as journal and the 2 others as write cache. This wastes capacity on the journal SSD and also make the failure domain be 8 HDD.
It will be too much for your 2 SSDs to serve as journal AND cache for your 16 HDDs. it will definitely affect throughput.
ghbiz
76 Posts
Quote from ghbiz on July 17, 2023, 10:07 pmAs a side note.. the following may work for your needs ...
We enabled auto-tiering and used a SSD pool to cache HDD pools.... something like this ....
#pool1-hdd == HDD Pool
#pool1-hdd-hot == SSD Pool to be used for auto tiering hot data ...ceph osd tier add pool1-hdd pool1-hdd-hot
ceph osd pool set pool1-hdd-hot hit_set_type bloom
ceph osd pool set pool1-hdd-hot hit_set_count 1
ceph osd pool set pool1-hdd-hot hit_set_period 3600 # 1 hour
ceph osd pool set pool1-hdd-hot target_max_bytes 1000000000000 # 100 GB
ceph osd pool set pool1-hdd-hot min_read_recency_for_promote 1
ceph osd pool set pool1-hdd-hot min_write_recency_for_promote 1
As a side note.. the following may work for your needs ...
We enabled auto-tiering and used a SSD pool to cache HDD pools.... something like this ....
#pool1-hdd == HDD Pool
#pool1-hdd-hot == SSD Pool to be used for auto tiering hot data ...
ceph osd tier add pool1-hdd pool1-hdd-hot
ceph osd pool set pool1-hdd-hot hit_set_type bloom
ceph osd pool set pool1-hdd-hot hit_set_count 1
ceph osd pool set pool1-hdd-hot hit_set_period 3600 # 1 hour
ceph osd pool set pool1-hdd-hot target_max_bytes 1000000000000 # 100 GB
ceph osd pool set pool1-hdd-hot min_read_recency_for_promote 1
ceph osd pool set pool1-hdd-hot min_write_recency_for_promote 1