Specification on leveraging caching with 2.4
ubx_cloud_steve
7 Posts
December 23, 2019, 11:35 pmQuote from ubx_cloud_steve on December 23, 2019, 11:35 pmHello PetaSAN team,
Congratulations on the caching feature introduced in this latest release. We have many questions on how to effectively leverage this in our setup.
1. How do you appropriately size the enterprise SSD for using a cache target?
2. Can we use the caching WITHOUT using journaling (we are using Filestore NOT Bluestore due to performance gains with our setup).
3. Can you share benchmark details with systems before and after caching has added?
4. What happens on the event of a power loss with this cache setup?
Hello PetaSAN team,
Congratulations on the caching feature introduced in this latest release. We have many questions on how to effectively leverage this in our setup.
1. How do you appropriately size the enterprise SSD for using a cache target?
2. Can we use the caching WITHOUT using journaling (we are using Filestore NOT Bluestore due to performance gains with our setup).
3. Can you share benchmark details with systems before and after caching has added?
4. What happens on the event of a power loss with this cache setup?
Last edited on December 23, 2019, 11:36 pm by ubx_cloud_steve · #1
admin
2,930 Posts
December 24, 2019, 1:19 pmQuote from admin on December 24, 2019, 1:19 pmThanks for the comment 🙂
1- an SSD cache should serve 4 HDDs well, so you can choose 4 partitions while creating the cache. Even with small size you will get good write performance, however the larger the size the more hits your will get while reading recently written data.
2-Yes you can, for bluestore it is better to have both as the journal device is used to speed up both read and write of metadata (object locations, crc). For filestore, the journal is quite similar to a write cache in behavior (although it flushes every 5 sec), so you could test using either journal or cache, it is important to test in your environment.
3-Cache benchmarks depend a lot on the io pattern. In most ideal cases, you will get close to native ssd speeds, even with pure random 4k blocks, the writebacks to hdd will be sorted in a way to help hdd iops, in many near-random writes, many will be merged and written to larger blocks, this is made better the larger the cache size. The best test for cache is under your real workload.
4-No problem, this is a persistent block cache, it is not in memory.
Thanks for the comment 🙂
1- an SSD cache should serve 4 HDDs well, so you can choose 4 partitions while creating the cache. Even with small size you will get good write performance, however the larger the size the more hits your will get while reading recently written data.
2-Yes you can, for bluestore it is better to have both as the journal device is used to speed up both read and write of metadata (object locations, crc). For filestore, the journal is quite similar to a write cache in behavior (although it flushes every 5 sec), so you could test using either journal or cache, it is important to test in your environment.
3-Cache benchmarks depend a lot on the io pattern. In most ideal cases, you will get close to native ssd speeds, even with pure random 4k blocks, the writebacks to hdd will be sorted in a way to help hdd iops, in many near-random writes, many will be merged and written to larger blocks, this is made better the larger the cache size. The best test for cache is under your real workload.
4-No problem, this is a persistent block cache, it is not in memory.
ubx_cloud_steve
7 Posts
December 24, 2019, 3:34 pmQuote from ubx_cloud_steve on December 24, 2019, 3:34 pmGreat!
I have access to these 1.2 TB fusion io cards. I would like to use them as cache. How many disks to cache ratio should I use with this guy?
Any concerns with using this card for caching?
https://www.storagereview.com/fusionio_iodrive2_duo_mlc_application_accelerator_review
Great!
I have access to these 1.2 TB fusion io cards. I would like to use them as cache. How many disks to cache ratio should I use with this guy?
Any concerns with using this card for caching?
https://www.storagereview.com/fusionio_iodrive2_duo_mlc_application_accelerator_review
admin
2,930 Posts
December 24, 2019, 7:03 pmQuote from admin on December 24, 2019, 7:03 pmwe have not used this device...can't really say. we allow a regular SSD/nvme to serve 1-8 hdds, if this is a high iops device and your workload is iops based, then you can shoot fot 8 partitions..if it is not high or your workload is throughput heavy then 2-4 would be better.
we have not used this device...can't really say. we allow a regular SSD/nvme to serve 1-8 hdds, if this is a high iops device and your workload is iops based, then you can shoot fot 8 partitions..if it is not high or your workload is throughput heavy then 2-4 would be better.
Specification on leveraging caching with 2.4
ubx_cloud_steve
7 Posts
Quote from ubx_cloud_steve on December 23, 2019, 11:35 pmHello PetaSAN team,
Congratulations on the caching feature introduced in this latest release. We have many questions on how to effectively leverage this in our setup.
1. How do you appropriately size the enterprise SSD for using a cache target?
2. Can we use the caching WITHOUT using journaling (we are using Filestore NOT Bluestore due to performance gains with our setup).
3. Can you share benchmark details with systems before and after caching has added?
4. What happens on the event of a power loss with this cache setup?
Hello PetaSAN team,
Congratulations on the caching feature introduced in this latest release. We have many questions on how to effectively leverage this in our setup.
1. How do you appropriately size the enterprise SSD for using a cache target?
2. Can we use the caching WITHOUT using journaling (we are using Filestore NOT Bluestore due to performance gains with our setup).
3. Can you share benchmark details with systems before and after caching has added?
4. What happens on the event of a power loss with this cache setup?
admin
2,930 Posts
Quote from admin on December 24, 2019, 1:19 pmThanks for the comment 🙂
1- an SSD cache should serve 4 HDDs well, so you can choose 4 partitions while creating the cache. Even with small size you will get good write performance, however the larger the size the more hits your will get while reading recently written data.
2-Yes you can, for bluestore it is better to have both as the journal device is used to speed up both read and write of metadata (object locations, crc). For filestore, the journal is quite similar to a write cache in behavior (although it flushes every 5 sec), so you could test using either journal or cache, it is important to test in your environment.
3-Cache benchmarks depend a lot on the io pattern. In most ideal cases, you will get close to native ssd speeds, even with pure random 4k blocks, the writebacks to hdd will be sorted in a way to help hdd iops, in many near-random writes, many will be merged and written to larger blocks, this is made better the larger the cache size. The best test for cache is under your real workload.
4-No problem, this is a persistent block cache, it is not in memory.
Thanks for the comment 🙂
1- an SSD cache should serve 4 HDDs well, so you can choose 4 partitions while creating the cache. Even with small size you will get good write performance, however the larger the size the more hits your will get while reading recently written data.
2-Yes you can, for bluestore it is better to have both as the journal device is used to speed up both read and write of metadata (object locations, crc). For filestore, the journal is quite similar to a write cache in behavior (although it flushes every 5 sec), so you could test using either journal or cache, it is important to test in your environment.
3-Cache benchmarks depend a lot on the io pattern. In most ideal cases, you will get close to native ssd speeds, even with pure random 4k blocks, the writebacks to hdd will be sorted in a way to help hdd iops, in many near-random writes, many will be merged and written to larger blocks, this is made better the larger the cache size. The best test for cache is under your real workload.
4-No problem, this is a persistent block cache, it is not in memory.
ubx_cloud_steve
7 Posts
Quote from ubx_cloud_steve on December 24, 2019, 3:34 pmGreat!
I have access to these 1.2 TB fusion io cards. I would like to use them as cache. How many disks to cache ratio should I use with this guy?
Any concerns with using this card for caching?
https://www.storagereview.com/fusionio_iodrive2_duo_mlc_application_accelerator_review
Great!
I have access to these 1.2 TB fusion io cards. I would like to use them as cache. How many disks to cache ratio should I use with this guy?
Any concerns with using this card for caching?
https://www.storagereview.com/fusionio_iodrive2_duo_mlc_application_accelerator_review
admin
2,930 Posts
Quote from admin on December 24, 2019, 7:03 pmwe have not used this device...can't really say. we allow a regular SSD/nvme to serve 1-8 hdds, if this is a high iops device and your workload is iops based, then you can shoot fot 8 partitions..if it is not high or your workload is throughput heavy then 2-4 would be better.
we have not used this device...can't really say. we allow a regular SSD/nvme to serve 1-8 hdds, if this is a high iops device and your workload is iops based, then you can shoot fot 8 partitions..if it is not high or your workload is throughput heavy then 2-4 would be better.