tcmalloc vs jemalloc
therm
121 Posts
August 16, 2017, 11:51 amQuote from therm on August 16, 2017, 11:51 amHi,
in an older thread http://www.petasan.org/forums/?view=thread&id=45 you told petasan is using jemalloc for performance.
Looking in /etc/default/ceph, it looks like it isnt:
root@ceph-node-mru-1:~# cat /etc/default/ceph
# /etc/default/ceph
#
# Environment file for ceph daemon systemd unit files.
#
# Increase tcmalloc cache size
TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728
use jemalloc instead of tcmalloc
#
# jemalloc is generally faster for small IO workloads and when
# ceph-osd is backed by SSDs. However, memory usage is usually
# higher by 200-300mb.
#
#LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
What is correct and where is it set?
Regards
Dennis
Hi,
in an older thread http://www.petasan.org/forums/?view=thread&id=45 you told petasan is using jemalloc for performance.
Looking in /etc/default/ceph, it looks like it isnt:
root@ceph-node-mru-1:~# cat /etc/default/ceph
# /etc/default/ceph
#
# Environment file for ceph daemon systemd unit files.
#
# Increase tcmalloc cache size
TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728
use jemalloc instead of tcmalloc
#
# jemalloc is generally faster for small IO workloads and when
# ceph-osd is backed by SSDs. However, memory usage is usually
# higher by 200-300mb.
#
#LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
What is correct and where is it set?
Regards
Dennis
admin
2,930 Posts
August 16, 2017, 2:11 pmQuote from admin on August 16, 2017, 2:11 pmwe do build Ceph using jemalloc, you can verify via
ldd /usr/bin/ceph-osd | grep malloc
libjemalloc.so.1 => /usr/lib/x86_64-linux-gnu/libjemalloc.so.1 (0x00007feded1f3000)
The TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES environment variable is used with default Ceph build but has no effect in PetaSAN
There are several links comparing them:
https://ceph.com/geen-categorie/the-ceph-and-tcmalloc-performance-story/
https://www.msi.umn.edu/sites/default/files/MN_RH_BOFSC15.pdf
we do build Ceph using jemalloc, you can verify via
ldd /usr/bin/ceph-osd | grep malloc
libjemalloc.so.1 => /usr/lib/x86_64-linux-gnu/libjemalloc.so.1 (0x00007feded1f3000)
The TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES environment variable is used with default Ceph build but has no effect in PetaSAN
There are several links comparing them:
https://ceph.com/geen-categorie/the-ceph-and-tcmalloc-performance-story/
https://www.msi.umn.edu/sites/default/files/MN_RH_BOFSC15.pdf
tcmalloc vs jemalloc
therm
121 Posts
Quote from therm on August 16, 2017, 11:51 amHi,
in an older thread http://www.petasan.org/forums/?view=thread&id=45 you told petasan is using jemalloc for performance.
Looking in /etc/default/ceph, it looks like it isnt:
root@ceph-node-mru-1:~# cat /etc/default/ceph
# /etc/default/ceph
#
# Environment file for ceph daemon systemd unit files.
## Increase tcmalloc cache size
TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728use jemalloc instead of tcmalloc
#
# jemalloc is generally faster for small IO workloads and when
# ceph-osd is backed by SSDs. However, memory usage is usually
# higher by 200-300mb.
#
#LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1What is correct and where is it set?
Regards
Dennis
Hi,
in an older thread http://www.petasan.org/forums/?view=thread&id=45 you told petasan is using jemalloc for performance.
Looking in /etc/default/ceph, it looks like it isnt:
root@ceph-node-mru-1:~# cat /etc/default/ceph
# /etc/default/ceph
#
# Environment file for ceph daemon systemd unit files.
## Increase tcmalloc cache size
TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES=134217728use jemalloc instead of tcmalloc
#
# jemalloc is generally faster for small IO workloads and when
# ceph-osd is backed by SSDs. However, memory usage is usually
# higher by 200-300mb.
#
#LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
What is correct and where is it set?
Regards
Dennis
admin
2,930 Posts
Quote from admin on August 16, 2017, 2:11 pmwe do build Ceph using jemalloc, you can verify via
ldd /usr/bin/ceph-osd | grep malloc
libjemalloc.so.1 => /usr/lib/x86_64-linux-gnu/libjemalloc.so.1 (0x00007feded1f3000)The TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES environment variable is used with default Ceph build but has no effect in PetaSAN
There are several links comparing them:
https://ceph.com/geen-categorie/the-ceph-and-tcmalloc-performance-story/
https://www.msi.umn.edu/sites/default/files/MN_RH_BOFSC15.pdf
we do build Ceph using jemalloc, you can verify via
ldd /usr/bin/ceph-osd | grep malloc
libjemalloc.so.1 => /usr/lib/x86_64-linux-gnu/libjemalloc.so.1 (0x00007feded1f3000)
The TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES environment variable is used with default Ceph build but has no effect in PetaSAN
There are several links comparing them:
https://ceph.com/geen-categorie/the-ceph-and-tcmalloc-performance-story/
https://www.msi.umn.edu/sites/default/files/MN_RH_BOFSC15.pdf