best mechanism to mounting a pool on a remote linux client environment instead of iscsi
ghbiz
76 Posts
August 4, 2023, 11:19 amQuote from ghbiz on August 4, 2023, 11:19 amCurrently, we are using Petasan to present an iSCSI Lun to a set of Linux servers that utilize KVM as a hypervisor for VMs.
The block device presented by the petasan iSCSI contains LVMs that are exposed to the KVM hypervisor for use in the VMs.
Instead of iSCSI, we would like to use the native ceph rbd driver to present the block devices to the Linux server and simply bypass iSCSI.
We wanted to use something like the following in our bash script on the client machine.
rbd map $CEPH_POOL/$CEPH_IMAGE --id admin --keyring /etc/ceph/ceph.client.admin.keyring
We would like to know what would be the best recommended mechanism in petasan to achieve this.
More specifically, creating a keyring that is unique and allocated to only a single RBD pool for access control preferences.
Currently, we are using Petasan to present an iSCSI Lun to a set of Linux servers that utilize KVM as a hypervisor for VMs.
The block device presented by the petasan iSCSI contains LVMs that are exposed to the KVM hypervisor for use in the VMs.
Instead of iSCSI, we would like to use the native ceph rbd driver to present the block devices to the Linux server and simply bypass iSCSI.
We wanted to use something like the following in our bash script on the client machine.
rbd map $CEPH_POOL/$CEPH_IMAGE --id admin --keyring /etc/ceph/ceph.client.admin.keyring
We would like to know what would be the best recommended mechanism in petasan to achieve this.
More specifically, creating a keyring that is unique and allocated to only a single RBD pool for access control preferences.
admin
2,967 Posts
August 4, 2023, 5:57 pmQuote from admin on August 4, 2023, 5:57 pmceph auth get-or-create client.USER1 mgr 'allow r' mon 'profile rbd' osd 'profile rbd pool=POOL1' >> /etc/ceph/ceph.client.USER1.keyring
ceph auth get-or-create client.USER1 mgr 'allow r' mon 'profile rbd' osd 'profile rbd pool=POOL1' >> /etc/ceph/ceph.client.USER1.keyring
ghbiz
76 Posts
August 4, 2023, 7:08 pmQuote from ghbiz on August 4, 2023, 7:08 pmok, this would need to be in both the petasan nodes and the client node or JUST the client node?
ok, this would need to be in both the petasan nodes and the client node or JUST the client node?
admin
2,967 Posts
August 4, 2023, 8:45 pmQuote from admin on August 4, 2023, 8:45 pmthe command needs to run on a host with admin keyring, it will create a user/keyring with restricted privileges. the keyring of the restricted user needs to be installed on client node.
the command needs to run on a host with admin keyring, it will create a user/keyring with restricted privileges. the keyring of the restricted user needs to be installed on client node.
ghbiz
76 Posts
ghbiz
76 Posts
August 8, 2023, 12:47 pmQuote from ghbiz on August 8, 2023, 12:47 pmi am trying to create a client key that allows for a specific image on a specific pool... but got an error as follows ....
ceph auth get-or-create client.devcomp-hdd mgr 'allow r' mon 'profile rbd' osd 'profile rbd pool=dev-hdd-pool image=image-00004' >> /etc/ceph/ceph.client.devcomp.keyring
Error EINVAL: osd capability parse failed, stopped at 'object_prefix rbd_data.image.image-00004' of 'profile rbd pool=dev-hdd-pool object_prefix rbd_data.image.image-00004'
i am trying to create a client key that allows for a specific image on a specific pool... but got an error as follows ....
ceph auth get-or-create client.devcomp-hdd mgr 'allow r' mon 'profile rbd' osd 'profile rbd pool=dev-hdd-pool image=image-00004' >> /etc/ceph/ceph.client.devcomp.keyring
Error EINVAL: osd capability parse failed, stopped at 'object_prefix rbd_data.image.image-00004' of 'profile rbd pool=dev-hdd-pool object_prefix rbd_data.image.image-00004'
admin
2,967 Posts
August 8, 2023, 1:26 pmQuote from admin on August 8, 2023, 1:26 pmyou cannot specify image
you cannot specify image
ghbiz
76 Posts
August 8, 2023, 5:41 pmQuote from ghbiz on August 8, 2023, 5:41 pmrunning "rbd showmapped" ... I see that petasan is not using any namespaces. is it possible to map an image to a namespace in petasan without it causing issues to petasan. I am asking because i CAN limit access to name spaces per the CEPH documentation....
root@cephdev1:~# rbd showmapped
id pool namespace image snap device
0 iscsi-test image-00007 - /dev/rbd0
1 iscsi-test-hdd image-00008 - /dev/rbd1
2 dev-ssd-pool image-00010 - /dev/rbd2
3 dev-ssd-pool image-00011 - /dev/rbd3
4 rbd image-00001 - /dev/rbd4
5 rbd image-00002 - /dev/rbd5
6 dev-hdd-pool image-00004 - /dev/rbd6
7 dev-hdd-pool image-00005 - /dev/rbd7
8 dev-ssd-pool image-00003 - /dev/rbd8
root@cephdev1:~#
running "rbd showmapped" ... I see that petasan is not using any namespaces. is it possible to map an image to a namespace in petasan without it causing issues to petasan. I am asking because i CAN limit access to name spaces per the CEPH documentation....
root@cephdev1:~# rbd showmapped
id pool namespace image snap device
0 iscsi-test image-00007 - /dev/rbd0
1 iscsi-test-hdd image-00008 - /dev/rbd1
2 dev-ssd-pool image-00010 - /dev/rbd2
3 dev-ssd-pool image-00011 - /dev/rbd3
4 rbd image-00001 - /dev/rbd4
5 rbd image-00002 - /dev/rbd5
6 dev-hdd-pool image-00004 - /dev/rbd6
7 dev-hdd-pool image-00005 - /dev/rbd7
8 dev-ssd-pool image-00003 - /dev/rbd8
root@cephdev1:~#
admin
2,967 Posts
August 8, 2023, 8:52 pmQuote from admin on August 8, 2023, 8:52 pmYou can limit access via namespaces, i am not sure how this will affect the rest of the system as it is not something we test. You can test yourself and see if there are any issues. or you can create different pools and have the access by pool as your original question which has no issues.
You can limit access via namespaces, i am not sure how this will affect the rest of the system as it is not something we test. You can test yourself and see if there are any issues. or you can create different pools and have the access by pool as your original question which has no issues.
best mechanism to mounting a pool on a remote linux client environment instead of iscsi
ghbiz
76 Posts
Quote from ghbiz on August 4, 2023, 11:19 amCurrently, we are using Petasan to present an iSCSI Lun to a set of Linux servers that utilize KVM as a hypervisor for VMs.
The block device presented by the petasan iSCSI contains LVMs that are exposed to the KVM hypervisor for use in the VMs.
Instead of iSCSI, we would like to use the native ceph rbd driver to present the block devices to the Linux server and simply bypass iSCSI.
We wanted to use something like the following in our bash script on the client machine.
rbd map $CEPH_POOL/$CEPH_IMAGE --id admin --keyring /etc/ceph/ceph.client.admin.keyring
We would like to know what would be the best recommended mechanism in petasan to achieve this.
More specifically, creating a keyring that is unique and allocated to only a single RBD pool for access control preferences.
Currently, we are using Petasan to present an iSCSI Lun to a set of Linux servers that utilize KVM as a hypervisor for VMs.
The block device presented by the petasan iSCSI contains LVMs that are exposed to the KVM hypervisor for use in the VMs.
Instead of iSCSI, we would like to use the native ceph rbd driver to present the block devices to the Linux server and simply bypass iSCSI.
We wanted to use something like the following in our bash script on the client machine.
rbd map $CEPH_POOL/$CEPH_IMAGE --id admin --keyring /etc/ceph/ceph.client.admin.keyring
We would like to know what would be the best recommended mechanism in petasan to achieve this.
More specifically, creating a keyring that is unique and allocated to only a single RBD pool for access control preferences.
admin
2,967 Posts
Quote from admin on August 4, 2023, 5:57 pmceph auth get-or-create client.USER1 mgr 'allow r' mon 'profile rbd' osd 'profile rbd pool=POOL1' >> /etc/ceph/ceph.client.USER1.keyring
ceph auth get-or-create client.USER1 mgr 'allow r' mon 'profile rbd' osd 'profile rbd pool=POOL1' >> /etc/ceph/ceph.client.USER1.keyring
ghbiz
76 Posts
Quote from ghbiz on August 4, 2023, 7:08 pmok, this would need to be in both the petasan nodes and the client node or JUST the client node?
ok, this would need to be in both the petasan nodes and the client node or JUST the client node?
admin
2,967 Posts
Quote from admin on August 4, 2023, 8:45 pmthe command needs to run on a host with admin keyring, it will create a user/keyring with restricted privileges. the keyring of the restricted user needs to be installed on client node.
the command needs to run on a host with admin keyring, it will create a user/keyring with restricted privileges. the keyring of the restricted user needs to be installed on client node.
ghbiz
76 Posts
ghbiz
76 Posts
Quote from ghbiz on August 8, 2023, 12:47 pmi am trying to create a client key that allows for a specific image on a specific pool... but got an error as follows ....
ceph auth get-or-create client.devcomp-hdd mgr 'allow r' mon 'profile rbd' osd 'profile rbd pool=dev-hdd-pool image=image-00004' >> /etc/ceph/ceph.client.devcomp.keyring
Error EINVAL: osd capability parse failed, stopped at 'object_prefix rbd_data.image.image-00004' of 'profile rbd pool=dev-hdd-pool object_prefix rbd_data.image.image-00004'
i am trying to create a client key that allows for a specific image on a specific pool... but got an error as follows ....
ceph auth get-or-create client.devcomp-hdd mgr 'allow r' mon 'profile rbd' osd 'profile rbd pool=dev-hdd-pool image=image-00004' >> /etc/ceph/ceph.client.devcomp.keyring
Error EINVAL: osd capability parse failed, stopped at 'object_prefix rbd_data.image.image-00004' of 'profile rbd pool=dev-hdd-pool object_prefix rbd_data.image.image-00004'
admin
2,967 Posts
Quote from admin on August 8, 2023, 1:26 pmyou cannot specify image
you cannot specify image
ghbiz
76 Posts
Quote from ghbiz on August 8, 2023, 5:41 pmrunning "rbd showmapped" ... I see that petasan is not using any namespaces. is it possible to map an image to a namespace in petasan without it causing issues to petasan. I am asking because i CAN limit access to name spaces per the CEPH documentation....
root@cephdev1:~# rbd showmapped
id pool namespace image snap device
0 iscsi-test image-00007 - /dev/rbd0
1 iscsi-test-hdd image-00008 - /dev/rbd1
2 dev-ssd-pool image-00010 - /dev/rbd2
3 dev-ssd-pool image-00011 - /dev/rbd3
4 rbd image-00001 - /dev/rbd4
5 rbd image-00002 - /dev/rbd5
6 dev-hdd-pool image-00004 - /dev/rbd6
7 dev-hdd-pool image-00005 - /dev/rbd7
8 dev-ssd-pool image-00003 - /dev/rbd8
root@cephdev1:~#
running "rbd showmapped" ... I see that petasan is not using any namespaces. is it possible to map an image to a namespace in petasan without it causing issues to petasan. I am asking because i CAN limit access to name spaces per the CEPH documentation....
root@cephdev1:~# rbd showmapped
id pool namespace image snap device
0 iscsi-test image-00007 - /dev/rbd0
1 iscsi-test-hdd image-00008 - /dev/rbd1
2 dev-ssd-pool image-00010 - /dev/rbd2
3 dev-ssd-pool image-00011 - /dev/rbd3
4 rbd image-00001 - /dev/rbd4
5 rbd image-00002 - /dev/rbd5
6 dev-hdd-pool image-00004 - /dev/rbd6
7 dev-hdd-pool image-00005 - /dev/rbd7
8 dev-ssd-pool image-00003 - /dev/rbd8
root@cephdev1:~#
admin
2,967 Posts
Quote from admin on August 8, 2023, 8:52 pmYou can limit access via namespaces, i am not sure how this will affect the rest of the system as it is not something we test. You can test yourself and see if there are any issues. or you can create different pools and have the access by pool as your original question which has no issues.
You can limit access via namespaces, i am not sure how this will affect the rest of the system as it is not something we test. You can test yourself and see if there are any issues. or you can create different pools and have the access by pool as your original question which has no issues.