Problem with doubbled ip adresses of iscsi disks
mathiasz
2 Posts
December 29, 2020, 12:34 pmQuote from mathiasz on December 29, 2020, 12:34 pmHi all,
I've "administrativly inherited" a ceph-cluster running petasan 2.3.1 with three nodes. Usage: shared storage for some ESX-Host in a vsphere cluster, three iSCSI-disks.
One "bug", mostly configuration: Two of the three iSCSI-disks have the same ip adresses (both disks: iscsi-1 = 10.0.2.100; iscsi-2 = 10.0.3.100). I've read in the documentation, that there is an option to set the ip adresses manually at the time of adding a new iSCSI disk. I dont know how this disks are created.
My question: Is there any possibility to change these adresses manually, or to force petasan to assign automatically a new (unused) ip adress from the auto IP range? (Auto IP range ist configured for iscsi-1 and iscsi-2, looks quite fine.)
Or another question to understand the mechanism:
When exactly is the IP address automatically assigned to the disks? Is it fixed when adding the disk; or whenever the entire cluster is restarted, petasan looks for free adresses? (Some weeks ago we have to shutdown everything, so all ESX-hosts and all ceph-nodes where offline and without power.) And yes: I have tried to change the ip adresses with the UI, but no success.
And another information: Because of understanding the error I've stopped and started one of the disks, and I see these messages in PetaSAN.log:
28/12/2020 12:59:43 INFO Stopping disk 00004
28/12/2020 12:59:43 ERROR Could not find ips for image-00004
When I started the disk again, no errors or warnings occurs in the Logfile.
I hope someone could me give some hints 🙂
Thanks in advance,
Mathias
Hi all,
I've "administrativly inherited" a ceph-cluster running petasan 2.3.1 with three nodes. Usage: shared storage for some ESX-Host in a vsphere cluster, three iSCSI-disks.
One "bug", mostly configuration: Two of the three iSCSI-disks have the same ip adresses (both disks: iscsi-1 = 10.0.2.100; iscsi-2 = 10.0.3.100). I've read in the documentation, that there is an option to set the ip adresses manually at the time of adding a new iSCSI disk. I dont know how this disks are created.
My question: Is there any possibility to change these adresses manually, or to force petasan to assign automatically a new (unused) ip adress from the auto IP range? (Auto IP range ist configured for iscsi-1 and iscsi-2, looks quite fine.)
Or another question to understand the mechanism:
When exactly is the IP address automatically assigned to the disks? Is it fixed when adding the disk; or whenever the entire cluster is restarted, petasan looks for free adresses? (Some weeks ago we have to shutdown everything, so all ESX-hosts and all ceph-nodes where offline and without power.) And yes: I have tried to change the ip adresses with the UI, but no success.
And another information: Because of understanding the error I've stopped and started one of the disks, and I see these messages in PetaSAN.log:
28/12/2020 12:59:43 INFO Stopping disk 00004
28/12/2020 12:59:43 ERROR Could not find ips for image-00004
When I started the disk again, no errors or warnings occurs in the Logfile.
I hope someone could me give some hints 🙂
Thanks in advance,
Mathias
admin
2,930 Posts
December 29, 2020, 4:46 pmQuote from admin on December 29, 2020, 4:46 pmIf you enter the ips manually, we do not check for duplicates. we only check that ip is valid within the subnet.
If you choose auto ip assignment, we select free ips from the range of to/from so there should be no duplicates.
IPs as well as other iSCSI data is stored in Ceph as attribute metadata within the rbd image. Adding a new disk or Attaching an exiting rbd image will create iSCSI attributes and write it to the image metadata, Detaching the iSCSI disk will erase all metadata. So you can detach and re-attach the images selecting auto ips and it will receive new ips and probably new iqn. Outside the ui, you can use the script:
/opt/petasan/scripts/util/disk_meta.py to read/ modify/ write disk metadata in json format. the script parameters in version 2.3.1 were a bit cryptic bu there should be a help within script itself. The script will give you more control for changing any value you want.
If you enter the ips manually, we do not check for duplicates. we only check that ip is valid within the subnet.
If you choose auto ip assignment, we select free ips from the range of to/from so there should be no duplicates.
IPs as well as other iSCSI data is stored in Ceph as attribute metadata within the rbd image. Adding a new disk or Attaching an exiting rbd image will create iSCSI attributes and write it to the image metadata, Detaching the iSCSI disk will erase all metadata. So you can detach and re-attach the images selecting auto ips and it will receive new ips and probably new iqn. Outside the ui, you can use the script:
/opt/petasan/scripts/util/disk_meta.py to read/ modify/ write disk metadata in json format. the script parameters in version 2.3.1 were a bit cryptic bu there should be a help within script itself. The script will give you more control for changing any value you want.
Last edited on December 29, 2020, 4:47 pm by admin · #2
mathiasz
2 Posts
December 31, 2020, 9:13 amQuote from mathiasz on December 31, 2020, 9:13 amHi,
Thanks a lot, detach and re-attach worked liked expected, with selecting auto ips the disk gets new adresses.
Mathias
Hi,
Thanks a lot, detach and re-attach worked liked expected, with selecting auto ips the disk gets new adresses.
Mathias
Problem with doubbled ip adresses of iscsi disks
mathiasz
2 Posts
Quote from mathiasz on December 29, 2020, 12:34 pmHi all,
I've "administrativly inherited" a ceph-cluster running petasan 2.3.1 with three nodes. Usage: shared storage for some ESX-Host in a vsphere cluster, three iSCSI-disks.
One "bug", mostly configuration: Two of the three iSCSI-disks have the same ip adresses (both disks: iscsi-1 = 10.0.2.100; iscsi-2 = 10.0.3.100). I've read in the documentation, that there is an option to set the ip adresses manually at the time of adding a new iSCSI disk. I dont know how this disks are created.
My question: Is there any possibility to change these adresses manually, or to force petasan to assign automatically a new (unused) ip adress from the auto IP range? (Auto IP range ist configured for iscsi-1 and iscsi-2, looks quite fine.)
Or another question to understand the mechanism:
When exactly is the IP address automatically assigned to the disks? Is it fixed when adding the disk; or whenever the entire cluster is restarted, petasan looks for free adresses? (Some weeks ago we have to shutdown everything, so all ESX-hosts and all ceph-nodes where offline and without power.) And yes: I have tried to change the ip adresses with the UI, but no success.
And another information: Because of understanding the error I've stopped and started one of the disks, and I see these messages in PetaSAN.log:
28/12/2020 12:59:43 INFO Stopping disk 00004
28/12/2020 12:59:43 ERROR Could not find ips for image-00004When I started the disk again, no errors or warnings occurs in the Logfile.
I hope someone could me give some hints 🙂
Thanks in advance,
Mathias
Hi all,
I've "administrativly inherited" a ceph-cluster running petasan 2.3.1 with three nodes. Usage: shared storage for some ESX-Host in a vsphere cluster, three iSCSI-disks.
One "bug", mostly configuration: Two of the three iSCSI-disks have the same ip adresses (both disks: iscsi-1 = 10.0.2.100; iscsi-2 = 10.0.3.100). I've read in the documentation, that there is an option to set the ip adresses manually at the time of adding a new iSCSI disk. I dont know how this disks are created.
My question: Is there any possibility to change these adresses manually, or to force petasan to assign automatically a new (unused) ip adress from the auto IP range? (Auto IP range ist configured for iscsi-1 and iscsi-2, looks quite fine.)
Or another question to understand the mechanism:
When exactly is the IP address automatically assigned to the disks? Is it fixed when adding the disk; or whenever the entire cluster is restarted, petasan looks for free adresses? (Some weeks ago we have to shutdown everything, so all ESX-hosts and all ceph-nodes where offline and without power.) And yes: I have tried to change the ip adresses with the UI, but no success.
And another information: Because of understanding the error I've stopped and started one of the disks, and I see these messages in PetaSAN.log:
28/12/2020 12:59:43 INFO Stopping disk 00004
28/12/2020 12:59:43 ERROR Could not find ips for image-00004
When I started the disk again, no errors or warnings occurs in the Logfile.
I hope someone could me give some hints 🙂
Thanks in advance,
Mathias
admin
2,930 Posts
Quote from admin on December 29, 2020, 4:46 pmIf you enter the ips manually, we do not check for duplicates. we only check that ip is valid within the subnet.
If you choose auto ip assignment, we select free ips from the range of to/from so there should be no duplicates.
IPs as well as other iSCSI data is stored in Ceph as attribute metadata within the rbd image. Adding a new disk or Attaching an exiting rbd image will create iSCSI attributes and write it to the image metadata, Detaching the iSCSI disk will erase all metadata. So you can detach and re-attach the images selecting auto ips and it will receive new ips and probably new iqn. Outside the ui, you can use the script:
/opt/petasan/scripts/util/disk_meta.py to read/ modify/ write disk metadata in json format. the script parameters in version 2.3.1 were a bit cryptic bu there should be a help within script itself. The script will give you more control for changing any value you want.
If you enter the ips manually, we do not check for duplicates. we only check that ip is valid within the subnet.
If you choose auto ip assignment, we select free ips from the range of to/from so there should be no duplicates.
IPs as well as other iSCSI data is stored in Ceph as attribute metadata within the rbd image. Adding a new disk or Attaching an exiting rbd image will create iSCSI attributes and write it to the image metadata, Detaching the iSCSI disk will erase all metadata. So you can detach and re-attach the images selecting auto ips and it will receive new ips and probably new iqn. Outside the ui, you can use the script:
/opt/petasan/scripts/util/disk_meta.py to read/ modify/ write disk metadata in json format. the script parameters in version 2.3.1 were a bit cryptic bu there should be a help within script itself. The script will give you more control for changing any value you want.
mathiasz
2 Posts
Quote from mathiasz on December 31, 2020, 9:13 amHi,
Thanks a lot, detach and re-attach worked liked expected, with selecting auto ips the disk gets new adresses.
Mathias
Hi,
Thanks a lot, detach and re-attach worked liked expected, with selecting auto ips the disk gets new adresses.
Mathias