Using two PetaSAN, provisionning targets from both to same machine . . . !
Pages: 1 2
sniffus
20 Posts
January 17, 2018, 3:06 pmQuote from sniffus on January 17, 2018, 3:06 pmPetaSAN1 and Peta2. (Ok yes, two of us working on the lab . . .)
so PetaSAN1 has disks 3, 4, 5, 7 (this is where we initially tested deletions).
and Peta2 has disks 1, 2, 3. And yep, we just added disk 3 to our hypervisor1 test machine.
Now, I expected this issue. So I changed the IQN prefix on Peta2 to say peta2 instead of petasan.
Still Microsoft 2016 iSCSI initiator sees both disk 3 as the same disk, likely because of MPIO. I would have expected the IQN to be enough, shows how wrong I can be.
Is there a way to change the range of disk numbering on PetaSAN, say start at 100 on petasan1 and 500 on peta2.
Our goal is because we will have different cluster for different performance disks. Or is there a better way to qualify the disks without going all CLI?
M.
PetaSAN1 and Peta2. (Ok yes, two of us working on the lab . . .)
so PetaSAN1 has disks 3, 4, 5, 7 (this is where we initially tested deletions).
and Peta2 has disks 1, 2, 3. And yep, we just added disk 3 to our hypervisor1 test machine.
Now, I expected this issue. So I changed the IQN prefix on Peta2 to say peta2 instead of petasan.
Still Microsoft 2016 iSCSI initiator sees both disk 3 as the same disk, likely because of MPIO. I would have expected the IQN to be enough, shows how wrong I can be.
Is there a way to change the range of disk numbering on PetaSAN, say start at 100 on petasan1 and 500 on peta2.
Our goal is because we will have different cluster for different performance disks. Or is there a better way to qualify the disks without going all CLI?
M.
sniffus
20 Posts
January 17, 2018, 3:38 pmQuote from sniffus on January 17, 2018, 3:38 pm
It's clearly an MPIO issue, somehow, the way it checks for disk info uniqueness is using something that PetaSAN has standardized and calculates the same way on both units, making it not so unique.
Are you creating a WWN? a physical disk scsi serial number?
https://social.technet.microsoft.com/Forums/windowsserver/en-US/6a6d6abd-61e9-47cc-8618-ffcee1e1a481/how-to-find-hardware-path-information-for-fc-mpio?forum=winserverfiles
Here they clarify that they are not using Disk Signatures. But I am not seeing how they actually id the disks.
https://technet.microsoft.com/sv-se/library/ee619734(v=ws.10).aspx
Unique storage device identifier
For dynamic discovery to work correctly, some form of identifier must be identified and obtainable regardless of the path from the host to the storage device. Each logical unit must have a unique hardware identifier. The MPIO driver package does not use disk signatures placed in the data area of a disk for identification purposes by software. Instead, the Microsoft-provided generic DSM generates a unique identifier from the data that is provided by the storage hardware. MPIO also provides for optionally using a unique hardware identifier assigned by the device manufacturer.
You might find more info in here: https://technet.microsoft.com/sv-se/library/ee619743(v=ws.10).aspx
Ran MPCLAIM -V, seems you do not randomize the disk's serial number... it's composed 60014050000x000000.... the x being the disk number. You could put the serial time instead of the zeros. Should be random enough and it would avoid this unpleasantness. 😉 Or allow for a manual entry.
M.
It's clearly an MPIO issue, somehow, the way it checks for disk info uniqueness is using something that PetaSAN has standardized and calculates the same way on both units, making it not so unique.
Are you creating a WWN? a physical disk scsi serial number?
Here they clarify that they are not using Disk Signatures. But I am not seeing how they actually id the disks.
https://technet.microsoft.com/sv-se/library/ee619734(v=ws.10).aspx
Unique storage device identifier
For dynamic discovery to work correctly, some form of identifier must be identified and obtainable regardless of the path from the host to the storage device. Each logical unit must have a unique hardware identifier. The MPIO driver package does not use disk signatures placed in the data area of a disk for identification purposes by software. Instead, the Microsoft-provided generic DSM generates a unique identifier from the data that is provided by the storage hardware. MPIO also provides for optionally using a unique hardware identifier assigned by the device manufacturer.
You might find more info in here: https://technet.microsoft.com/sv-se/library/ee619743(v=ws.10).aspx
Ran MPCLAIM -V, seems you do not randomize the disk's serial number... it's composed 60014050000x000000.... the x being the disk number. You could put the serial time instead of the zeros. Should be random enough and it would avoid this unpleasantness. 😉 Or allow for a manual entry.
M.
sniffus
20 Posts
January 17, 2018, 3:40 pmQuote from sniffus on January 17, 2018, 3:40 pmOk, found more into....
I get it, there is a method to the madness... a bit more complicated.
https://social.technet.microsoft.com/Forums/windowsserver/en-US/6a6d6abd-61e9-47cc-8618-ffcee1e1a481/how-to-find-hardware-path-information-for-fc-mpio?forum=winserverfiles
M.
Ok, found more into....
I get it, there is a method to the madness... a bit more complicated.
M.
sniffus
20 Posts
January 17, 2018, 3:48 pmQuote from sniffus on January 17, 2018, 3:48 pmMore info,
SPC3 standards?
https://blogs.msdn.microsoft.com/adioltean/2004/12/30/how-you-can-uniquely-identify-a-lun-in-a-storage-area-network/
More info,
SPC3 standards?
admin
2,930 Posts
January 17, 2018, 4:15 pmQuote from admin on January 17, 2018, 4:15 pmYes we do generate the wwn ourselves based on the disk #, typically in other systems a random number is used, but in PetaSAN since the disk is served many different nodes it is generated from the disk id so it is computed the same across all nodes.
disk wwn is used by MPIO to make sure it is talking to the same disk across all paths.
We did not think the case where the same initiator will be talking to the different clusters, however i think we can easily solve this by appending the cluster name to it, will this wok for you ? are you using different cluster names ? if so we can send you a small fix.
Yes we do generate the wwn ourselves based on the disk #, typically in other systems a random number is used, but in PetaSAN since the disk is served many different nodes it is generated from the disk id so it is computed the same across all nodes.
disk wwn is used by MPIO to make sure it is talking to the same disk across all paths.
We did not think the case where the same initiator will be talking to the different clusters, however i think we can easily solve this by appending the cluster name to it, will this wok for you ? are you using different cluster names ? if so we can send you a small fix.
sniffus
20 Posts
January 17, 2018, 4:52 pmQuote from sniffus on January 17, 2018, 4:52 pmYes, that would work, since the cluster names have to be unique also. Don't they?
In my mind, they should. In any case, it should solve the issue.
M.
Yes, that would work, since the cluster names have to be unique also. Don't they?
In my mind, they should. In any case, it should solve the issue.
M.
admin
2,930 Posts
January 18, 2018, 4:15 pmQuote from admin on January 18, 2018, 4:15 pmThe following appends the cluster uuid with the disk # to form wwn:
https://drive.google.com/file/d/11CGp4q3QF95f45G5X3tA75ccv9chngoh/view
place it in all nodes at:
/usr/lib/python2.7/dist-packages/PetaSAN/core/lio/api.py
restart iscsi service:
systemctl restart petasan-iscsi
The following appends the cluster uuid with the disk # to form wwn:
https://drive.google.com/file/d/11CGp4q3QF95f45G5X3tA75ccv9chngoh/view
place it in all nodes at:
/usr/lib/python2.7/dist-packages/PetaSAN/core/lio/api.py
restart iscsi service:
systemctl restart petasan-iscsi
Last edited on January 18, 2018, 4:16 pm by admin · #7
sniffus
20 Posts
January 18, 2018, 9:30 pmQuote from sniffus on January 18, 2018, 9:30 pm
Thank you again, so far, the other patch hasn't brought down the house. Been too busy to create and delete a target #1.
This will help, I will not have to juggle to make the test happen now 😉
Keep you posted on my results.
Love the stats btw. we noticed two disks were not performing as they should. took them down and tested them aside... boom, dead. That is the kind of data we need, that's what makes a great solution absolufkgamazing !!!!
M.
Thank you again, so far, the other patch hasn't brought down the house. Been too busy to create and delete a target #1.
This will help, I will not have to juggle to make the test happen now 😉
Keep you posted on my results.
Love the stats btw. we noticed two disks were not performing as they should. took them down and tested them aside... boom, dead. That is the kind of data we need, that's what makes a great solution absolufkgamazing !!!!
M.
sniffus
20 Posts
January 19, 2018, 2:59 pmQuote from sniffus on January 19, 2018, 2:59 pmLooks like it will work when connecting the initiator. MPIO shows two separate devices.
The problem starts with Disk Manager, shows both devices, you can initialize them (GPT was the choice). When trying to format the disk... (on the san that was patched) takes a long time and refuses to do so, Disk manager was not updated, you rescan and it shows the partition as raw... You can only delete the volume, cannot even do a properties on it.
Dam dam dummmm... Bit more complicated. 😉
M.
Looks like it will work when connecting the initiator. MPIO shows two separate devices.
The problem starts with Disk Manager, shows both devices, you can initialize them (GPT was the choice). When trying to format the disk... (on the san that was patched) takes a long time and refuses to do so, Disk manager was not updated, you rescan and it shows the partition as raw... You can only delete the volume, cannot even do a properties on it.
Dam dam dummmm... Bit more complicated. 😉
M.
admin
2,930 Posts
January 19, 2018, 5:53 pmQuote from admin on January 19, 2018, 5:53 pmStrange..we will look at this in more detail after the 2.0 release due end of Jan. If you do need to connect to 2 clusters from single client then for now try to use different disk ids.
We did test the new wwn naming but on a single cluster with a Windows client and it worked fine. it could also be a caching issue on your Win client, maybe clear the initiator targets and reboot and make sure the old disks do not show up in Disk Manager before you attempt to reconnect. Also make sure you use different iqn prefix names and last double check there is no ip conflict between the 2 clusters and both are accessible. If you find anything let us know else we will look into it after 2.0
Strange..we will look at this in more detail after the 2.0 release due end of Jan. If you do need to connect to 2 clusters from single client then for now try to use different disk ids.
We did test the new wwn naming but on a single cluster with a Windows client and it worked fine. it could also be a caching issue on your Win client, maybe clear the initiator targets and reboot and make sure the old disks do not show up in Disk Manager before you attempt to reconnect. Also make sure you use different iqn prefix names and last double check there is no ip conflict between the 2 clusters and both are accessible. If you find anything let us know else we will look into it after 2.0
Last edited on January 19, 2018, 5:55 pm by admin · #10
Pages: 1 2
Using two PetaSAN, provisionning targets from both to same machine . . . !
sniffus
20 Posts
Quote from sniffus on January 17, 2018, 3:06 pmPetaSAN1 and Peta2. (Ok yes, two of us working on the lab . . .)
so PetaSAN1 has disks 3, 4, 5, 7 (this is where we initially tested deletions).
and Peta2 has disks 1, 2, 3. And yep, we just added disk 3 to our hypervisor1 test machine.
Now, I expected this issue. So I changed the IQN prefix on Peta2 to say peta2 instead of petasan.
Still Microsoft 2016 iSCSI initiator sees both disk 3 as the same disk, likely because of MPIO. I would have expected the IQN to be enough, shows how wrong I can be.
Is there a way to change the range of disk numbering on PetaSAN, say start at 100 on petasan1 and 500 on peta2.
Our goal is because we will have different cluster for different performance disks. Or is there a better way to qualify the disks without going all CLI?
M.
PetaSAN1 and Peta2. (Ok yes, two of us working on the lab . . .)
so PetaSAN1 has disks 3, 4, 5, 7 (this is where we initially tested deletions).
and Peta2 has disks 1, 2, 3. And yep, we just added disk 3 to our hypervisor1 test machine.
Now, I expected this issue. So I changed the IQN prefix on Peta2 to say peta2 instead of petasan.
Still Microsoft 2016 iSCSI initiator sees both disk 3 as the same disk, likely because of MPIO. I would have expected the IQN to be enough, shows how wrong I can be.
Is there a way to change the range of disk numbering on PetaSAN, say start at 100 on petasan1 and 500 on peta2.
Our goal is because we will have different cluster for different performance disks. Or is there a better way to qualify the disks without going all CLI?
M.
sniffus
20 Posts
Quote from sniffus on January 17, 2018, 3:38 pm
It's clearly an MPIO issue, somehow, the way it checks for disk info uniqueness is using something that PetaSAN has standardized and calculates the same way on both units, making it not so unique.
Are you creating a WWN? a physical disk scsi serial number?
https://social.technet.microsoft.com/Forums/windowsserver/en-US/6a6d6abd-61e9-47cc-8618-ffcee1e1a481/how-to-find-hardware-path-information-for-fc-mpio?forum=winserverfiles
Here they clarify that they are not using Disk Signatures. But I am not seeing how they actually id the disks.
https://technet.microsoft.com/sv-se/library/ee619734(v=ws.10).aspx
Unique storage device identifier
For dynamic discovery to work correctly, some form of identifier must be identified and obtainable regardless of the path from the host to the storage device. Each logical unit must have a unique hardware identifier. The MPIO driver package does not use disk signatures placed in the data area of a disk for identification purposes by software. Instead, the Microsoft-provided generic DSM generates a unique identifier from the data that is provided by the storage hardware. MPIO also provides for optionally using a unique hardware identifier assigned by the device manufacturer.
You might find more info in here: https://technet.microsoft.com/sv-se/library/ee619743(v=ws.10).aspx
Ran MPCLAIM -V, seems you do not randomize the disk's serial number... it's composed 60014050000x000000.... the x being the disk number. You could put the serial time instead of the zeros. Should be random enough and it would avoid this unpleasantness. 😉 Or allow for a manual entry.
M.
It's clearly an MPIO issue, somehow, the way it checks for disk info uniqueness is using something that PetaSAN has standardized and calculates the same way on both units, making it not so unique.
Are you creating a WWN? a physical disk scsi serial number?
Here they clarify that they are not using Disk Signatures. But I am not seeing how they actually id the disks.
https://technet.microsoft.com/sv-se/library/ee619734(v=ws.10).aspx
Unique storage device identifier
For dynamic discovery to work correctly, some form of identifier must be identified and obtainable regardless of the path from the host to the storage device. Each logical unit must have a unique hardware identifier. The MPIO driver package does not use disk signatures placed in the data area of a disk for identification purposes by software. Instead, the Microsoft-provided generic DSM generates a unique identifier from the data that is provided by the storage hardware. MPIO also provides for optionally using a unique hardware identifier assigned by the device manufacturer.
You might find more info in here: https://technet.microsoft.com/sv-se/library/ee619743(v=ws.10).aspx
Ran MPCLAIM -V, seems you do not randomize the disk's serial number... it's composed 60014050000x000000.... the x being the disk number. You could put the serial time instead of the zeros. Should be random enough and it would avoid this unpleasantness. 😉 Or allow for a manual entry.
M.
sniffus
20 Posts
Quote from sniffus on January 17, 2018, 3:40 pmOk, found more into....
I get it, there is a method to the madness... a bit more complicated.
https://social.technet.microsoft.com/Forums/windowsserver/en-US/6a6d6abd-61e9-47cc-8618-ffcee1e1a481/how-to-find-hardware-path-information-for-fc-mpio?forum=winserverfiles
M.
Ok, found more into....
I get it, there is a method to the madness... a bit more complicated.
M.
sniffus
20 Posts
Quote from sniffus on January 17, 2018, 3:48 pmMore info,
SPC3 standards?
https://blogs.msdn.microsoft.com/adioltean/2004/12/30/how-you-can-uniquely-identify-a-lun-in-a-storage-area-network/
More info,
SPC3 standards?
admin
2,930 Posts
Quote from admin on January 17, 2018, 4:15 pmYes we do generate the wwn ourselves based on the disk #, typically in other systems a random number is used, but in PetaSAN since the disk is served many different nodes it is generated from the disk id so it is computed the same across all nodes.
disk wwn is used by MPIO to make sure it is talking to the same disk across all paths.
We did not think the case where the same initiator will be talking to the different clusters, however i think we can easily solve this by appending the cluster name to it, will this wok for you ? are you using different cluster names ? if so we can send you a small fix.
Yes we do generate the wwn ourselves based on the disk #, typically in other systems a random number is used, but in PetaSAN since the disk is served many different nodes it is generated from the disk id so it is computed the same across all nodes.
disk wwn is used by MPIO to make sure it is talking to the same disk across all paths.
We did not think the case where the same initiator will be talking to the different clusters, however i think we can easily solve this by appending the cluster name to it, will this wok for you ? are you using different cluster names ? if so we can send you a small fix.
sniffus
20 Posts
Quote from sniffus on January 17, 2018, 4:52 pmYes, that would work, since the cluster names have to be unique also. Don't they?
In my mind, they should. In any case, it should solve the issue.
M.
Yes, that would work, since the cluster names have to be unique also. Don't they?
In my mind, they should. In any case, it should solve the issue.
M.
admin
2,930 Posts
Quote from admin on January 18, 2018, 4:15 pmThe following appends the cluster uuid with the disk # to form wwn:
https://drive.google.com/file/d/11CGp4q3QF95f45G5X3tA75ccv9chngoh/view
place it in all nodes at:
/usr/lib/python2.7/dist-packages/PetaSAN/core/lio/api.py
restart iscsi service:
systemctl restart petasan-iscsi
The following appends the cluster uuid with the disk # to form wwn:
https://drive.google.com/file/d/11CGp4q3QF95f45G5X3tA75ccv9chngoh/view
place it in all nodes at:
/usr/lib/python2.7/dist-packages/PetaSAN/core/lio/api.py
restart iscsi service:
systemctl restart petasan-iscsi
sniffus
20 Posts
Quote from sniffus on January 18, 2018, 9:30 pm
Thank you again, so far, the other patch hasn't brought down the house. Been too busy to create and delete a target #1.
This will help, I will not have to juggle to make the test happen now 😉
Keep you posted on my results.
Love the stats btw. we noticed two disks were not performing as they should. took them down and tested them aside... boom, dead. That is the kind of data we need, that's what makes a great solution absolufkgamazing !!!!
M.
Thank you again, so far, the other patch hasn't brought down the house. Been too busy to create and delete a target #1.
This will help, I will not have to juggle to make the test happen now 😉
Keep you posted on my results.
Love the stats btw. we noticed two disks were not performing as they should. took them down and tested them aside... boom, dead. That is the kind of data we need, that's what makes a great solution absolufkgamazing !!!!
M.
sniffus
20 Posts
Quote from sniffus on January 19, 2018, 2:59 pmLooks like it will work when connecting the initiator. MPIO shows two separate devices.
The problem starts with Disk Manager, shows both devices, you can initialize them (GPT was the choice). When trying to format the disk... (on the san that was patched) takes a long time and refuses to do so, Disk manager was not updated, you rescan and it shows the partition as raw... You can only delete the volume, cannot even do a properties on it.
Dam dam dummmm... Bit more complicated. 😉
M.
Looks like it will work when connecting the initiator. MPIO shows two separate devices.
The problem starts with Disk Manager, shows both devices, you can initialize them (GPT was the choice). When trying to format the disk... (on the san that was patched) takes a long time and refuses to do so, Disk manager was not updated, you rescan and it shows the partition as raw... You can only delete the volume, cannot even do a properties on it.
Dam dam dummmm... Bit more complicated. 😉
M.
admin
2,930 Posts
Quote from admin on January 19, 2018, 5:53 pmStrange..we will look at this in more detail after the 2.0 release due end of Jan. If you do need to connect to 2 clusters from single client then for now try to use different disk ids.
We did test the new wwn naming but on a single cluster with a Windows client and it worked fine. it could also be a caching issue on your Win client, maybe clear the initiator targets and reboot and make sure the old disks do not show up in Disk Manager before you attempt to reconnect. Also make sure you use different iqn prefix names and last double check there is no ip conflict between the 2 clusters and both are accessible. If you find anything let us know else we will look into it after 2.0
Strange..we will look at this in more detail after the 2.0 release due end of Jan. If you do need to connect to 2 clusters from single client then for now try to use different disk ids.
We did test the new wwn naming but on a single cluster with a Windows client and it worked fine. it could also be a caching issue on your Win client, maybe clear the initiator targets and reboot and make sure the old disks do not show up in Disk Manager before you attempt to reconnect. Also make sure you use different iqn prefix names and last double check there is no ip conflict between the 2 clusters and both are accessible. If you find anything let us know else we will look into it after 2.0