iSCSI drive syncing issue, changes made by two host connected to the same drive do not apply
JG
26 Posts
July 17, 2020, 5:14 pmQuote from JG on July 17, 2020, 5:14 pmHello all,
We are installing a new three node cluster. The hardware of each node is as follows:
Dell R630 + (2) Xeon E5-2640 v3 + 128 GB RAM + (2) HDD 15k 320 GB RAID 1 + (2) SSD Enterprise (2 TB) for OSDs + (2) 1 Gb / s copper Ethernet ports + ( 4) SFP + 10Gb / s fiber
There are also 2 Dell S4112F-ON switches (linked through 1 QSFP 40 Gb / s port for VLT)
Two bond interfaces were created with the SFP + ports for the public network and for the PetaSAN iSCSI network. Jumbo frames have been enabled.
All connectivity tests have been successful, the cluster starts and the iSCSI disks are created (multipath enabled).
Issue:
Windows hosts are able to connect to the iSCSI drive created by PetaSAN but changes made to the drive by each host are not reflected on the other host unless the drive is Off-Line / On-Line in Windows Disk Manager. Windows reports a different number of used / free space on each of the drives. For example, if something is copied to the drive from one of the Windows servers, the used / available space on the other server is unchanged and changes made to the drive from the other server are also not reflected.
We have done a number of tests, below I mention some of them (for all iSCSI configurations we have followed the PDF "PetaSAN_Win2019_MPIO.pdf" available at petasan.org)
- We have connected different versions of Windows Server (2019 and 2012) to the iSCSI drive.
- We have disabled multipath when creating the disk in PetaSAN.
- We have used one of the PetaSAN servers as the initiating iSCSI and have been able to connect to the iSCSI drive but none of the changes made from Linux on the iSCSI drive are reflected on the iSCSI drive as well as they are not reflected on the iSCSI drive. any of the changes made from Windows servers.
- We have recreated in a virtual environment a PetaSAN cluster with 3 nodes and two Windows servers and we obtained the same results.
- We have tried creating the cluster with PetaSAN version 2.5 instead of version 2.5.3.
- PetaSAN Logs do not show anything out of the ordinary nor can we see any error in Linux ones (dmesg by example)
Please let me know if I can provide any further info.
Any help will be highly appreciated.
Thank you.
Hello all,
We are installing a new three node cluster. The hardware of each node is as follows:
Dell R630 + (2) Xeon E5-2640 v3 + 128 GB RAM + (2) HDD 15k 320 GB RAID 1 + (2) SSD Enterprise (2 TB) for OSDs + (2) 1 Gb / s copper Ethernet ports + ( 4) SFP + 10Gb / s fiber
There are also 2 Dell S4112F-ON switches (linked through 1 QSFP 40 Gb / s port for VLT)
Two bond interfaces were created with the SFP + ports for the public network and for the PetaSAN iSCSI network. Jumbo frames have been enabled.
All connectivity tests have been successful, the cluster starts and the iSCSI disks are created (multipath enabled).
Issue:
Windows hosts are able to connect to the iSCSI drive created by PetaSAN but changes made to the drive by each host are not reflected on the other host unless the drive is Off-Line / On-Line in Windows Disk Manager. Windows reports a different number of used / free space on each of the drives. For example, if something is copied to the drive from one of the Windows servers, the used / available space on the other server is unchanged and changes made to the drive from the other server are also not reflected.
We have done a number of tests, below I mention some of them (for all iSCSI configurations we have followed the PDF "PetaSAN_Win2019_MPIO.pdf" available at petasan.org)
- We have connected different versions of Windows Server (2019 and 2012) to the iSCSI drive.
- We have disabled multipath when creating the disk in PetaSAN.
- We have used one of the PetaSAN servers as the initiating iSCSI and have been able to connect to the iSCSI drive but none of the changes made from Linux on the iSCSI drive are reflected on the iSCSI drive as well as they are not reflected on the iSCSI drive. any of the changes made from Windows servers.
- We have recreated in a virtual environment a PetaSAN cluster with 3 nodes and two Windows servers and we obtained the same results.
- We have tried creating the cluster with PetaSAN version 2.5 instead of version 2.5.3.
- PetaSAN Logs do not show anything out of the ordinary nor can we see any error in Linux ones (dmesg by example)
Please let me know if I can provide any further info.
Any help will be highly appreciated.
Thank you.
admin
2,930 Posts
July 17, 2020, 5:26 pmQuote from admin on July 17, 2020, 5:26 pmTo have 2 client hosts connected actively on the same disk, you need a clustered file system: Windows CSV ( clustered shared volumes) which is used by Windows Clustering/Hyper-v, VMWare VMFS, Oracle OCFS2 ...etc. Look at the use cases we have for Hypwe-v and SOFS for how to setup Windows Clustering.
If you use NTFS then the passive node needs to be offline and in case of failover, you will connect the disk.
Windows clustering depends on support for Persistent Reservations, not all SANs support this. PetaSAN has custom kernel changes to support this.
Note that PetaSAN itself is active/active, so an NTFS client accesses a disk served from multiple PetaSAN nodes actively. Your questions deals with clustering on the client side.
To have 2 client hosts connected actively on the same disk, you need a clustered file system: Windows CSV ( clustered shared volumes) which is used by Windows Clustering/Hyper-v, VMWare VMFS, Oracle OCFS2 ...etc. Look at the use cases we have for Hypwe-v and SOFS for how to setup Windows Clustering.
If you use NTFS then the passive node needs to be offline and in case of failover, you will connect the disk.
Windows clustering depends on support for Persistent Reservations, not all SANs support this. PetaSAN has custom kernel changes to support this.
Note that PetaSAN itself is active/active, so an NTFS client accesses a disk served from multiple PetaSAN nodes actively. Your questions deals with clustering on the client side.
Last edited on July 18, 2020, 1:48 pm by admin · #2
JG
26 Posts
July 21, 2020, 2:03 pmQuote from JG on July 21, 2020, 2:03 pmThank you for your super quick response. Your reply answer all our questions. This topic can be considered closed.
Thank you for your super quick response. Your reply answer all our questions. This topic can be considered closed.
Last edited on July 21, 2020, 2:03 pm by JG · #3
iSCSI drive syncing issue, changes made by two host connected to the same drive do not apply
JG
26 Posts
Quote from JG on July 17, 2020, 5:14 pmHello all,
We are installing a new three node cluster. The hardware of each node is as follows:
Dell R630 + (2) Xeon E5-2640 v3 + 128 GB RAM + (2) HDD 15k 320 GB RAID 1 + (2) SSD Enterprise (2 TB) for OSDs + (2) 1 Gb / s copper Ethernet ports + ( 4) SFP + 10Gb / s fiber
There are also 2 Dell S4112F-ON switches (linked through 1 QSFP 40 Gb / s port for VLT)
Two bond interfaces were created with the SFP + ports for the public network and for the PetaSAN iSCSI network. Jumbo frames have been enabled.
All connectivity tests have been successful, the cluster starts and the iSCSI disks are created (multipath enabled).Issue:
Windows hosts are able to connect to the iSCSI drive created by PetaSAN but changes made to the drive by each host are not reflected on the other host unless the drive is Off-Line / On-Line in Windows Disk Manager. Windows reports a different number of used / free space on each of the drives. For example, if something is copied to the drive from one of the Windows servers, the used / available space on the other server is unchanged and changes made to the drive from the other server are also not reflected.We have done a number of tests, below I mention some of them (for all iSCSI configurations we have followed the PDF "PetaSAN_Win2019_MPIO.pdf" available at petasan.org)
- We have connected different versions of Windows Server (2019 and 2012) to the iSCSI drive.
- We have disabled multipath when creating the disk in PetaSAN.
- We have used one of the PetaSAN servers as the initiating iSCSI and have been able to connect to the iSCSI drive but none of the changes made from Linux on the iSCSI drive are reflected on the iSCSI drive as well as they are not reflected on the iSCSI drive. any of the changes made from Windows servers.
- We have recreated in a virtual environment a PetaSAN cluster with 3 nodes and two Windows servers and we obtained the same results.
- We have tried creating the cluster with PetaSAN version 2.5 instead of version 2.5.3.
- PetaSAN Logs do not show anything out of the ordinary nor can we see any error in Linux ones (dmesg by example)Please let me know if I can provide any further info.
Any help will be highly appreciated.
Thank you.
Hello all,
We are installing a new three node cluster. The hardware of each node is as follows:
Dell R630 + (2) Xeon E5-2640 v3 + 128 GB RAM + (2) HDD 15k 320 GB RAID 1 + (2) SSD Enterprise (2 TB) for OSDs + (2) 1 Gb / s copper Ethernet ports + ( 4) SFP + 10Gb / s fiber
There are also 2 Dell S4112F-ON switches (linked through 1 QSFP 40 Gb / s port for VLT)
Two bond interfaces were created with the SFP + ports for the public network and for the PetaSAN iSCSI network. Jumbo frames have been enabled.
All connectivity tests have been successful, the cluster starts and the iSCSI disks are created (multipath enabled).
Issue:
Windows hosts are able to connect to the iSCSI drive created by PetaSAN but changes made to the drive by each host are not reflected on the other host unless the drive is Off-Line / On-Line in Windows Disk Manager. Windows reports a different number of used / free space on each of the drives. For example, if something is copied to the drive from one of the Windows servers, the used / available space on the other server is unchanged and changes made to the drive from the other server are also not reflected.
We have done a number of tests, below I mention some of them (for all iSCSI configurations we have followed the PDF "PetaSAN_Win2019_MPIO.pdf" available at petasan.org)
- We have connected different versions of Windows Server (2019 and 2012) to the iSCSI drive.
- We have disabled multipath when creating the disk in PetaSAN.
- We have used one of the PetaSAN servers as the initiating iSCSI and have been able to connect to the iSCSI drive but none of the changes made from Linux on the iSCSI drive are reflected on the iSCSI drive as well as they are not reflected on the iSCSI drive. any of the changes made from Windows servers.
- We have recreated in a virtual environment a PetaSAN cluster with 3 nodes and two Windows servers and we obtained the same results.
- We have tried creating the cluster with PetaSAN version 2.5 instead of version 2.5.3.
- PetaSAN Logs do not show anything out of the ordinary nor can we see any error in Linux ones (dmesg by example)
Please let me know if I can provide any further info.
Any help will be highly appreciated.
Thank you.
admin
2,930 Posts
Quote from admin on July 17, 2020, 5:26 pmTo have 2 client hosts connected actively on the same disk, you need a clustered file system: Windows CSV ( clustered shared volumes) which is used by Windows Clustering/Hyper-v, VMWare VMFS, Oracle OCFS2 ...etc. Look at the use cases we have for Hypwe-v and SOFS for how to setup Windows Clustering.
If you use NTFS then the passive node needs to be offline and in case of failover, you will connect the disk.
Windows clustering depends on support for Persistent Reservations, not all SANs support this. PetaSAN has custom kernel changes to support this.
Note that PetaSAN itself is active/active, so an NTFS client accesses a disk served from multiple PetaSAN nodes actively. Your questions deals with clustering on the client side.
To have 2 client hosts connected actively on the same disk, you need a clustered file system: Windows CSV ( clustered shared volumes) which is used by Windows Clustering/Hyper-v, VMWare VMFS, Oracle OCFS2 ...etc. Look at the use cases we have for Hypwe-v and SOFS for how to setup Windows Clustering.
If you use NTFS then the passive node needs to be offline and in case of failover, you will connect the disk.
Windows clustering depends on support for Persistent Reservations, not all SANs support this. PetaSAN has custom kernel changes to support this.
Note that PetaSAN itself is active/active, so an NTFS client accesses a disk served from multiple PetaSAN nodes actively. Your questions deals with clustering on the client side.
JG
26 Posts
Quote from JG on July 21, 2020, 2:03 pmThank you for your super quick response. Your reply answer all our questions. This topic can be considered closed.
Thank you for your super quick response. Your reply answer all our questions. This topic can be considered closed.