Petasan disk list empty (reboot and UI reporting issues)
aaxworthy
2 Posts
June 7, 2023, 12:47 pmQuote from aaxworthy on June 7, 2023, 12:47 pmWe have a 4 node cluster all nodes are running 3.1.0. In the physical disk list of all 4 nodes the osds are showing up but with no disk information. Then all disks are listed below not assigned to any OSD.
We are also having issues where the OSDs do not come back up on reboot. I am able to get the osds back up with "ceph-volume lvm activate --all".
I have tried the ./print_osd_list.py and it does not return any osds.
I have also tried restarting petasan-start-osds.service with no affect. Then I have tried to manually run the start_all_osds.py script and it runs through and does not start any osds.
Could the issues be related, the dashboard reporting issues with OSDs, osds not starting on reboot, and the disk list being blank?
Any information on this would be greatly appreciated.
Recent patch for nodes over 26 disks per host did not resolve.
We have a 4 node cluster all nodes are running 3.1.0. In the physical disk list of all 4 nodes the osds are showing up but with no disk information. Then all disks are listed below not assigned to any OSD.
We are also having issues where the OSDs do not come back up on reboot. I am able to get the osds back up with "ceph-volume lvm activate --all".
I have tried the ./print_osd_list.py and it does not return any osds.
I have also tried restarting petasan-start-osds.service with no affect. Then I have tried to manually run the start_all_osds.py script and it runs through and does not start any osds.
Could the issues be related, the dashboard reporting issues with OSDs, osds not starting on reboot, and the disk list being blank?
Any information on this would be greatly appreciated.
Recent patch for nodes over 26 disks per host did not resolve.
admin
2,930 Posts
June 7, 2023, 6:12 pmQuote from admin on June 7, 2023, 6:12 pmWas there anything done that triggered this ?
Have you done any custom changes ? installed custom software ?
Do you see errors in
/opt/petasan/log/PetaSAN.log
Was there anything done that triggered this ?
Have you done any custom changes ? installed custom software ?
Do you see errors in
/opt/petasan/log/PetaSAN.log
aaxworthy
2 Posts
June 8, 2023, 4:14 pmQuote from aaxworthy on June 8, 2023, 4:14 pmNo errors are reporting in the Petasan.log.
Not custom changes/softwares were used on this cluster.
A thing to note, is that the cluster was built with a low amount of disks per node, then expanded to have 30+ osds per node.
No errors are reporting in the Petasan.log.
Not custom changes/softwares were used on this cluster.
A thing to note, is that the cluster was built with a low amount of disks per node, then expanded to have 30+ osds per node.
Petasan disk list empty (reboot and UI reporting issues)
aaxworthy
2 Posts
Quote from aaxworthy on June 7, 2023, 12:47 pmWe have a 4 node cluster all nodes are running 3.1.0. In the physical disk list of all 4 nodes the osds are showing up but with no disk information. Then all disks are listed below not assigned to any OSD.We are also having issues where the OSDs do not come back up on reboot. I am able to get the osds back up with "ceph-volume lvm activate --all".I have tried the ./print_osd_list.py and it does not return any osds.I have also tried restarting petasan-start-osds.service with no affect. Then I have tried to manually run the start_all_osds.py script and it runs through and does not start any osds.Could the issues be related, the dashboard reporting issues with OSDs, osds not starting on reboot, and the disk list being blank?
Any information on this would be greatly appreciated.Recent patch for nodes over 26 disks per host did not resolve.
Could the issues be related, the dashboard reporting issues with OSDs, osds not starting on reboot, and the disk list being blank?
admin
2,930 Posts
Quote from admin on June 7, 2023, 6:12 pmWas there anything done that triggered this ?
Have you done any custom changes ? installed custom software ?
Do you see errors in
/opt/petasan/log/PetaSAN.log
Was there anything done that triggered this ?
Have you done any custom changes ? installed custom software ?
Do you see errors in
/opt/petasan/log/PetaSAN.log
aaxworthy
2 Posts
Quote from aaxworthy on June 8, 2023, 4:14 pmNo errors are reporting in the Petasan.log.
Not custom changes/softwares were used on this cluster.
A thing to note, is that the cluster was built with a low amount of disks per node, then expanded to have 30+ osds per node.
No errors are reporting in the Petasan.log.
Not custom changes/softwares were used on this cluster.
A thing to note, is that the cluster was built with a low amount of disks per node, then expanded to have 30+ osds per node.