Admin web console intermittently unavailable
Vlad016
1 Post
June 2, 2022, 6:44 amQuote from Vlad016 on June 2, 2022, 6:44 amGood morning.
There is a cluster consisting of 3 nodes, 68 OSD by 6 Tb (Journal & Cache - SSD). Version 2.4.0. Operated 3 years.
The following problem periodically appears: when connecting to the web administration console, the console is not available.
There is no web server error. The browser is in connection standby mode.
With this error, on the node to which I am trying to connect, the admin.py daemon takes 100% of the processor resources.
Restarting the web server does not solve the problem.
This can only be fixed by completely restarting the node, but after that a very long cluster rebalance, even if the node is freed
from iSCSI disks.
Please help me to solve this problem. How can I restore access to the console without restarting
completely node.
Thanks to.
Vladimir.
Good morning.
There is a cluster consisting of 3 nodes, 68 OSD by 6 Tb (Journal & Cache - SSD). Version 2.4.0. Operated 3 years.
The following problem periodically appears: when connecting to the web administration console, the console is not available.
There is no web server error. The browser is in connection standby mode.
With this error, on the node to which I am trying to connect, the admin.py daemon takes 100% of the processor resources.
Restarting the web server does not solve the problem.
This can only be fixed by completely restarting the node, but after that a very long cluster rebalance, even if the node is freed
from iSCSI disks.
Please help me to solve this problem. How can I restore access to the console without restarting
completely node.
Thanks to.
Vladimir.
admin
2,930 Posts
June 2, 2022, 9:33 amQuote from admin on June 2, 2022, 9:33 amHow much ram do you have ? not clear are you using SSD for cache and journal but OSDs are HDDs ? how large are your cache disks / partitions ( you need 2% of cache partition in ram ) ?
The cluster has been running for 3 years, did the issue happen recently ? anything changed ? did you install additional software, did upgrades ? did you increase the load ?
Is the issue with Admin ui only ? is the cluster health OK ?
Do you see any errors in syslog or PetaSAN.log when the problem happens ?
From node stats charts, do you see any load saturation in ram/cpu/disks/network utilization ?
How much ram do you have ? not clear are you using SSD for cache and journal but OSDs are HDDs ? how large are your cache disks / partitions ( you need 2% of cache partition in ram ) ?
The cluster has been running for 3 years, did the issue happen recently ? anything changed ? did you install additional software, did upgrades ? did you increase the load ?
Is the issue with Admin ui only ? is the cluster health OK ?
Do you see any errors in syslog or PetaSAN.log when the problem happens ?
From node stats charts, do you see any load saturation in ram/cpu/disks/network utilization ?
Admin web console intermittently unavailable
Vlad016
1 Post
Quote from Vlad016 on June 2, 2022, 6:44 amGood morning.
There is a cluster consisting of 3 nodes, 68 OSD by 6 Tb (Journal & Cache - SSD). Version 2.4.0. Operated 3 years.
The following problem periodically appears: when connecting to the web administration console, the console is not available.
There is no web server error. The browser is in connection standby mode.
With this error, on the node to which I am trying to connect, the admin.py daemon takes 100% of the processor resources.
Restarting the web server does not solve the problem.
This can only be fixed by completely restarting the node, but after that a very long cluster rebalance, even if the node is freed
from iSCSI disks.
Please help me to solve this problem. How can I restore access to the console without restarting
completely node.Thanks to.
Vladimir.
Good morning.
There is a cluster consisting of 3 nodes, 68 OSD by 6 Tb (Journal & Cache - SSD). Version 2.4.0. Operated 3 years.
The following problem periodically appears: when connecting to the web administration console, the console is not available.
There is no web server error. The browser is in connection standby mode.
With this error, on the node to which I am trying to connect, the admin.py daemon takes 100% of the processor resources.
Restarting the web server does not solve the problem.
This can only be fixed by completely restarting the node, but after that a very long cluster rebalance, even if the node is freed
from iSCSI disks.
Please help me to solve this problem. How can I restore access to the console without restarting
completely node.
Thanks to.
Vladimir.
admin
2,930 Posts
Quote from admin on June 2, 2022, 9:33 amHow much ram do you have ? not clear are you using SSD for cache and journal but OSDs are HDDs ? how large are your cache disks / partitions ( you need 2% of cache partition in ram ) ?
The cluster has been running for 3 years, did the issue happen recently ? anything changed ? did you install additional software, did upgrades ? did you increase the load ?
Is the issue with Admin ui only ? is the cluster health OK ?
Do you see any errors in syslog or PetaSAN.log when the problem happens ?
From node stats charts, do you see any load saturation in ram/cpu/disks/network utilization ?
How much ram do you have ? not clear are you using SSD for cache and journal but OSDs are HDDs ? how large are your cache disks / partitions ( you need 2% of cache partition in ram ) ?
The cluster has been running for 3 years, did the issue happen recently ? anything changed ? did you install additional software, did upgrades ? did you increase the load ?
Is the issue with Admin ui only ? is the cluster health OK ?
Do you see any errors in syslog or PetaSAN.log when the problem happens ?
From node stats charts, do you see any load saturation in ram/cpu/disks/network utilization ?