Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

Storage Graph Empty


pros81
8 Posts
November 21, 2018, 5:56 pm

Hi,

I recently upgraded a 3 node cluster from 2.0.0 to 2.2.0.  Ever since the upgrade, the storage graph on the main dashboard page no longer displays any data.

Any idea how I can fix this?

#1

admin
2,930 Posts
November 21, 2018, 6:03 pm

are some stats working like the cluster stats or nodes stats or nothing is displayed

do you see the black charts background and title but no data points or you see a blank area instead of the chart ?

 

#2

pros81
8 Posts
November 21, 2018, 6:24 pm

Hi,

I checked all the charts and none of them are displaying data.  I see the black graph with the title and properly labeled X and Y axis, but no data points.

#3

admin
2,930 Posts
November 21, 2018, 6:35 pm

On the first 3 nodes do a

mount | grep shared

systemctl status carbon-cache

verify the shared mount should be present on all 3 nodes, the service should running on just 1 of them. also verify the folder /opt/petasan/config/shared does contain stats files in the graphite subdir.

#4

pros81
8 Posts
November 21, 2018, 7:00 pm

Hi,

mount | grep shared = All nodes have:  gfs-vol on /opt/petasan/config/shared type fuse.glusterfs

systemctl status carbon-cache = only active on node1

The graphite folder in /opt/petasan/config/shared contains the subfolder "whisper" but no immediate stat files, though there are several more subfolders below "whisper."

#5

admin
2,930 Posts
November 21, 2018, 9:31 pm

First double check the time and time zone is setup correctly on both the browser and on the nodes themselves.

If you go back a week or a month do you see data ?

Do you see any errors or warnings on the charts like no data found or time in the future ..etc ?

As a try, on the active node1 restart the stats server:

/opt/petasan/scripts/stats-stop.sh
/opt/petasan/scripts/stats-setup.sh
/opt/petasan/scripts/stats-start.sh

 

#6

pros81
8 Posts
November 21, 2018, 9:57 pm

Hi,

As instructed, I ran the three commands on node1:

/opt/petasan/scripts/stats-stop.sh
/opt/petasan/scripts/stats-setup.sh
/opt/petasan/scripts/stats-start.sh

And now the graphs are showing data again!  Thanks for the support!

#7

wailer
75 Posts
September 2, 2019, 8:48 am

Hi, I am facing the same problem right now, already executed the commands:

/opt/petasan/scripts/stats-stop.sh
/opt/petasan/scripts/stats-setup.sh
/opt/petasan/scripts/stats-start.sh

But still graphs not working, any hints?

 

 

 

#8

admin
2,930 Posts
September 2, 2019, 4:58 pm

i assume you did restated this only on 1 node as per the earlier threads. just in case you did this on all nodes stop on all nodes via

/opt/petasan/scripts/stats-stop.sh

then find the current stats server : from the browser, view the html page source and find what ip for the stats server is being used, this is the current active server. on this do a

/opt/petasan/scripts/stats-stop.sh
/opt/petasan/scripts/stats-setup.sh
/opt/petasan/scripts/stats-start.sh

If this is not the issue:

is the shared file system mounted ? :

mount | grep shared

Can you see some charts : like cluster related stats, but not others line node stats, or are you not able to see anything ?

Do you see charts with no data points (ie you see chart axis and chart labels with empty black background) : in this case make sure your time and time zones are correct on your servers as well as the browser.

Or are you getting an error connecting to the chats server and see a white/gray background ?

Last edited on September 2, 2019, 5:02 pm by admin · #9

wailer
75 Posts
September 2, 2019, 11:22 pm

Solved!

The issue was that stats server was node 2 , not node 1 as I initially thought. Checking html request as you suggested I realized the correct stats servers. Issued stop to all nodes an restart stat service to node 2.

Now all is back on! Thanks !

#10