problem with MON up/down
f.cuseo
66 Posts
February 1, 2024, 1:30 pmQuote from f.cuseo on February 1, 2024, 1:30 pmHello.
In my 10 host cluster, I always have this monitor down/election
I can't find anything useful in logfiles...
Do you have some suggestion?
2024-02-01T14:21:14.265743+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:21:14.404225+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:21:14.604433+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:21:14.604451+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:21:14.685276+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:24:06.760371+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:24:06.973041+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:24:07.281550+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:24:07.383463+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:24:07.651490+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:25:14.121389+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:25:14.234992+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:25:14.309124+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:25:14.309139+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:25:14.389404+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:26:04.449079+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:26:10.688098+0100 mon.petasan-01 [INF] mon.petasan-01 is new leader, mons petasan-01,petasan-02 in quorum (ranks 1,2)
2024-02-01T14:26:10.845181+0100 mon.petasan-01 [WRN] Health check failed: 1/3 mons down, quorum petasan-01,petasan-02 (MON_DOWN)
2024-02-01T14:26:10.909817+0100 mon.petasan-01 [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:26:10.909837+0100 mon.petasan-01 [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:26:10.909847+0100 mon.petasan-01 [WRN] mon.petasan-03 (rank 0) addr [v2:172.16.34.13:3300/0,v1:172.16.34.13:6789/0] is down (out of quorum)
2024-02-01T14:26:07.620379+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:26:19.622234+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:26:19.812316+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:26:19.812330+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:26:19.872207+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:27:21.335371+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:27:21.409207+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:27:26.488165+0100 mon.petasan-01 [INF] mon.petasan-01 is new leader, mons petasan-01,petasan-02 in quorum (ranks 1,2)
2024-02-01T14:27:26.617135+0100 mon.petasan-01 [WRN] Health check failed: 1/3 mons down, quorum petasan-01,petasan-02 (MON_DOWN)
2024-02-01T14:27:26.712139+0100 mon.petasan-01 [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:27:26.712156+0100 mon.petasan-01 [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:27:26.712165+0100 mon.petasan-01 [WRN] mon.petasan-03 (rank 0) addr [v2:172.16.34.13:3300/0,v1:172.16.34.13:6789/0] is down (out of quorum)
2024-02-01T14:27:35.886312+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:27:36.418033+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:27:36.418634+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:27:36.472644+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:27:36.598973+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:27:36.703889+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:27:36.703910+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:27:36.777158+0100 mon.petasan-03 [INF] overall HEALTH_OK
Hello.
In my 10 host cluster, I always have this monitor down/election
I can't find anything useful in logfiles...
Do you have some suggestion?
2024-02-01T14:21:14.265743+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:21:14.404225+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:21:14.604433+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:21:14.604451+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:21:14.685276+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:24:06.760371+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:24:06.973041+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:24:07.281550+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:24:07.383463+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:24:07.651490+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:25:14.121389+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:25:14.234992+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:25:14.309124+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:25:14.309139+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:25:14.389404+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:26:04.449079+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:26:10.688098+0100 mon.petasan-01 [INF] mon.petasan-01 is new leader, mons petasan-01,petasan-02 in quorum (ranks 1,2)
2024-02-01T14:26:10.845181+0100 mon.petasan-01 [WRN] Health check failed: 1/3 mons down, quorum petasan-01,petasan-02 (MON_DOWN)
2024-02-01T14:26:10.909817+0100 mon.petasan-01 [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:26:10.909837+0100 mon.petasan-01 [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:26:10.909847+0100 mon.petasan-01 [WRN] mon.petasan-03 (rank 0) addr [v2:172.16.34.13:3300/0,v1:172.16.34.13:6789/0] is down (out of quorum)
2024-02-01T14:26:07.620379+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:26:19.622234+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:26:19.812316+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:26:19.812330+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:26:19.872207+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:27:21.335371+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:27:21.409207+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:27:26.488165+0100 mon.petasan-01 [INF] mon.petasan-01 is new leader, mons petasan-01,petasan-02 in quorum (ranks 1,2)
2024-02-01T14:27:26.617135+0100 mon.petasan-01 [WRN] Health check failed: 1/3 mons down, quorum petasan-01,petasan-02 (MON_DOWN)
2024-02-01T14:27:26.712139+0100 mon.petasan-01 [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:27:26.712156+0100 mon.petasan-01 [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:27:26.712165+0100 mon.petasan-01 [WRN] mon.petasan-03 (rank 0) addr [v2:172.16.34.13:3300/0,v1:172.16.34.13:6789/0] is down (out of quorum)
2024-02-01T14:27:35.886312+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:27:36.418033+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:27:36.418634+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:27:36.472644+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:27:36.598973+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:27:36.703889+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:27:36.703910+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:27:36.777158+0100 mon.petasan-03 [INF] overall HEALTH_OK
problem with MON up/down
f.cuseo
66 Posts
Quote from f.cuseo on February 1, 2024, 1:30 pmHello.
In my 10 host cluster, I always have this monitor down/election
I can't find anything useful in logfiles...
Do you have some suggestion?
2024-02-01T14:21:14.265743+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:21:14.404225+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:21:14.604433+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:21:14.604451+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:21:14.685276+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:24:06.760371+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:24:06.973041+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:24:07.281550+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:24:07.383463+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:24:07.651490+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:25:14.121389+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:25:14.234992+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:25:14.309124+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:25:14.309139+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:25:14.389404+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:26:04.449079+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:26:10.688098+0100 mon.petasan-01 [INF] mon.petasan-01 is new leader, mons petasan-01,petasan-02 in quorum (ranks 1,2)
2024-02-01T14:26:10.845181+0100 mon.petasan-01 [WRN] Health check failed: 1/3 mons down, quorum petasan-01,petasan-02 (MON_DOWN)
2024-02-01T14:26:10.909817+0100 mon.petasan-01 [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:26:10.909837+0100 mon.petasan-01 [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:26:10.909847+0100 mon.petasan-01 [WRN] mon.petasan-03 (rank 0) addr [v2:172.16.34.13:3300/0,v1:172.16.34.13:6789/0] is down (out of quorum)
2024-02-01T14:26:07.620379+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:26:19.622234+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:26:19.812316+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:26:19.812330+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:26:19.872207+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:27:21.335371+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:27:21.409207+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:27:26.488165+0100 mon.petasan-01 [INF] mon.petasan-01 is new leader, mons petasan-01,petasan-02 in quorum (ranks 1,2)
2024-02-01T14:27:26.617135+0100 mon.petasan-01 [WRN] Health check failed: 1/3 mons down, quorum petasan-01,petasan-02 (MON_DOWN)
2024-02-01T14:27:26.712139+0100 mon.petasan-01 [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:27:26.712156+0100 mon.petasan-01 [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:27:26.712165+0100 mon.petasan-01 [WRN] mon.petasan-03 (rank 0) addr [v2:172.16.34.13:3300/0,v1:172.16.34.13:6789/0] is down (out of quorum)
2024-02-01T14:27:35.886312+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:27:36.418033+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:27:36.418634+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:27:36.472644+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:27:36.598973+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:27:36.703889+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:27:36.703910+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:27:36.777158+0100 mon.petasan-03 [INF] overall HEALTH_OK
Hello.
In my 10 host cluster, I always have this monitor down/election
I can't find anything useful in logfiles...
Do you have some suggestion?
2024-02-01T14:21:14.265743+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:21:14.404225+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:21:14.604433+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:21:14.604451+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:21:14.685276+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:24:06.760371+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:24:06.973041+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:24:07.281550+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:24:07.383463+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:24:07.651490+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:25:14.121389+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:25:14.234992+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:25:14.309124+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:25:14.309139+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:25:14.389404+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:26:04.449079+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:26:10.688098+0100 mon.petasan-01 [INF] mon.petasan-01 is new leader, mons petasan-01,petasan-02 in quorum (ranks 1,2)
2024-02-01T14:26:10.845181+0100 mon.petasan-01 [WRN] Health check failed: 1/3 mons down, quorum petasan-01,petasan-02 (MON_DOWN)
2024-02-01T14:26:10.909817+0100 mon.petasan-01 [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:26:10.909837+0100 mon.petasan-01 [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:26:10.909847+0100 mon.petasan-01 [WRN] mon.petasan-03 (rank 0) addr [v2:172.16.34.13:3300/0,v1:172.16.34.13:6789/0] is down (out of quorum)
2024-02-01T14:26:07.620379+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:26:19.622234+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:26:19.812316+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:26:19.812330+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:26:19.872207+0100 mon.petasan-03 [INF] overall HEALTH_OK
2024-02-01T14:27:21.335371+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:27:21.409207+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:27:26.488165+0100 mon.petasan-01 [INF] mon.petasan-01 is new leader, mons petasan-01,petasan-02 in quorum (ranks 1,2)
2024-02-01T14:27:26.617135+0100 mon.petasan-01 [WRN] Health check failed: 1/3 mons down, quorum petasan-01,petasan-02 (MON_DOWN)
2024-02-01T14:27:26.712139+0100 mon.petasan-01 [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:27:26.712156+0100 mon.petasan-01 [WRN] [WRN] MON_DOWN: 1/3 mons down, quorum petasan-01,petasan-02
2024-02-01T14:27:26.712165+0100 mon.petasan-01 [WRN] mon.petasan-03 (rank 0) addr [v2:172.16.34.13:3300/0,v1:172.16.34.13:6789/0] is down (out of quorum)
2024-02-01T14:27:35.886312+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:27:36.418033+0100 mon.petasan-01 [INF] mon.petasan-01 calling monitor election
2024-02-01T14:27:36.418634+0100 mon.petasan-02 [INF] mon.petasan-02 calling monitor election
2024-02-01T14:27:36.472644+0100 mon.petasan-03 [INF] mon.petasan-03 calling monitor election
2024-02-01T14:27:36.598973+0100 mon.petasan-03 [INF] mon.petasan-03 is new leader, mons petasan-03,petasan-01,petasan-02 in quorum (ranks 0,1,2)
2024-02-01T14:27:36.703889+0100 mon.petasan-03 [INF] Health check cleared: MON_DOWN (was: 1/3 mons down, quorum petasan-01,petasan-02)
2024-02-01T14:27:36.703910+0100 mon.petasan-03 [INF] Cluster is now healthy
2024-02-01T14:27:36.777158+0100 mon.petasan-03 [INF] overall HEALTH_OK