Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

[SOLVED] How to import OSD's

Pages: 1 2

tntbizhub
14 Posts
June 27, 2024, 5:19 pm

After reinstalling PetaSAN 3.3 on all nodes I have done the following

SAN 1

systemctl stop ceph-mgr@san-1 && systemctl stop ceph-mon@san-1&& systemctl stop ceph-osd@san-1&& systemctl stop ceph.target

 

SAN 2

systemctl stop ceph-mgr@san-2 && systemctl stop ceph-mon@san-2&& systemctl stop ceph-osd@san-2&& systemctl stop ceph.target

 

SAN 3

systemctl stop ceph-mgr@san-3 && systemctl stop ceph-mon@san-3&& systemctl stop ceph-osd@san-3&& systemctl stop ceph.target

 

SAN3

mkdir -p /tmp/monstore-san-3

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-1 --op update-mon-db --mon-store-path /tmp/monstore-san-3 --no-mon-config

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-2 --op update-mon-db --mon-store-path /tmp/monstore-san-3 --no-mon-config

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-4 --op update-mon-db --mon-store-path /tmp/monstore-san-3 --no-mon-config

rsync -azP --progress /tmp/monstore-san-3/ root@10.254.254.12:/tmp/monstore-san-2/

 

SAN2

mkdir -p /tmp/monstore-san-2

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-6 --op update-mon-db --mon-store-path /tmp/monstore-san-2 --no-mon-config

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-10 --op update-mon-db --mon-store-path /tmp/monstore-san-2 --no-mon-config

 

rsync -azP --progress /tmp/monstore-san-2/ root@10.254.254.11:/tmp/monstore-san-1/

 

SAN1

mkdir -p /tmp/monstore-san-1

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-0 --op update-mon-db --mon-store-path /tmp/monstore-san-1 --no-mon-config

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-11 --op update-mon-db --mon-store-path /tmp/monstore-san-1 --no-mon-config

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-7 --op update-mon-db --mon-store-path /tmp/monstore-san-1 --no-mon-config

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-8 --op update-mon-db --mon-store-path /tmp/monstore-san-1 --no-mon-config

ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-9 --op update-mon-db --mon-store-path /tmp/monstore-san-1 --no-mon-config

 

COPY FROM SAN-3 TO ALL NODES

rm -rf /var/lib/ceph/mon/ceph-san-3/store.db/

 

rsync -azP --progress /tmp/monstore-san-1/ root@10.254.254.13:/var/lib/ceph/mon/ceph-san-3/

rsync -azP --progress /tmp/monstore-san-1/ root@10.254.254.12:/var/lib/ceph/mon/ceph-san-2/

rsync -azP --progress /tmp/monstore-san-1/ /var/lib/ceph/mon/ceph-san-1/

 

 

SAN 1

ceph-monstore-tool /var/lib/ceph/mon/ceph-san-1/ rebuild -- --keyring /etc/ceph/ceph.client.admin.keyring  --mon-ids san-1 san-2 san-3

chown -R ceph:ceph /var/lib/ceph/mon/ceph-san-1/

systemctl start ceph-mon@san-1

 

SAN 2

ceph-monstore-tool /var/lib/ceph/mon/ceph-san-2/ rebuild -- --keyring /etc/ceph/ceph.client.admin.keyring  --mon-ids san-1 san-2 san-3

chown -R ceph:ceph /var/lib/ceph/mon/ceph-san-2/

systemctl start ceph-mon@san-2

SAN 3

ceph-monstore-tool /var/lib/ceph/mon/ceph-san-3/ rebuild -- --keyring /etc/ceph/ceph.client.admin.keyring  --mon-ids san-1 san-2 san-3

chown -R ceph:ceph /var/lib/ceph/mon/ceph-san-3/

systemctl start ceph-mon@san-3

--------------------------------------------------

Current status:

cluster:
id: 81acea00-fc7b-4131-a038-43ef1aa2efcc
health: HEALTH_WARN
mons are allowing insecure global_id reclaim
no active mgr

services:
mon: 3 daemons, quorum san-1,san-2,san-3 (age 11m)
mgr: no daemons active
osd: 12 osds: 10 up (since 7m), 10 in (since 4h); 3 remapped pgs

data:
pools: 0 pools, 0 pgs
objects: 0 objects, 0 B
usage: 0 B used, 0 B / 0 B avail
pgs:

 

I think mon's are working but not the the mgr.... and no data in it !?

Have I missed steps? I have been reading lots of documentation on this... Any help is appreciated...

Last edited on June 27, 2024, 5:25 pm by tntbizhub · #11

tntbizhub
14 Posts
July 15, 2024, 10:13 am

Finally the cluster is working again... Had to re-configure iSCSI services, detach iSCSI and re-attach... et violá!

Working again...  thank you !

#12
Pages: 1 2