Forums

Home / Forums

You need to log in to create posts and topics. Login · Register

New Node Installation on 2.3.1 Cluster


bill gottlieb
26 Posts
January 10, 2020, 4:44 pm
Greetings.
I am trying to add a new node to my existing cluster.
All goes well until I configure the backend IPs in the Deployment Wizard GUI, then hit "Submit"
Then I get this:
"Internal Server Error

The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application."

- Bill
#1

admin
2,930 Posts
January 10, 2020, 5:12 pm

do you see any errors in /opt/petasan/log/PetaSAN.log or in syslog ?

#2

bill gottlieb
26 Posts
January 22, 2020, 12:02 pm

Yes. May I email it to you for review? I don't see a way to upload files to this..

If so, to whom shall I send it?

#3

admin
2,930 Posts
January 22, 2020, 7:14 pm

yes to :

contact-us at petasan.org

i recommend you recheck your connections then re-try 1 more time before sending

 

#4

admin
2,930 Posts
January 23, 2020, 7:58 am

Were the existing nodes you want to join to, installed new from 2.3.1 or were they upgraded from prev releases ? if the later you need to follow the 2.3.1 upgrade guide which after using the installer, there is a sequence you have to follow to upgrade to ceph nautilus. once this is done you can then join new nodes.

#5

bill gottlieb
26 Posts
January 23, 2020, 5:03 pm

They were all installed new from 2.3.1

  • Bill
#6

Shiori
86 Posts
January 28, 2020, 6:10 pm

I have run into this a few times, its almost always a network issue. Make sure you select the same ethernet port when installing. If you used eth0 on the cluster, the new node must use eth0. Same for vlans and backend networks. It sounds stupid because the node will get new configs from the cluster but they are applied hot, without a reboot so the script wont allow it or it tries the new config then errors when it cant reach the cluster as expected. Remember that some servers (dells and sunfire servers are notorious) have different port to device naming conventions thus, port0 may not be eth0.

The remaining times I have found a boot drive or sas/sata port to be at fault.

#7

bill gottlieb
26 Posts
February 10, 2020, 3:45 pm

Hi.

It does not seem to be a network issue. I did a fresh install and seems that it joins the cluster, copies keys and files, but fails in getting the cluster uuid. Here is the output from the PetaSAN.log:

 

10/02/2020 20:31:00 INFO Start settings IPs
10/02/2020 20:34:36 INFO Start settings IPs
10/02/2020 20:35:57 ERROR getting cluster uuid from configuration failed
10/02/2020 20:35:59 ERROR getting cluster uuid from configuration failed
10/02/2020 20:36:00 ERROR 'in <string>' requires string as left operand, not NoneType
Traceback (most recent call last):
File "/opt/petasan/scripts/admin/node_manage_disks.py", line 121, in main_catch
func(args)
File "/opt/petasan/scripts/admin/node_manage_disks.py", line 134, in node_disk_list_json
print (json.dumps([o.get_dict() for o in ceph_disk_lib.get_full_disk_list(args.pid)]))
File "/usr/lib/python2.7/dist-packages/PetaSAN/core/ceph/ceph_disk_lib.py", line 288, in get_full_disk_list
ceph_disk_list = get_disk_list()
File "/usr/lib/python2.7/dist-packages/PetaSAN/core/ceph/ceph_disk_lib.py", line 196, in get_disk_list
ceph_disk_list = get_ceph_disk_list()
File "/usr/lib/python2.7/dist-packages/PetaSAN/core/ceph/ceph_disk_lib.py", line 104, in get_ceph_disk_list
for device in ceph_disk.list_devices():
File "/usr/lib/python2.7/dist-packages/PetaSAN/core/ceph/ceph_disk.py", line 720, in list_devices
space_map))
File "/usr/lib/python2.7/dist-packages/PetaSAN/core/ceph/ceph_disk.py", line 516, in list_dev
if ptype in (PTYPE['regular']['osd']['ready']):
TypeError: 'in <string>' requires string as left operand, not NoneType
10/02/2020 20:36:00 ERROR Error while run command.
10/02/2020 20:40:34 ERROR Config file error. The petaSAN os maybe just installed.
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/PetaSAN/backend/cluster/deploy.py", line 53, in get_node_status
node_name = config.get_node_info().name
File "/usr/lib/python2.7/dist-packages/PetaSAN/core/cluster/configuration.py", line 99, in get_node_info
with open(config.get_node_info_file_path(), 'r') as f:
IOError: [Errno 2] No such file or directory: '/opt/petasan/config/node_info.json'
10/02/2020 20:41:07 INFO Starting node join
10/02/2020 20:41:07 INFO Successfully copied public keys.
10/02/2020 20:41:07 INFO Successfully copied private keys.
10/02/2020 20:41:07 INFO password set successfully.
10/02/2020 20:41:08 INFO Start copying cluster info file.
10/02/2020 20:41:08 INFO Successfully copied cluster info file.
10/02/2020 20:41:08 INFO Joined cluster STORAGE-1
10/02/2020 20:42:27 INFO Set node info completed successfully.
10/02/2020 20:42:27 ERROR getting cluster uuid from configuration failed
10/02/2020 20:42:28 ERROR getting cluster uuid from configuration failed

  • Bill
#8