Switch-based Discovery

For switch based hardware discovery, the servers are identified through the switches and switchposts they are directly connected to.

In this document, the following configuration is used in the example

Management Node info:

MN Hostname: xcat1
MN NIC info for Management Network(Host network): eth1, 10.0.1.1/16
MN NIC info for Service Network(FSP/BMC nework): eth2, 50.0.1.1/16
Dynamic IP range for Hosts: 10.0.100.1-10.0.100.100
Dynamic IP range for FSP/BMC: 50.0.100.1-50.0.100.100

Compute Node info:

CN Hostname: cn1
Machine type/model: 8247-22L
Serial: 10112CA
IP Address: 10.0.101.1
Root Password: cluster
Desired FSP/BMC IP Address: 50.0.101.1
DHCP assigned FSP/BMC IP Address: 50.0.100.1
FSP/BMC username: ADMIN
FSP/BMC Password: admin

Switch info:

Switch name: switch1
Switch username: xcat
Switch password: passw0rd
Switch IP Address: 10.0.201.1
Switch port for Compute Node: port0

Configure xCAT

Configure network table

Normally, there will be at least two entries for the two subnet on MN in networks table after xCAT is installed:

#tabdump networks
#netname,net,mask,mgtifname,gateway,dhcpserver,tftpserver,nameservers,ntpservers,logservers,dynamicrange,staticrange,staticrangeincrement,nodehostname,ddnsdomain,vlanid,domain,comments,disable
"10_0_0_0-255_255_0_0","10.0.0.0","255.255.0.0","eth1","<xcatmaster>",,"10.0.1.1",,,,,,,,,,,,
"50_0_0_0-255_255_0_0","50.0.0.0","255.255.0.0","eth2","<xcatmaster>",,"50.0.1.1",,,,,,,,,,,,

Run the following command to add networks in networks table if there are no entries in it:

makenetworks

Setup DHCP

Set the correct NIC from which DHCP server provide service:

chdef -t site dhcpinterfaces=eth1,eth2

Add dynamic range in purpose of assigning temporary IP adddress for FSP/BMCs and hosts:

chdef -t network 10_0_0_0-255_255_0_0 dynamicrange="10.0.100.1-10.0.100.100"
chdef -t network 50_0_0_0-255_255_0_0 dynamicrange="50.0.100.1-50.0.100.100"

Update DHCP configuration file:

makedhcp -n
makedhcp -a

Config passwd table

Set required passwords for xCAT to do hardware management and/or OS provisioning by adding entries to the xCAT passwd table:

# tabedit passwd
# key,username,password,cryptmethod,authdomain,comments,disable

For hardware management with ipmi, add the following line:

"ipmi","ADMIN","admin",,,,

Verify the genesis packages

Genesis packages are used to create the root image for network boot and MUST be installed before doing hardware discovery.

  • [RH]:

    # rpm -qa |grep -i genesis
xCAT-genesis-scripts-ppc64-2.10-snap201507240527.noarch
xCAT-genesis-base-ppc64-2.10-snap201505172314.noarch

  • [ubuntu]:

    # dpkg -l | grep genesis
ii  xcat-genesis-base-ppc64 2.10-snap201505172314   all          xCAT Genesis netboot image
ii  xcat-genesis-scripts    2.10-snap201507240105   ppc64el      xCAT genesis

Note: If the two packages are not installed, install them first and then run mknb ppc64 to create the network boot root image.

Predefined Nodes

In order to differentiate one node from another, the admin needs to predefine node in xCAT database based on the switches information. This consists of two parts:

  1. predefined_switches_label
  2. predefined_server_nodes_label

Predefine Switches

The predefined switches will represent devices that the physical servers are connected to. xCAT need to access those switches to get server related information through SNMP v3.

So the admin need to make sure those switches are configured correctly with SNMP v3 enabled. <TODO: The document that Configure Ethernet Switches>

Then, define switch info into xCAT:

nodeadd switch1 groups=switch,all
chdef switch1 ip=10.0.201.1
tabch switch=switch1 switches.snmpversion=3 switches.username=xcat switches.password=passw0rd switches.auth=sha

Add switch into DNS using the following commands:

makehosts switch1
makedns -n

Predefine Server node

After switches are defined, the server node can be predefined with the following commands:

nodeadd cn1 groups=powerLE,all
chdef cn1 mgt=ipmi cons=ipmi ip=10.0.101.1 bmc=50.0.101.1 netboot=petitboot installnic=mac primarynic=mac
chdef cn1 switch=switch1 switchport=0

In order to do BMC configuration during the discovery process, set runcmd=bmcsetup. For more info about chain, please refer to Chain

chdef cn1 chain="runcmd=bmcsetup"

Add cn1 into DNS:

makehosts cn1
maekdns -n

Discover server and define

After environment is ready, and the server is powered, we can start server discovery process. The first thing to do is discovering the FSP/BMC of the server. It is automatically powered on when the physical server is powered.

The following command can be used to discovery BMC within an IP range and write the discovered node definition into xCAT database:

bmcdiscover -s nmap --range 50.0.100.1-100 -t -z -w

The discovered BMC node will be like this:

# lsdef node-8247-42l-10112ca
Object name: node-8247-42l-10112ca
bmc=50.0.100.1
cons=ipmi
groups=all
hwtype=bmc
mgt=ipmi
mtm=8247-42L
nodetype=mp
postbootscripts=otherpkgs
postscripts=syslog,remoteshell,syncfiles
serial=10112CA
Note:

  1. The BMC node is just used to control the physical during hardware discovery process, it will be deleted after the correct server node object is found.

  2. bmcdiscover will use username/password pair set in passwd table with key equal ipmi. If you’d like to use other username/password pair, you can use

    bmcdiscover -s nmap --range 50.0.100.1-100 -t -z -w -u <username> -p <password>

Start discovery process

To start discovery process, just need to power on the PBMC node remotely with the following command, and the discovery process will start automatically after the host is powered on:

rpower node-8247-42l-10112ca on

[Optional] If you’d like to monitor the discovery process, you can use:

makeconservercf node-8247-42l-10112ca
rcons node-8247-42l-10112ca

Verify node definition

The following is an example of the server node definition after hardware discovery:

#lsdef cn1
Object name: cn1
    arch=ppc64
    bmc=50.0.101.1
    cons=ipmi
    cpucount=192
    cputype=POWER8E (raw), altivec supported
    groups=powerLE,all
    installnic=mac
    ip=10.0.101.1
    mac=6c:ae:8b:02:12:50
    memory=65118MB
    mgt=ipmi
    mtm=8247-22L
    netboot=petitboot
    postbootscripts=otherpkgs
    postscripts=syslog,remoteshell,syncfiles
    primarynic=mac
    serial=10112CA
    supportedarchs=ppc64
    switch=switch1
    switchport=0