show sidebar Building Clusters with the KNOPPIX/OpenSSI Live CD next_inactive up previous


Building Clusters with the KNOPPIX/OpenSSI Live CD

These instructions describe how to build an OpenSSI cluster using the KNOPPIX/OpenSSI Live CD. All you need is two or more computers, connected with a private ethernet switch. This network is called the "interconnect", and it must be private because the first node automatically assigns itself a 10.0.0.1 IP address early in boot, which cannot be easily changed. This only affects the first network interface card (``NIC'') discovered by KNOPPIX/OpenSSI. If you have more than one NIC, you may attach the others to public networks.

Each individual computer in the cluster is called a "node". In this basic configuration, the CDROM and ramdisks mounted on the first node are automatically shared with the rest of the cluster via the interconnect. This works well for many users. To learn more about how filesystems are shared over the interconnect, please see README.cfs.

If you install the regular version of OpenSSI for Debian, Fedora Core, or Red Hat, you can make your filesystems highly-available (``HA''), either using shared disk hardware that is physically connected to two or more nodes, or using the Distributed Replicated Block Device (``DRBD'') project. Please see README.hardmounts for more information.

Note that any time another document is referenced, you can find it in the docs/ directory of this release tarball, as well as in your system's /usr/share/doc/openssi/ directory after you install OpenSSI.

A good document that explains more about OpenSSI clustering in general is Introduction-to-SSI. It was written by Bruce Walker, who is the project leader for OpenSSI.

This installation guide is provided in multiple formats for your convenience:

1 Booting the CD

  1. Insert the KNOPPIX/OpenSSI Live CD into the first node and boot from it. It should detect a network card, configure it, initialize the clustering software, and boot to a shell prompt. X Windows is not yet working, but this will probably be fixed in the near future.
  2. Your first node is configured as follows:

    1. The node number is 1.
    2. The first NIC detected is configured with the IP address 10.0.0.1, so it must be connected to a private network switch. If the node has any other NICs, they may be connected to public networks and manually configured after boot.
    3. The filesystem layout is very similar to KNOPPIX. There are some extra directories in /ramdisk, and in a few strategic locations there are context-dependent symlinks (``CDSL''), which is OpenSSI's mechanism for allowing per-node versions of files and directories. To learn more about CDSLs, please see the document entitled cdsl.
    4. Filesystem failover is not configured, since it does not make sense in the Live CD configuration. You can install a regular version of OpenSSI for Debian, Fedora Core, or Red Hat if you want to use this feature. You can learn more about filesystem failover in README.hardmounts.
    5. OpenSSI includes an integrated version of Linux Virtual Server (``LVS''), which lets you to configure a Cluster Virtual IP (``CVIP'') address that automatically load balances TCP connections across various nodes. This CVIP is highly available and can be configured to move to another node in the event of a failure. For more information, please see README.CVIP.
    6. The clustername is KNOPPIX-OpenSSI. If you want to run NFS server, it should resolve to your CVIP address, either in DNS or the cluster's /etc/hosts file. For more information, please see README.nfs-server.
    7. A simple mechanism for synchronizing time across the cluster is installed. Any time a node boots, it will synchronize its system clock with the initnode (the first node, where init is running). You can also run the ssi-timesync command at any time to force all nodes to synchronize with the initnode.

      This timesync mechanism synchronizes nodes to within a second or two of each other. If you need a higher degree of synchronization, you can configure Network Time Protocol (``NTP'') across the cluster. Instructions for how to do this are available in README.ntp.

    8. Automatic process load balancing will be installed as part of OpenSSI. To enable load-balancing for a program, add its name to /proc/cluster/loadlevellist. For more information, see README-mosixll.

      This Live CD includes a demo of process load-balancing in /demo-proclb, and the program to be load-balanced has already been added to /proc/cluster/loadlevellist. Please see /demo-proclb/README for more information.

2 Adding new nodes

Nodes in the OpenSSI cluster are booted using network booting method. This lets you avoid having to install a distribution on more than one node. To network boot a new node, first select one of its NICs for the cluster interconnect. It must have a chipset supported by PXE or Etherboot.

  1. If the selected NIC does not support PXE booting, download an appropriate Etherboot image from the following URL:

    http://rom-o-matic.net/5.2.4/
    Choose the appropriate chipset. Under Configure it is recommended that ASK_BOOT be set to 0. Floppy Bootable ROM Image is the easiest format to use. Just follow the instructions for writing it to a floppy.

  2. Connect the selected NIC to the cluster interconnect, insert an Etherboot floppy (if needed), and boot the computer. It should display the hardware address of the NIC it is attempting to boot with, then hang while it waits for a DHCP server to answer its request.
  3. On the first node (or any node already in the cluster), execute ssi-addnode. It will ask you few questions about how you want to configure your new node and they are as follows.

    1. Enter a unique node number between 2 and 125.
    2. Enter MAC address of the new node to be added in the cluster.
    3. Enter a static IP address for the NIC. It must be unique and it must be on the 10.x.x.x network.
    4. Select (P)XE or (E)therboot as the network boot protocol for this node. PXE is an Intel standard for network booting, and many professional grade NICs have a PXE implementation pre-installed on them. You can probably enable PXE with your BIOS configuration tool. If you do not have a NIC with PXE, you can use the open-source project Etherboot, which lets you generate a floppy or ROM image for a variety of different NICs.
    5. Enter a nodename. It should be unique in the cluster. If you want to run NFS client, the nodename should resolve to either the IP address you configured above for the interconnect, or to one of external IP addresses that you might configure after boot. The nodename can resolve to the IP address either in DNS or in the cluster's /etc/hosts file.

      The nodename is stored in /etc/nodename, which is a CDSL. In this case, the context is node number, which means each node you add will have it's own view of /etc/nodename containing its own hostname. To learn more about CDSLs, please see the document entitled cdsl.

    6. Save the configuration.
  4. The program will now do all the work to admit the node into the cluster. Wait for the new node to join. A ``nodeup'' message on the first node's console will indicate this. You can confirm its membership with the cluster command:

    # cluster -v
    If the new node is hung searching for the DHCP server, try manually restarting the ``dhcpd'' on the cluster's init node where DHCP server is running:

    # invoke-rc.d dhcp restart 
    It has been observed that sometimes the TFTP server will not respond to a client more than once. So restart the inetd on the init node if client could get IP address, but could not continue booting:

    # invoke-rc.d inetd restart
    If the new node is still hung, try rebooting it.

  5. If desired, configure the new node with one or more swap devices using fdisk (or a similar tool) and mkswap:

    # onnode <node_number> fdisk /dev/hda (device name)

    partition disk

    # onnode <node_number> mkswap /dev/hda3 (device partition name) 

    Add the device name(s) to the file /etc/fstab, as documented in README.fstab.

    Either reboot the node or manually activate the swap device(s) with the swapon command:

    # onnode <node_number> swapon <swap_device>
  6. Repeat above steps at any time to add other nodes to the cluster.
  7. Enjoy your new OpenSSI cluster!!!

    To learn more about OpenSSI, please read Introduction-to-SSI.

    One of the first things you can try is running the process load-balancing demo. It illustrates some of the features of OpenSSI clusters. You can find it in /demo-proclb.

    If you have questions or comments that are not addressed on the website, do not hesitate to send a message to the user's discussion forum:

    ssic-linux-users@lists.sf.net

About this document ...

Building Clusters with the KNOPPIX/OpenSSI Live CD

This document was generated using the LaTeX2HTML translator Version 2002 (1.62)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -no_subdir -split 0 -show_section_numbers /tmp/lyx_tmpdir6563riFIc5/lyx_tmpbuf0/README.tex

The translation was initiated by root on 2005-02-05


next_inactive up previous
root 2005-02-05

This page last updated on Wed Feb 16 15:12:45 2005 GMT
privacy and legal statement