show sidebar
Configuring NFS Client in an OpenSSI Cluster
============================================

NFS client capability in an OpenSSI cluster is very clusterwide, 
automatic, and transparent.  Any NFS mount done on any node is
automatically done on all nodes which are up, and by any node that 
subsequently joins.  Thus any mount should only be done once.  This 
automation avoids inconsistent views from different nodes and makes 
the cluster more SSI.  Requests generated from any node in the cluster 
go directly to the NFS server, so it doesn't matter which node does the 
original mount command.

This automation and mount consistency have several consequences, 
however.  First, each node must have network access to the remote NFS 
server.  Second, the NFS server must export to each node in the cluster.  
If either of these are not met, mounts will not succeed.  

Note that if a client node in the cluster fails, the only locks lost 
are those held by processes on that node at the time of the failure.

This is a general document for NFS client setup on an OpenSSI cluster. It
is written to apply to Fedora, as well as Debian, with indications
to explain differences between the distributions.

NOTE: On Fedora/Redhat, the equivalent commands for `invoke-rc.d` 
      and `update-rc.d` are `service` and `chkconfig`, respectively. 
      To start or stop a service, use command line arguments "on" 
      and "off".

Requirements:
	All cluster nodes must have external network interfaces
	All cluster nodes must have unique hostnames that correspond to an
		external interface
	NFS server must export filesystems to all cluster nodes

Limitations:
        Processes on different nodes in the cluster will only see standard
	NFS file coherency since each node is an independent client.


Client set-up steps:
--------------------

	Since each cluster node is an independent client, client failures
are handled just as any non-cluster machine would.

1. Make sure the hostname of each cluster node must corresponds to an external
interface.  This is never the CVIP address.

2. The portmap service is needed on all distributions.

3. On Fedora, the nfslock service should be enabled unless you mount with 
   the nolock option.  Also, the SSInfs service should be started, instead 
   of nfs.

   On Debian, the "nfs-common" service must be started and run on all nodes.  
   Standard startup should do this via `invoke-rc.d nfs_common start`.
   /etc/rc.nodeinfo should have an entry for starting portmap and nfs-common 
   on all nodes of the cluster.

4. Review standard Debian/Fedora NFS client configuration documentation

You can mount any nfs filesystems by hand or via /etc/fstab.  As an example 
of mounting by hand: 

# mount shadowman.example.com:/misc/export /misc/local

The only difference with an SSI cluster is that each node will perform the
mount given a single mount command.  That means, in the example above, that
the server shadowman.example.com must not preclude any of the cluster nodes
from mounting as specified in its /etc/export file.  If any node
can not mount the filesystem all other nodes will umount and an error
will be returned from the mount command above.

When a cluster node joins an existing cluster, it will attempt to mount all
client mounts that are already established in the cluster.  In the case that
an SSI node that joins the cluster can't mount a given NFS mount, the cluster
is in an inconsistent state.  Had he been part of the cluster during the mount,
the mount would have failed.  Processes migrating with an NFS file open in 
that filesystem, will be unable to migrate to the problem node.

The following /etc/fstab line from a standard Linux configuration:

server:/usr/local/pub   /pub    nfs  rsize=8192,wsize=8192,timeo=14,intr

would include a "node=*" option in an OpenSSI cluster's /etc/fstab:

server:/usr/local/pub   /pub    nfs  rsize=8192,wsize=8192,timeo=14,intr,node=*


Troubleshooting:
---------------

1. Verify that portmapper is running on each node.
   (ps -ef --shownode | grep portmapper)
2. Verify that each node can access the remote nfs server;
   (onall rpcinfo -p <sever>)
3. Verify that the remote nfs server is exporting to each clusternode;
   (run exportfs on the remote server and check output)
4. Verify that the nfs service is running on each node.

This page last updated on Thu Dec 15 17:15:45 2005 GMT
privacy and legal statement