BoKS troubleshooting: SSH daemon and client

2008-11-21 21:13:00

At $CLIENT we found that almost 60% of our time was being spent on troubleshooting SSH or SFTP in one of its many forms. Because each problem -seemed- unique we kept on reinventing the wheel, costing us precious time. To cut down on this I've set up a short procedure that should help in diagnosing the problem. I've also made a list of various symptoms that are linked to rather rare scenarios.

Troubleshooting example 1 also covers most of these steps with some sample output for additional detail.

Standard procedure is to follow these steps:

  1. Check the BoKS transaction log.
  2. Check the user's access rights.
  3. Check for authentication methods.
  4. Check the user's SSH keypair.

This should actually be enough to handle 70% of the cases. For the rest there's more:



1. Check the BoKS transaction log

While this may sound painfully obvious, the best place to see why a user cannot login is the BoKS transaction log. For each login request handled by BoKS these files will contain a log entry. It's easiest to search for the combination of hostname and username and to use the BoKS log parser to make the output legible.

For example:

$ for FILE in `ls -lrt | grep "Dec 13" | awk '{print $9}'`

> do

> grep $HOSTNAME $FILE | grep $USER | /opt/boksm/sbin/bkslog -f -

> done

Using either the output of the parsed BoKS log, or the list of error codes it should be trivial to find out what's going wrong. The most common errors in our environment are the following:



2. Check the user's access rights

As was mentioned, in the cases of a 200, 201 or a 203 you'll have to make sure whether the user actually has access to the requested resource. Crosscheck the following:

One of the most useful commands will be:

BoKS > lsbks -aTl *:$USER

The "lsbks" command lists information about a user. By using -a (all) and -T (access routes) you'll see everything you'll need to know. Hostgroup, userclass, uid/gid, is the account locked, when was the last login, and so on. You'll also see two lists of access routes: one for the individual user and one for his userclass.



3. Check for authentication methods

SSH is tricky insofar that it allows for (a combination of) multiple authentication methods. The most common are password, keyboard interactive and ssh_pk, aka key pair. The keyboard interactive method is actually forced by BoKS, thus disabling the "password" method, which isn't a problem at all since keyboard interactive -includes- password auth.

If the user's denied access it could be that the used authentication method isn't allowed. Per default, users have to use password authentication. In order to allow keypair authentication one has to set a particular flag on the account. This flag can be checked with either of these commands.

BoKS > authadm list -u *:$USER

BoKS > dumpbase -t31 | grep $USER

You'll notice the "must use" flag which indicates whether ssh_pk is optional or required. This value can be change using the -m and -M flags on the "authadm mod" command.



4. Check the user's SSH keypair

If the user is in fact making use of ssh_pk we should ensure that all relevant settings are correct.



5. Rare cases: further debugging

For those few cases that aren't solved by the aforementioned steps, there's a few other things we can try.



6. Scenarios and symptoms


kilala.nl tags: , ,

View or add comments (curr. 0)