Get Device Info From AIX

I needed to gather the serial numbers by device (rmt) in AIX for our VTL, and the grep command in AIX is not as functional as my Linux box at home, so the -A and -B options are not available. Those two flags allow you to grab X lines before and after the grep'ed expression which is really handy when running lscfg -vp. Here's a grep command to gather just the rmt(s) with their serial and WWN. If you need to change the number of lines just change the a (after) and b (before) value, and s is what is "grep'ed".

lscfg -vp | awk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r[(NR-c+1)%b];print;c=a}b{r[NR%b]=$0}' b=1 a=5 s="rmt"

Rosetta Stone for UNIX

Know AIX? Ever wonder what that simple command in AIX is on Solaris or HP/UX? Need to know what the command for tcp/ip management is in Linux compared to OS/X? Now you can! Check out this helpful website Rosetta Stone for UNIX. My boss passed this along a while ago and I thought I had passed it along to the community, but I cannot remember if I did. So, here is a great website for UNIX admins and those learning UNIX to help you transition from one vendor to another.

IBM Considers Buying Sun!

Did I.Q.'s just drop substantially around the world? Go sell crazy somewhere else! Why would IBM even consider buying Sun? Definitely not for their Server technology, it's gotta be for their software portfolio. Most people are not aware that IBM's makes more money from their software division than hardware. (At least they used to...still think they do). But still, I gotta think Sun would be better served by a buyout from HP or Cisco...maybe Dell, but Dell seems uninterested from what I have read.

TSM & File System Support

First off I hope everyone had a good holiday season. Now that we can focus on work again I wanted to discuss a topic of file system types. I just had an incident where a Solaris server had ZFS used for some newer file systems. The admins had added them without consulting us, and we didn't catch it because TSM didn't even attempt to back them up. Our client level was 5.4.1.0 and ZFS support was added with the 5.4.1.2 update. Once I updated the client the file systems were backed up successfully and show the correct format. We did see one file system was returning a type of UNKNOWN and that should have alerted us, but we were not receiving errors or failures on the backup of the server in question.

So here is the question, how do you keep something like this from happening in the future as new, more bleeding edge file system types are added? Obviously you need to inform your Unix Admins to work with you whenever they add a newer file system type, but if they don't alert you, and TSM doesn't report failures, how would you know? It's bound to happen as the Linux community adds newer, more robust file system types. Other than stay as current as possible with my TSM client levels (which wont always be the fix) what would you suggest?

Tabbed SSH/Telnet Client

As a system admin I am using an SSH terminal (PuTTY in most cases) and hate switching between the multiple open sessions I have. So in my search for a tabbed SSH client that was FREE (since I am cheap) I came across two options that fit the bill. The first product is a freeware telnet/ssh terminal utility called Poderosa that is not bad and I use it periodically. I must say that I am a PuTTY fan so I really hoped someone had written a tabbed PuTTY client, but I could not find one. I did, however, come across a nice little utility that lets me have my PuTTY and tabs too...it's called WinTabber and it grabs open windows and places them in tabs within its container window. To better understand I suggest you check out the screenshot on the website. If you have a tabbed based ssh client you would recommend that is free (again because I am cheap) then post a comment and leave a link so I can check it out.

DR Tests & System Restores

I know I have covered this before, but we have been doing DR tests and when restoring UNIX servers I find it easier and faster to use the Web/Java GUI to select all file systems required to restore the data than following the IBM guideline to build a small script that runs them serially. The thing I don't get is why they would push command line restore. Doing the restore file system by file system will require the same tapes be mounted multiple times where the GUI will restore all the data available on the tape for the selected backup. They do state that you can execute more than one restore simultaneously, but in every DR exercise I have run that would just cause tape contention since I do not collocate by file space. Unless I am wrong in how TSM handles a GUI based multiple file system restore wouldn't the GUI be the better solution when in a DR or full UNIX system restore scenario?

Of course this is all dependent on whether you have Java 1.4.1 or higher installed.

Backupsets & TCopy

OK, I am going to throw a question out into the TSM world and see what you people think. I was asked at work if a Backupset could be copied using the UNIX tcopy command? I thought about it and since Backupsets are independent of the TSM server I would think, in theory, it should. The only issue I could see would be needing to define the copy to TSM, since it would have no record of it. The benefit would be that I would not need to mount the tapes all over again to run the Backupset. The reason I ask is that a customer wanted two copies of a full backup of a server to archive forever. Anyone ever tried it, any theories? They are all welcome!

CDP For Unix?

Has anyone heard of when (if ever) Continuous Data Protection for Files will be available on the Unix platform? I could really use this feature with my GPFS system. Since the application creates hundreds of meta data files daily and is proprietary (hence no TDP support) I am getting killed by the backup timeframe since the each volume has in excess of 4 million files already and incrementals take close to 48hrs. to finish. Anyone heard anything at symposiums or seminars?

Setting Up A Secondary TSM Instance

Someone requested a post on how to setup a secondary instance of TSM on a UNIX server so here is the skinny on how that is setup:

First create a directory where the config files for the new server will be stored.

mkdir /usr/tivoli/tsm/serverb
mkdir /usr/tivoli/tsm/serverb/bin

then copy the dsmserv.opt over and modify the needed settings in it like devconfig and volhist to save in the new dir.  Then create DB and Log volumes that this instance will use. Once those are created you need to export the following environmental variables:

export DSMSERV_CONFIG=/usr/tivoli/tsm/serverb/bin/dsmserv.opt
export DSMSERV_DIR=/usr/tivoli/tsm/serverb/bin
export DSMSERV_ACCOUNTING_DIR=/usr/tivoli/tsm/serverb/bin

Now you can run the dsmserv format command to initialize the DB and Log volumes and it will create the dsmserv.dsk in the serverb directory.  Make sure you run the dsmserv runfile commands to load the scripts and webimages (even if TSM 5.3).  The final step is to create the startup script so that TSM initializes correctly.  Here is our script:

#!/bin/ksh

exec > /tmp/libserv.out 2>&1  #optional -> sends output to out file

ulimit -d unlimited

export DSMSERV_CONFIG=/usr/tivoli/tsm/serverb/bin/dsmserv.opt
export DSMSERV_DIR=/usr/tivoli/tsm/serverb/bin
export DSMSERV_ACCOUNTING_DIR=/usr/tivoli/tsm/serverb/bin

print "$(date '+%D %T') Starting Tivoli Storage Manager Server"
cd /usr/tivoli/tsm/serverb/bin

dsmserv

We use the following command to start TSM so we don’t have to deal with nohup:

echo “/usr/tivoli/tsm/serverb/bin/rc.adsmserv” | at now

This uses the at command to run the script immediately.  You can then edit the inittab and place a line in to start this instance on boot or put a script in the run level startup folders, your choice.  You should now be ready to run the second instance.  One note to those thinking of doing this and sharing a library, Tivoli recommends that you create a TSM instance to be just a library manager, no clients, no real work other than handling the library and tape mounts.  I agree with this and it has been a lot easier to manage and handle library issues.  Not knowing how large the DB could get I gave it 2GB and it is currently 3.8% utilized, and it has been in place for over a year and a half. Swapping a library manager from one system to another is not as hard as it would seem so consider it and if anyone wants docs on how to do the switch let me know I’ll post it.    

UNIX Permission Issues

Awhile back I ran into an issue with how TSM handles permissions on UNIX files and wanted to get some feedback from you readers out there on how you would handle it.  What happened was a user somehow was given root and he chown’ed the /home dir recursively.  It was made worse by the fact that he did that on Friday and didn’t alert anyone until the following Monday, and by the time it got to us another day had passed.  The customer of course wanted us to restore the directory and file permissions, but the kicker was that TSM does not back a UNIX file up again when the permissions change.  It just updates the database to reflect the permission changes (I got that directly from support and was floored; I had no idea it handled UNIX that way).  So here was our dilemma, if the file was the only version in backup I would not have any way of resetting its permissions.  Is the gravity of the situation hitting home?  Because it doesn’t backup the file again or track permissions I could not successfully restore to a point-in-time.  Sure I might get a good portion of the files fixed but there would still have been a large portion that we would be unable to get the permissions corrected.  The customer wasn’t happy and our only out was that the customer should not have been doing chown‘s as root.  I thought I once saw someone post a undocumented option you can set in the options file that will backup a file if it changes in any way, permissions included, but I can’t find it.  I thought I saw it on the new ADSM.org but am unable to locate it.  Anyone know the option or have an idea on how to approach this?  I brought it up with some Tivoli people who asked me what I thought should be added or changed in TSM, but so far I haven’t seen any change in their processing.