ISC+TSM AC

Many people have been complaining about the ISC+TSM AC and I have been one of them.  The concerns have been plentiful, “We have to learn a new interface”, “There’s no DRM functions”, and “It requires a server just to support it!”  All these are valid protests and the one that bothers me the most is the fact that in a DR situation you would have to rebuild the ISC system/instance along with the TSM server to have web accessibility.  This adds time to an already urgent situation.  So what good does the ISC+TSM AC provide?  For starters a single interface for accessing all your servers, a single login, more functions when it comes to hardware management, and in the event of a disaster it forces you to learn the command line.  I know the last item might frustrate a lot of people but the truth is you need to know the command line to be a proficient TSM administrator.  I love the web interface and I recommend TSMManager, but when in a DR situation you have to be able to handle the command line if you want to get back up and running. Granted you have no other choice than command line until the system comes back up and is running again, but afterwards you’ll need to do typical admin work and the ability to do it through the command line will increase you rebuild speed helping you meet SLA time frames.  I don’t think Tivoli had this in mind when they went to the ISC+TSM AC but in my opinion too many people rely on the web and don’t learn the commands needed to be truly proficient.  I find it amazing how many don’t even know how to use the HELP command.  So I could complain about the change in interface but change happens and although we don’t always like it (I don’t care for this one) we have to be able to change and adapt with it if we want to last in this work force.

King Of All Backups! (AKA LAN-Free to Disk)

About a year ago we were tasked to setup a large multi-clustered Exchange server and provide the best possible backup and restore performance.  After much debate and research we decided on using LAN-Free to disk.  The system was a 7 node Windows 2003 cluster connected to an SAN disk array (I can’t remember if it was EMC or Dell). The first 5 nodes were Exchange servers, the 6th was the failover node and the 7th was turned into a TSM server.  The TSM server instance had 5 500GB secondary disks assigned to it for the backup of the five Exchange servers.  These five disks would be mapped one to each Exchange server allowing for the backup to occur across the SAN to the disks owned by the TSM server. To utilize the LAN-Free to disk capability we had to install Tivoli’s SANergy product.  SANergy is no longer a separate product but is now part of a TDP/Agent type install package for TSM.  We actually installed and configured SANergy first, which was easier than it seemed in the directions, then mapped the drives.  When configured with SANergy the mapped drives become accessible across the SAN as long as the clients are on the same disk SAN fabric.  So we now had mapped SAN-accessible drives and could backup the Exchange servers to disk using the FILE device class.  The FILE device class is the device class used since TSM does not support LAN-Free backups to diskpools at this time.  The FILE device class works like a virtual tape and it was configured to migrate the data a few hours before the next backup would occur, or when the storage pool reached a specific usage threshold.  The reason for this was to allow almost a 24 hr. timeframe for a restore and along with the new Exchange 2003 restore capabilities internally; it provided a high performance backup/restore solution.  We tested the backups against a 360GB DB and backed it up in 90 minutes.  People were impressed, but they wanted to see how it performed on restore. We then restored the same amount, 360GB, in 91 minutes. WOW!  It was amazing to see those numbers (68MB/s).  We even tested it with the failover node by mapping all 5 SANergy defined drives to the failover node and still saw the same numbers. We had everything ready to go when the account decided they wanted to go in another direction.  Weeks spent configuring and implementing the solution all for not!  At least I have the experience and know it works.  So if anyone is looking to do LAN-Free to disk it works, it’s fast, it takes a lot of admin work, and it will be a good solution for anyone looking for a high performance backup/restore environment.

Submit A Question Or Topic

If anyone would like to submit a topic they would like to see covered or have a question about TSM or tape/SAN issues or strategy please e-mail me at chadsmal@us.ibm.com. With 5.3.2 out soon there will some things to cover. I am also thinking of posting an article on my LAN-Free to disk trial that was a great success. If anyone is interested in LAN-Free to disk let me know and I'll post my experience. If you would like to submit a post I am open to having guest contributors. Even though the name says TSMExpert I do not profess to know it all. Your contribution or questions help others out there.

Managing RAW Volumes

Well I was recently called by another IBMer and asked how to use RAW volumes. The person called to ask why sometimes DSMFMT will format quite fast on one machine then take forever on another machine. Well one thing you have to understand about DSMFMT is that it's taking that file and making the space within it RAW. If you've ever looked inside an unused TSM volume you'll see it is a text file filled with ADSM over and over (they might have changed the fill but when I was teaching TSM thats what we saw). So why use RAW instead of actual files (other than files being a redundant process)? FAST! EASY! and when speed in DR is key it's the only way to go. It's actually easier than one would think and with a little script you can manage your RAW volumes and hdisks easily. I'll post the script along with a script to querry the serial and WWN of your tape drives. These two scripts come courtesy of Hari Patel my co-worker who is a PERL mad man. (Download tar/zip)

Free Web Trainings!

I am providing this link for those of you unaware of some free TSM training available from IBM.  They have a number of web based training classes available like TSM 5.3 Overview and Differences for TSM 5.3. You can find the site here.  They are also providing a free TSM concepts poster for those that request it.  I would also suggest the free Linux trainings available on the IBM Developer Works website for those that would like to learn more about it.  You can find a list of web based trainings here, and personally if you can’t become a proficient Linux user with these trainings then you need to look for a new job!

NetApp TOC Issues

We have recently found out that the TOC file creation in TSM can fail when the NetApp volume has special characters in the filename.  The has led people to believe that the backups are unsuccessful and our group would be unable to restore data. That assumption could not be farther from the truth.  We can still restore an individual file, we just can’t load a graphical representation into the web based TSM client. Anyway, the response by Tivoli was that we could identify the file with the problem because an error will report when the TOC creation fails stating the filename that caused the problem.  So we would have to do this hundreds of times since we have, on our own, identified at least 400+ files with special characters. So I have good backups just can’t restore them easily, then the question is how does TSM react when trying to restore files with special characters?    

Restoring An Image Backup To A Larger Disk

Well just the other day my group was asked to help the NT System Admins with the combination of two large disks into one. Both of the disks have been performing image backups so we decided that the largest image would be restored first then the second drive would be restored to the new disk normally. To combine the disks the SA’s made one huge partition (400+ GB) and an SA on my team started the image restore. The following warning was issued when the image was initiated:

***************************** WARNING ********************************
The destination volume is larger than the source volume.  This will reduce the file system '\\machinename\x$' size to '    xxx.xx MB'.
Do you wish to continue? (Yes (Y)/No (N))

This was due to the target disk being larger than the image. We figured the image would not affect the space unused but we were wrong. After the image was restored explorer saw the disk as the size of the image backup but the Windows Disk Management tool saw it as the 400+ GB it was. So the problem was how to recover the disk.  Here is what MS support instructed and the problem was resolved.

When doing an image restore:

If the destination volume is larger than the source, after the restore operation you will lose the difference between the sizes. If the destination volume is located on a dynamic disk the lost space can be recovered by increasing the size of the volume. This will also increase the size of the restored volume.

We also have the following knowledge base doc:

When a Tivoli Storage Manager (TSM) image restore is performed to a target volume that is larger in size than the original volume (from which the image backup was taken), TSM will concatenate the target volume down to the original volume size.  However, within Windows Disk Management (within the Administrative Tools -> Computer Management utility), the disk will appear to still exist as the original larger volume size, even though Windows Explorer will show that the real volume size is now smaller.  In essence, this means that space has been lost, since it is not accessible on the drive and not available for use.  To prevent the loss of this space, the same volume size should be used on the target volume as on the original volume.

As far as I know this fix only works if the disk is dynamic (logical) and not a fixed (physical) disk. So the fix was to increase the disk size again to help it see the space again. I don’t think you need to increase it much but once done it resolves the issue. So if you happen to restore an image backup to a larger partition than it originated from just be aware of what you’ll need to do to reclaim your unused space after the restore finishes.