Sony announced they have developed a tape medium and write process that can support 185TB per tape. Whoa, That's huge! Now if we can see it hit the market before some other storage strategy catchphrase becomes the "it" thing. Check out the link below...."To the cloud!"
http://www.extremetech.com/computing/181560-sony-develops-tech-for-185tb-tapes-3700-times-more-storage-than-a-blu-ray-disc
Showing posts with label tape. Show all posts
Showing posts with label tape. Show all posts
Wednesday, April 30, 2014
Friday, March 28, 2014
Poor Performance
Currently I work in an environment where we have a specific TSM instance for a large SAP DB (99TB currently). We just upgraded the drives in the tape library (yes we use tape! I know...I know....) from MagStar 3592 TS1130 (E06) drives to TS1140 (E07) drives. The upgrade was pushed in hopes of a jump in write/backup performance, but I was skeptical. TSM adds so much overhead you cannot use the RAW tape read/write numbers from any manufacturer. Typically IBM is somewhat reasonable with their numbers, but in this case I have seen NO performance increase what-so-ever. Here is a query of the processes for storage pool backup.
UPDATE (04/04/2014): Let me give you some more specs, we have the 99TB DB split between 4 TSM Storage Agents each having 4 8Gb HBA's. Each storage agent runs 4 sessions (allocates 4 drives) for their backup process. So all 4 storage agents account for 16 simultaneous sessions and it still takes over 24 hours to perform the 99TB backup. The backups are averaging around 70-78MB/sec. Is this a TSM overhead issue or do I have a tuning issue with the TDP and TSM? I'm getting less than 50% of the throughput I should see.
Here's the command that is run to execute the DB backup:
ksh -c export DB2NODE=7 ; db2 "backup db DB8 LOAD /usr/tivoli/tsm/tdp_r3/db264/libtdpdb264.a OPEN 4 SESSIONS OPTIONS /db2/DB8/dbs/tsm_config/vendor.env.7 WITH 14 BUFFERS BUFFER 1024 PARALLELISM 8 WITHOUT PROMPTING" ; echo BACKUP_RC=$?
PROCESS_NUM: 2667
PROCESS: Backup Storage Pool
START_TIME: 03-27 23:21:54
DURATION: 00 23:20:13
BYTES: 6.0TB
AVG_THRPUT: 75.87 MB/s
PROCESS_NUM: 2668
PROCESS: Backup Storage Pool
START_TIME: 03-27 23:21:55
DURATION: 00 23:20:12
BYTES: 6.2TB
AVG_THRPUT: 78.48 MB/s
PROCESS_NUM: 2669
PROCESS: Backup Storage Pool
START_TIME: 03-27 23:21:55
DURATION: 00 23:20:12
BYTES: 6.2TB
AVG_THRPUT: 77.99 MB/s
PROCESS_NUM: 2670
PROCESS: Backup Storage Pool
START_TIME: 03-27 23:21:55
DURATION: 00 23:20:12
BYTES: 6.4TB
AVG_THRPUT: 80.13 MB/s
UPDATE (04/04/2014): Let me give you some more specs, we have the 99TB DB split between 4 TSM Storage Agents each having 4 8Gb HBA's. Each storage agent runs 4 sessions (allocates 4 drives) for their backup process. So all 4 storage agents account for 16 simultaneous sessions and it still takes over 24 hours to perform the 99TB backup. The backups are averaging around 70-78MB/sec. Is this a TSM overhead issue or do I have a tuning issue with the TDP and TSM? I'm getting less than 50% of the throughput I should see.
Here's the command that is run to execute the DB backup:
ksh -c export DB2NODE=7 ; db2 "backup db DB8 LOAD /usr/tivoli/tsm/tdp_r3/db264/libtdpdb264.a OPEN 4 SESSIONS OPTIONS /db2/DB8/dbs/tsm_config/vendor.env.7 WITH 14 BUFFERS BUFFER 1024 PARALLELISM 8 WITHOUT PROMPTING" ; echo BACKUP_RC=$?
PROCESS_NUM: 2667
PROCESS: Backup Storage Pool
START_TIME: 03-27 23:21:54
DURATION: 00 23:20:13
BYTES: 6.0TB
AVG_THRPUT: 75.87 MB/s
PROCESS_NUM: 2668
PROCESS: Backup Storage Pool
START_TIME: 03-27 23:21:55
DURATION: 00 23:20:12
BYTES: 6.2TB
AVG_THRPUT: 78.48 MB/s
PROCESS_NUM: 2669
PROCESS: Backup Storage Pool
START_TIME: 03-27 23:21:55
DURATION: 00 23:20:12
BYTES: 6.2TB
AVG_THRPUT: 77.99 MB/s
PROCESS_NUM: 2670
PROCESS: Backup Storage Pool
START_TIME: 03-27 23:21:55
DURATION: 00 23:20:12
BYTES: 6.4TB
AVG_THRPUT: 80.13 MB/s
I average anywhere from 75 to 80 MB/sec. Here is the Magstar performance chart. I am using JB media, not JC so I do take a little hit in performance for that.
So with JB media I could get as high as 200MB/sec but I am not even 50% of that number. Is there any specific tuning parameter I should look at that could be hindering the performance?
FYI - The backup of the 99TB DB runs LAN-Free using 16 tape drives over 26 hrs.
Wednesday, January 9, 2013
Do Large Corporations Need Tape?
I am dealing with a situation where I have to gone from a tapeless TSM environment to the standard TSM tape model and I have to wonder why you would use tape when you have multiple data centers? If you have multiple data centers why not backup to disk and replicate the data on disk to a disk solution at the alternate DC? I did this with Data Domains and it made life so much easier. Multiple DR tests showed it was efficient and successful, of course this also utilized deduplication so disk usage and costs didn't get out of hand. So I ask why is any large corporation still using tape?
Thursday, January 19, 2012
TSM Device Handling in Windows
I have to say that TSM on Windows is good for small to medium size solutions and I'm not ANTI-Windows. I just cringe when dealing with devices in Windows. I hate its driver handling and most of all I hate how Windows presents library and tape drives. So I was working with a TSM server where the tape library would not initialize. It was an older SCSI library, not Fiber. I tried restarting the library, the TSM server, reloading drivers, and updating the drivers and nothing worked.
Duh! <Head Slap!!!>
That's because on the initial reboot that caused the library to stop communicating the device ID's changed. So the library went from LB1.0.0.2 to LB1.0.0.3. Nobody touched the SCSI card or library but the device definition changed! Seriously? All the drives changed to mtX.X.X.3 also. Now I don't use Windows all that much but luckily I remembered the TSMDLST program that is installed with the TSM server. It's under C:\Porgram Files\tivoli\tsm\console and will pull the information from Windows for you in a readable format. So next time your library goes offline make sure you use it to compare the device definitions and serials with what is defined in TSM. It will save you a lot of time and headache. You can find more information on issues like this here.
Duh! <Head Slap!!!>
That's because on the initial reboot that caused the library to stop communicating the device ID's changed. So the library went from LB1.0.0.2 to LB1.0.0.3. Nobody touched the SCSI card or library but the device definition changed! Seriously? All the drives changed to mtX.X.X.3 also. Now I don't use Windows all that much but luckily I remembered the TSMDLST program that is installed with the TSM server. It's under C:\Porgram Files\tivoli\tsm\console and will pull the information from Windows for you in a readable format. So next time your library goes offline make sure you use it to compare the device definitions and serials with what is defined in TSM. It will save you a lot of time and headache. You can find more information on issues like this here.
Tuesday, August 30, 2011
Solaris Driver Issue
I have a Solaris (version 10) server that I am trying to configure LTO drives on for a Storage Agent. My problem is that I ran the procedure to assign the drives to the IBMtape driver but doc state a reboot is needed. Has anyone done a successful modunload/modload of the IBMtape driver thereby avoiding a reboot? The server is a production box and the DB takes 45 minutes to take down and come back up so reboots are usually scheduled and the next one is weeks off. Any ideas or help is appreciated.
Tuesday, May 31, 2011
Where, how and why are tapes still used for backup?
Not strictly a TSM only topic, but this fascinating thread has been raging on over at LinkedIn for the last few days and I thought it would be worth sharing here. The provocative question asked was:
"Is tape still used for backup? Why is tape still being used these days when disk and cloud are available?"
Link: http://linkd.in/lo6VnM
Cue an avalanche of fascinating pro- and anti-tape pitches and opinions. Wading through the comments may take a while (I pitched-in with a couple of comments too) but it's a useful read as, frankly, these are the questions that clients/customers/decision-makers - and indeed ourselves - should be asking to ensure that we continue to use the right technology for the right purpose.
Of course, TSM itself is well-positioned for in many ways given its long-running support of disk, both in random access and more recently in sequential file (virtual tapes!).
David Mc
London, UK
Tuesday, February 19, 2008
ACSLS Label Problem
For the first time in many many months our ACSLS library has experienced some problems and one that is perplexing is a LABEL LIBVOLUME issue. When a LABEL LIBVOL is run it is coming back with a SUCCESSFUL result code but the tape does not get checked into the library. I do have CHECKIN=SCRATCH in the command, but that part of the proceedure is failing due to the library stating it is unable to open the drive on eject (the tape does eject, however). So the tape does get labeled but not checked in. When I run a CHECKIN LIBVOL command the tapes are then found and listed as in the LIBVOL inventory. Anyone every seen this? I am having the tape drive looked at, but TSM returning the process as SUCCESSFUL is disconcerting. This is definitely a TSM flaw that I plan to address with support.
Wednesday, October 24, 2007
SpectraLogic Library Review
We recently acquired a Spectra Logic T950 library for one of our data centers and I thought I'd let you all know how it's been performing. Since I was with IBM previously my only experience was with STK's (which we were trying to push out the customers door ASAP) and IBM hardware. I can say I was no fan of the STK L700 and being an IBM'er at the time I touted the 3584 and 3494's like they walked on water (they don't). We frequently had to have maintenance on our IBM libraries. Was it due to the fact we were collocating over 2500 clients and mounts were through the roof? Probably!
The data center I worked at with IBM had a secondary library room that housed the 8 libraries (some were for mainframes) and tape shelves. The room was running out of floor space to accommodate another library or expansions to existing ones. This is where SpectraLogic has IBM and the competition beat, hands down. A single T950 frame can house up to 24 drives and a max of 950 tapes with a frame H 78.77 in, W 30.63 in, D 43.21 in (H 200.1 cm, W 77.8 cm, D 109.8 cm). That's a little taller but less deep than an IBM TS3500 frame at 70.9"H x 30.8"W x 47.7"D (1800 mm x 782 mm x 1212 mm), with the IBM L frame handling a max of only 12 drives and 287 tapes.
How does SpectraLogic get such great density? They go vertical with a twist. SpectraLogic libraries use "TeraPacks" that are 10 tape chassis that load the tapes so the barcode are vertical not horizontal like IBM and most other libraries. When the library needs a tape its robot removes the TeraPack and then the gripper mechanism grabs the tape. I can hardly tell if it adds more than a second or two to the mount time, but even if it does the density gain negates the ever so slightly increased mount time.
I could go on an on and mention every little thing that I like about this library, but one of the coolest features is the ability to add SATA RXT portable RAID media, making the T950 VTL capable. The RXT media (which stands for RAID eXchangeable TeraPack) fits in the SpectraLogic half inch tape drive openings and is composed of multiple SATA disks sealed in a enclosure capable of taking rugged handling with built in shock dampening technology. The TeraPacks range in size from 2TB to 1TB, but I'm sure you will see larger sizes in the very near future. The RXT media is compatible with all major backup applications and operating systems.
The final item that sealed the deal was of course price. This library came in at a great price point, lower than IBM and Sun by quite a bit. This, added up with all the other features/benefits it offers in expandability, made it a win/win. "So, how has it performed?" You ask. Well, so far it has performed above my expectations. I have shed my IBM favoritism and seen it for what it was "stubbornness". I would highly recommend considering SpectraLogic the next time you seek to buy new or refresh old equipment. They definitely have the features everyone is looking for available in their libraries, and with data center space at a premium you can count on the T950 to give you the capacity you need in less space than competing libraries.
***UPDATE***
The mount time is a a lit more excessive than thought. Because of the TeraPack having to be removed, then the tape grabbed and placed in the drive, then the terapack replaced; the mount times are quite a bit longer when you have a VERY busy queue. I think SpectraLogic needs to speed this process up somehow. I must say if they can speed it up the overall library density is great. It would be cool if there was some way they could utilize a tool that grouped all scratch together and tapes that are most frequently mounted. Almost like a tape slot reclamation. It might speed things along when say 5 systems are all waiting for scratch. I need to research this a little more. I'll see if I can get feedback from SpectraLogic.
NetApp Drive Definitions
I always forget the NetApp drive definitions so i thought I would post the example I found here so I can recall it later. If any of you wondered which definition TSM uses or would like to know why here is your answer:
rst4l - rewind device, format is: H Format 30 GB
nrst4l - no rewind device, format is: H Format 30 GB
urst4l - unload/reload device, format is: H Format 30 GB
rst4m - rewind device, format is: H Format 30 GB
nrst4m - no rewind device, format is: H Format 30 GB
urst4m - unload/reload device, format is: H Format 30 GB
rst4h - rewind device, format is: H Format 30 GB
nrst4h - no rewind device, format is: H Format 30 GB
urst4h - unload/reload device, format is: H Format 30 GB
rst4a - rewind device, format is: H Format 60 GB comp
nrst4a - no rewind device, format is: H Format 60 GB comp
urst4a - unload/reload device, format is: H Format 60 GB comp
The one is red is the definition TSM uses since it rewinds the tape, but leaves the unload/reload to TSM and not the device itself.
rst4l - rewind device, format is: H Format 30 GB
nrst4l - no rewind device, format is: H Format 30 GB
urst4l - unload/reload device, format is: H Format 30 GB
rst4m - rewind device, format is: H Format 30 GB
nrst4m - no rewind device, format is: H Format 30 GB
urst4m - unload/reload device, format is: H Format 30 GB
rst4h - rewind device, format is: H Format 30 GB
nrst4h - no rewind device, format is: H Format 30 GB
urst4h - unload/reload device, format is: H Format 30 GB
rst4a - rewind device, format is: H Format 60 GB comp
nrst4a - no rewind device, format is: H Format 60 GB comp
urst4a - unload/reload device, format is: H Format 60 GB comp
The one is red is the definition TSM uses since it rewinds the tape, but leaves the unload/reload to TSM and not the device itself.
Sunday, January 7, 2007
Tivoli Storage Manager 5.2 and Capacity on Demand gotcha.
A customer recently commissioned a new frame for their 3584 library and installed some new tape drives. TSM was left running under the assumption that paths and drives could be taken offline whilst work was performed on the library to ensure availability to backup to the disk storage pools.
Unfortunately this is not the case when you are commissioning additional storage slots (frames or capacity on demand).
The first indication that something was astray was that after the paths and drives online was that the the inability to check in any more tapes as the library was still showing that it was full.
The new frame allowed for over 400 more slots so this also didn’t make sense. After some thought I remembered the ’show slots’ command. After running the command I noticed that I was only being shown 704 slots rather then ther full number of slots I knew that were available.
The diagnostics and administration tool "tapeutil" showed the new and updated element range which confirmed that the configuration from an operating point of view was correct.
So the problem obviously was to do with TSM:
After some research, I discovered that the element count is only queried once upon TSM startup and it is not possible to update the this information without restarting TSM.
Unfortunately this is not the case when you are commissioning additional storage slots (frames or capacity on demand).
The first indication that something was astray was that after the paths and drives online was that the the inability to check in any more tapes as the library was still showing that it was full.
The new frame allowed for over 400 more slots so this also didn’t make sense. After some thought I remembered the ’show slots’ command. After running the command I noticed that I was only being shown 704 slots rather then ther full number of slots I knew that were available.
The diagnostics and administration tool "tapeutil" showed the new and updated element range which confirmed that the configuration from an operating point of view was correct.
So the problem obviously was to do with TSM:
tsm:> show slots 3584atl
PVR slot information for library L32ATL.
Library : L32ATL
Product Id : 03584L32
Support module : 4
Mount
count : 5
Drives : 16
Slots : 704
Changers : 2
Import/Exports :
10
Device : /dev/smc0
Drive 0, element 257
Drive 1, element 258
Drive 2, element 259
Drive 3, element 260
Drive 4, element 261
Drive 5, element 262
Drive 6, element 263
Drive 7, element 264
Drive 8, element 265
Drive 9, element 266
Drive 10, element 267
Drive 11, element 268
Drive 12, element 269
Drive 13, element 270
Drive 14, element 271
Drive 15, element 272
Changer 0, element 1
Changer 1, element 2
ImpExp 0, element number 769
ImpExp 1,
element number 770
ImpExp 2, element number 771
ImpExp 3, element number
772
ImpExp 4, element number 773
ImpExp 5, element number 774
ImpExp
6, element number 775
ImpExp 7, element number 776
ImpExp 8, element
number 777
ImpExp 9, element number 778
slot element range 1025 -
1728
After some research, I discovered that the element count is only queried once upon TSM startup and it is not possible to update the this information without restarting TSM.
tsm:> show slots 3584atl
PVR slot information for library L32ATL.
Library : L32ATL
Product Id : 03584L32
Support module : 4
Mount
count : 8
Drives : 20
Slots : 1130
Changers : 2
Import/Exports :
10
Device : /dev/smc0
Drive 0, element 257
Drive 1, element 258
Drive 2, element 259
Drive 3, element 260
Drive 4, element 261
Drive 5, element 262
Drive 6, element 263
Drive 7, element 264
Drive 8, element 265
Drive 9, element 266
Drive 10, element 267
Drive 11, element 268
Drive 12, element 269
Drive 13, element 270
Drive 14, element 271
Drive 15, element 272
Drive 16, element 273
Drive 17, element 274
Drive 18, element 275
Drive 19, element 276
Changer 0, element 1
Changer 1, element 2
ImpExp 0, element
number 769
ImpExp 1, element number 770
ImpExp 2, element number 771
ImpExp 3, element number 772
ImpExp 4, element number 773
ImpExp 5,
element number 774
ImpExp 6, element number 775
ImpExp 7, element number
776
ImpExp 8, element number 777
ImpExp 9, element number 778
slot element range 1025 - 2154
Wednesday, July 5, 2006
Import/Export Question!
Can someone explain to me why moving data (exporting/importing) from one TSM server to another TSM server is such a pain in the neck? Here is my scenario, I have some servers that moved from one location to another, network wise, and they now need to backup to a different TSM server. Both servers use the same media type (LTO-3). So why is it I have to either copy all the data across the network to the new server as it creates new tapes, or dump it all to tape(s) and then rewrite it to new tape(s) when imported? My question is this - why can’t I just export the DB info and pointers for the already existing tape to the new TSM server from the old? Why can’t the old server “hand-over” the tape to the new TSM server? It seems like a lot of wasted work to constantly have to copy the data (server to server export/import) or dump it to tape and then write the data to new tapes. I think the developers ought to work on a way of doing this. I would also think that this process could be done on a DB level so you could in a sense reorg the DB without the long DB Dump/Load and audit process, and the tapes would be handed over to the new TSM server. No rewrites necessary.
Subscribe to:
Posts (Atom)