IBM LTO5 data tape media.
In April 2010 the Linear Tape Open (LTO) consortium released LTO5, the fifth generation of the LTO technology with hardware and media availability from consortium members. Amongst other improvements, LTO5 introduced support for tape partitioning to the LTO technology. LTO5 partitioning supports a maximum of 2 partitions per cartridge and requires LTO5 tape hardware and tape media.
Tape partitioning is not compatible with LTO4 media even though the LTO5 hardware can write to LTO4 media. Adding partition support to LTO4 would require changes to the LTO4 specification and existing hardware products. If the LTO4 media is partitioned then this media would not be compatible with the LTO4 hardware already in the field.
The Linear Tape File System (LTFS) uses the partitioning support in LTO5 to create two partitions on the tape media. One partition, known as the Index Partition (IP), is used as a central place to store an Index of the contents of the tape. This Index holds the filesystem meta-data for all files written to the LTFS Volume. In addition to the filename, date stamps, extended attributes, and folder names, the Index records the location of the file content on the tape media. The Index Partition is occupies a relatively small amount of the total media capacity.
In general use, the file content is written to the larger of the two partitions on the tape media. This partition is known as the Data Partition (DP). In the current implementations of LTFS the Data Partition has been 93.7% of the raw data tape media capacity. This translates to roughly 1.43TB on LTO5 media.
The Linear Tape File System software interprets this partitioned media format to read the current Index from the media when the filesystem is mounted. The content of the Index is used by the software to build an in-memory structure that represents the folder and file structures on the LTFS Volume. This in-memory structure allows the filesystem to immediately respond to requests to list files and folders, access extended attributes, show timestamps, and read/write permissions. In this way the user can browse the filesystem using tools like ‘ls’, ‘find’, Windows Explorer, or OS X Finder without requiring the filesystem to touch the tape media.
When a user or an application performs an operation to access the content of a file stored in LTFS the filesystem moves the tape media to the location of the relevant data and performs a read. The LTFS filesystem does not need to unpack the file content from a packaging structure like ‘tar’ and the filesystem is not caching the file content on disk. Instead, the filesystem exposes stored files as byte-addressable data just like other POSIX filesystems.
Byte-addressable access to file data allows applications and users to extract segments out of the middle of a stored file. This kind of data extraction allows applications to retrieve a short clip of a few seconds of video directly from much larger video files. Most previous uses of data tape require the whole video file to be restored to a hard drive before a short clip can be extracted. With LTFS, the need for a full restore is eliminated.