In an earlier post I mentioned that prior to the introduction of LTFS an application typically relied on proprietary APIs to read and write data tape. These proprietary APIs typically have used proprietary data formats on the tape media. These proprietary APIs and data formats lock an application developer to a specific layer of software which mediates access to the data tape storage.
With LTFS we took a very different approach. The LTFS filesystem-based approach to accessing data stored on tape brings significant benefits to the tape market. For example, application developers no longer need to learn specialized APIs, and users get a fully self-describing storage medium that largely behaves the same as more common storage media such as hard-disk drives and memory sticks.
The benefits afforded by LTFS are arguably sufficient to support releasing LTFS as proprietary software that uses a closed data format on the tape media. Rather than treating LTFS as yet another proprietary on-tape format, the LTFS team at IBM elected to release the LTFS Single Drive Edition (LTFS-SDE) software as open-source[1. Open-source release of LTFS-SDE is available for Linux and Mac OS X. The LTFS-SDE implementation for MS Windows is freely available as a binary release. The MS Windows implementation could not be released as open-source due to licensing terms on necessary third-party components]. The LTFS open-source software release is licensed under the LGPL license to protect the LTFS format while providing application developers the ability to dynamically link with LTFS binary code without risking their own intellectual property. Each fix-pack released for LTFS-SDE has also been released under the LGPL.
In addition to the open-source software release, the LTFS team wrote the LTFS Format Specification document and released this specification to the public. This Format Specification describes the on-media data structures in sufficient detail for independent developers to read and write LTFS tapes that are compatible with other LTFS implementations.
During development of the LTFS software and the LTFS Format Specification the IBM team built a working relationship with developers at both HP and Quantum through the LTO Consortium. These relationships afforded an opportunity to inform HP and Quantum about the up-coming LTFS technology and allowed for some pre-release co-ordination between these three LTO 5 vendors. These on-going relationships have provided the opportunity to conduct inter-operablity testing between all three vendors. We are fortunate to enjoy continuing discussions between vendors on the direction and development of the LTFS format.
This communication, the open-specification, and the open-source approach to software distribution is motivated by a desire for LTFS to become an industry-wide standard. During LTFS development we recognized that if LTFS was a proprietary release then we would be adding to the fragmentation and incompatibility of data tape use. Instead, a release of LTFS as an open implementation with an open specification would create an environment that may lead to broad industry adoption. Broad adoption along with the ease of use benefits may result in expanded use of tape media for data storage. Expansion of the tape market would be good for the industry and all of us involved in the data tape business.
Hello Michael,
First of all, Happy New Year.
Now, my company is looking for a way to integrate LTFS into our system. We made some tests with the LTFS-SDR (windows version) and it looks great but it is unusable in an automated environment.
I read that you don’t work on the project anymore but I tough you could point me out to the right place where to get a proper ltfs sdk or an ltfs-sdr build without all the ui requirements (pop-up on mount / eject trough interface / etc / … ) for windows as its source code was never released.
Best Regards.
Hello,
During my time working on LTFS I argued strongly that the LTFS Windows implementation should be implemented as a stand-alone command-line tool just like the Linux and OS X implementations. My personal view is that the GUI integration for Windows should be a wrapper around a command-line implementation specifically to allow system integrators to have the option of completely removing the GUI elements. Unfortunately IBM management were not willing to adopt this approach.
With regard to an ltfs sdk or ltfs-sdr build, I know of there is no such build available to the public. The LTFS codebase on Windows contains roughly 98% of the Linux LTFS codebase. But on Windows IBM uses a third-party library to hook into the Windows kernel. This library is only licensed by IBM for binary distribution so there no good options for distributing the LTFS Windows source given this dependency. I do not know the current IBM thinking regarding the Windows codebase.
With regard to integrating into a Windows-based system, I know of two approaches that have been used by system integrators. Approach one is to use the LTFS for Windows binary code and actively monitor the LTFS log events to identify when LTFS transitions to/from an active filesystem state when a tape is inserted/ejected. Approach two is to use a virtualization product like VMware Player to host a guest Linux OS on your Windows machine. This Linux guest can run Linux LTFS and export the LTFS volume via a CIFS/Samba share. The Windows machine running on the physical hardware can then mount the exported LTFS volume using Windows file sharing.
Both of these approaches have their own particular trade-offs and rough-edges. However, both of these approaches have been used by system integrators in shipping solutions delivered to significant numbers of customers.
Hi Michael,
Thanks for the great article. So this third party library that talks to the windows kernel is why Quantum and HP have not released windows LTFS drivers? I’m a Mac user but I was shocked when I bought a Quantum LTO5 drive to be installed on my Windows machine and realized that only IBM supports windows LTFS (and costs almost twice the price of a HP or Quantum LTO5 drive). Since I’m developing a software for digital camera logging, report and archival, I thought of hiring someone to implement LTFS on it and bypass the vendors software, but reading your article now, I imagine this will be a problem on Windows, since it’s gui based, right? Do you recommend the VMware route? Thank you, and congratulations for the excellent work in revolutionizing storage.
Thank you for your kind words.