OpenSFS
| Consortium | |
| Founded | 2010 |
| Headquarters | Beaverton, Oregon, United States |
Key people | Charlie Carroll (Interim Chairman) |
| Website |
www |
Open Scalable File Systems, Inc. (OpenSFS) is a nonprofit organization dedicated to the success of the Lustre file system. OpenSFS was founded in 2010 to advance Lustre, ensuring it remains vendor-neutral, open, and free. Since its inception, OpenSFS has been responsible for advancing Lustre and delivering new releases on behalf of the open source community. Through working groups, events, and ongoing funding initiatives, OpenSFS intends to fuel innovation and growth of Lustre worldwide.[1]
History
The Lustre file system architecture started in 1999 as a research project by Peter Braam, who was on staff at Carnegie Mellon University (CMU) at the time. Braam then founded Cluster File Systems in 2001, with work from the InterMezzo file system in the Coda project at CMU.[2] Lustre was developed under the Accelerated Strategic Computing Initiative Path Forward project funded by the United States Department of Energy, which included Hewlett-Packard and Intel.[3] In September 2007, Sun Microsystems acquired the assets of Cluster File Systems Inc. including its IP.[4][5] Sun intended to bring Lustre technologies to Sun's ZFS file system and the Solaris operating system. In November 2008, Braam left Sun Microsystems, and Eric Barton and Andreas Dilger took control of the project. After having acquired Sun in 2010, Oracle Corporation began to manage and release Lustre.
Lustre is widely deployed in key computational centers worldwide including many TOP500 systems. When Oracle acquired Sun in 2010, however, ongoing development of Lustre was discontinued, prompting a number of prominent Lustre developers to leave Oracle/Sun for other entities committed to taking Lustre forward.[6] OpenSFS was subsequently founded in 2010 to steward the new open source Lustre community.
In 2011, Lustre 2.1 was the first community release endorsed by OpenSFS and other community entities. OpenSFS subsequently began direct funding of all Lustre community releases in early 2012. These releases are focused on introducing new features to Lustre. Feature releases are targeted every six months. There are also Lustre maintenance releases based on the 'current maintenance branch' of Lustre; these releases focus solely on bug fixes and stability. Maintenance releases are targeted every three months.[7] All Lustre feature and maintenance releases are regularly updated on the OpenSFS community Lustre roadmap.
Timeline history of OpenSFS
2010
- OpenSFS was founded to steward the open source Lustre community. The four founding "Participants" (paying members) of OpenSFS were Cray, DataDirect Networks, Lawrence Livermore National Laboratory, and Oak Ridge National Laboratory.
2011
- OpenSFS, and others, endorsed the release of Lustre 2.1 by the open source community. Various Lustre community entities then consolidated to create a unified, global effort to support Lustre via OpenSFS.
- OpenSFS solicited proposals, based on community requirements, and funded a 2-year development contract for key new features, including improved Lustre metadata performance and scalability.
- OpenSFS took on ongoing responsibility for the annual Lustre Users Group (LUG) event.
2012
- OpenSFS took on responsibility for Lustre tree maintenance, a critical element of community based development.
- OpenSFS began funding ongoing Lustre feature releases. Lustre 2.2 was the first OpenSFS funded release.
2013
- OpenSFS funded releases continue, including Lustre 2.5 in October 2013 containing the much anticipated Lustre+HSM integration capability.[8][9][10][11]
- OpenSFS established the Lustre Community Portal to support the open source community.[12] This portal supports Lustre community developers, Lustre releases (downloadable), documentation, working groups, bug tracking, and more.
- OpenSFS solicited proposals for Lustre feature development, parallel file system tools, addressing Lustre technical debt, and parallel file system incubators.[13]
- OpenSFS held the first annual APAC Lustre Users Group events in China and Japan, supplementing the annual U.S. based LUG event (with worldwide participation).
- OpenSFS membership continued to grow, totaling 21 active members in 2013.
Goals
OpenSFS supports the continued evolution and success of the Lustre file system by ensuring that Lustre remains vendor-neutral, open, and free. Goals of OpenSFS investments include:[14]
- Further the Lustre roadmap to meet the highest priority requirements defined by the community
- Develop production quality tools to ease administration and use of open source scalable file systems
- Address Lustre technical debt to improve the code base and documentation thereof
- Encourage new efforts in open source scalable file systems for high performance and data intensive computing to broaden the set of solutions available to the community
Working Groups
OpenSFS Working Groups are collaborative teams of volunteers from OpenSFS Participants who contribute their expertise and ideas to the continued success of Lustre. Structured with specific group leaders, mailing list(s), and regularly scheduled teleconferences, Working Groups are designed to keep the group organized and informed no matter their location or work schedule.
Lustre Working Group[15]
- Gather requirements from the community
- Propose and manage development projects
- Generate Lustre feature roadmap
- Manage Lustre releases
- Coordinate release roadmap for new features
- Coordinate community development
Benchmarking Working Group[16]
- Research I/O workloads in high performance parallel file systems configurations
- Investigate HPC user workloads
- Define tests to evaluate system performance
Wide Area File System Working Group[17]
- Coordinate use cases and features for wide area Lustre
Marketing Working Group
- Brand management: guide and direct use of OpenSFS brand footprint
- Event marketing: promote and exhibit opportunities for OpenSFS and OpenSFS Participants
Lustre.org Working Group [18]
The Lustre.org Working Group manages the lustre.org web domain and its associated content. The lustre.org domain is a shared asset of European Open File Systems (EOFS) and Open Scalable File Systems (OpenSFS).
Lustre User Group (LUG) and other Lustre events
LUG: An annual event held by OpenSFS in April for discussion and seminars on the Lustre parallel file system and other open source file system technologies that provides more than 50 sessions and panels where attendees are able to immerse in the strong Lustre community, working collaboratively to further the development of Lustre.[19] LUG 2015 will be taking place April 13–15 in Denver, CO.
Supercomputing Conference (SC): The main high performance supercomputing (HPC) event annually brings together the international supercomputing community spotlighting the most advanced scientific and technical applications in the world, as well as providing a program of technical papers, tutorials and timely research posters. OpenSFS and EOFS jointly hold an annual Lustre BoF as part of this conference.
LAD: The annual Lustre Administrators and Developers Workshop is held by EOFS in September in France. It is a great opportunity for Lustre experts and new comers alike to gather and exchange experiences, developments, tools, best practices, and more.
APAC LUG: The annual APAC LUG event is held in Asia in the fall. These meetings are primarily held in the local languages although some English talks are included. APAC LUG events help expand the global Lustre community and maintain the same mission as the annual LUG in North America by providing an event consisting of discussion and seminars to gather and share expertise and experience.
International Supercomputing Conference (ISC): ISC is an IEEE-recognized key global conference and exhibition for high performance computing, networking and storage that takes place every June in Germany. EOFS and OpenSFS jointly hold a breakfast and BoF at this event.
Participants
Through the support and donation of the Participants in OpenSFS, the Lustre file system will remain open and free. Annual donations allow OpenSFS to provide an ongoing collaborative environment to ensure the continued success of the Lustre file system. There are three membership levels in OpenSFS: Promoter, Adopter and Supporter. Membership levels are based on annual donations.
Promoters
Cray, Intel, Oak Ridge National Laboratory, Seagate
Adopters
Argon National Laboratory, DDN, EMC2, Lawrence Livermore National Laboratory, NCSA, Sandia National Laboratories
Supporters
Aeon Computing, AWE, BP, Fermilab, Fujitsu, Indiana University, NASA, NetApp, SGI, TACC, Terascala, University of Florida
| Promoter | Adopter | Supporter |
|---|---|---|
| $300,000 annually | $50,000 annually | $5,000 annually |
| Voting seat on Board of Directors | NA | NA |
| Serve on Board Committees | NA | NA |
| Chair or co-chair working groups | Chair or co-chair working groups | NA |
| Participate in working groups | Participate in working groups | Participate in working groups |
| Vote on OpenSFS Stack | Vote on OpenSFS Stack | Vote on OpenSFS Stack |
Note: One representative can be nominated by the Adopter and Supporter community to serve as a voting Community Representative Director to the Board of Directors.
Funding
OpenSFS has a total annual operating budget and funds based solely on Participant donations that are made available for Lustre advancement and awareness.
References
- ↑ "About Us". OpenSFS. Retrieved 1 October 2013.
- ↑ Peter J. Braam (August 4, 2002). "Lustre, The Inter-Galactic File System" (PDF). Presentation slides. Lawrence Livermore National Laboratory. Retrieved September 23, 2013.
- ↑ R. Kent Koeninger (June 2003). "The Ultra-Scalable HPTC Lustre Filesystem" (PDF). Slides for presentation at Cluster World 2003. Retrieved September 23, 2013.
- ↑ Britta Wülfing (September 13, 2007). "Sun Assimilates Lustre Filesystem". Linux Magazine. Retrieved September 23, 2013.
- ↑ "Sun Microsystems Expands High Performance Computing Portfolio with Definitive Agreement to Acquire Assets of Cluster File Systems, Including the Lustre File System". Press release (Sun Microsystems). September 12, 2007. Archived from the original on October 2, 2007. Retrieved September 23, 2013.
- ↑ "Oracle has Kicked Lustre to the Curb". Inside HPC. 2011-01-10.
- ↑ "Lustre Releases". Intel. Retrieved 8 October 2013.
- ↑ Prickett Morgan, Timothy. "OpenSFS Announces Availability of Lustre 2.5". EnterpriseTech.
- ↑ Brueckner, Rich. "Video: New Lustre 2.5 Release Offers HSM Capabilities". Inside Big Data. Retrieved 11 December 2013.
- ↑ Hemsoth, Nicole. "Lustre Gets Business Class Upgrade with HSM". HPCwire. Retrieved 11 December 2013.
- ↑ "Lustre 2.5". Scientific Computing World. Retrieved 11 December 2013.
- ↑ "Lustre Community Portal". OpenSFS. Retrieved 1 October 2013.
- ↑ Brueckner, Rich. "With New RFP, OpenSFS to Invest in Critical Open Source Technologies for HPC". insideHPC. Retrieved 1 October 2013.
- ↑ "OpenSFS Increases Investment in Open Source for HPC". HPCwire. Retrieved 2 October 2013.
- ↑ "Lustre Working Group". OpenSFS. Retrieved 1 October 2014.
- ↑ "Benchmarking Working Group". OpenSFS. Retrieved 1 October 2013.
- ↑ "Wide Area Filesystem Working Group". OpenSFS. Retrieved 1 October 2013.
- ↑ "Lustre.org Working Group". OpenSFS. Retrieved 1 February 2015.
- ↑ "OpenSFS Published LUG 2013 Agenda". insideHPC. Retrieved 1 October 2013.