CITI: Projects: ASC

Introduction

Large data stores are pushing the limits of modern technology. Parallel file systems provide high I/O throughput to large data stores, but are limited to particular operating system and hardware platforms, lack seamless integration and modern security features, and suffer from slow offsite performance. Meanwhile, advanced research collaborations are requiring higher bandwidth as well as concurrent and secure access to large datasets across myriad platforms and parallel file systems, forming a schism between file systems and their users.

The above issues are nowhere more apparent than with the Advanced Simulation and Computing program (ASC), formerly known as the Accelerated Strategic Computing Initiative (ASCI). With majority of the world's fastest computers, speed and standards are paramount. NFSv4 is designed for speed and is the nascent Internet standard protocol for distributed file systems. This project investigates the application of NFSv4 technologies to the massive data handling requirements of ASC scientists. The result of this investigation may ultimately enable wide-area access to data repositories for data collection and post-analysis with orders of magnitude improvements in capacity and bandwidth while enforcing and preserving consistent and secure shared access.

Personnel

Peter Honeyman	Scientific Director, CITI.
William A. (Andy) Adamson	Assistant Director, CITI.
J. Bruce Fields	Assistant Research Scientist, CITI.
Dean Hildebrand	Graduate Student

Project Sponsors

ASC at Livermore, Sandia, and Los Alamos.
The Scalable I/O research groups at Sandia, and Livermore.

Publications

07-2 pdf
Dean Hildebrand and Peter Honeyman, "Direct-pNFS: Scalable, Transparent, and Versatile Access to Parallel File Systems," June 2007. [Proceedings of the 16th IEEE International Symposium on High Performance Distributed Computing, Monterey, CA, (June 2007).]
07-1 pdf
Dean Hildebrand, Peter Honeyman, and Wm. A. (Andy) Adamson, "pNFS and Linux: Working Towards a Heterogeneous Future," May 2007. [Proceedings the 8th LCI International Conference on High-Performance Clustered Computing, Lake Tahoe, CA, (May 2007).] Supercedes Center for Information Technology Integration Technical Report CITI-TR-06-6.
06-6 pdf
Dean Hildebrand and Peter Honeyman, "pNFS and Linux: Working Towards a Heterogeneous Future," May 2006
06-5 pdf
Wm. A. (Andy) Adamson, Dean Hildebrand, Peter Honeyman, Shawn McKee, and Jiaying Zhang, "Extending NFSv4 for Petascale Data Management," in Proceedings of the HPDC Workshop on Next-Generation Distributed Data Management, Paris, France, June 2006.
06-4 pdf
Dean Hildebrand, Lee Ward, and Peter Honeyman, "Large Files, Small Writes, and pNFS," in Proceedings of the 20th ACM International Conference on Supercomputing (ICS06), Cairns, Australia, June 2006.
05-2 pdf
Dean Hildebrand and Peter Honeyman, "Scaling NFSv4 with Parallel File Systems," in Proceedings of Cluster Computing and Grid (CCGrid05), Cardiff, United Kingdom, May 2005.
05-1 pdf
Dean Hildebrand and Peter Honeyman, "Exporting Storage Systems in a Scalable Manner with pNFS," in Proceedings of the 22nd IEEE - 13th NASA Goddard (MSST2005) Conference on Mass Storage Systems and Technologies, Monterey, California, April 2005.
04-2 pdf
Dean Hildebrand and Peter Honeyman, "NFSv4 and High Performance File Systems: Positioning to Scale," in Proceedings of NFS Extensions for Parallel Storage Workshop, Ann Arbor, Michigan, December 2004.
doc
"Distributed Filing in the Grid," Draft 2 (in preparation), February 2003.

pNFS

Use the NFSv4.1 protocol to perform direct access to storage while preserving operating system and hardware platform independence.

Split-Server NFS

Achieve full utilization of an unmodified storage system's available bandwidth while retaining the security, consistency, and heterogeneity features of NFSv4.

Projects: ASC

Introduction

Personnel

Project Sponsors

Publications

pNFS

Split-Server NFS

ASC/ICSI Linux NFSv4 Testing

Development Resources

References and Links