SAM2 Toolkit: Scalable and Adaptive Metadata Management for High-end Computing

Sponsored by NSF-HECURA

Collaborated project between University of Nebraska, University of Maine and University of Central Florida

Abstracts: The increasing demand for Exa-byte-scale storage capacity by high end computing applications requires a higher level of scalability and dependability than that provided by current file and storage systems. The proposal deals with file systems research for metadata management of scalable cluster-based parallel and distributed file storage systems in the HEC environment. It aims to develop a scalable and adaptive metadata management (SAM2) toolkit to extend features of and fully leverage the peak performance promised by state-of-the-art cluster-based parallel and distributed file storage systems used by the high performance computing community. There is a large body of research on data movement and management scaling, however, the need to scale up the attributes of cluster-based file systems and I/O, that is, metadata, has been underestimated. An understanding of the characteristics of metadata traffic, and an application of proper load-balancing, caching, prefetching and grouping mechanisms to perform metadata management correspondingly, will lead to a high scalability. It is anticipated that by appropriately plugging the scalable and adaptive metadata management components into the state-of-the-art cluster-based parallel and distributed file storage systems one could potentially increase the performance of applications and file systems, and help translate the promise and potential of high peak performance of such systems to real application performance improvements. The project involves the following components:

  • Develop multi-variable forecasting models to analyze and predict file metadata access patterns
  • Develop scalable and adaptive file name mapping schemes to enforce load balance and increase scalability
  • Develop decentralized, locality-aware metadata grouping schemes to facilitate the bulk metadata operations such as prefetching
  • Develop an adaptive cache coherence protocol using a distributed shared object model
  • Prototype the SAM^2 toolkit into PVFS^2 and dCache

Research Facilities

  • RCF Supercomputers:
    • Homestead - Our 32 processor SGI Origin 300 is our original machine.
    • Prairiefire - 128 dual processor AMD machines packed together into a small space make for one hot cluster. It also makes up our beloved Prairiefire.
    • Red - This cluster is the resource for UNL's US CMS Tier-2 site. It consists of 256 compute cores and 50 TB of storage.
    • Prairieview - A visualization cluster powering RCF's Tiled Display wall for high-resolution images.
  • UMaine High Performance Computing:
    • Bender: 256-node dual Xserve G5 cluster
    • Blackbear: 208-node dual P3 cluster
    • Diego: 8-node Sun Fire T1000. Each server has 8GB of DDR2 memory and an 8-core 1.0GHz UltraSPARC T1 processor supporting 32 simultaneous processing threads.
    • Manfred: 16-node dual P3 cluster

Principle Investigators

Personnel Supported

  • Brian Bockleman, University of Nebraska (PhD Student)
  • Lin Lin, University of Nebraska (PhD Student)
  • Dongyuan Zhan, University of Nebraska (PhD Student)
  • Cai Zhao, University of Maine (PhD Student)
  • Jihanhui Yue, University of Maine (PhD Student)
  • Charles Jones, University of Maine (Master Student)
  • Peng Gu, University of Central Florida (PhD Student)
  • Huijun Zhu, University of Central Forida (PhD Student)
  • Dr. Xueming Li, a visiting scholar from China (Post Dr.)
  • Lei Tian, a visiting research scholar from Huazhong Univerisity of Science Technology (Post Dr.)

Recent publications

  • Y. Zhu, H. Jiang, J. Wang and F. Xian, "HBA: Distributed Metadata Management System for Large Cluster-based Storage" IEEE Transaction on Distributed and Parallel Systems, Vol. 19, No. 6, pp. 750-763, June 2008.
  • Y. Zhu, and H. Jiang, "RACE: A Robust Adaptive Caching Strategy for Buffer Cache", IEEE Transaction on Computers, Vol. 57, No. 1, pp. 25-40, January 2008
  • Peng Xia, Dan Feng, Hong Jiang, Lei Tian, and Fang Wang, "FARMER: A Novel Approach to File Access Correlation Mining And Evaluating Reference Model for Optimizing Peta-Scale File Systems Performance," To appear in the Proceedings of the 17th ACM/IEEE International Symposium on High Performance Distributed Computing (HPDC 2008), Boston, MA, June 23-27, 2008 (Acceptance rate: 17%)
  • Bo Mao, Dan Feng, Hong Jiang, Suzhen Wu, Jianxi Chen, Lingfang Zeng, "GRAID: A Green RAID Storage Architecture with Improved Energy Efficiency and Reliability," to appear in the Proceedings of the 16th Annual Meeting of the IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), Baltimore, MD. USA, September 8-10, 2008. (acceptance rate: 36/94 = 38%)
  • Q. Zou, D. Feng, Y. Zhu, H. Jiang, X. Ge, and Z. Zhou, "A Novel and Generic Model for Synthesizing Disk I/O Traffic Based on The Alpha-stable Process", to appear in Proceedings of 16th Annual Meeting of the IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), Baltimore, MD, Sept 8-10, 2008 (acceptance rate: 36/94 = 38%)
  • J. Yue, Y. Zhu, Z. Cai, "Energy Efficient Buffer Cache Replacement", to appear in Proceedings of 16th Annual Meeting of the IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), Baltimore, MD, Sept 8-10, 2008 (Poster Paper)
  • J. Yue, Y. Zhu, Z. Cai, "Impacts of Indirect Blocks on Buffer Cache Energy Efficiency", to appear in Proceedings of the 37th International Conference on Parallel Processing (ICPP 2008), Portland, Oregon, USA, September 2008 (acceptance rate: 81/263 = 31%)
  • Peng Gu, Jun Wang, Robert Ross. Bridging The Gap Between Parallel File Systems and Local File Systems: A Case Study with PVFS. The 37th International Conference on Parallel Processing 2008. September 8–12, Portland, Oregon, USA. ICPP 2008.
  • Hailong Cai, Ping Ge, Jun Wang. Applications of Bloom Filters in Peer-to-peer Systems: Issues and Questions. International Conference on Networking, Architecture, and Storage (NAS) 2008.
  • Zhongying Niu, Ke Zhou, Dan Feng, Hong Jiang, Frank Wang, Hua Chai, Wei Xiao,and Chunhua Li, "Implementing and Evaluating Security Controls for an Object-Based Storage System", to appear in the Proceedings of the 24th IEEE Conference on Mass Storage Systems and Technologies (MSST'07), San Diego, California, USA September 24-27, 2007.
  • Peng Gu, Jun Wang, Hailong Cai. ASAP: An Advertisement-based Search Scheme for Unstructured Peer-to-Peer Systems. International Conference on Parallel Processing (ICPP 2007), Sept. 2007. Xi'an, China.
  • Zhongying Niu, Ke Zhou, Dan Feng, Hong Jiang, Frank Wang, Hua Chai, Wei Xiao,and Chunhua Li, Implementing and Evaluating Security Controls for an Object-Based Storage System, in the Proceedings of the 24th IEEE Conference on Mass Storage Systems and Technologies (MSST'07), San Diego, California, USA September 24-27, 2007, pp. 87-99 (acceptance rate: 18.7%).
  • J. Yue, Y. Zhu and Z. Cai, "Evaluating Memory Energy Efficiency in Parallel I/O Workloads", in Proceedings of IEEE International Conference on Cluster Computing, 2007 (Best Paper Award)
  • Yu Hua, Dan Feng, Hong Jiang and Lei Tian, RBF: A New Storage Structure for Space-Efficient Queries for Multidimensional Metadata in OSS, in the 5th USENIX Conference on File and Storage Technologies (FAST '07) Work-in-Progress (WiP) Report, San Jose, CA, February 13-16, 2007
  • Lei Tian, Hong Jiang, Dan Feng, Qin Xin and Xing Shu, "Implementation and Evaluation of a Popularity-Based Reconstruction Optimization Algorithm in Availability-Oriented Disk Arrays", to appear in the Proceedings of the 24th IEEE Conference on Mass Storage Systems and Technologies (MSST'07), San Diego, California, USA September 24-27, 2007
  • Lei Tian, Dan Feng, Hong Jiang, Ke Zhou, Lingfang Zeng, Jianxi Chen, Zhikun Wang, and Zhenlei Song, "PRO: A Popularity-based Multi-threaded Reconstruction Optimization for RAID-Structured Storage Systems" in Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST '07), San Jose, CA, February 13-16, 2007 (Acceptance Rate: 19%)
  • Yu Hua, Dan Feng, and Hong Jiang, RBF: A New Storage Structure for Space-Efficient Queries for Multidimensional Metadata in OSS, in the 5th USENIX Conference on File and Storage Technologies (FAST '07) Work-in-Progress (WiP) Report, San Jose, CA, February 13-16, 2007
  • Y. Zhu and H. Jiang, "On the Analysis and Impact of False Rates of Bloom Filters in Distributed Systems," in Proceedings of the 35th International Conference on Parallel Processing (ICPP), Columbus, pp. 255-262, OH, August, 2006 (Acceptance rate: 64/200 = 32%).
  • P. Gu, Y. Zhu, H. Jiang, and J. Wang, "Nexus: A Novel Weighted-Graph-Based Prefetching Algorithm for Metadata Servers in Petabyte-Scale Storage Systems," in the Proceedings of International Symposium on Cluster Computing and the Grid (CCGrid, 2006), May 16-19, 2006, Singapore, pp. 409-416. (Acceptance rate: 61/257 = 25.7%)

99 visitors since August, 2006.