发布时间：2025-04-30

论文名称：An optimized approach for storing and accessing small files on cloud storage

发表刊物：Journal of Network and Computer Applications

摘要：This paper firstly analyzes and points out the reasons of smallfile problem of HDFS: (1) large numbers of smallfiles impose heavy burden on NameNode of HDFS; (2) correlations between smallfiles are not considered for data placement; and (3) no optimization mechanism, such as prefetching, is provided to improve I/O performance. Secondly, in the context of HDFS, the clear cut-off point between large and smallfiles is determined through experimentation, which helps determine ‘how small is small’. Thirdly, according to file correlation features, files are classified into three types: structurally-related files, logically-related files, and independent files. Finally, based on the above three steps, an optimizedapproach is designed to improve the storage and access efficiencies of smallfiles on HDFS. File merging and prefetching scheme is applied for structurally-related smallfiles, while file grouping and prefetching scheme is used for managing logically-related smallfiles. Experimental results demonstrate that the proposed schemes effectively improve the storage and access efficiencies of smallfiles, compared with native HDFS and a Hadoop file archiving facility.（http://www.sciencedirect.com/science/article/pii/S1084804512001610）

合写作者：董博，郑庆华，田锋（通信作者）等

卷号：**

页面范围：**

是否译文：否

发表时间：2012-08-03