Load Balancing Using Adaptive Overlapped Data Chained Declustering For Distributed File Systems
Abstract
integration of data from heterogeneous databases is inevitable.
Virtual Database Technology (VDB) is one of the effective
solutions for integration of data from heterogeneous sources.
This will become complex when size of the database is very
large. MapReduce is a new framework specifically designed
for processing huge datasets on distributed sources. Apaches
Hadoop is an implementation of MapReduce. Distributed file
systems (DFS) are key building blocks for cloud computing
applications based on the MapReduce programming paradigm. In
such file systems, nodes simultaneously serve computing and
storage functions; a file is partitioned into a number of chunks
allocated in distinct nodes so that MapReduce tasks can be
performed in parallel over the nodes.
However, in a cloud computing environment, failure is the norm,
and nodes may be upgraded, replaced, and added in the system.
Files can also be dynamically created, deleted, and appended. This
results in load imbalance; that is, the file chunks are not distributed
as uniformly as possible in the nodes. The performance of the
proposal implemented in the Hadoop distributed file system is
further investigated in a cluster environment.
Full Text:
PDFReferences
Hsueh-Yi Chung, Che-Wei hang Hung-Chang Hsiao, Yu-Chang Chao,"The
Load Rebalancing Problem in Distributed File Systems,“, 2012
Hung-Chang Hsiao, Hsueh-Yi Chung, HaiyingShen, Yu-Chang Chao,"Load
Rebalancing for Distributed File Systems in Clouds,", 2013.
WenqiuZeng, Ying Li, Jian Wu, Qingqing Zhong, Qi Zhang, "Load
rebalancing in Large-Scale Distributed File System,” 2009.
L.Kirankumar,V.Ranjithkumar, "Application of Hadoop MapReduce
Technique to Virtual Database System Design," 2011.
FeiHu ,JimZiobro, Jason Tillett, Neeraj K. Sharma,"CATCH: A Cloud-based
Adaptive Data Transfer Service for HPC," 2011.
SabaSehrish, Grant Mackey, Pengju Shang, Jun Wang, "Supporting HPC
Analytics Applications with Access Patterns Using Data Restructuring and Data-
Centric Scheduling Techniques in MapReduce," 2013.
Bin Wu, Shengqi Yang, Haizhou Zhao, and Bai Wang, “A Distributed
Algorithm to Enumerate All Maximal Cliques in MapReduce,” .
Suresh M., ShafiUllah Z., Santhosh Kumar B., “An Analysis of Load
Balancing in Cloud Computing“2013.
R. X. T. and X. F. Z.” A Load Balancing Strategy Based on the Combination
of Static and Dynamic, (DBTA)”, 2010 2nd International Workshop 2010.
Abhijit A.Rajguru, S.S. Apte, “A Comparative Performance Analysis
of Load Balancing Algorithms In Distributed Systems Using Qualitative
Parameters”, International Journal of Recent Technology and Engineering, Vol.
, Issue 3, August 2012.
Eager, D., Lazowska, E., and J. Zahorjan, ”Adaptive Load Sharing in
Homo- geneous Distributed Systems,” IEEE Transactions on Software
Engineering, Vol. SE-12, No. 5, May 1986.
Carey, M., Livny, M., and H. Lu, ”Dynamic Task Allocation in a
Distributed Database System,” Proceedings of the 5th International Conference
on Dis- tributed Computer Systems, Denver, May 1985.
Y. Hua, Y. Zhu, H. Jiang, D. Feng, “Supporting Scalable and Adaptive
Metadata Management in Ultra Large-scale File Systems,” IEEE Trans. Parallel
Distrib. Syst., vol. 22, no. 4, pp. 580–593, Apr. 2011.
Hadoop Distributed File System, http://hadoop.apache.org/hdfs.
Refbacks
- There are currently no refbacks.
Copyright © IJETT, International Journal on Emerging Trends in Technology