Blocks
A disk has a block size, which is the minimum
amount of data that it can read or write.
Hadoop also has a concept of a block, minimum block
size—64 MB by default.
Files in HDFS are broken into block-sized chunks, which
are stored as independent units.
Except the last block remaining all
blocks are of equal size.
The Blocks cannot be divided or shared
between the files.
Hadoop can process large files
efficiently instead of small files.
Block size can also be changed with the
help of configurations.
When we install Hadoop, the
Hadoop folder contains some configuration files the Ex: path is /Hadoop/conf/hdfs-site.xml in this file we have to configure the
block size.
<configuration>
<property>
<name>dfs.block.size</name>
<value>141213058</value>
</property>
</configuration>
Replication in HADOOP
Replication is nothing but maintaining
same copy of file in another node.
-In our Traditional distributed system,
if one node is fail we have to wait until it is recovered or backup is done,
till then the result cannot be released.
-But in Hadoop we can get the data from
another node as soon as the node fails. It will be possible with the help of
replication.
Formula for replication is = n-1 where n = no. of slave nodes
Hadoop is always expecting commodity
Software and Hardware.
The default replication factor is “3”,
we can also configure using configuration file.
/Hadoop/conf/hdfs-site.xml in this file
we have set the property.
<configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
</configuration>
-If we change the replication
factor or block size in configuration file, we have restart the cluster to
apply the changes for coming Data.
No comments:
Post a Comment