top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

Hadoop: namenode doesn't update block locations when data directories of a datanode is changed?

+2 votes
300 views

I am running hadoop-2.4.0 cluster. Each datanode has 10 disks, directories for 10 disks are specified in dfs.datanode.data.dir.

A few days ago, I modified dfs.datanode.data.dir of a datanode () to reduce disks. so two disks were excluded from dfs.datanode.data.dir, after the datanode was restarted, I expected that the namenode would update block locations. In other words, I thought the namenode should remove from block locations associated with blocks which were stored on excluded disks, but the namenode didnt update the block locations...

In my understanding, datanode send a block report to the namenode when datanode start so the namenode should update block locations immediately.

Is a bug? Could anyone please explain?

posted Jul 13, 2015 by anonymous

Looking for an answer?  Promote on:
Facebook Share Button Twitter Share Button LinkedIn Share Button

Similar Questions
+1 vote

I have an old Hadoop install that Im looking to update to Hadoop 2. In the old setup, I have a /conf/masters file that specifies the secondary namenode.

Looking through the Hadoop 2 documentation I cant find any mention of a "masters" file, or how to setup a secondary namenode.

Thanks in advance?

+4 votes

I am using Hadoop cluster with 9 nodes. I would like to know what is the basic datanode configuration in Hadoop cluster.

I am using following configuration on Namenode and Datanode.

RAM = 4GB Cores = 4 Disk = 8 ( Total 16GB storage space)

Running sample sort and word count jobs to check the hadoop network performance.
Is the configuration I have chosen is right ?

...