Client and NN communication in Hadoop?

Question

I have a Hadoop cluster running with 3 slaves and 1 Master. Slaves are Datanodes and running Tasktarckers. Namenode is running jobtracker and secondary namenode. I am running sample mapreduce after loading a file into HDFS.

According to Hadoop architecture ,before writing a file into HDFS , client will contact the Namenode and get the location details of DNs and then client directly write the file into DN.

What is this client ? is it an application running on Namenode ? Is user and client both are different ? How can I see the messages between client and datanodes?

Answer 1

The application that try to write the file, for example the binary "hadoop" found in the bin dir of hadoop is a hadoop client, whoever wants to read or write from to hadoop is a client, you can also write hadoop client using java apis which hadoop provide.
like following is an example of hadoop client which tries to read file from hadoop

import java.io.*; 
import java.util.*;
import java.net.*; 
import org.apache.hadoop.fs.*;
import org.apache.hadoop.conf.*; 
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapred.*; 
import org.apache.hadoop.util.*;

public class Cat {
  public static void main (String [] args) throws Exception{
  try{
    Path pt=new Path("hdfs://xxxx:9000/home/xxxx/fileonhdfstoread.txt");
    FileSystem filesysObject = FileSystem.get(new Configuration());
    BufferedReader bufferReaderObject=new BufferedReader(new InputStreamReader(filesysObject.open(pt)));
    String line;
    line=bufferReaderObject.readLine();
    while (line != null){
      System.out.println(line); 
      line=bufferReaderObject.readLine();
    }catch(Exception e){
  } 
}}

Hope this clear some of your doubts...