top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

NFS HDFS gateway with kerberos

+2 votes
612 views

Does anyone know if the NFS HDFS gateway is currently supported on secure clusters using Kerberos for Hadoop 2.2.0? We are using HDP 2.0 and looking to use NFS gateway

posted Jan 31, 2014 by Jai Prakash

Share this question
Facebook Share Button Twitter Share Button LinkedIn Share Button

1 Answer

+1 vote

HDFS-5804 (https://issues.apache.org/jira/browse/HDFS-5804 ) allows the NFS gateway to communicate with a secured Hadoop cluster using Kerberos. HDFS-5086 will add RPCSEC_GSS authentication support to NFS gateway.

answer Jan 31, 2014 by Seema Siddique
Similar Questions
+2 votes
Error while writing to HDFS NFS from windows

I have a use case where I need to transfer files from windows to HDFS. I cannot use flume because the folder and file structure need to be replicated as they are. HDFS NFS is the perfect thing for this use case, I was able to mount hdfs on windows, but when I try to write to it I get the following error.

ERROR nfs3.RpcProgramNfs3: Setting file size is not supported when creating file: 
New Text Document.txt dir fileId:463160

Is there any way to resolve this, so that I can successfully write from windows to HDFS NFS?

0 votes
Unable to use ./hdfs dfsadmin -report with HDFS Federation

Our hadoop cluster is using HDFS Federation, but when use the following command to report the HDFS status

$ ./hdfs dfsadmin -reportreport: FileSystem viewfs://nsX/ is not an HDFS file system
Usage: hdfs dfsadmin [-report] [-live] [-dead] [-decommissioning]

It gives me the following message that viewfs is NOT HDFS filesystem. Then how can I proceed to report the hdfs status

+2 votes
Tips for optimizing HDFS writes with replication=1?

I am writing temp files to HDFS with replication=1, so I expect the blocks to be stored on the writing node. Are there any tips, in general, for optimizing write performance to HDFS? I use 128K buffers in the write() calls. Are there any parameters that can be set on the connection or in HDFS configuration to optimize this use pattern?

0 votes
How to get info about which data in hdfs or file system that a MapReduce job visits?

I was trying to implement a Hadoop/Spark audit tool, but l met a problem that I can't get the input file location and file name. I can get username, IP address, time, user command, all of these info from hdfs-audit.log. But When I submit a MapReduce job, I can't see input file location neither in Hadoop logs or Hadoop ResourceManager.

Does hadoop have API or log that contains these info through some configuration ?If it have, what should I configure?

...