Best practice of migrating hadoop 1.0.1 to hadoop 2.2.3

Question

We plan to migrate a 30 nodes hadoop 1.0.1 cluster to the version 2.3.0. We dont have extra machines to setup a separate new cluster, thus hope to do an in-place migration by replacing the components on the existing computers. So the questions are:

1) Is it possible to do an in-place migration, while keeping all data in HDFS safely?
2) If it is yes, is there any doc/guidance to do this?
3) Is the 2.0.3 MR API binary compatible with the one of 1.0.1?

Answer 1

1) Is it possible to do an in-place migration, while keeping all data in HDFS safely?
yes. stop the HDFS firstly, then run "start-dfs.sh -upgrade"

2) If it is yes, is there any doc/guidance to do this?
you just want a HDFS upgrade, so I dont think there are some useful doc.

3) Is the 2.0.3 MR API binary compatible with the one of 1.0.1?
Not much compatible of the FileSystem API. and there are some new HDFS configurations and some deprecates.

Answer 2

Refer to the following links for reference these should be helpful:

http://www.michael-noll.com/blog/2011/08/23/performing-an-hdfs-upgrade-of-an-hadoop-cluster/
http://wiki.apache.org/hadoop/Hadoop_Upgrade

Hadoop version used in the doc may be different from yours, but they are good references to understand the basic flow. I would suggest create a test cluster that could mimic your production environment, try it out on the test cluster before on the production and backup your namenode meta data, which may help you to recover.