Hadoop Rack Awareness

Hadoop Rack Awareness

In core-site.xml, add the below contents:
<property>
 <name>topology.script.file.name</name>
 <value>conf/rack-awareness.sh</value>
 </property>
 <property>
 <name>topology.script.number.args</name>
 <value>1000</value>
</property>
<property>
      <name>topology.node.switch.mapping.impl</name>
      <value>org.apache.hadoop.net.ScriptBasedMapping</value>
      <description> The default implementation of the DNSToSwitchMapping. It
            invokes a script specified in topology.script.file.name to resolve
            node names. If the value for topology.script.file.name is not set, the
            default value of DEFAULT_RACK is returned for all node names.
      </description>
 </property>




create a file in $HADOOP_HOME/conf/rack-awareness.sh with the below content:

 #!/bin/sh

 HADOOP_CONF=/opt/hadoop-1.0.3/conf

 while [ $# -gt 0 ] ; do
   nodeArg=$1
   exec< ${HADOOP_CONF}/topology.data
   result=""
   while read line ; do
     ar=( $line )
     if [ "${ar[0]}" = "$nodeArg" ] ; then
       result="${ar[1]}"
     fi
   done
   shift
   if [ -z "$result" ] ; then
     echo -n "/default-rack "
   else
     echo -n "$result "
   fi
 done


create a file in /opt/hadoop-1.0.3/conf/topology.data：

dn1 rackA
dn2 rackA
dn3 rackB
dn4 rackB