by Rahul

Categories

  • Big Data
  • hbase
  • java

Tags

  • hadoop
  • hbase
  • java

Lanuch Hbase Mapreduce Job
Add all the required jars to a variable separated by(,)
like jar=a,b
in my case I want to supply all hbase jars to job

jar=/home/hadoop/Desktop/techsquids/repos/techsquids/map-reduce/target/map-reduce-1.0-SNAPSHOT.jar classpath=$jar for f in $HBASE\_HOME/lib/\*.jar;do classpath="${classpath},$f" done

Next prepare the Hadoop classpath from previously formed jar files and jars separated by colon(:).

path=`echo $classpath | sed 's/,/:/g'`

Export LIBJARS & HADOOP_CLASSPATH\

export LIBJARS=`hadoop classpath`:$classpath export HADOOP\_CLASSPATH=$path

Run Hadoop job

hadoop jar $jar com.ts.mapreduce.hbase.ColumnsCounter -libjars ${classpath} "$@"

if you get ClassNotFountException in java then register your jars

HBaseConfiguration.addHbaseResources(job.getConfiguration()); TableMapReduceUtil.addDependencyJars(job); TableMapReduceUtil.addDependencyJars(job.getConfiguration(), org.apache.hadoop.hbase.client.Put.class); job.setJarByClass(ColumnsCounter.class);

Complete source at git(techsquids)