Categories

  • Big Data
  • hive
  • java

Tags

  • bigdata
  • hive
  • java

Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error-

add jar /xxx/hive-customserde.jar; add jar /xxx/solr-solrj.jar;

Above commands will add resource to hive class path but suppose your custom library has a dependency on other jar then you need to do some work around to resolve class path issue. My custom serde is dependent on Solr libraries and jar file is already added to class path but still it is not getting reflected to class path.
Exception trace-

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/solr/client/solrj/SolrServerException at com.aexp.ims.atworks.hive.solr.SolrSplit.getSplits(SolrSplit.java:84) at com.aexp.ims.atworks.hive.solr.SolrInputFormat.getSplits(SolrInputFormat.java:91) at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:419) at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:573) at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:546) at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138) at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1595) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:285) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)

Solution is you need to add dependent libraries to Hive auxiliary path. Use below command to add libraries to hive aux path-

hive --auxpath /xxx/solr-solrj-5.2.1.jar, /xxx/httpclient-4.5.1.jar

If you still face any class path issue then post your queries in comment :)