kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhou, Qianhao" <qianz...@ebay.com>
Subject Re: upgrading kylin's hadoop depedencies version
Date Wed, 14 Jan 2015 13:01:10 GMT
There are also some users who has reported the same issue.
However the list of supported hdp version is kind of out-of-date.

Best Regard
Zhou QianHao





On 1/14/15, 8:20 PM, "hongbin ma" <mahongbin@apache.org> wrote:

>hi, yang
>
>I tried our deploy script on hdp 2.2 (with current kylin dependencies in
>pom untouched), it turns out that MR job submission will fail. the log is
>attached at the end.
>I found that "mapreduce.application.framework.path" is something new in
>hdp
>2.2's mapred-site.xml. I guess the obsolete MR client might not be able to
>handle it properly.
>
>I suggest we upgrade the hadoop dependencies versions in our pom, and see
>if it will work out in both hdp 2.1 and 2.2.
>I remember you tried upgrading the versions recently, Did you checked them
>in?
>
>hongbin.
>
>=================================
>
>[QuartzScheduler_Worker-1]:[2015-01-14
>09:24:45,832][ERROR][com.kylinolap.job.hadoop.cube.FactDistinctColumnsJob.
>run(FactDistinctColumnsJob.java:83)]
>- Unable to parse
>'/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a
>URI, check the setting for mapreduce.application.framework.path
>
>java.lang.IllegalArgumentException: Unable to parse
>'/hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework' as a
>URI, check the setting for mapreduce.application.framework.path
>
>at
>org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(
>JobSubmitter.java:739)
>
>at
>org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.ja
>va:435)
>
>at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
>
>at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1293)
>
>at java.security.AccessController.doPrivileged(Native Method)
>
>at javax.security.auth.Subject.doAs(Subject.java:415)
>
>at
>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.
>java:1628)
>
>at org.apache.hadoop.mapreduce.Job.submit(Job.java:1293)
>
>at
>com.kylinolap.job.hadoop.AbstractHadoopJob.waitForCompletion(AbstractHadoo
>pJob.java:114)
>
>at
>com.kylinolap.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColum
>nsJob.java:79)
>
>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>
>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>
>at com.kylinolap.job.cmd.JavaHadoopCmd.execute(JavaHadoopCmd.java:53)
>
>at 
>com.kylinolap.job.flow.AsyncJobFlowNode.execute(AsyncJobFlowNode.java:70)
>
>at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
>
>at
>org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:5
>73)
>
>Caused by: java.net.URISyntaxException: Illegal character in path at index
>11: /hdp/apps/${hdp.version}/mapreduce/mapreduce.tar.gz#mr-framework
>
>at java.net.URI$Parser.fail(URI.java:2829)
>
>at java.net.URI$Parser.checkChars(URI.java:3002)
>
>at java.net.URI$Parser.parseHierarchical(URI.java:3086)
>
>at java.net.URI$Parser.parse(URI.java:3044)
>
>at java.net.URI.<init>(URI.java:595)
>
>at
>org.apache.hadoop.mapreduce.JobSubmitter.addMRFrameworkToDistributedCache(
>JobSubmitter.java:737)
>
>... 15 more


Mime
View raw message