Mailing-List: contact dev-help@kylin.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@kylin.incubator.apache.org
Date: Fri, 8 May 2015 08:04:19 +0000 (UTC)
From: Abhijit Deka <abhijit.deka@rocketmail.com>
Reply-To: Abhijit Deka <abhijit.deka@rocketmail.com>
To: hongbin ma <mahongbin@apache.org>
Cc: "Shi, Shaofeng" <shaoshi@ebay.com>,
	"dev@kylin.incubator.apache.org" <dev@kylin.incubator.apache.org>
Message-ID: <274125455.1635682.1431072259733.JavaMail.yahoo@mail.yahoo.com>
In-Reply-To: 
 <CAF7etTk-7KigySY5jqk91yDaFTTLJe9qpzUnzbpVh79i5ATuvg@mail.gmail.com>
References: 
 <CAF7etTk-7KigySY5jqk91yDaFTTLJe9qpzUnzbpVh79i5ATuvg@mail.gmail.com>
Subject: Re: Cube build failed
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_1635681_142823984.1431072259714"

------=_Part_1635681_142823984.1431072259714
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Thanks.I'll try with the latest=C2=A0Regards
Abhijit DekaComputer ScientistAdobe SystemsBangalore
Ph-+91 80884 39067=20


     On Friday, 8 May 2015 1:26 PM, hongbin ma <mahongbin@apache.org> wrote=
:
  =20

 to be honest we barely had the bandwidth to test on chd,i would recommend =
the latest cdh
On Fri, May 8, 2015 at 3:54 PM, Abhijit Deka <abhijit.deka@rocketmail.com> =
wrote:

Please let me know which version of CHD should I download.(tested with all =
the Kylin components)=C2=A0Regards
Abhijit DekaComputer ScientistAdobe SystemsBangalore
Ph-+91 80884 39067=20


     On Friday, 8 May 2015 12:33 PM, hongbin ma <mahongbin@apache.org> wrot=
e:
  =20

 hi,cdh 5.1 is using hive 0.12, however we require 0.14 at leastto save all=
 the troubles, you can change to hdp 2.2 if possible
On Fri, May 8, 2015 at 2:52 PM, Abhijit Deka <abhijit.deka@rocketmail.com> =
wrote:

Hi,
That issue is resolved when i moved from External table.
but I am getitng a new error now .In the same build step I am getting this
java.lang.IncompatibleClassChangeError: Found class org.apache.hive.hcatalo=
g.data.transfer.ReaderContext, but interface was expected
=09at org.apache.kylin.dict.lookup.HiveTableReader.initialize(HiveTableRead=
er.java:85)
=09at org.apache.kylin.dict.lookup.HiveTableReader.<init>(HiveTableReader.j=
ava:74)
=09at org.apache.kylin.dict.lookup.HiveTableReader.<init>(HiveTableReader.j=
ava:60)
=09at org.apache.kylin.dict.lookup.HiveTable.getReader(HiveTable.java:66)
=09at org.apache.kylin.dict.DictionaryGenerator.loadColumnValues(Dictionary=
Generator.java:173)
=09at org.apache.kylin.dict.DictionaryGenerator.buildDictionary(DictionaryG=
enerator.java:112)
=09at org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryMan=
ager.java:172)
=09at org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:15=
4)
=09at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(Dicti=
onaryGeneratorCLI.java:53)
=09at org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(Dicti=
onaryGeneratorCLI.java:42)
=09at org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictio=
naryJob.java:53)
=09at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
=09at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
=09at org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellE=
xecutable.java:63)
=09at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExe=
cutable.java:107)
=09at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(Defaul=
tChainedExecutable.java:50)
=09at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExe=
cutable.java:107)
=09at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(D=
efaultScheduler.java:132)
=09at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.=
java:1145)
=09at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor=
.java:615)
=09at java.lang.Thread.run(Thread.java:745)Do you know what is causing this=
 error?=20

Regards
Abhijit DekaComputer ScientistAdobe SystemsBangalore
Ph-+91 80884 39067=20


     On Friday, 8 May 2015 11:09 AM, hongbin ma <mahongbin@apache.org> wrot=
e:
  =20

 what will you get if you run=C2=A0hadoop fs -cat=C2=A0/user/hive/warehouse=
/glaastest_dim_vendor ?
On Fri, May 8, 2015 at 1:37 PM, Abhijit Deka <abhijit.deka@rocketmail.com> =
wrote:

Hi Shi,
It's Hive.=C2=A0Regards
Abhijit DekaComputer ScientistAdobe SystemsBangalore
Ph-+91 80884 39067=20


     On Friday, 8 May 2015 10:09 AM, "Shi, Shaofeng" <shaoshi@ebay.com> wro=
te:
  =20

 What=E2=80=99s the owner info for
hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_vendor ?
Didn=E2=80=99t see it here.

On 5/8/15, 12:35 PM, "Abhijit Deka" <abhijit.deka@rocketmail.com> wrote:

>Hi Bin,
>Thanks for the reply.I have already mentioned that I have verified and
>the file exists in HDFS.But the owner is Hive not Cloudera.
>drwxrwxrwx=C2=A0 - hive=C2=A0 =C2=A0 hive=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 0 2015-05-07 04:22
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_product
>drwxrwxrwx=C2=A0 - hive=C2=A0 =C2=A0 hive=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 0 2015-05-07 04:20
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_fact
>drwxrwxrwx=C2=A0 - cloudera supergroup=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=
 2015-05-04 07:18
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_cal_dt
>drwxrwxrwx=C2=A0 - cloudera supergroup=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=
 2015-05-04 07:18
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_category_groupin
>gs
>drwxrwxrwx=C2=A0 - cloudera supergroup=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=
 2015-05-04 07:18
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_sales
>
>For the example bundled with Kylin the user is Cloudera. Is it some kind
>of permission issue I am running into.Meanwhile i'll try to run it with
>hive table (not external)
>
> Regards
>Abhijit DekaComputer ScientistAdobe SystemsBangalore
>Ph-+91 80884 39067
>
>
>=C2=A0 =C2=A0 On Friday, 8 May 2015 6:59 AM, hongbin ma <mahongbin@apache.=
org>
>wrote:
>=C2=A0=20
>
> for all the lookup tables, for example glaastest_dim_vendor=C2=A0 in your
>case, kylin asks hive the hdfs location of it, and will try to read the
>hdfs file directly.
>it seems hive returned the location:
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_vendor
>does , but the file does not exist. can you verify why hive returned such
>location(as you described it's a external table in hbase?)
>An alternative is that you convert all the lookup tables into
>non-external tables
>On Thu, May 7, 2015 at 4:40 PM, Abhijit Deka
><abhijit.deka@rocketmail.com> wrote:
>
>Forgot to add the machine info.I am running it in CHD 5.1.0
>virtualbox.Not a cluster.
> Regards
>Abhijit DekaComputer ScientistAdobe SystemsBangalore
>Ph-+91 80884 39067
>
>
>=C2=A0 =C2=A0 On Thursday, 7 May 2015 1:36 AM, Abhijit Deka
><abhijit.deka@rocketmail.com> wrote:
>
>
> Hi,
>I am new to Kylin and i was trying out a simple cube.I did the below
>steps.Please let me know if i have done anything wrong.
>1.Done ETL in Pig.2.Stored the Fact and Dim tables in Hbase.3.Created
>external table in Hive to query the data and also use in Kylin4.Created
>the cube as instructed in documentation.5.Build the cube.
>But the cube build failed in 3rd step.
>Below are the logs
>java.io.FileNotFoundException: File
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_vendor
>does not exist.
>=C2=A0 =C2=A0 at=20
>org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(Distribute
>dFileSystem.java:654)
>=C2=A0 =C2=A0 at=20
>org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSys
>tem.java:102)
>=C2=A0 =C2=A0 at=20
>org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSyst
>em.java:712)
>=C2=A0 =C2=A0 at=20
>org.apache.hadoop.hdfs.DistributedFileSystem$14.doCall(DistributedFileSyst
>em.java:708)
>=C2=A0 =C2=A0 at=20
>org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver
>.java:81)
>=C2=A0 =C2=A0 at=20
>org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSys
>tem.java:708)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.dict.lookup.HiveTable.findOnlyFile(HiveTable.java:116)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.dict.lookup.HiveTable.computeHDFSLocation(HiveTable.java:
>107)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.dict.lookup.HiveTable.getHDFSLocation(HiveTable.java:83)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.dict.lookup.HiveTable.getFileTable(HiveTable.java:76)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.dict.lookup.HiveTable.getSignature(HiveTable.java:71)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.dict.DictionaryManager.buildDictionary(DictionaryManager.
>java:164)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.cube.CubeManager.buildDictionary(CubeManager.java:154)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(Dictionary
>GeneratorCLI.java:53)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.cube.cli.DictionaryGeneratorCLI.processSegment(Dictionary
>GeneratorCLI.java:42)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.job.hadoop.dict.CreateDictionaryJob.run(CreateDictionaryJ
>ob.java:53)
>=C2=A0 =C2=A0 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>=C2=A0 =C2=A0 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.job.common.HadoopShellExecutable.doWork(HadoopShellExecut
>able.java:63)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutab
>le.java:107)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChai
>nedExecutable.java:50)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutab
>le.java:107)
>=C2=A0 =C2=A0 at=20
>org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(Defaul
>tScheduler.java:132)
>=C2=A0 =C2=A0 at=20
>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>1145)
>=C2=A0 =C2=A0 at=20
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:615)
>=C2=A0 =C2=A0 at java.lang.Thread.run(Thread.java:745)
>Now when I check for the file in HDFS the file is there but it looks like
>a permission issue.(??)
>drwxrwxrwx=C2=A0 - hive=C2=A0 =C2=A0 hive=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 0 2015-05-07 04:22
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_dim_product
>drwxrwxrwx=C2=A0 - hive=C2=A0 =C2=A0 hive=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 0 2015-05-07 04:20
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/glaastest_fact
>drwxrwxrwx=C2=A0 - cloudera supergroup=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=
 2015-05-04 07:18
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_cal_dt
>drwxrwxrwx=C2=A0 - cloudera supergroup=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=
 2015-05-04 07:18
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_category_groupin
>gs
>drwxrwxrwx=C2=A0 - cloudera supergroup=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0=
 2015-05-04 07:18
>hdfs://quickstart.cloudera:8020/user/hive/warehouse/kylin_sales
>How do i solve this issue.Thanks in advance.
> Regards
>Abhijit DekaComputer ScientistAdobe SystemsBangalore
>
>
>=C2=A0=20
>
>
>
>--=20
>Regards,
>Bin Mahone | =E9=A9=AC=E6=B4=AA=E5=AE=BE
>Apache Kylin: http://kylin.io
>Github: https://github.com/binmahone
>
>=C2=A0=20


  =20


--=20
Regards,
Bin Mahone | =E9=A9=AC=E6=B4=AA=E5=AE=BE
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone=20

  =20


--=20
Regards,
Bin Mahone | =E9=A9=AC=E6=B4=AA=E5=AE=BE
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone=20

  =20


--=20
Regards,
Bin Mahone | =E9=A9=AC=E6=B4=AA=E5=AE=BE
Apache Kylin: http://kylin.io
Github: https://github.com/binmahone=20

  
------=_Part_1635681_142823984.1431072259714--