kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: Error using GlobalDictionary
Date Mon, 05 Jun 2017 04:50:52 GMT
It looks like seeking the file in local disk, instead of HDFS. You need
check whether the proper core-site.xml (with hdfs as default file system)
is used by kylin. Or you can upgrade to kylin 2.0 to see whether it works.

2017-06-05 11:25 GMT+08:00 Sonny Heer <sonnyheer@gmail.com>:

> Any ideas on this?  Not sure ,but appears the dictionary is looking on
> local FS vs HDFS? ...what am i missing here?
>
> On Sat, Jun 3, 2017 at 9:05 PM, Sonny Heer <sonnyheer@gmail.com> wrote:
>
>> Kylin version 1.6.0
>>
>> Our data has High cardinality columns that require count distinct
>> measures.  Therefore using GlobalDictionary.  The .index file exists on
>> HDFS, but kylin errors out with FileNotFound exception (see below).  RowKey
>> is set to "dict".  Any ideas if this is a known issue or something
>> misconfigured...?
>>
>> At stage: "#7 Step Name: Build Base Cuboid Data"
>>
>> Error:
>>
>> Caused by: java.io.FileNotFoundException: File
>> /kylin/kylin_metadata/resources/GlobalDict/dict/MY_HIVE_DB.MY_HIVE_VIEW/MY_MEASURE/.index
>> does not exist
>>
>>         at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileSta
>> tus(RawLocalFileSystem.java:609)
>>
>>         at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInt
>> ernal(RawLocalFileSystem.java:822)
>>
>>         at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLoc
>> alFileSystem.java:599)
>>
>>         at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFi
>> leSystem.java:421)
>>
>>         at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputCheck
>> er.<init>(ChecksumFileSystem.java:140)
>>
>>         at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSys
>> tem.java:341)
>>
>>         at org.apache.kylin.dict.AppendTrieDictionary.readFields(Append
>> TrieDictionary.java:1076)
>>
>>         at org.apache.kylin.dict.DictionaryInfoSerializer.deserialize(D
>> ictionaryInfoSerializer.java:74)
>>
>>         at org.apache.kylin.dict.DictionaryInfoSerializer.deserialize(D
>> ictionaryInfoSerializer.java:34)
>>
>>         at org.apache.kylin.common.persistence.ResourceStore.getResourc
>> e(ResourceStore.java:146)
>>
>>         at org.apache.kylin.dict.DictionaryManager.load(DictionaryManag
>> er.java:421)
>>
>>         at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryMan
>> ager.java:103)
>>
>>         at org.apache.kylin.dict.DictionaryManager$1.load(DictionaryMan
>> ager.java:100)
>>
>>         at com.google.common.cache.LocalCache$LoadingValueReference.loa
>> dFuture(LocalCache.java:3568)
>>
>>         at com.google.common.cache.LocalCache$Segment.loadSync(LocalCac
>> he.java:2350)
>>
>>         at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(
>> LocalCache.java:2313)
>>
>>         at com.google.common.cache.LocalCache$Segment.get(LocalCache.
>> java:2228)
>>
>>         at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
>>
>>         at com.google.common.cache.LocalCache.getOrLoad(LocalCache.
>> java:3969)
>>
>>         at com.google.common.cache.LocalCache$LocalManualCache.get(
>> LocalCache.java:4829)
>>
>>         at org.apache.kylin.dict.DictionaryManager.getDictionaryInfo(Di
>> ctionaryManager.java:120)
>>
>>         ... 17 more
>> --
>>
>>
>>
>
>
> --
>
>
> Pushpinder S. Heer
> Senior Software Engineer
> m: 360-434-4354 h: 509-884-2574
>



-- 
Best regards,

Shaofeng Shi 史少锋

Mime
View raw message