kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "李巍" <lwd...@fhzz.com.cn>
Subject Re: How to bulid cube with long string in fact table
Date Tue, 14 Nov 2017 13:22:19 GMT
Hi Kaige,
Thanks for reply!
 
However, I add the columns as measures(RAW) instead of dimensions in order to show the images
with a given ID number(images are not querying condition), so it doesn't involve dimension
encoding issues.  
 
In this situation, should I still changed base64? Or there are other solutions?
 
------------------ Original ------------------
From:  "Kaige Liu"<kaige.liu@kyligence.io>;
Date:  Tue, Nov 14, 2017 03:57 PM
To:  "user@kylin.apache.org"<user@kylin.apache.org>; 

Subject:  Re: How to bulid cube with long string in fact table

 
  
Hi Wei,
 
 
 
It’s not recommended to add such columns to Cube dimensions. You choose “Dict” encoding
for the dimension, but the base64 value of image data is too long to encode. 
 
Do you have requirements about querying those images? If not, I suggest you remove the image
column from the cube. If you only use images as a filter, maybe transform the base64 value
to  MD5(or other short digest) could be another thought.  Thanks.
 
 
  
Best regards,
 
 
 
Kaige Liu(刘凯歌)
 
Mail:  kaige.liu@kyligence.io
 
Shanghai Kyligence Information Technology Co., Ltd
 
上海市浦东新区亮秀路112号Y1座405室
 
 
 
"Do small things with great love." 
 
 
 
 
 
  
From:  李巍 <lwdata@fhzz.com.cn>
 Reply-To: "user@kylin.apache.org" <user@kylin.apache.org>
 Date: Tuesday, November 14, 2017 at 12:33 PM
 To: user <user@kylin.apache.org>
 Subject: How to bulid cube with long string in fact table
 
  
 
 
    
Hi all:
 
  
I have a fact table include image data which is encoded to base64, so that the column contains
long string. 
 
  
When I builded this cube, a error happened in step 3 (Extract Fact Table Distinct Columns),
the log showed 'Job Counters Failed reduce tasks=4', details as the follow picture.
 
  

 
  
Could Kylin deal with this situation?
 
  
 
 
  
PS: my data form like this
Mime
View raw message