kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhou, Qianhao" <qianz...@ebay.com>
Subject Re: need help: Reg Dictionary
Date Wed, 18 Mar 2015 11:24:01 GMT
#1 Dictionary is mainly for storage saving
   Consider a column like ³country² , by using dictionary, we can encode
the country into only 1 byte (let¹s say there are no more than 256
countries in the world) while if not using dictionary ³People¹s republic
of China² will cost a lot more space to store.

#2. 
   Dictionary is more like a k-v map, k is a order-preserved id with same
size(currently, maybe in the future some other encoding like Huffman will
be introduced), and v is the actual value.
   By using dictionary, all value in the same column can have the same
size, thus the size of a record is also the same. This will make us easier
to decode and encode. (Maybe some other encoding like Huffman)
   Currently we use TRIE to build dictionary.

Best Regard
Zhou QianHao





On 3/18/15, 5:51 PM, "koteswara Vemu" <koti@infoworks.io> wrote:

>Hi,
>
>       What is the use of creating dimension dictionary? It will be a
>great
>help if you can explain how the data is stored in dictionary
>
>
>Regards,
>Koti


Mime
View raw message