kylin-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From roger shi <rogershijich...@hotmail.com>
Subject Re: 回复: 使用全局字典报错AppendTrieDictionary can't retrive value from id
Date Tue, 06 Dec 2016 02:34:39 GMT
Hi wangdan,

Would you please attach the cube desc json? The cube definition in previous email is cube
instance json.

Thanks,
Roger

On 06/12/2016, 10:20 AM, "wangdan@snqu.com" <wangdan@snqu.com> wrote:

    查看了kylin.properties文件中kylin.dictionary.max.cardinality=5000000修改为kylin.dictionary.max.cardinality=20000000
同时修改cube, 添加rowkey,重新编译成功
    但是查询时以下的两个语句可以成功
    select label, count(label) from USERCASE_20161204 group by label order by label desc
    select name, count(name) from USERCASE_20161204 group by name order by name desc
    
    以下的两个语句
    select rowkey, count(rowkey) from USERCASE_20161204 group by rowkey order by rowkey desc
    select job, count(job) from USERCASE_20161204 group by job order by job des
    执行后报错:
    Error while executing SQL "select rowkey, count(rowkey) from USERCASE_20161204 group by
rowkey order by rowkey desc LIMIT 50000": AppendTrieDictionary can't retrive value from id
    Error while executing SQL "select job, count(job) from USERCASE_20161204 group by job
order by job desc LIMIT 50000": AppendTrieDictionary can't retrive value from id
    
    cube的定义如下:
    {
      "uuid": "d4671695-96a1-4981-bb4c-2263de45f2ee",
      "last_modified": 1480939514479,
      "version": "1.6.0",
      "name": "dmp_cube_590w",
      "owner": "ADMIN",
      "descriptor": "dmp_cube_590w",
      "cost": 50,
      "status": "READY",
      "segments": [
        {
          "uuid": "4237b09b-8d2e-4c5c-be19-afc67e6524f5",
          "name": "19700101000000_20161205000000",
          "storage_location_identifier": "KYLIN_IE2V4DQUY4",
          "date_range_start": 0,
          "date_range_end": 1480896000000,
          "source_offset_start": 0,
          "source_offset_end": 0,
          "status": "READY",
          "size_kb": 8180652,
          "input_records": 5978388,
          "input_records_size": 666419108,
          "last_build_time": 1480939514367,
          "last_build_job_id": "d563a6b8-c6cd-41c7-93c4-47bb319bf21b",
          "create_time_utc": 1480931400144,
          "cuboid_shard_nums": {
            "1": 2,
            "2": 2,
            "3": 3,
            "4": 2,
            "5": 3,
            "6": 3,
            "7": 4,
            "8": 2,
            "9": 3,
            "10": 3,
            "11": 4,
            "12": 3,
            "13": 4,
            "14": 4,
            "15": 5,
            "32": 2,
            "33": 3,
            "34": 3,
            "35": 4,
            "36": 3,
            "37": 4,
            "38": 4,
            "39": 5,
            "40": 3,
            "41": 4,
            "42": 4,
            "43": 5,
            "44": 4,
            "45": 5,
            "46": 5,
            "47": 6,
            "64": 6,
            "65": 6,
            "66": 6,
            "67": 6,
            "68": 6,
            "69": 6,
            "70": 6,
            "71": 6,
            "72": 6,
            "73": 6,
            "74": 6,
            "75": 6,
            "76": 6,
            "77": 6,
            "78": 6,
            "79": 6,
            "96": 6,
            "97": 6,
            "98": 6,
            "99": 6,
            "100": 6,
            "101": 6,
            "102": 6,
            "103": 6,
            "104": 6,
            "105": 6,
            "106": 6,
            "107": 6,
            "108": 6,
            "109": 6,
            "110": 6,
            "111": 6,
            "127": 6
          },
          "total_shards": 11,
          "blackout_cuboids": [],
          "binary_signature": null,
          "dictionaries": {
            "DEFAULT.USERCASE_20161204/SEX": "/dict/DEFAULT.USERCASE_20161204/SEX/17d36c0b-e7a7-4bb4-941f-47bc78a24751.dict",
            "DEFAULT.USERCASE_20161204/TIMESTAMP": "/dict/DEFAULT.USERCASE_20161204/TIMESTAMP/87ce791b-3de3-491f-901f-d28721a25e94.dict",
            "DEFAULT.USERCASE_20161204/NAME": "/dict/DEFAULT.USERCASE_20161204/NAME/73a59cfb-eaa5-4531-ba7e-16ba2adeaea9.dict",
            "DEFAULT.USERCASE_20161204/LABEL": "/dict/DEFAULT.USERCASE_20161204/LABEL/71c633ee-dffb-4d80-9844-768b6ee21782.dict",
            "DEFAULT.USERCASE_20161204/LOCAL": "/dict/DEFAULT.USERCASE_20161204/LOCAL/31ed5b68-aae2-40b7-ba09-83abf1d64953.dict",
            "DEFAULT.USERCASE_20161204/ROWKEY": "/dict/DEFAULT.USERCASE_20161204/ROWKEY/736822fd-5103-4814-bfcd-b6af80609970.dict",
            "DEFAULT.USERCASE_20161204/JOB": "/dict/DEFAULT.USERCASE_20161204/JOB/a47cc0f8-80ab-46fa-953a-59a326412395.dict"
          },
          "snapshots": null,
          "index_path": "/kylin/kylin_metadata/kylin-d563a6b8-c6cd-41c7-93c4-47bb319bf21b/dmp_cube_590w/secondary_index/",
          "rowkey_stats": [
            [
              "ROWKEY",
              5978389,
              4
            ],
            [
              "NAME",
              1195682,
              3
            ],
            [
              "TIMESTAMP",
              1,
              1
            ],
            [
              "SEX",
              1195680,
              4
            ],
            [
              "LOCAL",
              1195679,
              4
            ],
            [
              "JOB",
              1195679,
              4
            ],
            [
              "LABEL",
              1195676,
              3
            ]
          ]
        }
      ],
      "create_time_utc": 1480907805715,
      "size_kb": 8180652,
      "input_records_count": 5978388,
      "input_records_size": 666419108
    }
     
    发件人: wangdan@snqu.com
    发送时间: 2016-12-05 15:11
    收件人: dev
    主题: 使用全局字典报错AppendTrieDictionary can't retrive value from id
    hi, 
     每个维度的基数有590万,当在rowkey中选择dict时,编译产生错误:
    “Too high cardinality is not suitable for dictionary -- cardinality: 5978388“
    
    所以修改了model, 没有定义rowkey, 对所有维度定义了全局字典,build成功,查询时报错:

    “AppendTrieDictionary can't retrive value from id“
    
    
    

Mime
View raw message