hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao (JIRA)" <>
Subject [jira] [Created] (HIVE-10209) FetchTask with VC may fail because of ExecMapper.done is true
Date Fri, 03 Apr 2015 05:59:52 GMT
Chao created HIVE-10209:

             Summary: FetchTask with VC may fail because of ExecMapper.done is true
                 Key: HIVE-10209
             Project: Hive
          Issue Type: Bug
          Components: Query Processor
    Affects Versions: 1.1.0
            Reporter: Chao
            Assignee: Chao

ExecMapper.done is a static variable, and may cause issues in the following example:

set hive.fetch.task.conversion=minimal;
select * from src where key < 10 limit 1;
set hive.fetch.task.conversion=more;
select *, BLOCK__OFFSET_INSIDE__FILE from src where key < 10;

The second select won't return any result.

The issue is, the first select query will be converted to a MapRedTask with only a mapper.
And, when the task is done, because of the limit operator, ExecMapper.done will be set to

Then, when the second select query begin to execute, it will call {{FetchOperator::getRecordReader()}},
and since here we have virtual column, an instance of {{HiveRecordReader}} will be returned.
The problem is, {{HiveRecordReader::doNext()}} will check ExecMapper.done. In this case, since
the value is true, it will quit immediately.

In short, I think making ExecMapper.done static is a bad idea. The first query should in no
way affect the second one.

This message was sent by Atlassian JIRA

View raw message