hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rudd Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14411) selecting Hive on Hbase table may cause FileNotFoundException
Date Wed, 03 Aug 2016 11:29:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15405760#comment-15405760
] 

Rudd Chen commented on HIVE-14411:
----------------------------------

  @SuppressWarnings("rawtypes")
  private static Path createDummyFileForEmptyPartition(Path path, JobConf job, MapWork work,
      Path hiveScratchDir, String alias, int sequenceNumber)
          throws Exception {

    String strPath = path.toString();

    // The input file does not exist, replace it by a empty file
    PartitionDesc partDesc = work.getPathToPartitionInfo().get(strPath);
    if (partDesc.getTableDesc().isNonNative()) {
      // if this isn't a hive table we can't create an empty file for it.
      return path;
    }

    Properties props = SerDeUtils.createOverlayedProperties(
        partDesc.getTableDesc().getProperties(), partDesc.getProperties());
    HiveOutputFormat outFileFormat = HiveFileFormatUtils.getHiveOutputFormat(job, partDesc);

    boolean oneRow = partDesc.getInputFileFormatClass() == OneNullRowInputFormat.class;

    Path newPath = createEmptyFile(hiveScratchDir, outFileFormat, job,

> selecting Hive on Hbase table may cause FileNotFoundException
> -------------------------------------------------------------
>
>                 Key: HIVE-14411
>                 URL: https://issues.apache.org/jira/browse/HIVE-14411
>             Project: Hive
>          Issue Type: Bug
>          Components: Physical Optimizer
>    Affects Versions: 1.3.0
>            Reporter: Rudd Chen
>
> 1. create a Hbase table hbase_table
> 2. create a external Hive table test_table mapping to the hbase table 
> example: 
> create 'hbase_t' ,{NAME=>'cf',COMPRESSION=>'snappy'},{NUMREGIONS=>15,SPLITALGO=>'HexStringSplit'}
> create external table hbase_t_hive(key1 string,cf_train string,cf_flight string,cf_wbsw
string,cf_wbxw string,cf_bgrz string,cf_bgtf string) 
> stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' 
> with serdeproperties("hbase.columns.mapping"=":key,cf:train,cf:flight,cf:wbsw,cf:wbxw,cf:bgrz,cf:bgtf")
tblproperties("hbase.table.name"="hbase_t");
> create table test3 as select * from hbase_t_hive where 1=2;
> ====
> if hive.optimize.null.scan=true, it will return an FileNotFoundException



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message