hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-17215) Streaming Ingest API writing unbucketed tables
Date Thu, 24 Aug 2017 23:20:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eugene Koifman resolved HIVE-17215.
-----------------------------------
       Resolution: Fixed
    Fix Version/s: 3.0.0

fix included in HIVE-17205

> Streaming Ingest API writing unbucketed tables
> ----------------------------------------------
>
>                 Key: HIVE-17215
>                 URL: https://issues.apache.org/jira/browse/HIVE-17215
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Transactions
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>             Fix For: 3.0.0
>
>
> Currently the API expects the target table to be bucketed.
> It creates 1 writer per bucket per connection/partition.
> The simplest is to allow the API to create a single writer for unbucketed tables.  
> If this doesn't provide enough write throughput, the client can create another connection.
> Could add a parameter to the API to specify writer parallelism for unbucketed tables.
 If it's set to 2 for example, the writer will write delta_x_y_0000 and delta_x_y_00001 using
statementId.  Maybe as a followup.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message