Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3E79411935 for ; Tue, 23 Sep 2014 10:25:34 +0000 (UTC) Received: (qmail 34253 invoked by uid 500); 23 Sep 2014 10:25:33 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 34181 invoked by uid 500); 23 Sep 2014 10:25:33 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 34170 invoked by uid 500); 23 Sep 2014 10:25:33 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 34167 invoked by uid 99); 23 Sep 2014 10:25:33 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 23 Sep 2014 10:25:33 +0000 Date: Tue, 23 Sep 2014 10:25:33 +0000 (UTC) From: "Damien Carol (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HIVE-8231) Error in directory layer when insert into empty table with ACID MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-8231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damien Carol updated HIVE-8231: ------------------------------- Description: Steps to show the bug : 1. create table {code} create table encaissement_1b_64m like encaissement_1b; {code} 2. check table {code} desc encaissement_1b_64m; dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m; {code} everything is ok: {noformat} 0: jdbc:hive2://nc-h04:10000/casino> desc encaissement_1b_64m; +------------+------------+----------+--+ | col_name | data_type | comment | +------------+------------+----------+--+ | id | int | | | idmagasin | int | | | zibzin | string | | | cheque | int | | | montant | double | | | date | timestamp | | | col_6 | string | | | col_7 | string | | | col_8 | string | | +------------+------------+----------+--+ 9 rows selected (0.158 seconds) 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/; +-------------+--+ | DFS Output | +-------------+--+ +-------------+--+ No rows selected (0.01 seconds) {noformat} 3. Insert values into the new table {noformat} insert into table encaissement_1b_64m VALUES (1, 1, '800000000909000000000000', 1, 12.5, '12/05/2014', '','',''); {noformat} 4. Check {noformat} 0: jdbc:hive2://nc-h04:10000/casino> select id from encaissement_1b_64m; +-----+--+ | id | +-----+--+ +-----+--+ No rows selected (0.091 seconds) {noformat} There are already a pb. I don't see the inserted row. 5. When I'm checking HDFS directory, there are something strange, I see {{delta_0000421_0000421}} folder {noformat} 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/; +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+ | DFS Output | +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+ | Found 1 items | | drwxr-xr-x - hduser supergroup 0 2014-09-23 12:17 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/delta_0000421_0000421 | +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+ 2 rows selected (0.014 seconds) {noformat} 6. Doing a major compaction solves the bug {noformat} 0: jdbc:hive2://nc-h04:10000/casino> alter table encaissement_1b_64m compact 'major'; No rows affected (0.046 seconds) 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/; +--------------------------------------------------------------------------------------------------------------------------------------------+--+ | DFS Output | +--------------------------------------------------------------------------------------------------------------------------------------------+--+ | Found 1 items | | drwxr-xr-x - hduser supergroup 0 2014-09-23 12:21 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/base_0000421 | +--------------------------------------------------------------------------------------------------------------------------------------------+--+ 2 rows selected (0.02 seconds) {noformat} was: Steps to show the bug : 1. create table {code} create table encaissement_1b_64m like encaissement_1b; {code} 2. check table {code} desc encaissement_1b_64m; dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m; {code} everything is ok: {noformat} 0: jdbc:hive2://nc-h04:10000/casino> desc encaissement_1b_64m; +------------+------------+----------+--+ | col_name | data_type | comment | +------------+------------+----------+--+ | id | int | | | idmagasin | int | | | zibzin | string | | | cheque | int | | | montant | double | | | date | timestamp | | | col_6 | string | | | col_7 | string | | | col_8 | string | | +------------+------------+----------+--+ 9 rows selected (0.158 seconds) 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/; +-------------+--+ | DFS Output | +-------------+--+ +-------------+--+ No rows selected (0.01 seconds) {noformat} 3. Insert values into the new table {noformat} insert into table encaissement_1b_64m VALUES (1, 1, '800000000909000000000000', 1, 12.5, '12/05/2014', '','',''); {noformat} 4. Check {noformat} 0: jdbc:hive2://nc-h04:10000/casino> select id from encaissement_1b_64m; +-----+--+ | id | +-----+--+ +-----+--+ No rows selected (0.091 seconds) {noformat} There are already a pb. I don't see the inserted row. 5. When I'm checking HDFS directory, there are something strange, I see {{delta_0000421_0000421}} folder {noformat} 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/; +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+ | DFS Output | +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+ | Found 1 items | | drwxr-xr-x - hduser supergroup 0 2014-09-23 12:17 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/delta_0000421_0000421 | +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+ 2 rows selected (0.014 seconds) 6. Doing a major compaction solves the bug {noformat} 0: jdbc:hive2://nc-h04:10000/casino> alter table encaissement_1b_64m compact 'major'; No rows affected (0.046 seconds) 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/; +--------------------------------------------------------------------------------------------------------------------------------------------+--+ | DFS Output | +--------------------------------------------------------------------------------------------------------------------------------------------+--+ | Found 1 items | | drwxr-xr-x - hduser supergroup 0 2014-09-23 12:21 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/base_0000421 | +--------------------------------------------------------------------------------------------------------------------------------------------+--+ 2 rows selected (0.02 seconds) {noformat} > Error in directory layer when insert into empty table with ACID > --------------------------------------------------------------- > > Key: HIVE-8231 > URL: https://issues.apache.org/jira/browse/HIVE-8231 > Project: Hive > Issue Type: Bug > Affects Versions: 0.14.0 > Reporter: Damien Carol > > Steps to show the bug : > 1. create table > {code} > create table encaissement_1b_64m like encaissement_1b; > {code} > 2. check table > {code} > desc encaissement_1b_64m; > dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m; > {code} > everything is ok: > {noformat} > 0: jdbc:hive2://nc-h04:10000/casino> desc encaissement_1b_64m; +------------+------------+----------+--+ > | col_name | data_type | comment | > +------------+------------+----------+--+ > | id | int | | > | idmagasin | int | | > | zibzin | string | | > | cheque | int | | > | montant | double | | > | date | timestamp | | > | col_6 | string | | > | col_7 | string | | > | col_8 | string | | > +------------+------------+----------+--+ > 9 rows selected (0.158 seconds) > 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/; > +-------------+--+ > | DFS Output | > +-------------+--+ > +-------------+--+ > No rows selected (0.01 seconds) > {noformat} > 3. Insert values into the new table > {noformat} > insert into table encaissement_1b_64m VALUES (1, 1, '800000000909000000000000', 1, 12.5, '12/05/2014', '','',''); > {noformat} > 4. Check > {noformat} > 0: jdbc:hive2://nc-h04:10000/casino> select id from encaissement_1b_64m; > +-----+--+ > | id | > +-----+--+ > +-----+--+ > No rows selected (0.091 seconds) > {noformat} > There are already a pb. I don't see the inserted row. > 5. When I'm checking HDFS directory, there are something strange, I see {{delta_0000421_0000421}} folder > {noformat} > 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/; > +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+ > | DFS Output | > +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+ > | Found 1 items | > | drwxr-xr-x - hduser supergroup 0 2014-09-23 12:17 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/delta_0000421_0000421 | > +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+ > 2 rows selected (0.014 seconds) > {noformat} > 6. Doing a major compaction solves the bug > {noformat} > 0: jdbc:hive2://nc-h04:10000/casino> alter table encaissement_1b_64m compact 'major'; > No rows affected (0.046 seconds) > 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/; > +--------------------------------------------------------------------------------------------------------------------------------------------+--+ > | DFS Output | > +--------------------------------------------------------------------------------------------------------------------------------------------+--+ > | Found 1 items | > | drwxr-xr-x - hduser supergroup 0 2014-09-23 12:21 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/base_0000421 | > +--------------------------------------------------------------------------------------------------------------------------------------------+--+ > 2 rows selected (0.02 seconds) > {noformat} > -- This message was sent by Atlassian JIRA (v6.3.4#6332)