hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Damien Carol (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-8231) Error in directory layer when insert into empty table with ACID
Date Tue, 23 Sep 2014 10:25:33 GMT

     [ https://issues.apache.org/jira/browse/HIVE-8231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Damien Carol updated HIVE-8231:
-------------------------------
    Description: 
Steps to show the bug :
1. create table 
{code}
create table encaissement_1b_64m like encaissement_1b;
{code}
2. check table 
{code}
desc encaissement_1b_64m;
dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m;
{code}
everything is ok:
{noformat}
0: jdbc:hive2://nc-h04:10000/casino> desc encaissement_1b_64m;                        
                                                                                     +------------+------------+----------+--+
|  col_name  | data_type  | comment  |
+------------+------------+----------+--+
| id         | int        |          |
| idmagasin  | int        |          |
| zibzin     | string     |          |
| cheque     | int        |          |
| montant    | double     |          |
| date       | timestamp  |          |
| col_6      | string     |          |
| col_7      | string     |          |
| col_8      | string     |          |
+------------+------------+----------+--+
9 rows selected (0.158 seconds)
0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/;
+-------------+--+
| DFS Output  |
+-------------+--+
+-------------+--+
No rows selected (0.01 seconds)

{noformat}

3. Insert values into the new table
{noformat}
insert into table encaissement_1b_64m VALUES (1, 1, '800000000909000000000000', 1, 12.5, '12/05/2014',
'','','');
{noformat}

4. Check
{noformat}
0: jdbc:hive2://nc-h04:10000/casino> select id from encaissement_1b_64m;
+-----+--+
| id  |
+-----+--+
+-----+--+
No rows selected (0.091 seconds)
{noformat}
There are already a pb. I don't see the inserted row.

5. When I'm checking HDFS directory, there are something strange, I see {{delta_0000421_0000421}}
folder
{noformat}
0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/;
+-----------------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                     DFS Output         
                                                            |
+-----------------------------------------------------------------------------------------------------------------------------------------------------+--+
| Found 1 items                                                                          
                                                            |
| drwxr-xr-x   - hduser supergroup          0 2014-09-23 12:17 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/delta_0000421_0000421
 |
+-----------------------------------------------------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.014 seconds)
{noformat}

6. Doing a major compaction solves the bug
{noformat}
0: jdbc:hive2://nc-h04:10000/casino> alter table encaissement_1b_64m compact 'major';
No rows affected (0.046 seconds)
0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/;
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                 DFS Output             
                                                   |
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
| Found 1 items                                                                          
                                                   |
| drwxr-xr-x   - hduser supergroup          0 2014-09-23 12:21 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/base_0000421
 |
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.02 seconds)
{noformat}
 


  was:
Steps to show the bug :
1. create table 
{code}
create table encaissement_1b_64m like encaissement_1b;
{code}
2. check table 
{code}
desc encaissement_1b_64m;
dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m;
{code}
everything is ok:
{noformat}
0: jdbc:hive2://nc-h04:10000/casino> desc encaissement_1b_64m;                        
                                                                                     +------------+------------+----------+--+
|  col_name  | data_type  | comment  |
+------------+------------+----------+--+
| id         | int        |          |
| idmagasin  | int        |          |
| zibzin     | string     |          |
| cheque     | int        |          |
| montant    | double     |          |
| date       | timestamp  |          |
| col_6      | string     |          |
| col_7      | string     |          |
| col_8      | string     |          |
+------------+------------+----------+--+
9 rows selected (0.158 seconds)
0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/;
+-------------+--+
| DFS Output  |
+-------------+--+
+-------------+--+
No rows selected (0.01 seconds)

{noformat}

3. Insert values into the new table
{noformat}
insert into table encaissement_1b_64m VALUES (1, 1, '800000000909000000000000', 1, 12.5, '12/05/2014',
'','','');
{noformat}

4. Check
{noformat}
0: jdbc:hive2://nc-h04:10000/casino> select id from encaissement_1b_64m;
+-----+--+
| id  |
+-----+--+
+-----+--+
No rows selected (0.091 seconds)
{noformat}
There are already a pb. I don't see the inserted row.

5. When I'm checking HDFS directory, there are something strange, I see {{delta_0000421_0000421}}
folder
{noformat}
0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/;
+-----------------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                     DFS Output         
                                                            |
+-----------------------------------------------------------------------------------------------------------------------------------------------------+--+
| Found 1 items                                                                          
                                                            |
| drwxr-xr-x   - hduser supergroup          0 2014-09-23 12:17 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/delta_0000421_0000421
 |
+-----------------------------------------------------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.014 seconds)


6. Doing a major compaction solves the bug
{noformat}
0: jdbc:hive2://nc-h04:10000/casino> alter table encaissement_1b_64m compact 'major';
No rows affected (0.046 seconds)
0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/;
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                 DFS Output             
                                                   |
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
| Found 1 items                                                                          
                                                   |
| drwxr-xr-x   - hduser supergroup          0 2014-09-23 12:21 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/base_0000421
 |
+--------------------------------------------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.02 seconds)
{noformat}
 



> Error in directory layer when insert into empty table with ACID
> ---------------------------------------------------------------
>
>                 Key: HIVE-8231
>                 URL: https://issues.apache.org/jira/browse/HIVE-8231
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 0.14.0
>            Reporter: Damien Carol
>
> Steps to show the bug :
> 1. create table 
> {code}
> create table encaissement_1b_64m like encaissement_1b;
> {code}
> 2. check table 
> {code}
> desc encaissement_1b_64m;
> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m;
> {code}
> everything is ok:
> {noformat}
> 0: jdbc:hive2://nc-h04:10000/casino> desc encaissement_1b_64m;                   
                                                                                         
+------------+------------+----------+--+
> |  col_name  | data_type  | comment  |
> +------------+------------+----------+--+
> | id         | int        |          |
> | idmagasin  | int        |          |
> | zibzin     | string     |          |
> | cheque     | int        |          |
> | montant    | double     |          |
> | date       | timestamp  |          |
> | col_6      | string     |          |
> | col_7      | string     |          |
> | col_8      | string     |          |
> +------------+------------+----------+--+
> 9 rows selected (0.158 seconds)
> 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/;
> +-------------+--+
> | DFS Output  |
> +-------------+--+
> +-------------+--+
> No rows selected (0.01 seconds)
> {noformat}
> 3. Insert values into the new table
> {noformat}
> insert into table encaissement_1b_64m VALUES (1, 1, '800000000909000000000000', 1, 12.5,
'12/05/2014', '','','');
> {noformat}
> 4. Check
> {noformat}
> 0: jdbc:hive2://nc-h04:10000/casino> select id from encaissement_1b_64m;
> +-----+--+
> | id  |
> +-----+--+
> +-----+--+
> No rows selected (0.091 seconds)
> {noformat}
> There are already a pb. I don't see the inserted row.
> 5. When I'm checking HDFS directory, there are something strange, I see {{delta_0000421_0000421}}
folder
> {noformat}
> 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/;
> +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+
> |                                                                     DFS Output    
                                                                 |
> +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+
> | Found 1 items                                                                     
                                                                 |
> | drwxr-xr-x   - hduser supergroup          0 2014-09-23 12:17 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/delta_0000421_0000421
 |
> +-----------------------------------------------------------------------------------------------------------------------------------------------------+--+
> 2 rows selected (0.014 seconds)
> {noformat}
> 6. Doing a major compaction solves the bug
> {noformat}
> 0: jdbc:hive2://nc-h04:10000/casino> alter table encaissement_1b_64m compact 'major';
> No rows affected (0.046 seconds)
> 0: jdbc:hive2://nc-h04:10000/casino> dfs -ls hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/;
> +--------------------------------------------------------------------------------------------------------------------------------------------+--+
> |                                                                 DFS Output        
                                                        |
> +--------------------------------------------------------------------------------------------------------------------------------------------+--+
> | Found 1 items                                                                     
                                                        |
> | drwxr-xr-x   - hduser supergroup          0 2014-09-23 12:21 hdfs://nc-h04/user/hive/warehouse/casino.db/encaissement_1b_64m/base_0000421
 |
> +--------------------------------------------------------------------------------------------------------------------------------------------+--+
> 2 rows selected (0.02 seconds)
> {noformat}
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message