drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khurram Faraaz (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-4356) Subsequent refresh table metadata command returns in-correct message to user.
Date Fri, 05 Feb 2016 06:43:39 GMT
Khurram Faraaz created DRILL-4356:
-------------------------------------

             Summary: Subsequent refresh table metadata command returns in-correct message
to user.
                 Key: DRILL-4356
                 URL: https://issues.apache.org/jira/browse/DRILL-4356
             Project: Apache Drill
          Issue Type: Bug
          Components: Storage - Parquet
    Affects Versions: 1.5.0
         Environment: 4 node cluster CentOS
            Reporter: Khurram Faraaz


Subsequent refresh table metadata command returns in-correct message to user.

git commit ID: ca53c244
MapR Drill 1.5.0
JDK8
MapR FS 5.0.0 GA 
 
{noformat}
CREATE TABLE drill4349 AS SELECT CAST(columns[0] AS INT) AS id, CAST(NULLIF(columns[1], '')
AS DOUBLE) AS val from `manyNullsInColum.csv`;

0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
+-------+-----------------------------------------------------+
|  ok   |                       summary                       |
+-------+-----------------------------------------------------+
| true  | Successfully updated metadata for table drill4349.  |
+-------+-----------------------------------------------------+
1 row selected (0.336 seconds)

[root@centos-01 ~]# hadoop fs -ls /tmp/drill4349
Found 2 items
-rwxr-xr-x   3 mapr mapr        764 2016-02-04 09:00 /tmp/drill4349/.drill.parquet_metadata
-rwxr-xr-x   3 mapr mapr     201205 2016-02-04 08:44 /tmp/drill4349/0_0_0.parquet

Delete the parquet_metadata cache file.

[root@centos-01 ~]# hadoop fs -rm -r /tmp/drill4349/.drill.parquet_metadata
16/02/05 06:11:49 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated. Instead,
use dfs.bytes-per-checksum
16/02/05 06:11:49 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion interval
= 0 minutes, Emptier interval = 0 minutes.
Deleted /tmp/drill4349/.drill.parquet_metadata

Re-run refresh table metadata command, this one succeeds as expected.

0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
+-------+-----------------------------------------------------+
|  ok   |                       summary                       |
+-------+-----------------------------------------------------+
| true  | Successfully updated metadata for table drill4349.  |
+-------+-----------------------------------------------------+
1 row selected (0.336 seconds)

However, a subsequent refresh table metadata command fails with message that says, Support
is currently limited to single-directory-based Parquet tables.

0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
+----+---------+
| ok | summary |
+----+---------+
| false | Table drill4349 does not support metadata refresh.  Support is currently limited
to single-directory-based Parquet tables. |
+----+---------+
1 row selected (0.299 seconds)
{noformat}

Interesting to note that in drillbit.log, in the entry corresponding to the second failed
refresh table metadata command, is also reported as FINISHED and no errors/message is logged
to indicate that it failed.

{noformat}
2016-02-05 06:11:59,494 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 294bc2cf-8a84-2f98-68a2-2aefe3d27b43: refresh table metadata drill4349
2016-02-05 06:11:59,728 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO  o.a.d.exec.store.parquet.Metadata
- Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Time: 3ms total, 3.565858ms
avg, 3ms max.
2016-02-05 06:11:59,728 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO  o.a.d.exec.store.parquet.Metadata
- Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Earliest start: 1.093000 μs,
Latest start: 1.093000 μs, Average start: 1.093000 μs .
2016-02-05 06:11:59,756 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor
- 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State change requested AWAITING_ALLOCATION -->
RUNNING
2016-02-05 06:11:59,756 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter
- 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State to report: RUNNING
2016-02-05 06:11:59,761 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor
- 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State change requested RUNNING --> FINISHED
2016-02-05 06:11:59,761 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter
- 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State to report: FINISHED

2016-02-05 06:12:07,564 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 294bc2c7-9f94-16b1-442f-eb6dc74d707a: refresh table metadata drill4349
2016-02-05 06:12:07,778 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:foreman] INFO  o.a.d.exec.store.parquet.Metadata
- Took 13 ms to read metadata from cache file
2016-02-05 06:12:07,791 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor
- 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State change requested AWAITING_ALLOCATION -->
RUNNING
2016-02-05 06:12:07,791 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter
- 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State to report: RUNNING
2016-02-05 06:12:07,796 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor
- 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State change requested RUNNING --> FINISHED
2016-02-05 06:12:07,796 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter
- 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State to report: FINISHED
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message