drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arina Ielchiieva (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (DRILL-4356) Subsequent refresh table metadata command returns in-correct message to user.
Date Tue, 04 Jul 2017 15:41:00 GMT

     [ https://issues.apache.org/jira/browse/DRILL-4356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Arina Ielchiieva resolved DRILL-4356.
-------------------------------------
    Resolution: Fixed

> Subsequent refresh table metadata command returns in-correct message to user.
> -----------------------------------------------------------------------------
>
>                 Key: DRILL-4356
>                 URL: https://issues.apache.org/jira/browse/DRILL-4356
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - Parquet
>    Affects Versions: 1.5.0
>         Environment: 4 node cluster CentOS
>            Reporter: Khurram Faraaz
>
> Subsequent refresh table metadata command returns in-correct message to user.
> git commit ID: ca53c244
> MapR Drill 1.5.0
> JDK8
> MapR FS 5.0.0 GA 
>  
> {noformat}
> CREATE TABLE drill4349 AS SELECT CAST(columns[0] AS INT) AS id, CAST(NULLIF(columns[1],
'') AS DOUBLE) AS val from `manyNullsInColum.csv`;
> 0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
> +-------+-----------------------------------------------------+
> |  ok   |                       summary                       |
> +-------+-----------------------------------------------------+
> | true  | Successfully updated metadata for table drill4349.  |
> +-------+-----------------------------------------------------+
> 1 row selected (0.336 seconds)
> [root@centos-01 ~]# hadoop fs -ls /tmp/drill4349
> Found 2 items
> -rwxr-xr-x   3 mapr mapr        764 2016-02-04 09:00 /tmp/drill4349/.drill.parquet_metadata
> -rwxr-xr-x   3 mapr mapr     201205 2016-02-04 08:44 /tmp/drill4349/0_0_0.parquet
> Delete the parquet_metadata cache file.
> [root@centos-01 ~]# hadoop fs -rm -r /tmp/drill4349/.drill.parquet_metadata
> 16/02/05 06:11:49 INFO Configuration.deprecation: io.bytes.per.checksum is deprecated.
Instead, use dfs.bytes-per-checksum
> 16/02/05 06:11:49 INFO fs.TrashPolicyDefault: Namenode trash configuration: Deletion
interval = 0 minutes, Emptier interval = 0 minutes.
> Deleted /tmp/drill4349/.drill.parquet_metadata
> Re-run refresh table metadata command, this one succeeds as expected.
> 0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
> +-------+-----------------------------------------------------+
> |  ok   |                       summary                       |
> +-------+-----------------------------------------------------+
> | true  | Successfully updated metadata for table drill4349.  |
> +-------+-----------------------------------------------------+
> 1 row selected (0.336 seconds)
> However, a subsequent refresh table metadata command fails with message that says, Support
is currently limited to single-directory-based Parquet tables.
> 0: jdbc:drill:schema=dfs.tmp> refresh table metadata drill4349;
> +----+---------+
> | ok | summary |
> +----+---------+
> | false | Table drill4349 does not support metadata refresh.  Support is currently limited
to single-directory-based Parquet tables. |
> +----+---------+
> 1 row selected (0.299 seconds)
> {noformat}
> Interesting to note that in drillbit.log, in the entry corresponding to the second failed
refresh table metadata command, is also reported as FINISHED and no errors/message is logged
to indicate that it failed.
> {noformat}
> 2016-02-05 06:11:59,494 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 294bc2cf-8a84-2f98-68a2-2aefe3d27b43: refresh table metadata drill4349
> 2016-02-05 06:11:59,728 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO  o.a.d.exec.store.parquet.Metadata
- Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Time: 3ms total, 3.565858ms
avg, 3ms max.
> 2016-02-05 06:11:59,728 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:foreman] INFO  o.a.d.exec.store.parquet.Metadata
- Fetch parquet metadata: Executed 1 out of 1 using 1 threads. Earliest start: 1.093000 μs,
Latest start: 1.093000 μs, Average start: 1.093000 μs .
> 2016-02-05 06:11:59,756 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor
- 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State change requested AWAITING_ALLOCATION -->
RUNNING
> 2016-02-05 06:11:59,756 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter
- 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State to report: RUNNING
> 2016-02-05 06:11:59,761 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor
- 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State change requested RUNNING --> FINISHED
> 2016-02-05 06:11:59,761 [294bc2cf-8a84-2f98-68a2-2aefe3d27b43:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter
- 294bc2cf-8a84-2f98-68a2-2aefe3d27b43:0:0: State to report: FINISHED
> 2016-02-05 06:12:07,564 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:foreman] INFO  o.a.drill.exec.work.foreman.Foreman
- Query text for query id 294bc2c7-9f94-16b1-442f-eb6dc74d707a: refresh table metadata drill4349
> 2016-02-05 06:12:07,778 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:foreman] INFO  o.a.d.exec.store.parquet.Metadata
- Took 13 ms to read metadata from cache file
> 2016-02-05 06:12:07,791 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor
- 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State change requested AWAITING_ALLOCATION -->
RUNNING
> 2016-02-05 06:12:07,791 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter
- 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State to report: RUNNING
> 2016-02-05 06:12:07,796 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO  o.a.d.e.w.fragment.FragmentExecutor
- 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State change requested RUNNING --> FINISHED
> 2016-02-05 06:12:07,796 [294bc2c7-9f94-16b1-442f-eb6dc74d707a:frag:0:0] INFO  o.a.d.e.w.f.FragmentStatusReporter
- 294bc2c7-9f94-16b1-442f-eb6dc74d707a:0:0: State to report: FINISHED
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message