hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tao Li (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-17496) Bootstrap repl is not cleaning up staging dirs
Date Mon, 18 Sep 2017 15:29:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16170144#comment-16170144
] 

Tao Li edited comment on HIVE-17496 at 9/18/17 3:28 PM:
--------------------------------------------------------

According to the logs below, I did not see any error from HDFS deletion for the testDeleteStagingDir
test, according to change in HIVE-17496.5.patch. Looks like the file should have been deleted
successfully. So I don't know why the test fails.

{noformat}
2017-09-18T02:25:52,399  INFO [main] ql.Context: Deleting scratch dir: pfile:/home/hiveptest/104.198.168.233-hiveptest-0/apache-github-source-source/itests/hive-unit/target/warehouse/deletestagingdir_org_apache_hadoop_hive_ql_parse_testreplicationscenarios_1505726246743_dupe.db/unptned/.hive-staging_hive_2017-09-18_02-25-52_081_7805993483018006783-1
{noformat}

But in some other repl test logs, I do see some deletion that failed. It indicates some possibility
there.

{noformat}
2017-09-18T02:25:49,746  INFO [main] ql.Context: Deleting scratch dir: file:/home/hiveptest/104.198.168.233-hiveptest-0/apache-github-source-source/itests/hive-unit/target/tmp/localscratchdir/6b15a44f-aec0-43f8-851d-81db07e4e329/hive_2017-09-18_02-25-49_687_551878613091072616-1/-mr-10001/.hive-staging_hive_2017-09-18_02-25-49_687_551878613091072616-1
2017-09-18T02:25:49,746 ERROR [main] ql.Context: File failes to be deleted when removing Scratch
(HIVE-17496)
{noformat}


was (Author: taoli-hwx):
According to the logs below, I did not see any error from HDFS deletion for the testDeleteStagingDir
test, according to change in HIVE-17496.5.patch. Looks like the file should have been deleted
successfully. So I don't know why the test fails.

{noformat}
2017-09-18T02:25:52,399  INFO [main] ql.Context: Deleting scratch dir: pfile:/home/hiveptest/104.198.168.233-hiveptest-0/apache-github-source-source/itests/hive-unit/target/warehouse/deletestagingdir_org_apache_hadoop_hive_ql_parse_testreplicationscenarios_1505726246743_dupe.db/unptned/.hive-staging_hive_2017-09-18_02-25-52_081_7805993483018006783-1
{noformat}

But in some other repl test logs, I do see some deletion that failed:

{noformat}
2017-09-18T02:25:49,746  INFO [main] ql.Context: Deleting scratch dir: file:/home/hiveptest/104.198.168.233-hiveptest-0/apache-github-source-source/itests/hive-unit/target/tmp/localscratchdir/6b15a44f-aec0-43f8-851d-81db07e4e329/hive_2017-09-18_02-25-49_687_551878613091072616-1/-mr-10001/.hive-staging_hive_2017-09-18_02-25-49_687_551878613091072616-1
2017-09-18T02:25:49,746 ERROR [main] ql.Context: File failes to be deleted when removing Scratch
(HIVE-17496)
{noformat}

> Bootstrap repl is not cleaning up staging dirs
> ----------------------------------------------
>
>                 Key: HIVE-17496
>                 URL: https://issues.apache.org/jira/browse/HIVE-17496
>             Project: Hive
>          Issue Type: Bug
>          Components: repl
>            Reporter: Tao Li
>            Assignee: Tao Li
>         Attachments: HIVE-17496.1.patch, HIVE-17496.2.patch, HIVE-17496.3.patch, HIVE-17496.4.patch,
HIVE-17496.5.patch, HIVE-17496.6.patch
>
>
> This will put more pressure on the HDFS file limit.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message