hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16898) Validation of source file after distcp in repl load
Date Fri, 29 Sep 2017 16:59:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16186080#comment-16186080
] 

ASF GitHub Bot commented on HIVE-16898:
---------------------------------------

GitHub user sankarh opened a pull request:

    https://github.com/apache/hive/pull/254

    HIVE-16898: Validation of source file after distcp in repl load

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sankarh/hive HIVE-16898

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/hive/pull/254.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #254
    
----
commit 9d3a04379e519f788e6b83bdbef70d7f7ef7f421
Author: Sankar Hariappan <mailtosankarh@gmail.com>
Date:   2017-09-28T17:06:19Z

    HIVE-16898: Validation of source file after distcp in repl load

----


> Validation of source file after distcp in repl load 
> ----------------------------------------------------
>
>                 Key: HIVE-16898
>                 URL: https://issues.apache.org/jira/browse/HIVE-16898
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 3.0.0
>            Reporter: anishek
>            Assignee: Sankar Hariappan
>              Labels: pull-request-available
>             Fix For: 3.0.0
>
>         Attachments: HIVE-16898.1.patch, HIVE-16898.2.patch, HIVE-16898.3.patch, HIVE-16898.4.patch,
HIVE-16898.5.patch, HIVE-16898.6.patch, HIVE-16898.7.patch, HIVE-16898.8.patch
>
>
> time between deciding the source and destination path for distcp to invoking of distcp
can have a change of the source file, hence distcp might copy the wrong file to destination,
hence we should an additional check on the checksum of the source file path after distcp finishes
to make sure the path didnot change during the copy process. if it has take additional steps
to delete the previous file on destination and copy the new source and repeat the same process
as above till we copy the correct file. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message