hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-231) validation of files in ship/cache specs
Date Wed, 07 May 2008 17:40:56 GMT

    [ https://issues.apache.org/jira/browse/PIG-231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12594974#action_12594974
] 

Olga Natkovich commented on PIG-231:
------------------------------------

Arun, a couple of questions/comments:

(1) For skip path, you need to also check that it is a directory. If this is the only change
needed, I can make it.
(2) It was not clear to me why you could not validate everything in the grunt parser. You
should be able to find if file exists on dfs. We already do it for dfs operations such as
ls.

> validation of files in ship/cache specs
> ---------------------------------------
>
>                 Key: PIG-231
>                 URL: https://issues.apache.org/jira/browse/PIG-231
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Olga Natkovich
>            Assignee: Arun C Murthy
>         Attachments: PIG-231_0_20080507.patch
>
>
> Currently the code fails after map reduce job starts if files for ship/cache don't exist.
> We should be able to detect that on the client side.
> For ship, make sure that the file(s) to be shipped exist on the client
> For cache, make sure it exists on the server.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message