hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pradeep Kamath (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-934) Merge join implementation currently does not seek to right point on the right side input based on the offset provided by the index
Date Mon, 31 Aug 2009 23:05:33 GMT

    [ https://issues.apache.org/jira/browse/PIG-934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749669#action_12749669
] 

Pradeep Kamath commented on PIG-934:
------------------------------------

Agree with both the above comments. I was wondering if instead of returning an InputStream,
if the code could return a SeekableInputStream it would be usable in other scenarios (like
creating a CBZip2InputStream out of it - this would be needed for http://issues.apache.org/jira/browse/PIG-930
for example). Callers only needing an InputStream would still be able to use the method.

> Merge join implementation currently does not seek to right point on the right side input
based on the offset provided by the index
> ----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-934
>                 URL: https://issues.apache.org/jira/browse/PIG-934
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.3.1
>            Reporter: Pradeep Kamath
>            Assignee: Ashutosh Chauhan
>         Attachments: pig-934.patch
>
>
> We use POLoad to seek into right file which has the following code: 
> {noformat}
>    public void setUp() throws IOException{
>         String filename = lFile.getFileName();
>         loader = (LoadFunc)PigContext.instantiateFuncFromSpec(lFile.getFuncSpec()); 
      
>         is = FileLocalizer.open(filename, pc);
>         loader.bindTo(filename , new BufferedPositionedInputStream(is), this.offset,
Long.MAX_VALUE);
>     }
> {noformat}
> Between opening the stream and bindTo we do not seek to the right offset. bindTo itself
does not perform any seek.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message