hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (PIG-86) load and store function should be file independent
Date Mon, 26 Jan 2009 19:57:59 GMT

     [ https://issues.apache.org/jira/browse/PIG-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Olga Natkovich resolved PIG-86.
-------------------------------

    Resolution: Fixed

Custom slicer takes care of this

> load and store function should be file independent
> --------------------------------------------------
>
>                 Key: PIG-86
>                 URL: https://issues.apache.org/jira/browse/PIG-86
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Stefan Groschupf
>            Priority: Critical
>
> Currently pig is very file system centric for input LoadFunc and output StoreFunc.
> However in many buzz usecases the data are not in the dfs directly but come from a remote
service. Also it is common to store the results into a database for a webbased ui.
> There for I suggest to make filepath elements in the gramar optional.
> For example:
> A = load 'mydata' using PigStorage() // " 'myData' " would be optional  
> STORE A into 'output.bz2' USING myStore(); //  " into 'output.bz2' " would be optional.
> Current workaround for implementing Load and Store functions that work without file system
access is to create temporary dummy files since pig is checking for this files, even if the
load and store functions do not need those file.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message