pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitriy V. Ryaboy (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (PIG-2115) Pig HBaseStorage configuration and setup issues
Date Thu, 27 Sep 2012 21:27:09 GMT

     [ https://issues.apache.org/jira/browse/PIG-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dmitriy V. Ryaboy reassigned PIG-2115:
--------------------------------------

    Assignee: Greg Bowyer

credit where credit's due :)
                
> Pig HBaseStorage configuration and setup issues
> -----------------------------------------------
>
>                 Key: PIG-2115
>                 URL: https://issues.apache.org/jira/browse/PIG-2115
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.9.0
>            Reporter: Greg Bowyer
>            Assignee: Greg Bowyer
>              Labels: hbase, load, storage
>             Fix For: 0.10.0
>
>         Attachments: HBaseStorage-correctly-configure-child-MapR-task-v1.patch, HBaseStorage-correctly-configure-child-MapR-task-v2.patch,
PIG-2115.3.patch
>
>
> HBase storage currently configures child MapR jobs assuming that the HBase jars and cluster
configuration are present on the hadoop cluster that will run the pig program. In some circumstances,
and for some configurations this causes problems with both the configuration suddenly becoming
defaulted during a job run as well as ClassNotFound exceptions occurring due to the child
MapReduce job being incorrectly configured.
> As such, the hbase built in configuration is merged with the job configuration using
the inbuilt pig classes for this functionality, this prevents the jobs configuration from
being overwritten by defaults.
> At the same time this change has been mirrored in the relevant setup methods called when
the HBaseStorage is used as a StoreFunc; this means that the StoreFunc also correctly sets
up the relevant classpath preventing ClassNotFound exceptions in clusters that are not adding
the HBase classes to the HADOOP_CLASSPATH.  
> Attached is a patch that hopefully fixes these issues for others.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message