incubator-hcatalog-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject Re: HCatalog Tests, Forrest, Hive, and SemanticException
Date Fri, 07 Jun 2013 06:29:29 GMT
Ohhhh ok! This explains alot!

- yes building off trunk, seems to be functionally equivalent.

- how are the hive / hcat builds dependent?  Shouldn't the class namespace be enough to induce
API compatibility?

I think the relevant JIRA should be updated then..  Very important to make it clear that there
are runtime dependency conflicts/deps between hive and hcat 

On Jun 6, 2013, at 4:19 AM, Alan Gates <gates@hortonworks.com> wrote:

> Are you building off of trunk?
> 
> Forrest should not be required for building off of trunk, as that dependency was removed
a while ago.  You only need forrest if you wish to generate the website docs.  
> 
> The SemanticException failures are due to an error in the way Hive and HCatalog build
together.  The way Hive's parser is built, each time it builds some of the token values change.
 This means if you don't build the HCatalog and Hive packages in sync, Hive will have a different
set of token values than HCatalog, and thus HCatalog's jars will produce commands that cause
Hive to throw a SemanticException.  If you take the build.xml from the Hive 0.11 branch and
use that to build from, and then install the resulting tar ball with both HCatalog and Hive,
this issue should go away.  We are working on fixing this (sorry, don't have the JIRA at the
moment as I'm not online).
> 
> Alan.
> 
> On Jun 6, 2013, at 1:48 AM, Jay Vyas wrote:
> 
>> 
>> Hi : Im trying to run HCatalog tests.  I have some preliminary questions:
>> 
>> 1) Why Is forest needed for the build?  It appears that "ant package" is sufficient
to build an HCatalog deployable. 
>> 
>> 2) I've noticed that I get alot of SemanticException failures.  Maybe this is related
to the fact that I did not build with forest, or else, maybe it is due to incorrect HIVE configuration.
 Any thoughts on this? 
>> 
>> 3) (possible hive setup error?)
>> 
>> Finally, Im noticing that my HCat test operations fail due to hive failures, which
result in the SemanticException.   However, the thrown "SemanticException" error seems to
configuration related, because by printing the offending tokens out, it seems that the tokens
are not being matched in the following case statement. 
>> 
>> Is there some connection with the behaviour of this class (HCatSemanticAnalyzer)
and the hive runtime configurations?  If so that could explain why valid tokens are flagged
as SemanticException.
>> 
>>        case HiveParser.TOK_ALTERTABLE_
>> TOUCH:
>>        case HiveParser.TOK_DESCTABLE:
>>        case HiveParser.TOK_DROPTABLE:
>>        case HiveParser.TOK_SHOW_TABLESTATUS:
>>        case HiveParser.TOK_SHOWPARTITIONS:
>>        case HiveParser.TOK_SHOWTABLES:
>>            return ast;
>> 
>>        // In all other cases, throw an exception. Its a white-list of allowed operations.
>>        default:
>>            throw new SemanticException("Operation not supported.");
>> 
>>        }
>> 
>> 
>> -- 
>> Jay Vyas
>> http://jayunit100.blogspot.com
> 

Mime
View raw message