hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lavelle, Shawn" <>
Subject Upgrading w/Custom Storage Handler
Date Thu, 30 Jun 2016 13:19:11 GMT
   I'm working on a custom storage handler for Cassandra that was built in HIVE 0.11 and run
on SPARK 0.9.2 to HIVE 1.2.1 and SPARK 1.6. I appreciate your help with three questions:

1)    Hive 0.11 populated a parameter in the jobConf, TableScanDesc.FILTER_TEXT_CONF_STR,
and it appears to me that HIVE on SPARK does not do this.  Is this confirmed? We are implementing
the DecomposedPredicate Interface, but it does not seem to be called and our ParsedPredicate
relied on the FILTER information. Sarg.pushdown is also empty.  I was able to find some references
to SPARK not doing this with patches to support parquet, but is there a universal workaround?

2)    We wrote the storage handler to operate as an External Table in Hive.  I know that SPARK
also would support a C* storage handler, but thought that we could continue to use our storage
handler at Hive's level. Is this the case?

3)    Do you have any further advice for this upgrade?

Thank you,

~ Shawn M Lavelle

View raw message