hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13992) Integrate SparkOnHBase into HBase
Date Thu, 23 Jul 2015 18:53:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639357#comment-14639357
] 

Steve Loughran commented on HBASE-13992:
----------------------------------------

There's not much in the way of tests here, in particular, not much in the way of generation
of failure conditions and validation of outcome

Ideally, there'd be one test to generate each failure condition: the exception handling including
those which downgrade a failure to a log message...the test should verify that such actions
are the correct response.

At the very least, I'd recommend

# test against non-existent database
# attempt to work with a table that doesn't exist
# attempt to read a column that doesn't exist


I'd also make sure test teardown is robust, catching exceptions & downgrading to logs.
That way, if something didn't get set up properly, the root cause of the failure isn't hidden
by any exception generated in teardown.

> Integrate SparkOnHBase into HBase
> ---------------------------------
>
>                 Key: HBASE-13992
>                 URL: https://issues.apache.org/jira/browse/HBASE-13992
>             Project: HBase
>          Issue Type: New Feature
>          Components: spark
>            Reporter: Ted Malaska
>            Assignee: Ted Malaska
>             Fix For: 2.0.0
>
>         Attachments: HBASE-13992.5.patch, HBASE-13992.6.patch, HBASE-13992.7.patch, HBASE-13992.8.patch,
HBASE-13992.9.patch, HBASE-13992.patch, HBASE-13992.patch.3, HBASE-13992.patch.4, HBASE-13992.patch.5
>
>
> This Jira is to ask if SparkOnHBase can find a home in side HBase core.
> Here is the github: 
> https://github.com/cloudera-labs/SparkOnHBase
> I am the core author of this project and the license is Apache 2.0
> A blog explaining this project is here
> http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/
> A spark Streaming example is here
> http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/
> A real customer using this in produce is blogged here
> http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/
> Please debate and let me know what I can do to make this happen.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message