spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolae Marasoiu <nicolae.maras...@adswizz.com>
Subject Re: How to save DataFrame as a Table in Hbase?
Date Fri, 02 Oct 2015 06:18:57 GMT
Hi,

Phoenix, an SQL coprocessor for HBase has ingestion integration with dataframes in 4.x version.
For HBase and RDD in general there are multiple solutions: hbase-spark module by Cloudera,
which wil be part of a future HBase release, hbase-rdd by unicredit, and many others.
I am not sure if the fact that an RDD is dataframe or normal RDD is relevant for storage.
I think main advantage of dataframe is economical memory usage and efficient scans thru the
data in memory and processing in general but when mapping to outside schema, you have the
same data to map to a schema specific to external db. For instance saving granular values
in separate columns or bundling them together in arrays of concatenated values is a choice
that seems to be independent of how the rdd is on the spark side - normal rdd or data frames,
but more like a storage tradeoff between space & speed for various use cases (data access
patterns).

Nicu
________________________________________
From: unk1102 <umesh.kacha@gmail.com>
Sent: Friday, October 2, 2015 1:15 AM
To: user@spark.apache.org
Subject: How to save DataFrame as a Table in Hbase?

Hi anybody tried to save DataFrame in HBase? I have processed data in
DataFrame which I need to store in HBase so that my web ui can access it
from Hbase? Please guide. Thanks in advance.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-save-DataFrame-as-a-Table-in-Hbase-tp24903.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message