hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Haviv <daniel.ha...@veracity-group.com>
Subject Re: Imporve performance in displaying content of data frame
Date Wed, 01 Jul 2015 10:03:44 GMT
Hi Vinod,
A better place to ask this would be at Spark's mailing list.

Your select isn't executed until you're running the foreach on it, so you get the impression
that the select ran fast.

Daniel

> On 1 ביולי 2015, at 12:56, Vinod Kuamr <vinod.rajan1991@yahoo.com> wrote:
> 
> Hi Everyone,
> 
> I am using following sqlContext
> 
> var df=sqlContext.sql("SELECT fullname,SUM(CAST(contactid AS decimal(38,6))) FROM adventurepersoncontacts
GROUP BY fullname ORDER BY fullname ASC");
> 
> It executes fine but when I display the content of the data frame by using println method
it take very more time to retrun the result
> 
> df.foreach(println)
> 
> can you please let me know how get the content of data frame in a optimized way?
> 
> My Environment is:
> Spark 1.3.1
> Windows 8
> Sample Data with  15000 records
> 
> Thank you,
> Vinod

Mime
View raw message