spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sarath Chandra <sarathchandra.jos...@algofusiontech.com>
Subject PermGen Space Error
Date Wed, 29 Jul 2015 09:09:18 GMT
Dear All,

I'm using -
 => Spark 1.2.0
 => Hive 0.13.1
 => Mesos 0.18.1
 => Spring
 => JDK 1.7

I've written a scala program which
  => instantiates a spark and hive context
  => parses an XML file which provides the where clauses for queries
  => generates full fledged hive queries to be run on hive tables
  => registers obtained SchemaRDD as temp tables to get reduced data sets
to be queried further
  => prints the count of finally obtained data set

I'm running this scala programatically through java command (command
invokes a controller program to create some useful value objects using
input parameters and properties files and then calls the above scala
program).

*I'm getting PermGen Space error when it hits the last line to print the
count.*

I'm printing to console the generated hive queries from the scala program.
When I run the same from a spark shell it works fine.

As mentioned in some posts and blogs I tried using the option
spark.driver.extraJavaOptions to increase the size, tried with 256 and 512
but still no luck.

Please help me in resolving the space issue

Thanks & Regards,
Sarath.

Mime
View raw message