spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew O'Reilly <moreill...@qub.ac.uk>
Subject Encryption on RDDs or in-memory/cache on Apache Spark
Date Fri, 31 Jul 2015 08:17:27 GMT
Hi, 

I am currently working on the latest version of Apache Spark (1.4.1), pre-built package for
Hadoop 2.6+.

Is there any feature in Spark/Hadoop to encrypt RDDs or in-memory/cache (something similar
is Altibase's HDB: http://altibase.com/in-memory-database-computing-solutions/security/)
when running applications in Spark? Or is there an external library/framework which could
be used to encrypt RDDs or in-memory/cache in Spark?

I discovered it is possible to encrypt the data, and encapsulate it into RDD. However, I feel
this affects Spark's fast data processing as it is slower to encrypt the data, and then encapsulate
it to RDD; it's then a two step process. Encryption and storing data should be done parallel.

Any help would be appreciated.

Many thanks,
Matthew


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Mime
View raw message