spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <>
Subject Re: Merging two Spark SQL tables?
Date Mon, 25 Aug 2014 19:49:44 GMT
> SO I tried the above (why doesn't union or ++ have the same behavior
> btw?)

I don't think there is a good reason for this.  I'd open a JIRA.

> and it works, but is slow because the original Rdds are not
> cached and files must be read from disk.
> I also discovered you can recover the InMemoryCached versions of the
> Rdds using sqlContext.table("table1").

Yeah, this is an unfortunate consequence of the way we handle caching.
 I've opened this JIRA for the 1.2 roadmap:

View raw message