spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiao Li <gatorsm...@gmail.com>
Subject Spark SQL: what does an exclamation mark mean in the plan?
Date Mon, 19 Oct 2015 06:38:57 GMT
Hi, all,

After turning on the trace, I saw a strange exclamation mark in
the intermediate plans. This happened in catalyst analyzer.

Join Inner, Some((col1#0 = col1#6))
 Project [col1#0,col2#1,col3#2,col2_alias#24,col3#2 AS col3_alias#13]
  Project [col1#0,col2#1,col3#2,col2#1 AS col2_alias#24]
   LogicalRDD [col1#0,col2#1,col3#2], MapPartitionsRDD[1] at
createDataFrame at SimpleApp.scala:32
 Aggregate [col1#6], [col1#6,count(col1#6) AS count(col1)#5L]
  *!Project [col1#6,col2#7,col3#8,col2_alias#24,col3#8 AS col3_alias#4]*
   Project [col1#6,col2#7,col3#8,col2#7 AS col2_alias#3]
    LogicalRDD [col1#6,col2#7,col3#8], MapPartitionsRDD[1] at
createDataFrame at SimpleApp.scala:32

Could anybody give me a hint why there exists a !(exclamation mark) before
the node name (Project)? This ! mark does not disappear in the subsequent
query plan.

Thank you!

Xiao Li

Mime
View raw message