spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From marmbrus <>
Subject [GitHub] spark pull request: [SPARK-5212][SQL] Add support of schema-less, ...
Date Mon, 02 Feb 2015 02:09:28 GMT
Github user marmbrus commented on a diff in the pull request:
    --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala ---
    @@ -167,8 +167,10 @@ private[hive] trait HiveStrategies {
       object Scripts extends Strategy {
         def apply(plan: LogicalPlan): Seq[SparkPlan] = plan match {
    -      case logical.ScriptTransformation(input, script, output, child) =>
    -        ScriptTransformation(input, script, output, planLater(child))(hiveContext) ::
    +      case logical.ScriptTransformation(input, script, output, child, schema) =>
    +          ScriptTransformation(input, script, output,
    +            planLater(child),{ case s: HiveScriptIOSchema => s }.get
    --- End diff --
    Given that it is safe to just call `get` here it seems like `Option` might not be the
correct choice for the data model.  I'd also do the casting by place a type annotation in
the match.
    case logical.ScriptTransformation(input, script, output, child, schema: HiveScriptIOSchema)

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message