spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chenghao-intel <...@git.apache.org>
Subject [GitHub] spark pull request: [SPARK-2866][SQL] Support attributes in ORDER ...
Date Wed, 06 Aug 2014 02:21:45 GMT
Github user chenghao-intel commented on a diff in the pull request:

    https://github.com/apache/spark/pull/1795#discussion_r15853338
  
    --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
---
    @@ -121,6 +122,51 @@ class Analyzer(catalog: Catalog, registry: FunctionRegistry, caseSensitive:
Bool
       }
     
       /**
    +   * In many dialects of SQL is it valid to sort by attributes that are not present in
the SELECT
    +   * clause.  This rule detects such queries and adds the required attributes to the
original
    +   * projection, so that they will be available during sorting. Another projection is
added to
    +   * remove these attributes after sorting.
    +   */
    +  object ResolveSortReferences extends Rule[LogicalPlan] {
    +    def apply(plan: LogicalPlan): LogicalPlan = plan transformUp {
    +      case s@Sort(ordering, p@Project(projectList, child)) if !s.resolved &&
p.resolved =>
    +        val unresolved = ordering.flatMap(_.collect { case UnresolvedAttribute(name)
=> name})
    +        val resolved = unresolved.flatMap(child.resolveChildren)
    +        val requiredAttributes = resolved.collect { case a: Attribute => a }.toSet
    +
    +        val missingInProject = requiredAttributes -- p.output
    +        if (missingInProject.nonEmpty) {
    +          // Add missing attributes and then project them away after the sort.
    +          Project(projectList,
    +            Sort(ordering,
    +              Project(projectList ++ missingInProject, child)))
    +        } else {
    +          s // Nothing we can do here. Return original plan.
    +        }
    +      case s@Sort(ordering, a@Aggregate(grouping, aggs, child)) if !s.resolved &&
a.resolved =>
    --- End diff --
    
    Spaces before and after `@`


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message