spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ueshin <...@git.apache.org>
Subject [GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...
Date Fri, 01 Jun 2018 03:31:27 GMT
Github user ueshin commented on a diff in the pull request:

    https://github.com/apache/spark/pull/21370#discussion_r192291498
  
    --- Diff: python/pyspark/sql/dataframe.py ---
    @@ -351,8 +352,70 @@ def show(self, n=20, truncate=True, vertical=False):
             else:
                 print(self._jdf.showString(n, int(truncate), vertical))
     
    +    @property
    +    def _eager_eval(self):
    +        """Returns true if the eager evaluation enabled.
    +        """
    +        return self.sql_ctx.getConf(
    +            "spark.sql.repl.eagerEval.enabled", "false").lower() == "true"
    +
    +    @property
    +    def _max_num_rows(self):
    +        """Returns the max row number for eager evaluation.
    +        """
    +        return int(self.sql_ctx.getConf(
    +            "spark.sql.repl.eagerEval.maxNumRows", "20"))
    +
    +    @property
    +    def _truncate(self):
    +        """Returns the truncate length for eager evaluation.
    +        """
    +        return int(self.sql_ctx.getConf(
    +            "spark.sql.repl.eagerEval.truncate", "20"))
    +
         def __repr__(self):
    -        return "DataFrame[%s]" % (", ".join("%s: %s" % c for c in self.dtypes))
    +        if not self._support_repr_html and self._eager_eval:
    +            vertical = False
    +            return self._jdf.showString(
    +                self._max_num_rows, self._truncate, vertical)
    +        else:
    +            return "DataFrame[%s]" % (", ".join("%s: %s" % c for c in self.dtypes))
    +
    +    def _repr_html_(self):
    +        """Returns a dataframe with html code when you enabled eager evaluation
    +        by 'spark.sql.repl.eagerEval.enabled', this only called by REPL you are
    +        using support eager evaluation with HTML.
    +        """
    +        import cgi
    +        if not self._support_repr_html:
    +            self._support_repr_html = True
    +        if self._eager_eval:
    +            max_num_rows = self._max_num_rows
    --- End diff --
    
    I see. I think it's okay with max(0) only.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message