spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HyukjinKwon <...@git.apache.org>
Subject [GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader
Date Wed, 01 Nov 2017 16:18:46 GMT
Github user HyukjinKwon commented on the issue:

    https://github.com/apache/spark/pull/19439
  
    I opened a PR to deal with docstring issue and completing https://github.com/apache/spark/pull/19439#discussion_r148027184
here https://github.com/imatiach-msft/spark/pull/1
    
    Merging it into this PR and discussing it here, or discussing there before merging it
into this PR. Either is fine to me.
    
    I am not sure if that's the best way and there are few hacks ... but I could not think
of a better way.
    
    I actually tried the another approach taken in graphframes - https://github.com/apache/spark/pull/19439#discussion_r148049352
but seems not working with Sphinx:
    
    
    ```python
    # This workaround was taken after graphframes.
    # See https://github.com/graphframes/graphframes/pull/169.
    class _ClassProperty(object):
        """Custom read-only class property descriptor.
    
        The underlying method should take the class as the sole argument.
        """
    
        def __init__(self, f):
            self.f = f
            self.__doc__ = f.__doc__
    
        def __get__(self, _, cls):
            return self.f(cls)
    ```
    
    ```
    .../spark/python/docs/pyspark.ml.rst:103: WARNING: autodoc: failed to import attribute
u'ImageSchema.imageSchema' from module u'pyspark.ml.image'; the following exception was raised:
    Traceback (most recent call last):
      File "/Library/Python/2.7/site-packages/sphinx/ext/autodoc.py", line 514, in import_object
        obj = self.get_attr(obj, part)
      File "/Library/Python/2.7/site-packages/sphinx/ext/autodoc.py", line 410, in get_attr
        return safe_getattr(obj, name, *defargs)
      File "/Library/Python/2.7/site-packages/sphinx/util/inspect.py", line 115, in safe_getattr
        raise AttributeError(name)
    AttributeError: imageSchema
    ```
    
    So, I just tried to find some hack with the current singleton-like way.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message