spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Cheung <felixcheun...@hotmail.com>
Subject Re: Help needed in R documentation generation
Date Tue, 27 Feb 2018 19:09:18 GMT
I had agreed it was a compromise when it was proposed back in May 2017.

I don’t think I can capture the long reviews and many discussed that went in, for further
discussion please start from JIRA SPARK-20889.



________________________________
From: Marcelo Vanzin <vanzin@cloudera.com>
Sent: Tuesday, February 27, 2018 10:26:23 AM
To: Felix Cheung
Cc: Mihály Tóth; Mihály Tóth; dev@spark.apache.org
Subject: Re: Help needed in R documentation generation

I followed Misi's instructions:
- click on https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc5-docs/_site/api/R/index.html
- click on "s" at the top
- find "sin" and click on it

And that does not give me the documentation for the "sin" function.
That leads to you to a really ugly list of functions that's basically
unreadable. There's lots of things like this:

## S4 method for signature 'Column'
abs(x)

Which look to me like the docs weren't properly generated. So it
doesn't look like it's a discoverability problem, it seems there's
something odd going on with the new docs.

On the previous version those same steps take me to a nicely formatted
doc for the "sin" function.



On Tue, Feb 27, 2018 at 10:14 AM, Felix Cheung
<felixcheung_m@hotmail.com> wrote:
> I think what you are calling out is discoverability of names from index - I
> agree this should be improved.
>
> There are several reasons for this change, if I recall, some are:
>
> - we have too many doc pages and a very long index page because of the
> atypical large number of functions - many R packages only have dozens (or a
> dozen) and we have hundreds; this also affects discoverability
>
> - a side effect of high number of functions is that we have hundreds of
> pages of cross links between functions in the same and different categories
> that are very hard to read or find
>
> - many function examples are too simple or incomplete - it would be good to
> make them runnable, for instance
>
> There was a proposal for a search feature on the doc index at one point, IMO
> that would be very useful and would address the discoverability issue.
>
>
> ________________________________
> From: Mihály Tóth <misutoth@gmail.com>
> Sent: Tuesday, February 27, 2018 9:13:18 AM
> To: Felix Cheung
> Cc: Mihály Tóth; dev@spark.apache.org
>
> Subject: Re: Help needed in R documentation generation
>
> Hi,
>
> Earlier, at https://spark.apache.org/docs/latest/api/R/index.html I see
>
> sin as a title
> description describes what sin does
> usage, arguments, note, see also are specific to sin function
>
> When opening sin from
> https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc5-docs/_site/api/R/index.html:
>
> Title is 'Math functions for Column operations', not very specific to sin
> Description is 'Math functions defined for Column.'
> Usage contains a list of functions, scrolling down you can see sin as well
> though ...
>
> To me that sounds like a problem. Do I overlook something here?
>
> Best Regards,
>   Misi
>
>
> 2018-02-27 16:15 GMT+00:00 Felix Cheung <felixcheung_m@hotmail.com>:
>>
>> The help content on sin is in
>>
>> https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc5-docs/_site/api/R/column_math_functions.html
>>
>> It’s a fairly long list but sin is in there. Is that not what you are
>> seeing?
>>
>>
>> ________________________________
>> From: Mihály Tóth <mtoth@cloudera.com>
>> Sent: Tuesday, February 27, 2018 8:03:34 AM
>> To: dev@spark.apache.org
>> Subject: Fwd: Help needed in R documentation generation
>>
>> Hi,
>>
>> Actually, when I open the link you provided and click on - for example -
>> 'sin' the page does not seem to describe that function at all. Actually I
>> get same effect that I get locally. I have attached a screenshot about that:
>>
>>
>>
>>
>>
>> I tried with Chrome and then with Safari too and got the same result.
>>
>> When I go to https://spark.apache.org/docs/latest/api/R/index.html (Spark
>> 2.2.1) and select 'sin' I get a proper Description, Usage, Arguments, etc.
>> sections.
>>
>> This sounds like a bug in the documentation of Spark R, does'nt it? Shall
>> I file a Jira about it?
>>
>> Locally I ran SPARK_HOME/R/create-docs.sh and it returned successfully.
>> Unfortunately with the result mentioned above.
>>
>> Best Regards,
>>
>>   Misi
>>
>>
>>>
>>> --------------------
>>>
>>> From: Felix Cheung <felixcheung_m@hotmail.com>
>>> Date: 2018-02-26 20:42 GMT+00:00
>>> Subject: Re: Help needed in R documentation generation
>>> To: Mihály Tóth <misutoth@gmail.com>
>>> Cc: "dev@spark.apache.org" <dev@spark.apache.org>
>>>
>>>
>>> Could you tell me more about the steps you are taking? Which page you are
>>> clicking on?
>>>
>>> Could you try
>>> https://dist.apache.org/repos/dist/dev/spark/v2.3.0-rc5-docs/_site/api/R/index.html
>>>
>>> ________________________________
>>> From: Mihály Tóth <misutoth@gmail.com>
>>> Sent: Monday, February 26, 2018 8:06:59 AM
>>> To: Felix Cheung
>>> Cc: dev@spark.apache.org
>>> Subject: Re: Help needed in R documentation generation
>>>
>>> I see.
>>>
>>> When I click on such a selected function, like 'sin' the page falls apart
>>> and does not tell anything about sin function. How is it supposed to work
>>> when all functions link to the same column_math_functions.html ?
>>>
>>> Thanks,
>>>
>>>   Misi
>>>
>>>
>>> On Sun, Feb 25, 2018, 22:53 Felix Cheung <felixcheung_m@hotmail.com>
>>> wrote:
>>>>
>>>> This is recent change. The html file column_math_functions.html should
>>>> have the right help content.
>>>>
>>>> What is the problem you are experiencing?
>>>>
>>>> ________________________________
>>>> From: Mihály Tóth <misutoth@gmail.com>
>>>> Sent: Sunday, February 25, 2018 10:42:50 PM
>>>> To: dev@spark.apache.org
>>>> Subject: Help needed in R documentation generation
>>>>
>>>> Hi,
>>>>
>>>> I am having difficulties generating R documentation.
>>>>
>>>> In R/pkg/html/index.html file at the individual function entries it
>>>> reference
>>>> column_math_functions.html instead of the function page itself. Like
>>>>
>>>> <a href="column_math_functions.html">asin</a>
>>>>
>>>> Have you met with such a problem?
>>>>
>>>> Thanks,
>>>>
>>>>   Misi
>>>>
>>>>
>>>
>>
>>
>



--
Marcelo

Mime
View raw message