predictionio-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shinsuke Sugaya <shinsuke.sug...@gmail.com>
Subject Re: Update default build targets
Date Tue, 06 Jun 2017 04:44:55 GMT
> What is the policy driving dependency upgrades?

Although it might be difficult to define it,
how about the following policy:

- Select newer dependencies
- If the above one is not supported in major platforms(ex. AWS,
  Cloudera, Horton), change to the lower version
- Review the version of dependencies at every releases if needed

As for elasticsearch, I would like to keep a newer version
since it will be released monthly.

> I don’t run hadoop 2.7 locally and many users that have Cloudera or Horton
> contracts may not either.

Thank you for the info.
CDH seems not to support hadoop 2.7...
Updated as below:

<UPDATE>
0.12.0:
 - PIO_SCALA_VERSION=2.11.8
 - PIO_SPARK_VERSION=2.1.1
 - PIO_ELASTICSEARCH_VERSION=5.4.1
 - PIO_HADOOP_VERSION=2.6.5


For Python 2/3, I fixed some templates, such as recommender.
Since I think template scripts can support both Python 2/3,
I'll fix them.

Regards,
 shinsuke

2017-06-06 4:57 GMT+09:00 Pat Ferrel <pat@occamsmachete.com>:
> What is the policy driving dependency upgrades?
>
> I don’t run hadoop 2.7 locally and many users that have Cloudera or Horton
> contracts may not either. Not sure why this should be the default until it’s
> the most popular of we need some feature of it.
>
> I’d agree with most of what @Shinsuke suggests as long as there is an easy
> way to build for any reasonable combination of deps.
>
> The hard one will be Python 3. All existing python scripts in templates will
> need upgrading since it’s very difficult to support mixed py2 and py3 where
> scala 2.10 and 2,11 are much easier. I still think it’s time to do this but
> mention it because with each upgrade we need to consider how may templates
> are left even further behind. Many now do not work with Apache PIO, this may
> put them further behind.
>
> Though we work on PIO we must remember that PIO does nothing interesting
> without templates and ask ourselves what pain we may cause for template
> users.
>
>
>
> On Jun 5, 2017, at 11:06 AM, Donald Szeto <donald@apache.org> wrote:
>
> Hey all, this has a huge impact to the default build, so If you see any
> issue with this, please let us know as soon as possible.
>
> On Sun, Jun 4, 2017 at 10:25 PM, Shinsuke Sugaya <shinsuke.sugaya@gmail.com>
> wrote:
>>
>> Hi all,
>>
>> We have a plan to change default build targets in PIO-83 and PIO-84.
>> Current versions look too old, so it will be better to support
>> newer versions as default.
>>
>> Current:
>>  - PIO_SCALA_VERSION=2.10.6
>>  - PIO_SPARK_VERSION=1.6.3
>>  - PIO_ELASTICSEARCH_VERSION=1.7.6
>>  - PIO_HADOOP_VERSION=2.6.5
>>
>> They will be changed to:
>>
>> 0.12.0:
>>  - PIO_SCALA_VERSION=2.11.8
>>  - PIO_SPARK_VERSION=2.1.1
>>  - PIO_ELASTICSEARCH_VERSION=5.4.1
>>  - PIO_HADOOP_VERSION=2.7.3
>>
>> Note that this change does not discard old version supports.
>> If you want to use old versions, you can build PIO with them.
>>
>> Please let us know if you have any concerns.
>>
>> https://issues.apache.org/jira/browse/PIO-83
>> https://issues.apache.org/jira/browse/PIO-84
>>
>> Regards,
>>  shinsuke
>
>
>

Mime
View raw message