spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <mich...@databricks.com>
Subject Re: Spark 1.6.1
Date Tue, 02 Feb 2016 18:48:12 GMT
I'm waiting for a few last fixes to be merged.  Hoping to cut an RC in the
next few days.

On Tue, Feb 2, 2016 at 10:43 AM, Mingyu Kim <mkim@palantir.com> wrote:

> Hi all,
>
> Is there an estimated timeline for 1.6.1 release? Just wanted to check how
> the release is coming along. Thanks!
>
> Mingyu
>
> From: Romi Kuntsman <romi@totango.com>
> Date: Tuesday, February 2, 2016 at 3:16 AM
> To: Michael Armbrust <michael@databricks.com>
> Cc: Hamel Kothari <hamelkothari@gmail.com>, Ted Yu <yuzhihong@gmail.com>,
> "dev@spark.apache.org" <dev@spark.apache.org>
> Subject: Re: Spark 1.6.1
>
> Hi Michael,
> What about the memory leak bug?
> https://issues.apache.org/jira/browse/SPARK-11293
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SPARK-2D11293&d=CwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=ennQJq47pNnObsDh-88a9YUrUulcYQoV8giPASqXB84&m=tI8Pjfii7XuX3Suiky8mImD7S5BoAq6fgOSdJ7rt2Wo&s=R_B4rDig-0VPE5Q4YeLEs2HUIg-A8St1OtDjD89d_zY&e=>
> Even after the memory rewrite in 1.6.0, it still happens in some cases.
> Will it be fixed for 1.6.1?
> Thanks,
>
> *Romi Kuntsman*, * Big Data Engineer*
> http://www.totango.com
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.totango.com_&d=CwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=ennQJq47pNnObsDh-88a9YUrUulcYQoV8giPASqXB84&m=tI8Pjfii7XuX3Suiky8mImD7S5BoAq6fgOSdJ7rt2Wo&s=Z4TgGF0h7oetD4O6u_3qjrYbe0ZtW2g_In7V8tkByPg&e=>
>
> On Mon, Feb 1, 2016 at 9:59 PM, Michael Armbrust <michael@databricks.com>
> wrote:
>
>> We typically do not allow changes to the classpath in maintenance
>> releases.
>>
>> On Mon, Feb 1, 2016 at 8:16 AM, Hamel Kothari <hamelkothari@gmail.com>
>> wrote:
>>
>>> I noticed that the Jackson dependency was bumped to 2.5 in master for
>>> something spark-streaming related. Is there any reason that this upgrade
>>> can't be included with 1.6.1?
>>>
>>> According to later comments on this thread:
>>> https://issues.apache.org/jira/browse/SPARK-8332
>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.apache.org_jira_browse_SPARK-2D8332&d=CwMFaQ&c=izlc9mHr637UR4lpLEZLFFS3Vn2UXBrZ4tFb6oOnmz8&r=ennQJq47pNnObsDh-88a9YUrUulcYQoV8giPASqXB84&m=tI8Pjfii7XuX3Suiky8mImD7S5BoAq6fgOSdJ7rt2Wo&s=i-ngQFHfxOmgkYx_5NiCaHdIlm7zi2LYpUxm9I3RfR4&e=>
>>> and my personal experience using with Spark with Jackson 2.5 hasn't caused
>>> any issues but it does have some useful new features. It should be fully
>>> backwards compatible according to the Jackson folks.
>>>
>>> On Mon, Feb 1, 2016 at 10:29 AM Ted Yu <yuzhihong@gmail.com> wrote:
>>>
>>>> SPARK-12624 has been resolved.
>>>> According to Wenchen, SPARK-12783 is fixed in 1.6.0 release.
>>>>
>>>> Are there other blockers for Spark 1.6.1 ?
>>>>
>>>> Thanks
>>>>
>>>> On Wed, Jan 13, 2016 at 5:39 PM, Michael Armbrust <
>>>> michael@databricks.com> wrote:
>>>>
>>>>> Hey All,
>>>>>
>>>>> While I'm not aware of any critical issues with 1.6.0, there are
>>>>> several corner cases that users are hitting with the Dataset API that
are
>>>>> fixed in branch-1.6.  As such I'm considering a 1.6.1 release.
>>>>>
>>>>> At the moment there are only two critical issues targeted for 1.6.1:
>>>>>  - SPARK-12624 - When schema is specified, we should treat undeclared
>>>>> fields as null (in Python)
>>>>>  - SPARK-12783 - Dataset map serialization error
>>>>>
>>>>> When these are resolved I'll likely begin the release process.  If
>>>>> there are any other issues that we should wait for please contact me.
>>>>>
>>>>> Michael
>>>>>
>>>>
>>>>
>>
>

Mime
View raw message