Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@phoenix.apache.org
Date: Sun, 10 Jul 2016 09:42:19 +0000 (UTC)
From: <larsh@apache.org>
Reply-To: <larsh@apache.org>
To: "dev@phoenix.apache.org" <dev@phoenix.apache.org>
Message-ID: <2139908888.1437981.1468143739500.JavaMail.yahoo@mail.yahoo.com>
In-Reply-To: <749E441F-AF13-44D3-9FFE-016A35836786@gmail.com>
References: <CAAF1Jdj7x6G=K9PyOZJk_cQqF5eeOp5Wnfi_KTAsAWA2wJXVhQ@mail.gmail.com> <CANZa=Gs+1g=jf0L2hQWzNo1kMj2uJpioJ1HqHeCCn2Ot-24ifg@mail.gmail.com> <CA+RK=_DTh6azYFMVH9fX9_8fJ4NSD-48yRBr4T-_wSX24u9jMw@mail.gmail.com> <CANZa=Gv1H=escVosNUjmqtcYG7Dd0z6323R34H6_nW5oYnnysg@mail.gmail.com> <D1D43607-D69B-4AD8-BAA6-C595419FC8E1@gmail.com> <CA+RK=_BLaOpfz3X4efpUb64Cx39kdFAQF7WO9sPFKaj4zpct0w@mail.gmail.com> <CAMUu0w88toT15aPR6sADzYG=DkxqjsxeXk+uvx8s76xu+WhStw@mail.gmail.com> <1515219485.243521.1467884674411.JavaMail.yahoo@mail.yahoo.com> <CA+ap0wYofyPipT0Dv1Wsf+xWX8pGSWi8aXhU6EMWJj2s6xh4qA@mail.gmail.com> <CANZa=Gsxxn40pni85LON4EL8bogDDjFFvvMy5ZJyE+GoexHfuw@mail.gmail.com> <CAMUu0w8z_ywNJ5o29Tj-zC=UHchLqideFSS+hSioDwouig46Cg@mail.gmail.com> <CAAF1JdhgbdhjS69ZgWwyAZTCyA4Rtj5EUzDTvDe1oUb3Vv-=UQ@mail.gmail.com> <203497731.1211488.1468072555405.JavaMail.yahoo@mail.yahoo.com> <749E441F-AF13-44D3-9FFE-016A35836786@gmail.com>
Subject: Re: [DISCUSS] RM for next release
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_1437980_904055.1468143739488"
archived-at: Sun, 10 Jul 2016 09:44:14 -0000

------=_Part_1437980_904055.1468143739488
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

0.94 never took that much time from me.Sure some days it's a lot of work, b=
ut on average I found it a task that can be easily done on the side.
I relied a lot on the Apache Jenkins, triggering runs and checking back the=
 next day, etc.I also think you're more diligent in your RM duties :)

-- Lars

      From: Andrew Purtell <andrew.purtell@gmail.com>
 To: dev@phoenix.apache.org=20
 Sent: Saturday, July 9, 2016 9:02 AM
 Subject: Re: [DISCUSS] RM for next release
  =20
Huh. RMing HBase 0.98 consumes whole days with testing and patching. YMMV=
=20

> On Jul 9, 2016, at 6:55 AM, <larsh@apache.org> <larsh@apache.org> wrote:
>=20
> +1 on 4.8.x should support HBase 1.0. (although it would safe us maintena=
nce of=20
> For 4.9 we can drop that (but likely should pull in support for HBase 1.3=
).We'll need to continue with 0.98 support until 0.98 is EOL'd, I think.
>=20
> I'm happy to do 4.8.1 and 4.9 for one release, to confirm Stack's nicknam=
e for me: "Iron Hand" :)After the first release more folks should tag team.=
 RM'ing is actually not that much work, maybe 15-30 mins/day.
>=20
> I'm actually quire exited about the potential of 4.9. We'll bring more of=
 the power of knowing the schema together with the proven stability of HBas=
e:
> PHOENIX-1598 Encode column names to save space and improve performance
> PHOENIX-2565 Store data for immutable tables in single KeyValue
> The trick will be to add these in an absolutely backwards compatible way =
(where an old 4.8 client will be able to run against a 4.9 server indefinit=
ely).We may have to start putting a schema version number into the SYSTEM.C=
ATALOG if we do not have this already.
>=20
> I'll propose merging those into 4.9 early (assuming they are ready and st=
able - so that's something to ensure now in their resp branches).Bug fixes =
would go into both 4.9 and 4.8.1.
>=20
> As soon as 4.8 is out (hopefully soon now), I'll propose the branches and=
 create them once we all agree.
>=20
> -- Lars
>=20
> ps. There's no 4.8-HBase-1.0, yet, right?! At least I don't see it.
>=20
>=C2=A0 =C2=A0 =C2=A0 From: James Taylor <jamestaylor@apache.org>
> To: "dev@phoenix.apache.org" <dev@phoenix.apache.org>=20
> Sent: Friday, July 8, 2016 3:01 PM
> Subject: Re: [DISCUSS] RM for next release
>=20
> I think we should go ahead with the 4.8 release for HBase 1.0, but not do
> one for 4.9. As far as I recall, there were some objections about not doi=
ng
> an HBase 1.0 release because CDH was based off of 1.0. Seems like CDH has
> moved past this, though, so it's likely not necessary to continue. Probab=
ly
> something we should propose on the user list.
>=20
>> On Fri, Jul 8, 2016 at 10:51 PM, Enis S=C3=B6ztutar <enis@apache.org> wr=
ote:
>>=20
>> Let's start with planning for 4.8.1 then. It should be independent of 4.=
9.0
>> plans.
>>=20
>> The burden is on committers, that they have to commit all bug fixes to t=
he
>> 4.8 branches as well (not just master and 4.9 branches as previously). T=
he
>> RM for 4.8.1 can spend a couple of extra cycles each day to gently remin=
d
>> the committers to do the backport or RM can do the cherry-pick. In HBase
>> for example, it is a combination of both that RM can actively look for b=
ug
>> fixes to backport, but committers also backport themselves if they think
>> that it is a good fix. Usually pinging the RM in the issue helps.
>>=20
>> hbase-1.0 is EOL'ed, and I have proposed in two @dev threads to drop 1.0
>> branches. It was lazy consensus, but we kept the branches without deleti=
ng.
>> Shall I go ahead and delete the branches for 4.8-HBase-1.0 and> 4.x-HBas=
e-1.0? If we do it now, 4.8-HBase-1.0 will not be released.
>> Alternatively, we can do it after the 4.8 release so that 4.9+ will be
>> 1.1+.
>>=20
>> Enis
>>=20
>>> On Thu, Jul 7, 2016 at 7:59 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote=
:
>>>=20
>>> +1 for patch releases on a more frequent cadence. I volunteered to do a
>>> 4.7.x line in support of my own requirements. I think folks are settlin=
g
>> on
>>> a Phoenix minor release for longer periods, and they'll benefit from
>>> receiving bug fixes along the way. Defining and enforcing compatibility
>> is
>>> a part of that.
>>>=20
>>> On the branch issue, I think we should re-consider the compatibility
>> shim.
>>> This is working well for HBase on HDFS, it seems like a reasonable
>> approach
>>> for Phoenix on HBase.
>>>=20
>>>> For 4.9, can we drop support for HBase 1.0 (and perhaps 1.1), since
>> those
>>> HBase releases are EOL'd?
>>>=20
>>> 1.1 isn't EOL'd yet, but we're working in that direction. Maybe for 4.1=
0?
>>> How long does Phoenix want to support users who are sticking to 1.1?
>> Maybe
>>> Phoenix moving on is a good forcing function to get folks upgrading to
>>> 1.2+.
>>>=20
>>> On Thu, Jul 7, 2016 at 10:59 AM, Jan Fernando <jfernando@salesforce.com=
>
>>> wrote:
>>>=20
>>>> As a consumer of Phoenix, moving to where we have regular patch
>> releases
>>> on
>>>> a predicable cadence that contain bug fixes would be incredibly
>>> beneficial.
>>>> For the most recent few releases we have only needed bug fixes and wer=
e
>>> not
>>>> dependent on any of the new features. Therefore having to wait until a
>>>> release with large feature changes stabilizes adds a lot of risk for
>> us.
>>>>=20
>>>> So I am +1 on Lars' proposal.
>>>>=20
>>>> --Jan
>>>>=20
>>>>> On Thu, Jul 7, 2016 at 2:44 AM, <larsh@apache.org> wrote:
>>>>>=20
>>>>> Agreed. On all counts.
>>>>> There are some bigger changes in the pipeline (column remapping and
>>>>> "dense" column storage).Those are powerful and combine the power of
>> SQL
>>>> and
>>>>> HBase quite nicely. I propose we get those soon after 4.8 and then
>>> spin a
>>>>> 4.9 from those.
>>>>>=20
>>>>> A 4.8.1 would be of great value as well. I think Phoenix has reached
>>> that
>>>>> level of maturity now (it's part of Hortonwork, and now finally in
>>>>> Cloudera).To drive adoption and "satisfaction" now I think we need to
>>>>> provide a stable release.
>>>>>=20
>>>>> Questions:
>>>>> - Can we do both a 4.8.1 and 4.9?- For 4.9, can we drop support for
>>> HBase
>>>>> 1.0 (and perhaps 1.1), since those HBase releases are EOL'd?- Maybe
>> for
>>>> 4.9
>>>>> we only support 0.98 and 1.2? That would reduce the number of
>> branches
>>> to
>>>>> maintain.
>>>>> - Support HBase 1.3? If we have a release in time.- If the two main
>>>>> features above are the only "major" changes in 4.9, do we need a
>> 4.8.1?
>>>>>=20
>>>>> If so we need to cover the
>>>>> following:4.8.1-0.984.8.1-1.04.8.1-1.14.8.1-1.24.9-0.984.9-1.1
>>> (perhaps)
>>>>> 4.9-1.2
>>>>> 4.9-1.3 (perhaps)
>>>>> With 4.8.1 we could EOL 4.8.I can start RM'ing 4.8.1 as well (the
>> patch
>>>>> releases are usually little effort for the RM, it's mostly the
>>> committers
>>>>> who have to be diligent backporting bugfixes).But at some point I
>> think
>>>>> we'd need 2 RMs at any given time.
>>>>>=20
>>>>> -- Lars
>>>>>=20
>>>>>=C2=A0 =C2=A0 =C2=A0 From: Enis S=C3=B6ztutar <enis@apache.org>
>>>>>=C2=A0 To: "dev@phoenix.apache.org" <dev@phoenix.apache.org>
>>>>>=C2=A0 Sent: Wednesday, July 6, 2016 2:57 PM
>>>>>=C2=A0 Subject: Re: [DISCUSS] RM for next release
>>>>>=20
>>>>> I would really like to get maintenance releases. Release cadence for
>>>> minor
>>>>> releases and patch releases should be orthogonal in theory, but we
>> are
>>>> all
>>>>> human and there are only so many hours in a day. I would opt for
>> doing
>>>>> actual maintenance releases rather than more frequent minor releases.
>>>>> Having smaller changes is definitely good to get a release out the
>>> door,
>>>>> but hbase/phoenix is a database. Nobody on their right mind updates
>>> their
>>>>> database every month.
>>>>>=20
>>>>> +1 for Lars as always.
>>>>>=20
>>>>> Enis
>>>>>=20
>>>>> On Tue, Jul 5, 2016 at 6:51 PM, Andrew Purtell <
>>> andrew.purtell@gmail.com
>>>>>=20
>>>>> wrote:
>>>>>=20
>>>>>> Stating it another way: As far as I know, all bugs found in the
>> 4.7.0
>>>>>> release are going to be fixed with 4.8.0, not a 4.7.1, and there's
>>>> nobody
>>>>>> planning to maintain a 4.7.x line of releases. It was this way with
>>>> 4.6.0
>>>>>> as well, all bug fixes for problems in 4.6.0 were put in the 4.7.0
>>>>> release,
>>>>>> not a 4.6.1 patch release. I don't think there will ever be a 4.6.x
>>>> patch
>>>>>> release.
>>>>>>=20
>>>>>> Like Nick asked, with a monthly release cadence do we see this
>>>> changing?
>>>>>>=20
>>>>>> Let me put forward this thought: It will be easy to hit a monthly
>>>> release
>>>>>> cadence if we treat bug fixes and bigger works like transactions
>>> (4.7)
>>>>> and
>>>>>> local index reimplementations (4.8) differently. We branch
>>>> appropriately
>>>>>> for making patch releases but don't take advantage of them. That's
>>> easy
>>>>> to
>>>>>> change. Commit bug fixes to development heads and
>> maintenance/release
>>>>>> branches both. Cut releases from the maintenance branches monthly.
>>>>> Simple.
>>>>>> When the time comes, just do it. Meanwhile as the bigger things
>> fully
>>>>> bake
>>>>>> do a new minor or even major rev to release them. Bug fixes will
>> not
>>>> have
>>>>>> been held up no matter how long it may have taken for next new
>>> feature
>>>> X
>>>>> to
>>>>>> bake. In exchange, there will be point releases to make. The HBase
>>>>> "branch
>>>>>> RM" model could be helpful for distributing that work.
>>>>>>=20
>>>>>>=20
>>>>>>=20
>>>>>> On Tue, Jul 5, 2016 at 1:27 PM, Andrew Purtell <
>>>> andrew.purtell@gmail.com
>>>>>>=20
>>>>>> wrote:
>>>>>>=20
>>>>>>> But I don't see patch version releases generally. Right? So if
>> you
>>>> look
>>>>>> at
>>>>>>> release history you'd expect new minors not new patches.
>>>>>>>=20
>>>>>>>> On Jul 5, 2016, at 11:32 AM, Nick Dimiduk <ndimiduk@apache.org
>>>=20
>>>>> wrote:
>>>>>>>>=20
>>>>>>>> We already support multiple release code lines via
>>> branch-per-hbase
>>>>>>> version.
>>>>>>>>=20
>>>>>>>>> On Tue, Jul 5, 2016 at 10:05 AM, Andrew Purtell <
>>>>> apurtell@apache.org>
>>>>>>> wrote:
>>>>>>>>>=20
>>>>>>>>> Will under a new monthly cadence the project still produce new
>>>> minor
>>>>>>>>> versions at every release until the community decides to do a
>>>> major
>>>>>>>>> increment, then continue with minors again?
>>>>>>>>>=20
>>>>>>>>> Or is the plan as Nick wonders to support released minor
>>> versions
>>>>>>> longer,
>>>>>>>>> via patch versions?=C2=A0 If so, I suppose this would mean active
>>>>>>> maintenance of
>>>>>>>>> multiple code lines, and, if so, are we considering or should
>> we
>>>>>>> consider
>>>>>>>>> the HBase "branch RM" style management for that?
>>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>>> On Tue, Jul 5, 2016 at 9:53 AM, Nick Dimiduk <
>>>> ndimiduk@apache.org>
>>>>>>> wrote:
>>>>>>>>>>=20
>>>>>>>>>> Is this thread to discuss Lars for RM, for moving to a
>> monthly
>>>>>> release
>>>>>>>>>> cadence, or propose specific JIRAs for the next release? One
>>> the
>>>>>> above:
>>>>>>>>>>=20
>>>>>>>>>> +1 for Lars, he knows how to make releases happen :)
>>>>>>>>>>=20
>>>>>>>>>> Is this monthly cadence for patch releases? So far this
>>> community
>>>>>>> hasn't
>>>>>>>>>> seen fit to make patch releases, so I'm wondering what's
>>> changed
>>>>> now.
>>>>>>> Are
>>>>>>>>>> we thinking the rate of change in the product has
>>>> slowed/stabilized
>>>>>> and
>>>>>>>>> now
>>>>>>>>>> we're going to support released versions longer? Have we
>>> decided
>>>> on
>>>>>>>>> policy
>>>>>>>>>> re: what makes a change suitable for a patch release vs. the
>>> next
>>>>>>> minor?
>>>>>>>>>>=20
>>>>>>>>>> Re: these tickets, those all look like good improvements and
>>>> fixes
>>>>> to
>>>>>>> get
>>>>>>>>>> shipped. Hopefully the last two would qualify as patch
>> release
>>>>>>> material.
>>>>>>>>>>=20
>>>>>>>>>> On Sat, Jul 2, 2016 at 12:16 AM, James Taylor <
>>>>>> jamestaylor@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>=20
>>>>>>>>>>> Lars has bravely volunteered to be our RM for our next
>> release
>>>>> with
>>>>>> an
>>>>>>>>>> aim
>>>>>>>>>>> to help us get on a monthly release cadence. Big +1 from me.
>>> We
>>>>>> have a
>>>>>>>>>> few
>>>>>>>>>>> features teed up on the encodecolumns branch:
>>>>>>>>>>>=20
>>>>>>>>>>> PHOENIX-1598 Encode column names to save space and improve
>>>>>> performance
>>>>>>>>>>> PHOENIX-2565 Store data for immutable tables in single
>>> KeyValue
>>>>>>>>>>>=20
>>>>>>>>>>> These have both been implemented in a b/w compatible manner.
>>>>>> Existing
>>>>>>>>>>> tables would continue to work as-is and new tables would
>> take
>>>>>>> advantage
>>>>>>>>>> of
>>>>>>>>>>> these new formats.
>>>>>>>>>>>=20
>>>>>>>>>>> A couple of other important JIRAs that I know about are:
>>>>>>>>>>>=20
>>>>>>>>>>> PHOENIX-2995 Write performance severely degrades with large
>>>> number
>>>>>> of
>>>>>>>>>> views
>>>>>>>>>>> PHOENIX-2724 Query with large number of guideposts is slower
>>>>>> compared
>>>>>>>>> to
>>>>>>>>>> no
>>>>>>>>>>> stats
>>>>>>>>>>>=20
>>>>>>>>>>> Hopefully these can make it in, but it'd be up to the
>>> digression
>>>>> of
>>>>>>> the
>>>>>>>>>> RM,
>>>>>>>>>>> of course.
>>>>>>>>>>>=20
>>>>>>>>>>> Thanks,
>>>>>>>>>>> James
>>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>> --
>>>>>>>>> Best regards,
>>>>>>>>>=20
>>>>>>>>>=C2=A0 - Andy
>>>>>>>>>=20
>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
>>>> Piet
>>>>>> Hein
>>>>>>>>> (via Tom White)
>=20
>=20

  
------=_Part_1437980_904055.1468143739488--