phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <la...@apache.org>
Subject Re: [DISCUSS] RM for next release
Date Sun, 10 Jul 2016 09:42:19 GMT
0.94 never took that much time from me.Sure some days it's a lot of work, but on average I
found it a task that can be easily done on the side.
I relied a lot on the Apache Jenkins, triggering runs and checking back the next day, etc.I
also think you're more diligent in your RM duties :)

-- Lars

      From: Andrew Purtell <andrew.purtell@gmail.com>
 To: dev@phoenix.apache.org 
 Sent: Saturday, July 9, 2016 9:02 AM
 Subject: Re: [DISCUSS] RM for next release
   
Huh. RMing HBase 0.98 consumes whole days with testing and patching. YMMV 

> On Jul 9, 2016, at 6:55 AM, <larsh@apache.org> <larsh@apache.org> wrote:
> 
> +1 on 4.8.x should support HBase 1.0. (although it would safe us maintenance of 
> For 4.9 we can drop that (but likely should pull in support for HBase 1.3).We'll need
to continue with 0.98 support until 0.98 is EOL'd, I think.
> 
> I'm happy to do 4.8.1 and 4.9 for one release, to confirm Stack's nickname for me: "Iron
Hand" :)After the first release more folks should tag team. RM'ing is actually not that much
work, maybe 15-30 mins/day.
> 
> I'm actually quire exited about the potential of 4.9. We'll bring more of the power of
knowing the schema together with the proven stability of HBase:
> PHOENIX-1598 Encode column names to save space and improve performance
> PHOENIX-2565 Store data for immutable tables in single KeyValue
> The trick will be to add these in an absolutely backwards compatible way (where an old
4.8 client will be able to run against a 4.9 server indefinitely).We may have to start putting
a schema version number into the SYSTEM.CATALOG if we do not have this already.
> 
> I'll propose merging those into 4.9 early (assuming they are ready and stable - so that's
something to ensure now in their resp branches).Bug fixes would go into both 4.9 and 4.8.1.
> 
> As soon as 4.8 is out (hopefully soon now), I'll propose the branches and create them
once we all agree.
> 
> -- Lars
> 
> ps. There's no 4.8-HBase-1.0, yet, right?! At least I don't see it.
> 
>      From: James Taylor <jamestaylor@apache.org>
> To: "dev@phoenix.apache.org" <dev@phoenix.apache.org> 
> Sent: Friday, July 8, 2016 3:01 PM
> Subject: Re: [DISCUSS] RM for next release
> 
> I think we should go ahead with the 4.8 release for HBase 1.0, but not do
> one for 4.9. As far as I recall, there were some objections about not doing
> an HBase 1.0 release because CDH was based off of 1.0. Seems like CDH has
> moved past this, though, so it's likely not necessary to continue. Probably
> something we should propose on the user list.
> 
>> On Fri, Jul 8, 2016 at 10:51 PM, Enis Söztutar <enis@apache.org> wrote:
>> 
>> Let's start with planning for 4.8.1 then. It should be independent of 4.9.0
>> plans.
>> 
>> The burden is on committers, that they have to commit all bug fixes to the
>> 4.8 branches as well (not just master and 4.9 branches as previously). The
>> RM for 4.8.1 can spend a couple of extra cycles each day to gently remind
>> the committers to do the backport or RM can do the cherry-pick. In HBase
>> for example, it is a combination of both that RM can actively look for bug
>> fixes to backport, but committers also backport themselves if they think
>> that it is a good fix. Usually pinging the RM in the issue helps.
>> 
>> hbase-1.0 is EOL'ed, and I have proposed in two @dev threads to drop 1.0
>> branches. It was lazy consensus, but we kept the branches without deleting.
>> Shall I go ahead and delete the branches for 4.8-HBase-1.0 and> 4.x-HBase-1.0?
If we do it now, 4.8-HBase-1.0 will not be released.
>> Alternatively, we can do it after the 4.8 release so that 4.9+ will be
>> 1.1+.
>> 
>> Enis
>> 
>>> On Thu, Jul 7, 2016 at 7:59 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
>>> 
>>> +1 for patch releases on a more frequent cadence. I volunteered to do a
>>> 4.7.x line in support of my own requirements. I think folks are settling
>> on
>>> a Phoenix minor release for longer periods, and they'll benefit from
>>> receiving bug fixes along the way. Defining and enforcing compatibility
>> is
>>> a part of that.
>>> 
>>> On the branch issue, I think we should re-consider the compatibility
>> shim.
>>> This is working well for HBase on HDFS, it seems like a reasonable
>> approach
>>> for Phoenix on HBase.
>>> 
>>>> For 4.9, can we drop support for HBase 1.0 (and perhaps 1.1), since
>> those
>>> HBase releases are EOL'd?
>>> 
>>> 1.1 isn't EOL'd yet, but we're working in that direction. Maybe for 4.10?
>>> How long does Phoenix want to support users who are sticking to 1.1?
>> Maybe
>>> Phoenix moving on is a good forcing function to get folks upgrading to
>>> 1.2+.
>>> 
>>> On Thu, Jul 7, 2016 at 10:59 AM, Jan Fernando <jfernando@salesforce.com>
>>> wrote:
>>> 
>>>> As a consumer of Phoenix, moving to where we have regular patch
>> releases
>>> on
>>>> a predicable cadence that contain bug fixes would be incredibly
>>> beneficial.
>>>> For the most recent few releases we have only needed bug fixes and were
>>> not
>>>> dependent on any of the new features. Therefore having to wait until a
>>>> release with large feature changes stabilizes adds a lot of risk for
>> us.
>>>> 
>>>> So I am +1 on Lars' proposal.
>>>> 
>>>> --Jan
>>>> 
>>>>> On Thu, Jul 7, 2016 at 2:44 AM, <larsh@apache.org> wrote:
>>>>> 
>>>>> Agreed. On all counts.
>>>>> There are some bigger changes in the pipeline (column remapping and
>>>>> "dense" column storage).Those are powerful and combine the power of
>> SQL
>>>> and
>>>>> HBase quite nicely. I propose we get those soon after 4.8 and then
>>> spin a
>>>>> 4.9 from those.
>>>>> 
>>>>> A 4.8.1 would be of great value as well. I think Phoenix has reached
>>> that
>>>>> level of maturity now (it's part of Hortonwork, and now finally in
>>>>> Cloudera).To drive adoption and "satisfaction" now I think we need to
>>>>> provide a stable release.
>>>>> 
>>>>> Questions:
>>>>> - Can we do both a 4.8.1 and 4.9?- For 4.9, can we drop support for
>>> HBase
>>>>> 1.0 (and perhaps 1.1), since those HBase releases are EOL'd?- Maybe
>> for
>>>> 4.9
>>>>> we only support 0.98 and 1.2? That would reduce the number of
>> branches
>>> to
>>>>> maintain.
>>>>> - Support HBase 1.3? If we have a release in time.- If the two main
>>>>> features above are the only "major" changes in 4.9, do we need a
>> 4.8.1?
>>>>> 
>>>>> If so we need to cover the
>>>>> following:4.8.1-0.984.8.1-1.04.8.1-1.14.8.1-1.24.9-0.984.9-1.1
>>> (perhaps)
>>>>> 4.9-1.2
>>>>> 4.9-1.3 (perhaps)
>>>>> With 4.8.1 we could EOL 4.8.I can start RM'ing 4.8.1 as well (the
>> patch
>>>>> releases are usually little effort for the RM, it's mostly the
>>> committers
>>>>> who have to be diligent backporting bugfixes).But at some point I
>> think
>>>>> we'd need 2 RMs at any given time.
>>>>> 
>>>>> -- Lars
>>>>> 
>>>>>      From: Enis Söztutar <enis@apache.org>
>>>>>  To: "dev@phoenix.apache.org" <dev@phoenix.apache.org>
>>>>>  Sent: Wednesday, July 6, 2016 2:57 PM
>>>>>  Subject: Re: [DISCUSS] RM for next release
>>>>> 
>>>>> I would really like to get maintenance releases. Release cadence for
>>>> minor
>>>>> releases and patch releases should be orthogonal in theory, but we
>> are
>>>> all
>>>>> human and there are only so many hours in a day. I would opt for
>> doing
>>>>> actual maintenance releases rather than more frequent minor releases.
>>>>> Having smaller changes is definitely good to get a release out the
>>> door,
>>>>> but hbase/phoenix is a database. Nobody on their right mind updates
>>> their
>>>>> database every month.
>>>>> 
>>>>> +1 for Lars as always.
>>>>> 
>>>>> Enis
>>>>> 
>>>>> On Tue, Jul 5, 2016 at 6:51 PM, Andrew Purtell <
>>> andrew.purtell@gmail.com
>>>>> 
>>>>> wrote:
>>>>> 
>>>>>> Stating it another way: As far as I know, all bugs found in the
>> 4.7.0
>>>>>> release are going to be fixed with 4.8.0, not a 4.7.1, and there's
>>>> nobody
>>>>>> planning to maintain a 4.7.x line of releases. It was this way with
>>>> 4.6.0
>>>>>> as well, all bug fixes for problems in 4.6.0 were put in the 4.7.0
>>>>> release,
>>>>>> not a 4.6.1 patch release. I don't think there will ever be a 4.6.x
>>>> patch
>>>>>> release.
>>>>>> 
>>>>>> Like Nick asked, with a monthly release cadence do we see this
>>>> changing?
>>>>>> 
>>>>>> Let me put forward this thought: It will be easy to hit a monthly
>>>> release
>>>>>> cadence if we treat bug fixes and bigger works like transactions
>>> (4.7)
>>>>> and
>>>>>> local index reimplementations (4.8) differently. We branch
>>>> appropriately
>>>>>> for making patch releases but don't take advantage of them. That's
>>> easy
>>>>> to
>>>>>> change. Commit bug fixes to development heads and
>> maintenance/release
>>>>>> branches both. Cut releases from the maintenance branches monthly.
>>>>> Simple.
>>>>>> When the time comes, just do it. Meanwhile as the bigger things
>> fully
>>>>> bake
>>>>>> do a new minor or even major rev to release them. Bug fixes will
>> not
>>>> have
>>>>>> been held up no matter how long it may have taken for next new
>>> feature
>>>> X
>>>>> to
>>>>>> bake. In exchange, there will be point releases to make. The HBase
>>>>> "branch
>>>>>> RM" model could be helpful for distributing that work.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Tue, Jul 5, 2016 at 1:27 PM, Andrew Purtell <
>>>> andrew.purtell@gmail.com
>>>>>> 
>>>>>> wrote:
>>>>>> 
>>>>>>> But I don't see patch version releases generally. Right? So if
>> you
>>>> look
>>>>>> at
>>>>>>> release history you'd expect new minors not new patches.
>>>>>>> 
>>>>>>>> On Jul 5, 2016, at 11:32 AM, Nick Dimiduk <ndimiduk@apache.org
>>> 
>>>>> wrote:
>>>>>>>> 
>>>>>>>> We already support multiple release code lines via
>>> branch-per-hbase
>>>>>>> version.
>>>>>>>> 
>>>>>>>>> On Tue, Jul 5, 2016 at 10:05 AM, Andrew Purtell <
>>>>> apurtell@apache.org>
>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Will under a new monthly cadence the project still produce
new
>>>> minor
>>>>>>>>> versions at every release until the community decides
to do a
>>>> major
>>>>>>>>> increment, then continue with minors again?
>>>>>>>>> 
>>>>>>>>> Or is the plan as Nick wonders to support released minor
>>> versions
>>>>>>> longer,
>>>>>>>>> via patch versions?  If so, I suppose this would mean
active
>>>>>>> maintenance of
>>>>>>>>> multiple code lines, and, if so, are we considering or
should
>> we
>>>>>>> consider
>>>>>>>>> the HBase "branch RM" style management for that?
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On Tue, Jul 5, 2016 at 9:53 AM, Nick Dimiduk <
>>>> ndimiduk@apache.org>
>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Is this thread to discuss Lars for RM, for moving
to a
>> monthly
>>>>>> release
>>>>>>>>>> cadence, or propose specific JIRAs for the next release?
One
>>> the
>>>>>> above:
>>>>>>>>>> 
>>>>>>>>>> +1 for Lars, he knows how to make releases happen
:)
>>>>>>>>>> 
>>>>>>>>>> Is this monthly cadence for patch releases? So far
this
>>> community
>>>>>>> hasn't
>>>>>>>>>> seen fit to make patch releases, so I'm wondering
what's
>>> changed
>>>>> now.
>>>>>>> Are
>>>>>>>>>> we thinking the rate of change in the product has
>>>> slowed/stabilized
>>>>>> and
>>>>>>>>> now
>>>>>>>>>> we're going to support released versions longer?
Have we
>>> decided
>>>> on
>>>>>>>>> policy
>>>>>>>>>> re: what makes a change suitable for a patch release
vs. the
>>> next
>>>>>>> minor?
>>>>>>>>>> 
>>>>>>>>>> Re: these tickets, those all look like good improvements
and
>>>> fixes
>>>>> to
>>>>>>> get
>>>>>>>>>> shipped. Hopefully the last two would qualify as
patch
>> release
>>>>>>> material.
>>>>>>>>>> 
>>>>>>>>>> On Sat, Jul 2, 2016 at 12:16 AM, James Taylor <
>>>>>> jamestaylor@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Lars has bravely volunteered to be our RM for
our next
>> release
>>>>> with
>>>>>> an
>>>>>>>>>> aim
>>>>>>>>>>> to help us get on a monthly release cadence.
Big +1 from me.
>>> We
>>>>>> have a
>>>>>>>>>> few
>>>>>>>>>>> features teed up on the encodecolumns branch:
>>>>>>>>>>> 
>>>>>>>>>>> PHOENIX-1598 Encode column names to save space
and improve
>>>>>> performance
>>>>>>>>>>> PHOENIX-2565 Store data for immutable tables
in single
>>> KeyValue
>>>>>>>>>>> 
>>>>>>>>>>> These have both been implemented in a b/w compatible
manner.
>>>>>> Existing
>>>>>>>>>>> tables would continue to work as-is and new tables
would
>> take
>>>>>>> advantage
>>>>>>>>>> of
>>>>>>>>>>> these new formats.
>>>>>>>>>>> 
>>>>>>>>>>> A couple of other important JIRAs that I know
about are:
>>>>>>>>>>> 
>>>>>>>>>>> PHOENIX-2995 Write performance severely degrades
with large
>>>> number
>>>>>> of
>>>>>>>>>> views
>>>>>>>>>>> PHOENIX-2724 Query with large number of guideposts
is slower
>>>>>> compared
>>>>>>>>> to
>>>>>>>>>> no
>>>>>>>>>>> stats
>>>>>>>>>>> 
>>>>>>>>>>> Hopefully these can make it in, but it'd be up
to the
>>> digression
>>>>> of
>>>>>>> the
>>>>>>>>>> RM,
>>>>>>>>>>> of course.
>>>>>>>>>>> 
>>>>>>>>>>> Thanks,
>>>>>>>>>>> James
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Best regards,
>>>>>>>>> 
>>>>>>>>>  - Andy
>>>>>>>>> 
>>>>>>>>> Problems worthy of attack prove their worth by hitting
back. -
>>>> Piet
>>>>>> Hein
>>>>>>>>> (via Tom White)
> 
> 

  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message