hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: HDFS-1599 status? (HDFS tickets to improve HBase)
Date Thu, 09 Jun 2011 21:02:42 GMT
HDFS-941 just got committed to TRUNK in a coordinated effort led by
our man Todd (Hopefully it makes it into 0.22!).  HDFS-1148 is next!

On Sun, Jun 5, 2011 at 10:09 PM, Todd Lipcon <todd@cloudera.com> wrote:
> OK guys, I did my part: I rebased HDFS-941 and HDFS-1148 to trunk. Also
> attempted to rebase HDFS-1323 (nee 918) but it's failing tests.
> If you want to help, please take a look at the failing tests on HDFS-1323
> and see if you can understand what might be going wrong.
> Unfortunately this isn't my top priority at work right now (HA for the
> namenode is), but I'm happy to spend some nights and weekends to help push
> these through if they really work.
> -Todd
> On Sun, Jun 5, 2011 at 3:40 PM, Todd Lipcon <todd@cloudera.com> wrote:
>> On Sat, Jun 4, 2011 at 1:46 AM, Andrew Purtell <apurtell@apache.org>wrote:
>>> This is not discouraging. :-)
>>> HBasers patch CDH because trunk -- anything > 0.20 actually -- is not
>>> trusted by consensus if you look at all of the production deployments. Does
>>> ANYONE run trunk under anything approaching "production"? And trunk/upstream
>>> has a history of ignoring any HBase specific concern. So the use of and
>>> trading of patches will probably continue for a while, maybe forever.
>> Right - I wasn't suggesting that you run trunk in production as of yet. But
>> there has been very little activity in terms of HBase people running trunk
>> in dev/test clusters in the past. Stack has done some awesome work here in
>> the last few weeks, so that should open it up for some more people to jump
>> on board.
>> I agree that HBase has been treated as a second-class citizen in recent
>> years from HDFS's performance, but I think that has changed. All of the
>> major HDFS contributors now have serious stakes in HBase, and so long as
>> there are tests with sufficient testing that apply against trunk, I don't
>> see a reason they wouldn't be included.
>>> Part of the problem is the expectation that any patch provided against
>>> trunk may generate months of back and forth, as we have seen, which presents
>>> difficulities to a potential contributor who does not work on e.g. HDFS
>>> matters full time. Alternatively it may pick up a committer as sponsor and
>>> then be vetoed by Yahoo because they're mad at Cloudera over some unrelated
>>> issue and a patch appears to have a Cloudera sponsor and/or or vice versa.
>>> Now, that situation I describe _is_ discouraging. It's not enough to say
>>> that we must contribute through trunk. Trunk needs to earn back our trust.
>> Yes, there have been some unfortunate things in the past. There have also
>> been some half-finished or untested patches proposed, and you can't blame
>> HDFS folks for not taking a big patch that doesn't have a lot of confidence
>> behind it.
>> I've been thinking about this this afternoon, and have an idea. It may
>> prove to be an awful one, but maybe it's a good one, only time will tell :)
>> I'll create a branch off of HDFS trunk specifically for HBase performance
>> work. We can commit these "90% done" patches there, which will make it
>> easier for others to test and gain confidence. Branches also can make it
>> easier to maintain patches over time with a changing trunk.
>> How does this sound to the HBase community? If it seems like a good idea,
>> *and* there are some people who would be willing to set it up on some small
>> dev clusters and run load tests, I'll move forward with it.
>>> I believe I recently saw discussion that append should be removed or
>>> disabled by default on 0.22 or trunk. Did you see anything like this? If I
>>> am mistaken, fine. If not, this is going in the wrong direction, for
>>> example.
>> Not sure what you're referring to - I don't remember any discussion like
>> this.
>> -Todd
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
> --
> Todd Lipcon
> Software Engineer, Cloudera

View raw message