accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jorge Machado <jorge.w.mach...@hotmail.com>
Subject Re: [DISCUSS] Hadoop3 support target?
Date Tue, 05 Dec 2017 22:37:18 GMT
About this Client config is a Mess. Why can’t we just do it like in hdfs where for the client
to run it needs or to pass that I to the configuration object or we just read it from a resource.
My ideia is to get rid of this client.conf

Jorge Machado
Jorge@jmachado.me<mailto:Jorge@jmachado.me>


Am 05.12.2017 um 21:29 schrieb Keith Turner <keith@deenlo.com<mailto:keith@deenlo.com>>:

On Tue, Dec 5, 2017 at 2:53 PM, Josh Elser <elserj@apache.org<mailto:elserj@apache.org>>
wrote:
Interesting. What makes you want to deprecate ClientConfig entirely?

I'd be worried about removing without sufficient thought of replacement
around. It would be a bit "churn-y" to introduce yet another way that
clients have to connect (since it was introduced in 1.6-ish?). Working
around the ClientConfig changes was irritating for the downstream
integrations (Hive, most notably).

Ok maybe thats a bad idea, not looking to cause pain.  Here were some
of my goals.

* Remove commons config from API completely via deprecation cycle.
* Introduce API that supports putting all props needed to connect to
Accumulo in an API.

I suppose if we want to keep ClientConfig class in API, then there is
no way to remove commons config via a deprecation cycle??  We can't
deprecate the extension of commons config, all we can do is just drop
it at some point.



On 12/5/17 1:13 PM, Keith Turner wrote:

I was thinking of a slightly different path forward.

 * Add new entry point and deprecate clientconfig in 1.9
 * Branch 1.9 off 1.8
 * Stop releasing 1.8.x in favor of 1.9.x (they are the same except
for new API)
 * Release 1.9 ASAP
 * Drop clientconfig in 2.0.0
 * Release 2.0.0 early next year... maybe target March

On Tue, Dec 5, 2017 at 12:51 PM, Josh Elser <elserj@apache.org<mailto:elserj@apache.org>>
wrote:

Ok, a bridge version seems to be a general path forward. Generally this
would be...

* 1.8 gets relevant commons-config classes/methods deprecated
* 1.9 is 1.8 with those deprecation points removed
* 1.9 has commons-config shaded (maybe?)

IMO, it's critical that we remove the commons-config stuff from our
public
API (shame this somehow was let in to begin).

I think shading our use of commons-config would be a good idea and lessen
our ClientConfiguration scope to being able to read from a file. Trying
to
support the breadth of what commons-configuration can do will just get us
into more trouble.


On 12/5/17 12:18 PM, Keith Turner wrote:


If we are going to deprecate, then it would be nice to have a
replacement.  One thing that has irked me about the current Accumulo
entry point is that one can not specify everything needed to connect
to in a single props file.  Specifically, credentials can not be
specified.  It would be really nice to have a new entry point that
allows this.

We could release a 1.9 bridge version.  This version would be based on
1.8 and only include a new entry point. Base it on 1.8 in order to
allow a low risk upgrade for anyone currently using 1.8.  Once people
start using 1.9 they can have code that uses the old and new entry
point running at the same time.  In 2.0 we can drop the problematic
entry point.

Below is a commit to 1.8 where I was experimenting with a new entry
point.



https://github.com/keith-turner/accumulo/commit/1c07fa62e9c57bde7e60907595d50f898d03c9d5

This new API would need review, its rough and there are some things I
don't like about it.  Just sharing for discussion of general concept,
not advocating for this specific API.

On Mon, Dec 4, 2017 at 6:27 PM, Dave Marion <dmarion18@gmail.com<mailto:dmarion18@gmail.com>>
wrote:


There is no reason that you can't mark the offending API methods as
deprecated in a 1.8.x release, then immediately branch off of that to
create
a 2.0 and remove the method. Alternatively, we could decide to forego
the
semver rules for a specific release and make sure to point it out in
the
release notes.

-----Original Message-----
From: Josh Elser [mailto:elserj@apache.org]
Sent: Monday, December 4, 2017 6:19 PM
To: dev@accumulo.apache.org<mailto:dev@accumulo.apache.org>
Subject: Re: [DISCUSS] Hadoop3 support target?

Also, just to be clear for everyone else:

This means that we have *no roadmap* at all for Hadoop 3 support
because
Accumulo 2.0 is in a state of languish.

This is a severe enough problem to me that I would consider breaking
API
compatibility and fixing the API leak in 1.7/1.8. I'm curious what
people
other than Christopher think (assuming from his comments/JIRA work that
he
disagrees with me).

On 12/4/17 6:12 PM, Christopher wrote:


Agreed.

On Mon, Dec 4, 2017 at 6:01 PM Josh Elser <elserj@apache.org<mailto:elserj@apache.org>>
wrote:

Ah, I'm seeing now -- didn't check my inbox appropriately.

I think the fact that code that we don't own has somehow been allowed
to be public API is the smell. That's something that needs to be
rectified sooner than later. By that measure, it can *only* land on
Accumulo 2.0 (which is going to be a major issue for the project).

On 12/4/17 5:58 PM, Josh Elser wrote:


Sorry, I don't follow. Why do you think 4611/4753 is a show-stopper?
Cuz, uh... I made it work already :)

Thanks for the JIRA cleanup. Forgot about that one.

On 12/4/17 5:55 PM, Christopher wrote:


I don't think we can support it with 1.8 or earlier, because of
some serious incompatibilities (namely, ACCUMULO-4611/4753) I think
people are still patching 1.7, so I don't think we've "officially"
EOL'd it.
I think 2.0 could require Hadoop 3, if Hadoop 3 is sufficiently
stable.

On Mon, Dec 4, 2017 at 1:14 PM Josh Elser <elserj@apache.org<mailto:elserj@apache.org>>
wrote:

What branch do we want to consider Hadoop3 support?

There is a 3.0.0-beta1 release that's been out for a while, and
Hadoop PMC has already done a 3.0.0 RC0. I think it's the right
time to start considering this.

In my poking so far, I've filed ACCUMULO-4753 which I'm working
through now. This does raise the question: where do we want to say
we support Hadoop3? 1.8 or 2.0? (have we "officially" deprecated
1.7?)

- Josh

https://issues.apache.org/jira/browse/ACCUMULO-4753







Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message