accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Donald Miner <dmi...@clearedgeit.com>
Subject Re: OpenTSDB
Date Mon, 12 May 2014 01:45:39 GMT
I've thought about this a lot actually (and finally have some time to sit
down and type it out). This plays into my larger idea of an automated
hbase->accumulo suite. A system that moves data from one to the other is
one piece, the other piece is letting existing hbase applications hit
accumulo instead. Anyways...

The most efficient thing to do is to have Accumulo tablet servers speak
something that a native hbase client can talk to.  What's going on in a
client is pretty complex, so it's not going to be as easy as writing some
interfaces. I think.

The proxy idea is okay, but I think we'll see some performance problems.

The other option is to do it client side, which would mean the applications
would need to be rebuilt with our "special" APIs that look the same as
before, but really behind the scenes are translating it to an accumulo
call. Maybe they'd have to change an import or something.

Maybe there is a larger project here, something like a bigtable API that
Cassandra, HBase, and Accumulo all can sign on to supporting. Or it is just
client-side code. Not sure. Then something like OpenTSDB or zipkin would
just write to the bigtable API and you wouldn't have to write new code.

Lot of options... not sure which one i like. Sorry for rambling.

-d


On Sun, May 11, 2014 at 8:56 PM, Donald Miner <dminer@clearedgeit.com>wrote:

> We are having a hackathon the day of during the conference. Not exclusive
> to the idea of having a separate hack day the day before. I know some of us
> are participating in the conference and can't participate in the hackathon.
>
>
>
> On May 11, 2014, at 8:41 PM, Sean Busbey <busbey@cloudera.com> wrote:
>
> Don,
>
> Yeah I was thinking covering most of the bigtable implementations at the
> source level would be an easy place to start.
>
> Any interest in organizing a hack day near the Accumulo Summit to try to
> get an initial implementation done?
>
> --
> Sean
> On May 11, 2014 7:28 PM, "Donald Miner" <dminer@clearedgeit.com> wrote:
>
>> "Commands" like scan, put, create table, etc.
>>
>> It would respond to hbase thrift and pretend it was hbase.... But it is
>> using accumulo behind the scenes. basically be a translation layer. There
>> are obviously some things that don't translate.
>>
>> I presume you could do something across all of the big table
>> implementations.
>>
>> On May 11, 2014, at 7:33 PM, David Medinets <david.medinets@gmail.com>
>> wrote:
>>
>> Please define "hbase commands".
>>
>>
>> On Sun, May 11, 2014 at 7:29 PM, Donald Miner <dminer@clearedgeit.com>wrote:
>>
>>> Crazy/bad idea I've had before.... we could develop a hbase->accumulo
>>> proxy that receives basic hbase commands and then writes out accumulo.
>>>
>>> On May 11, 2014, at 4:57 PM, Eric Newton <eric.newton@gmail.com> wrote:
>>>
>>> It is being maintained.  I have tried very hard not to modify the core
>>> OpenTSDB to support it.  But, it would be nice if we could define a
>>> storage-independent layer to which it could adhere.
>>>
>>> I don't believe the OTSDB team is interested, but a basic scalable
>>> back-end abstraction would be nice.  As would a standard java build
>>> environment, but that doesn't seem to be wanted, either.
>>>
>>> We could work towards a common storage abstraction, which would be a
>>> reasonable request.
>>>
>>> Zipkin does a good job of being storage independent.  I would work
>>> towards their model.
>>>
>>> -Eric
>>>
>>> On May 11, 2014 10:28 AM, "Arshak Navruzyan" <arshakn@gmail.com> wrote:
>>>
>>>> I noticed Eric Newton's opentsdb adapter for Accumulo.  Is this still
>>>> being maintained?
>>>>
>>>> https://github.com/ericnewton/accumulo-opentsdb
>>>>
>>>> Also wondering if the StumbleUpon folks are willing to merge it in as
>>>> an alternative to HBase back end.
>>>>
>>>> Thanks
>>>>
>>>> Arshak
>>>>
>>>
>>


-- 

Donald Miner
Chief Technology Officer
ClearEdge IT Solutions, LLC
Cell: 443 799 7807
www.clearedgeit.com

Mime
View raw message