accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Vines (Updated) (JIRA)" <>
Subject [jira] [Updated] (ACCUMULO-482) Add a thrift proxy server
Date Fri, 23 Mar 2012 03:16:22 GMT


John Vines updated ACCUMULO-482:

    Priority: Major  (was: Minor)

We've (the team) discussed this quite a bit offline. While one can simply write a translation
API, there is something else we would like to accomplish with a proxy. With the column visibilities,
we contend that any part of the Key/Value pair could be the piece that is worth restricting.
And we attempt to uphold this to teh highest degree. The one place where this falters is in
the !METADATA table. Whenever we split, we base the split point off of an existing row ID
in the table. And if the reason a column/value pair is put in a particular visiblity is because
of the row part of the key, then we're potentially leaking it. All clients must be able to
read the !METADATA table in order to determine what tablet (and tserver) they need to hit
to get their next piece of information.

That said, we would actually like to make server side proxies. We've never really concluded
on tserver thread vs. independent process, I think that's still up in the air. But regardless,
what we should do is have all clients come in through this proxy service, and it will handle
all of the !METADATA table lookups and funnel all of the data through itself back to the client.
This can prevent a possible attack to guesstimate where the tablets split (Adam can talk more
about this). We should have it utilize the current client API, as we still want to use it
for some operations (MR InputForamat off the top of my head).

However, aside from this, it should work as a proxy for all other purposes as this is intended.
We can simplify the API for the client user, letting all of the !METADATA lookups occur on
the proxy, we can do all of the error translation, etc. This means we could have a much, MUCH
lighter client api that does easily trascend languages.

As for the language, I don't really care. We use thrift, so it would allow for some object
reuse (but then we will have to reimplement some objects, like Key, Value, and Mutation, in
the various languages I think). But I wouldn't object to seeing something else used like Avro
or any of those other similar projects. However, I don't really see the need for the flexibility
for Avro, but I think it offers more than just that. Just my two cents.

I'll get off my soap box now.
> Add a thrift proxy server
> -------------------------
>                 Key: ACCUMULO-482
>                 URL:
>             Project: Accumulo
>          Issue Type: New Feature
>            Reporter: Sapan Shah
>            Assignee: Sapan Shah
> Add a thrift proxy server to make integration with other languages besides Java a bit
easier.  This should work like

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message