incubator-blur-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aaron McCurry <>
Subject Re: RPC question - Time for a change?
Date Fri, 14 Sep 2012 14:36:20 GMT
As a follow up I decided to create a little sample API that would be
exposed to the clients (the controller).

//Table information
list<string> clusterList()
list<string> tableList(1:string cluster)
TableDescriptor describe(1:string table)
TableSchema schema(1:string table)

// Takes the place of fetch and query

// A non blocking call that starts the query and
// return a handle to the query.
QuerySession query(1:Query query)

// A blocking call that will fetch the first query result,
// and subsequent would iterate through the results.
QueryResult fetchQueryResult(1:QuerySession session)

// Closes the query and releases resources, there will have
// to be an automatic session clean up on the server side
// incase close is not called.
void closeQuerySession(1:QuerySession session)

// Tries to query the query in progress
bool cancelQuery(1:QuerySession session)

// Gets the status of a running query, current
// result counts, percentage complete, etc
QueryStatus queryStatus(1:QuerySession session)

// Term, TermDoc, and TermPosition iteration

TermInfoResultsSession termInfo(1:TermInfo info)
TermInfoResult fetchNextTermInfoResult(1:TermInfoResultsSession session)
void closeTermInfoSession(1:TermInfoResultsSession session)

// Mutation session, I am less sure of this section.
// But with this API, we could support transaction like semantics.

MutateSession mutateSession()
void mutate(1:Mutation mutate)
void close(1:MutateSession session)

On Fri, Sep 14, 2012 at 9:56 AM, Aaron McCurry <> wrote:

> In recent weeks I have had several discussions with developers over the
> complexity of the RPC and how we need to make it easier to use.  I agree,
> but I have struggling to come up with a way to make things easier to use
> while still having all the components necessary to allow all the different
> functions to operate.
> To date the Blur shard server and the Blur controller server have shared
> the same Thrift service API, perhaps that should change.  A lot of the
> extra pieces to the API have come from this fact.  Basically the
> controllers needs to be able to pass different information to shard servers
> than the clients need to pass to the Blur controllers.
> The change that I am proposing, split the API into the 3 services.
> - User / Client API (for the controller)
> - Shard Server API
> - Admin API
> One pro to this design is that changes to the internal API (Shard Server)
> could be changed and have less of an impact to the clients if any.
> The only drawback to this approach is that the controller has to be
> deployed, even for testing.  Currently you can just deploy a single shard
> server and have the same features as a whole cluster.  We could get around
> this by allowing the controller API (and the admin API) to be run in the
> same process as the shard server so that the same small scale testing could
> occur.  This then begs the question, do we need separate controller
> processes?  Should we allow for all 3 API types to always be in every
> process?  or should we keep things the way they are?  or should we just
> allow users to configure things they way they want?
> Aaron

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message