hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From fding hbase <fding.hb...@gmail.com>
Subject EndPoint Coprocessor could be dealocked?
Date Mon, 14 May 2012 09:35:32 GMT
Hi all,

Is it possible to use table scanner (different from the host table region)
or
execute coprocessor of another table, in the endpoint coprocessor?
It looks like chaining coprocessors. But I found a possible deadlock!
Can anyone help me with this?

In my testing environment I deployed the 0.92.0 version from CDH.
I wrote an Endpoint coprocessor to do composite secondary index queries.
The index is stored in another table and the index update is maintained
by the client through a extended HTable. While a single index query
works fine through Scanners of index table, soon after we realized
we need to do multi-index queries at the same time.
At first we tried to pull every row keys queried from a single index table
and do the merge (just set intersection) on the client,
but that overruns the network bandwidth. So I proposed to try
the endpoint coprocessor. The idea is to use coprocessors, one
in master table (the indexed table) and the other for each index table
regions.
Each master table region coprocessor instance invokes the index table
coprocessor instances with its regioninfo (the startKey and endKey) and the
scan,
the index table region coprocessor instance scans and returns the row keys
within the range of startKey and endKey passed in.

The cluster blocks sometimes in invoking the index table coprocessor. I
traced
into the code and found that when HConnection locates regions it will rpc
to the same regionserver.

(After a while I found the index table coprocessor is equivalent to
just a plain scan with filter, so I switched to scanners with filter, but
the problem
remains.)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message