accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From keith-turner <...@git.apache.org>
Subject [GitHub] accumulo pull request #224: ACCUMULO-4500 ACCUMULO-96 Added summarization
Date Mon, 13 Mar 2017 20:16:46 GMT
Github user keith-turner commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/224#discussion_r105760463
  
    --- Diff: core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
---
    @@ -1661,4 +1676,138 @@ public Locations locate(String tableName, Collection<Range>
ranges) throws Accum
     
         return new LoctionsImpl(binnedRanges);
       }
    +
    +  @Override
    +  public SummaryRetriever getSummaries(String tableName) {
    +
    +    return new SummaryRetriever() {
    +
    +      private Text startRow = null;
    +      private Text endRow = null;
    +      private List<TSummarizerConfiguration> summariesToFetch = Collections.emptyList();
    +      private String summarizerClassRegex;
    +      private boolean flush = false;
    +
    +      @Override
    +      public SummaryRetriever startRow(Text startRow) {
    +        Objects.requireNonNull(startRow);
    +        if (endRow != null) {
    +          Preconditions.checkArgument(startRow.compareTo(endRow) < 0, "Start row must
be less than end row : %s >= %s", startRow, endRow);
    +        }
    +        this.startRow = startRow;
    +        return this;
    +      }
    +
    +      @Override
    +      public SummaryRetriever startRow(CharSequence startRow) {
    +        return startRow(new Text(startRow.toString()));
    +      }
    +
    +      @Override
    +      public List<Summary> retrieve() throws AccumuloException, AccumuloSecurityException,
TableNotFoundException {
    +        String tableId = Tables.getTableId(context.getInstance(), tableName);
    +        if (Tables.getTableState(context.getInstance(), tableId) == TableState.OFFLINE)
    +          throw new TableOfflineException(context.getInstance(), tableId);
    +
    +        TRowRange range = new TRowRange(TextUtil.getByteBuffer(startRow), TextUtil.getByteBuffer(endRow));
    +        TSummaryRequest request = new TSummaryRequest(tableId, range, summariesToFetch,
summarizerClassRegex);
    +        if (flush) {
    +          _flush(tableId, startRow, endRow, true);
    +        }
    +
    +        ClientContext cct = new ClientContext(context.getInstance(), context.getCredentials(),
context.getConfiguration()) {
    +          @Override
    +          public long getClientTimeoutInMillis() {
    +            return Math.max(super.getClientTimeoutInMillis(), 60 * 60 * 1000);
    --- End diff --
    
    > Correct me if I'm wrong, but don't we usually do that for client-facing RPCs?
    
    That is a good point.  I think so.
    
    > My worry is that if we do this inside the TabletServer, the risk is larger since
that is a Java process which we expect to run for days/months/indefinitely.
    
    One thing I am unsure about is the relationship between the timeout and TCP keepalive.
 From the research I have done this morning and previously, I think the timeout works mechanism
above the socket and just closes it if the operation takes longer than expected (even if the
remote machine is fine and working).  In this case I would rather wait indefinitely and rely
on something like TCP keep alive mechanism to close the socket if the remote machine is unresponsive.
 I am going to do an experiment where I kill a tserver that I am waiting on with no timeout
and see what happens (if something is enabling tcp keepalive, the socket should still be closed
eventually causing a retry).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message