accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From keith-turner <...@git.apache.org>
Subject [GitHub] accumulo pull request #224: ACCUMULO-4500 ACCUMULO-96 Added summarization
Date Wed, 01 Mar 2017 20:38:37 GMT
Github user keith-turner commented on a diff in the pull request:

    https://github.com/apache/accumulo/pull/224#discussion_r103783042
  
    --- Diff: core/src/main/java/org/apache/accumulo/core/client/impl/TableOperationsImpl.java
---
    @@ -1661,4 +1676,138 @@ public Locations locate(String tableName, Collection<Range>
ranges) throws Accum
     
         return new LoctionsImpl(binnedRanges);
       }
    +
    +  @Override
    +  public SummaryRetriever getSummaries(String tableName) {
    +
    +    return new SummaryRetriever() {
    +
    +      private Text startRow = null;
    +      private Text endRow = null;
    +      private List<TSummarizerConfiguration> summariesToFetch = Collections.emptyList();
    +      private String summarizerClassRegex;
    +      private boolean flush = false;
    +
    +      @Override
    +      public SummaryRetriever startRow(Text startRow) {
    +        Objects.requireNonNull(startRow);
    +        if (endRow != null) {
    +          Preconditions.checkArgument(startRow.compareTo(endRow) < 0, "Start row must
be less than end row : %s >= %s", startRow, endRow);
    +        }
    +        this.startRow = startRow;
    +        return this;
    +      }
    +
    +      @Override
    +      public SummaryRetriever startRow(CharSequence startRow) {
    +        return startRow(new Text(startRow.toString()));
    +      }
    +
    +      @Override
    +      public List<Summary> retrieve() throws AccumuloException, AccumuloSecurityException,
TableNotFoundException {
    +        String tableId = Tables.getTableId(context.getInstance(), tableName);
    +        if (Tables.getTableState(context.getInstance(), tableId) == TableState.OFFLINE)
    +          throw new TableOfflineException(context.getInstance(), tableId);
    +
    +        TRowRange range = new TRowRange(TextUtil.getByteBuffer(startRow), TextUtil.getByteBuffer(endRow));
    +        TSummaryRequest request = new TSummaryRequest(tableId, range, summariesToFetch,
summarizerClassRegex);
    +        if (flush) {
    +          _flush(tableId, startRow, endRow, true);
    +        }
    +
    +        ClientContext cct = new ClientContext(context.getInstance(), context.getCredentials(),
context.getConfiguration()) {
    +          @Override
    +          public long getClientTimeoutInMillis() {
    +            return Math.max(super.getClientTimeoutInMillis(), 60 * 60 * 1000);
    --- End diff --
    
    Ugh.  Glad you found this.  I meant to circle back to this, but forgot about it.  When
I did this hack I Was worried about the case where getting summaries is slow (because none
are in cache) and a result the client times and initiates a new operation (while threads for
old operation are still churning).  I need a better solution.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message