phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-1954) Reserve chunks of numbers for a sequence
Date Thu, 09 Jul 2015 04:58:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14619891#comment-14619891
] 

James Taylor commented on PHOENIX-1954:
---------------------------------------

Excellent work, [~jfernando_sfdc]. Thanks for outlining the changes as that's super helpful.
For the CYCLE error check, just move this logic up before the return in SequenceRegionObserver
when VALIDATE is the operation and we'll catch that at validate time (which is what we really
want):
{code}
+					// Bulk Allocations are expressed by NEXT <n> VALUES FOR
+					if (SequenceUtil.isBulkAllocation(numSlotsToAllocate)) {
+					    if (cycle) {
+	                        // We don't support Bulk Allocations on sequences that have the
CYCLE flag set to true
+					        return getErrorResult(row, maxTimestamp, SQLExceptionCode.NUM_SEQ_TO_ALLOCATE_NOT_SUPPORTED.getErrorCode());
{code}
So move that check (and the KeyValue cycleKV = Sequence.getCycleKV(result) call) before this
return:
{code}
                if (validateOnly) {
                    return result;
                }
{code}

For (1), it's ok to get the numToAllocate from the cache if there are enough values there.
Do this check instead of the == check:
{code}
         if (value.currentValue + numToAllocate * value.incrementBy > value.nextValue)
{
             if (op == ValueOp.VALIDATE_SEQUENCE) {
                 return value.currentValue;
             }
             throw EMPTY_SEQUENCE_CACHE_EXCEPTION;
         }    
{code}

Good catch on (2). That seems right to me too.


> Reserve chunks of numbers for a sequence
> ----------------------------------------
>
>                 Key: PHOENIX-1954
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1954
>             Project: Phoenix
>          Issue Type: New Feature
>            Reporter: Lars Hofhansl
>            Assignee: Jan Fernando
>         Attachments: PHOENIX-1954-rebased.patch, PHOENIX-1954-wip.patch, PHOENIX-1954-wip2.patch.txt,
PHOENIX-1954-wip3.patch, PHOENIX-1954-wip4.patch, PHOENIX-1954-wip5-rebased.patch
>
>
> In order to be able to generate many ids in bulk (for example in map reduce jobs) we
need a way to generate or reserve large sets of ids. We also need to mix ids reserved with
incrementally generated ids from other clients. 
> For this we need to atomically increment the sequence and return the value it had when
the increment happened.
> If we're OK to throw the current cached set of values away we can do
> {{NEXT VALUE FOR <seq>(,<N>)}}, that needs to increment value and return
the value it incremented from (i.e. it has to throw the current cache away, and return the
next value it found at the server).
> Or we can invent a new syntax {{RESERVE VALUES FOR <seq>, <N>}} that does
the same, but does not invalidate the cache.
> Note that in either case we won't retrieve the reserved set of values via {{NEXT VALUE
FOR}} because we'd need to be idempotent in our case, all we need to guarantee is that after
a call to {{RESERVE VALUES FOR <seq>, <N>}}, which returns a value <M> is
that the range [M, M+N) won't be used by any other user of the sequence. My might need reserve
1bn ids this way ahead of a map reduce run.
> Any better ideas?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message