phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3860) Implement TAL functionality for Omid
Date Thu, 01 Mar 2018 22:32:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382782#comment-16382782
] 

James Taylor commented on PHOENIX-3860:
---------------------------------------

bq. Could you please refer me to this code? We also have problem with the shadow cells in
this case.
The server-side code that does local index population is in UngroupedAggregateRegionObserver
here:
{code}
                        } else if (buildLocalIndex) {
                            for (IndexMaintainer maintainer : indexMaintainers) {
                                if (!results.isEmpty()) {
                                    result.getKey(ptr);
                                    ValueGetter valueGetter =
                                            maintainer.createGetterFromKeyValues(
                                                ImmutableBytesPtr.copyBytesIfNecessary(ptr),
                                                results);
                                    Put put = maintainer.buildUpdateMutation(kvBuilder,
                                        valueGetter, ptr, results.get(0).getTimestamp(),
                                        env.getRegion().getRegionInfo().getStartKey(),
                                        env.getRegion().getRegionInfo().getEndKey());
                                    indexMutations.add(put);
                                }
                            }
                            result.setKeyValues(results);
{code}

bq. Do you create an index using a transaction and commit when you finish populate all the
data table info to the index? What happened if the commit fails? You drop the index table,
take a new fence and start the process again?
No, we don't use a transaction during the initial index population. This population of an
index is done synchronously by default, but you can tack on the ASYNC keyword to the CREATE
INDEX call and then a MR job will asynchronously build it. Either way, the index is only marked
active (and hence used) if the population is completed successfully. If the table is transactional,
then we do start and end a transaction solely for the purpose of getting a transaction ID
to use for the initial population. If the population fails, then it's up to the user to issue
a DROP INDEX and then call CREATE INDEX again (though it'd be nice if we did this cleanup
ourselves). One more detail: as the index is being populated, we also perform the index maintenance
as writes are done to the table. These mutations may lay down delete markers (our own tx delete
marker) for rows that have not yet been populated. This all works out because the mutations
will have a later timestamp than the timestamp of the rows created during the initial population.

For the issue with the garbage collector, can we determine in the coprocessor that the index
is in a "building" state and make sure that no garbage collection is done? Or perhaps the
commit table can be updated right before the index is moved to an active state?

> Implement TAL functionality for Omid
> ------------------------------------
>
>                 Key: PHOENIX-3860
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3860
>             Project: Phoenix
>          Issue Type: Sub-task
>            Reporter: Ohad Shacham
>            Assignee: Ohad Shacham
>            Priority: Major
>
> Implement TAL functionality for Omid in order to be able to use Omid as Phoenix's transaction
processing engine. 
> To support the integration jira [OMID-82] was opened that encapsulates all Omid related
development for Phoenix.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message