phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geoffrey Jacoby (Jira)" <j...@apache.org>
Subject [jira] [Created] (PHOENIX-5449) Schema-aware Message Consumer
Date Mon, 19 Aug 2019 21:25:00 GMT
Geoffrey Jacoby created PHOENIX-5449:
----------------------------------------

             Summary: Schema-aware Message Consumer
                 Key: PHOENIX-5449
                 URL: https://issues.apache.org/jira/browse/PHOENIX-5449
             Project: Phoenix
          Issue Type: Sub-task
            Reporter: Geoffrey Jacoby
            Assignee: Geoffrey Jacoby


The change data capture ReplicationEndpoint will be persisting data to the message bus using
a low-level, generic schema that includes a pointer to schema information, plus a list of
Cells (like an Avro-encoded WAL Entry). 

This is done both because some low-level use cases such as backup don't the strong typing
until restore-time, but also to simplify the endpoint and improve performance. The endpoint
will be decoupled from any consumer that wants to do higher level processing on any entity
type. 

There will be a need, however, for a message bus consumer that reads in the low-level messages
from the ReplicationEndpoint, looks up the schema using the metadata in the message and the
API in PHOENIX-5443, and then "rehydrates" the low-level message into a higher-level, strongly-typed
message suitable for running business logic on. 





--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message