phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chinmay Kulkarni (Jira)" <j...@apache.org>
Subject [jira] [Created] (PHOENIX-5943) Consider splitting up SYSTEM.TASK into 2 tables - one for queued jobs and one for task history
Date Tue, 09 Jun 2020 00:19:00 GMT
Chinmay Kulkarni created PHOENIX-5943:
-----------------------------------------

             Summary: Consider splitting up SYSTEM.TASK into 2 tables - one for queued jobs
and one for task history
                 Key: PHOENIX-5943
                 URL: https://issues.apache.org/jira/browse/PHOENIX-5943
             Project: Phoenix
          Issue Type: Improvement
    Affects Versions: 4.15.0
            Reporter: Chinmay Kulkarni
             Fix For: 4.16.0


We can split SYSTEM.TASK into 2 tables:
 # SYSTEM.TASK_QUEUE: used as an ephemeral task queue with potentially a smaller TTL which
contains tasks that are new and yet to be picked up.
 # SYSTEM.TASK_HISTORY: used to store tasks that have already been picked up and may or may
not have succeeded, along with more data about the task (stack trace, job IDs, etc.). This
table can have a longer TTL and can be used for auditing/debugging purposes.

The advantages of doing this are:
 * Makes SYSTEM.TASK_QUEUE more lightweight.
 * Less data scanned when picking up new tasks.
 * Combined with a PK change, we can introduce a natural priority queue using the TASK_PRIORITY
field.
 * Different TTL value on the TASK_HISTORY table so we can maintain past tasks for a longer
period of time.
 * Potentially makes this more in-line with other use-cases that want to maintain a similar
kind of task and task history table (for ex: View TTL's async delete functionality).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message