hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5715) introduce entity prefix for return and sort order
Date Mon, 17 Oct 2016 20:37:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15583379#comment-15583379
] 

Sangjin Lee commented on YARN-5715:
-----------------------------------

Thanks for the patch [~rohithsharma]!

I think one thing we need to be clear we’re on the same page is whether the user-provided
id prefix should be already inverted so that they should be stored as is, or the user provides
a more natural value and the storage inverts it. I think it’s fine either way, although
I do think the latter is slightly more user-friendly. What do others think?

Either way, we should be *crystal clear* about this in the javadoc so that users do not forget
what needs to be done. If we go with the former (the user inverts the prefix), users would
need to use {{LongConverter.invertLong()}}. It may also mean we need to move the {{LongConverter}}
class to where {{TimelineEntity}} is so that users can use it.

Can we add this to javadoc of {{setIdPrefix()}} so that first, setting it with a consistent
value is a requirement and second, whatever approach we're going with? We should remove the
comments on the private {{idPrefix}} variable and state it as javadoc on {{setIdPrefix()}}.

(EntityRowKeyPrefix.java)
- l.48: I’m not sure if this second form of the constructor is needed

(GenericEntityReader.java)
- If we’re going to separate the reader work to YARN-5585, is this change needed?

(TimelineReaderContext.java)
- if we’re going to separate the reader work to YARN-5585, is this change needed?
- I see it use {{Long}} instead of {{long}}. We should use {{long}}.


> introduce entity prefix for return and sort order
> -------------------------------------------------
>
>                 Key: YARN-5715
>                 URL: https://issues.apache.org/jira/browse/YARN-5715
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Sangjin Lee
>            Assignee: Rohith Sharma K S
>            Priority: Critical
>         Attachments: YARN-5715-YARN-5355.01.patch, YARN-5715-YARN-5355.02.patch, YARN-5715-YARN-5355.03.patch
>
>
> While looking into YARN-5585, we have come across the need to provide a sort order different
than the current entity id order. The current entity id order returns entities strictly in
the lexicographical order, and as such it returns the earliest entities first. This may not
be the most natural return order. A more natural return/sort order would be from the most
recent entities.
> To solve this, we would like to add what we call the "entity prefix" in the row key for
the entity table. It is a number (long) that can be easily provided by the client on write.
In the row key, it would be added before the entity id itself.
> The entity prefix would be considered mandatory. On all writes (including updates) the
correct entity prefix should be set by the client so that the correct row key is used. The
entity prefix needs to be unique only within the scope of the application and the entity type.
> For queries that return a list of entities, the prefix values will be returned along
with the entity id's. Queries that specify the prefix and the id should be returned quickly
using the row key. If the query omits the prefix but specifies the id (query by id), the query
may be less efficient.
> This JIRA should add the entity prefix to the entity API and add its handling to the
schema and the write path. The read path will be addressed in YARN-5585.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message