hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Junping Du (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4987) Read cache concurrency issue between read and evict in EntityGroupFS timeline store
Date Thu, 26 May 2016 16:54:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-4987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302411#comment-15302411
] 

Junping Du commented on YARN-4987:
----------------------------------

Thanks [~gtCarrera9] for updating the patch!
bq.  The factor to enforce cache item release is more of an implementation level detail, therefore
I'm not exposing that as a config. I've added more explanation on it in the latest patch to
make its meaning more clear. 
Sounds good to me. We can revisit the value setting of CACHE_ITEM_OVERFLOW_FACTOR after going
through production deployment.

004 patch LGTM. +1. Will commit it tomorrow if no further comments from others.


> Read cache concurrency issue between read and evict in EntityGroupFS timeline store 
> ------------------------------------------------------------------------------------
>
>                 Key: YARN-4987
>                 URL: https://issues.apache.org/jira/browse/YARN-4987
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Li Lu
>            Assignee: Li Lu
>            Priority: Critical
>         Attachments: YARN-4987-trunk.001.patch, YARN-4987-trunk.002.patch, YARN-4987-trunk.003.patch,
YARN-4987-trunk.004.patch
>
>
> To handle concurrency issues, key value based timeline storage may return null on reads
that are concurrent to service stop. This is actually caused by a concurrency issue between
cache reads and evicts. Specifically, if the storage is being read when it gets evicted, the
storage may turn into null. EntityGroupFS timeline store needs to handle this case gracefully.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message