phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PHOENIX-4903) HashCache recreated on client for every RegionServer it is sent to
Date Tue, 18 Sep 2018 00:00:33 GMT

     [ https://issues.apache.org/jira/browse/PHOENIX-4903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Elser updated PHOENIX-4903:
--------------------------------
    Summary: HashCache recreated on client for every RegionServer it is sent to  (was: Hash
cache RPC uses O(N) memory on master)

> HashCache recreated on client for every RegionServer it is sent to
> ------------------------------------------------------------------
>
>                 Key: PHOENIX-4903
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4903
>             Project: Phoenix
>          Issue Type: Improvement
>            Reporter: Marcell Ortutay
>            Priority: Major
>
> To distribute the hash cache to region servers, the master node makes an `AddServerCacheRequest`
RPC to each region servers. If there are N region servers, it makes N of these RPC's. For
each of the region servers, it generates a serialized RPC message and sends it out. This happens
concurrently, and the result is that it uses O(N) memory on the master.
> As an example, if the `AddServerCacheRequest` RPC message is 100MB, and you have a cluster
of 100 nodes, it would use 10GB memory on the master, potentially resulting in an "OutOfMemory"
exception.
> It would be better if the master could use O(1) memory for the RPC.
> I observed this behavior in Phoenix 4.14.1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message