ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ALEKSEY KUZNETSOV <alkuznetsov...@gmail.com>
Subject Re: Merging all network components to a single one
Date Tue, 07 Mar 2017 10:25:37 GMT
What tasks do discovery component responsible for ?

вт, 7 мар. 2017 г. в 13:15, Yakov Zhdanov <yzhdanov@apache.org>:

> Guys,
>
> I have an idea of merging all net components to one.
>
> Now we have the following components interacting via network:
> 1. discovery
> 2. communication
> 3. rest
> 4. odbc
> 5. ignite-hadoop
> 6. time processor (being removed together with clock mode)
> 7. IPC communication endpoint
>
> 2-6 use GridNioServer each with different set of selector threads which may
> result to exceeding the number of cores. Tcp discovery uses blocking socket
> API.
>
> All above mean that we may require many TCP ports opened on nodes. When it
> comes to some secured environments with firewalls and gathering special
> permissions to open new ports Ignite installation may become painful.
>
> What if we have the only TCP port per node (of course we can still bind all
> the components to different ports) and single component that encapsulates
> all the network activities and resource management? All components that
> need network interaction may register filter chain to the network component
> and start getting/sending network messages.
>
> In other words, I suggest to have a single set of nio-selectors and clean
> API to install network listeners to satisfy demands of all other
> components. E.g. discovery, communication and rest will not open their own
> servers but go to the new NetworkProcessor and setup listeners chain on
> some ports (possibly on the default one and NetworkProcessor will properly
> dispatch incoming connections between the components)
>
> Current implementation has the following drawbacks that will be fixed by
> new approach.
> 1. may require too many ports opened
> 2. may have selector threads count exceeding number of CPU which may lead
> to performance degradation.
>
> Please share your thoughts or ask questions.
>
> --Yakov
>
-- 

*Best Regards,*

*Kuznetsov Aleksey*

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message