asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Carey <dtab...@gmail.com>
Subject Re: Parallel feed ingestion
Date Wed, 17 May 2017 19:11:30 GMT
@Xikui?  @Abdullah?


On 5/17/17 11:40 AM, Ildar Absalyamov wrote:
> In light of Steven’s discussion about feeds in parallel thread I was wondering what
would be a correct way to push parallel ingestion as far as possible in multinode\multipartition
environment.
> In one of my experiments I am trying to saturate the ingestion to see the effect of computing
stats in background.
> Several things I’ve tried:
> 1) Open a socket adapter on all NC:
> create feed Feed using socket_adapter
> (
>      ("sockets”="NC1:10001,NC2:10001,…”),
> …)
>
> 2) Connect several Feeds to a single dataset.
> create feed Feed1 using socket_adapter
> (
>      ("sockets”="NC1:10001”),
> …)
> create feed Feed2 using socket_adapter
> (
>      ("sockets”="NC2:10001”),
> …)
>
> 3) Have several nodes sending data into a single socket.
>
> In my previous experiments the parallelization did not quite show that the bottleneck
was on the sender side, but I am wondering if that will still be the case, since a lot of
things happened under the hood since the last time.
>
> Best regards,
> Ildar


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message