Mailing-List: contact dev-help@spark.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: 
 <CACEJUc4vd1cCnPtku0wQfTsHAE2Mv_NWZZ_wMSnskTnA1W1mAQ@mail.gmail.com>
References: 
 <CACEJUc4vd1cCnPtku0wQfTsHAE2Mv_NWZZ_wMSnskTnA1W1mAQ@mail.gmail.com>
Date: Mon, 7 Mar 2016 12:42:38 +0900
Message-ID: 
 <CACVFzXC9+s7L0cpmn4SHRsJ08P4DYY++ewMwM=qrXX4WdtPp6g@mail.gmail.com>
Subject: Re: GraphX optimizations
From: Takeshi Yamamuro <linguin.m.s@gmail.com>
To: Khaled Ammar <khaled.ammar@gmail.com>
Cc: "dev@spark.apache.org" <dev@spark.apache.org>
Content-Type: multipart/alternative; boundary=001a114388c4973c41052d6d41f4

--001a114388c4973c41052d6d41f4
Content-Type: text/plain; charset=UTF-8

Hi,

mapReduceTriplets you said has been removed in master and you need to use a
newer api,
aggregateMessages, instead of it (See SPARK-3936 and SPARK-12995 for
details).
The memory-based shuffling opt. is a topic of not only graphx but also
spark itself.
You can see SPARK-3376 for related discussions.

Thanks,


On Sat, Mar 5, 2016 at 2:53 AM, Khaled Ammar <khaled.ammar@gmail.com> wrote:

> Hi all,
>
> I wonder if the optimizations mentioned in the GraphX paper (
> https://amplab.cs.berkeley.edu/wp-content/uploads/2014/09/graphx.pdf )
> are currently implemented. In particular, I am looking for mrTriplets
> optimizations and memory-based shuffle.
>
> --
> Thanks,
> -Khaled
>


-- 
---
Takeshi Yamamuro

--001a114388c4973c41052d6d41f4
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hi,</div><div><br><div>mapReduceTriplets you said has=
 been removed in master and you need to use a newer api,</div><div>aggregat=
eMessages, instead of it (See SPARK-3936 and SPARK-12995 for details).<br><=
/div></div><div>The memory-based shuffling opt. is a topic of not only grap=
hx but also spark itself.</div><div>You can see=C2=A0SPARK-3376 for related=
 discussions.</div><div><br></div><div>Thanks,</div><div><br></div></div><d=
iv class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Sat, Mar 5, 2016=
 at 2:53 AM, Khaled Ammar <span dir=3D"ltr">&lt;<a href=3D"mailto:khaled.am=
mar@gmail.com" target=3D"_blank">khaled.ammar@gmail.com</a>&gt;</span> wrot=
e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l=
eft:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hi all,<div><br></div=
><div>I wonder if the optimizations mentioned in the GraphX paper (=C2=A0<a=
 href=3D"https://amplab.cs.berkeley.edu/wp-content/uploads/2014/09/graphx.p=
df" target=3D"_blank">https://amplab.cs.berkeley.edu/wp-content/uploads/201=
4/09/graphx.pdf</a> ) are currently implemented. In particular, I am lookin=
g for mrTriplets optimizations and memory-based shuffle.</div><span class=
=3D"HOEnZb"><font color=3D"#888888"><div><br></div><div>-- <br><div>Thanks,=
<div>-Khaled</div></div>
</div></font></span></div>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div class=
=3D"gmail_signature"><div dir=3D"ltr"><div>---<br>Takeshi Yamamuro<br></div=
></div></div>
</div>

--001a114388c4973c41052d6d41f4--