Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 1 Feb 2016 16:36:06 -0800 (PST)
From: Jia Zou <jacquelinezou@gmail.com>
To: Tachyon Users <tachyon-users@googlegroups.com>
Cc: user@spark.apache.org
Message-Id: <2c5d3441-d5cf-4597-b13a-4d9efca9ef88@googlegroups.com>
In-Reply-To: <bb4f2166-342e-48de-af7b-86d4d7769b3a@googlegroups.com>
References: 
 <CANxD3tLfs0Nh2eToLV0=Bb0aJiDWa_R5D42_QADyObj_O3ggYg@mail.gmail.com>
 <CANxD3t++KnJnJXEhrzxptOD=kbbowh3L8s6sCEWcSXQ7BVE8Pw@mail.gmail.com>
 <CANxD3tLK800iArRnE3NfS0cs8K-cgCLreAXsvB8YCMqSeNWTAQ@mail.gmail.com>
 <bb4f2166-342e-48de-af7b-86d4d7769b3a@googlegroups.com>
Subject: Re: TTransportException when using Spark 1.6.0 on top of Tachyon
 0.8.2
MIME-Version: 1.0
Content-Type: multipart/mixed;
	boundary="----=_Part_1658_1535558727.1454373366218"

------=_Part_1658_1535558727.1454373366218
Content-Type: multipart/alternative;
	boundary="----=_Part_1659_1530717105.1454373366218"

------=_Part_1659_1530717105.1454373366218
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

Hi, Calvin, I am running  24GB data Spark KMeans in a c3.2xlarge AWS 
instance with 30GB physical memory.
Spark will cache data off-heap to Tachyon, the input data is also stored in 
Tachyon.
Tachyon is configured to use 15GB memory, and use tired store.
Tachyon underFS is /tmp.

The only configuration I've changed is Tachyon data block size.

Above experiment is a part of a research project.

Best Regards,
Jia

On Thursday, January 28, 2016 at 9:11:19 PM UTC-6, Calvin Jia wrote:
>
> Hi,
>
> Thanks for the detailed information. How large is the dataset you are 
> running against? Also did you change any Tachyon configurations?
>
> Thanks,
> Calvin
>

------=_Part_1659_1530717105.1454373366218
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi, Calvin, I am running =C2=A024GB data Spark KMeans in a=
 c3.2xlarge AWS instance with 30GB physical memory.<div>Spark will cache da=
ta off-heap to Tachyon, the input data is also stored in Tachyon.</div><div=
>Tachyon is configured to use 15GB memory, and use tired store.</div><div>T=
achyon underFS is /tmp.</div><div><br></div><div>The only configuration I&#=
39;ve changed is Tachyon data block size.</div><div><br></div><div>Above ex=
periment is a part of a research project.</div><div><br></div><div>Best Reg=
ards,</div><div>Jia</div><div><br>On Thursday, January 28, 2016 at 9:11:19 =
PM UTC-6, Calvin Jia wrote:<blockquote class=3D"gmail_quote" style=3D"margi=
n: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><di=
v dir=3D"ltr">Hi,<div><br></div><div>Thanks for the detailed information. H=
ow large is the dataset you are running against? Also did you change any Ta=
chyon configurations?</div><div><br></div><div>Thanks,</div><div>Calvin</di=
v></div></blockquote></div></div>
------=_Part_1659_1530717105.1454373366218--


------=_Part_1658_1535558727.1454373366218
Content-Type: text/plain; charset=us-ascii


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org
------=_Part_1658_1535558727.1454373366218--