spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepak Goel <deic...@gmail.com>
Subject Re: processing 50 gb data using just one machine
Date Wed, 15 Jun 2016 17:14:33 GMT
If it is just for test purpose, why not use a smaller size of data and test
it on your notebook. When you go for the cluster, you can go for 50GB (I am
a newbie so my thought would be very naive)

Hey

Namaskara~Nalama~Guten Tag~Bonjour


   --
Keigu

Deepak
73500 12833
www.simtree.net, deepak@simtree.net
deicool@gmail.com

LinkedIn: www.linkedin.com/in/deicool
Skype: thumsupdeicool
Google talk: deicool
Blog: http://loveandfearless.wordpress.com
Facebook: http://www.facebook.com/deicool

"Contribute to the world, environment and more : http://www.gridrepublic.org
"

On Wed, Jun 15, 2016 at 10:40 PM, Sergio Fernández <wikier@apache.org>
wrote:

> In theory yes... the common sense say that:
>
> volume / resources = time
>
> So more volume on the same processing resources would just take more time.
> On Jun 15, 2016 6:43 PM, "spR" <data.smart25@gmail.com> wrote:
>
>> I have 16 gb ram, i7
>>
>> Will this config be able to handle the processing without my ipythin
>> notebook dying?
>>
>> The local mode is for testing purpose. But, I do not have any cluster at
>> my disposal. So can I make this work with the configuration that I have?
>> Thank you.
>> On Jun 15, 2016 9:40 AM, "Deepak Goel" <deicool@gmail.com> wrote:
>>
>>> What do you mean by "EFFECIENTLY"?
>>>
>>> Hey
>>>
>>> Namaskara~Nalama~Guten Tag~Bonjour
>>>
>>>
>>>    --
>>> Keigu
>>>
>>> Deepak
>>> 73500 12833
>>> www.simtree.net, deepak@simtree.net
>>> deicool@gmail.com
>>>
>>> LinkedIn: www.linkedin.com/in/deicool
>>> Skype: thumsupdeicool
>>> Google talk: deicool
>>> Blog: http://loveandfearless.wordpress.com
>>> Facebook: http://www.facebook.com/deicool
>>>
>>> "Contribute to the world, environment and more :
>>> http://www.gridrepublic.org
>>> "
>>>
>>> On Wed, Jun 15, 2016 at 9:33 PM, spR <data.smart25@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> can I use spark in local mode using 4 cores to process 50gb data
>>>> effeciently?
>>>>
>>>> Thank you
>>>>
>>>> misha
>>>>
>>>
>>>

Mime
View raw message