Mailing-List: contact dev-help@spark.apache.org; run by ezmlm
Precedence: bulk
From: Andreas Hechenberger <internet@hechenberger.me>
Subject: Get size of intermediate results
To: dev@spark.apache.org
Message-ID: <b1518d80-bc29-2922-4c8f-f089dfc6f218@hechenberger.me>
Date: Thu, 20 Oct 2016 11:17:46 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.4.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
archived-at: Thu, 20 Oct 2016 09:17:57 -0000

Hey awesome Spark-Dev's :)

i am new to spark and i read a lot but now i am stuck :( so please be
kind, if i ask silly questions.

I want to analyze some algorithms and strategies in spark and for one
experiment i want to know the size of the intermediate results between
iterations/jobs. Some of them are written to disk and some are in the
cache, i guess. I am not afraid of looking into the code (i already did)
but its complex and have no clue where to start :( It would be nice if
someone can point me in the right direction or where i can find more
information about the structure of spark core devel :)

I already setup the devel environment and i can compile spark. It was
really awesome how smoothly the setup was :) Thx for that.

Servus
Andy

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org