giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Reisman <apache.mail...@gmail.com>
Subject Re: [jira] [Commented] (GIRAPH-474) Add an option not to use direct byte buffers
Date Wed, 23 Jan 2013 04:33:05 GMT
Update: I built a new hadop-2.0.2 profile of Giraph today for psuedo tests,
gets through build but hangs mysteriously forever at 100% map completion on
PR benchmark. Don't know why. Maybe something odd has crept into the code
in the last week I didn't notice?

On Tue, Jan 22, 2013 at 8:31 PM, Eli Reisman <apache.mailbox@gmail.com>wrote:

> I was getting some wierd test failures in TestBspBasic late last week when
> trying to evaluate a newbie patch that should not have messed up anything.
> Thought I was nuts, ran pure trunk again, compiled fine. No problem since.
> This was on several build profiles. I haven't been able to reproduce it
> since.
>
>
> On Sat, Jan 19, 2013 at 6:48 AM, Claudio Martella <
> claudio.martella@gmail.com> wrote:
>
>> Good news is I just downloaded 0.20.205 and ran tests on it, and I
>> have the same problem. Bad news is that I haven't figured out what's
>> wrong :)
>>
>> On Sat, Jan 19, 2013 at 3:06 PM, Claudio Martella
>> <claudio.martella@gmail.com> wrote:
>> > I do not know when it started precisely, IIRC it was fine when I
>> > worked on GIRAPH-459.
>> >
>> > Currently, TestBspBasic keeps hanging, one worker dies for:
>> >
>> > java.lang.Throwable: Child Error
>> >         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
>> > Caused by: java.io.IOException: Task process exit with nonzero status
>> of 1.
>> >         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>> >
>> > Task attempt logs do not present any error. Basically they reach the
>> > 20th superstep and then one dies after SendPartitionCache, while the
>> > others hang waiting. I will have to look into it better to understand
>> > what's going on, but if you could please just run a test on your
>> > cluster just to cancel out the obvious, that would help.
>> >
>> > On Thu, Jan 17, 2013 at 7:33 PM, Eli Reisman <apache.mailbox@gmail.com>
>> wrote:
>> >> I have a hadoop 1.0.3 setup on another machine I could run giraph
>> against
>> >> later, is Giraph trunk not working with hadoop_1.0 profile any more? I
>> >> havent run Giraph against Hadoop 1.0.x in a few weeks, when did the
>> trouble
>> >> start? After the git switchover?
>> >>
>> >> In case there is some confusion, my last email regarding 0.1 was the
>> Giraph
>> >> 0.1 release as opposed to Hadoop 1.0.x My thought was our Giraph
>> >> "branch-0.1" on our source tree was from when they rolled the Giraph
>> 0.1
>> >> release, the "giraph474" branch is an accident, and "trunk" is trunk.
>> Trunk
>> >> should have working profiles for Hadoop 1.0.x built via a "mvn
>> -Phadoop_1.0
>> >> clean package" command. If not, thats a new JIRA for sure ;)
>> >>
>> >> I do have trouble getting the contrib stuff to build on any profile
>> other
>> >> than hadoop_2.0.2 running against Hadoop-2.0.2-alpha, and maybe the
>> >> hadoop_trunk Giraph profile (Hadoop-3.0-SNAPSHOT) as someone just
>> >> re-reported on GIRAPH-481. Is that the build problem you're referring
>> to?
>> >>
>> >> On Thu, Jan 17, 2013 at 3:53 AM, Claudio Martella <
>> >> claudio.martella@gmail.com> wrote:
>> >>
>> >>> I am afraid this is what might have broken currently trunk against
>> hadoop
>> >>> 1.0.
>> >>>
>> >>> On Thu, Jan 17, 2013 at 2:11 AM, Eli Reisman <
>> apache.mailbox@gmail.com>
>> >>> wrote:
>> >>> > Thanks for the link, Claudio, I'll check that out. I think
>> Alessandro is
>> >>> > right about which branches are which, I committed the "long way"
>> using a
>> >>> > copy of trunk for 474 so I think my attempt is the one that made
it
>> into
>> >>> > trunk. I think the giraph474 branch is a failed commit from
>> someone's dev
>> >>> > branch (not mine, I typically name something equally creative like
>> >>> > "jira474") so we can safely prune it out.
>> >>> >
>> >>> > The branch marked "branch-0.1" should not be deleted as far as
I
>> know,
>> >>> > since it seems to be the source tree for our 0.1 release?
>> >>> >
>> >>> > On Mon, Jan 14, 2013 at 3:02 PM, Claudio Martella <
>> >>> > claudio.martella@gmail.com> wrote:
>> >>> >
>> >>> >> Hi Eli,
>> >>> >>
>> >>> >> you may want to have a look at:
>> >>> >>
>> >>> >> http://www.apache.org/dev/git.html#workflow
>> >>> >>
>> >>> >> which gives a few command examples of the workflow described
by
>> >>> Alessandro.
>> >>> >>
>> >>> >> On Mon, Jan 14, 2013 at 11:56 PM, Eli Reisman <
>> apache.mailbox@gmail.com
>> >>> >
>> >>> >> wrote:
>> >>> >> > Nice, thanks! I had specified the branches for push like
that on
>> my
>> >>> own
>> >>> >> > repos, but was hoping to see the exact syntax for doing
it to our
>> >>> apache
>> >>> >> > repo online by someone who had successfully done it without
>> breaking
>> >>> >> things
>> >>> >> > before I could bring myself to brave it. Now I have seen
it,
>> I'll try
>> >>> >> it! I
>> >>> >> > had been using a separate clean git repo + patch on trunk
for my
>> >>> commits
>> >>> >> > last week :)
>> >>> >> >
>> >>> >> > thanks again!
>> >>> >> >
>> >>> >> > Alessandro: feeling brave enough to delete the "giraph474"
>> branch from
>> >>> >> the
>> >>> >> > remote repo? Do you know if that commit actually made
it into
>> trunk or
>> >>> >> not?
>> >>> >> >
>> >>> >> > On Mon, Jan 14, 2013 at 11:11 AM, Alessandro Presta <
>> >>> alessandro@fb.com
>> >>> >> >wrote:
>> >>> >> >
>> >>> >> >> It shouldn't be a problem to work and commit from
the same
>> checkout.
>> >>> >> This
>> >>> >> >> is what I do:
>> >>> >> >> 1) Create a new branch (say GIRAPH-XYZ)
>> >>> >> >> 2) Apply the patch
>> >>> >> >> 3) Update the CHANGELOG
>> >>> >> >> 4) git commit -a
>> >>> >> >> 5) git push origin GIRAPH-XYZ:trunk
>> >>> >> >> Those extra branches are the result of someone inadvertently
>> pushing
>> >>> a
>> >>> >> new
>> >>> >> >> branch to the remote.
>> >>> >> >> Making changes directly on your master branch is generally
not
>> a good
>> >>> >> >> practice in git.
>> >>> >> >>
>> >>> >> >> On 1/14/13 11:05 AM, "Eli Reisman" <apache.mailbox@gmail.com>
>> wrote:
>> >>> >> >>
>> >>> >> >> >Hey, Giraph-Dev folks. Looking at our
>> >>> >> >> >https://git-wip-us.apache.org/repos/asf/giraph.git
repo, we
>> have
>> >>> some
>> >>> >> >> >funny
>> >>> >> >> >branches creeping in. Please disambiguate the
situation for me
>> if
>> >>> you
>> >>> >> can,
>> >>> >> >> >but here's what I think:
>> >>> >> >> >
>> >>> >> >> >When you do dev work, have a repo with your dev
branches.
>> >>> >> >> >
>> >>> >> >> >when you're ready to commit, get the patch itself
from the
>> JIRA in
>> >>> >> >> >question, and download (or maintain) a SEPARATE
repo. This is
>> so you
>> >>> >> can
>> >>> >> >> >commit the patch to TRUNK. What worked for me
is:
>> >>> >> >> >
>> >>> >> >> >1. dl this repo from the HTTPS URL only. Alternately,
keep this
>> >>> repo on
>> >>> >> >> >TRUNK all the time, and call "git pull" before
a new commit
>> >>> >> >> >2. apply the patch
>> >>> >> >> >3. run mvn verify etc.
>> >>> >> >> >4. update the CHANGELOG
>> >>> >> >> >5. run git commit -a
>> >>> >> >> >6. run git push
>> >>> >> >> >
>> >>> >> >> >this pushes the current updated commit to TRUNK
on the remote
>> repo.
>> >>> If
>> >>> >> you
>> >>> >> >> >commit directly from your dev branch on your dev
repo, you
>> have to
>> >>> tell
>> >>> >> >> >git
>> >>> >> >> >to apply to trunk or it will make a new alternate
branch
>> matching
>> >>> your
>> >>> >> dev
>> >>> >> >> >branch.
>> >>> >> >> >
>> >>> >> >> >I think this is the method for doing commits now.
Hows this
>> sound to
>> >>> >> you
>> >>> >> >> >folks?
>> >>> >> >> >
>> >>> >> >> >
>> >>> >> >> >On Sun, Jan 13, 2013 at 9:50 AM, Eli Reisman (JIRA)
<
>> >>> jira@apache.org>
>> >>> >> >> >wrote:
>> >>> >> >> >
>> >>> >> >> >>
>> >>> >> >> >>     [
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >>
>> >>>
>> https://issues.apache.org/jira/browse/GIRAPH-474?page=com.atlassian.jira
>> >>> >> >> .
>> >>> >> >>
>> >>> >>
>> >>>
>> >>plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13552260#c
>> >>> >> >> >>omment-13552260]
>> >>> >> >> >>
>> >>> >> >> >> Eli Reisman commented on GIRAPH-474:
>> >>> >> >> >> ------------------------------------
>> >>> >> >> >>
>> >>> >> >> >> Hey Maja, try a 'git branch -a' in your Giraph
repo,
>> something
>> >>> went
>> >>> >> >> >>wrong
>> >>> >> >> >> with this commit, it seems to be on its own
branch or
>> something.
>> >>> >> >> >>
>> >>> >> >> >>
>> >>> >> >> >> > Add an option not to use direct byte
buffers
>> >>> >> >> >> > --------------------------------------------
>> >>> >> >> >> >
>> >>> >> >> >> >                 Key: GIRAPH-474
>> >>> >> >> >> >                 URL:
>> >>> >> https://issues.apache.org/jira/browse/GIRAPH-474
>> >>> >> >> >> >             Project: Giraph
>> >>> >> >> >> >          Issue Type: Improvement
>> >>> >> >> >> >            Reporter: Maja Kabiljo
>> >>> >> >> >> >            Assignee: Maja Kabiljo
>> >>> >> >> >> >         Attachments: GIRAPH-474.patch,
GIRAPH-474.patch
>> >>> >> >> >> >
>> >>> >> >> >> >
>> >>> >> >> >> > Using direct byte buffers makes it harder
to control the
>> memory
>> >>> >> used
>> >>> >> >> >>by
>> >>> >> >> >> giraph. It's a bit faster than regular buffers
though (from
>> some
>> >>> >> >> >> experiments I run the difference was about
10%), so I am
>> adding it
>> >>> >> as an
>> >>> >> >> >> option.
>> >>> >> >> >>
>> >>> >> >> >> --
>> >>> >> >> >> This message is automatically generated by
JIRA.
>> >>> >> >> >> If you think it was sent incorrectly, please
contact your
>> JIRA
>> >>> >> >> >> administrators
>> >>> >> >> >> For more information on JIRA, see:
>> >>> >> >> >>http://www.atlassian.com/software/jira
>> >>> >> >> >>
>> >>> >> >>
>> >>> >> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >>    Claudio Martella
>> >>> >>    claudio.martella@gmail.com
>> >>> >>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>>    Claudio Martella
>> >>>    claudio.martella@gmail.com
>> >>>
>> >
>> >
>> >
>> > --
>> >    Claudio Martella
>> >    claudio.martella@gmail.com
>>
>>
>>
>> --
>>    Claudio Martella
>>    claudio.martella@gmail.com
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message