giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: [jira] [Commented] (GIRAPH-474) Add an option not to use direct byte buffers
Date Sat, 19 Jan 2013 14:48:27 GMT
Good news is I just downloaded 0.20.205 and ran tests on it, and I
have the same problem. Bad news is that I haven't figured out what's
wrong :)

On Sat, Jan 19, 2013 at 3:06 PM, Claudio Martella
<claudio.martella@gmail.com> wrote:
> I do not know when it started precisely, IIRC it was fine when I
> worked on GIRAPH-459.
>
> Currently, TestBspBasic keeps hanging, one worker dies for:
>
> java.lang.Throwable: Child Error
>         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of 1.
>         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>
> Task attempt logs do not present any error. Basically they reach the
> 20th superstep and then one dies after SendPartitionCache, while the
> others hang waiting. I will have to look into it better to understand
> what's going on, but if you could please just run a test on your
> cluster just to cancel out the obvious, that would help.
>
> On Thu, Jan 17, 2013 at 7:33 PM, Eli Reisman <apache.mailbox@gmail.com> wrote:
>> I have a hadoop 1.0.3 setup on another machine I could run giraph against
>> later, is Giraph trunk not working with hadoop_1.0 profile any more? I
>> havent run Giraph against Hadoop 1.0.x in a few weeks, when did the trouble
>> start? After the git switchover?
>>
>> In case there is some confusion, my last email regarding 0.1 was the Giraph
>> 0.1 release as opposed to Hadoop 1.0.x My thought was our Giraph
>> "branch-0.1" on our source tree was from when they rolled the Giraph 0.1
>> release, the "giraph474" branch is an accident, and "trunk" is trunk. Trunk
>> should have working profiles for Hadoop 1.0.x built via a "mvn -Phadoop_1.0
>> clean package" command. If not, thats a new JIRA for sure ;)
>>
>> I do have trouble getting the contrib stuff to build on any profile other
>> than hadoop_2.0.2 running against Hadoop-2.0.2-alpha, and maybe the
>> hadoop_trunk Giraph profile (Hadoop-3.0-SNAPSHOT) as someone just
>> re-reported on GIRAPH-481. Is that the build problem you're referring to?
>>
>> On Thu, Jan 17, 2013 at 3:53 AM, Claudio Martella <
>> claudio.martella@gmail.com> wrote:
>>
>>> I am afraid this is what might have broken currently trunk against hadoop
>>> 1.0.
>>>
>>> On Thu, Jan 17, 2013 at 2:11 AM, Eli Reisman <apache.mailbox@gmail.com>
>>> wrote:
>>> > Thanks for the link, Claudio, I'll check that out. I think Alessandro is
>>> > right about which branches are which, I committed the "long way" using a
>>> > copy of trunk for 474 so I think my attempt is the one that made it into
>>> > trunk. I think the giraph474 branch is a failed commit from someone's dev
>>> > branch (not mine, I typically name something equally creative like
>>> > "jira474") so we can safely prune it out.
>>> >
>>> > The branch marked "branch-0.1" should not be deleted as far as I know,
>>> > since it seems to be the source tree for our 0.1 release?
>>> >
>>> > On Mon, Jan 14, 2013 at 3:02 PM, Claudio Martella <
>>> > claudio.martella@gmail.com> wrote:
>>> >
>>> >> Hi Eli,
>>> >>
>>> >> you may want to have a look at:
>>> >>
>>> >> http://www.apache.org/dev/git.html#workflow
>>> >>
>>> >> which gives a few command examples of the workflow described by
>>> Alessandro.
>>> >>
>>> >> On Mon, Jan 14, 2013 at 11:56 PM, Eli Reisman <apache.mailbox@gmail.com
>>> >
>>> >> wrote:
>>> >> > Nice, thanks! I had specified the branches for push like that on
my
>>> own
>>> >> > repos, but was hoping to see the exact syntax for doing it to our
>>> apache
>>> >> > repo online by someone who had successfully done it without breaking
>>> >> things
>>> >> > before I could bring myself to brave it. Now I have seen it, I'll
try
>>> >> it! I
>>> >> > had been using a separate clean git repo + patch on trunk for my
>>> commits
>>> >> > last week :)
>>> >> >
>>> >> > thanks again!
>>> >> >
>>> >> > Alessandro: feeling brave enough to delete the "giraph474" branch
from
>>> >> the
>>> >> > remote repo? Do you know if that commit actually made it into trunk
or
>>> >> not?
>>> >> >
>>> >> > On Mon, Jan 14, 2013 at 11:11 AM, Alessandro Presta <
>>> alessandro@fb.com
>>> >> >wrote:
>>> >> >
>>> >> >> It shouldn't be a problem to work and commit from the same
checkout.
>>> >> This
>>> >> >> is what I do:
>>> >> >> 1) Create a new branch (say GIRAPH-XYZ)
>>> >> >> 2) Apply the patch
>>> >> >> 3) Update the CHANGELOG
>>> >> >> 4) git commit -a
>>> >> >> 5) git push origin GIRAPH-XYZ:trunk
>>> >> >> Those extra branches are the result of someone inadvertently
pushing
>>> a
>>> >> new
>>> >> >> branch to the remote.
>>> >> >> Making changes directly on your master branch is generally
not a good
>>> >> >> practice in git.
>>> >> >>
>>> >> >> On 1/14/13 11:05 AM, "Eli Reisman" <apache.mailbox@gmail.com>
wrote:
>>> >> >>
>>> >> >> >Hey, Giraph-Dev folks. Looking at our
>>> >> >> >https://git-wip-us.apache.org/repos/asf/giraph.git repo,
we have
>>> some
>>> >> >> >funny
>>> >> >> >branches creeping in. Please disambiguate the situation
for me if
>>> you
>>> >> can,
>>> >> >> >but here's what I think:
>>> >> >> >
>>> >> >> >When you do dev work, have a repo with your dev branches.
>>> >> >> >
>>> >> >> >when you're ready to commit, get the patch itself from
the JIRA in
>>> >> >> >question, and download (or maintain) a SEPARATE repo. This
is so you
>>> >> can
>>> >> >> >commit the patch to TRUNK. What worked for me is:
>>> >> >> >
>>> >> >> >1. dl this repo from the HTTPS URL only. Alternately, keep
this
>>> repo on
>>> >> >> >TRUNK all the time, and call "git pull" before a new commit
>>> >> >> >2. apply the patch
>>> >> >> >3. run mvn verify etc.
>>> >> >> >4. update the CHANGELOG
>>> >> >> >5. run git commit -a
>>> >> >> >6. run git push
>>> >> >> >
>>> >> >> >this pushes the current updated commit to TRUNK on the
remote repo.
>>> If
>>> >> you
>>> >> >> >commit directly from your dev branch on your dev repo,
you have to
>>> tell
>>> >> >> >git
>>> >> >> >to apply to trunk or it will make a new alternate branch
matching
>>> your
>>> >> dev
>>> >> >> >branch.
>>> >> >> >
>>> >> >> >I think this is the method for doing commits now. Hows
this sound to
>>> >> you
>>> >> >> >folks?
>>> >> >> >
>>> >> >> >
>>> >> >> >On Sun, Jan 13, 2013 at 9:50 AM, Eli Reisman (JIRA) <
>>> jira@apache.org>
>>> >> >> >wrote:
>>> >> >> >
>>> >> >> >>
>>> >> >> >>     [
>>> >> >> >>
>>> >> >> >>
>>> >>
>>> https://issues.apache.org/jira/browse/GIRAPH-474?page=com.atlassian.jira
>>> >> >> .
>>> >> >>
>>> >>
>>> >>plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13552260#c
>>> >> >> >>omment-13552260]
>>> >> >> >>
>>> >> >> >> Eli Reisman commented on GIRAPH-474:
>>> >> >> >> ------------------------------------
>>> >> >> >>
>>> >> >> >> Hey Maja, try a 'git branch -a' in your Giraph repo,
something
>>> went
>>> >> >> >>wrong
>>> >> >> >> with this commit, it seems to be on its own branch
or something.
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> > Add an option not to use direct byte buffers
>>> >> >> >> > --------------------------------------------
>>> >> >> >> >
>>> >> >> >> >                 Key: GIRAPH-474
>>> >> >> >> >                 URL:
>>> >> https://issues.apache.org/jira/browse/GIRAPH-474
>>> >> >> >> >             Project: Giraph
>>> >> >> >> >          Issue Type: Improvement
>>> >> >> >> >            Reporter: Maja Kabiljo
>>> >> >> >> >            Assignee: Maja Kabiljo
>>> >> >> >> >         Attachments: GIRAPH-474.patch, GIRAPH-474.patch
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > Using direct byte buffers makes it harder to
control the memory
>>> >> used
>>> >> >> >>by
>>> >> >> >> giraph. It's a bit faster than regular buffers though
(from some
>>> >> >> >> experiments I run the difference was about 10%), so
I am adding it
>>> >> as an
>>> >> >> >> option.
>>> >> >> >>
>>> >> >> >> --
>>> >> >> >> This message is automatically generated by JIRA.
>>> >> >> >> If you think it was sent incorrectly, please contact
your JIRA
>>> >> >> >> administrators
>>> >> >> >> For more information on JIRA, see:
>>> >> >> >>http://www.atlassian.com/software/jira
>>> >> >> >>
>>> >> >>
>>> >> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >>    Claudio Martella
>>> >>    claudio.martella@gmail.com
>>> >>
>>>
>>>
>>>
>>> --
>>>    Claudio Martella
>>>    claudio.martella@gmail.com
>>>
>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com



-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message