mesos-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ben Mahler" <benjamin.mah...@gmail.com>
Subject Re: Review Request: Hadoop tutorial version bump (CDH4.2.0 -> 4.2.1).
Date Mon, 06 May 2013 18:35:26 GMT


> On May 3, 2013, 8:40 p.m., Vinod Kone wrote:
> > can't open the patch again :(
> 
> Brenden Matthews wrote:
>     I still don't understand review board.
>     
>     Here are all of the patches:
>     
>     http://ompldr.org/vaWIzYw/patches.tar.bz2
>     
>     Here's that specific patch:
>     
>     http://ompldr.org/vaWIzZA/0011-Hadoop-tutorial-version-bump-CDH4.2.0-4.2.1.patch
> 
> Ben Mahler wrote:
>     Any way you can fix the ReviewBoard diffs on this? We'd love to get this change :)
> 
> Brenden Matthews wrote:
>     I don't know how to fix it.  The patch depends on >1 previous patches, and review
board only permits adding 1 parent patch.
>     
>     When I submitted one patch with multiple changes, I was asked to split the changes
into multiple patches.  I can't do both.
> 
> Ben Mahler wrote:
>     Ah, this is what we use support/post-reviews.py for, which allows you to send a chain
of reviews based on the commits in your current branch.
> 
> Brenden Matthews wrote:
>     I'll try post-reviews again, though it gave me the same results before.
> 
> Brenden Matthews wrote:
>     I might be doing it wrong, but I think post-reviews has brain damage:
>     
>     brenden@airbob ~/dev/mesos $ post-review -gd --target-groups=mesos --repository-url=git://git.apache.org/mesos.git
--revision-range=5e1961bc -r 10926 --debug
>     >>> RBTools 0.5
>     >>> Python 2.7.3 (default, Mar  4 2013, 14:57:34) 
>     [GCC 4.7.2]
>     >>> Running on Linux-3.8.5-x86_64-with-debian-7.0
>     >>> Home = /home/brenden
>     >>> Current Directory = /home/brenden/dev/mesos
>     >>> Checking the repository type. Errors shown below are mostly harmless.
>     DEBUG:root:Checking for a Bazaar repository...
>     DEBUG:root:Checking for a CVS repository...
>     DEBUG:root:Checking for a ClearCase repository...
>     DEBUG:root:Checking for a Git repository...
>     DEBUG:root:Running: git rev-parse --git-dir
>     DEBUG:root:Running: git config core.bare
>     DEBUG:root:Running: git rev-parse --show-toplevel
>     DEBUG:root:Running: git symbolic-ref -q HEAD
>     DEBUG:root:Running: git config --get branch.testing.merge
>     DEBUG:root:Running: git config --get branch.testing.remote
>     DEBUG:root:Running: git config --get remote.origin.url
>     DEBUG:root:repository info: Path: git://git.apache.org/mesos.git, Base path: , Supports
changesets: False
>     >>> Finished checking the repository type.
>     DEBUG:root:Running: git config --get reviewboard.url
>     >>> HTTP GETting api/info/
>     DEBUG:root:Running: git merge-base origin/testing refs/heads/testing
>     DEBUG:root:Running: git branch -r --contains 5e1961bc
>     DEBUG:root:Running: git log --pretty=format:%s 5e1961bc..
>     DEBUG:root:Running: git log --pretty=format:%s%n%n%b 5e1961bc..
>     DEBUG:root:Running: git diff --no-color --full-index --no-ext-diff --ignore-submodules
--no-renames 5e1961bc
>     >>> HTTP GETting api/
>     >>> HTTP GETting https://reviews.apache.org/api/info/
>     >>> Using the new web API
>     >>> HTTP GETting https://reviews.apache.org/api/review-requests/10926/
>     >>> Attempting to set field 'target_groups' to 'mesos' for review request
'10926'
>     >>> HTTP PUTting to https://reviews.apache.org/api/review-requests/10926/draft/:
{'target_groups': 'mesos'}
>     >>> Attempting to set field 'summary' to 'Terminate correct tasks when a
slave disconnects. Turn down noise from ZooKeeper. Make sure you clean up the cgroups test.
Terminate executors that aren't needed. Kill tasks that never properly launch. Log executor
output to syslog. Slave feature: maximum system load. Shuffle offers before sending to frameworks.'
for review request '10926'
>     >>> HTTP PUTting to https://reviews.apache.org/api/review-requests/10926/draft/:
{'summary': "Terminate correct tasks when a slave disconnects. Turn down noise from ZooKeeper.
Make sure you clean up the cgroups test. Terminate executors that aren't needed. Kill tasks
that never properly launch. Log executor output to syslog. Slave feature: maximum system load.
Shuffle offers before sending to frameworks."}
>     >>> Attempting to set field 'description' to 'Terminate correct tasks when
a slave disconnects.
>     
>     Previously, when a slave disconnected all tasks for that framework would
>     be removed and it would result in a bad state for a given framework.  In
>     the case of Hadoop, it would result in a bunch of zombie tasks running
>     on the slaves which never terminate.
>     
>     Added some `operator !=' type utilities.
>     
>     Turn down noise from ZooKeeper.
>     
>     
>     Make sure you clean up the cgroups test.
>     
>     Sometimes the cgroup test fails because the directory is already there.
>     This should ensure that it gets cleanup up in that case before we exit.
>     
>     Terminate executors that aren't needed.
>     
>     If we launch an executor and then kill the task immediately after, make
>     sure we also terminate the executor when there are no other tasks.
>     
>     Kill tasks that never properly launch.
>     
>     After trying to launch a task tracker, we'll wait up to 5 minutes before
>     giving up and killing the task.
>     
>     Log executor output to syslog.
>     
>     
>     Slave feature: maximum system load.
>     
>     When the load exceeds a specified value, don't accept tasks.  Some nodes
>     may become unstable under excessive load (i.e., heavy disk I/O), and
>     this helps prevent the assigning of further tasks to busy slaves.
>     
>     Shuffle offers before sending to frameworks.
>     
>     This helps ensure fair allocation of resources.' for review request '10926'
>     >>> HTTP PUTting to https://reviews.apache.org/api/review-requests/10926/draft/:
{'description': "Terminate correct tasks when a slave disconnects.\n\nPreviously, when a slave
disconnected all tasks for that framework would\nbe removed and it would result in a bad state
for a given framework.  In\nthe case of Hadoop, it would result in a bunch of zombie tasks
running\non the slaves which never terminate.\n\nAdded some `operator !=' type utilities.\n\nTurn
down noise from ZooKeeper.\n\n\nMake sure you clean up the cgroups test.\n\nSometimes the
cgroup test fails because the directory is already there.\nThis should ensure that it gets
cleanup up in that case before we exit.\n\nTerminate executors that aren't needed.\n\nIf we
launch an executor and then kill the task immediately after, make\nsure we also terminate
the executor when there are no other tasks.\n\nKill tasks that never properly launch.\n\nAfter
trying to launch a task tracker, we'll wait up to 5 minutes before\ngiving up and killing
the task.\n\nLog executor output to syslog.\n\n\nSlave feature: maximum system load.\n\nWhen
the load exceeds a specified value, don't accept tasks.  Some nodes\nmay become unstable under
excessive load (i.e., heavy disk I/O), and\nthis helps prevent the assigning of further tasks
to busy slaves.\n\nShuffle offers before sending to frameworks.\n\nThis helps ensure fair
allocation of resources."}
>     >>> Uploading diff, size: 11461
>     >>> HTTP POSTing to https://reviews.apache.org/api/review-requests/10926/diffs/:
{}
>     >>> Got API Error 105 (HTTP code 400): One or more fields had errors
>     >>> Error data: {u'fields': {u'path': [u'error: unable to find 24c2adc039a6920a0d14d8c2b0e79989567897b6\nfatal:
git cat-file 24c2adc039a6920a0d14d8c2b0e79989567897b6: bad file\n']}, u'stat': u'fail', u'err':
{u'msg': u'One or more fields had errors', u'code': 105}}
>     
>     Error uploading diff
>     
>     The generated diff file was empty. This usually means no files were
>     modified in this change.
>     
>     Try running with --output-diff and --debug for more information.
>     
>     Your review request still exists, but the diff is not attached.
>     brenden@airbob ~/dev/mesos $ 
>

Looks like you were running post-review instead of post-reviews.py?

If you have a branch with the following commits:
A
B
C

post-reviews.py will send out 3 reviews:
A parented off of trunk
B parented off of A
C parented off of B


- Ben


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10926/#review20145
-----------------------------------------------------------


On May 3, 2013, 6:45 p.m., Brenden Matthews wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10926/
> -----------------------------------------------------------
> 
> (Updated May 3, 2013, 6:45 p.m.)
> 
> 
> Review request for mesos.
> 
> 
> Description
> -------
> 
> From 5e1961bc57316d128663bd964512c549d94088a9 Mon Sep 17 00:00:00 2001
> From: Brenden Matthews <brenden.matthews@airbnb.com>
> Date: Fri, 26 Apr 2013 15:36:54 -0700
> Subject: [PATCH 3/9] Hadoop tutorial version bump (CDH4.2.0 -> 4.2.1).
> 
> ---
>  hadoop/HadoopPipes.cc.patch                        |    4 ++--
>  hadoop/Makefile.am                                 |   24 ++++++++++----------
>  hadoop/TUTORIAL.sh                                 |   12 +++++-----
>  .../hadoop-2.0.0-mr1-cdh4.2.0_hadoop-env.sh.patch  |   14 ------------
>  hadoop/hadoop-2.0.0-mr1-cdh4.2.0_mesos.patch       |   22 ------------------
>  .../hadoop-2.0.0-mr1-cdh4.2.1_hadoop-env.sh.patch  |   14 ++++++++++++
>  hadoop/hadoop-2.0.0-mr1-cdh4.2.1_mesos.patch       |   22 ++++++++++++++++++
>  7 files changed, 56 insertions(+), 56 deletions(-)
>  delete mode 100644 hadoop/hadoop-2.0.0-mr1-cdh4.2.0_hadoop-env.sh.patch
>  delete mode 100644 hadoop/hadoop-2.0.0-mr1-cdh4.2.0_mesos.patch
>  create mode 100644 hadoop/hadoop-2.0.0-mr1-cdh4.2.1_hadoop-env.sh.patch
>  create mode 100644 hadoop/hadoop-2.0.0-mr1-cdh4.2.1_mesos.patch
> 
> 
> Diffs
> -----
> 
>   hadoop/HadoopPipes.cc.patch aa5df7c 
>   hadoop/Makefile.am f32f268 
>   hadoop/TUTORIAL.sh d81c555 
>   hadoop/hadoop-2.0.0-mr1-cdh4.2.0_hadoop-env.sh.patch b915b1d 
>   hadoop/hadoop-2.0.0-mr1-cdh4.2.0_mesos.patch 8a39444 
>   hadoop/hadoop-2.0.0-mr1-cdh4.2.1_hadoop-env.sh.patch PRE-CREATION 
>   hadoop/hadoop-2.0.0-mr1-cdh4.2.1_mesos.patch PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/10926/diff/
> 
> 
> Testing
> -------
> 
> Used in production at airbnb.
> 
> 
> Thanks,
> 
> Brenden Matthews
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message