Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAHfHakGNGXt1mZ7zr-D32FbU+Xvp-JWOJCwW+qhATmMHDzdSwQ@mail.gmail.com>
References: 
 <CAHfHakGh=MZDwip5xGyfJcxh-xEih0p74Mt_C72+UGTS6004Hg@mail.gmail.com>
	<CADx-ob0AzpevE77jEDL=114eP-MxXke6uhrFXKfpGTTQB+EOpw@mail.gmail.com>
	<CAOmV22scHhhpbhuZRPRqay=8k4tN+==GBHgzMjnV8PzqVbqrSQ@mail.gmail.com>
	<CAOmV22v1STyuRvdJQcqG7koLwqQrgrPRx4xHpCQwfOfG6JGaPg@mail.gmail.com>
	<CAHfHakGdbaFRYpTU0sW3-vvRPgy3T161fc4KaXqqZkCD+v0hdg@mail.gmail.com>
	<CAOmV22vdGn5jhVnmXOXoBLN07O+JQcTXrvNZnJvyARJ_rPVu1w@mail.gmail.com>
	<CAOmV22sCGhD20NfpyZ1OhyGzYdhx_VpOOecwJ-hXVsVfh25tSg@mail.gmail.com>
	<CAHfHakHrF0uK=BRVAsUwad=tGdVrNwD8B_kq-JJwDuP-bNsC_w@mail.gmail.com>
	<D274B817.442A7%sergey@hortonworks.com>
	<CAOmV22vcJ1zQyD_hciGpPsBaMNnCRhpP8Lk=h9RvwWszCMsHMw@mail.gmail.com>
	<CAHfHakGNGXt1mZ7zr-D32FbU+Xvp-JWOJCwW+qhATmMHDzdSwQ@mail.gmail.com>
Date: Fri, 20 Nov 2015 14:07:16 -0800
Message-ID: 
 <CAHfHakFh1pjEq90QycS0ak4z7gThX-AuiHVGJMfWjQo8g=qvgA@mail.gmail.com>
Subject: Re: file too large -
 itests/thirdparty/spark-1.2.0-bin-hadoop2-without-hive.tgz
From: "Owen O'Malley" <omalley@apache.org>
To: "dev@hive.apache.org" <dev@hive.apache.org>
Content-Type: multipart/alternative; boundary=089e0149c0c63103610525001992

--089e0149c0c63103610525001992
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Ok, I just talked to them over on the Infra hipchat and Sam Ruby suggested
that I take it to the board to get permission to replace our master branch.

.. Owen

On Fri, Nov 20, 2015 at 1:41 PM, Owen O'Malley <omalley@apache.org> wrote:

> I've been maintaining the master-fixed branch, but I missed the fact that
> infra asked us to try pushing it ourself. Let me try that.
>
> .. Owen
>
> On Fri, Nov 20, 2015 at 12:27 PM, Jimmy Xiang <jxiang@cloudera.com> wrote=
:
>
>> I can get the latest of master branch, clean it up, and force-push to
>> master-fixed again.
>>
>> Another solution is to abandon branch master, then use a new branch like
>> "trunk".
>>
>> Thanks,
>> Jimmy
>>
>> On Fri, Nov 20, 2015 at 12:07 PM, Sergey Shelukhin <
>> sergey@hortonworks.com>
>> wrote:
>>
>> > Did this ever go anywhere?
>> > Infra told us to try ourselves.
>> > I wonder if we should do it before releasing 2.0...
>> >
>> > I am assuming master-fixed diverged by now. It can be rebased as
>> follows:
>> >
>> >  git rebase --onto master-fixed `git log master --grep=3D"$(git show -=
s
>> > master-fixed --format=3D"%s")" --format=3D"%H"` master
>> >
>> >  git push origin master:master-fixed (!!!)
>> >
>> > Or with safety checks:
>> > * OLD_HEAD=3D`git log master --grep=3D"$(git show -s master-fixed
>> > --format=3D"%s")" --format=3D"%H"`
>> > * Optional git show $OLD_HEAD and git show master-fixed to see if they
>> > match
>> > * git rebase --onto master-fixed `git log master --grep=3D"$(git show =
-s
>> > master-fixed --format=3D"%s")" --format=3D"%H"` master
>> >   This will update the branch correctly but change master, not
>> > master-fixed (because essentially you are rebasing new commits added t=
o
>> > master on top of master-fixed)=E2=80=A6 If someone knows a better way =
to do it
>> let
>> > me know.
>> > * Optionally git diff master origin/master to check.
>> > * git push origin master:master-fixed.
>> > * For good measure, git reset --hard origin/master.
>> >
>> >
>> >
>> >
>> >
>> > On 15/11/6, 13:36, "Owen O'Malley" <omalley@apache.org> wrote:
>> >
>> > >Ok, I've filled the infra jira for replacing the master branch:
>> > >
>> > > https://issues.apache.org/jira/browse/INFRA-10731
>> > >
>> > >.. Owen
>> > >
>> > >On Fri, Nov 6, 2015 at 9:36 AM, Jimmy Xiang <jxiang@cloudera.com>
>> wrote:
>> > >
>> > >> Done. Branch master-fixed is clean now. What should we do next?
>> > >>
>> > >> On Fri, Nov 6, 2015 at 9:26 AM, Jimmy Xiang <jxiang@cloudera.com>
>> > wrote:
>> > >>
>> > >> > Let me do that.
>> > >> >
>> > >> > On Fri, Nov 6, 2015 at 9:24 AM, Owen O'Malley <omalley@apache.org=
>
>> > >> wrote:
>> > >> >
>> > >> >> Can someone push a fixed master history to "master-fixed"? Then =
we
>> > >>can
>> > >> >> start a discussion with infra about replacing master with
>> > >>master-fixed.
>> > >> >>
>> > >> >> .. Owen
>> > >> >>
>> > >> >> On Fri, Nov 6, 2015 at 9:04 AM, Jimmy Xiang <jxiang@cloudera.com=
>
>> > >> wrote:
>> > >> >>
>> > >> >> > Right. The file is not in source control any more. But it was
>> > >>there at
>> > >> >> some
>> > >> >> > moment. That's why it is in the git history. So we need to
>> clean up
>> > >> the
>> > >> >> git
>> > >> >> > history.
>> > >> >> >
>> > >> >> > To push to a branch other than master, for example, hive-11890=
,
>> as
>> > >>a
>> > >> >> > work-around, you can try
>> > https://rtyley.github.io/bfg-repo-cleaner/
>> > >> to
>> > >> >> do
>> > >> >> > the clean up first, then push.
>> > >> >> >
>> > >> >> > On Fri, Nov 6, 2015 at 9:00 AM, Jimmy Xiang <
>> jxiang@cloudera.com>
>> > >> >> wrote:
>> > >> >> >
>> > >> >> > > I ran into the same problem too. What we need to do is to
>> clean
>> > >>up
>> > >> the
>> > >> >> > git
>> > >> >> > > history. However, the master branch can't be force-pushed. I
>> was
>> > >> >> > wondering
>> > >> >> > > if this is something the infra team can help.
>> > >> >> > >
>> > >> >> > > On Fri, Nov 6, 2015 at 8:54 AM, Xuefu Zhang <
>> xzhang@cloudera.com
>> > >
>> > >> >> wrote:
>> > >> >> > >
>> > >> >> > >> I don't quite follow. That file is not in source control, b=
ut
>> > >>only
>> > >> >> > >> downloaded when running test. Why a git push will mass with
>> that
>> > >> >> file?
>> > >> >> > >>
>> > >> >> > >>
>> > >> >> > >> On Fri, Nov 6, 2015 at 7:44 AM, Owen O'Malley
>> > >><omalley@apache.org>
>> > >> >> > wrote:
>> > >> >> > >>
>> > >> >> > >> > Github apparently has a limit on pushes to personal
>> accounts
>> > >>that
>> > >> >> no
>> > >> >> > >> file
>> > >> >> > >> > may be larger than 100mb. When I try to push a branch to =
my
>> > >> >> personal
>> > >> >> > >> clone
>> > >> >> > >> > of hive, I get:
>> > >> >> > >> >
>> > >> >> > >> > rockfleet:hive owen$ git push omalley master:hive-11890
>> > >> >> > >> > > Counting objects: 21159, done.
>> > >> >> > >> > > Delta compression using up to 4 threads.
>> > >> >> > >> > > Compressing objects: 100% (6561/6561), done.
>> > >> >> > >> > > Writing objects: 100% (21159/21159), 110.16 MiB | 3.19
>> > >>MiB/s,
>> > >> >> done.
>> > >> >> > >> > > Total 21159 (delta 11142), reused 18555 (delta 9014)
>> > >> >> > >> > > remote: error: GH001: Large files detected. You may wan=
t
>> to
>> > >>try
>> > >> >> Git
>> > >> >> > >> Large
>> > >> >> > >> > > File Storage - https://git-lfs.github.com.
>> > >> >> > >> > > remote: error: Trace: 5cfe5408835563c7f661c3086c84930a
>> > >> >> > >> > > remote: error: See http://git.io/iEPt8g for more
>> > >>information.
>> > >> >> > >> > > remote: error: File
>> > >> >> > >> > >
>> itests/thirdparty/spark-1.2.0-bin-hadoop2-without-hive.tgz
>> > >>is
>> > >> >> 103.62
>> > >> >> > >> MB;
>> > >> >> > >> > > this exceeds GitHub's file size limit of 100.00 MB
>> > >> >> > >> > > To git@github.com:omalley/hive.git
>> > >> >> > >> > >  ! [remote rejected] master -> hive-11890 (pre-receive
>> hook
>> > >> >> > declined)
>> > >> >> > >> > > error: failed to push some refs to 'git@github.com:
>> > >> >> > omalley/hive.git'
>> > >> >> > >> >
>> > >> >> > >> >
>> > >> >> > >> > This makes it very difficult to work with. Can we do
>> something
>> > >> else
>> > >> >> > with
>> > >> >> > >> > the file? If not, can we store it with the git large file
>> > >>storage
>> > >> >> as
>> > >> >> > >> > suggested in the error message?
>> > >> >> > >> >
>> > >> >> > >> > .. Owen
>> > >> >> > >> >
>> > >> >> > >>
>> > >> >> > >
>> > >> >> > >
>> > >> >> >
>> > >> >>
>> > >> >
>> > >> >
>> > >>
>> >
>> >
>>
>
>

--089e0149c0c63103610525001992--