hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley" <omal...@apache.org>
Subject Re: file too large - itests/thirdparty/spark-1.2.0-bin-hadoop2-without-hive.tgz
Date Fri, 20 Nov 2015 21:41:19 GMT
I've been maintaining the master-fixed branch, but I missed the fact that
infra asked us to try pushing it ourself. Let me try that.

.. Owen

On Fri, Nov 20, 2015 at 12:27 PM, Jimmy Xiang <jxiang@cloudera.com> wrote:

> I can get the latest of master branch, clean it up, and force-push to
> master-fixed again.
>
> Another solution is to abandon branch master, then use a new branch like
> "trunk".
>
> Thanks,
> Jimmy
>
> On Fri, Nov 20, 2015 at 12:07 PM, Sergey Shelukhin <sergey@hortonworks.com
> >
> wrote:
>
> > Did this ever go anywhere?
> > Infra told us to try ourselves.
> > I wonder if we should do it before releasing 2.0...
> >
> > I am assuming master-fixed diverged by now. It can be rebased as follows:
> >
> >  git rebase --onto master-fixed `git log master --grep="$(git show -s
> > master-fixed --format="%s")" --format="%H"` master
> >
> >  git push origin master:master-fixed (!!!)
> >
> > Or with safety checks:
> > * OLD_HEAD=`git log master --grep="$(git show -s master-fixed
> > --format="%s")" --format="%H"`
> > * Optional git show $OLD_HEAD and git show master-fixed to see if they
> > match
> > * git rebase --onto master-fixed `git log master --grep="$(git show -s
> > master-fixed --format="%s")" --format="%H"` master
> >   This will update the branch correctly but change master, not
> > master-fixed (because essentially you are rebasing new commits added to
> > master on top of master-fixed)… If someone knows a better way to do it
> let
> > me know.
> > * Optionally git diff master origin/master to check.
> > * git push origin master:master-fixed.
> > * For good measure, git reset --hard origin/master.
> >
> >
> >
> >
> >
> > On 15/11/6, 13:36, "Owen O'Malley" <omalley@apache.org> wrote:
> >
> > >Ok, I've filled the infra jira for replacing the master branch:
> > >
> > > https://issues.apache.org/jira/browse/INFRA-10731
> > >
> > >.. Owen
> > >
> > >On Fri, Nov 6, 2015 at 9:36 AM, Jimmy Xiang <jxiang@cloudera.com>
> wrote:
> > >
> > >> Done. Branch master-fixed is clean now. What should we do next?
> > >>
> > >> On Fri, Nov 6, 2015 at 9:26 AM, Jimmy Xiang <jxiang@cloudera.com>
> > wrote:
> > >>
> > >> > Let me do that.
> > >> >
> > >> > On Fri, Nov 6, 2015 at 9:24 AM, Owen O'Malley <omalley@apache.org>
> > >> wrote:
> > >> >
> > >> >> Can someone push a fixed master history to "master-fixed"? Then
we
> > >>can
> > >> >> start a discussion with infra about replacing master with
> > >>master-fixed.
> > >> >>
> > >> >> .. Owen
> > >> >>
> > >> >> On Fri, Nov 6, 2015 at 9:04 AM, Jimmy Xiang <jxiang@cloudera.com>
> > >> wrote:
> > >> >>
> > >> >> > Right. The file is not in source control any more. But it
was
> > >>there at
> > >> >> some
> > >> >> > moment. That's why it is in the git history. So we need to
clean
> up
> > >> the
> > >> >> git
> > >> >> > history.
> > >> >> >
> > >> >> > To push to a branch other than master, for example, hive-11890,
> as
> > >>a
> > >> >> > work-around, you can try
> > https://rtyley.github.io/bfg-repo-cleaner/
> > >> to
> > >> >> do
> > >> >> > the clean up first, then push.
> > >> >> >
> > >> >> > On Fri, Nov 6, 2015 at 9:00 AM, Jimmy Xiang <jxiang@cloudera.com
> >
> > >> >> wrote:
> > >> >> >
> > >> >> > > I ran into the same problem too. What we need to do
is to clean
> > >>up
> > >> the
> > >> >> > git
> > >> >> > > history. However, the master branch can't be force-pushed.
I
> was
> > >> >> > wondering
> > >> >> > > if this is something the infra team can help.
> > >> >> > >
> > >> >> > > On Fri, Nov 6, 2015 at 8:54 AM, Xuefu Zhang <
> xzhang@cloudera.com
> > >
> > >> >> wrote:
> > >> >> > >
> > >> >> > >> I don't quite follow. That file is not in source
control, but
> > >>only
> > >> >> > >> downloaded when running test. Why a git push will
mass with
> that
> > >> >> file?
> > >> >> > >>
> > >> >> > >>
> > >> >> > >> On Fri, Nov 6, 2015 at 7:44 AM, Owen O'Malley
> > >><omalley@apache.org>
> > >> >> > wrote:
> > >> >> > >>
> > >> >> > >> > Github apparently has a limit on pushes to
personal accounts
> > >>that
> > >> >> no
> > >> >> > >> file
> > >> >> > >> > may be larger than 100mb. When I try to push
a branch to my
> > >> >> personal
> > >> >> > >> clone
> > >> >> > >> > of hive, I get:
> > >> >> > >> >
> > >> >> > >> > rockfleet:hive owen$ git push omalley master:hive-11890
> > >> >> > >> > > Counting objects: 21159, done.
> > >> >> > >> > > Delta compression using up to 4 threads.
> > >> >> > >> > > Compressing objects: 100% (6561/6561),
done.
> > >> >> > >> > > Writing objects: 100% (21159/21159), 110.16
MiB | 3.19
> > >>MiB/s,
> > >> >> done.
> > >> >> > >> > > Total 21159 (delta 11142), reused 18555
(delta 9014)
> > >> >> > >> > > remote: error: GH001: Large files detected.
You may want
> to
> > >>try
> > >> >> Git
> > >> >> > >> Large
> > >> >> > >> > > File Storage - https://git-lfs.github.com.
> > >> >> > >> > > remote: error: Trace: 5cfe5408835563c7f661c3086c84930a
> > >> >> > >> > > remote: error: See http://git.io/iEPt8g
for more
> > >>information.
> > >> >> > >> > > remote: error: File
> > >> >> > >> > > itests/thirdparty/spark-1.2.0-bin-hadoop2-without-hive.tgz
> > >>is
> > >> >> 103.62
> > >> >> > >> MB;
> > >> >> > >> > > this exceeds GitHub's file size limit
of 100.00 MB
> > >> >> > >> > > To git@github.com:omalley/hive.git
> > >> >> > >> > >  ! [remote rejected] master -> hive-11890
(pre-receive
> hook
> > >> >> > declined)
> > >> >> > >> > > error: failed to push some refs to 'git@github.com:
> > >> >> > omalley/hive.git'
> > >> >> > >> >
> > >> >> > >> >
> > >> >> > >> > This makes it very difficult to work with.
Can we do
> something
> > >> else
> > >> >> > with
> > >> >> > >> > the file? If not, can we store it with the
git large file
> > >>storage
> > >> >> as
> > >> >> > >> > suggested in the error message?
> > >> >> > >> >
> > >> >> > >> > .. Owen
> > >> >> > >> >
> > >> >> > >>
> > >> >> > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >
> > >> >
> > >>
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message