Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
MIME-Version: 1.0
References: <76AA899A-F4E2-48D1-82F1-0E786089A309@gmail.com> <CAGB5yNn51A=tn4WqqsYyJMGy00wUJnerqod-dFj5rDULtuLkcg@mail.gmail.com>
In-Reply-To: <CAGB5yNn51A=tn4WqqsYyJMGy00wUJnerqod-dFj5rDULtuLkcg@mail.gmail.com>
From: Edoardo Causarano <edoardo.causarano@gmail.com>
Date: Thu, 17 Nov 2016 21:38:50 +0000
Message-ID: <CAJoAOG6VM8i8-b1=cPcwMv3jiF++QBDEF4kj7se66AhFV_qF5Q@mail.gmail.com>
Subject: Re: OOMs with Dropbox Camel module
To: users@camel.apache.org
Content-Type: multipart/alternative; boundary=047d7b471d6494002705418604b3
archived-at: Thu, 17 Nov 2016 21:39:09 -0000

--047d7b471d6494002705418604b3
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi there, I've done some work on this but I'd like to know a couple things
now:

Should I branch from master and you take care of cherry picking?

What language level should I use? Can I go with Java8?

Does it need to be 100% backwards compatible with the existing component?
Some of the headers in the current impl can be improved IMHO but current
usage would break if I changed them arbitrarily... is there a deprecation
mechanism for headers?

Advice on testing: Dropbox doesn't provide a testing module, is there a
robust way to create a test harness? Some sort of record/replay tool
perhaps?


Best,
Edoardo

On Thu, 10 Nov 2016 at 14:56, Claus Ibsen <claus.ibsen@gmail.com> wrote:

> Hi
>
> Yeah you can take a look at how some of the dataformats does that as
> they have out of bands streaming support with Camel's stream caching
> http://camel.apache.org/stream-caching.html
>
> We love contributions, so you are very welcome to look into this and
> provide a PR / patch. And to log a JIRA ticket.
> http://camel.apache.org/contributing
>
> On Thu, Nov 10, 2016 at 11:50 AM, Edoardo Causarano
> <edoardo.causarano@gmail.com> wrote:
> > Hi all,
> >
> > just moved code from dev to testing and found that in real-life the
> DropBox component blows apart in OOMs. Seems that it=E2=80=99s using plai=
n BAOS
> (sic) to buffer remote data which is not a particularly good idea when yo=
u
> have no idea how big the files will be (or you=E2=80=99re pretty certain =
they=E2=80=99re in
> the multiple GB range.)
> >
> > I=E2=80=99d rather not implement a client from scratch so I=E2=80=99d a=
ppreciate some
> guidance on how to fix the existing camel-dropbox component to use
> stream-caching.
> >
> >
> > Best,
> > Edoardo
>
>
>
> --
> Claus Ibsen
> -----------------
> http://davsclaus.com @davsclaus
> Camel in Action 2: https://www.manning.com/ibsen2
>

--047d7b471d6494002705418604b3--