Mailing-List: contact users-help@camel.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@camel.apache.org
MIME-Version: 1.0
In-Reply-To: <DM5PR13MB15152F60F4DF265F459AB05DBBE40@DM5PR13MB1515.namprd13.prod.outlook.com>
References: <DM5PR13MB15152F60F4DF265F459AB05DBBE40@DM5PR13MB1515.namprd13.prod.outlook.com>
From: Zoran Regvart <zoran@regvart.com>
Date: Thu, 18 May 2017 09:40:29 +0200
Message-ID: <CABD_Zr9B_upuaeR5nK2UYFSf6MJFLiX4eTahmw3d+yZXhoZr3g@mail.gmail.com>
Subject: Re: Camel processes only the first 150 rows in CSV
To: users@camel.apache.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
archived-at: Thu, 18 May 2017 07:40:55 -0000

Hi,
you have specified `noop=3Dtrue` and I guess you didn't configure
`idempotent=3Dtrue` and `idempotentRepository` (if you wish to preserve
the state of processing between nodes and restarts) -- that keeps the
file as is on the SFTP server.

Also the file might be half-consumed if you're not using
`doneFileName`. It works like this: Camel notices a file on the SFTP
server and starts processing it while the app that is placing the file
is still writing to the file. It could be just that the app that is
writing the file in the first place has a buffer, so on first buffer
flush only first 150 lines are written.

The best course of action is to combine `readLock` and `idempotent*`
settings with `doneFileName`, i.e. the app that is writing the file to
the SFTP server leaves another file (a marker file) that indicates
that it's done writing.

zoran

zoran

On Thu, May 18, 2017 at 8:58 AM, Ambika Das <dasap@outlook.com> wrote:
> Hi All,
>
>
> I am developing a aggregator application which is supposed to consume a C=
SV and generate the message. I have developed it as a Spring boot applicati=
on. The problem is when I transfer a file it takes only the first 150 recor=
d and processes. The file is still there in the folder. If I restart the ap=
plication the whole file gets processed.
>
>
> my code looks as follows.
>
>
> @override
>
> public void configure() {
>
> String end point =3D "sftp://server:port/dropbox?delay=3D10000&moveFailed=
=3Dfailed&noop=3Dtrue&preMove=3Ddone"
>
> from(endpoint).to("direct:file_Processor")
>
>
> }
>
>
> Regards,
>
> Prasad


--=20
Zoran Regvart