Mailing-List: contact users-help@nifi.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@nifi.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAL15SkBOp2ioUmgu_z6se5Zn5xnAGLOaFh_+1WOYRvjW6KMG1Q@mail.gmail.com>
References: 
 <CAL15SkBOp2ioUmgu_z6se5Zn5xnAGLOaFh_+1WOYRvjW6KMG1Q@mail.gmail.com>
Date: Wed, 24 Feb 2016 09:55:06 -0500
Message-ID: 
 <CAC9dF2f4xnwP2LE67NLvVyr_zstvOyB_L7524-NJAjY1GSJGbg@mail.gmail.com>
Subject: Re: ExtractText processor
From: Matthew Clarke <matt.clarke.138@gmail.com>
To: users@nifi.apache.org
Content-Type: multipart/alternative; boundary=001a1145ba2c7019c1052c8540e4

--001a1145ba2c7019c1052c8540e4
Content-Type: text/plain; charset=UTF-8

Sudeep,
       You need to be cautious when extracted the entire contents of a file
in to an attribute.  Attributes are stored in JVM memory.  Having
exceptionally large attributes will consume considerable amounts of that
memory. To use the extractText processor to grab the entire content, you
first need to set/adjust teh following properties:

- Maximum Buffer Size   <-- default is 1 MB but needs to be large enough to
accommodate the entire file.
- Maximum Capture Group Size  <-- in your case since your capture group
will be the entire file, this also must be large enough to handle entire
content.  if set to low and characters beyond they set file will be
truncated.
- Enable DOTALL Mode   <-- needs to be set to true so that line returns are
matched by your capture group as well.
- Include Capture Group 0  <-- you should set this to False to lessen your
JVM memory footprint here.
- Finally you need to add a "New property" which will contain your capture
group
     - for example:
property name: MyContent
value: (.*)

The above value is a Java regular expression contained in a capture group.

Matt

On Wed, Feb 24, 2016 at 9:22 AM, sudeep mishra <sudeepshekharm@gmail.com>
wrote:

> Hi,
>
> Can someone please guide how to use the ExtractText processor to add
> entire flowfile content to an attribute?
>
>
> Thanks & Regards,
>
> Sudeep
>

--001a1145ba2c7019c1052c8540e4
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Sudeep,<div>=C2=A0 =C2=A0 =C2=A0 =C2=A0You need to be caut=
ious when extracted the entire contents of a file in to an attribute.=C2=A0=
 Attributes are stored in JVM memory.=C2=A0 Having exceptionally large attr=
ibutes will consume considerable amounts of that memory. To use the extract=
Text processor to grab the entire content, you first need to set/adjust teh=
 following properties:<br><br>- Maximum Buffer Size =C2=A0 &lt;-- default i=
s 1 MB but needs to be large enough to accommodate the entire file.<br>- Ma=
ximum Capture Group Size =C2=A0&lt;-- in your case since your capture group=
 will be the entire file, this also must be large enough to handle entire c=
ontent. =C2=A0if set to low and characters beyond they set file will be tru=
ncated.<br>- Enable DOTALL Mode =C2=A0 &lt;-- needs to be set to true so th=
at line returns are matched by your capture group as well.<br>- Include Cap=
ture Group 0 =C2=A0&lt;-- you should set this to False to lessen your JVM m=
emory footprint here.<br>- Finally you need to add a &quot;New property&quo=
t; which will contain your capture group<br>=C2=A0 =C2=A0 =C2=A0- for examp=
le:<br>property name: MyContent<br>value: (.*)=C2=A0<br><br>The above value=
 is a Java regular expression contained in a capture group.</div><div><br><=
/div><div>Matt</div></div><div class=3D"gmail_extra"><br><div class=3D"gmai=
l_quote">On Wed, Feb 24, 2016 at 9:22 AM, sudeep mishra <span dir=3D"ltr">&=
lt;<a href=3D"mailto:sudeepshekharm@gmail.com" target=3D"_blank">sudeepshek=
harm@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div=
 dir=3D"ltr">Hi,<div><br></div><div>Can someone please guide how to use the=
 ExtractText processor to add entire flowfile content to an attribute?<br c=
lear=3D"all"><div><br></div>=C2=A0<br><div><div dir=3D"ltr">Thanks &amp; Re=
gards,<div><br></div><div>Sudeep=C2=A0<br></div></div></div>
</div></div>
</blockquote></div><br></div>

--001a1145ba2c7019c1052c8540e4--