Mailing-List: contact user-help@manifoldcf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@manifoldcf.apache.org
MIME-Version: 1.0
In-Reply-To: <CALUFAGAJPN8oq6eKMVq-TSG6vW851gv5iqgHGuo632C9tdm1QQ@mail.gmail.com>
References: <CADNgPDgyTkqx8Ds1TDCQ1dYruEvuo646seRm9meF_9eFYzCLvQ@mail.gmail.com>
 <CALUFAGCpSLHkfDXA9KLNQwWH-JQdCmKgyZyXdCQ3m-E4YfuPhg@mail.gmail.com> <CALUFAGAJPN8oq6eKMVq-TSG6vW851gv5iqgHGuo632C9tdm1QQ@mail.gmail.com>
From: Karl Wright <daddywri@gmail.com>
Date: Tue, 7 Feb 2017 14:36:05 -0500
Message-ID: <CALUFAGCs24tkn6xrCjzg974QT_hK_OfRr01dWWCoankmRkmpow@mail.gmail.com>
Subject: Re: extract email attachment
To: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
Content-Type: multipart/alternative; boundary=001a1144e164f1833b0547f5dbcd
archived-at: Tue, 07 Feb 2017 19:36:19 -0000

--001a1144e164f1833b0547f5dbcd
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I've created a ticket and attached a patch to it.  CONNECTORS-1375.  Please
let me know if it works for you; if not, I'll fix what doesn't work.

Karl


On Tue, Feb 7, 2017 at 1:19 PM, Karl Wright <daddywri@gmail.com> wrote:

> Correction: the only metadata attribute we set is the attachment(s)
> mimetype (as a multivalued field) -- this doesn't currently include the
> attachment data.
>
> Karl
>
>
> On Tue, Feb 7, 2017 at 1:14 PM, Karl Wright <daddywri@gmail.com> wrote:
>
>> Hi Cihad,
>>
>> The email connector is providing the attachment data unextracted to the
>> output connector as metadata attribute data.  There are no transformatio=
n
>> connectors that look at this metadata.  Solr cell also probably does not
>> handle binary in random metadata attributes the proper way.
>>
>> The connector's attachment code therefore seems to be designed only to
>> deal with textual attachments.  The right solution is to have individual
>> IDs for each attachment.  But that would also require there to be a URL =
we
>> could construct for each attachment.  We could provide an additional URI
>> template for attachments, but I'd wonder if your system has the ability =
to
>> serve attachments by their own URLs.  Please let me know if this would w=
ork
>> and if so I can create a ticket and work on making these changes.
>>
>> Thanks,
>> Karl
>>
>>
>> On Tue, Feb 7, 2017 at 12:56 PM, Cihad Guzel <cguzelg@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I try the email connector with gmail. I attach the file [1] in my new
>>> email. And sent to my test email adress.
>>>
>>> My mail content body is like: "this is test mail for mfc"
>>>
>>> Then I run my email job and the email is indexed to Solr successfully.
>>> But, the solr's content field have not my attachment's content body. So=
lr
>>> content filed looks like:
>>>
>>> "content":" \n \n  \n  \n  \n  \n  \n  \n  \n \n
>>>  --94eb2c1910841bc55f0547f43443\r\nContent-Type: multipart/alternative;
>>> boundary=3D94eb2c1910841bc5530547f43441\r\n\r\n--94eb2c1910841
>>> bc5530547f43441\r\nContent-Type: text/plain; charset=3DUTF-8\r\n\r\nthi=
s
>>> is test mail for mfc.\r\n\r\n--94eb2c1910841bc5530547f43441\r\nContent-=
Type:
>>> text/html; charset=3DUTF-8\r\n\r\n<div dir=3D\"ltr\">this is test mail =
for
>>> mfc.\r\n</div>\r\n\r\n--94eb2c1910841bc5530547f43441--\r\n--
>>> 94eb2c1910841bc55f0547f43443\r\nContent-Type: application/pdf;
>>> name=3D\"pdf-test.pdf\"\r\nContent-Disposition: attachment;
>>> filename=3D\"pdf-test.pdf\"\r\nContent-Transfer-Encoding:
>>> base64\r\nX-Attachment-Id: f_iyvt78qa0\r\n\r\nJVBERi0xLjY
>>> NJeLjz9MNCjM3IDAgb2JqIDw8L0xpbmVhcml6ZWQgMS9MIDIwNTk3L08gNDA
>>> vRSAx\r\nNDExNS9OIDEvVCAxOTc5NS9IIFsgMTAwNSAyMTVdPj4NZW5kb2J
>>> qDSAgICAgICAgICAgICAgICAg\r\nDQp4cmVmDQozNyAzNA0KMDAwMDAwMDA
>>> xNiAwMDAwMCBuDQowMDAwMDAxMzg2IDAwMDAwIG4NCjAw\r\nMDAwMDE1MjIgMDAwM ..."
>>>
>>> Does the MFC email connector know that the attachment's file type is
>>> pdf? Does not extract the contents?
>>>
>>> [1] http://www.orimi.com/pdf-test.pdf
>>> --
>>> Regards
>>> Cihad G=C3=BCzel
>>>
>>
>>
>

--001a1144e164f1833b0547f5dbcd
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">I&#39;ve created a ticket and attached a patch to it.=C2=
=A0 CONNECTORS-1375.=C2=A0 Please let me know if it works for you; if not, =
I&#39;ll fix what doesn&#39;t work.<div><br></div><div>Karl</div><div><br><=
/div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Tue=
, Feb 7, 2017 at 1:19 PM, Karl Wright <span dir=3D"ltr">&lt;<a href=3D"mail=
to:daddywri@gmail.com" target=3D"_blank">daddywri@gmail.com</a>&gt;</span> =
wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bord=
er-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr">Correction: the o=
nly metadata attribute we set is the attachment(s) mimetype (as a multivalu=
ed field) -- this doesn&#39;t currently include the attachment data.<span c=
lass=3D"HOEnZb"><font color=3D"#888888"><div><br></div><div>Karl</div><div>=
<br></div></font></span></div><div class=3D"HOEnZb"><div class=3D"h5"><div =
class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Tue, Feb 7, 2017 at=
 1:14 PM, Karl Wright <span dir=3D"ltr">&lt;<a href=3D"mailto:daddywri@gmai=
l.com" target=3D"_blank">daddywri@gmail.com</a>&gt;</span> wrote:<br><block=
quote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc=
 solid;padding-left:1ex"><div dir=3D"ltr">Hi Cihad,<div><br></div><div>The =
email connector is providing the attachment data unextracted to the output =
connector as metadata attribute data.=C2=A0 There are no transformation con=
nectors that look at this metadata.=C2=A0 Solr cell also probably does not =
handle binary in random metadata attributes the proper way.</div><div><br><=
/div><div>The connector&#39;s attachment code therefore seems to be designe=
d only to deal with textual attachments.=C2=A0 The right solution is to hav=
e individual IDs for each attachment.=C2=A0 But that would also require the=
re to be a URL we could construct for each attachment.=C2=A0 We could provi=
de an additional URI template for attachments, but I&#39;d wonder if your s=
ystem has the ability to serve attachments by their own URLs.=C2=A0 Please =
let me know if this would work and if so I can create a ticket and work on =
making these changes.</div><div><br></div><div>Thanks,</div><div>Karl</div>=
<div><br></div></div><div class=3D"m_-2317487741658762886HOEnZb"><div class=
=3D"m_-2317487741658762886h5"><div class=3D"gmail_extra"><br><div class=3D"=
gmail_quote">On Tue, Feb 7, 2017 at 12:56 PM, Cihad Guzel <span dir=3D"ltr"=
>&lt;<a href=3D"mailto:cguzelg@gmail.com" target=3D"_blank">cguzelg@gmail.c=
om</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"marg=
in:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"=
>Hi,<div><br></div><div>I try the email connector with gmail. I attach the =
file [1] in my new email. And sent to my test email adress.=C2=A0</div><div=
><br></div><div>My mail content body is like: &quot;this is test mail for m=
fc&quot;</div><div><div><br></div><div>Then I run my email job and the emai=
l is indexed to Solr successfully. But, the solr&#39;s content field have n=
ot my attachment&#39;s content body. Solr content filed looks like:</div><d=
iv><br></div><div>&quot;content&quot;:&quot; \n \n =C2=A0\n =C2=A0\n =C2=A0=
\n =C2=A0\n =C2=A0\n =C2=A0\n =C2=A0\n \n =C2=A0--94eb2c1910841bc55f0547f43=
44<wbr>3\r\nContent-Type: multipart/alternative; boundary=3D94eb2c1910841bc=
553054<wbr>7f43441\r\n\r\n--94eb2c1910841<wbr>bc5530547f43441\r\nContent-Ty=
p<wbr>e: text/plain; charset=3DUTF-8\r\n\r\nthis is test mail for mfc.\r\n\=
r\n--94eb2c1910841bc5<wbr>530547f43441\r\nContent-Type: text/html; charset=
=3DUTF-8\r\n\r\n&lt;div dir=3D\&quot;ltr\&quot;&gt;this is test mail for mf=
c.\r\n&lt;/div&gt;\r\n\r\n--94eb2c<wbr>1910841bc5530547f43441--\r\n--<wbr>9=
4eb2c1910841bc55f0547f43443\r<wbr>\nContent-Type: application/pdf; name=3D\=
&quot;pdf-test.pdf\&quot;\r\nConte<wbr>nt-Disposition: attachment; filename=
=3D\&quot;pdf-test.pdf\&quot;\r\nC<wbr>ontent-Transfer-Encoding: base64\r\n=
X-Attachment-Id: f_iyvt78qa0\r\n\r\nJVBERi0xLjY<wbr>NJeLjz9MNCjM3IDAgb2JqID=
w8L0xpb<wbr>mVhcml6ZWQgMS9MIDIwNTk3L08gNDA<wbr>vRSAx\r\nNDExNS9OIDEvVCAxOTc=
5N<wbr>S9IIFsgMTAwNSAyMTVdPj4NZW5kb2J<wbr>qDSAgICAgICAgICAgICAgICAg\r\nD<wb=
r>Qp4cmVmDQozNyAzNA0KMDAwMDAwMDA<wbr>xNiAwMDAwMCBuDQowMDAwMDAxMzg2I<wbr>DAw=
MDAwIG4NCjAw\r\nMDAwMDE1MjI<wbr>gMDAwM ...&quot;<br></div><div><br></div><d=
iv>Does the MFC email connector know that the attachment&#39;s file type is=
 pdf? Does not extract the contents?<br></div><div><br></div><div>[1]=C2=A0=
<a href=3D"http://www.orimi.com/pdf-test.pdf" target=3D"_blank">http://www.=
orimi.com/pdf-t<wbr>est.pdf</a></div><span class=3D"m_-2317487741658762886m=
_-3162714987407554782HOEnZb"><font color=3D"#888888">-- <br><div class=3D"m=
_-2317487741658762886m_-3162714987407554782m_1400571431449843537gmail_signa=
ture"><div dir=3D"ltr"><div style=3D"font-size:small">Regards</div><div sty=
le=3D"font-size:small">Cihad G=C3=BCzel</div></div></div>
</font></span></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>

--001a1144e164f1833b0547f5dbcd--