Mailing-List: contact user-help@manifoldcf.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@manifoldcf.apache.org
Received-SPF: softfail (athena.apache.org: transitioning domain of
 bhoesel@scamander.com does not designate 195.245.231.130 as permitted sender)
From: Bert van Hoesel <bhoesel@scamander.com>
To: "user@manifoldcf.apache.org" <user@manifoldcf.apache.org>
Date: Mon, 11 Feb 2013 09:57:38 +0100
Subject: Oracle jdbc queried documents not read and not ingested into solr
Thread-Topic: Oracle jdbc queried documents not read and not ingested into
 solr
Thread-Index: Ac4INdhe0hFO3OJUQdqBdS6I/fg2jQ==
Message-ID: <5118B282.7000309@scamander.com>
References: <1360568921.72224.ezmlm@manifoldcf.apache.org>
In-Reply-To: <1360568921.72224.ezmlm@manifoldcf.apache.org>
Accept-Language: en-US
user-agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130107
 Thunderbird/17.0.2
acceptlanguage: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Hi,

I have setup an Oracle JDBC repository and a Solr output. Obvious I want=20
to get the Oracle documents ingested into the Solr output. But when=20
running a job to get this done, nothing gets read and ingested into the=20
Solr output.

The Solr output is working since I can get documents ingested into the=20
Solr output when using a file system  repository.

The Oracle queries are correct as far as I can check. The seeding and=20
data queries work when issued directly to Oracle.

But the oracle - solr combination is not working. The Simple History=20
shows the job is started and the two external queries execute without=20
error. But no 'read' en 'ingest' actions show up in the history. The=20
document count in the job status shows the number of documents and the=20
number processed. These are the expected numbers.

As far as I can see in the Oracle database the queries are indeed=20
executed by the database. When copying and executing (after a little=20
edditing) the external queries from the history including shown bind=20
data to a direct Oracle interface they do retrieve the expected rows.

What am I missing to get the documents (rows) from Oracle ingested into=20
Solr?

Thanks in advance.

Regards,

Bert van Hoesel.

PS:
Please find below the seeding and data queries, both from definition and=20
history

SEED:
=3D=3D=3D=3D=3D
select mi.menu_id as "$(IDCOLUMN)"
from   sn_menu_items mi
where  1=3D1
and    mi.menu_type     =3D 'W'
and    greatest(mi.dt_created, nvl(mi.dt_updated, mi.dt_created)) >=20
to_date( '1970/01/01:00:00:00', 'yyyy/mm/dd:hh24:mi:ss') +=20
round($(STARTTIME)/86400000)
and    greatest(mi.dt_created, nvl(mi.dt_updated, mi.dt_created)) <=3D=20
to_date( '1970/01/01:00:00:00', 'yyyy/mm/dd:hh24:mi:ss') +=20
round($(ENDTIME)/86400000)
and    greatest(mi.dt_created, nvl(mi.dt_updated, mi.dt_created)) >=20
sysdate - 200 /* just for testing */
connect by prior mi.menu_id =3D mi.top_menu_id
start with mi.menu_id in (185837,275)

DATA:
=3D=3D=3D=3D=3D
select mi.menu_id "$(IDCOLUMN)"
,      'Thiz iz id: ' || to_char(mi.menu_id) "$(DATACOLUMN)"
,      '<a=20
href=3D"http://www.somesite.com/xx/xx_wiki.ht_show?p_id=3D'||mi.wiki_id||ch=
r(38)||'p_fld=3D'||mi.top_menu_id||'">'||mi.display_text||'</a>'=20
"$(URLCOLUMN)"
from   sn_menu_items mi
,      sn_wiki_item  wi
where  mi.menu_id    in $(IDLIST)
and    mi.wiki_id     =3D wi.id

The external queries from the historie:

SEED:
=3D=3D=3D=3D=3D
select mi.menu_id as "lcf__id" from sn_menu_items mi wher...
e 1=3D1 and mi.menu_type =3D 'W' and greatest(mi.dt_...
created, nvl(mi.dt_updated, mi.dt_created)) > to_date( '1970...
/01/01:00:00:00', 'yyyy/mm/dd:hh24:mi:ss') + round(?/86400000...
) and greatest(mi.dt_created, nvl(mi.dt_updated, mi.dt_cr...
eated)) <=3D to_date( '1970/01/01:00:00:00', 'yyyy/mm/dd:hh24:m...
i:ss') + round(?/86400000) and greatest(mi.dt_created, nv...
l(mi.dt_updated, mi.dt_created)) > sysdate - 200 connect by...
prior mi.menu_id =3D mi.top_menu_id start with mi.menu_id in ...
(185837,275); arguments =3D (0,1360570171797)

DATA:
=3D=3D=3D=3D=3D
select mi.menu_id "lcf__id" , 'Thiz iz id: ' || to_char...
(mi.menu_id) "lcf__data" , '<a href=3D"http://www.some...
site.com/xx/xx_wiki.ht_show?p_id=3D'||mi.wiki_id||chr(38)||'p...
_fld=3D'||mi.top_menu_id||'">'||mi.display_text||'</a>' "lcf__u...
rl" from sn_menu_items mi , sn_wiki_item wi where ...
mi.menu_id in (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?...
,?,?,?,?,?,?,?,?,?,?,?,?) and mi.wiki_id =3D wi.id; ar...
guments =3D ('185904','185641','185885','188488','184738','1856...
12','185853','185889','186158','185117','184723','185901','18...
6249','185263','185886','185229','190366','185103','185900','...
185892','184696','185613','185104','185903','184822','185116'...
,'185979','185866','185896','157508','185893','186185','185902')