manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Oracle as backend DB
Date Mon, 21 Feb 2011 10:30:53 GMT
Hi Jeffrey,

I was wondering how this was going, as I had not seen a jira ticket
get created.  Do you want me to create one?

Karl

On Fri, Feb 11, 2011 at 5:10 AM, Karl Wright <daddywri@gmail.com> wrote:
> I don't have any knowledge of anyone else that is working on an Oracle
> port at this time.  I presume someone will pipe up if they are.
>
> A procedural point of order...  You will want to create a ticket in
> jira to cover your work.  https://issues.apache.org/jira .  I will
> assign the ticket to myself.  Questions and answers should be attached
> as comments to that ticket, and when your code is ready, you will need
> to attach it to that ticket as well, and click the "grant license to
> ASF" button when you attach it.  Then, I can commit it to the code
> base.
>
> Thanks!
> Karl
>
> On Thu, Feb 10, 2011 at 10:49 PM, Jeffrey Guo <jeff@semtific.com> wrote:
>> Karl, thanks!  I am interested in implementing the Oracle port, if it has
>> not been done or is not being worked on.  I have looked at the class
>> DBInterfacePostgreSQL (similar ones for Derby, HSQLDB, and MySQL), if there
>> are any tips you could share with me, I'd greatly appreciate it, but
>> otherwise, I will post my questions as I move forward with the porting.
>>
>> On another note, does anyone on this list have experiences with Open
>> Pipeline (http://www.openpipeline.org/)?
>>
>> Jeffrey
>>
>> -----Original Message-----
>> From: Karl Wright [mailto:daddywri@gmail.com]
>> Sent: Tuesday, February 08, 2011 1:03 AM
>> To: connectors-user@incubator.apache.org
>> Subject: Re: Running ManifoldCF on Resin
>>
>> I take it you mean the Oracle database, not the Oracle app server?
>> Open-source software relies on the contributions of individuals, not
>> really on road-maps.  ManifoldCF's database architecture is reasonably
>> well-suited for adding support for new kinds of databases, although in
>> my experience adding new database support is much harder than adding a
>> connector, because of the unique operating characteristics of
>> individual databases, which leads to the possibility of failure.
>> Nevertheless, if you want to volunteer, I'd be happy to lend you
>> whatever assistance you might need.
>>
>> Karl
>>
>> On Tue, Feb 8, 2011 at 12:41 AM, Jeffrey Guo <jeff@semtific.com> wrote:
>>> Is Oracle on the road map to be supported?
>>>
>>> -----Original Message-----
>>> From: Karl Wright [mailto:daddywri@gmail.com]
>>> Sent: Friday, February 04, 2011 7:31 AM
>>> To: connectors-user@incubator.apache.org
>>> Subject: Re: Running ManifoldCF on Resin
>>>
>>> Sure, feel free to create a ticket and try to come up with a patch.
>>> Since the DBInterfacePostgreSQL class is responsible for any remote
>>> communication, I would make those parameters local to the postgresql
>>> implementation.  So the parameters would be something like:
>>>
>>> org.apache.manifoldcf.postgresql.hostname
>>> org.apache.manifoldcf.postgresql.ssl
>>>
>>> or some such.  Then, you'd want to load the necessary parameters in
>>> DBInterfacePostgreSQL.java itself.  (I wouldn't bother with MySQL
>>> since that's not officially supported at this time.)
>>>
>>> The only problem I see is that the constructor architecture of
>>> Database.java may make it a challenge to load the appropriate
>>> parameters in the DBInterfacePostgreSQL constructor.  But I think you
>>> could make it work, and if not, we should probably fix this anyway to
>>> make it easier.
>>>
>>> Thanks,
>>> Karl
>>>
>>> On Fri, Feb 4, 2011 at 10:17 AM, Erlend Garåsen <e.f.garasen@usit.uio.no>
>>> wrote:
>>>>
>>>> You're right! I didn't read the command section in the documentation
>>>> properly. ManifoldCF does also run on Resin now. The only problem with
>>> Resin
>>>> is that the necessary resources, such as CSS, are not loaded. All the
>>> other
>>>> errors have disappeared, which means that it should be possible to run
>> the
>>>> application on Resin.
>>>>
>>>> Do you think I should create a ticket about running an external
>> PostgreSQL
>>>> database as well? I think I know which classes to modify. I see there are
>>>> classes for MySQL as well, so perhaps the issue should be generally, i.e.
>>>> not PG specific.
>>>>
>>>> I was thinking about extending the property list like this:
>>>> org.apache.manifoldcf.database.hostname (not required, defaults to
>>>> localhost)
>>>> org.apache.manifoldcf.database.ssl (not required, defaults to false)
>>>>
>>>> For PostgreSQL I think it should be quite easy to implement this, but I'm
>>>> not sure when it comes to MySQL. Haven't touched that database system for
>>>> years.
>>>>
>>>> Erlend
>>>>
>>>>
>>>> On 04.02.11 15.42, Karl Wright wrote:
>>>>>
>>>>> Ok, have you followed the instructions for a multi-process
>>>>> installation in how-to-build-and-deploy.html?
>>>>> You will need to create the database (using DBCreate), and install the
>>>>> schema (using Install), and register the crawler agent (using
>>>>> Register).  You will also need to create the properties.xml and
>>>>> logging.ini files by hand.
>>>>>
>>>>> Let me know if this helps.
>>>>> Karl
>>>>>
>>>>> On Fri, Feb 4, 2011 at 9:35 AM, Erlend Garåsen<e.f.garasen@usit.uio.no>
>>>>>  wrote:
>>>>>>
>>>>>> I guess the problem is that the database does not exist. When I
>>> installed
>>>>>> it
>>>>>> on Tomcat, I got an error message telling me that the schema TEST
does
>>>>>> not
>>>>>> exist. I tried to build the database with the username and password
>>>>>> "test"
>>>>>> by the following command:
>>>>>> ./processes/script/executecommand.sh
>> org.apache.manifoldcf.core.DBCreate
>>>>>> test test
>>>>>>
>>>>>> And I also configured these two lines in my properties.xml file:
>>>>>>  <property name="org.apache.manifoldcf.database.username"
>> value="test"/>
>>>>>>  <property name="org.apache.manifoldcf.database.password"
>> value="test"/>
>>>>>>
>>>>>> I'm using the Derby database since I cannot see how I could contact
an
>>>>>> external PostgreSQL server on another host. PG is one of several
>>> database
>>>>>> systems the university supports. I guess there should be an option
for
>>>>>> configuring external hosts and optionally turn on SSL encryption.
Maybe
>>>>>> another ticket to create.
>>>>>>
>>>>>> But first I want to run ManifoldCF on Tomcat using Derby, then deploy
>> it
>>>>>> on
>>>>>> Resin and then try to configure an external PostgreSQL server.
>>>>>>
>>>>>> Erlend
>>>>>>
>>>>>>
>>>>>> On 03.02.11 16.32, Karl Wright wrote:
>>>>>>>
>>>>>>> Sure, if you want, create a ticket, and see if you can figure
out what
>>>>>>> the problems are.  I'd start with not being able to reach the
>>>>>>> stylesheet, actually.  The "redirection" error sound like the
crawler
>>>>>>> UI is trying to present an error and resin is handling one of
the java
>>>>>>> standard tags in a non-standard way.
>>>>>>>
>>>>>>> Karl
>>>>>>>
>>>>>>> On Thu, Feb 3, 2011 at 10:28 AM, Erlend
>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>  wrote:
>>>>>>>>
>>>>>>>> Resin is de facto application server at the university, but
we also
>>>>>>>> support
>>>>>>>> Tomcat.
>>>>>>>>
>>>>>>>> I can try to deploy the war files on Tomcat instead. The
error
>> message
>>>>>>>> comes
>>>>>>>> from Resin, probably caused by a redirect after the http
header is
>>>>>>>> written.
>>>>>>>> I can notify you, eventually create a patch, if I manage
to run
>>>>>>>> ManifoldCF
>>>>>>>> on Resin.
>>>>>>>>
>>>>>>>> Erlend
>>>>>>>>
>>>>>>>> On 03.02.11 16.16, Karl Wright wrote:
>>>>>>>>>
>>>>>>>>> I have never tried this on Resin, just on Tomcat and
Jetty.  Clearly
>>>>>>>>> Resin as an application server is having difficulty doing
some very
>>>>>>>>> basic stuff.
>>>>>>>>>
>>>>>>>>> FWIW, when we first started the Jetty development, I
had to do
>>> nothing
>>>>>>>>> whatsoever to the crawler UI or the war packaging to
make it work.
>>>  It
>>>>>>>>> just worked.  Your experiences with Resin are making
me wonder if
>>>>>>>>> Resin is indeed working up to Java standards.
>>>>>>>>>
>>>>>>>>> Karl
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Feb 3, 2011 at 9:31 AM, Erlend
>>>>>>>>> Garåsen<e.f.garasen@usit.uio.no>
>>>>>>>>>  wrote:
>>>>>>>>>>
>>>>>>>>>> I have placed the three war files into Resin's webapps
directory
>> and
>>>>>>>>>> set
>>>>>>>>>> the
>>>>>>>>>> following system variable inside resin.xml:
>>>>>>>>>>
>>>>>>>>>> <system-property
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>
>> org.apache.manifoldcf.configfile="/Users/erlendfg/apps/manifoldcf/properties
>>> .xml"
>>>>>>>>>> />
>>>>>>>>>>
>>>>>>>>>> I have also created my own home directory for ManifoldCF
and copied
>>>>>>>>>> the
>>>>>>>>>> necessary files from the example directory. When
I start Resin and
>>>>>>>>>> goes
>>>>>>>>>> to
>>>>>>>>>> ... mcf-crawler-ui/, the start page shows up, but
the page looks
>>> ugly
>>>>>>>>>> (I
>>>>>>>>>> guess the path to CSS files etc. are broken). And
when I try to
>>> click
>>>>>>>>>> on
>>>>>>>>>> a
>>>>>>>>>> link, e.g. "List Output Connections", I get the following
error:
>>>>>>>>>> java.lang.IllegalStateException: can't forward after
writing HTTP
>>>>>>>>>> headers
>>>>>>>>>>
>>>>>>>>>> I find nothing in manifoldcf.log.
>>>>>>>>>>
>>>>>>>>>> BTW: PostgreSQL is not configured. I will start using
Derby until
>> it
>>>>>>>>>> runs
>>>>>>>>>> stable on Resin.
>>>>>>>>>>
>>>>>>>>>> E
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Erlend Garåsen
>>>>>>>>>> Center for Information Technology Services
>>>>>>>>>> University of Oslo
>>>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile:
(+47) 91380968,
>>> VIP:
>>>>>>>>>> 31050
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Erlend Garåsen
>>>>>>>> Center for Information Technology Services
>>>>>>>> University of Oslo
>>>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968,
VIP:
>>>>>>>> 31050
>>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Erlend Garåsen
>>>>>> Center for Information Technology Services
>>>>>> University of Oslo
>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968,
VIP:
>>>>>> 31050
>>>>>>
>>>>
>>>>
>>>> --
>>>> Erlend Garåsen
>>>> Center for Information Technology Services
>>>> University of Oslo
>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP:
>>> 31050
>>>>
>>>
>>>
>>
>>
>

Mime
View raw message