camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bengt Rodehav <be...@rodehav.com>
Subject Re: "Unconnected sockets not implemented" exception in camel-ftp when using ftps
Date Mon, 21 Jun 2010 07:11:05 GMT
OK

2010/6/21 Claus Ibsen <claus.ibsen@gmail.com>

> On Mon, Jun 21, 2010 at 8:52 AM, Bengt Rodehav <bengt@rodehav.com> wrote:
> > I have now created a JIRA in commons-net for this issue:
> >
> > https://issues.apache.org/jira/browse/NET-327
> >
>
> Good
>
> > <https://issues.apache.org/jira/browse/NET-327>However, I still think we
> > must fix this in camel-ftp by some kind of work-around (maybe similar to
> the
> > one I proposed in https://issues.apache.org/activemq/browse/CAMEL-2829).
> > Have you had a chance to look at it Claus?
> >
>
> No sorry I am to busy with other issues but we got the ticket so we
> will look into it before 2.4 is cut.
>
>
> > /Bengt
> >
> > 2010/6/18 Bengt Rodehav <bengt@rodehav.com>
> >
> >> JIRA created:
> >>
> >> https://issues.apache.org/activemq/browse/CAMEL-2829
> >>
> >> <https://issues.apache.org/activemq/browse/CAMEL-2829>/Bengt
> >>
> >> 2010/6/18 Bengt Rodehav <bengt@rodehav.com>
> >>
> >> Claus,
> >>>
> >>> I'm not sure what you mean by:
> >>>
> >>> "I think we should add an option on the endpoint to allow people to
> >>> turn this on/off.
> >>> Some would like to reuse existing connections to avoid the connect ->
> >>> upload -> disconnect cycle when they upload many files etc."
> >>>
> >>> IMO we would not disconnect more often than today. We would just change
> >>> the connect() method in FtpsOperations so that it creates a new
> instance of
> >>> FTPSClient prior to a connection attempt. I assume that since the
> connect()
> >>> method is called in the first place, we are not currently connected and
> >>> there is no way to reuse an existing connection. Am I right?
> >>>
> >>> I have made some modifications in camel-ftp along these lines. I will
> >>> create a JIRA ticket for this bug and attach my changes as diff files
> to the
> >>> issue. I just tested my changes and they do solve the problem I have
> >>> encountered but I haven't tested whether I have introduced any new
> problems
> >>> or not.
> >>>
> >>> Anyway, this is what I did:
> >>>
> >>> *FtpsEndpoint*
> >>> I added the following members to keep track of the configuration of the
> >>> FTPSClient:
> >>>
> >>>   private boolean isNeedClientAuth = false;
> >>>   private KeyManager keyManager;
> >>>   private TrustManager trustManager;
> >>>
> >>> I modified the createFtpClient() method so that it doesn't actually
> create
> >>> a FTPSClient but only calculates and stores the configuration in the
> above
> >>> members. I also renamed it to createFtpsConfiguration() to reflect
> that.
> >>>
> >>> The createRemoteFileOperations() method was also changed accordingly.
> It
> >>> no longer interacts directly with an FTPSClient (since it is not
> created
> >>> yet).
> >>>
> >>> *FtpsOperations*
> >>> I added the private createFtpsClient() method that takes care of
> creating
> >>> an FTPSClient from configuration stored in the FtpsEndpoint. The
> >>> createFtpsClient() is called in the connect() method just before the
> call to
> >>> the "real" connect() method in the superclass (FtpOperations).
> >>>
> >>> I changed the constructor so that it no longer takes a client as an
> >>> argument. I send null as client to the super class constructor. This is
> of
> >>> course a major change in behavior that you need to take a look at it.
> >>> Neither FtpsOperations nor its super class FtpOperations can no longer
> >>> assume that a client exists until the first connection attempt has been
> >>> made.
> >>>
> >>> *FtpOperations*
> >>> I had to make the "client" member non-final since I now set it on every
> >>> call to the connect() method in FtpsOperations.
> >>>
> >>>
> >>> BTW, I agree with you that the SFTPClient in commons-net is flawed. It
> >>> should provide a way to re-initialize itself so that a new connect()
> could
> >>> be made. The FTPClient seems to work that way and it also provides a
> >>> disconnect() method. If they do not intend to support this, then they
> should
> >>> at least state, as part of the documentation, that the SFTPClient
> cannot be
> >>> reused across multiple connections. I've posted about this on their
> mailing
> >>> list but hasn't got any reply yet. Meanwhile the above can be regarded
> as a
> >>> (complicated...) work-around.
> >>>
> >>> /Bengt
> >>>
> >>>
> >>> 2010/6/18 Claus Ibsen <claus.ibsen@gmail.com>
> >>>
> >>> On Thu, Jun 17, 2010 at 11:56 PM, Bengt Rodehav <bengt@rodehav.com>
> >>>> wrote:
> >>>> > Claus,
> >>>> >
> >>>> > Having looked a bit more at the commons-net code it's becoming
clear
> to
> >>>> me
> >>>> > that you probably can't just reconnect an FTPSClient once it has
> >>>> entered
> >>>> > secure communication mode. I can't find a way to re-initialise
an
> >>>> > FTPSClient. I tried the disconnect() method (in the super class
> >>>> FTPClient)
> >>>> > but it didn't get the FTPSClient out of secure communication mode.
> >>>> >
> >>>> > To be safe I think we should always start with a newly instantiated
> >>>> > FTPSClient when we try to connect. Today the FTPSClient is created
> by
> >>>> the
> >>>> > FtpsEndpoint. I don't think it has to. It could just get all the
> righ
> >>>> > configuration parameters and pass it to FtpsOperations which would
> then
> >>>> > instantiate a new FTPSClient at every connect attempt.
> >>>> >
> >>>> > If we do it this way for ftps I guess it would make sense with
the
> same
> >>>> > setup for plain ftp.
> >>>> >
> >>>> > What do you think?
> >>>> >
> >>>>
> >>>> I think we should add an option on the endpoint to allow people to
> >>>> turn this on/off.
> >>>> Some would like to reuse existing connections to avoid the connect ->
> >>>> upload -> disconnect cycle when they upload many files etc.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> And frankly I can't see the problem why a secure connection cannot
> >>>> self heal and be able to re-establish itself. After all FTPSClient got
> >>>> all the connection information. And hence I would consider it a
> >>>> missing feature / glitch in the FTP library.
> >>>>
> >>>> Imagine if using secure connections for JMS brokers which couldn't
> >>>> self heal just because a network outage. That would cause a lot of
> >>>> pain for clients having to re-start.
> >>>>
> >>>>
> >>>> > /Bengt
> >>>> >
> >>>> > 2010/6/17 Bengt Rodehav <bengt@rodehav.com>
> >>>> >
> >>>> >> Claus,
> >>>> >>
> >>>> >> Yeah, maybe this is a commons-net issue. As always I find it
> >>>> surprising
> >>>> >> that no one else has encountered this problem.
> >>>> >>
> >>>> >> As I started writing an email on the commons user list, I took
> another
> >>>> look
> >>>> >> at the code in commons-net. It turns out that the FTPSSocketFactory
> >>>> class
> >>>> >> actually does override the createSocket() methods in SocketFactory.
> A
> >>>> >> similar work-around that I was talking about but on the factory
> class
> >>>> >> instead of in the SocketClient class. FTPSSocketFactory delegates
> the
> >>>> >> createSocket() calls to its SSLContect's socket factory.
> >>>> >>
> >>>> >> So why then are we initially able to connect but not after
we have
> >>>> called
> >>>> >> execProt() and thus changed the connection factory?
> >>>> >>
> >>>> >> Moving back to the FTPSClient class now. When the initial
> connection
> >>>> is up,
> >>>> >> the _connectAction() method is called. It in turn calls
> >>>> sslNegotiation()
> >>>> >> which in turn sets up the secure socket. In the sslNegotiation()
> >>>> method, an
> >>>> >> SSLSocketFactory is not instantiated directly. It is given
by the
> >>>> >> SSLContext's getSocketFactory() method. This connection factory
> >>>> obviously
> >>>> >> works otherwise the secure connection wouldn't work even initially.
> >>>> >>
> >>>> >> Compare this with what's being done in the execProt() method.
Here
> the
> >>>> >> FTPSSocketFactory is instantiated directly with the SSLContext
> passed
> >>>> as a
> >>>> >> constructor argument. Obviously this factory does NOT work...
> >>>> >>
> >>>> >> Another question that comes to mind is why the socket factory
> created
> >>>> in
> >>>> >> the sslNegotiation() method is NOT set as the SocketClient's
socket
> >>>> factory
> >>>> >> but the one created in the execProt() method is? It turns out
that
> >>>> >> execProt() calls a generic sendCommand() method that sends
the
> command
> >>>> and
> >>>> >> if the reply is OK, then sets the socket factory to null! One
> wonders
> >>>> why
> >>>> >> (and there is also a comment in the code: "Check this - is
this
> >>>> necessary at
> >>>> >> all?"). Anyway, this is probably the reason why execProt()
needs to
> >>>> "reset"
> >>>> >> the connection factory.
> >>>> >>
> >>>> >> Still sounds like a commons-net problem though. I'll try to
post
> this
> >>>> at
> >>>> >> their user mailing list.
> >>>> >>
> >>>> >> Just one thought. Do we need to reuse the FTPSClient instance?
> Can't
> >>>> we
> >>>> >> just create a fresh new instance everytime  we need to connect?
> >>>> >>
> >>>> >> /Bengt
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> 2010/6/17 Claus Ibsen <claus.ibsen@gmail.com>
> >>>> >>
> >>>> >> Hi Bengt
> >>>> >>>
> >>>> >>> I think the issue should be created as a ticket for Apache
Commons
> >>>> >>> Net. Then at least the people there can take a look, and
maybe
> they
> >>>> >>> got some ideas. And can fix it in a future release.
> >>>> >>>
> >>>> >>> Bengt fell free to experiment yourself with the subclassing,
as it
> >>>> >>> sounds like a good workaround.
> >>>> >>>
> >>>> >>> I assume the FTPSClient doesn't offer other methods to
re connect
> >>>> >>> which can recover this error?
> >>>> >>>
> >>>> >>>
> >>>> >>> On Wed, Jun 16, 2010 at 2:44 PM, Bengt Rodehav <bengt@rodehav.com
> >
> >>>> wrote:
> >>>> >>> > I have mentioned this problem in a previous conversation
but
> after
> >>>> >>> > investigating the subject further I decided to start
a new
> thread
> >>>> for
> >>>> >>> this.
> >>>> >>> >
> >>>> >>> > If some kind of problem is encountered while using
ftps (I
> tested
> >>>> this
> >>>> >>> using
> >>>> >>> > Filezilla server by copying a file that already existed
which
> >>>> Filezilla
> >>>> >>> does
> >>>> >>> > not allow), then at the next attempt to connect to
the ftps
> server,
> >>>> I
> >>>> >>> get
> >>>> >>> > the following exception in my logfile:
> >>>> >>> >
> >>>> >>> > 13:03:27,606 | ERROR | %7Bfile%3Aext%7D |
> GenericFileOnCompletion
> >>>> >>>    |
> >>>> >>> > rg.apache.camel.processor.Logger  248 | Caused by:
> >>>> >>> >
> >>>> [org.apache.camel.component.file.GenericFileOperationFailedException
-
> >>>> >>> File
> >>>> >>> > operation failed: null Unconnected sockets not implemented.
> Code:
> >>>> 221]
> >>>> >>> >
> >>>> org.apache.camel.component.file.GenericFileOperationFailedException:
> >>>> >>> File
> >>>> >>> > operation failed: null Unconnected sockets not implemented.
> Code:
> >>>> 221
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:108)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:123)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[76:org.apache.camel.camel-ftp:2.4.0.SNAPSHOT]
> >>>> >>> > at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.processor.SendProcessor$1.doInProducer(SendProcessor.java:91)[65:org.apache.camel.camel-core:2.4.0.SNAPSHOT]
> >>>> >>> >
> >>>> >>> > I added some logging to see the actual stack trace
which is:
> >>>> >>> >
> >>>> >>> > java.net.SocketException: Unconnected sockets not
implemented
> >>>> >>> >        at
> >>>> javax.net.SocketFactory.createSocket(SocketFactory.java:104)
> >>>> >>> >        at
> >>>> >>> >
> org.apache.commons.net.SocketClient.connect(SocketClient.java:175)
> >>>> >>> >        at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.FtpOperations.connect(FtpOperations.java:91)
> >>>> >>> >        at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.FtpsOperations.connect(FtpsOperations.java:40)
> >>>> >>> >        at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.connectIfNecessary(RemoteFileProducer.java:170)
> >>>> >>> >        at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.preWriteCheck(RemoteFileProducer.java:99)
> >>>> >>> >        at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:75)
> >>>> >>> >        at
> >>>> >>> >
> >>>> >>>
> >>>>
> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)
> >>>> >>> >
> >>>> >>> > commons-net SocketClient class tries to call the socket
> factory's
> >>>> >>> >  createSocket() method (with no parameters):
> >>>> >>> >
> >>>> >>> >        _socket_= _socketFactory_.createSocket();
> >>>> >>> >        _socket_.connect(new InetSocketAddress(hostname,
port),
> >>>> >>> > connectTimeout);
> >>>> >>> >
> >>>> >>> > Then intention is to create an "unconnected socket"
(hence the
> >>>> error
> >>>> >>> > message) and then subsequently connect it. In my case,
the
> >>>> connection
> >>>> >>> > factory being used is an FTPSSocketFactory
> >>>> >>> > (package org.apache.commons.net.ftp) that inherits
from
> >>>> >>> > SocketFactory(package javax.net). The SocketFactory
class
> >>>> implements
> >>>> >>> the
> >>>> >>> > createSocket() method but throws the exception ("Unconnected
> >>>> sockets not
> >>>> >>> > implemented").
> >>>> >>> >
> >>>> >>> > The reason why this happens is that after having executed
the
> >>>> >>> > FTPSClient.execPROT() method (package
> org.apache.commons.net.ftp),
> >>>> with
> >>>> >>> any
> >>>> >>> > other parameter than "C" (in my case I used "P"),
the socket
> >>>> factory is
> >>>> >>> set
> >>>> >>> > to an FTPSSocketFactory as follows:
> >>>> >>> >
> >>>> >>> >            setSocketFactory(new FTPSSocketFactory(context));
> >>>> >>> >
> >>>> >>> > After this, no connection attempt will succeed. This
problem has
> >>>> been
> >>>> >>> > introduced recently when Claus (on my initiative)
added support
> for
> >>>> >>> secure
> >>>> >>> > data channel in ftps. I'm thus to blame...
> >>>> >>> >
> >>>> >>> > I'm not sure how this should be fixed.
> >>>> >>> >
> >>>> >>> > It's a bit strange that all the connect methods
> >>>> >>> > in org.apache.commons.net.SocketClient (which is the
ultimate
> base
> >>>> class
> >>>> >>> of
> >>>> >>> > FTPSClient) always try to create an unconnected socket
first and
> >>>> then
> >>>> >>> > connects it. If commons-net uses this pattern, then
it should
> make
> >>>> sure
> >>>> >>> that
> >>>> >>> > all its connection factor's support unconnected sockets
but the
> >>>> >>> > FTPSSocketFactory dosn't. This sounds like a bug in
commons-net.
> >>>> >>> >
> >>>> >>> > Knowing that commons-net doesn't release very frequently
I think
> we
> >>>> need
> >>>> >>> a
> >>>> >>> > workaround for this. I guess it would be possible
to subclass
> the
> >>>> >>> FTPClient
> >>>> >>> > class and override all the connect methods to make
sure that no
> >>>> attempt
> >>>> >>> is
> >>>> >>> > made to create unconnected sockets. We would then
change the
> >>>> >>> > createFtpClient() method in the FtpsEndpoint class
to
> instantiate
> >>>> our
> >>>> >>> own
> >>>> >>> > subclass instead of an FTPSClient. Doesn't sound like
a nice
> clean
> >>>> >>> solution
> >>>> >>> > though.
> >>>> >>> >
> >>>> >>> > Any ideas?
> >>>> >>> >
> >>>> >>> > /Bengt
> >>>> >>> >
> >>>> >>>
> >>>> >>>
> >>>> >>>
> >>>> >>> --
> >>>> >>> Claus Ibsen
> >>>> >>> Apache Camel Committer
> >>>> >>>
> >>>> >>> Author of Camel in Action: http://www.manning.com/ibsen/
> >>>> >>> Open Source Integration: http://fusesource.com
> >>>> >>> Blog: http://davsclaus.blogspot.com/
> >>>> >>> Twitter: http://twitter.com/davsclaus
> >>>> >>>
> >>>> >>
> >>>> >>
> >>>> >
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Claus Ibsen
> >>>> Apache Camel Committer
> >>>>
> >>>> Author of Camel in Action: http://www.manning.com/ibsen/
> >>>> Open Source Integration: http://fusesource.com
> >>>> Blog: http://davsclaus.blogspot.com/
> >>>> Twitter: http://twitter.com/davsclaus
> >>>>
> >>>
> >>>
> >>
> >
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message