activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Norman Maurer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AMQ-3017) Add support for stream data to filesystem when using BlobMessages
Date Thu, 25 Nov 2010 09:33:27 GMT

    [ https://issues.apache.org/activemq/browse/AMQ-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=63539#action_63539
] 

Norman Maurer commented on AMQ-3017:
------------------------------------

Are you sure that your BlobTransferPolicy class was patched as well ? It should use the FileSystemBlobStrategy...


As you see it works here:

maurer@maurer-desktop:~/workspace/activemq$ mvn test -Dtest=FilesystemBlobTest
[INFO] Scanning for projects...
[INFO] Reactor build order: 
[INFO]   ActiveMQ
[INFO]   ActiveMQ :: KahaDB
[INFO]   ActiveMQ :: JAAS
[INFO]   Openwire Generator
[INFO]   ActiveMQ :: jmdns 1.0
[INFO]   ActiveMQ :: Core
[INFO]   ActiveMQ :: RA
[INFO]   ActiveMQ :: Pool
[INFO]   ActiveMQ :: Camel
[INFO]   ActiveMQ :: Console
[INFO]   ActiveMQ :: Optional
[INFO]   ActiveMQ :: Jar Bundle
[INFO]   ActiveMQ :: File Server
[INFO]   ActiveMQ :: Blueprint
[INFO]   ActiveMQ :: Apache Karaf
[INFO]   ActiveMQ :: RAR
[INFO]   ActiveMQ :: Run Jar
[INFO]   ActiveMQ :: Spring
[INFO]   ActiveMQ Tooling
[INFO]   ActiveMQ :: Memory Usage Test Plugin
[INFO]   ActiveMQ :: Performance Test Plugin
[INFO]   ActiveMQ :: StartUp Plugin
[INFO]   ActiveMQ :: Web
[INFO]   ActiveMQ :: Web Demo
[INFO]   ActiveMQ :: XMPP
[INFO]   ActiveMQ :: Web Console
[INFO]   ActiveMQ :: Assembly
[INFO] ------------------------------------------------------------------------
[INFO] Building ActiveMQ
[INFO]    task-segment: [test]
[INFO] ------------------------------------------------------------------------
[INFO] [enforcer:enforce {execution: default}]
[INFO] [bundle:cleanVersions {execution: cleanVersions}]
[INFO] Setting property: classpath.resource.loader.class => 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
[INFO] Setting property: velocimacro.messages.on => 'false'.
[INFO] Setting property: resource.loader => 'classpath'.
[INFO] Setting property: resource.manager.logwhenfound => 'false'.
[INFO] [remote-resources:process {execution: default}]
[INFO] ------------------------------------------------------------------------
[INFO] Building ActiveMQ :: KahaDB
[INFO]    task-segment: [test]
[INFO] ------------------------------------------------------------------------
[INFO] [enforcer:enforce {execution: default}]
[INFO] [bundle:cleanVersions {execution: cleanVersions}]
[INFO] [remote-resources:process {execution: default}]
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 3 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources {execution: default-testResources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[WARNING] DEPRECATED [systemProperties]: Use systemPropertyVariables instead.
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: /home/maurer/workspace/activemq/kahadb/target/surefire-reports
[INFO] ------------------------------------------------------------------------
[INFO] Building ActiveMQ :: JAAS
[INFO]    task-segment: [test]
[INFO] ------------------------------------------------------------------------
[INFO] [enforcer:enforce {execution: default}]
[INFO] [bundle:cleanVersions {execution: cleanVersions}]
[INFO] [remote-resources:process {execution: default}]
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/maurer/workspace/activemq/activemq-jaas/src/main/resources
[INFO] Copying 3 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources {execution: default-testResources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 4 resources
[INFO] Copying 3 resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[WARNING] DEPRECATED [systemProperties]: Use systemPropertyVariables instead.
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: /home/maurer/workspace/activemq/activemq-jaas/target/surefire-reports
[INFO] ------------------------------------------------------------------------
[INFO] Building Openwire Generator
[INFO]    task-segment: [test]
[INFO] ------------------------------------------------------------------------
[INFO] [enforcer:enforce {execution: default}]
[INFO] [bundle:cleanVersions {execution: cleanVersions}]
[INFO] [remote-resources:process {execution: default}]
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/maurer/workspace/activemq/activemq-openwire-generator/src/main/resources
[INFO] Copying 3 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources {execution: default-testResources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/maurer/workspace/activemq/activemq-openwire-generator/src/test/resources
[INFO] Copying 3 resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] No sources to compile
[INFO] [surefire:test {execution: default-test}]
[INFO] Tests are skipped.
[INFO] ------------------------------------------------------------------------
[INFO] Building ActiveMQ :: jmdns 1.0
[INFO]    task-segment: [test]
[INFO] ------------------------------------------------------------------------
[INFO] [enforcer:enforce {execution: default}]
[INFO] [bundle:cleanVersions {execution: cleanVersions}]
[INFO] [remote-resources:process {execution: default}]
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Nothing to compile - all classes are up to date
[INFO] [resources:testResources {execution: default-testResources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/maurer/workspace/activemq/activemq-jmdns_1.0/src/test/resources
[INFO] Copying 3 resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] No sources to compile
[INFO] [surefire:test {execution: default-test}]
[INFO] Tests are skipped.
[INFO] ------------------------------------------------------------------------
[INFO] Building ActiveMQ :: Core
[INFO]    task-segment: [test]
[INFO] ------------------------------------------------------------------------
[INFO] [enforcer:enforce {execution: default}]
[INFO] [bundle:cleanVersions {execution: cleanVersions}]
[WARNING] While downloading javacc:javacc:3.2
  This artifact has been relocated to net.java.dev.javacc:javacc:3.2.


[INFO] [javacc:javacc {execution: default}]
[INFO] Nothing to process - all grammars are up to date
[INFO] [activemq-protobuf:compile {execution: default}]
[INFO] Compiling: /home/maurer/workspace/activemq/activemq-core/src/main/proto/journal-data.proto
[INFO] [remote-resources:process {execution: default}]
[INFO] [resources:resources {execution: default-resources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 52 resources
[INFO] Copying 3 resources
[INFO] [compiler:compile {execution: default-compile}]
[INFO] Compiling 15 source files to /home/maurer/workspace/activemq/activemq-core/target/classes
[INFO] [xbean:mapping {execution: default}]
[INFO] Generating META-INF properties file: /home/maurer/workspace/activemq/activemq-core/target/classes/META-INF/services/org/apache/xbean/spring/http/activemq.apache.org/schema/core
for namespace: http://activemq.apache.org/schema/core
[INFO] Updating Spring 2.0 handler mapping: /home/maurer/workspace/activemq/activemq-core/target/classes/META-INF/spring.handlers
for namespace: http://activemq.apache.org/schema/core
[INFO] Generating HTML documentation file: /home/maurer/workspace/activemq/activemq-core/target/classes/activemq.xsd.html
for namespace: http://activemq.apache.org/schema/core
Warning, could not load class: T: java.lang.ClassNotFoundException: T
Warning, could not load class: T: java.lang.ClassNotFoundException: T
Warning, could not load class: T: java.lang.ClassNotFoundException: T
[INFO] Generating XSD file: /home/maurer/workspace/activemq/activemq-core/target/classes/activemq.xsd
for namespace: http://activemq.apache.org/schema/core
[INFO] Generating WIKI documentation file: /home/maurer/workspace/activemq/activemq-core/target/classes/activemq.xsd.wiki
for namespace: http://activemq.apache.org/schema/core
[INFO] ...done.
[INFO] [resources:testResources {execution: default-testResources}]
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 144 resources
[INFO] Copying 3 resources
[INFO] [compiler:testCompile {execution: default-testCompile}]
[INFO] Nothing to compile - all classes are up to date
[WARNING] DEPRECATED [systemProperties]: Use systemPropertyVariables instead.
[INFO] [surefire:test {execution: default-test}]
[INFO] Surefire report directory: /home/maurer/workspace/activemq/activemq-core/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.activemq.blob.FilesystemBlobTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.589 sec

Results :

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0


> Add support for stream data to filesystem when using BlobMessages
> -----------------------------------------------------------------
>
>                 Key: AMQ-3017
>                 URL: https://issues.apache.org/activemq/browse/AMQ-3017
>             Project: ActiveMQ
>          Issue Type: New Feature
>          Components: Broker
>            Reporter: Norman Maurer
>            Assignee: Dejan Bosanac
>         Attachments: AMQ-3017-v2.diff, AMQ-3017.diff
>
>
> Using BlobMessages can help to reduce the memory usage very much when you have the need
to transfer many big messages via activemq. However for this you need some kind of storage
which can be used to transfer the payload to. ActiveMQ can handle the transfer with FTP and
HTTP at the moment. However if you only run an embed ActiveMQ broker this is often overkill
and just plain slow. Here it would be cool to just stream the payload from and to the filesystem,
to get the memory improvement when using BlobMessages but keep the complex level low + have
high speed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message