ctakes-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jay vyas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CTAKES-314) BigTop/Hadoop cTAKES integration
Date Fri, 10 Oct 2014 16:38:34 GMT

    [ https://issues.apache.org/jira/browse/CTAKES-314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14167084#comment-14167084 ] 

jay vyas commented on CTAKES-314:
---------------------------------

Update.... the spark side of the house for medical terms is working ! 

Now glueing ctakes stuff in to process them !  

https://gist.github.com/jayunit100/06baf96af7592b7ff202

output of a sample looks like this 
{noformat}
/usr/lib/jvm/java-1.7.0-openjdk/bin/java -Didea.launcher.port=7536 -Didea.launcher.bin.path=/opt/idea-IU-135.1230/bin -Dfile.encoding=UTF-8 -classpath /usr/lib/jvm/java-1.7.0-openjdk/jre/lib/management-agent.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/rhino.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/jsse.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/charsets.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/jce.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/rt.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/resources.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/ext/zipfs.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/ext/sunjce_provider.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/ext/dnsns.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/ext/sunpkcs11.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/ext/localedata.jar:/usr/lib/jvm/java-1.7.0-openjdk/jre/lib/ext/pulse-java.jar:/home/jay/Development/SparkBlueprint/target/scala-2.10/classes:/home/jay/.ivy2/cache/colt/colt/jars/colt-1.2.0.jar:/home/jay/.ivy2/cache/com.codahale.metrics/metrics-core/bundles/metrics-core-3.0.0.jar:/home/jay/.ivy2/cache/com.codahale.metrics/metrics-graphite/bundles/metrics-graphite-3.0.0.jar:/home/jay/.ivy2/cache/com.codahale.metrics/metrics-json/bundles/metrics-json-3.0.0.jar:/home/jay/.ivy2/cache/com.codahale.metrics/metrics-jvm/bundles/metrics-jvm-3.0.0.jar:/home/jay/.ivy2/cache/com.esotericsoftware.kryo/kryo/bundles/kryo-2.21.jar:/home/jay/.ivy2/cache/com.esotericsoftware.minlog/minlog/jars/minlog-1.2.jar:/home/jay/.ivy2/cache/com.esotericsoftware.reflectasm/reflectasm/jars/reflectasm-1.07-shaded.jar:/home/jay/.ivy2/cache/com.fasterxml.jackson.core/jackson-annotations/bundles/jackson-annotations-2.3.0.jar:/home/jay/.ivy2/cache/com.google.code.findbugs/jsr305/jars/jsr305-1.3.9.jar:/home/jay/.ivy2/cache/com.google.guava/guava/bundles/guava-14.0.1.jar:/home/jay/.ivy2/cache/com.ning/compress-lzf/bundles/compress-lzf-1.0.0.jar:/home/jay/.ivy2/cache/com.thoughtworks.paranamer/paranamer/jars/paranamer-2.6.jar:/home/jay/.ivy2/cache/com.twitter/chill-java/jars/chill-java-0.3.6.jar:/home/jay/.ivy2/cache/com.twitter/chill_2.10/jars/chill_2.10-0.3.6.jar:/home/jay/.ivy2/cache/com.typesafe/config/bundles/config-1.0.2.jar:/home/jay/.ivy2/cache/commons-beanutils/commons-beanutils/jars/commons-beanutils-1.7.0.jar:/home/jay/.ivy2/cache/commons-beanutils/commons-beanutils-core/jars/commons-beanutils-core-1.8.0.jar:/home/jay/.ivy2/cache/commons-codec/commons-codec/jars/commons-codec-1.5.jar:/home/jay/.ivy2/cache/commons-collections/commons-collections/jars/commons-collections-3.2.1.jar:/home/jay/.ivy2/cache/commons-configuration/commons-configuration/jars/commons-configuration-1.6.jar:/home/jay/.ivy2/cache/commons-digester/commons-digester/jars/commons-digester-1.8.jar:/home/jay/.ivy2/cache/commons-el/commons-el/jars/commons-el-1.0.jar:/home/jay/.ivy2/cache/commons-httpclient/commons-httpclient/jars/commons-httpclient-3.1.jar:/home/jay/.ivy2/cache/commons-io/commons-io/jars/commons-io-2.4.jar:/home/jay/.ivy2/cache/commons-lang/commons-lang/jars/commons-lang-2.4.jar:/home/jay/.ivy2/cache/commons-net/commons-net/jars/commons-net-2.2.jar:/home/jay/.ivy2/cache/concurrent/concurrent/jars/concurrent-1.3.4.jar:/home/jay/.ivy2/cache/hsqldb/hsqldb/jars/hsqldb-1.8.0.10.jar:/home/jay/.ivy2/cache/io.netty/netty/bundles/netty-3.6.6.Final.jar:/home/jay/.ivy2/cache/jline/jline/jars/jline-0.9.94.jar:/home/jay/.ivy2/cache/log4j/log4j/bundles/log4j-1.2.17.jar:/home/jay/.ivy2/cache/net.java.dev.jets3t/jets3t/jars/jets3t-0.7.1.jar:/home/jay/.ivy2/cache/org.apache.commons/commons-lang3/jars/commons-lang3-3.3.2.jar:/home/jay/.ivy2/cache/org.apache.commons/commons-math/jars/commons-math-2.1.jar:/home/jay/.ivy2/cache/org.apache.curator/curator-client/bundles/curator-client-2.4.0.jar:/home/jay/.ivy2/cache/org.apache.curator/curator-framework/bundles/curator-framework-2.4.0.jar:/home/jay/.ivy2/cache/org.apache.curator/curator-recipes/bundles/curator-recipes-2.4.0.jar:/home/jay/.ivy2/cache/org.apache.hadoop/hadoop-client/jars/hadoop-client-1.0.4.jar:/home/jay/.ivy2/cache/org.apache.hadoop/hadoop-core/jars/hadoop-core-1.0.4.jar:/home/jay/.ivy2/cache/org.apache.mesos/mesos/jars/mesos-0.18.1-shaded-protobuf.jar:/home/jay/.ivy2/cache/org.apache.zookeeper/zookeeper/jars/zookeeper-3.4.5.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-continuation/jars/jetty-continuation-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-http/jars/jetty-http-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-io/jars/jetty-io-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-jndi/jars/jetty-jndi-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-plus/jars/jetty-plus-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-security/jars/jetty-security-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-server/jars/jetty-server-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-servlet/jars/jetty-servlet-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-util/jars/jetty-util-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-webapp/jars/jetty-webapp-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty/jetty-xml/jars/jetty-xml-8.1.14.v20131031.jar:/home/jay/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:/home/jay/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:/home/jay/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:/home/jay/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:/home/jay/.ivy2/cache/org.objenesis/objenesis/jars/objenesis-1.2.jar:/home/jay/.ivy2/cache/org.scala-lang/scala-actors/jars/scala-actors-2.10.0-M4.jar:/home/jay/.sbt/boot/scala-2.10.4/lib/scala-compiler.jar:/home/jay/.sbt/boot/scala-2.10.4/lib/scala-library.jar:/home/jay/.sbt/boot/scala-2.10.4/lib/scala-reflect.jar:/home/jay/.ivy2/cache/org.scala-lang/scalap/jars/scalap-2.10.4.jar:/home/jay/.ivy2/cache/org.scalatest/scalatest_2.10.0-M4/jars/scalatest_2.10.0-M4-1.9-2.10.0-M4-B1.jar:/home/jay/.ivy2/cache/org.slf4j/jcl-over-slf4j/jars/jcl-over-slf4j-1.7.5.jar:/home/jay/.ivy2/cache/org.slf4j/jul-to-slf4j/jars/jul-to-slf4j-1.7.5.jar:/home/jay/.ivy2/cache/org.slf4j/slf4j-api/jars/slf4j-api-1.7.5.jar:/home/jay/.ivy2/cache/org.slf4j/slf4j-log4j12/jars/slf4j-log4j12-1.7.5.jar:/home/jay/.ivy2/cache/org.spark-project/pyrolite/jars/pyrolite-2.0.1.jar:/home/jay/.ivy2/cache/org.spark-project.akka/akka-actor_2.10/jars/akka-actor_2.10-2.2.3-shaded-protobuf.jar:/home/jay/.ivy2/cache/org.spark-project.akka/akka-remote_2.10/bundles/akka-remote_2.10-2.2.3-shaded-protobuf.jar:/home/jay/.ivy2/cache/org.spark-project.akka/akka-slf4j_2.10/bundles/akka-slf4j_2.10-2.2.3-shaded-protobuf.jar:/home/jay/.ivy2/cache/org.spark-project.protobuf/protobuf-java/jars/protobuf-java-2.4.1-shaded.jar:/home/jay/.ivy2/cache/org.uncommons.maths/uncommons-maths/jars/uncommons-maths-1.2.2a.jar:/home/jay/.ivy2/cache/oro/oro/jars/oro-2.0.8.jar:/home/jay/.ivy2/cache/xmlenc/xmlenc/jars/xmlenc-0.52.jar:/home/jay/.ivy2/cache/com.clearspring.analytics/stream/jars/stream-2.7.0.jar:/home/jay/.ivy2/cache/com.fasterxml.jackson.core/jackson-core/bundles/jackson-core-2.3.1.jar:/home/jay/.ivy2/cache/com.fasterxml.jackson.core/jackson-databind/bundles/jackson-databind-2.3.1.jar:/home/jay/.ivy2/cache/com.github.fommil.netlib/core/jars/core-1.1.2.jar:/home/jay/.ivy2/cache/com.github.rwl/jtransforms/jars/jtransforms-2.4.0.jar:/home/jay/.ivy2/cache/com.google.code.gson/gson/jars/gson-2.3.jar:/home/jay/.ivy2/cache/com.twitter/parquet-column/jars/parquet-column-1.4.3.jar:/home/jay/.ivy2/cache/com.twitter/parquet-common/jars/parquet-common-1.4.3.jar:/home/jay/.ivy2/cache/com.twitter/parquet-encoding/jars/parquet-encoding-1.4.3.jar:/home/jay/.ivy2/cache/com.twitter/parquet-format/jars/parquet-format-2.0.0.jar:/home/jay/.ivy2/cache/com.twitter/parquet-generator/jars/parquet-generator-1.4.3.jar:/home/jay/.ivy2/cache/com.twitter/parquet-hadoop/jars/parquet-hadoop-1.4.3.jar:/home/jay/.ivy2/cache/com.twitter/parquet-jackson/jars/parquet-jackson-1.4.3.jar:/home/jay/.ivy2/cache/commons-cli/commons-cli/jars/commons-cli-1.2.jar:/home/jay/.ivy2/cache/commons-logging/commons-logging/jars/commons-logging-1.1.1.jar:/home/jay/.ivy2/cache/io.netty/netty-all/jars/netty-all-4.0.23.Final.jar:/home/jay/.ivy2/cache/net.jpountz.lz4/lz4/jars/lz4-1.2.0.jar:/home/jay/.ivy2/cache/net.sf.opencsv/opencsv/jars/opencsv-2.3.jar:/home/jay/.ivy2/cache/net.sf.py4j/py4j/jars/py4j-0.8.2.1.jar:/home/jay/.ivy2/cache/net.sourceforge.f2j/arpack_combined_all/jars/arpack_combined_all-0.1-javadoc.jar:/home/jay/.ivy2/cache/net.sourceforge.f2j/arpack_combined_all/jars/arpack_combined_all-0.1.jar:/home/jay/.ivy2/cache/org.apache.spark/spark-catalyst_2.10/jars/spark-catalyst_2.10-1.1.0.jar:/home/jay/.ivy2/cache/org.apache.spark/spark-core_2.10/jars/spark-core_2.10-1.1.0.jar:/home/jay/.ivy2/cache/org.apache.spark/spark-mllib_2.10/jars/spark-mllib_2.10-1.1.0.jar:/home/jay/.ivy2/cache/org.apache.spark/spark-sql_2.10/jars/spark-sql_2.10-1.1.0.jar:/home/jay/.ivy2/cache/org.apache.spark/spark-streaming-twitter_2.10/jars/spark-streaming-twitter_2.10-1.1.0.jar:/home/jay/.ivy2/cache/org.apache.spark/spark-streaming_2.10/jars/spark-streaming_2.10-1.1.0.jar:/home/jay/.ivy2/cache/org.codehaus.jackson/jackson-core-asl/jars/jackson-core-asl-1.9.11.jar:/home/jay/.ivy2/cache/org.jblas/jblas/jars/jblas-1.2.3.jar:/home/jay/.ivy2/cache/org.json4s/json4s-ast_2.10/jars/json4s-ast_2.10-3.2.10.jar:/home/jay/.ivy2/cache/org.json4s/json4s-core_2.10/jars/json4s-core_2.10-3.2.10.jar:/home/jay/.ivy2/cache/org.json4s/json4s-jackson_2.10/jars/json4s-jackson_2.10-3.2.10.jar:/home/jay/.ivy2/cache/org.scalamacros/quasiquotes_2.10/jars/quasiquotes_2.10-2.0.1.jar:/home/jay/.ivy2/cache/org.scalanlp/breeze-macros_2.10/jars/breeze-macros_2.10-0.3.1.jar:/home/jay/.ivy2/cache/org.scalanlp/breeze_2.10/jars/breeze_2.10-0.9.jar:/home/jay/.ivy2/cache/org.spire-math/spire-macros_2.10/jars/spire-macros_2.10-0.7.4.jar:/home/jay/.ivy2/cache/org.spire-math/spire_2.10/jars/spire_2.10-0.7.4.jar:/home/jay/.ivy2/cache/org.tachyonproject/tachyon/jars/tachyon-0.5.0.jar:/home/jay/.ivy2/cache/org.tachyonproject/tachyon-client/jars/tachyon-client-0.5.0.jar:/home/jay/.ivy2/cache/org.twitter4j/twitter4j-core/jars/twitter4j-core-3.0.3.jar:/home/jay/.ivy2/cache/org.twitter4j/twitter4j-stream/jars/twitter4j-stream-3.0.3.jar:/home/jay/.ivy2/cache/org.xerial.snappy/snappy-java/bundles/snappy-java-1.0.5.3.jar:/home/jay/.ivy2/cache/org.codehaus.jackson/jackson-mapper-asl/jars/jackson-mapper-asl-1.9.11.jar:/opt/idea-IU-135.1230/lib/idea_rt.jar com.intellij.rt.execution.application.AppMain sparkapps.Collect
Usage: Collect$<outputDirectory> <numTweetsToCollect> <intervalInSeconds> <partitionsEachInterval>
START
running w/ defaults[Ljava.lang.String;@31109962
START
ARRAY
Setting sys prop twitter4j.oauth.consumerKey scnGvGpBwNtWu1ztIW2Q
Setting sys prop twitter4j.oauth.consumerSecret CPFVbwy240gTgpq88FaBpL7nJZ9tZW4dsMGHnQmIYU
Setting sys prop twitter4j.oauth.accessToken 312897818-LwyhlLe7vSUG3W8Kwm75v5bWHU6lQ779OflzAUkY
Setting sys prop twitter4j.oauth.accessTokenSecret JQfwRojeNbirTv75QFzj0TedCS7IGRuAeT304hBb7kSj4
Params = seconds=1 tweets=10, /tmp/OUTPUT_1412958706542 partitions=1
~~~~~~ Checkpoint ~~~~~
Initializing Streaming Spark Context...
14/10/10 12:31:47 INFO spark.SecurityManager: Changing view acls to: jay,
14/10/10 12:31:47 INFO spark.SecurityManager: Changing modify acls to: jay,
14/10/10 12:31:47 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jay, ); users with modify permissions: Set(jay, )
14/10/10 12:31:47 INFO slf4j.Slf4jLogger: Slf4jLogger started
14/10/10 12:31:47 INFO Remoting: Starting remoting
14/10/10 12:31:47 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@dhcp-25-121.bos.redhat.com:42685]
14/10/10 12:31:47 INFO Remoting: Remoting now listens on addresses: [akka.tcp://sparkDriver@dhcp-25-121.bos.redhat.com:42685]
14/10/10 12:31:47 INFO util.Utils: Successfully started service 'sparkDriver' on port 42685.
14/10/10 12:31:47 INFO spark.SparkEnv: Registering MapOutputTracker
14/10/10 12:31:47 INFO spark.SparkEnv: Registering BlockManagerMaster
14/10/10 12:31:47 INFO storage.DiskBlockManager: Created local directory at /tmp/spark-local-20141010123147-1dcc
14/10/10 12:31:47 INFO util.Utils: Successfully started service 'Connection manager for block manager' on port 43503.
14/10/10 12:31:47 INFO network.ConnectionManager: Bound socket to port 43503 with id = ConnectionManagerId(dhcp-25-121.bos.redhat.com,43503)
14/10/10 12:31:47 INFO storage.MemoryStore: MemoryStore started with capacity 1890.3 MB
14/10/10 12:31:47 INFO storage.BlockManagerMaster: Trying to register BlockManager
14/10/10 12:31:47 INFO storage.BlockManagerMasterActor: Registering block manager dhcp-25-121.bos.redhat.com:43503 with 1890.3 MB RAM
14/10/10 12:31:47 INFO storage.BlockManagerMaster: Registered BlockManager
14/10/10 12:31:47 INFO spark.HttpFileServer: HTTP File server directory is /tmp/spark-adbe60eb-6087-4905-b54c-2c51a2a32d55
14/10/10 12:31:47 INFO spark.HttpServer: Starting HTTP Server
14/10/10 12:31:47 INFO server.Server: jetty-8.1.14.v20131031
14/10/10 12:31:47 INFO server.AbstractConnector: Started SocketConnector@0.0.0.0:47066
14/10/10 12:31:47 INFO util.Utils: Successfully started service 'HTTP file server' on port 47066.
14/10/10 12:31:47 INFO server.Server: jetty-8.1.14.v20131031
14/10/10 12:31:47 INFO server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:4040
14/10/10 12:31:47 INFO util.Utils: Successfully started service 'SparkUI' on port 4040.
14/10/10 12:31:47 INFO ui.SparkUI: Started SparkUI at http://dhcp-25-121.bos.redhat.com:4040
14/10/10 12:31:48 INFO util.AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver@dhcp-25-121.bos.redhat.com:42685/user/HeartbeatReceiver
14/10/10 12:31:48 INFO scheduler.ReceiverTracker: ReceiverTracker started
14/10/10 12:31:48 INFO dstream.ForEachDStream: metadataCleanupDelay = -1
14/10/10 12:31:48 INFO dstream.FilteredDStream: metadataCleanupDelay = -1
14/10/10 12:31:48 INFO dstream.MappedDStream: metadataCleanupDelay = -1
14/10/10 12:31:48 INFO sparkapps.TwitterInputDStreamJ: metadataCleanupDelay = -1
14/10/10 12:31:48 INFO sparkapps.TwitterInputDStreamJ: Slide time = 1000 ms
14/10/10 12:31:48 INFO sparkapps.TwitterInputDStreamJ: Storage level = StorageLevel(false, false, false, false, 1)
14/10/10 12:31:48 INFO sparkapps.TwitterInputDStreamJ: Checkpoint interval = null
14/10/10 12:31:48 INFO sparkapps.TwitterInputDStreamJ: Remember duration = 1000 ms
14/10/10 12:31:48 INFO sparkapps.TwitterInputDStreamJ: Initialized and validated sparkapps.TwitterInputDStreamJ@2fbb1447
14/10/10 12:31:48 INFO dstream.MappedDStream: Slide time = 1000 ms
14/10/10 12:31:48 INFO dstream.MappedDStream: Storage level = StorageLevel(false, false, false, false, 1)
14/10/10 12:31:48 INFO dstream.MappedDStream: Checkpoint interval = null
14/10/10 12:31:48 INFO dstream.MappedDStream: Remember duration = 1000 ms
14/10/10 12:31:48 INFO dstream.MappedDStream: Initialized and validated org.apache.spark.streaming.dstream.MappedDStream@509137c0
14/10/10 12:31:48 INFO dstream.FilteredDStream: Slide time = 1000 ms
14/10/10 12:31:48 INFO dstream.FilteredDStream: Storage level = StorageLevel(false, false, false, false, 1)
14/10/10 12:31:48 INFO dstream.FilteredDStream: Checkpoint interval = null
14/10/10 12:31:48 INFO dstream.FilteredDStream: Remember duration = 1000 ms
14/10/10 12:31:48 INFO dstream.FilteredDStream: Initialized and validated org.apache.spark.streaming.dstream.FilteredDStream@1a6b3a9a
14/10/10 12:31:48 INFO dstream.ForEachDStream: Slide time = 1000 ms
14/10/10 12:31:48 INFO dstream.ForEachDStream: Storage level = StorageLevel(false, false, false, false, 1)
14/10/10 12:31:48 INFO dstream.ForEachDStream: Checkpoint interval = null
14/10/10 12:31:48 INFO dstream.ForEachDStream: Remember duration = 1000 ms
14/10/10 12:31:48 INFO dstream.ForEachDStream: Initialized and validated org.apache.spark.streaming.dstream.ForEachDStream@73b0a5b
14/10/10 12:31:48 INFO scheduler.ReceiverTracker: Starting 1 receivers
14/10/10 12:31:48 INFO spark.SparkContext: Starting job: runJob at ReceiverTracker.scala:275
14/10/10 12:31:48 INFO scheduler.DAGScheduler: Got job 0 (runJob at ReceiverTracker.scala:275) with 1 output partitions (allowLocal=false)
14/10/10 12:31:48 INFO scheduler.DAGScheduler: Final stage: Stage 0(runJob at ReceiverTracker.scala:275)
14/10/10 12:31:48 INFO scheduler.DAGScheduler: Parents of final stage: List()
14/10/10 12:31:48 INFO scheduler.DAGScheduler: Missing parents: List()
14/10/10 12:31:48 INFO scheduler.DAGScheduler: Submitting Stage 0 (ParallelCollectionRDD[0] at makeRDD at ReceiverTracker.scala:253), which has no missing parents
14/10/10 12:31:48 INFO util.RecurringTimer: Started timer for JobGenerator at time 1412958709000
14/10/10 12:31:48 INFO scheduler.JobGenerator: Started JobGenerator at 1412958709000 ms
14/10/10 12:31:48 INFO scheduler.JobScheduler: Started JobScheduler
14/10/10 12:31:48 INFO storage.MemoryStore: ensureFreeSpace(1216) called with curMem=0, maxMem=1982091755
14/10/10 12:31:48 INFO storage.MemoryStore: Block broadcast_0 stored as values in memory (estimated size 1216.0 B, free 1890.3 MB)
14/10/10 12:31:48 INFO scheduler.DAGScheduler: Submitting 1 missing tasks from Stage 0 (ParallelCollectionRDD[0] at makeRDD at ReceiverTracker.scala:253)
14/10/10 12:31:48 INFO scheduler.TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
14/10/10 12:31:48 INFO scheduler.TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, PROCESS_LOCAL, 4293 bytes)
14/10/10 12:31:48 INFO executor.Executor: Running task 0.0 in stage 0.0 (TID 0)
14/10/10 12:31:48 INFO receiver.ReceiverSupervisorImpl: Registered receiver 0
14/10/10 12:31:48 INFO util.RecurringTimer: Started timer for BlockGenerator at time 1412958708600
14/10/10 12:31:48 INFO receiver.BlockGenerator: Started BlockGenerator
14/10/10 12:31:48 INFO receiver.BlockGenerator: Started block pushing thread
14/10/10 12:31:48 INFO receiver.ReceiverSupervisorImpl: Starting receiver
14/10/10 12:31:48 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from akka://sparkDriver
14/10/10 12:31:48 INFO twitter4j.TwitterStreamImpl: Establishing connection.
14/10/10 12:31:48 INFO sparkapps.TwitterReceiver: Twitter receiver started
14/10/10 12:31:48 INFO receiver.ReceiverSupervisorImpl: Called receiver onStart
14/10/10 12:31:48 INFO scheduler.ReceiverTracker: Registered receiver for stream 0 from akka://sparkDriver
14/10/10 12:31:49 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:31:49 INFO scheduler.JobScheduler: Added jobs for time 1412958709000 ms
14/10/10 12:31:49 INFO scheduler.JobScheduler: Starting job streaming job 1412958709000 ms.0 from job set of time 1412958709000 ms
14/10/10 12:31:49 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:49 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 8.17949E-4 s
RDD 0 0
14/10/10 12:31:49 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:49 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 3.0243E-5 s
14/10/10 12:31:49 INFO scheduler.JobScheduler: Finished job streaming job 1412958709000 ms.0 from job set of time 1412958709000 ms
14/10/10 12:31:49 INFO scheduler.JobScheduler: Total delay: 0.035 s for time 1412958709000 ms (execution: 0.011 s)
14/10/10 12:31:49 INFO twitter4j.TwitterStreamImpl: Connection established.
14/10/10 12:31:49 INFO twitter4j.TwitterStreamImpl: Receiving status stream.
Storing Survey: 1 in 3 Seniors Went Without Medical Care Due to High Energy Prices http://t.co/O8Ry69ZxPq
14/10/10 12:31:50 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
RDD 1 0
14/10/10 12:31:50 INFO scheduler.JobScheduler: Added jobs for time 1412958710000 ms
14/10/10 12:31:50 INFO scheduler.JobScheduler: Starting job streaming job 1412958710000 ms.0 from job set of time 1412958710000 ms
14/10/10 12:31:50 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:50 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 3.9205E-5 s
14/10/10 12:31:50 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:50 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 2.0842E-5 s
14/10/10 12:31:50 INFO scheduler.JobScheduler: Finished job streaming job 1412958710000 ms.0 from job set of time 1412958710000 ms
14/10/10 12:31:50 INFO scheduler.JobScheduler: Total delay: 0.009 s for time 1412958710000 ms (execution: 0.004 s)
14/10/10 12:31:50 INFO rdd.FilteredRDD: Removing RDD 3 from persistence list
14/10/10 12:31:50 INFO storage.BlockManager: Removing RDD 3
14/10/10 12:31:50 INFO rdd.MappedRDD: Removing RDD 2 from persistence list
14/10/10 12:31:50 INFO storage.BlockManager: Removing RDD 2
14/10/10 12:31:50 INFO rdd.BlockRDD: Removing RDD 1 from persistence list
14/10/10 12:31:50 INFO storage.BlockManager: Removing RDD 1
14/10/10 12:31:50 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[1] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958710000 ms
Storing @xkelgx to u that sounds like a turn out but that's my medical kit looool
Storing RT @BrettSnodgrass1: Now this is deeply disturbing, not a social media post that offends a student or medical board. @GMCUK #MedEd 
14/10/10 12:31:51 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks

http:/…
14/10/10 12:31:51 INFO scheduler.JobScheduler: Added jobs for time 1412958711000 ms
14/10/10 12:31:51 INFO scheduler.JobScheduler: Starting job streaming job 1412958711000 ms.0 from job set of time 1412958711000 ms
14/10/10 12:31:51 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:51 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 3.4724E-5 s
14/10/10 12:31:51 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:51 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 1.7054E-5 s
14/10/10 12:31:51 INFO scheduler.JobScheduler: Finished job streaming job 1412958711000 ms.0 from job set of time 1412958711000 ms
14/10/10 12:31:51 INFO scheduler.JobScheduler: Total delay: 0.017 s for time 1412958711000 ms (execution: 0.003 s)
14/10/10 12:31:51 INFO rdd.FilteredRDD: Removing RDD 6 from persistence list
14/10/10 12:31:51 INFO storage.BlockManager: Removing RDD 6
14/10/10 12:31:51 INFO rdd.MappedRDD: Removing RDD 5 from persistence list
14/10/10 12:31:51 INFO storage.BlockManager: Removing RDD 5
RDD 2 0
14/10/10 12:31:51 INFO rdd.BlockRDD: Removing RDD 4 from persistence list
14/10/10 12:31:51 INFO storage.BlockManager: Removing RDD 4
14/10/10 12:31:51 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[4] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958711000 ms
Storing Calixte is currently undergoing a medical evaluation at the #Brockton District Court and is awaiting a bail hearing.
Storing RT @LynHornerAuthor: #Savelives #Amazon #ASMSG #BookBoost #IARTG @BarbaraEbel
#LiverAwarenessMonth
A CASCADING #MEDICAL #CRISIS!
OUTCOME ht…
14/10/10 12:31:52 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:31:52 INFO scheduler.JobScheduler: Added jobs for time 1412958712000 ms
14/10/10 12:31:52 INFO scheduler.JobScheduler: Starting job streaming job 1412958712000 ms.0 from job set of time 1412958712000 ms
14/10/10 12:31:52 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:52 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.3417E-5 s
14/10/10 12:31:52 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:52 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 1.7845E-5 s
14/10/10 12:31:52 INFO scheduler.JobScheduler: Finished job streaming job 1412958712000 ms.0 from job set of time 1412958712000 ms
14/10/10 12:31:52 INFO rdd.FilteredRDD: Removing RDD 9 from persistence list
14/10/10 12:31:52 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958712000 ms (execution: 0.004 s)
14/10/10 12:31:52 INFO storage.BlockManager: Removing RDD 9
14/10/10 12:31:52 INFO rdd.MappedRDD: Removing RDD 8 from persistence list
RDD 3 0
14/10/10 12:31:52 INFO storage.BlockManager: Removing RDD 8
14/10/10 12:31:52 INFO rdd.BlockRDD: Removing RDD 7 from persistence list
14/10/10 12:31:52 INFO storage.BlockManager: Removing RDD 7
14/10/10 12:31:52 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[7] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958712000 ms
Storing I'm raising money for Emergency Help Housing,Life,medical. Click to Donate: http://t.co/qUdQpxaJjO via @gofundme
14/10/10 12:31:53 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:31:53 INFO scheduler.JobScheduler: Added jobs for time 1412958713000 ms
14/10/10 12:31:53 INFO scheduler.JobScheduler: Starting job streaming job 1412958713000 ms.0 from job set of time 1412958713000 ms
RDD 4 0
14/10/10 12:31:53 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:53 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.5568E-5 s
14/10/10 12:31:53 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:53 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 1.7881E-5 s
14/10/10 12:31:53 INFO scheduler.JobScheduler: Finished job streaming job 1412958713000 ms.0 from job set of time 1412958713000 ms
14/10/10 12:31:53 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958713000 ms (execution: 0.004 s)
14/10/10 12:31:53 INFO rdd.FilteredRDD: Removing RDD 12 from persistence list
14/10/10 12:31:53 INFO storage.BlockManager: Removing RDD 12
14/10/10 12:31:53 INFO rdd.MappedRDD: Removing RDD 11 from persistence list
14/10/10 12:31:53 INFO storage.BlockManager: Removing RDD 11
14/10/10 12:31:53 INFO rdd.BlockRDD: Removing RDD 10 from persistence list
14/10/10 12:31:53 INFO storage.BlockManager: Removing RDD 10
14/10/10 12:31:53 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[10] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958713000 ms
Storing You're probably a cat - If you can conquer tension without medical help.
Storing Bilingual Medical Assistant - Miami, FL http://t.co/2LHvo4gftt
14/10/10 12:31:54 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
Storing @BZB so many things can and do go wrong. That’s why I have trouble understanding people who don’t want medical care in pregnancy/birth.
RDD 5 0
14/10/10 12:31:54 INFO scheduler.JobScheduler: Added jobs for time 1412958714000 ms
14/10/10 12:31:54 INFO scheduler.JobScheduler: Starting job streaming job 1412958714000 ms.0 from job set of time 1412958714000 ms
14/10/10 12:31:54 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:54 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 4.4516E-5 s
14/10/10 12:31:54 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:54 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 3.532E-5 s
14/10/10 12:31:54 INFO scheduler.JobScheduler: Finished job streaming job 1412958714000 ms.0 from job set of time 1412958714000 ms
14/10/10 12:31:54 INFO scheduler.JobScheduler: Total delay: 0.009 s for time 1412958714000 ms (execution: 0.005 s)
14/10/10 12:31:54 INFO rdd.FilteredRDD: Removing RDD 15 from persistence list
14/10/10 12:31:54 INFO storage.BlockManager: Removing RDD 15
14/10/10 12:31:54 INFO rdd.MappedRDD: Removing RDD 14 from persistence list
14/10/10 12:31:54 INFO storage.BlockManager: Removing RDD 14
14/10/10 12:31:54 INFO rdd.BlockRDD: Removing RDD 13 from persistence list
14/10/10 12:31:54 INFO storage.BlockManager: Removing RDD 13
14/10/10 12:31:54 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[13] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958714000 ms
Storing Check out this #job: #Medical #Assistant at #Cigna in #Chicago http://t.co/cnCYhe0JIz http://t.co/CA81aQO2N6
Storing RT @BrettSnodgrass1: @natlawreview @FTC How does 1 address corruption of Medical Boards? They lack oversight, assume truth of #PatientSafet…
Storing RT @DavidJo52951945: Labour peer backs UKIP call for immigrants with serious medical conditions not to be allowed into the UK http://t.co/E…
14/10/10 12:31:55 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:31:55 INFO scheduler.JobScheduler: Added jobs for time 1412958715000 ms
14/10/10 12:31:55 INFO scheduler.JobScheduler: Starting job streaming job 1412958715000 ms.0 from job set of time 1412958715000 ms
14/10/10 12:31:55 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:55 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.2807E-5 s
14/10/10 12:31:55 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:55 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 1.6911E-5 s
14/10/10 12:31:55 INFO scheduler.JobScheduler: Finished job streaming job 1412958715000 ms.0 from job set of time 1412958715000 ms
14/10/10 12:31:55 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958715000 ms (execution: 0.003 s)
14/10/10 12:31:55 INFO rdd.FilteredRDD: Removing RDD 18 from persistence list
14/10/10 12:31:55 INFO storage.BlockManager: Removing RDD 18
14/10/10 12:31:55 INFO rdd.MappedRDD: Removing RDD 17 from persistence list
14/10/10 12:31:55 INFO storage.BlockManager: Removing RDD 17
14/10/10 12:31:55 INFO rdd.BlockRDD: Removing RDD 16 from persistence list
14/10/10 12:31:55 INFO storage.BlockManager: Removing RDD 16
14/10/10 12:31:55 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[16] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958715000 ms
RDD 6 0
Storing ignorant people drive me CRAZY. b4 u spit out medical jargon that's over ur head to begin with, RESEARCH. u won't look like such an idiot.
14/10/10 12:31:56 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:31:56 INFO scheduler.JobScheduler: Added jobs for time 1412958716000 ms
14/10/10 12:31:56 INFO scheduler.JobScheduler: Starting job streaming job 1412958716000 ms.0 from job set of time 1412958716000 ms
RDD 7 0
14/10/10 12:31:56 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:56 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 3.345E-5 s
14/10/10 12:31:56 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:56 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 3.648E-5 s
14/10/10 12:31:56 INFO scheduler.JobScheduler: Finished job streaming job 1412958716000 ms.0 from job set of time 1412958716000 ms
14/10/10 12:31:56 INFO rdd.FilteredRDD: Removing RDD 21 from persistence list
14/10/10 12:31:56 INFO scheduler.JobScheduler: Total delay: 0.010 s for time 1412958716000 ms (execution: 0.005 s)
14/10/10 12:31:56 INFO storage.BlockManager: Removing RDD 21
14/10/10 12:31:56 INFO rdd.MappedRDD: Removing RDD 20 from persistence list
14/10/10 12:31:56 INFO storage.BlockManager: Removing RDD 20
14/10/10 12:31:56 INFO rdd.BlockRDD: Removing RDD 19 from persistence list
14/10/10 12:31:56 INFO storage.BlockManager: Removing RDD 19
14/10/10 12:31:56 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[19] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958716000 ms
Storing Bilingual Medical Assistant - Miami, FL http://t.co/zZpKatYf4b
14/10/10 12:31:57 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
RDD 8 0
14/10/10 12:31:57 INFO scheduler.JobScheduler: Added jobs for time 1412958717000 ms
14/10/10 12:31:57 INFO scheduler.JobScheduler: Starting job streaming job 1412958717000 ms.0 from job set of time 1412958717000 ms
14/10/10 12:31:57 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:57 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.1857E-5 s
14/10/10 12:31:57 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:57 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 1.9361E-5 s
14/10/10 12:31:57 INFO scheduler.JobScheduler: Finished job streaming job 1412958717000 ms.0 from job set of time 1412958717000 ms
14/10/10 12:31:57 INFO rdd.FilteredRDD: Removing RDD 24 from persistence list
14/10/10 12:31:57 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958717000 ms (execution: 0.003 s)
14/10/10 12:31:57 INFO storage.BlockManager: Removing RDD 24
14/10/10 12:31:57 INFO rdd.MappedRDD: Removing RDD 23 from persistence list
14/10/10 12:31:57 INFO storage.BlockManager: Removing RDD 23
14/10/10 12:31:57 INFO rdd.BlockRDD: Removing RDD 22 from persistence list
14/10/10 12:31:57 INFO storage.BlockManager: Removing RDD 22
14/10/10 12:31:57 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[22] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958717000 ms
14/10/10 12:31:58 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
RDD 9 0
14/10/10 12:31:58 INFO scheduler.JobScheduler: Added jobs for time 1412958718000 ms
14/10/10 12:31:58 INFO scheduler.JobScheduler: Starting job streaming job 1412958718000 ms.0 from job set of time 1412958718000 ms
14/10/10 12:31:58 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:58 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.141E-5 s
14/10/10 12:31:58 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:58 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 1.7445E-5 s
14/10/10 12:31:58 INFO scheduler.JobScheduler: Finished job streaming job 1412958718000 ms.0 from job set of time 1412958718000 ms
14/10/10 12:31:58 INFO scheduler.JobScheduler: Total delay: 0.007 s for time 1412958718000 ms (execution: 0.003 s)
14/10/10 12:31:58 INFO rdd.FilteredRDD: Removing RDD 27 from persistence list
14/10/10 12:31:58 INFO storage.BlockManager: Removing RDD 27
14/10/10 12:31:58 INFO rdd.MappedRDD: Removing RDD 26 from persistence list
14/10/10 12:31:58 INFO storage.BlockManager: Removing RDD 26
14/10/10 12:31:58 INFO rdd.BlockRDD: Removing RDD 25 from persistence list
14/10/10 12:31:58 INFO storage.BlockManager: Removing RDD 25
14/10/10 12:31:58 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[25] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958718000 ms
Storing Medical micro tests make me feel so stupid.
Storing Am I in a classroom or medical ward? Can't tell with all the coughing going on
14/10/10 12:31:59 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:31:59 INFO scheduler.JobScheduler: Added jobs for time 1412958719000 ms
14/10/10 12:31:59 INFO scheduler.JobScheduler: Starting job streaming job 1412958719000 ms.0 from job set of time 1412958719000 ms
RDD 10 0
14/10/10 12:31:59 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:31:59 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.6839E-5 s
14/10/10 12:31:59 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:31:59 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 2.0727E-5 s
14/10/10 12:31:59 INFO scheduler.JobScheduler: Finished job streaming job 1412958719000 ms.0 from job set of time 1412958719000 ms
14/10/10 12:31:59 INFO rdd.FilteredRDD: Removing RDD 30 from persistence list
14/10/10 12:31:59 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958719000 ms (execution: 0.004 s)
14/10/10 12:31:59 INFO storage.BlockManager: Removing RDD 30
14/10/10 12:31:59 INFO rdd.MappedRDD: Removing RDD 29 from persistence list
14/10/10 12:31:59 INFO storage.BlockManager: Removing RDD 29
14/10/10 12:31:59 INFO rdd.BlockRDD: Removing RDD 28 from persistence list
14/10/10 12:31:59 INFO storage.BlockManager: Removing RDD 28
14/10/10 12:31:59 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[28] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958719000 ms
Storing RT @UNMEER: "The highest priorities are #Ebola treatment centres with skilled managers and medical personnel." - @UNMEER Head Banbury to #U…
Storing General Pharmaceuticals Limited, Post: Medical Promotion Officer & Medical Information… http://t.co/7cWm1NUP58
14/10/10 12:32:00 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:32:00 INFO scheduler.JobScheduler: Added jobs for time 1412958720000 ms
14/10/10 12:32:00 INFO scheduler.JobScheduler: Starting job streaming job 1412958720000 ms.0 from job set of time 1412958720000 ms
RDD 11 0
14/10/10 12:32:00 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:00 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 3.7558E-5 s
14/10/10 12:32:00 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:00 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 2.351E-5 s
14/10/10 12:32:00 INFO scheduler.JobScheduler: Finished job streaming job 1412958720000 ms.0 from job set of time 1412958720000 ms
14/10/10 12:32:00 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958720000 ms (execution: 0.004 s)
14/10/10 12:32:00 INFO rdd.FilteredRDD: Removing RDD 33 from persistence list
14/10/10 12:32:00 INFO storage.BlockManager: Removing RDD 33
14/10/10 12:32:00 INFO rdd.MappedRDD: Removing RDD 32 from persistence list
14/10/10 12:32:00 INFO storage.BlockManager: Removing RDD 32
14/10/10 12:32:00 INFO rdd.BlockRDD: Removing RDD 31 from persistence list
14/10/10 12:32:00 INFO storage.BlockManager: Removing RDD 31
14/10/10 12:32:00 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[31] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958720000 ms
14/10/10 12:32:01 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:32:01 INFO scheduler.JobScheduler: Added jobs for time 1412958721000 ms
14/10/10 12:32:01 INFO scheduler.JobScheduler: Starting job streaming job 1412958721000 ms.0 from job set of time 1412958721000 ms
RDD 12 0
14/10/10 12:32:01 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:01 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 4.4982E-5 s
14/10/10 12:32:01 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:01 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 4.4584E-5 s
14/10/10 12:32:01 INFO scheduler.JobScheduler: Finished job streaming job 1412958721000 ms.0 from job set of time 1412958721000 ms
14/10/10 12:32:01 INFO scheduler.JobScheduler: Total delay: 0.012 s for time 1412958721000 ms (execution: 0.008 s)
14/10/10 12:32:01 INFO rdd.FilteredRDD: Removing RDD 36 from persistence list
14/10/10 12:32:01 INFO storage.BlockManager: Removing RDD 36
14/10/10 12:32:01 INFO rdd.MappedRDD: Removing RDD 35 from persistence list
14/10/10 12:32:01 INFO storage.BlockManager: Removing RDD 35
14/10/10 12:32:01 INFO rdd.BlockRDD: Removing RDD 34 from persistence list
14/10/10 12:32:01 INFO storage.BlockManager: Removing RDD 34
14/10/10 12:32:01 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[34] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958721000 ms
14/10/10 12:32:02 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
RDD 13 0
14/10/10 12:32:02 INFO scheduler.JobScheduler: Added jobs for time 1412958722000 ms
14/10/10 12:32:02 INFO scheduler.JobScheduler: Starting job streaming job 1412958722000 ms.0 from job set of time 1412958722000 ms
14/10/10 12:32:02 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:02 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.5593E-5 s
14/10/10 12:32:02 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:02 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 1.7689E-5 s
14/10/10 12:32:02 INFO scheduler.JobScheduler: Finished job streaming job 1412958722000 ms.0 from job set of time 1412958722000 ms
14/10/10 12:32:02 INFO rdd.FilteredRDD: Removing RDD 39 from persistence list
14/10/10 12:32:02 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958722000 ms (execution: 0.004 s)
14/10/10 12:32:02 INFO storage.BlockManager: Removing RDD 39
14/10/10 12:32:02 INFO rdd.MappedRDD: Removing RDD 38 from persistence list
14/10/10 12:32:02 INFO storage.BlockManager: Removing RDD 38
14/10/10 12:32:02 INFO rdd.BlockRDD: Removing RDD 37 from persistence list
14/10/10 12:32:02 INFO storage.BlockManager: Removing RDD 37
14/10/10 12:32:02 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[37] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958722000 ms
Storing RT @khanjangli: Do you have the stomach to check out all 15 of these disgusting medical procedures? ====> http://t.co/AY6RHDUFUZ http://t.c…
14/10/10 12:32:03 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:32:03 INFO scheduler.JobScheduler: Added jobs for time 1412958723000 ms
14/10/10 12:32:03 INFO scheduler.JobScheduler: Starting job streaming job 1412958723000 ms.0 from job set of time 1412958723000 ms
14/10/10 12:32:03 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:03 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.456E-5 s
RDD 14 0
14/10/10 12:32:03 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:03 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 2.6834E-5 s
14/10/10 12:32:03 INFO scheduler.JobScheduler: Finished job streaming job 1412958723000 ms.0 from job set of time 1412958723000 ms
14/10/10 12:32:03 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958723000 ms (execution: 0.003 s)
14/10/10 12:32:03 INFO rdd.FilteredRDD: Removing RDD 42 from persistence list
14/10/10 12:32:03 INFO storage.BlockManager: Removing RDD 42
14/10/10 12:32:03 INFO rdd.MappedRDD: Removing RDD 41 from persistence list
14/10/10 12:32:03 INFO storage.BlockManager: Removing RDD 41
14/10/10 12:32:03 INFO rdd.BlockRDD: Removing RDD 40 from persistence list
14/10/10 12:32:03 INFO storage.BlockManager: Removing RDD 40
14/10/10 12:32:03 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[40] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958723000 ms
14/10/10 12:32:04 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
RDD 15 0
14/10/10 12:32:04 INFO scheduler.JobScheduler: Added jobs for time 1412958724000 ms
14/10/10 12:32:04 INFO scheduler.JobScheduler: Starting job streaming job 1412958724000 ms.0 from job set of time 1412958724000 ms
14/10/10 12:32:04 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:04 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.4787E-5 s
14/10/10 12:32:04 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:04 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 2.319E-5 s
14/10/10 12:32:04 INFO scheduler.JobScheduler: Finished job streaming job 1412958724000 ms.0 from job set of time 1412958724000 ms
14/10/10 12:32:04 INFO rdd.FilteredRDD: Removing RDD 45 from persistence list
14/10/10 12:32:04 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958724000 ms (execution: 0.004 s)
14/10/10 12:32:04 INFO storage.BlockManager: Removing RDD 45
14/10/10 12:32:04 INFO rdd.MappedRDD: Removing RDD 44 from persistence list
14/10/10 12:32:04 INFO storage.BlockManager: Removing RDD 44
14/10/10 12:32:04 INFO rdd.BlockRDD: Removing RDD 43 from persistence list
14/10/10 12:32:04 INFO storage.BlockManager: Removing RDD 43
14/10/10 12:32:04 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[43] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958724000 ms
Storing RT @JoneseysWords: Absolutely read. Reflect. Update your thinking: Teen died from gunshot to right cheek, medical examiner says http://t.co…
Storing Schizophrenia: shattering the stigma - Medical News Today http://t.co/BLBqMRlSWb #google #health
14/10/10 12:32:05 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:32:05 INFO scheduler.JobScheduler: Added jobs for time 1412958725000 ms
14/10/10 12:32:05 INFO scheduler.JobScheduler: Starting job streaming job 1412958725000 ms.0 from job set of time 1412958725000 ms
14/10/10 12:32:05 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:05 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 3.2758E-5 s
14/10/10 12:32:05 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:05 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 2.2976E-5 s
14/10/10 12:32:05 INFO scheduler.JobScheduler: Finished job streaming job 1412958725000 ms.0 from job set of time 1412958725000 ms
14/10/10 12:32:05 INFO scheduler.JobScheduler: Total delay: 0.008 s for time 1412958725000 ms (execution: 0.004 s)
14/10/10 12:32:05 INFO rdd.FilteredRDD: Removing RDD 48 from persistence list
14/10/10 12:32:05 INFO storage.BlockManager: Removing RDD 48
14/10/10 12:32:05 INFO rdd.MappedRDD: Removing RDD 47 from persistence list
14/10/10 12:32:05 INFO storage.BlockManager: Removing RDD 47
RDD 16 0
14/10/10 12:32:05 INFO rdd.BlockRDD: Removing RDD 46 from persistence list
14/10/10 12:32:05 INFO storage.BlockManager: Removing RDD 46
14/10/10 12:32:05 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[46] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958725000 ms
Storing Medical school moralization volume is slabby: EwMKTi http://t.co/LSjhuOJfFJ
14/10/10 12:32:06 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:32:06 INFO scheduler.JobScheduler: Added jobs for time 1412958726000 ms
14/10/10 12:32:06 INFO scheduler.JobScheduler: Starting job streaming job 1412958726000 ms.0 from job set of time 1412958726000 ms
14/10/10 12:32:06 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:06 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.2276E-5 s
14/10/10 12:32:06 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:06 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 1.5381E-5 s
14/10/10 12:32:06 INFO scheduler.JobScheduler: Finished job streaming job 1412958726000 ms.0 from job set of time 1412958726000 ms
14/10/10 12:32:06 INFO rdd.FilteredRDD: Removing RDD 51 from persistence list
14/10/10 12:32:06 INFO scheduler.JobScheduler: Total delay: 0.007 s for time 1412958726000 ms (execution: 0.003 s)
14/10/10 12:32:06 INFO storage.BlockManager: Removing RDD 51
14/10/10 12:32:06 INFO rdd.MappedRDD: Removing RDD 50 from persistence list
14/10/10 12:32:06 INFO storage.BlockManager: Removing RDD 50
RDD 17 0
14/10/10 12:32:06 INFO rdd.BlockRDD: Removing RDD 49 from persistence list
14/10/10 12:32:06 INFO storage.BlockManager: Removing RDD 49
14/10/10 12:32:06 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[49] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958726000 ms
14/10/10 12:32:07 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:32:07 INFO scheduler.JobScheduler: Added jobs for time 1412958727000 ms
14/10/10 12:32:07 INFO scheduler.JobScheduler: Starting job streaming job 1412958727000 ms.0 from job set of time 1412958727000 ms
14/10/10 12:32:07 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:07 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.2817E-5 s
14/10/10 12:32:07 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:07 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 1.6015E-5 s
14/10/10 12:32:07 INFO scheduler.JobScheduler: Finished job streaming job 1412958727000 ms.0 from job set of time 1412958727000 ms
14/10/10 12:32:07 INFO scheduler.JobScheduler: Total delay: 0.007 s for time 1412958727000 ms (execution: 0.003 s)
14/10/10 12:32:07 INFO rdd.FilteredRDD: Removing RDD 54 from persistence list
14/10/10 12:32:07 INFO storage.BlockManager: Removing RDD 54
14/10/10 12:32:07 INFO rdd.MappedRDD: Removing RDD 53 from persistence list
14/10/10 12:32:07 INFO storage.BlockManager: Removing RDD 53
14/10/10 12:32:07 INFO rdd.BlockRDD: Removing RDD 52 from persistence list
14/10/10 12:32:07 INFO storage.BlockManager: Removing RDD 52
14/10/10 12:32:07 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[52] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958727000 ms
RDD 18 0
14/10/10 12:32:08 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:32:08 INFO scheduler.JobScheduler: Added jobs for time 1412958728000 ms
14/10/10 12:32:08 INFO scheduler.JobScheduler: Starting job streaming job 1412958728000 ms.0 from job set of time 1412958728000 ms
14/10/10 12:32:08 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:08 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 3.235E-5 s
RDD 19 0
14/10/10 12:32:08 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:08 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 2.5517E-5 s
14/10/10 12:32:08 INFO scheduler.JobScheduler: Finished job streaming job 1412958728000 ms.0 from job set of time 1412958728000 ms
14/10/10 12:32:08 INFO scheduler.JobScheduler: Total delay: 0.014 s for time 1412958728000 ms (execution: 0.005 s)
14/10/10 12:32:08 INFO rdd.FilteredRDD: Removing RDD 57 from persistence list
14/10/10 12:32:08 INFO storage.BlockManager: Removing RDD 57
14/10/10 12:32:08 INFO rdd.MappedRDD: Removing RDD 56 from persistence list
14/10/10 12:32:08 INFO storage.BlockManager: Removing RDD 56
14/10/10 12:32:08 INFO rdd.BlockRDD: Removing RDD 55 from persistence list
14/10/10 12:32:08 INFO storage.BlockManager: Removing RDD 55
14/10/10 12:32:08 INFO sparkapps.TwitterInputDStreamJ: Removing blocks of RDD BlockRDD[55] at BlockRDD at ReceiverInputDStream.scala:69 of time 1412958728000 ms
14/10/10 12:32:09 INFO scheduler.ReceiverTracker: Stream 0 received 0 blocks
14/10/10 12:32:09 INFO scheduler.JobScheduler: Added jobs for time 1412958729000 ms
14/10/10 12:32:09 INFO scheduler.JobScheduler: Starting job streaming job 1412958729000 ms.0 from job set of time 1412958729000 ms
RDD 20 0
14/10/10 12:32:09 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:102
14/10/10 12:32:09 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:102, took 2.3605E-5 s
14/10/10 12:32:09 INFO spark.SparkContext: Starting job: count at DataBricksTwitterApp.scala:103
14/10/10 12:32:09 INFO spark.SparkContext: Job finished: count at DataBricksTwitterApp.scala:103, took 2.9644E-5 s
14/10/10 12:32:09 INFO scheduler.ReceiverTracker: Sent stop signal to all 1 receivers
14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Received stop signal
14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Stopping receiver with message: Stopped by driver: 
14/10/10 12:32:09 INFO twitter4j.TwitterStreamImpl: Inflater has been closed
14/10/10 12:32:09 INFO sparkapps.TwitterReceiver: Twitter receiver stopped
14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Called receiver onStop
14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Deregistering receiver 0
14/10/10 12:32:09 WARN receiver.ReceiverSupervisorImpl: Restarting receiver with delay 2000 ms: Error receiving tweets
java.lang.NullPointerException: Inflater has been closed
	at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
	at java.util.zip.Inflater.inflate(Inflater.java:257)
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
	at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:116)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.fill(BufferedReader.java:154)
	at java.io.BufferedReader.readLine(BufferedReader.java:317)
	at java.io.BufferedReader.readLine(BufferedReader.java:382)
	at twitter4j.StatusStreamBase.handleNextElement(StatusStreamBase.java:85)
	at twitter4j.StatusStreamImpl.next(StatusStreamImpl.java:57)
	at twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:478)
14/10/10 12:32:09 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver
14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Stopped receiver 0
14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Stopping receiver with message: Restarting receiver with delay 2000ms: Error receiving tweets: java.lang.NullPointerException: Inflater has been closed
14/10/10 12:32:09 INFO sparkapps.TwitterReceiver: Twitter receiver stopped
14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Called receiver onStop
14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Deregistering receiver 0
14/10/10 12:32:09 INFO receiver.BlockGenerator: Stopping BlockGenerator
14/10/10 12:32:09 ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Restarting receiver with delay 2000ms: Error receiving tweets - java.lang.NullPointerException: Inflater has been closed
	at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
	at java.util.zip.Inflater.inflate(Inflater.java:257)
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
	at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:116)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.fill(BufferedReader.java:154)
	at java.io.BufferedReader.readLine(BufferedReader.java:317)
	at java.io.BufferedReader.readLine(BufferedReader.java:382)
	at twitter4j.StatusStreamBase.handleNextElement(StatusStreamBase.java:85)
	at twitter4j.StatusStreamImpl.next(StatusStreamImpl.java:57)
	at twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:478)

14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Stopped receiver 0
14/10/10 12:32:09 INFO util.RecurringTimer: Stopped timer for BlockGenerator after time 1412958729200
14/10/10 12:32:09 INFO receiver.BlockGenerator: Waiting for block pushing thread
14/10/10 12:32:09 INFO receiver.BlockGenerator: Pushing out the last 0 blocks
14/10/10 12:32:09 INFO receiver.BlockGenerator: Stopped block pushing thread
14/10/10 12:32:09 INFO receiver.BlockGenerator: Stopped BlockGenerator
14/10/10 12:32:09 INFO receiver.ReceiverSupervisorImpl: Waiting for executor stop is over
14/10/10 12:32:09 WARN receiver.ReceiverSupervisorImpl: Stopped executor without error
14/10/10 12:32:09 INFO executor.Executor: Finished task 0.0 in stage 0.0 (TID 0). 826 bytes result sent to driver
14/10/10 12:32:09 INFO scheduler.TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 20674 ms on localhost (1/1)
14/10/10 12:32:09 INFO scheduler.DAGScheduler: Stage 0 (runJob at ReceiverTracker.scala:275) finished in 20.684 s
14/10/10 12:32:09 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 
14/10/10 12:32:09 INFO spark.SparkContext: Job finished: runJob at ReceiverTracker.scala:275, took 20.855293194 s
14/10/10 12:32:09 INFO scheduler.ReceiverTracker: All of the receivers have been terminated
14/10/10 12:32:09 WARN scheduler.ReceiverTracker: All of the receivers have not deregistered, Map(0 -> ReceiverInfo(0,TwitterReceiver-0,null,false,localhost,Restarting receiver with delay 2000ms: Error receiving tweets,java.lang.NullPointerException: Inflater has been closed
	at java.util.zip.Inflater.ensureOpen(Inflater.java:389)
	at java.util.zip.Inflater.inflate(Inflater.java:257)
	at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:152)
	at java.util.zip.GZIPInputStream.read(GZIPInputStream.java:116)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.fill(BufferedReader.java:154)
	at java.io.BufferedReader.readLine(BufferedReader.java:317)
	at java.io.BufferedReader.readLine(BufferedReader.java:382)
	at twitter4j.StatusStreamBase.handleNextElement(StatusStreamBase.java:85)
	at twitter4j.StatusStreamImpl.next(StatusStreamImpl.java:57)
	at twitter4j.TwitterStreamImpl$TwitterStreamConsumer.run(TwitterStreamImpl.java:478)
))
14/10/10 12:32:09 INFO scheduler.ReceiverTracker: ReceiverTracker stopped
14/10/10 12:32:09 INFO scheduler.JobGenerator: Stopping JobGenerator immediately
14/10/10 12:32:09 INFO util.RecurringTimer: Stopped timer for JobGenerator after time 1412958729000
14/10/10 12:32:09 INFO scheduler.JobGenerator: Stopped JobGenerator
14/10/10 12:32:11 INFO receiver.ReceiverSupervisorImpl: Starting receiver again
14/10/10 12:32:11 INFO receiver.ReceiverSupervisorImpl: Starting receiver
14/10/10 12:32:11 INFO sparkapps.TwitterReceiver: Twitter receiver started
14/10/10 12:32:11 INFO twitter4j.TwitterStreamImpl: Establishing connection.
14/10/10 12:32:11 INFO receiver.ReceiverSupervisorImpl: Called receiver onStart
14/10/10 12:32:11 INFO scheduler.JobScheduler: Stopped JobScheduler
14/10/10 12:32:11 INFO streaming.StreamingContext: StreamingContext stopped successfully
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/streaming/json,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/streaming,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/metrics/json,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/stages/stage/kill,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/static,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/executors/json,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/executors,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/environment/json,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/environment,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/storage/rdd/json,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/storage/rdd,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/storage/json,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/storage,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/stages/pool/json,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/stages/pool,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/stages/stage/json,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/stages/stage,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/stages/json,null}
14/10/10 12:32:11 INFO handler.ContextHandler: stopped o.e.j.s.ServletContextHandler{/stages,null}
14/10/10 12:32:11 INFO ui.SparkUI: Stopped Spark web UI at http://dhcp-25-121.bos.redhat.com:4040
14/10/10 12:32:11 INFO scheduler.DAGScheduler: Stopping DAGScheduler
14/10/10 12:32:11 INFO twitter4j.TwitterStreamImpl: Connection established.
14/10/10 12:32:11 INFO twitter4j.TwitterStreamImpl: Receiving status stream.
Storing #Job #Kingston Medical Technologist - Clinical Laboratory Scientist - (Med Tech - MT - CLS): Poughkeepsie Job ... http://t.co/JtIPCkPwR2
Storing #Job #Kingston Medical Laboratory Technician - (Med Lab Tech - MLT): Poughkeepsie Job Description & Requiremen... http://t.co/KlhIyACpG1
14/10/10 12:32:12 INFO spark.MapOutputTrackerMasterActor: MapOutputTrackerActor stopped!
14/10/10 12:32:12 INFO network.ConnectionManager: Selector thread was interrupted!
14/10/10 12:32:12 INFO network.ConnectionManager: ConnectionManager stopped
14/10/10 12:32:12 INFO storage.MemoryStore: MemoryStore cleared
14/10/10 12:32:12 INFO storage.BlockManager: BlockManager stopped
14/10/10 12:32:12 INFO storage.BlockManagerMaster: BlockManagerMaster stopped
14/10/10 12:32:12 INFO spark.SparkContext: Successfully stopped SparkContext 

{noformat}


> BigTop/Hadoop cTAKES integration
> --------------------------------
>
>                 Key: CTAKES-314
>                 URL: https://issues.apache.org/jira/browse/CTAKES-314
>             Project: cTAKES
>          Issue Type: New Feature
>    Affects Versions: 3.2.0
>            Reporter: Pei Chen
>             Fix For: 3.2.3
>
>         Attachments: Napkin_cTAKES_Hadoop.JPG
>
>
> Placeholder to-
> Create a simple application that can take in different datasources (public forums, twitter, etc.), scale up cTAKES using BigTop/Hadoop ecosystem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message