spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yssharma <...@git.apache.org>
Subject [GitHub] spark issue #18029: [SPARK-20168][WIP][DStream] Add changes to use kinesis f...
Date Mon, 05 Jun 2017 13:34:46 GMT
Github user yssharma commented on the issue:

    https://github.com/apache/spark/pull/18029
  
    
    ```
    ./bin/spark-shell --jars /Users/ysharma/.m2/repository/org/apache/spark/spark-streaming-kinesis-asl_2.11/2.3.0-SNAPSHOT/spark-streaming-kinesis-asl_2.11-2.3.0-SNAPSHOT.jar
--packages com.amazonaws:amazon-kinesis-client:1.0.0 -Ylog-classpath
    
    
    import org.apache.spark.SparkConf
    import org.apache.spark.storage.StorageLevel
    import org.apache.spark.streaming.kinesis.KinesisInputDStream
    import org.apache.spark.streaming.{Seconds, StreamingContext}
    import com.amazonaws.services.kinesis.clientlibrary.lib.worker.InitialPositionInStream
    
    
    val streamingContext = new StreamingContext(sc, Seconds(30))
    val kinesisStream = KinesisInputDStream.builder
        .streamingContext(streamingContext)
        .endpointUrl("https://kinesis.us-east-1.amazonaws.com")
        .regionName("us-east-1")
        .streamName("a-very-nice-kinesis-app")
        .initialPositionInStream(InitialPositionInStream.TRIM_HORIZON)
        .checkpointAppName("a-very-nice-kinesis-app")
        .checkpointInterval(Seconds(30))
        .storageLevel(StorageLevel.MEMORY_AND_DISK_2)
        .build()
    
    Error:
    error: bad symbolic reference. A signature in KinesisInputDStream.class refers to term
InterfaceStability
    in package org.apache.spark.annotation which is not available.
    It may be completely missing from the current classpath, or the version on
    the classpath might be incompatible with the version used when compiling KinesisInputDStream.class.
    
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message