hudi-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [incubator-hudi] tooptoop4 opened a new issue #1359: [SUPPORT]
Date Wed, 26 Feb 2020 17:08:47 GMT
tooptoop4 opened a new issue #1359: [SUPPORT]
URL: https://github.com/apache/incubator-hudi/issues/1359
 
 
   I am writing data to S3 and partitioning on a string column that may contain colon. How
to make hudi handle this?
   
   2020-02-26 15:28:15,404 [task-result-getter-0] WARN  org.apache.spark.scheduler.TaskSetManager
- Lost task 0.0 in stage 7.0 (TID 51, 10.166.209.219, executor 0): java.lang.RuntimeException:
com.uber.hoodie.exception.HoodieException: com.uber.hoodie.exception.HoodieException: java.util.concurrent.ExecutionException:
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute
URI: 2007-09-26 13:14:35.981
   	at com.uber.hoodie.func.LazyIterableIterator.next(LazyIterableIterator.java:121)
   	at scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:43)
   	at scala.collection.Iterator$$anon$12.nextCur(Iterator.scala:434)
   	at scala.collection.Iterator$$anon$12.hasNext(Iterator.scala:440)
   	at org.apache.spark.storage.memory.MemoryStore.putIteratorAsBytes(MemoryStore.scala:378)
   	at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1111)
   	at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1085)
   	at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:1020)
   	at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1085)
   	at org.apache.spark.storage.BlockManager.getOrElseUpdate(BlockManager.scala:811)
   	at org.apache.spark.rdd.RDD.getOrCompute(RDD.scala:335)
   	at org.apache.spark.rdd.RDD.iterator(RDD.scala:286)
   	at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:49)
   	at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
   	at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
   	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
   	at org.apache.spark.scheduler.Task.run(Task.scala:109)
   	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   Caused by: com.uber.hoodie.exception.HoodieException: com.uber.hoodie.exception.HoodieException:
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: java.net.URISyntaxException:
Relative path in absolute URI: 2007-09-26 13:14:35.981
   	at com.uber.hoodie.func.CopyOnWriteLazyInsertIterable.computeNext(CopyOnWriteLazyInsertIterable.java:106)
   	at com.uber.hoodie.func.CopyOnWriteLazyInsertIterable.computeNext(CopyOnWriteLazyInsertIterable.java:45)
   	at com.uber.hoodie.func.LazyIterableIterator.next(LazyIterableIterator.java:119)
   	... 20 more
   Caused by: com.uber.hoodie.exception.HoodieException: java.util.concurrent.ExecutionException:
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute
URI: 2007-09-26 13:14:35.981
   	at com.uber.hoodie.common.util.queue.BoundedInMemoryExecutor.execute(BoundedInMemoryExecutor.java:146)
   	at com.uber.hoodie.func.CopyOnWriteLazyInsertIterable.computeNext(CopyOnWriteLazyInsertIterable.java:102)
   	... 22 more
   Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException:
java.net.URISyntaxException: Relative path in absolute URI: 2007-09-26 13:14:35.981
   	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
   	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
   	at com.uber.hoodie.common.util.queue.BoundedInMemoryExecutor.execute(BoundedInMemoryExecutor.java:144)
   	... 23 more
   Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path
in absolute URI: 2007-09-26 13:14:35.981
   	at org.apache.hadoop.fs.Path.initialize(Path.java:254)
   	at org.apache.hadoop.fs.Path.<init>(Path.java:212)
   	at org.apache.hadoop.fs.Path.<init>(Path.java:120)
   	at org.apache.hadoop.fs.Globber.doGlob(Globber.java:228)
   	at org.apache.hadoop.fs.Globber.glob(Globber.java:148)
   	at org.apache.hadoop.fs.FileSystem.globStatus(FileSystem.java:1686)
   	at org.apache.hadoop.fs.s3a.S3AFileSystem.globStatus(S3AFileSystem.java:2020)
   	at com.uber.hoodie.io.HoodieIOHandle.cleanupTmpFilesFromCurrentCommit(HoodieIOHandle.java:76)
   	at com.uber.hoodie.func.CopyOnWriteLazyInsertIterable$CopyOnWriteInsertHandler.consumeOneRecord(CopyOnWriteLazyInsertIterable.java:141)
   	at com.uber.hoodie.func.CopyOnWriteLazyInsertIterable$CopyOnWriteInsertHandler.consumeOneRecord(CopyOnWriteLazyInsertIterable.java:127)
   	at com.uber.hoodie.common.util.queue.BoundedInMemoryQueueConsumer.consume(BoundedInMemoryQueueConsumer.java:38)
   	at com.uber.hoodie.common.util.queue.BoundedInMemoryExecutor.lambda$null$77(BoundedInMemoryExecutor.java:124)
   	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   	... 3 more
   Caused by: java.net.URISyntaxException: Relative path in absolute URI: 2007-09-26 13:14:35.981
   	at java.net.URI.checkPath(URI.java:1823)
   	at java.net.URI.<init>(URI.java:745)
   	at org.apache.hadoop.fs.Path.initialize(Path.java:251)
   	... 15 more

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message