flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hagersaleh <loveallah1...@yahoo.com>
Subject when run progrm on big data customer 2.5GB orders 5GB disply error why
Date Thu, 18 Jun 2015 19:51:10 GMT
when run progrm on big data customer 2.5GB orders 5GB disply error why


DataSource (at getCustomerDataSet(TPCHQuery3.java:252)
(org.apache.flink.api.java.io.CsvInputFormat)) (1/1) switched to FAILED
org.apache.flink.api.common.io.ParseException: Row too short:
14999999|Customer#014999999|3emQ49UZtlfjeK
at
org.apache.flink.api.common.io.GenericCsvInputFormat.parseRecord(GenericCsvInputFormat.java:283)
at
org.apache.flink.api.java.io.CsvInputFormat.readRecord(CsvInputFormat.java:236)
at
org.apache.flink.api.java.io.CsvInputFormat.readRecord(CsvInputFormat.java:42)
at
org.apache.flink.api.common.io.DelimitedInputFormat.nextRecord(DelimitedInputFormat.java:489)
at
org.apache.flink.api.java.io.CsvInputFormat.nextRecord(CsvInputFormat.java:204)
at
org.apache.flink.api.java.io.CsvInputFormat.nextRecord(CsvInputFormat.java:42)
at
org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:148)
at
org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:257)
at java.lang.Thread.run(Thread.java:724)


ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();

DataSet<Customer> customers = getCustomerDataSet(env,mask);

DataSet<Orders> orders= getOrdersDataSet(env,maskorder);

DefaultJoin<Customer,Orders> result =
customers.join(orders).where(0).equalTo(1);

result.print();
result .writeAsCsv("/home/hadoop/Desktop/Dataset/inoptmization.csv", "\n",
"|",WriteMode.OVERWRITE);
env.execute();
}
public static class Customer extends Tuple3<Long,String,String> {

}
public static class Orders extends Tuple2<Long,Long> {

}
private static DataSet<Customer> getCustomerDataSet(ExecutionEnvironment
env,String mask) {
return env.readCsvFile("/home/hadoop/Desktop/Dataset/customer.csv")
.fieldDelimiter('|')
.includeFields(mask).ignoreFirstLine()
.tupleType(Customer.class);


} 





--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/when-run-progrm-on-big-data-customer-2-5GB-orders-5GB-disply-error-why-tp1699.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message