hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Biju .B" <bijub...@gmail.com>
Subject Help for Sqlserver querying with hadoop
Date Fri, 24 Sep 2010 08:36:45 GMT
Hi

Need urgent help on using sql server with hadoop

am using following code to connect to database

DBConfiguration.configureDB(conf,"com.microsoft.sqlserver.jdbc.SQLServerDriver","jdbc:sqlserver://xxx.xxx.xxx.xxx;user=abc;password=abc;DatabaseName=dbname");
String [] fields = { "id", "url" };
String [] fields = { "id", "url" };
DBInputFormat.setInput(conf,MyRecord.class,"urls",null,"id", fields);

Am getting following error

10/09/24 13:26:42 INFO mapred.JobClient: Task Id :
attempt_201009231924_0008_m_000001_2, Status : FAILED
java.io.IOException: Incorrect syntax near 'LIMIT'.
    at
org.apache.hadoop.mapreduce.lib.db.DBRecordReader.nextKeyValue(DBRecordReader.java:235)
    at
org.apache.hadoop.mapreduce.lib.db.DBRecordReader.next(DBRecordReader.java:204)
    at
org.apache.hadoop.mapred.lib.db.DBInputFormat$DBRecordReaderWrapper.next(DBInputFormat.java:118)
    at
org.apache.hadoop.mapred.lib.db.DBInputFormat$DBRecordReaderWrapper.next(DBInputFormat.java:87)
    at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:192)
    at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:176)
    at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.Child.main(Child.java:170)



Found that the error is due to query that each task tries to execute

SELECT id, url FROM urls AS urls ORDER BY id LIMIT 13228 OFFSET 13228


the "LIMIT" and "OFFSET" are not valid in Sqlserver and it returns error

Pls tell me how to solve this problem

Regards
Biju

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message