apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (APEXMALHAR-2172) Update JDBC poll input operator to fix issues
Date Tue, 09 Aug 2016 07:57:20 GMT

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413150#comment-15413150
] 

ASF GitHub Bot commented on APEXMALHAR-2172:
--------------------------------------------

Github user bhupeshchawda commented on a diff in the pull request:

    https://github.com/apache/apex-malhar/pull/358#discussion_r74013531
  
    --- Diff: library/src/main/java/com/datatorrent/lib/db/jdbc/JdbcPOJOPollInputOperator.java
---
    @@ -0,0 +1,321 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + *   http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing,
    + * software distributed under the License is distributed on an
    + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    + * KIND, either express or implied.  See the License for the
    + * specific language governing permissions and limitations
    + * under the License.
    + */
    +package com.datatorrent.lib.db.jdbc;
    +
    +import java.math.BigDecimal;
    +import java.sql.Date;
    +import java.sql.PreparedStatement;
    +import java.sql.ResultSet;
    +import java.sql.ResultSetMetaData;
    +import java.sql.SQLException;
    +import java.sql.Time;
    +import java.sql.Timestamp;
    +import java.sql.Types;
    +import java.util.List;
    +import java.util.Map;
    +
    +import javax.validation.constraints.NotNull;
    +
    +import org.slf4j.Logger;
    +import org.slf4j.LoggerFactory;
    +
    +import org.apache.hadoop.classification.InterfaceStability.Evolving;
    +
    +import com.google.common.collect.Lists;
    +import com.google.common.collect.Maps;
    +
    +import com.datatorrent.api.Context;
    +import com.datatorrent.api.Context.OperatorContext;
    +import com.datatorrent.api.DefaultOutputPort;
    +import com.datatorrent.api.annotation.OutputPortFieldAnnotation;
    +import com.datatorrent.lib.db.jdbc.JdbcPOJOInputOperator.ActiveFieldInfo;
    +import com.datatorrent.lib.util.FieldInfo;
    +import com.datatorrent.lib.util.PojoUtils;
    +
    +/**
    + * A concrete implementation for {@link AbstractJdbcPollInputOperator} for
    + * consuming data from MySQL using JDBC interface <br>
    + * User needs to provide tableName,dbConnection,setEmitColumnList,look-up key <br>
    + * Optionally batchSize,pollInterval,Look-up key and a where clause can be given <br>
    + * This operator uses static partitioning to arrive at range queries for exactly
    + * once reads<br>
    + * Assumption is that there is an ordered column using which range queries can
    + * be formed<br>
    + * If an emitColumnList is provided, please ensure that the keyColumn is the
    + * first column in the list<br>
    + * Range queries are formed using the {@link JdbcMetaDataUtility} Output - comma
    + * separated list of the emit columns eg columnA,columnB,columnC
    + * 
    + * @displayName Jdbc Polling Input Operator
    + * @category Input
    + * @tags database, sql, jdbc
    + */
    +@Evolving
    +public class JdbcPOJOPollInputOperator extends AbstractJdbcPollInputOperator<Object>
    +{
    +  private final transient List<ActiveFieldInfo> columnFieldSetters = Lists.newArrayList();
    +  protected List<Integer> columnDataTypes;
    +  protected transient Class<?> pojoClass;
    +  @NotNull
    +  private List<FieldInfo> fieldInfos;
    --- End diff --
    
    For consistency with Jdbc output operator, can you please use ```JdbcFieldInfo``` instead?


> Update JDBC poll input operator to fix issues
> ---------------------------------------------
>
>                 Key: APEXMALHAR-2172
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2172
>             Project: Apache Apex Malhar
>          Issue Type: Improvement
>            Reporter: Priyanka Gugale
>            Assignee: Priyanka Gugale
>
> Update JDBCPollInputOperator to:
> 1. Fix small bugs
> 2. Use jooq query dsl library to construct sql queries
> 3. Make code more readable
> 4. Use row counts rather than key column values to partition reads



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message