beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-3437) Support schema in PCollections
Date Tue, 03 Apr 2018 00:05:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-3437?focusedWorklogId=86864&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-86864
]

ASF GitHub Bot logged work on BEAM-3437:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 03/Apr/18 00:04
            Start Date: 03/Apr/18 00:04
    Worklog Time Spent: 10m 
      Work Description: akedin commented on a change in pull request #4964: [BEAM-3437] Introduce
Schema class, and use it in BeamSQL
URL: https://github.com/apache/beam/pull/4964#discussion_r178680493
 
 

 ##########
 File path: sdks/java/core/src/main/java/org/apache/beam/sdk/values/Row.java
 ##########
 @@ -75,191 +83,230 @@ public static Row nullRow(RowType rowType) {
    * if type doesn't match.
    */
   public <T> T getValue(String fieldName) {
-    return getValue(getRowType().indexOf(fieldName));
+    return getValue(getSchema().indexOf(fieldName));
   }
 
   /**
    * Get value by field index, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * if schema doesn't match.
    */
   @Nullable
   public <T> T getValue(int fieldIdx) {
     return (T) getValues().get(fieldIdx);
   }
 
   /**
-   * Get a {@link Byte} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#BYTE} value by field name, {@link IllegalStateException} is thrown
+   * if schema doesn't match.
    */
-  public Byte getByte(String fieldName) {
-    return getValue(fieldName);
+  public byte getByte(String fieldName) {
+    return getByte(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link Short} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#INT16} value by field name, {@link IllegalStateException} is thrown
+   * if schema doesn't match.
    */
-  public Short getShort(String fieldName) {
-    return getValue(fieldName);
+  public short getInt16(String fieldName) {
+    return getInt16(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link Integer} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#INT32} value by field name, {@link IllegalStateException} is thrown
+   * if schema doesn't match.
    */
-  public Integer getInteger(String fieldName) {
-    return getValue(fieldName);
+  public int getInt32(String fieldName) {
+    return getInt32(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link Float} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#INT64} value by field name, {@link IllegalStateException} is thrown
+   * if schema doesn't match.
    */
-  public Float getFloat(String fieldName) {
-    return getValue(fieldName);
+  public long getInt64(String fieldName) {
+    return getInt64(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link Double} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#DECIMAL} value by field name, {@link IllegalStateException} is
thrown
+   * if schema doesn't match.
    */
-  public Double getDouble(String fieldName) {
-    return getValue(fieldName);
+  public BigDecimal getDecimal(String fieldName) {
+    return getDecimal(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link Long} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#FLOAT} value by field name, {@link IllegalStateException} is thrown
+   * if schema doesn't match.
    */
-  public Long getLong(String fieldName) {
-    return getValue(fieldName);
+  public float getFloat(String fieldName) {
+    return getFloat(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link String} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#DOUBLE} value by field name, {@link IllegalStateException} is
thrown
+   * if schema doesn't match.
+   */
+  public double getDouble(String fieldName) {
+    return getDouble(getSchema().indexOf(fieldName));
+  }
+
+  /**
+   * Get a {@link TypeName#STRING} value by field name, {@link IllegalStateException} is
thrown
+   * if schema doesn't match.
    */
   public String getString(String fieldName) {
-    return getValue(fieldName);
+    return getString(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link Date} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#DATETIME} value by field name, {@link IllegalStateException} is
thrown
+   * if schema doesn't match.
    */
-  public Date getDate(String fieldName) {
-    return getValue(fieldName);
+  public ReadableDateTime getDateTime(String fieldName) {
+    return getDateTime(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link GregorianCalendar} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#BOOLEAN} value by field name, {@link IllegalStateException} is
thrown
+   * if schema doesn't match.
    */
-  public GregorianCalendar getGregorianCalendar(String fieldName) {
-    return getValue(fieldName);
+  public boolean getBoolean(String fieldName) {
+    return getBoolean(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link BigDecimal} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get an array value by field name, {@link IllegalStateException} is thrown
+   * if schema doesn't match.
    */
-  public BigDecimal getBigDecimal(String fieldName) {
-    return getValue(fieldName);
+  public <T> List<T> getArray(String fieldName) {
+    return getArray(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link Boolean} value by field name, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#ROW} value by field name, {@link IllegalStateException} is thrown
+   * if schema doesn't match.
    */
-  public Boolean getBoolean(String fieldName) {
-    return getValue(fieldName);
+  public Row getRow(String fieldName) {
+    return getRow(getSchema().indexOf(fieldName));
   }
 
   /**
-   * Get a {@link Byte} value by field index, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#BYTE} value by field index, {@link ClassCastException} is thrown
+   * if schema doesn't match.
    */
   public Byte getByte(int idx) {
+    checkState(
+        TypeName.BYTE.equals(getSchema().getField(idx).getTypeDescriptor().getType()));
     return getValue(idx);
   }
 
   /**
-   * Get a {@link Short} value by field index, {@link ClassCastException} is thrown
-   * if type doesn't match.
+   * Get a {@link TypeName#INT16} value by field index, {@link ClassCastException} is thrown
+   * if schema doesn't match.
    */
-  public Short getShort(int idx) {
+  public Short getInt16(int idx) {
+    checkState(
+        TypeName.INT16.equals(getSchema().getField(idx).getTypeDescriptor().getType()));
 
 Review comment:
   I think that ClassCastExecptions better describe the issue here than IllegalStateExceptions

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 86864)

> Support schema in PCollections
> ------------------------------
>
>                 Key: BEAM-3437
>                 URL: https://issues.apache.org/jira/browse/BEAM-3437
>             Project: Beam
>          Issue Type: Wish
>          Components: beam-model
>            Reporter: Jean-Baptiste Onofré
>            Assignee: Jean-Baptiste Onofré
>            Priority: Major
>          Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> As discussed with some people in the team, it would be great to add schema support in
{{PCollections}}. It will allow us:
> 1. To expect some data type in {{PTransforms}}
> 2. Improve some runners with additional features (I'm thinking about Spark runner with
data frames for instance).
> A technical draft document has been created: 
> https://docs.google.com/document/d/1tnG2DPHZYbsomvihIpXruUmQ12pHGK0QIvXS1FOTgRc/edit?disco=AAAABhykQIs&ts=5a203b46&usp=comment_email_document
> I also started a PoC on a branch, I will update this Jira with a "discussion" PR.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message