beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-4707) Change fields and table names in nexmark perfkit tables
Date Thu, 05 Jul 2018 15:16:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-4707?focusedWorklogId=119399&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-119399
]

ASF GitHub Bot logged work on BEAM-4707:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 05/Jul/18 15:15
            Start Date: 05/Jul/18 15:15
    Worklog Time Spent: 10m 
      Work Description: echauchot commented on a change in pull request #5889: [BEAM-4707]
insert timestamp to bigQuery as seconds as expected by BigQuery
URL: https://github.com/apache/beam/pull/5889#discussion_r200390879
 
 

 ##########
 File path: sdks/java/nexmark/src/test/java/org/apache/beam/sdk/nexmark/PerfsToBigQueryTest.java
 ##########
 @@ -92,7 +91,11 @@ public void testSavePerfsToBigQuery() throws IOException, InterruptedException
{
     HashMap<NexmarkConfiguration, NexmarkPerf> perfs = new HashMap<>(2);
     perfs.put(nexmarkConfiguration1, nexmarkPerf1);
     perfs.put(nexmarkConfiguration2, nexmarkPerf2);
-    Instant start = Instant.now();
+    // when read using TableRowJsonCoder the timstamp field is boxed into an Integer, cast
it to int
+    // to have same types between expected and actual rows. To avoid averflow on int capacity,
+    // set the instant to a fixed date, Beam incubation date :)
+    java.time.Instant start = java.time.Instant.ofEpochSecond(1454284800L);
 
 Review comment:
   1. the overflow was just if we used Instant.now() which will be above int capacity in 2038.
Now that the date is fixed, no more overflow problem.
   
   
    +1 to your proposed changes, but as I mentioned I will try using only joda time to avoid
mixing joda.time and java.time
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 119399)
    Time Spent: 3h 50m  (was: 3h 40m)

> Change fields and table names in nexmark perfkit tables
> -------------------------------------------------------
>
>                 Key: BEAM-4707
>                 URL: https://issues.apache.org/jira/browse/BEAM-4707
>             Project: Beam
>          Issue Type: Improvement
>          Components: testing
>            Reporter: Etienne Chauchot
>            Assignee: Etienne Chauchot
>            Priority: Major
>             Fix For: 2.6.0
>
>          Time Spent: 3h 50m
>  Remaining Estimate: 0h
>
> Nexmark BQ tables for perfkit lack timestamp field. Also the table name contains a boolean
than shows the mode of execution. It would be better to have batch or streaming label in place
of the boolean.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message