carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhichao Zhang (JIRA)" <>
Subject [jira] [Created] (CARBONDATA-1351) When 'SORT_SCOPE'='GLOBAL_SORT' and 'enable.unsafe.columnpage'='true', 'ThreadLocalTaskInfo.getCarbonTaskInfo()' return null
Date Tue, 01 Aug 2017 08:41:01 GMT
Zhichao  Zhang created CARBONDATA-1351:

             Summary: When 'SORT_SCOPE'='GLOBAL_SORT' and 'enable.unsafe.columnpage'='true',
'ThreadLocalTaskInfo.getCarbonTaskInfo()' return null
                 Key: CARBONDATA-1351
             Project: CarbonData
          Issue Type: Bug
          Components: data-load, spark-integration
    Affects Versions: 1.2.0
            Reporter: Zhichao  Zhang
            Assignee: Zhichao  Zhang
            Priority: Minor
             Fix For: 1.2.0

When 'SORT_SCOPE'='GLOBAL_SORT' and 'enable.unsafe.columnpage'='true', it uses native RDD
of Spark to load data, the method of 'ThreadLocalTaskInfo.setCarbonTaskInfo(carbonTaskInfo)'
in ‘CarbonRDD.compute’ does not be called, so 'ThreadLocalTaskInfo.getCarbonTaskInfo()'
will return null in some unsafe related classes, such as: UnsafeFixLengthColumnPage, UnsafeVarLengthColumnPage,
UnsafeMemoryDMStore and so on.

Solution: Set the CarbonTaskInfo in the method of 'ThreadLocalTaskInfo.getCarbonTaskInfo()'
when 'threadLocal.get()' is null.

This message was sent by Atlassian JIRA

View raw message