spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hemant Bhanawat <hemant9...@gmail.com>
Subject Re: GenericMutableRow and Row mismatch on Spark 1.5?
Date Wed, 07 Oct 2015 12:50:48 GMT
Will send you the code on your email id.

On Wed, Oct 7, 2015 at 4:37 PM, Ophir Cohen <ophchu@gmail.com> wrote:

> Thanks!
> Can you check if you can provide example of the conversion?
>
>
> On Wed, Oct 7, 2015 at 2:05 PM, Hemant Bhanawat <hemant9379@gmail.com>
> wrote:
>
>> Oh, this is an internal class of our project and I had used it without
>> realizing the source.
>>
>> Anyway, the idea is to  wrap the InternalRow in a class that derives from
>> Row. When you implement the functions of the trait 'Row ', the type
>> conversions from Row types to InternalRow types has to be done for each of
>> the types. But, as I can see, the primitive types (apart from String) don't
>> need conversions. Map and Array would need some handling.
>>
>> I will check with the author of this code, I think this code can be
>> contributed to Spark.
>>
>> Hemant
>> www.snappydata.io
>> linkedin.com/company/snappydata
>>
>> On Wed, Oct 7, 2015 at 3:30 PM, Ophir Cohen <ophchu@gmail.com> wrote:
>>
>>> From which jar WrappedInternalRow comes from?
>>> It seems that I can't find it.
>>>
>>> BTW
>>> What I'm trying to do now is to create scala array from the fields and
>>> than create Row out of that array.
>>> The problem is that I get types mismatches...
>>>
>>> On Wed, Oct 7, 2015 at 8:03 AM, Hemant Bhanawat <hemant9379@gmail.com>
>>> wrote:
>>>
>>>> An approach can be to wrap your MutableRow in WrappedInternalRow which
>>>> is a child class of Row.
>>>>
>>>> Hemant
>>>> www.snappydata.io
>>>> linkedin.com/company/snappydata
>>>>
>>>>
>>>> On Tue, Oct 6, 2015 at 3:21 PM, Ophir Cohen <ophchu@gmail.com> wrote:
>>>>
>>>>> Hi Guys,
>>>>> I'm upgrading to Spark 1.5.
>>>>>
>>>>> In our previous version (Spark 1.3 but it was OK on 1.4 as well) we
>>>>> created GenericMutableRow
>>>>> (org.apache.spark.sql.catalyst.expressions.GenericMutableRow) and return
it
>>>>> as org.apache.spark.sql.Row
>>>>>
>>>>> Starting from Spark 1.5 GenericMutableRow isn't extends Row.
>>>>>
>>>>> What do you suggest to do?
>>>>> How can I convert GenericMutableRow to Row?
>>>>>
>>>>> Prompt answer will be highly appreciated!
>>>>> Thanks,
>>>>> Ophir
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message