Apologies, I checked colab and it's 0.15.0 there, I confirm that
In [8]: pa.table([data2], schema=sss1)
does not work due to the missing cast implementation. It would be
great if someone would contribute an implementation for ARROW6990
On Fri, Oct 25, 2019 at 9:42 AM Wes McKinney <wesmckinn@gmail.com> wrote:
>
> These code examples both work for me in 0.15.0 and master. What
> version of the project are you using?
>
> In [6]: pa.array(data1).type
> Out[6]: Decimal128Type(decimal(14, 2))
>
> In [7]: pa.table([data1], schema=sss1)
> Out[7]:
> pyarrow.Table
> decimal_col: decimal(22, 2)
>
> On Fri, Oct 25, 2019 at 8:46 AM Дима Фомин <datagy@yandex.ru> wrote:
> >
> > Hello, Micah!
> >
> > Gotcha! Seems the problem in in pyarrow.array() handling of decimal values.
> >
> > data1 = [decimal.Decimal('234234234234.43')]
> >
> > #pa.array somehow broke precision and scale?
> > data2 = pa.array(data1)
> >
> > sss1 = pa.schema({'decimal_col': pa.decimal128(22,2)})
> >
> > pa.Table.from_arrays([data1], schema=sss1)
> >
> > is OK
> >
> > bur this one
> > pa.Table.from_arrays([data2], schema=sss1)
> >
> > raise ArrowNotImplementedError: No cast implemented from decimal(14, 2) to decimal(22,
2)
> >
> > see it here
> > https://colab.research.google.com/drive/1ejOyFBVs8GPFTosuRMDyTwPvt4145rgV
> >
> > Should I post in to JIRA as well? Or this is the same reason posted above?
> >
> >
> > 25.10.2019, 08:00, "Micah Kornfield" <emkornfield@gmail.com>:
> >
> > Hi Dima,
> > Thanks for finding this bug would you mind filing a JIRA (https://issues.apache.org/jira).
> >
> > Note that this code works fine if you put quotes around the decimal value (e.g.
Decimal('123.234')), but as you state it crashes python because the Decimal created without
quotes requires more then 38 decimal places to represent.
> >
> > Thanks,
> > Micah
> >
> >
> >
> > On Thu, Oct 24, 2019 at 7:53 AM Дима Фомин <datagy@yandex.ru> wrote:
> >
> > Digging this I just found that this code breaks my Python Kernel :(
> >
> > import sqlalchemy as sq
> > import pyarrow as pa
> >
> > aaa, bbb = sq.sql.sqltypes.decimal.Decimal(123.234), sq.sql.sqltypes.decimal.Decimal(123.234)
> > ddd = [aaa, bbb]
> > pa.array(ddd)
> >
> > So seems pyarrow.array is limited with types of array elements, doesnt'it?
> >
> > WBR,
> > Dima Fomin
> >
> > 24.10.2019, 15:47, "Dima Fomin" <datagy@yandex.ru>:
> >
> > Hello!
> > First of all thank you very much for your efforts on Apache Arrow! This looks very
useful and prominent tech to be able use it in selfBI area.
> >
> > I faced such issues  get Subj Error
> >
> > ArrowNotImplementedError: No cast implemented from decimal(2,2) to decimal(12,2)
> >
> > when trying to pyarrow.Table.from_batches array of values defined in schema as pyarrow.decimal128(12,2).
> >
> > Which values are in fact column of sqlAlchemy.sql.sqltypes.DECIMAL()="DECIMAL(precision=12,scale=2)"
values, which I map in schema like this pyarrow.decimal128(12,2).
> >
> > What could be a reason of issue?
> > Could it be that somehow declared decimal type of data is reduced to minimum value
which met in data? :/
> >
> > With best regards,
> > Dima Fomin
