hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Satyajit varma (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HIVE-13203) STRUCT type not allowing more than 266 fields. Table get created with more fields but throws error while querying against the table. SHOW CREATE TABLE skips having some fields in STRUCT.
Date Fri, 04 Mar 2016 19:24:40 GMT

     [ https://issues.apache.org/jira/browse/HIVE-13203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Satyajit varma reassigned HIVE-13203:
-------------------------------------

    Assignee: Satyajit varma  (was: VISHNU S NAIR)

> STRUCT type not allowing more than 266 fields. Table get created with more fields but
throws error while querying against the table. SHOW CREATE TABLE skips having some fields
in STRUCT.
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-13203
>                 URL: https://issues.apache.org/jira/browse/HIVE-13203
>             Project: Hive
>          Issue Type: Bug
>          Components: Database/Schema
>    Affects Versions: 1.2.0
>            Reporter: Satyajit varma
>            Assignee: Satyajit varma
>
> I am trying to create a table with multiple STRUCT column types, one of which having
more than 300 fields.
> I get the table created with no problem, but when i load i face few errors in loading
up the data.when i check SHOW CREATE TABLE TABLENAME, i get fields only till 266th column
and rest of them do not appear in the table DDL.(check the issue at StateOrProvinc, search
for StateOrProvinc in the second DDL).
> Would like to know, if there is any restriction on the number of fields to be using in
STRUCT datatype.
> EXAMPLE:
> BELOW IS THE DDL i run in hive and the second below DLL is the TABLE DDL i get from SHOW
CREATE TABLE.
> CREATE TABLE property_feature.property_feature_new(
> `_id` STRUCT<`mls_sysid`:String,`type`:String>,
> all_attrs STRUCT<`1`: String ,`10`: String ,`100`: String ,`105`: String ,`113`: String
,`118`: String ,`126`: String ,`1299`: String ,`1301`: String ,`1305`: String ,`1307`: String
,`1319`: String ,`1325`: String ,`1327`: String ,`1329`: String ,`1339`: String ,`1341`: String
,`1345`: String ,`1351`: String ,`1353`: String ,`1359`: String ,`1363`: String ,`1365`: String
,`1367`: String ,`1373`: String ,`1377`: String ,`1397`: String ,`1401`: String ,`1403`: String
,`1405`: String ,`1409`: String ,`1411`: String ,`1413`: String ,`1417`: String ,`1419`: String
,`1421`: String ,`1423`: String ,`1431`: String ,`1437`: String ,`1439`: String ,`144`: String
,`1441`: String ,`145`: String ,`1451`: String ,`1453`: String ,`146`: String ,`1467`: String
,`1469`: String ,`1483`: String ,`1485`: String ,`1491`: String ,`1493`: String ,`15`: String
,`1505`: String ,`1519`: String ,`1521`: String ,`1523`: String ,`155`: String ,`156`: String
,`158`: String ,`1586`: String ,`1678`: String ,`169`: String ,`17`: String ,`177`: String
,`1783`: String ,`1785`: String ,`1787`: String ,`1789`: String ,`1793`: String ,`1795`: String
,`1797`: String ,`1799`: String ,`1801`: String ,`1803`: String ,`1805`: String ,`1807`: String
,`1809`: String ,`1811`: String ,`1815`: String ,`1818`: String ,`1826`: String ,`1829`: String
,`1874`: String ,`1876`: String ,`1878`: String ,`1880`: String ,`1882`: String ,`207`: String
,`215`: String ,`226`: String ,`2293`: String ,`232`: String ,`239`: String ,`241`: String
,`245`: String ,`2456`: String ,`248`: String ,`2482`: String ,`2490`: String ,`2491`: String
,`2492`: String ,`2493`: String ,`2495`: String ,`25`: String ,`2502`: String ,`2503`: String
,`2517`: String ,`2522`: String ,`255`: String ,`256`: String ,`2573`: String ,`2574`: String
,`260`: String ,`263`: String ,`265`: String ,`276`: String ,`278`: String ,`286`: String
,`293`: String ,`296`: String ,`299`: String ,`3`: String ,`30`: String ,`307`: String ,`312`:
String ,`320`: String ,`328`: String ,`340`: String ,`348`: String ,`354`: String ,`363`:
String ,`364`: String ,`369`: String ,`375`: String ,`376`: String ,`383`: String ,`401`:
String ,`405`: String ,`409`: String ,`412`: String ,`42`: String ,`423`: String ,`444`: String
,`445`: String ,`446`: String ,`447`: String ,`450`: String ,`451`: String ,`452`: String
,`453`: String ,`455`: String ,`458`: String ,`461`: String ,`463`: String ,`464`: String
,`51`: String ,`543472980`: String ,`57`: String ,`75`: String ,`90`: String ,`92`: String
,`AVMYN`: String ,`AccessibilityAmenities`: String ,`AddressPublic`: String ,`Amenities`:
String ,`AmountofWaterFront`: String ,`AppliancesIncluded`: String ,`AppraiserInitials`: String
,`AppraiserName`: String ,`Basement`: String ,`BathsFull`: String ,`BathsHalf`: String ,`BathsTotal`:
String ,`BedsTotal`: String ,`Block`: String ,`Book`: String ,`BuildingNumber`: String ,`City`:
String ,`CloseDate`: String ,`ClosePrice`: String ,
> `CoListAgentDirectWorkPhone`: String ,`CoListAgentEmail`: String ,`CoListAgentFullName`:
String ,`CoListAgentMLSID`: String ,`CoListAgent_MUI`: String ,`CoListOfficeMLSID`: String
,`CoListOfficeName`: String ,`CoListOfficePhone`: String ,`CoListOffice_MUI`: String ,`Color`:
String ,`CommentsYN`: String ,`Construction`: String ,`Cooling`: String ,`CountyOrParish`:
String ,`DOM`: String ,`Deed`: String ,`Directions`: String ,`Driveway`: String ,`Electric`:
String ,`Equipment`: String ,`Exterior`: String ,`Fireplace`: String ,`FloorNumber`: String
,`Floors`: String ,`FoundationMaterials`: String ,`FullTaxAmount`: String ,`Gas`: String ,`HeatFuel`:
String ,`HeatSystem`: String ,`Internet`: String ,`ListAgentDirectWorkPhone`: String ,`ListAgentEmail`:
String ,`ListAgentFullName`: String ,`ListAgentMLSID`: String ,`ListAgent_MUI`: String ,`ListOfficeMLSID`:
String ,`ListOfficeName`: String ,`ListOfficePhone`: String ,`ListOffice_MUI`: String ,`ListPrice`:
String ,`ListingContractDate`: String ,`Location`: String ,`Lot`: String ,`LotSizeArea`: String
,`MLSNumber`: String ,`Make`: String ,`MapNumber`: String ,`MatrixModifiedDT`: String ,`MobileHomeIncludes`:
String ,`Model`: String ,`MonthlyAssociationFee`: String ,`NeighborhoodAssoc`: String ,`NumberOfRooms`:
String ,`OneBedroomUnits`: String ,`OriginalListPrice`: String ,`Page`: String ,`Parking`:
String ,`PhotoCount`: String ,`PhotoModificationTimestamp`: String ,`PostalCode`: String ,`PropertySubType`:
String ,`PropertyType`: String ,`PublicAddress`: String ,`RecreationalWater`: String ,`Remarks`:
String ,`Restrictions`: String ,`RoadFrontage`: String ,`Roads`: String ,`Roof`: String ,`SchoolDistrict`:
String ,`Seasonal`: String ,`SellingAgentDirectWorkPhone`: String ,`SellingAgentEmail`: String
,`SellingAgentFullName`: String ,`SellingAgentMLSID`: String ,`SellingAgent_MUI`: String ,`SellingOfficeMLSID`:
String ,`SellingOfficeName`: String ,`SellingOfficePhone`: String ,`SellingOffice_MUI`: String
,`SerialNumber`: String ,`Site`: String ,`SoldTerms`: String ,`SourceofSquareFootage`: String
,`SqFtFinishedTotal`: String ,`StateOrProvince`: String ,`Status`: String ,`Style`: String
,`SupplementPublicCount`: String ,`Surveyed`: String ,`TaxYear`: String ,`ThreeBedroomUnits`:
String ,`Transportation`: String ,`TwoBedroomUnits`: String ,`UnitType`: String ,`UnitsInBuilding`:
String ,`VehicleStorage`: String ,`View`: String ,`VirtualTour`: String ,`WasteWaterDisposal`:
String ,`Water`: String ,`WaterBodyName`: String ,`WaterBodyType`: String ,`WaterFrontage`:
String ,`WaterFrontageOwned`: String ,`WaterFrontageShared`: String ,`WaterHeater`: String
,`YearBuilt`: String ,`Zoning`: String ,`matrix_unique_id`: String ,`removed_created_at`:
String ,`sysid`: String>
> basement` string
> ) STORED AS parquet;
> CREATE TABLE `property_feature`(
>   `_id` struct<mls_sysid:string,type:string>,
>   `all_attrs` struct<1:string,10:string,100:string,105:string,113:string,118:string,126:string,1299:string,
> 1301:string,1305:string,1307:string,1319:string,1325:string,1327:string,1329:string,
> 1339:string,1341:string,1345:string,1351:string,1353:string,1359:string,1363:string,
> 1365:string,1367:string,1373:string,1377:string,1397:string,1401:string,1403:string,
> 1405:string,1409:string,1411:string,1413:string,1417:string,1419:string,1421:string,
> 1423:string,1431:string,1437:string,1439:string,144:string,1441:string,145:string,1451:string,
> 1453:string,146:string,1467:string,1469:string,1483:string,1485:string,1491:string,1493:string,
> 15:string,1505:string,1519:string,1521:string,1523:string,155:string,156:string,158:string,
> 1586:string,1678:string,169:string,17:string,177:string,1783:string,1785:string,1787:string,
> 1789:string,1793:string,1795:string,1797:string,1799:string,1801:string,1803:string,1805:string,
> 1807:string,1809:string,1811:string,1815:string,1818:string,1826:string,1829:string,1874:string,
> 1876:string,1878:string,1880:string,1882:string,207:string,215:string,226:string,2293:string,
> 232:string,239:string,241:string,245:string,2456:string,248:string,2482:string,2490:string,
> 2491:string,2492:string,2493:string,2495:string,25:string,2502:string,2503:string,2517:string,
> 2522:string,255:string,256:string,2573:string,2574:string,260:string,263:string,265:string,
> 276:string,278:string,286:string,293:string,296:string,299:string,3:string,30:string,307:string,
> 312:string,320:string,328:string,340:string,348:string,354:string,363:string,364:string,369:string,
> 375:string,376:string,383:string,401:string,405:string,409:string,412:string,42:string,423:string,444:string,445:string,446:string,447:string,450:string,451:string,452:string,453:string,455:string,458:string,461:string,463:string,464:string,51:string,543472980:string,57:string,75:string,90:string,92:string,AVMYN:string,AccessibilityAmenities:string,
> AddressPublic:string,Amenities:string,AmountofWaterFront:string,AppliancesIncluded:string,AppraiserInitials:string,AppraiserName:string,Basement:string,BathsFull:string,BathsHalf:string,BathsTotal:string,BedsTotal:string,Block:string,Book:string,BuildingNumber:string,City:string,CloseDate:string,ClosePrice:string,CoListAgentDirectWorkPhone:string,CoListAgentEmail:string,CoListAgentFullName:string,CoListAgentMLSID:string,CoListAgent_MUI:string,CoListOfficeMLSID:string,CoListOfficeName:string,CoListOfficePhone:string,
> CoListOffice_MUI:string,Color:string,CommentsYN:string,Construction:string,Cooling:string,CountyOrParish:string,DOM:string,Deed:string,Directions:string,Driveway:string,Electric:string,Equipment:string,Exterior:string,Fireplace:string,FloorNumber:string,Floors:string,FoundationMaterials:string,FullTaxAmount:string,Gas:string,HeatFuel:string,HeatSystem:string,Internet:string,ListAgentDirectWorkPhone:string,ListAgentEmail:string,ListAgentFullName:string,ListAgentMLSID:string,ListAgent_MUI:string,ListOfficeMLSID:string,ListOfficeName:string,ListOfficePhone:string,ListOffice_MUI:string,ListPrice:string,ListingContractDate:string,Location:string,Lot:string,LotSizeArea:string,MLSNumber:string,Make:string,MapNumber:string,MatrixModifiedDT:string,MobileHomeIncludes:string,Model:string,MonthlyAssociationFee:string,NeighborhoodAssoc:string,NumberOfRooms:string,OneBedroomUnits:string,OriginalListPrice:string,Page:string,Parking:string,PhotoCount:string,PhotoModificationTimestamp:string,PostalCode:string,PropertySubType:string,PropertyType:string,PublicAddress:string,RecreationalWater:string,Remarks:string,Restrictions:string,RoadFrontage:string,Roads:string,Roof:string,SchoolDistrict:string,Seasonal:string,SellingAgentDirectWorkPhone:string,SellingAgentEmail:string,SellingAgentFullName:string,SellingAgentMLSID:string,SellingAgent_MUI:string,SellingOfficeMLSID:string,SellingOfficeName:string,SellingOfficePhone:string,SellingOffice_MUI:string,SerialNumber:string,Site:string,SoldTerms:string,
> SourceofSquareFootage:string,SqFtFinishedTotal:string,StateOrProvinc,
> `basement` string
> ) STORED AS parquet;
> Please help out in case if there is any workaround for it or any other restriction details.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message