From notifications-return-158278-archive-asf-public=cust-asf.ponee.io@asterixdb.apache.org Mon Jun 14 22:47:23 2021 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id 434DD180181 for ; Tue, 15 Jun 2021 00:47:23 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id 672023EAF7 for ; Mon, 14 Jun 2021 22:47:22 +0000 (UTC) Received: (qmail 76047 invoked by uid 500); 14 Jun 2021 22:47:22 -0000 Mailing-List: contact notifications-help@asterixdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.apache.org Delivered-To: mailing list notifications@asterixdb.apache.org Received: (qmail 76036 invoked by uid 99); 14 Jun 2021 22:47:21 -0000 Received: from spamproc1-he-fi.apache.org (HELO spamproc1-he-fi.apache.org) (95.217.134.168) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 14 Jun 2021 22:47:21 +0000 Received: from localhost (localhost [127.0.0.1]) by spamproc1-he-fi.apache.org (ASF Mail Server at spamproc1-he-fi.apache.org) with ESMTP id 5CB03C0484 for ; Mon, 14 Jun 2021 22:47:18 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamproc1-he-fi.apache.org X-Spam-Flag: NO X-Spam-Score: 2.186 X-Spam-Level: ** X-Spam-Status: No, score=2.186 tagged_above=-999 required=6.31 tests=[HTML_MESSAGE=0.2, MPART_ALT_DIFF_COUNT=1.483, SPF_NONE=0.001, URIBL_BLOCKED=0.001, URI_NOVOWEL=0.5, WEIRD_PORT=0.001] autolearn=disabled Received: from mx1-ec2-va.apache.org ([116.203.227.195]) by localhost (spamproc1-he-fi.apache.org [95.217.134.168]) (amavisd-new, port 10024) with ESMTP id VeZpPr4M6RRl for ; Mon, 14 Jun 2021 22:47:05 +0000 (UTC) Received-SPF: None (mailfrom) identity=mailfrom; client-ip=128.195.1.133; helo=adrian-monk-v3.ics.uci.edu; envelope-from=do-not-reply@asterix-gerrit.ics.uci.edu; receiver= Received: from adrian-monk-v3.ics.uci.edu (adrian-monk-v3.ics.uci.edu [128.195.1.133]) by mx1-ec2-va.apache.org (ASF Mail Server at mx1-ec2-va.apache.org) with ESMTPS id EFD21BC48A for ; Mon, 14 Jun 2021 22:47:03 +0000 (UTC) Received: from 4f688a390847 (vitalstatistix.ics.uci.edu [128.195.52.38]) by adrian-monk-v3.ics.uci.edu (Postfix) with ESMTP id B3608C02A0C9; Mon, 14 Jun 2021 15:46:55 -0700 (PDT) X-Gerrit-PatchSet: 1 Date: Mon, 14 Jun 2021 22:46:55 +0000 From: AsterixDB Code Review To: Till Westmann Message-ID: Auto-Submitted: auto-generated X-Gerrit-MessageType: newchange Subject: Change in asterixdb[master]: [ASTERIXDB-2791][IDX] Set null flags when writing an index entry X-Gerrit-Change-Id: I5fc9c40af68c7b6aaa54ecb440c8fe1e90046231 X-Gerrit-Change-Number: 11944 X-Gerrit-Project: asterixdb X-Gerrit-ChangeURL: X-Gerrit-Commit: 85790396cc65aeafd28d5c492435c3b2c800dc23 References: Reply-To: ali.al.solaiman@gmail.com, dmitry.lychagin@couchbase.com, notifications@asterixdb.apache.org, lwhaywhu@gmail.com, tillw@apache.org MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/3.0.12 Content-Type: multipart/alternative; boundary="nYDT6NJUqDk="; charset=UTF-8 X-ICS-MailScanner-Information: Please send mail to helpdesk@ics.uci.edu or more information X-ICS-MailScanner-ID: B3608C02A0C9.A71E5 X-ICS-MailScanner: Found to be clean X-ICS-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=0.988, required 5, ALL_TRUSTED -1.00, FSL_HELO_NON_FQDN_1 0.00, HELO_NO_DOMAIN 0.00, HTML_MESSAGE 0.00, MPART_ALT_DIFF_COUNT 1.48, URIBL_BLOCKED 0.00, URI_NOVOWEL 0.50, WEIRD_PORT 0.00) X-ICS-MailScanner-From: do-not-reply@asterix-gerrit.ics.uci.edu --nYDT6NJUqDk= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable From Ali Alsuliman : Ali Alsuliman has up= loaded this change for review=2E ( https://asterix-gerrit=2Eics=2Euci=2Eedu= /c/asterixdb/+/11944 ) Change subject: [ASTERIXDB-2791][IDX] Set null fla= gs when writing an index entry =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E= =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E= =2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E=2E= =2E=2E=2E=2E=2E=2E [ASTERIXDB-2791][IDX] Set null flags when writing an in= dex entry - user model changes: no - storage format changes: no - interfac= e changes: yes Details: When writing an index entry, set the corresponding= null flags for keys that are considered null=2E When reading an index entr= y, examine the null flags to read the index entry properly=2E - add INull= Introspector interface that can be used by Hyracks to know if a key is n= ull or not=2E Change-Id: I5fc9c40af68c7b6aaa54ecb440c8fe1e90046231 --- M a= sterixdb/asterix-app/src/main/java/org/apache/asterix/app/io/PersistedResou= rceRegistry=2Ejava M asterixdb/asterix-metadata/src/main/java/org/apache/as= terix/metadata/MetadataNode=2Ejava M asterixdb/asterix-metadata/src/main/ja= va/org/apache/asterix/metadata/bootstrap/MetadataBootstrap=2Ejava M asterix= db/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/Arra= yBTreeResourceFactoryProvider=2Ejava M asterixdb/asterix-metadata/src/main/= java/org/apache/asterix/metadata/declared/BTreeResourceFactoryProvider=2Eja= va M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/v= alueextractors/TupleCopyValueExtractor=2Ejava A asterixdb/asterix-om/src/ma= in/java/org/apache/asterix/formats/nontagged/NullIntrospector=2Ejava M aste= rixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/TypeTra= itProvider=2Ejava M hyracks-fullstack/hyracks/hyracks-examples/hyracks-inte= gration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryI= ndexScanOperatorTest=2Ejava M hyracks-fullstack/hyracks/hyracks-examples/hy= racks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTr= eePrimaryIndexSearchOperatorTest=2Ejava M hyracks-fullstack/hyracks/hyracks= -examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/= am/btree/BTreePrimaryIndexStatsOperatorTest=2Ejava M hyracks-fullstack/hyra= cks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyr= acks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest=2Ejava M hyracks-= fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/= org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest=2Ej= ava M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/= src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOp= eratorTest=2Ejava M hyracks-fullstack/hyracks/hyracks-examples/hyracks-inte= gration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeO= peratorTestHelper=2Ejava M hyracks-fullstack/hyracks/hyracks-examples/hyrac= ks-integration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/Abstra= ctRTreeOperatorTest=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-b= tree/src/main/java/org/apache/hyracks/storage/am/btree/compressors/FieldPre= fixCompressor=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-btree/s= rc/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeResource=2Ej= ava M hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/= apache/hyracks/storage/am/btree/dataflow/BTreeResourceFactory=2Ejava M hyra= cks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyr= acks/storage/am/btree/frames/BTreeFieldPrefixNSMLeafFrame=2Ejava M hyracks-= fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks= /storage/am/btree/impls/FieldPrefixPrefixTupleReference=2Ejava M hyracks-fu= llstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/s= torage/am/btree/tuples/BTreeTypeAwareTupleReference=2Ejava M hyracks-fullst= ack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/stora= ge/am/btree/tuples/BTreeTypeAwareTupleWriter=2Ejava M hyracks-fullstack/hyr= acks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/b= tree/tuples/BTreeTypeAwareTupleWriterFactory=2Ejava M hyracks-fullstack/hyr= acks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/b= tree/util/BTreeUtils=2Ejava A hyracks-fullstack/hyracks/hyracks-storage-am-= common/src/main/java/org/apache/hyracks/storage/am/common/api/INullIntrospe= ctor=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/= java/org/apache/hyracks/storage/am/common/tuples/SimpleTupleWriter=2Ejava M= hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apac= he/hyracks/storage/am/common/tuples/TypeAwareTupleReference=2Ejava M hyrack= s-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyra= cks/storage/am/common/tuples/TypeAwareTupleWriter=2Ejava M hyracks-fullstac= k/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storag= e/am/common/tuples/TypeAwareTupleWriterFactory=2Ejava M hyracks-fullstack/h= yracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storag= e/am/lsm/btree/dataflow/ExternalBTreeLocalResource=2Ejava M hyracks-fullsta= ck/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/st= orage/am/lsm/btree/dataflow/ExternalBTreeLocalResourceFactory=2Ejava M hyra= cks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache= /hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResource= =2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/j= ava/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddy= LocalResourceFactory=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-= lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LS= MBTreeLocalResource=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-l= sm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSM= BTreeLocalResourceFactory=2Ejava M hyracks-fullstack/hyracks/hyracks-storag= e-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples= /LSMBTreeCopyTupleWriter=2Ejava M hyracks-fullstack/hyracks/hyracks-storage= -am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/= LSMBTreeCopyTupleWriterFactory=2Ejava M hyracks-fullstack/hyracks/hyracks-s= torage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/t= uples/LSMBTreeTupleReference=2Ejava M hyracks-fullstack/hyracks/hyracks-sto= rage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tup= les/LSMBTreeTupleWriter=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-= am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/L= SMBTreeTupleWriterFactory=2Ejava M hyracks-fullstack/hyracks/hyracks-storag= e-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/utils/= LSMBTreeUtil=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-comm= on/src/main/java/org/apache/hyracks/storage/am/lsm/common/dataflow/LsmResou= rce=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/ma= in/java/org/apache/hyracks/storage/am/lsm/common/dataflow/LsmResourceFactor= y=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/s= rc/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/dataflow/LSMIn= vertedIndexLocalResource=2Ejava M hyracks-fullstack/hyracks/hyracks-storage= -am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/inver= tedindex/dataflow/LSMInvertedIndexLocalResourceFactory=2Ejava M hyracks-ful= lstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apach= e/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndex=2Ejava M hyra= cks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/or= g/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedInde= x=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/s= rc/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskI= nvertedIndex=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inve= rtedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ond= isk/variablesize/VariableSizeElementInvertedListBuilder=2Ejava M hyracks-fu= llstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apac= he/hyracks/storage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeEle= mentOnDiskInvertedListCursor=2Ejava M hyracks-fullstack/hyracks/hyracks-sto= rage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/i= nvertedindex/ondisk/variablesize/VariableSizeInvertedListSearchResultFrameT= upleAccessor=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inve= rtedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ond= isk/variablesize/VariableSizeInvertedListTupleReference=2Ejava M hyracks-fu= llstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apac= he/hyracks/storage/am/lsm/invertedindex/search/InvertedIndexSearchResult=2E= java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/m= ain/java/org/apache/hyracks/storage/am/lsm/invertedindex/util/InvertedIndex= Utils=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/m= ain/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeLocalReso= urce=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/ma= in/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeLocalResou= rceFactory=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/= src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeWith= AntiMatterLocalResource=2Ejava M hyracks-fullstack/hyracks/hyracks-storage-= am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow= /LSMRTreeWithAntiMatterLocalResourceFactory=2Ejava M hyracks-fullstack/hyra= cks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/a= m/lsm/rtree/tuples/LSMRTreeTupleReferenceForPointMBR=2Ejava M hyracks-fulls= tack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/= storage/am/lsm/rtree/tuples/LSMRTreeTupleWriterForPointMBR=2Ejava M hyracks= -fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hy= racks/storage/am/lsm/rtree/utils/LSMRTreeUtils=2Ejava M hyracks-fullstack/h= yracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/storage/am= /rtree/tuples/RTreeTypeAwareTupleReference=2Ejava M hyracks-fullstack/hyrac= ks/hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/storage/am/rtr= ee/tuples/RTreeTypeAwareTupleWriter=2Ejava M hyracks-fullstack/hyracks/hyra= cks-storage-am-rtree/src/main/java/org/apache/hyracks/storage/am/rtree/tupl= es/RTreeTypeAwareTupleWriterFactory=2Ejava M hyracks-fullstack/hyracks/hyra= cks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/st= orage/am/btree/BTreeExamplesTest=2Ejava M hyracks-fullstack/hyracks/hyracks= -tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/stora= ge/am/btree/BTreeModificationOperationCallbackTest=2Ejava M hyracks-fullsta= ck/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/ap= ache/hyracks/storage/am/btree/BTreeSearchCursorTest=2Ejava M hyracks-fullst= ack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/a= pache/hyracks/storage/am/btree/BTreeSearchOperationCallbackTest=2Ejava M hy= racks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/tes= t/java/org/apache/hyracks/storage/am/btree/BTreeStatsTest=2Ejava M hyracks-= fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java= /org/apache/hyracks/storage/am/btree/BTreeUpdateSearchTest=2Ejava M hyracks= -fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/jav= a/org/apache/hyracks/storage/am/btree/FieldPrefixNSMTest=2Ejava M hyracks-f= ullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/= org/apache/hyracks/storage/am/btree/multithread/BTreeMultiThreadTest=2Ejava= M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/sr= c/test/java/org/apache/hyracks/storage/am/btree/util/BTreeTestContext=2Ejav= a M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-te= st/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeExamplesTe= st=2Ejava M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-= btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeMo= dificationOperationCallbackTest=2Ejava M hyracks-fullstack/hyracks/hyracks-= tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/st= orage/am/lsm/btree/LSMBTreeSearchOperationCallbackTest=2Ejava M hyracks-ful= lstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/jav= a/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeUpdateInPlaceTest=2Ejava = M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test= /src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeLoc= alResource=2Ejava M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage= -am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/im= pl/TestLsmBtreeLocalResourceFactory=2Ejava M hyracks-fullstack/hyracks/hyra= cks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyrack= s/storage/am/lsm/btree/impl/TestLsmBtreeUtil=2Ejava M hyracks-fullstack/hyr= acks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apac= he/hyracks/storage/am/lsm/btree/multithread/LSMBTreeMultiThreadTest=2Ejava = M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test= /src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/BTreeRunner=2Ej= ava M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-= test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/ConcurrentS= kipListRunner=2Ejava M hyracks-fullstack/hyracks/hyracks-tests/hyracks-stor= age-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree= /perf/InMemoryBTreeRunner=2Ejava M hyracks-fullstack/hyracks/hyracks-tests/= hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/= am/lsm/btree/perf/InMemorySortRunner=2Ejava M hyracks-fullstack/hyracks/hyr= acks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyrac= ks/storage/am/lsm/btree/perf/LSMTreeRunner=2Ejava M hyracks-fullstack/hyrac= ks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache= /hyracks/storage/am/lsm/btree/tuples/LSMBTreeTuplesTest=2Ejava M hyracks-fu= llstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/ja= va/org/apache/hyracks/storage/am/lsm/btree/util/LSMBTreeTestContext=2Ejava = M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-common-tes= t/src/test/java/org/apache/hyracks/storage/am/lsm/common/test/LSMComponentF= ilterReferenceTest=2Ejava 90 files changed, 506 insertions(+), 194 deletion= s(-) git pull ssh://asterix-gerrit=2Eics=2Euci=2Eedu:29418/asterixdb r= efs/changes/44/11944/1 diff --git a/asterixdb/asterix-app/src/main/java/or= g/apache/asterix/app/io/PersistedResourceRegistry=2Ejava b/asterixdb/asteri= x-app/src/main/java/org/apache/asterix/app/io/PersistedResourceRegistry=2Ej= ava index 4cc6e43=2E=2E35ee6dd 100644 --- a/asterixdb/asterix-app/src/main/= java/org/apache/asterix/app/io/PersistedResourceRegistry=2Ejava +++ b/aster= ixdb/asterix-app/src/main/java/org/apache/asterix/app/io/PersistedResourceR= egistry=2Ejava @@ -53,6 +53,7 @@ import org=2Eapache=2Easterix=2Edataflow= =2Edata=2Enontagged=2Ecomparators=2EListItemBinaryComparatorFactory; impor= t org=2Eapache=2Easterix=2Edataflow=2Edata=2Enontagged=2Evalueproviders=2EP= rimitiveValueProviderFactory; import org=2Eapache=2Easterix=2Eformats=2Eno= ntagged=2EAnyBinaryComparatorFactory; +import org=2Eapache=2Easterix=2Eform= ats=2Enontagged=2ENullIntrospector; import org=2Eapache=2Easterix=2Eformat= s=2Enontagged=2EOrderedBinaryComparatorFactory; import org=2Eapache=2Easte= rix=2Eformats=2Enontagged=2EOrderedLinearizeComparatorFactory; import org= =2Eapache=2Easterix=2Emetadata=2Eutils=2ESecondaryCorrelatedTreeIndexOperat= ionsHelper; @@ -175,6 +176,9 @@ registeredClasses=2Eput("FixedLeng= thTypeTrait", FixedLengthTypeTrait=2Eclass); registeredClasses=2Ep= ut("VarLengthTypeTrait", VarLengthTypeTrait=2Eclass); + // INullIn= trospector + registeredClasses=2Eput("NullIntrospector", NullIntrosp= ector=2Eclass); + // ILSMOperationTrackerFactory register= edClasses=2Eput("PrimaryIndexOperationTrackerFactory", PrimaryIndexOperatio= nTrackerFactory=2Eclass); registeredClasses=2Eput("SecondaryIndexO= perationTrackerFactory", SecondaryIndexOperationTrackerFactory=2Eclass); di= ff --git a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/meta= data/MetadataNode=2Ejava b/asterixdb/asterix-metadata/src/main/java/org/apa= che/asterix/metadata/MetadataNode=2Ejava index f8ba574=2E=2E9975f4c 100644 = --- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/= MetadataNode=2Ejava +++ b/asterixdb/asterix-metadata/src/main/java/org/apac= he/asterix/metadata/MetadataNode=2Ejava @@ -50,7 +50,9 @@ import org=2Eapa= che=2Easterix=2Ecommon=2Etransactions=2ETxnId; import org=2Eapache=2Easter= ix=2Ecommon=2Eutils=2EStoragePathUtil; import org=2Eapache=2Easterix=2Eext= ernal=2Eindexing=2EExternalFile; +import org=2Eapache=2Easterix=2Eformats= =2Enontagged=2ENullIntrospector; import org=2Eapache=2Easterix=2Eformats= =2Enontagged=2ESerializerDeserializerProvider; +import org=2Eapache=2Easter= ix=2Eformats=2Enontagged=2ETypeTraitProvider; import org=2Eapache=2Easteri= x=2Emetadata=2Eapi=2EExtensionMetadataDataset; import org=2Eapache=2Easter= ix=2Emetadata=2Eapi=2EExtensionMetadataDatasetId; import org=2Eapache=2Eas= terix=2Emetadata=2Eapi=2EIExtensionMetadataEntity; @@ -1539,7 +1541,8 @@ = private ITupleReference getTupleToBeDeleted(TxnId txnId, IMetadataInde= x metadataIndex, ITupleReference searchKey) throws AlgebricksE= xception, HyracksDataException { - IValueExtractor = valueExtractor =3D new TupleCopyValueExtractor(metadataIndex=2EgetTypeTrait= s()); + IValueExtractor valueExtractor =3D new Tupl= eCopyValueExtractor(metadataIndex=2EgetTypeTraits(), + TypeT= raitProvider=2EINSTANCE=2EgetTypeTrait(BuiltinType=2EANULL), NullIntrospect= or=2EINSTANCE); List results =3D new ArrayList<>(= ); searchIndex(txnId, metadataIndex, searchKey, valueExtractor, re= sults); if (results=2EisEmpty()) { diff --git a/asterixdb/asterix-= metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootst= rap=2Ejava b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/me= tadata/bootstrap/MetadataBootstrap=2Ejava index 047b823=2E=2E125db21 100644= --- a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata= /bootstrap/MetadataBootstrap=2Ejava +++ b/asterixdb/asterix-metadata/src/ma= in/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap=2Ejava @@ -= 45,6 +45,8 @@ import org=2Eapache=2Easterix=2Eexternal=2Eapi=2EITypedAdapt= erFactory; import org=2Eapache=2Easterix=2Eexternal=2Edataset=2Eadapter=2E= AdapterIdentifier; import org=2Eapache=2Easterix=2Eexternal=2Eindexing=2EE= xternalFile; +import org=2Eapache=2Easterix=2Eformats=2Enontagged=2ENullInt= rospector; +import org=2Eapache=2Easterix=2Eformats=2Enontagged=2ETypeTrait= Provider; import org=2Eapache=2Easterix=2Emetadata=2EIDatasetDetails; imp= ort org=2Eapache=2Easterix=2Emetadata=2EMetadataManager; import org=2Eapac= he=2Easterix=2Emetadata=2EMetadataNode; @@ -66,6 +68,7 @@ import org=2Eapa= che=2Easterix=2Emetadata=2Efeeds=2EBuiltinFeedPolicies; import org=2Eapach= e=2Easterix=2Emetadata=2Eutils=2EMetadataConstants; import org=2Eapache=2E= asterix=2Emetadata=2Eutils=2EMetadataUtil; +import org=2Eapache=2Easterix= =2Eom=2Etypes=2EBuiltinType; import org=2Eapache=2Easterix=2Eom=2Etypes=2E= IAType; import org=2Eapache=2Easterix=2Eruntime=2Eformats=2ENonTaggedDataF= ormat; import org=2Eapache=2Easterix=2Etransaction=2Emanagement=2Eopcallba= cks=2EPrimaryIndexOperationTrackerFactory; @@ -428,7 +431,8 @@ = storageComponentProvider=2EgetIoOperationSchedulerProvider= (), appContext=2EgetMetadataMergePolicyFactory= (), StorageConstants=2EDEFAULT_COMPACTION_POLI= CY_PROPERTIES, true, bloomFilterKeyFields, - blo= omFilterFalsePositiveRate, true, null, NoOpCompressorDecompressorFactory=2E= INSTANCE, true); + bloomFilterFalsePositiveRate,= true, null, NoOpCompressorDecompressorFactory=2EINSTANCE, true, + = TypeTraitProvider=2EINSTANCE=2EgetTypeTrait(BuiltinType= =2EANULL), NullIntrospector=2EINSTANCE); DatasetLocalResourceF= actory dsLocalResourceFactory =3D new DatasetLocalReso= urceFactory(datasetId, lsmBtreeFactory); // TODO(amoudi) Creat= ing the index should be done through the same code path as diff --git a/ast= erixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/= ArrayBTreeResourceFactoryProvider=2Ejava b/asterixdb/asterix-metadata/src/m= ain/java/org/apache/asterix/metadata/declared/ArrayBTreeResourceFactoryProv= ider=2Ejava index 7c19282=2E=2E5654d16 100644 --- a/asterixdb/asterix-metad= ata/src/main/java/org/apache/asterix/metadata/declared/ArrayBTreeResourceFa= ctoryProvider=2Ejava +++ b/asterixdb/asterix-metadata/src/main/java/org/apa= che/asterix/metadata/declared/ArrayBTreeResourceFactoryProvider=2Ejava @@ -= 26,12 +26,14 @@ import org=2Eapache=2Easterix=2Ecommon=2Econtext=2EIStorag= eComponentProvider; import org=2Eapache=2Easterix=2Ecommon=2Eexceptions=2E= CompilationException; import org=2Eapache=2Easterix=2Ecommon=2Eexceptions= =2EErrorCode; +import org=2Eapache=2Easterix=2Eformats=2Enontagged=2ENullIn= trospector; import org=2Eapache=2Easterix=2Emetadata=2Eapi=2EIResourceFact= oryProvider; import org=2Eapache=2Easterix=2Emetadata=2Eentities=2EDataset= ; import org=2Eapache=2Easterix=2Emetadata=2Eentities=2EIndex; import org= =2Eapache=2Easterix=2Emetadata=2Eutils=2EArrayIndexUtil; import org=2Eapac= he=2Easterix=2Emetadata=2Eutils=2EIndexUtil; import org=2Eapache=2Easterix= =2Eom=2Etypes=2EARecordType; +import org=2Eapache=2Easterix=2Eom=2Etypes=2E= BuiltinType; import org=2Eapache=2Easterix=2Eom=2Etypes=2EIAType; import = org=2Eapache=2Ehyracks=2Ealgebricks=2Ecommon=2Eexceptions=2EAlgebricksExcep= tion; import org=2Eapache=2Ehyracks=2Ealgebricks=2Ecommon=2Eutils=2EPair; = @@ -66,6 +68,7 @@ int[] filterFields =3D IndexUtil=2EgetFilterFiel= ds(dataset, index, filterTypeTraits); int[] btreeFields =3D IndexU= til=2EgetBtreeFieldsIfFiltered(dataset, index); IStorageComponentP= rovider storageComponentProvider =3D mdProvider=2EgetStorageComponentProvid= er(); + ITypeTraitProvider typeTraitProvider =3D storageComponentPro= vider=2EgetTypeTraitProvider(); ITypeTraits[] typeTraits =3D getTy= peTraits(mdProvider, dataset, index, recordType, metaType); IBinar= yComparatorFactory[] cmpFactories =3D getCmpFactories(mdProvider, dataset, = index, recordType, metaType); double bloomFilterFalsePositiveRate = =3D mdProvider=2EgetStorageProperties()=2EgetBloomFilterFalsePositiveRate()= ; @@ -97,7 +100,8 @@ filterCmpFactories, filterFie= lds, opTrackerFactory, ioOpCallbackFactory, pageWr= iteCallbackFactory, metadataPageManagerFactory, vbcProvider, ioSchedulerPro= vider, mergePolicyFactory, mergePolicyProperties, = true, null, bloomFilterFalsePositiveRate, - index=2E= isPrimaryIndex(), btreeFields, compDecompFactory, false); + = index=2EisPrimaryIndex(), btreeFields, compDecompFactory, false, + = typeTraitProvider=2EgetTypeTrait(BuiltinType=2EANULL= ), NullIntrospector=2EINSTANCE); default: thr= ow new CompilationException(ErrorCode=2ECOMPILATION_UNKNOWN_DATASET_TYPE, = dataset=2EgetDatasetType()=2EtoString()); diff --gi= t a/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/de= clared/BTreeResourceFactoryProvider=2Ejava b/asterixdb/asterix-metadata/src= /main/java/org/apache/asterix/metadata/declared/BTreeResourceFactoryProvide= r=2Ejava index 7f56f2c=2E=2E6276044 100644 --- a/asterixdb/asterix-metadata= /src/main/java/org/apache/asterix/metadata/declared/BTreeResourceFactoryPro= vider=2Ejava +++ b/asterixdb/asterix-metadata/src/main/java/org/apache/aste= rix/metadata/declared/BTreeResourceFactoryProvider=2Ejava @@ -28,11 +28,13 = @@ import org=2Eapache=2Easterix=2Ecommon=2Eexceptions=2EErrorCode; impor= t org=2Eapache=2Easterix=2Eexternal=2Eindexing=2EFilesIndexDescription; im= port org=2Eapache=2Easterix=2Eexternal=2Eindexing=2EIndexingConstants; +imp= ort org=2Eapache=2Easterix=2Eformats=2Enontagged=2ENullIntrospector; impor= t org=2Eapache=2Easterix=2Emetadata=2Eapi=2EIResourceFactoryProvider; impo= rt org=2Eapache=2Easterix=2Emetadata=2Eentities=2EDataset; import org=2Eap= ache=2Easterix=2Emetadata=2Eentities=2EIndex; import org=2Eapache=2Easteri= x=2Emetadata=2Eutils=2EIndexUtil; import org=2Eapache=2Easterix=2Eom=2Etyp= es=2EARecordType; +import org=2Eapache=2Easterix=2Eom=2Etypes=2EBuiltinType= ; import org=2Eapache=2Easterix=2Eom=2Etypes=2EIAType; import org=2Eapach= e=2Ehyracks=2Ealgebricks=2Ecommon=2Eexceptions=2EAlgebricksException; impo= rt org=2Eapache=2Ehyracks=2Ealgebricks=2Ecommon=2Eutils=2EPair; @@ -82,6 +8= 4,7 @@ ILSMIOOperationSchedulerProvider ioSchedulerProvider =3D = storageComponentProvider=2EgetIoOperationSchedulerProvider()= ; boolean hasBloomFilter =3D bloomFilterFields !=3D null; + = ITypeTraitProvider typeTraitProvider =3D mdProvider=2EgetDataFormat()=2Ege= tTypeTraitProvider(); switch (dataset=2EgetDatasetType()) { = case EXTERNAL: return index=2EgetIndexName()=2Eequa= ls(IndexingConstants=2EgetFilesIndexName(dataset=2EgetDatasetName())) @@ -8= 9,12 +92,14 @@ filterTypeTraits, filterCmp= Factories, filterFields, opTrackerFactory, = ioOpCallbackFactory, pageWriteCallbackFactory, metadataPageManagerFactory,= ioSchedulerProvider, mergePolicyFactory, = mergePolicyProperties, true, bloomFilterFields, - = bloomFilterFalsePositiveRate, false, btreeFields, hasBloomFilter) + = bloomFilterFalsePositiveRate, false, btreeFie= lds, hasBloomFilter, + typeTraitProvider=2Eg= etTypeTrait(BuiltinType=2EANULL), NullIntrospector=2EINSTANCE) = : new ExternalBTreeWithBuddyLocalResourceFactory(storageManage= r, typeTraits, cmpFactories, filterTypeTra= its, filterCmpFactories, filterFields, opTrackerFactory, = ioOpCallbackFactory, pageWriteCallbackFactory, metadataPageM= anagerFactory, ioSchedulerProvider, mergeP= olicyFactory, mergePolicyProperties, true, bloomFilterFields, - = bloomFilterFalsePositiveRate, false, btreeFields, hasBl= oomFilter); + bloomFilterFalsePositiveRate, = false, btreeFields, hasBloomFilter, + typeTr= aitProvider=2EgetTypeTrait(BuiltinType=2EANULL), NullIntrospector=2EINSTANC= E); case INTERNAL: AsterixVirtualBufferCacheP= rovider vbcProvider =3D new AsterixVirtualBufferCa= cheProvider(dataset=2EgetDatasetId()); @@ -112,7 +117,7 @@ = pageWriteCallbackFactory, metadataPageManagerFactory, vbcProvider,= ioSchedulerProvider, mergePolicyFactory, mergePol= icyProperties, true, bloomFilterFields, bloomFilte= rFalsePositiveRate, index=2EisPrimaryIndex(), btreeFields, compDecompFactor= y, - hasBloomFilter); + hasBl= oomFilter, typeTraitProvider=2EgetTypeTrait(BuiltinType=2EANULL), NullIntro= spector=2EINSTANCE); default: throw new Compi= lationException(ErrorCode=2ECOMPILATION_UNKNOWN_DATASET_TYPE, = dataset=2EgetDatasetType()=2EtoString()); diff --git a/asterixd= b/asterix-metadata/src/main/java/org/apache/asterix/metadata/valueextractor= s/TupleCopyValueExtractor=2Ejava b/asterixdb/asterix-metadata/src/main/java= /org/apache/asterix/metadata/valueextractors/TupleCopyValueExtractor=2Ejava= index c9fb39c=2E=2E9b19bb5 100644 --- a/asterixdb/asterix-metadata/src/mai= n/java/org/apache/asterix/metadata/valueextractors/TupleCopyValueExtractor= =2Ejava +++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/m= etadata/valueextractors/TupleCopyValueExtractor=2Ejava @@ -27,6 +27,7 @@ i= mport org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits; impor= t org=2Eapache=2Ehyracks=2Eapi=2Eexceptions=2EHyracksDataException; import= org=2Eapache=2Ehyracks=2Edataflow=2Ecommon=2Edata=2Eaccessors=2EITupleRefe= rence; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINull= Introspector; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Etupl= es=2ETypeAwareTupleReference; import org=2Eapache=2Ehyracks=2Estorage=2Eam= =2Ecommon=2Etuples=2ETypeAwareTupleWriter; @@ -42,8 +43,9 @@ private= byte[] tupleBytes; private ByteBuffer buf; - public TupleCopyVal= ueExtractor(ITypeTraits[] typeTraits) { - this=2EtupleWriter =3D new= TypeAwareTupleWriter(typeTraits); + public TupleCopyValueExtractor(ITyp= eTraits[] typeTraits, ITypeTraits nullTypeTraits, + INullIntrosp= ector nullIntrospector) { + this=2EtupleWriter =3D new TypeAwareTupl= eWriter(typeTraits, nullTypeTraits, nullIntrospector); this=2Etupl= eReference =3D tupleWriter=2EcreateTupleReference(); } diff --git a/= asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/Nul= lIntrospector=2Ejava b/asterixdb/asterix-om/src/main/java/org/apache/asteri= x/formats/nontagged/NullIntrospector=2Ejava new file mode 100644 index 0000= 000=2E=2Ef496efe --- /dev/null +++ b/asterixdb/asterix-om/src/main/java/org= /apache/asterix/formats/nontagged/NullIntrospector=2Ejava @@ -0,0 +1,55 @@ = +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or m= ore contributor license agreements=2E See the NOTICE file + * distributed = with this work for additional information + * regarding copyright ownership= =2E The ASF licenses this file + * to you under the Apache License, Versio= n 2=2E0 (the + * "License"); you may not use this file except in compliance= + * with the License=2E You may obtain a copy of the License at + * + * = http://www=2Eapache=2Eorg/licenses/LICENSE-2=2E0 + * + * Unless required b= y applicable law or agreed to in writing, + * software distributed under th= e License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CON= DITIONS OF ANY + * KIND, either express or implied=2E See the License for = the + * specific language governing permissions and limitations + * under t= he License=2E + */ +package org=2Eapache=2Easterix=2Eformats=2Enontagged; += +import static org=2Eapache=2Easterix=2Eom=2Etypes=2EATypeTag=2ESERIALIZED= _MISSING_TYPE_TAG; +import static org=2Eapache=2Easterix=2Eom=2Etypes=2EATy= peTag=2ESERIALIZED_NULL_TYPE_TAG; + +import org=2Eapache=2Ehyracks=2Eapi=2E= exceptions=2EHyracksDataException; +import org=2Eapache=2Ehyracks=2Eapi=2Ei= o=2EIJsonSerializable; +import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIPersist= edResourceRegistry; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon= =2Eapi=2EINullIntrospector; + +import com=2Efasterxml=2Ejackson=2Edatabind= =2EJsonNode; + +public class NullIntrospector implements INullIntrospector = { + + private static final long serialVersionUID =3D 1L; + + public s= tatic final INullIntrospector INSTANCE =3D new NullIntrospector(); + + p= rivate NullIntrospector() { + } + + @Override + public boolean isN= ull(byte[] bytes, int offset, int length) { + return (bytes[offset] = =3D=3D SERIALIZED_MISSING_TYPE_TAG || bytes[offset] =3D=3D SERIALIZED_NULL_= TYPE_TAG) + && length =3D=3D 1; + } + + @Override + = public JsonNode toJson(IPersistedResourceRegistry registry) throws Hyracks= DataException { + return registry=2EgetClassIdentifier(getClass(), s= erialVersionUID); + } + + @SuppressWarnings("squid:S1172") // unused = parameter + public static IJsonSerializable fromJson(IPersistedResourceR= egistry registry, JsonNode json) { + return INSTANCE; + } +} diff= --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nont= agged/TypeTraitProvider=2Ejava b/asterixdb/asterix-om/src/main/java/org/apa= che/asterix/formats/nontagged/TypeTraitProvider=2Ejava index 1788445=2E=2E9= 7fe9c1 100644 --- a/asterixdb/asterix-om/src/main/java/org/apache/asterix/f= ormats/nontagged/TypeTraitProvider=2Ejava +++ b/asterixdb/asterix-om/src/ma= in/java/org/apache/asterix/formats/nontagged/TypeTraitProvider=2Ejava @@ -2= 8,6 +28,7 @@ // WARNING: the byte sizes depend on the serializer! = // currently assuming a serializer that adds a 1-byte type indicator bef= ore the data + private static final ITypeTraits ZERO_BYTE_TYPE_TRAIT =3D= new FixedLengthTypeTrait(1); private static final ITypeTraits ONE_BYT= E_TYPE_TRAIT =3D new FixedLengthTypeTrait(1 + 1); private static final= ITypeTraits TWO_BYTE_TYPE_TRAIT =3D new FixedLengthTypeTrait(2 + 1); = private static final ITypeTraits FOUR_BYTE_TYPE_TRAIT =3D new FixedLengthTy= peTrait(4 + 1); @@ -48,6 +49,9 @@ return null; } = switch (type=2EgetTypeTag()) { + case NULL: + ca= se MISSING: + return ZERO_BYTE_TYPE_TRAIT; case= BOOLEAN: case TINYINT: return ONE_BYTE_TYPE_= TRAIT; diff --git a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-inte= gration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryI= ndexScanOperatorTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-examples/hy= racks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTr= eePrimaryIndexScanOperatorTest=2Ejava index 336204c=2E=2Ee4214cc 100644 ---= a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src= /test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorT= est=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integra= tion-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryInde= xScanOperatorTest=2Ejava @@ -98,12 +98,12 @@ @Override protected = IResourceFactory createPrimaryResourceFactory() { return new BTree= ResourceFactory(storageManager, DataSetConstants=2EprimaryTypeTraits, - = DataSetConstants=2EprimaryComparatorFactories, pageManagerFacto= ry); + DataSetConstants=2EprimaryComparatorFactories, pageMa= nagerFactory, null, null); } @Override protected IResource= Factory createSecondaryResourceFactory() { return new BTreeResourc= eFactory(storageManager, DataSetConstants=2EsecondaryTypeTraits, - = DataSetConstants=2EsecondaryComparatorFactories, pageManagerFactory)= ; + DataSetConstants=2EsecondaryComparatorFactories, pageMan= agerFactory, null, null); } } diff --git a/hyracks-fullstack/hyracks/= hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks= /tests/am/btree/BTreePrimaryIndexSearchOperatorTest=2Ejava b/hyracks-fullst= ack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/ap= ache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest=2Ejava inde= x 870cfe3=2E=2E65bb2f0 100644 --- a/hyracks-fullstack/hyracks/hyracks-examp= les/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btr= ee/BTreePrimaryIndexSearchOperatorTest=2Ejava +++ b/hyracks-fullstack/hyrac= ks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyra= cks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest=2Ejava @@ -103,12 +1= 03,12 @@ @Override protected IResourceFactory createPrimaryResour= ceFactory() { return new BTreeResourceFactory(storageManager, Data= SetConstants=2EprimaryTypeTraits, - DataSetConstants=2Eprima= ryComparatorFactories, pageManagerFactory); + DataSetConstan= ts=2EprimaryComparatorFactories, pageManagerFactory, null, null); } = @Override protected IResourceFactory createSecondaryResourceFacto= ry() { return new BTreeResourceFactory(storageManager, DataSetCons= tants=2EsecondaryTypeTraits, - DataSetConstants=2EsecondaryC= omparatorFactories, pageManagerFactory); + DataSetConstants= =2EsecondaryComparatorFactories, pageManagerFactory, null, null); } }= diff --git a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integratio= n-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSt= atsOperatorTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-examples/hyracks= -integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePri= maryIndexStatsOperatorTest=2Ejava index 5cffd95=2E=2Ea1f53bd 100644 --- a/h= yracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/tes= t/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest= =2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integratio= n-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSt= atsOperatorTest=2Ejava @@ -67,12 +67,12 @@ @Override protected IR= esourceFactory createPrimaryResourceFactory() { return new BTreeRe= sourceFactory(storageManager, DataSetConstants=2EprimaryTypeTraits, - = DataSetConstants=2EprimaryComparatorFactories, pageManagerFactory= ); + DataSetConstants=2EprimaryComparatorFactories, pageMana= gerFactory, null, null); } @Override protected IResourceFa= ctory createSecondaryResourceFactory() { return new BTreeResourceF= actory(storageManager, DataSetConstants=2EsecondaryTypeTraits, - = DataSetConstants=2EsecondaryComparatorFactories, pageManagerFactory); = + DataSetConstants=2EsecondaryComparatorFactories, pageManag= erFactory, null, null); } } diff --git a/hyracks-fullstack/hyracks/hy= racks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/t= ests/am/btree/BTreeSecondaryIndexInsertOperatorTest=2Ejava b/hyracks-fullst= ack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/ap= ache/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest=2Ejava in= dex b2060b6=2E=2Eb489474 100644 --- a/hyracks-fullstack/hyracks/hyracks-exa= mples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/b= tree/BTreeSecondaryIndexInsertOperatorTest=2Ejava +++ b/hyracks-fullstack/h= yracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/= hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest=2Ejava @@ -121= ,12 +121,12 @@ @Override protected IResourceFactory createPrimary= ResourceFactory() { return new BTreeResourceFactory(storageManager= , DataSetConstants=2EprimaryTypeTraits, - DataSetConstants= =2EprimaryComparatorFactories, pageManagerFactory); + DataSe= tConstants=2EprimaryComparatorFactories, pageManagerFactory, null, null); = } @Override protected IResourceFactory createSecondaryResou= rceFactory() { return new BTreeResourceFactory(storageManager, Dat= aSetConstants=2EsecondaryTypeTraits, - DataSetConstants=2Ese= condaryComparatorFactories, pageManagerFactory); + DataSetCo= nstants=2EsecondaryComparatorFactories, pageManagerFactory, null, null); = } } diff --git a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-int= egration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSeconda= ryIndexSearchOperatorTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-exampl= es/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btre= e/BTreeSecondaryIndexSearchOperatorTest=2Ejava index e73a11e=2E=2Ea1d1502 1= 00644 --- a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-= tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSe= archOperatorTest=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-examples/hy= racks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTr= eeSecondaryIndexSearchOperatorTest=2Ejava @@ -117,12 +117,12 @@ @Overr= ide protected IResourceFactory createPrimaryResourceFactory() { = return new BTreeResourceFactory(storageManager, DataSetConstants=2Eprima= ryTypeTraits, - DataSetConstants=2EprimaryComparatorFactorie= s, pageManagerFactory); + DataSetConstants=2EprimaryComparat= orFactories, pageManagerFactory, null, null); } @Override = protected IResourceFactory createSecondaryResourceFactory() { retu= rn new BTreeResourceFactory(storageManager, DataSetConstants=2EsecondaryTyp= eTraits, - DataSetConstants=2EsecondaryComparatorFactories, = pageManagerFactory); + DataSetConstants=2EsecondaryComparato= rFactories, pageManagerFactory, null, null); } } diff --git a/hyracks= -fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java= /org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest=2E= java b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests= /src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertO= peratorTest=2Ejava index a333dc4=2E=2Ef9cdfcf 100644 --- a/hyracks-fullstac= k/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apac= he/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest=2Ejava +++ = b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/= test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpsertOperat= orTest=2Ejava @@ -120,12 +120,12 @@ @Override protected IResource= Factory createPrimaryResourceFactory() { return new BTreeResourceF= actory(storageManager, DataSetConstants=2EprimaryTypeTraits, - = DataSetConstants=2EprimaryComparatorFactories, pageManagerFactory); + = DataSetConstants=2EprimaryComparatorFactories, pageManagerFact= ory, null, null); } @Override protected IResourceFactory c= reateSecondaryResourceFactory() { return new BTreeResourceFactory(= storageManager, DataSetConstants=2EsecondaryTypeTraits, - Da= taSetConstants=2EsecondaryComparatorFactories, pageManagerFactory); + = DataSetConstants=2EsecondaryComparatorFactories, pageManagerFacto= ry, null, null); } } diff --git a/hyracks-fullstack/hyracks/hyracks-e= xamples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am= /lsm/btree/LSMBTreeOperatorTestHelper=2Ejava b/hyracks-fullstack/hyracks/hy= racks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/t= ests/am/lsm/btree/LSMBTreeOperatorTestHelper=2Ejava index d46f5f5=2E=2Ebcd7= 1a3 100644 --- a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integra= tion-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOper= atorTestHelper=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-examples/hyra= cks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/L= SMBTreeOperatorTestHelper=2Ejava @@ -49,6 +49,6 @@ getVirt= ualBufferCacheProvider(), SynchronousSchedulerProvider=2EINSTANCE, MERGE_PO= LICY_FACTORY, MERGE_POLICY_PROPERTIES, DURABLE, bloomFilte= rKeyFields, LSMTreeOperatorTestHelper=2EDEFAULT_BLOOM_FILT= ER_FALSE_POSITIVE_RATE, true, btreefields, - NoOpCompressorD= ecompressorFactory=2EINSTANCE, bloomFilterKeyFields !=3D null); + = NoOpCompressorDecompressorFactory=2EINSTANCE, bloomFilterKeyFields != =3D null, null, null); } } diff --git a/hyracks-fullstack/hyracks/hyr= acks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/te= sts/am/rtree/AbstractRTreeOperatorTest=2Ejava b/hyracks-fullstack/hyracks/h= yracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/= tests/am/rtree/AbstractRTreeOperatorTest=2Ejava index 2d092f1=2E=2E872502b = 100644 --- a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration= -tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperato= rTest=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integ= ration-tests/src/test/java/org/apache/hyracks/tests/am/rtree/AbstractRTreeO= peratorTest=2Ejava @@ -228,7 +228,7 @@ protected void createPrimaryInd= ex() throws Exception { JobSpecification spec =3D new JobSpecifica= tion(); btreeFactory =3D new BTreeResourceFactory(storageManager, = primaryTypeTraits, primaryComparatorFactories, - pageManager= Factory); + pageManagerFactory, null, null); IIndex= BuilderFactory indexBuilderFactory =3D new IndexBuilderFac= tory(storageManager, primarySplitProvider, btreeFactory, false); I= ndexCreateOperatorDescriptor primaryCreateOp =3D new IndexCreateOperatorDes= criptor(spec, indexBuilderFactory); diff --git a/hyracks-fullstack/hyracks/= hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/= compressors/FieldPrefixCompressor=2Ejava b/hyracks-fullstack/hyracks/hyrack= s-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/compre= ssors/FieldPrefixCompressor=2Ejava index 8c327f7=2E=2E17fe593 100644 --- a/= hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache= /hyracks/storage/am/btree/compressors/FieldPrefixCompressor=2Ejava +++ b/hy= racks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/h= yracks/storage/am/btree/compressors/FieldPrefixCompressor=2Ejava @@ -32,6 += 32,7 @@ import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Eimpls=2EBTr= eeFieldPrefixTupleReference; import org=2Eapache=2Ehyracks=2Estorage=2Eam= =2Ebtree=2Eimpls=2EFieldPrefixSlotManager; import org=2Eapache=2Ehyracks= =2Estorage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriter; +import org= =2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; imp= ort org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EITreeIndexFrame;= import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EITreeIndexF= rameCompressor; import org=2Eapache=2Ehyracks=2Estorage=2Ecommon=2EMultiCo= mparator; @@ -46,10 +47,17 @@ private final ITypeTraits[] typeTraits= ; - public FieldPrefixCompressor(ITypeTraits[] typeTraits, float ratio= Threshold, int occurrenceThreshold) { + private final ITypeTraits nullTy= peTraits; + + private final INullIntrospector nullIntrospector; + + p= ublic FieldPrefixCompressor(ITypeTraits[] typeTraits, float ratioThreshold,= int occurrenceThreshold, + ITypeTraits nullTypeTraits, INullInt= rospector nullIntrospector) { this=2EtypeTraits =3D typeTraits; = this=2EratioThreshold =3D ratioThreshold; this=2Eoccurrence= Threshold =3D occurrenceThreshold; + this=2EnullTypeTraits =3D nullT= ypeTraits; + this=2EnullIntrospector =3D nullIntrospector; } = @Override @@ -163,7 +171,8 @@ int prefixTupleIndex =3D 0; = uncompressedTupleCount =3D 0; - BTreeTypeAwareTupleWriter tu= pleWriter =3D new BTreeTypeAwareTupleWriter(typeTraits, false); + BT= reeTypeAwareTupleWriter tupleWriter =3D + new BTreeTypeAware= TupleWriter(typeTraits, false, nullTypeTraits, nullIntrospector); = BTreeFieldPrefixTupleReference tupleToWrite =3D new BTreeF= ieldPrefixTupleReference(tupleWriter=2EcreateTupleReference()); tu= pleToWrite=2EsetFieldCount(fieldCount); @@ -340,7 +349,8 @@ KeyPar= tition kp =3D new KeyPartition(maxCmps); keyPartitions=2Eadd(kp); = - BTreeTypeAwareTupleWriter tupleWriter =3D new BTreeTypeAwareTupl= eWriter(typeTraits, false); + BTreeTypeAwareTupleWriter tupleWriter = =3D + new BTreeTypeAwareTupleWriter(typeTraits, false, nullT= ypeTraits, nullIntrospector); BTreeFieldPrefixTupleReference pre= vTuple =3D new BTreeFieldPrefixTupleReference(tupleWriter= =2EcreateTupleReference()); diff --git a/hyracks-fullstack/hyracks/hyracks-= storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow= /BTreeResource=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/= src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeResource=2E= java index 4cf145b=2E=2E4589bed1 100644 --- a/hyracks-fullstack/hyracks/hyr= acks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dat= aflow/BTreeResource=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-= am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeRe= source=2Ejava @@ -26,6 +26,7 @@ import org=2Eapache=2Ehyracks=2Eapi=2Eio= =2EIIOManager; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Efram= es=2EBTreeLeafFrameType; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebt= ree=2Eutil=2EBTreeUtils; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Eco= mmon=2Eapi=2EINullIntrospector; import org=2Eapache=2Ehyracks=2Estorage=2E= am=2Ecommon=2Eapi=2EIPageManagerFactory; import org=2Eapache=2Ehyracks=2Es= torage=2Ecommon=2EIIndex; import org=2Eapache=2Ehyracks=2Estorage=2Ecommon= =2EIResource; @@ -34,20 +35,25 @@ public class BTreeResource implements = IResource { - private static final long serialVersionUID =3D 1L; + = private static final long serialVersionUID =3D 2L; private String path= ; private final IStorageManager storageManager; private final ITy= peTraits[] typeTraits; private final IBinaryComparatorFactory[] compar= atorFactories; private final IPageManagerFactory pageManagerFactory; += private final ITypeTraits nullTypeTraits; + private final INullIntro= spector nullIntrospector; public BTreeResource(String path, IStorage= Manager storageManager, ITypeTraits[] typeTraits, - IBinaryCompa= ratorFactory[] comparatorFactories, IPageManagerFactory pageManagerFactory)= { + IBinaryComparatorFactory[] comparatorFactories, IPageManage= rFactory pageManagerFactory, + ITypeTraits nullTypeTraits, INull= Introspector nullIntrospector) { this=2Epath =3D path; th= is=2EstorageManager =3D storageManager; this=2EtypeTraits =3D type= Traits; this=2EcomparatorFactories =3D comparatorFactories; = this=2EpageManagerFactory =3D pageManagerFactory; + this=2EnullTy= peTraits =3D nullTypeTraits; + this=2EnullIntrospector =3D nullIntro= spector; } @Override @@ -56,7 +62,8 @@ IIOManager ioMa= nager =3D ctx=2EgetIoManager(); FileReference resourceRef =3D ioMa= nager=2Eresolve(path); return BTreeUtils=2EcreateBTree(bufferCache= , typeTraits, comparatorFactories, BTreeLeafFrameType=2EREGULAR_NSM, - = resourceRef, pageManagerFactory=2EcreatePageManager(bufferCache)= , false); + resourceRef, pageManagerFactory=2EcreatePageMana= ger(bufferCache), false, nullTypeTraits, + nullIntrospector)= ; } @Override diff --git a/hyracks-fullstack/hyracks/hyracks-st= orage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/B= TreeResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-b= tree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeResour= ceFactory=2Ejava index 7a21495=2E=2Ed817327 100644 --- a/hyracks-fullstack/= hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/a= m/btree/dataflow/BTreeResourceFactory=2Ejava +++ b/hyracks-fullstack/hyrack= s/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btre= e/dataflow/BTreeResourceFactory=2Ejava @@ -21,6 +21,7 @@ import org=2Eapac= he=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EIBinaryComparatorFactory; import o= rg=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits; import org= =2Eapache=2Ehyracks=2Eapi=2Eio=2EFileReference; +import org=2Eapache=2Ehyra= cks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; import org=2Eapache= =2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIPageManagerFactory; import org= =2Eapache=2Ehyracks=2Estorage=2Ecommon=2EIResource; import org=2Eapache=2E= hyracks=2Estorage=2Ecommon=2EIResourceFactory; @@ -28,24 +29,29 @@ publi= c class BTreeResourceFactory implements IResourceFactory { - private s= tatic final long serialVersionUID =3D 1L; + private static final long se= rialVersionUID =3D 2L; private final IStorageManager storageManager; = private final ITypeTraits[] typeTraits; private final IBinaryCompa= ratorFactory[] comparatorFactories; private final IPageManagerFactory = pageManagerFactory; + private final ITypeTraits nullTypeTraits; + pri= vate final INullIntrospector nullIntrospector; public BTreeResourceF= actory(IStorageManager storageManager, ITypeTraits[] typeTraits, - = IBinaryComparatorFactory[] comparatorFactories, IPageManagerFactory page= ManagerFactory) { + IBinaryComparatorFactory[] comparatorFactori= es, IPageManagerFactory pageManagerFactory, + ITypeTraits nullTy= peTraits, INullIntrospector nullIntrospector) { this=2EstorageMana= ger =3D storageManager; this=2EtypeTraits =3D typeTraits; = this=2EcomparatorFactories =3D comparatorFactories; this=2EpageMa= nagerFactory =3D pageManagerFactory; + this=2EnullTypeTraits =3D nul= lTypeTraits; + this=2EnullIntrospector =3D nullIntrospector; } = @Override public IResource createResource(FileReference fileRef= ) { return new BTreeResource(fileRef=2EgetRelativePath(), storageM= anager, typeTraits, comparatorFactories, - pageManagerFactor= y); + pageManagerFactory, nullTypeTraits, nullIntrospector);= } } diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-btre= e/src/main/java/org/apache/hyracks/storage/am/btree/frames/BTreeFieldPrefix= NSMLeafFrame=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/sr= c/main/java/org/apache/hyracks/storage/am/btree/frames/BTreeFieldPrefixNSML= eafFrame=2Ejava index 6584df3=2E=2E10eab69 100644 --- a/hyracks-fullstack/h= yracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am= /btree/frames/BTreeFieldPrefixNSMLeafFrame=2Ejava +++ b/hyracks-fullstack/h= yracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am= /btree/frames/BTreeFieldPrefixNSMLeafFrame=2Ejava @@ -75,8 +75,10 @@ = this=2EslotManager =3D new FieldPrefixSlotManager(); ITypeTra= its[] typeTraits =3D tupleWriter=2EgetTypeTraits(); - this=2EframePr= efixTuple =3D new FieldPrefixPrefixTupleReference(typeTraits); - thi= s=2Ecompressor =3D new FieldPrefixCompressor(typeTraits, 0=2E001f, 2); + = ITypeTraits nullTypeTraits =3D tupleWriter=2EgetNullTypeTraits(); + = this=2EframePrefixTuple =3D new FieldPrefixPrefixTupleReference(typeTr= aits, nullTypeTraits); + this=2Ecompressor =3D + new = FieldPrefixCompressor(typeTraits, 0=2E001f, 2, nullTypeTraits, tupleWriter= =2EgetNullIntrospector()); } @Override diff --git a/hyracks-ful= lstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/st= orage/am/btree/impls/FieldPrefixPrefixTupleReference=2Ejava b/hyracks-fulls= tack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/stor= age/am/btree/impls/FieldPrefixPrefixTupleReference=2Ejava index b767537=2E= =2E2c728a7 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/= src/main/java/org/apache/hyracks/storage/am/btree/impls/FieldPrefixPrefixTu= pleReference=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-btre= e/src/main/java/org/apache/hyracks/storage/am/btree/impls/FieldPrefixPrefix= TupleReference=2Ejava @@ -27,8 +27,8 @@ public class FieldPrefixPrefixTu= pleReference extends TypeAwareTupleReference { - public FieldPrefixPre= fixTupleReference(ITypeTraits[] typeTraits) { - super(typeTraits); += public FieldPrefixPrefixTupleReference(ITypeTraits[] typeTraits, ITypeT= raits nullTypeTraits) { + super(typeTraits, nullTypeTraits); } = // assumes tuple index refers to prefix tuples diff --git a/hyracks-= fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks= /storage/am/btree/tuples/BTreeTypeAwareTupleReference=2Ejava b/hyracks-full= stack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/sto= rage/am/btree/tuples/BTreeTypeAwareTupleReference=2Ejava index 5b89a19=2E= =2Ef4b1948 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/= src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupl= eReference=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/= src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupl= eReference=2Ejava @@ -23,14 +23,15 @@ import org=2Eapache=2Ehyracks=2Estor= age=2Eam=2Ecommon=2Etuples=2ETypeAwareTupleReference; import org=2Eapache= =2Ehyracks=2Estorage=2Eam=2Ecommon=2Eutil=2EBitOperationUtils; +// TODO(a= li): this is broken if updateAware is enabled public class BTreeTypeAwareT= upleReference extends TypeAwareTupleReference implements IBTreeIndexTupleRe= ference { public static final byte UPDATE_BIT_OFFSET =3D 6; p= rotected final boolean updateAware; - public BTreeTypeAwareTupleRefere= nce(ITypeTraits[] typeTraits, boolean updateAware) { - super(typeTra= its); + public BTreeTypeAwareTupleReference(ITypeTraits[] typeTraits, bo= olean updateAware, ITypeTraits nullTypeTraits) { + super(typeTraits,= nullTypeTraits); this=2EupdateAware =3D updateAware; } dif= f --git a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/= org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupleWriter=2Ejava= b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apa= che/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupleWriter=2Ejava index = 083462b=2E=2E87f46b2 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage= -am-btree/src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTyp= eAwareTupleWriter=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am= -btree/src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAw= areTupleWriter=2Ejava @@ -19,21 +19,23 @@ package org=2Eapache=2Ehyracks= =2Estorage=2Eam=2Ebtree=2Etuples; import org=2Eapache=2Ehyracks=2Eapi=2E= dataflow=2Evalue=2EITypeTraits; -import org=2Eapache=2Ehyracks=2Estorage=2E= am=2Ecommon=2Eapi=2EITreeIndexTupleWriter; +import org=2Eapache=2Ehyracks= =2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; import org=2Eapache=2E= hyracks=2Estorage=2Eam=2Ecommon=2Etuples=2ETypeAwareTupleWriter; -public = class BTreeTypeAwareTupleWriter extends TypeAwareTupleWriter implements ITr= eeIndexTupleWriter { +// TODO(ali): this is broken if updateAware is enable= d +public class BTreeTypeAwareTupleWriter extends TypeAwareTupleWriter { = protected final boolean updateAware; protected boolean isUpdated; = - public BTreeTypeAwareTupleWriter(ITypeTraits[] typeTraits, boolean up= dateAware) { - super(typeTraits); + public BTreeTypeAwareTupleWri= ter(ITypeTraits[] typeTraits, boolean updateAware, ITypeTraits nullTypeTrai= ts, + INullIntrospector nullIntrospector) { + super(typeT= raits, nullTypeTraits, nullIntrospector); this=2EupdateAware =3D u= pdateAware; } @Override public BTreeTypeAwareTupleReferenc= e createTupleReference() { - return new BTreeTypeAwareTupleReference= (typeTraits, updateAware); + return new BTreeTypeAwareTupleReference= (typeTraits, updateAware, nullTypeTraits); } @Override diff --g= it a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/a= pache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupleWriterFactory=2Eja= va b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/a= pache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupleWriterFactory=2Eja= va index 7089e79=2E=2Ee2aa759 100644 --- a/hyracks-fullstack/hyracks/hyrack= s-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/tuples= /BTreeTypeAwareTupleWriterFactory=2Ejava +++ b/hyracks-fullstack/hyracks/hy= racks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/tu= ples/BTreeTypeAwareTupleWriterFactory=2Ejava @@ -19,6 +19,7 @@ package org= =2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Etuples; import org=2Eapache= =2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits; +import org=2Eapache=2Eh= yracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; import org=2Eapa= che=2Ehyracks=2Estorage=2Eam=2Ecommon=2Etuples=2ETypeAwareTupleWriterFactor= y; public class BTreeTypeAwareTupleWriterFactory extends TypeAwareTupleW= riterFactory { @@ -26,13 +27,14 @@ private static final long serialVer= sionUID =3D 1L; protected final boolean updateAware; - public BTr= eeTypeAwareTupleWriterFactory(ITypeTraits[] typeTraits, boolean updateAware= ) { - super(typeTraits); + public BTreeTypeAwareTupleWriterFactor= y(ITypeTraits[] typeTraits, boolean updateAware, ITypeTraits nullTypeTraits= , + INullIntrospector nullIntrospector) { + super(typeTra= its, nullTypeTraits, nullIntrospector); this=2EupdateAware =3D upd= ateAware; } @Override public BTreeTypeAwareTupleWriter cre= ateTupleWriter() { - return new BTreeTypeAwareTupleWriter(typeTraits= , updateAware); + return new BTreeTypeAwareTupleWriter(typeTraits, u= pdateAware, nullTypeTraits, nullIntrospector); } } diff --git a/hyrac= ks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyra= cks/storage/am/btree/util/BTreeUtils=2Ejava b/hyracks-fullstack/hyracks/hyr= acks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/uti= l/BTreeUtils=2Ejava index 0c06bc3=2E=2E09340b6 100644 --- a/hyracks-fullsta= ck/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storag= e/am/btree/util/BTreeUtils=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-s= torage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/util/BTre= eUtils=2Ejava @@ -31,6 +31,7 @@ import org=2Eapache=2Ehyracks=2Estorage=2E= am=2Ebtree=2Eimpls=2EBTree; import org=2Eapache=2Ehyracks=2Estorage=2Eam= =2Ebtree=2Eimpls=2EDiskBTree; import org=2Eapache=2Ehyracks=2Estorage=2Eam= =2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriterFactory; +import org=2Eapache= =2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; import org= =2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIPageManager; import o= rg=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EITreeIndexFrameFactor= y; import org=2Eapache=2Ehyracks=2Estorage=2Ecommon=2EMultiComparator; @@ = -43,9 +44,10 @@ public static BTree createBTree(IBufferCache bufferC= ache, ITypeTraits[] typeTraits, IBinaryComparatorFactory[] cmp= Factories, BTreeLeafFrameType leafType, FileReference file, - IP= ageManager freePageManager, boolean updateAware) throws HyracksDataExceptio= n { + IPageManager freePageManager, boolean updateAware, ITypeTr= aits nullTypeTraits, + INullIntrospector nullIntrospector) throw= s HyracksDataException { BTreeTypeAwareTupleWriterFactory tupleWri= terFactory =3D - new BTreeTypeAwareTupleWriterFactory(typeTr= aits, updateAware); + new BTreeTypeAwareTupleWriterFactory(t= ypeTraits, updateAware, nullTypeTraits, nullIntrospector); ITreeIn= dexFrameFactory leafFrameFactory =3D getLeafFrameFactory(tupleWriterFactory= , leafType); ITreeIndexFrameFactory interiorFrameFactory =3D new B= TreeNSMInteriorFrameFactory(tupleWriterFactory); return new BTree(= bufferCache, freePageManager, interiorFrameFactory, leafFrameFactory, cmpFa= ctories, @@ -54,9 +56,10 @@ public static DiskBTree createDiskBTree(= IBufferCache bufferCache, ITypeTraits[] typeTraits, IBinaryCom= paratorFactory[] cmpFactories, BTreeLeafFrameType leafType, FileReference f= ile, - IPageManager freePageManager, boolean updateAware) throws= HyracksDataException { + IPageManager freePageManager, boolean = updateAware, ITypeTraits nullTypeTraits, + INullIntrospector nul= lIntrospector) throws HyracksDataException { BTreeTypeAwareTupleWr= iterFactory tupleWriterFactory =3D - new BTreeTypeAwareTuple= WriterFactory(typeTraits, updateAware); + new BTreeTypeAware= TupleWriterFactory(typeTraits, updateAware, nullTypeTraits, nullIntrospecto= r); ITreeIndexFrameFactory leafFrameFactory =3D getLeafFrameFactor= y(tupleWriterFactory, leafType); ITreeIndexFrameFactory interiorFr= ameFactory =3D new BTreeNSMInteriorFrameFactory(tupleWriterFactory); = return new DiskBTree(bufferCache, freePageManager, interiorFrameFactory,= leafFrameFactory, cmpFactories, @@ -65,9 +68,10 @@ public static BT= ree createBTree(IBufferCache bufferCache, IPageManager freePageManager, ITy= peTraits[] typeTraits, IBinaryComparatorFactory[] cmpFactories= , BTreeLeafFrameType leafType, FileReference file, - boolean upd= ateAware) throws HyracksDataException { + boolean updateAware, I= TypeTraits nullTypeTraits, INullIntrospector nullIntrospector) + = throws HyracksDataException { BTreeTypeAwareTupleWriterFactory tu= pleWriterFactory =3D - new BTreeTypeAwareTupleWriterFactory(= typeTraits, updateAware); + new BTreeTypeAwareTupleWriterFac= tory(typeTraits, updateAware, nullTypeTraits, nullIntrospector); I= TreeIndexFrameFactory leafFrameFactory =3D getLeafFrameFactory(tupleWriterF= actory, leafType); ITreeIndexFrameFactory interiorFrameFactory =3D= new BTreeNSMInteriorFrameFactory(tupleWriterFactory); return new = BTree(bufferCache, freePageManager, interiorFrameFactory, leafFrameFactory,= cmpFactories, diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-co= mmon/src/main/java/org/apache/hyracks/storage/am/common/api/INullIntrospect= or=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/ja= va/org/apache/hyracks/storage/am/common/api/INullIntrospector=2Ejava new fi= le mode 100644 index 0000000=2E=2Ee159612 --- /dev/null +++ b/hyracks-fulls= tack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/sto= rage/am/common/api/INullIntrospector=2Ejava @@ -0,0 +1,33 @@ +/* + * Licens= ed to the Apache Software Foundation (ASF) under one + * or more contributo= r license agreements=2E See the NOTICE file + * distributed with this work= for additional information + * regarding copyright ownership=2E The ASF l= icenses this file + * to you under the Apache License, Version 2=2E0 (the += * "License"); you may not use this file except in compliance + * with the = License=2E You may obtain a copy of the License at + * + * http://www=2E= apache=2Eorg/licenses/LICENSE-2=2E0 + * + * Unless required by applicable l= aw or agreed to in writing, + * software distributed under the License is d= istributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY= + * KIND, either express or implied=2E See the License for the + * specif= ic language governing permissions and limitations + * under the License=2E = + */ +package org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi; + +impo= rt java=2Eio=2ESerializable; + +import org=2Eapache=2Ehyracks=2Eapi=2Eio=2E= IJsonSerializable; + +/** + * An introspector that tells whether data is {@= code NULL} or not=2E + */ +@FunctionalInterface +public interface INullIntr= ospector extends Serializable, IJsonSerializable { + + boolean isNull(by= te[] bytes, int offset, int length); + +} diff --git a/hyracks-fullstack/hy= racks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am= /common/tuples/SimpleTupleWriter=2Ejava b/hyracks-fullstack/hyracks/hyracks= -storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/tuple= s/SimpleTupleWriter=2Ejava index ca7217e=2E=2E6273b68 100644 --- a/hyracks-= fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyrack= s/storage/am/common/tuples/SimpleTupleWriter=2Ejava +++ b/hyracks-fullstack= /hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage= /am/common/tuples/SimpleTupleWriter=2Ejava @@ -69,6 +69,7 @@ int r= unner =3D targetOff; int nullFlagsBytes =3D getNullFlagsBytes(tupl= e); int fieldSlotsBytes =3D getFieldSlotsBytes(tuple); + //= TODO(ali): fix if needed for (int i =3D 0; i < nullFlagsBytes; i+= +) { targetBuf[runner++] =3D (byte) 0; } diff --git a= /hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apac= he/hyracks/storage/am/common/tuples/TypeAwareTupleReference=2Ejava b/hyrack= s-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyra= cks/storage/am/common/tuples/TypeAwareTupleReference=2Ejava index 11f86c9= =2E=2E8eff435 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-com= mon/src/main/java/org/apache/hyracks/storage/am/common/tuples/TypeAwareTupl= eReference=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-common= /src/main/java/org/apache/hyracks/storage/am/common/tuples/TypeAwareTupleRe= ference=2Ejava @@ -35,11 +35,13 @@ protected int dataStartOff; = protected final ITypeTraits[] typeTraits; - protected VarLenIntDecoder e= ncDec =3D VarLenIntEncoderDecoder=2EcreateDecoder(); + protected final I= TypeTraits nullTypeTraits; // can be null + protected final VarLenIntDec= oder encDec =3D VarLenIntEncoderDecoder=2EcreateDecoder(); protected i= nt[] decodedFieldSlots; - public TypeAwareTupleReference(ITypeTraits[]= typeTraits) { + public TypeAwareTupleReference(ITypeTraits[] typeTraits= , ITypeTraits nullTypeTraits) { this=2EtypeTraits =3D typeTraits; = + this=2EnullTypeTraits =3D nullTypeTraits; this=2EfieldSta= rtIndex =3D 0; setFieldCount(typeTraits=2Elength); } @@ -57,1= 1 +59,12 @@ for (int i =3D fieldStartIndex; i < end; i++) { = if (!typeTraits[i]=2EisFixedLength()) { cumul +=3D = encDec=2Edecode(); - decodedFieldSlots[field++] =3D cumul; += } else if (nullTypeTraits !=3D null && isNull(buf, tupleStartOf= f, i)) { + cumul +=3D nullTypeTraits=2EgetFixedLength(); = } else { cumul +=3D typeTraits[i]=2EgetFixedLeng= th(); - decodedFieldSlots[field++] =3D cumul; }= + decodedFieldSlots[field++] =3D cumul; } dat= aStartOff =3D encDec=2EgetPos(); } @@ -127,4 +130,21 @@ public in= t getTupleSize() { return dataStartOff - tupleStartOff + decodedFi= eldSlots[fieldCount - 1]; } + + private boolean isNull(byte[] flags= , int flagsOffset, int fieldIdx) { + int adjustedFieldIdx =3D getAdj= ustedFieldIdx(fieldIdx); + int flagByteIdx =3D adjustedFieldIdx / 8;= + int flagBitIdx =3D 7 - (adjustedFieldIdx % 8); + return Bi= tOperationUtils=2EgetBit(flags, flagsOffset + flagByteIdx, (byte) flagBitId= x); + } + + /** + * Adjusts the field index in case the null flag= s section starts with some other special-purpose fields=2E + * + * = @param fieldIdx logical field index + * @return adjusted field index + = */ + protected int getAdjustedFieldIdx(int fieldIdx) { + retu= rn fieldIdx; + } } diff --git a/hyracks-fullstack/hyracks/hyracks-stora= ge-am-common/src/main/java/org/apache/hyracks/storage/am/common/tuples/Type= AwareTupleWriter=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-comm= on/src/main/java/org/apache/hyracks/storage/am/common/tuples/TypeAwareTuple= Writer=2Ejava index eb6fe2a=2E=2E2528518 100644 --- a/hyracks-fullstack/hyr= acks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/= common/tuples/TypeAwareTupleWriter=2Ejava +++ b/hyracks-fullstack/hyracks/h= yracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common= /tuples/TypeAwareTupleWriter=2Ejava @@ -23,6 +23,7 @@ import org=2Eapach= e=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits; import org=2Eapache=2E= hyracks=2Edataflow=2Ecommon=2Edata=2Eaccessors=2EITupleReference; +import o= rg=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; i= mport org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EITreeIndexTupl= eWriter; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eutil=2EBi= tOperationUtils; import org=2Eapache=2Ehyracks=2Eutil=2Eencoding=2EVarLenI= ntEncoderDecoder; @@ -30,10 +31,14 @@ public class TypeAwareTupleWriter im= plements ITreeIndexTupleWriter { protected final ITypeTraits[] typeT= raits; - protected VarLenIntEncoderDecoder=2EVarLenIntDecoder decoder = =3D VarLenIntEncoderDecoder=2EcreateDecoder(); + protected final ITypeTr= aits nullTypeTraits; // can be null + protected final INullIntrospector = nullIntrospector; // can be null - public TypeAwareTupleWriter(ITypeTr= aits[] typeTraits) { + public TypeAwareTupleWriter(ITypeTraits[] typeTra= its, ITypeTraits nullTypeTraits, + INullIntrospector nullIntrosp= ector) { this=2EtypeTraits =3D typeTraits; + this=2EnullTyp= eTraits =3D nullTypeTraits; + this=2EnullIntrospector =3D nullIntros= pector; } @Override @@ -56,7 +61,7 @@ @Override pub= lic TypeAwareTupleReference createTupleReference() { - return new Ty= peAwareTupleReference(typeTraits); + return new TypeAwareTupleRefere= nce(typeTraits, nullTypeTraits); } @Override @@ -68,7 +73,7 @@ = public int writeTuple(ITupleReference tuple, byte[] targetBuf, int tar= getOff) { int runner =3D targetOff; int nullFlagsBytes = =3D getNullFlagsBytes(tuple); - // write null indicator bits + = // reset null indicator bits for (int i =3D 0; i < nullFlagsByte= s; i++) { targetBuf[runner++] =3D (byte) 0; } @@ -80,= 10 +85,16 @@ } } - // write data fields + = // write data fields and set null indicator bits for (int i = =3D 0; i < tuple=2EgetFieldCount(); i++) { - System=2Earraycopy(= tuple=2EgetFieldData(i), tuple=2EgetFieldStart(i), targetBuf, runner, tuple= =2EgetFieldLength(i)); - runner +=3D tuple=2EgetFieldLength(i); = + byte[] fieldData =3D tuple=2EgetFieldData(i); + int= fieldOffset =3D tuple=2EgetFieldStart(i); + int fieldLength =3D= tuple=2EgetFieldLength(i); + if (nullIntrospector !=3D null && = nullIntrospector=2EisNull(fieldData, fieldOffset, fieldLength)) { + = setNullFlag(targetBuf, targetOff, i); + } + S= ystem=2Earraycopy(fieldData, fieldOffset, targetBuf, runner, fieldLength); = + runner +=3D fieldLength; } return runner -= targetOff; @@ -93,7 +104,7 @@ public int writeTupleFields(ITupleRefer= ence tuple, int startField, int numFields, byte[] targetBuf, int targetOff)= { int runner =3D targetOff; int nullFlagsBytes =3D getNu= llFlagsBytes(numFields); - // write null indicator bits + // = reset null indicator bits for (int i =3D 0; i < nullFlagsBytes; i+= +) { targetBuf[runner++] =3D (byte) 0; } @@ -105,9 +1= 16,15 @@ } } - for (int i =3D startField; i = < startField + numFields; i++) { - System=2Earraycopy(tuple=2Ege= tFieldData(i), tuple=2EgetFieldStart(i), targetBuf, runner, tuple=2EgetFiel= dLength(i)); - runner +=3D tuple=2EgetFieldLength(i); + f= or (int i =3D startField, targetField =3D 0; i < startField + numFields; i+= +, targetField++) { + byte[] fieldData =3D tuple=2EgetFieldData(= i); + int fieldOffset =3D tuple=2EgetFieldStart(i); + = int fieldLength =3D tuple=2EgetFieldLength(i); + if (nullIntros= pector !=3D null && nullIntrospector=2EisNull(fieldData, fieldOffset, field= Length)) { + setNullFlag(targetBuf, targetOff, targetField);= + } + System=2Earraycopy(fieldData, fieldOffset, tar= getBuf, runner, fieldLength); + runner +=3D fieldLength; = } return runner - targetOff; @@ -145,8 +162,40 @@ ret= urn typeTraits; } + public ITypeTraits getNullTypeTraits() { + = return nullTypeTraits; + } + + public INullIntrospector getNullI= ntrospector() { + return nullIntrospector; + } + @Override = public int getCopySpaceRequired(ITupleReference tuple) { retur= n bytesRequired(tuple); } + + /** + * Given a field index, this= method finds its corresponding bit in the null flags section and sets it= =2E + * + * @param flags data + * @param flagsOffset start of t= he null flags data + * @param fieldIdx logical field index + */ + = private void setNullFlag(byte[] flags, int flagsOffset, int fieldIdx) { += int adjustedFieldIdx =3D getAdjustedFieldIdx(fieldIdx); + in= t flagByteIdx =3D adjustedFieldIdx / 8; + int flagBitIdx =3D 7 - (ad= justedFieldIdx % 8); + BitOperationUtils=2EsetBit(flags, flagsOffset= + flagByteIdx, (byte) flagBitIdx); + } + + /** + * Adjusts the f= ield index in case the null flags section starts with some other special-pu= rpose fields=2E + * + * @param fieldIdx logical field index + *= @return adjusted field index + */ + protected int getAdjustedFieldI= dx(int fieldIdx) { + return fieldIdx; + } } diff --git a/hyracks= -fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyrac= ks/storage/am/common/tuples/TypeAwareTupleWriterFactory=2Ejava b/hyracks-fu= llstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/= storage/am/common/tuples/TypeAwareTupleWriterFactory=2Ejava index a0fa505= =2E=2E586f243 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-com= mon/src/main/java/org/apache/hyracks/storage/am/common/tuples/TypeAwareTupl= eWriterFactory=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-co= mmon/src/main/java/org/apache/hyracks/storage/am/common/tuples/TypeAwareTup= leWriterFactory=2Ejava @@ -20,20 +20,26 @@ package org=2Eapache=2Ehyracks= =2Estorage=2Eam=2Ecommon=2Etuples; import org=2Eapache=2Ehyracks=2Eapi= =2Edataflow=2Evalue=2EITypeTraits; +import org=2Eapache=2Ehyracks=2Estorage= =2Eam=2Ecommon=2Eapi=2EINullIntrospector; import org=2Eapache=2Ehyracks=2E= storage=2Eam=2Ecommon=2Eapi=2EITreeIndexTupleWriterFactory; public class= TypeAwareTupleWriterFactory implements ITreeIndexTupleWriterFactory { - = private static final long serialVersionUID =3D 1L; - protected ITypeT= raits[] typeTraits; + private static final long serialVersionUID =3D 2L;= + protected final ITypeTraits[] typeTraits; + protected final ITypeT= raits nullTypeTraits; + protected final INullIntrospector nullIntrospect= or; - public TypeAwareTupleWriterFactory(ITypeTraits[] typeTraits) { += public TypeAwareTupleWriterFactory(ITypeTraits[] typeTraits, ITypeTrait= s nullTypeTraits, + INullIntrospector nullIntrospector) { = this=2EtypeTraits =3D typeTraits; + this=2EnullTypeTraits =3D nul= lTypeTraits; + this=2EnullIntrospector =3D nullIntrospector; } = @Override public TypeAwareTupleWriter createTupleWriter() { - = return new TypeAwareTupleWriter(typeTraits); + return new Type= AwareTupleWriter(typeTraits, nullTypeTraits, nullIntrospector); } }= diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/ma= in/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeLocal= Resource=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/sr= c/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeL= ocalResource=2Ejava index d94a5e1=2E=2Ee16baf9 100644 --- a/hyracks-fullsta= ck/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/st= orage/am/lsm/btree/dataflow/ExternalBTreeLocalResource=2Ejava +++ b/hyracks= -fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hy= racks/storage/am/lsm/btree/dataflow/ExternalBTreeLocalResource=2Ejava @@ -2= 9,6 +29,7 @@ import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIJsonSerializable;= import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIPersistedResourceRegistry; i= mport org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMetadataPageM= anagerFactory; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi= =2EINullIntrospector; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm= =2Ebtree=2Eutils=2ELSMBTreeUtil; import org=2Eapache=2Ehyracks=2Estorage= =2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallbackFactory; import org=2E= apache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationSched= ulerProvider; @@ -54,12 +55,13 @@ ILSMOperationTrackerFactory = opTrackerProvider, ILSMIOOperationCallbackFactory ioOpCallbackFactory, = ILSMPageWriteCallbackFactory pageWriteCallbackFactory, = IMetadataPageManagerFactory metadataPageManagerFactory, - ILSM= IOOperationSchedulerProvider ioSchedulerProvider, boolean durable, boolean = hasBloomFilter) { + ILSMIOOperationSchedulerProvider ioScheduler= Provider, boolean durable, boolean hasBloomFilter, + ITypeTraits= nullTypeTraits, INullIntrospector nullIntrospector) { super(typeT= raits, cmpFactories, bloomFilterKeyFields, bloomFilterFalsePositiveRate, is= Primary, path, storageManager, mergePolicyFactory, mergePo= licyProperties, filterTypeTraits, filterCmpFactories, btre= eFields, filterFields, opTrackerProvider, ioOpCallbackFactory, pageWriteCal= lbackFactory, metadataPageManagerFactory, null, ioSchedule= rProvider, durable, - NoOpCompressorDecompressorFactory=2EIN= STANCE, hasBloomFilter); + NoOpCompressorDecompressorFactory= =2EINSTANCE, hasBloomFilter, nullTypeTraits, nullIntrospector); } = private ExternalBTreeLocalResource(IPersistedResourceRegistry registry, = JsonNode json, int[] bloomFilterKeyFields, @@ -80,7 +82,7 @@ = mergePolicyFactory=2EcreateMergePolicy(mergePolicyProperties, serviceCtx= ), opTrackerProvider=2EgetOperationTracker(serviceCtx, thi= s), ioSchedulerProvider=2EgetIoScheduler(serviceCtx), ioOp= CallbackFactory, pageWriteCallbackFactory, durable, metadataPageManagerFact= ory, - serviceCtx=2EgetTracer()); + serviceCt= x=2EgetTracer(), nullTypeTraits, nullIntrospector); } @Override= diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/ma= in/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeLocal= ResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-b= tree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/Externa= lBTreeLocalResourceFactory=2Ejava index f398370=2E=2Ee4c29ba 100644 --- a/h= yracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apa= che/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeLocalResourceFactory= =2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/ma= in/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeLocal= ResourceFactory=2Ejava @@ -24,6 +24,7 @@ import org=2Eapache=2Ehyracks=2Ea= pi=2Edataflow=2Evalue=2EITypeTraits; import org=2Eapache=2Ehyracks=2Eapi= =2Eio=2EFileReference; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecomm= on=2Eapi=2EIMetadataPageManagerFactory; +import org=2Eapache=2Ehyracks=2Est= orage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; import org=2Eapache=2Ehyrac= ks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallbackFactory; i= mport org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOp= erationSchedulerProvider; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2El= sm=2Ecommon=2Eapi=2EILSMMergePolicyFactory; @@ -45,12 +46,13 @@ = IMetadataPageManagerFactory metadataPageManagerFactory, ILSM= IOOperationSchedulerProvider ioSchedulerProvider, ILSMMergePolicyFactory me= rgePolicyFactory, Map mergePolicyProperties, b= oolean durable, int[] bloomFilterKeyFields, - double bloomFilter= FalsePositiveRate, boolean isPrimary, int[] btreeFields, boolean hasBloomFi= lter) { + double bloomFilterFalsePositiveRate, boolean isPrimary= , int[] btreeFields, boolean hasBloomFilter, + ITypeTraits nullT= ypeTraits, INullIntrospector nullIntrospector) { super(storageMana= ger, typeTraits, cmpFactories, filterTypeTraits, filterCmpFactories, filter= Fields, opTrackerFactory, ioOpCallbackFactory, pageWriteCa= llbackFactory, metadataPageManagerFactory, null, ioSchedul= erProvider, mergePolicyFactory, mergePolicyProperties, durable, bloomFilter= KeyFields, bloomFilterFalsePositiveRate, isPrimary, btreeF= ields, NoOpCompressorDecompressorFactory=2EINSTANCE, - hasBl= oomFilter); + hasBloomFilter, nullTypeTraits, nullIntrospect= or); } @Override @@ -59,6 +61,6 @@ bloomFilter= FalsePositiveRate, isPrimary, fileRef=2EgetRelativePath(), storageManager, = mergePolicyFactory, mergePolicyProperties, filterTypeTrait= s, filterCmpFactories, btreeFields, bloomFilterKeyFields, = opTrackerProvider, ioOpCallbackFactory, pageWriteCallbackFactory, metadataP= ageManagerFactory, - ioSchedulerProvider, durable, hasBloomF= ilter); + ioSchedulerProvider, durable, hasBloomFilter, null= TypeTraits, nullIntrospector); } } diff --git a/hyracks-fullstack/hyr= acks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/= am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResource=2Ejava b/hyracks-= fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyr= acks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResource=2Eja= va index bf4b06c=2E=2E863ae56 100644 --- a/hyracks-fullstack/hyracks/hyrack= s-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btre= e/dataflow/ExternalBTreeWithBuddyLocalResource=2Ejava +++ b/hyracks-fullsta= ck/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/st= orage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResource=2Ejava @@ -= 29,6 +29,7 @@ import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIJsonSerializable= ; import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIPersistedResourceRegistry; = import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMetadataPage= ManagerFactory; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eap= i=2EINullIntrospector; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm= =2Ebtree=2Eutils=2ELSMBTreeUtil; import org=2Eapache=2Ehyracks=2Estorage= =2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallbackFactory; import org=2E= apache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationSched= ulerProvider; @@ -57,12 +58,13 @@ ILSMOperationTrackerFactory = opTrackerProvider, ILSMIOOperationCallbackFactory ioOpCallbackFactory, = ILSMPageWriteCallbackFactory pageWriteCallbackFactory, = IMetadataPageManagerFactory metadataPageManagerFactory, - ILSM= IOOperationSchedulerProvider ioSchedulerProvider, boolean durable, boolean = hasBloomFilter) { + ILSMIOOperationSchedulerProvider ioScheduler= Provider, boolean durable, boolean hasBloomFilter, + ITypeTraits= nullTypeTraits, INullIntrospector nullIntrospector) { super(typeT= raits, cmpFactories, buddyBtreeFields, bloomFilterFalsePositiveRate, isPrim= ary, path, storageManager, mergePolicyFactory, mergePolicy= Properties, filterTypeTraits, filterCmpFactories, btreeFields, = filterFields, opTrackerProvider, ioOpCallbackFactory, pageWriteCallbac= kFactory, metadataPageManagerFactory, null, ioSchedulerPro= vider, durable, - NoOpCompressorDecompressorFactory=2EINSTAN= CE, hasBloomFilter); + NoOpCompressorDecompressorFactory=2EI= NSTANCE, hasBloomFilter, nullTypeTraits, nullIntrospector); } p= rivate ExternalBTreeWithBuddyLocalResource(IPersistedResourceRegistry regis= try, JsonNode json, @@ -83,7 +85,7 @@ mergePolicyFactory= =2EcreateMergePolicy(mergePolicyProperties, serviceCtx), o= pTrackerProvider=2EgetOperationTracker(serviceCtx, this), ioSchedulerProvid= er=2EgetIoScheduler(serviceCtx), ioOpCallbackFactory, page= WriteCallbackFactory, bloomFilterKeyFields, durable, - metad= ataPageManagerFactory, serviceCtx=2EgetTracer()); + metadata= PageManagerFactory, serviceCtx=2EgetTracer(), nullTypeTraits, nullIntrospec= tor); } @Override diff --git a/hyracks-fullstack/hyracks/hyrack= s-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btre= e/dataflow/ExternalBTreeWithBuddyLocalResourceFactory=2Ejava b/hyracks-full= stack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks= /storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResourceFactory= =2Ejava index 91ebee1=2E=2E39c8aed 100644 --- a/hyracks-fullstack/hyracks/h= yracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm= /btree/dataflow/ExternalBTreeWithBuddyLocalResourceFactory=2Ejava +++ b/hyr= acks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apach= e/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResource= Factory=2Ejava @@ -24,6 +24,7 @@ import org=2Eapache=2Ehyracks=2Eapi=2Edat= aflow=2Evalue=2EITypeTraits; import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EFi= leReference; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi= =2EIMetadataPageManagerFactory; +import org=2Eapache=2Ehyracks=2Estorage=2E= am=2Ecommon=2Eapi=2EINullIntrospector; import org=2Eapache=2Ehyracks=2Esto= rage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallbackFactory; import or= g=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationS= chedulerProvider; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecom= mon=2Eapi=2EILSMMergePolicyFactory; @@ -45,12 +46,13 @@ IMetad= ataPageManagerFactory metadataPageManagerFactory, ILSMIOOperat= ionSchedulerProvider ioSchedulerProvider, ILSMMergePolicyFactory mergePolic= yFactory, Map mergePolicyProperties, boolean d= urable, int[] buddyBtreeFields, - double bloomFilterFalsePositiv= eRate, boolean isPrimary, int[] btreeFields, boolean hasBloomFilter) { + = double bloomFilterFalsePositiveRate, boolean isPrimary, int[] btre= eFields, boolean hasBloomFilter, + ITypeTraits nullTypeTraits, I= NullIntrospector nullIntrospector) { super(storageManager, typeTra= its, cmpFactories, filterTypeTraits, filterCmpFactories, filterFields, = opTrackerProvider, ioOpCallbackFactory, pageWriteCallbackFacto= ry, metadataPageManagerFactory, null, ioSchedulerProvider,= mergePolicyFactory, mergePolicyProperties, durable, buddyBtreeFields, = bloomFilterFalsePositiveRate, isPrimary, btreeFields, NoOpComp= ressorDecompressorFactory=2EINSTANCE, - hasBloomFilter); + = hasBloomFilter, nullTypeTraits, nullIntrospector); } = @Override @@ -59,6 +61,6 @@ bloomFilterFalsePositiveRa= te, isPrimary, fileRef=2EgetRelativePath(), storageManager, mergePolicyFact= ory, mergePolicyProperties, filterTypeTraits, filterCmpFac= tories, btreeFields, filterFields, opTrackerProvider, ioOp= CallbackFactory, pageWriteCallbackFactory, metadataPageManagerFactory, - = ioSchedulerProvider, durable, hasBloomFilter); + = ioSchedulerProvider, durable, hasBloomFilter, nullTypeTraits, nullIntrosp= ector); } } diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am= -lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/L= SMBTreeLocalResource=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-= lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LS= MBTreeLocalResource=2Ejava index 6098a1d=2E=2E656ae02 100644 --- a/hyracks-= fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyr= acks/storage/am/lsm/btree/dataflow/LSMBTreeLocalResource=2Ejava +++ b/hyrac= ks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/= hyracks/storage/am/lsm/btree/dataflow/LSMBTreeLocalResource=2Ejava @@ -31,6= +31,7 @@ import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIJsonSerializable; i= mport org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIPersistedResourceRegistry; impo= rt org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMetadataPageMana= gerFactory; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2E= INullIntrospector; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ebt= ree=2Eutils=2ELSMBTreeUtil; import org=2Eapache=2Ehyracks=2Estorage=2Eam= =2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallbackFactory; import org=2Eapach= e=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationSchedulerP= rovider; @@ -68,10 +69,12 @@ ILSMPageWriteCallbackFactory page= WriteCallbackFactory, IMetadataPageManagerFactory metadataPage= ManagerFactory, IVirtualBufferCacheProvider vbcProvider, ILSMI= OOperationSchedulerProvider ioSchedulerProvider, boolean durable, - = ICompressorDecompressorFactory compressorDecompressorFactory, boolean h= asBloomFilter) { + ICompressorDecompressorFactory compressorDeco= mpressorFactory, boolean hasBloomFilter, + ITypeTraits nullTypeT= raits, INullIntrospector nullIntrospector) { super(path, storageMa= nager, typeTraits, cmpFactories, filterTypeTraits, filterCmpFactories, filt= erFields, opTrackerProvider, ioOpCallbackFactory, pageWrit= eCallbackFactory, metadataPageManagerFactory, - vbcProvider,= ioSchedulerProvider, mergePolicyFactory, mergePolicyProperties, durable); = + vbcProvider, ioSchedulerProvider, mergePolicyFactory, merg= ePolicyProperties, durable, nullTypeTraits, + nullIntrospect= or); this=2EbloomFilterKeyFields =3D bloomFilterKeyFields; = this=2EbloomFilterFalsePositiveRate =3D bloomFilterFalsePositiveRate; = this=2EisPrimary =3D isPrimary; @@ -108,7 +111,7 @@ = opTrackerProvider=2EgetOperationTracker(serviceCtx, this), ioSchedulerProvi= der=2EgetIoScheduler(serviceCtx), ioOpCallbackFactory, pag= eWriteCallbackFactory, isPrimary, filterTypeTraits, filterCmpFactories, = btreeFields, filterFields, durable, metadataPageManagerFactor= y, updateAware, serviceCtx=2EgetTracer(), - compressorDecomp= ressorFactory, hasBloomFilter); + compressorDecompressorFact= ory, hasBloomFilter, nullTypeTraits, nullIntrospector); } @Over= ride diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/sr= c/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSMBTreeLocalR= esourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-bt= ree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSMBTree= LocalResourceFactory=2Ejava index a2c16b8=2E=2Ee926a48 100644 --- a/hyracks= -fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hy= racks/storage/am/lsm/btree/dataflow/LSMBTreeLocalResourceFactory=2Ejava +++= b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org= /apache/hyracks/storage/am/lsm/btree/dataflow/LSMBTreeLocalResourceFactory= =2Ejava @@ -26,6 +26,7 @@ import org=2Eapache=2Ehyracks=2Eapi=2Edataflow= =2Evalue=2EITypeTraits; import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EFileRef= erence; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMet= adataPageManagerFactory; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Eco= mmon=2Eapi=2EINullIntrospector; import org=2Eapache=2Ehyracks=2Estorage=2E= am=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallbackFactory; import org=2Eapa= che=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationSchedule= rProvider; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Ea= pi=2EILSMMergePolicyFactory; @@ -57,10 +58,12 @@ ILSMIOOperati= onSchedulerProvider ioSchedulerProvider, ILSMMergePolicyFactory mergePolicy= Factory, Map mergePolicyProperties, boolean du= rable, int[] bloomFilterKeyFields, double bloomFilterFalsePosi= tiveRate, boolean isPrimary, int[] btreeFields, - ICompressorDec= ompressorFactory compressorDecompressorFactory, boolean hasBloomFilter) { += ICompressorDecompressorFactory compressorDecompressorFactory, b= oolean hasBloomFilter, + ITypeTraits nullTypeTraits, INullIntros= pector nullIntrospector) { super(storageManager, typeTraits, cmpFa= ctories, filterTypeTraits, filterCmpFactories, filterFields, = opTrackerFactory, ioOpCallbackFactory, pageWriteCallbackFactory, metadat= aPageManagerFactory, - vbcProvider, ioSchedulerProvider, mer= gePolicyFactory, mergePolicyProperties, durable); + vbcProvi= der, ioSchedulerProvider, mergePolicyFactory, mergePolicyProperties, durabl= e, nullTypeTraits, + nullIntrospector); this=2EhasB= loomFilter =3D hasBloomFilter; this=2EbloomFilterKeyFields =3D blo= omFilterKeyFields; this=2EbloomFilterFalsePositiveRate =3D bloomFi= lterFalsePositiveRate; @@ -75,7 +78,7 @@ isPrimary, fileRe= f=2EgetRelativePath(), storageManager, mergePolicyFactory, mergePolicyPrope= rties, filterTypeTraits, filterCmpFactories, btreeFields, = filterFields, opTrackerProvider, ioOpCallbackFactory, page= WriteCallbackFactory, metadataPageManagerFactory, vbcProvider, ioSchedulerP= rovider, durable, - compressorDecompressorFactory, hasBloomF= ilter); + compressorDecompressorFactory, hasBloomFilter, nul= lTypeTraits, nullIntrospector); } private void readObject(java= =2Eio=2EObjectInputStream in) throws IOException, ClassNotFoundException { = diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/mai= n/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeCopyTupleWrit= er=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main= /java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeCopyTupleWrite= r=2Ejava index 7c924b5=2E=2E76916ec 100644 --- a/hyracks-fullstack/hyracks/= hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/ls= m/btree/tuples/LSMBTreeCopyTupleWriter=2Ejava +++ b/hyracks-fullstack/hyrac= ks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am= /lsm/btree/tuples/LSMBTreeCopyTupleWriter=2Ejava @@ -21,11 +21,14 @@ imp= ort org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits; import = org=2Eapache=2Ehyracks=2Edataflow=2Ecommon=2Edata=2Eaccessors=2EITupleRefer= ence; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullI= ntrospector; public class LSMBTreeCopyTupleWriter extends LSMBTreeTupleW= riter { - public LSMBTreeCopyTupleWriter(ITypeTraits[] typeTraits, int n= umKeyFields, boolean updateAware) { + + public LSMBTreeCopyTupleWriter(I= TypeTraits[] typeTraits, int numKeyFields, boolean updateAware, + = ITypeTraits nullTypeTraits, INullIntrospector nullIntrospector) { = // Third parameter is never used locally, just give false=2E - sup= er(typeTraits, numKeyFields, false, updateAware); + super(typeTraits= , numKeyFields, false, updateAware, nullTypeTraits, nullIntrospector); = } @Override diff --git a/hyracks-fullstack/hyracks/hyracks-storage-= am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/L= SMBTreeCopyTupleWriterFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-st= orage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tu= ples/LSMBTreeCopyTupleWriterFactory=2Ejava index 02cd900=2E=2Edd43ef4 10064= 4 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/jav= a/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeCopyTupleWriterFac= tory=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/sr= c/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeCopyTupl= eWriterFactory=2Ejava @@ -22,18 +22,20 @@ import org=2Eapache=2Ehyracks=2E= api=2Edataflow=2Evalue=2EITypeTraits; import org=2Eapache=2Ehyracks=2Estor= age=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriter; import org=2Eapache= =2Ehyracks=2Estorage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriterFacto= ry; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullInt= rospector; public class LSMBTreeCopyTupleWriterFactory extends BTreeType= AwareTupleWriterFactory { private static final long serialVersionUID = =3D 1L; private final int numKeyFields; - public LSMBTreeCopyTupl= eWriterFactory(ITypeTraits[] typeTraits, int numKeyFields, boolean updateAw= are) { - super(typeTraits, updateAware); + public LSMBTreeCopyTup= leWriterFactory(ITypeTraits[] typeTraits, int numKeyFields, boolean updateA= ware, + ITypeTraits nullTypeTraits, INullIntrospector nullIntros= pector) { + super(typeTraits, updateAware, nullTypeTraits, nullIntro= spector); this=2EnumKeyFields =3D numKeyFields; } @Ove= rride public BTreeTypeAwareTupleWriter createTupleWriter() { - = return new LSMBTreeCopyTupleWriter(typeTraits, numKeyFields, updateAware); = + return new LSMBTreeCopyTupleWriter(typeTraits, numKeyFields, updat= eAware, nullTypeTraits, nullIntrospector); } } diff --git a/hyracks-f= ullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyra= cks/storage/am/lsm/btree/tuples/LSMBTreeTupleReference=2Ejava b/hyracks-ful= lstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyrack= s/storage/am/lsm/btree/tuples/LSMBTreeTupleReference=2Ejava index 2a5e232= =2E=2Ee5c365a 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm= -btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTre= eTupleReference=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-l= sm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBT= reeTupleReference=2Ejava @@ -33,8 +33,9 @@ private boolean resetFieldC= ount =3D false; private final int numKeyFields; - public LSMBTree= TupleReference(ITypeTraits[] typeTraits, int numKeyFields, boolean updateAw= are) { - super(typeTraits, updateAware); + public LSMBTreeTupleRe= ference(ITypeTraits[] typeTraits, int numKeyFields, boolean updateAware, + = ITypeTraits nullTypeTraits) { + super(typeTraits, updateA= ware, nullTypeTraits); this=2EnumKeyFields =3D numKeyFields; = } @@ -94,4 +95,10 @@ public int getTupleStart() { return tu= pleStartOff; } + + @Override + protected int getAdjustedFieldIdx= (int fieldIdx) { + // 2 bits when update-aware: 1 for antimatter and= 1 for the update, otherwise, only 1 bit for the antimatter + return= updateAware ? fieldIdx + 2 : fieldIdx + 1; + } } diff --git a/hyracks-= fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyr= acks/storage/am/lsm/btree/tuples/LSMBTreeTupleWriter=2Ejava b/hyracks-fulls= tack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/= storage/am/lsm/btree/tuples/LSMBTreeTupleWriter=2Ejava index 09ced10=2E=2Eb= cbf93f 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/= src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTupleW= riter=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/s= rc/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTupleWr= iter=2Ejava @@ -25,6 +25,7 @@ import org=2Eapache=2Ehyracks=2Eapi=2Edatafl= ow=2Evalue=2EITypeTraits; import org=2Eapache=2Ehyracks=2Edataflow=2Ecommo= n=2Edata=2Eaccessors=2EITupleReference; import org=2Eapache=2Ehyracks=2Est= orage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriter; +import org=2Eapac= he=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; import org= =2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eutil=2EBitOperationUtils; im= port org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMTreeT= upleWriter; @@ -33,8 +34,9 @@ private boolean isAntimatter; pri= vate final int numKeyFields; - public LSMBTreeTupleWriter(ITypeTraits[= ] typeTraits, int numKeyFields, boolean isAntimatter, boolean updateAware) = { - super(typeTraits, updateAware); + public LSMBTreeTupleWriter(= ITypeTraits[] typeTraits, int numKeyFields, boolean isAntimatter, boolean u= pdateAware, + ITypeTraits nullTypeTraits, INullIntrospector null= Introspector) { + super(typeTraits, updateAware, nullTypeTraits, nul= lIntrospector); this=2EnumKeyFields =3D numKeyFields; thi= s=2EisAntimatter =3D isAntimatter; } @@ -56,7 +58,7 @@ @Overrid= e public LSMBTreeTupleReference createTupleReference() { - retu= rn new LSMBTreeTupleReference(typeTraits, numKeyFields, updateAware); + = return new LSMBTreeTupleReference(typeTraits, numKeyFields, updateAware= , nullTypeTraits); } @Override @@ -83,6 +85,7 @@ public in= t writeTuple(ITupleReference tuple, byte[] targetBuf, int targetOff) { = int bytesWritten =3D -1; if (isAntimatter) { + // = TODO(ali): check this with secondary primary-key index bytesWr= itten =3D super=2EwriteTupleFields(tuple, 0, numKeyFields, targetBuf, targe= tOff); // Set antimatter bit to 1=2E BitOperation= Utils=2EsetBit(targetBuf, targetOff, ANTIMATTER_BIT_OFFSET); @@ -100,4 +103= ,10 @@ public void setAntimatter(boolean isDelete) { this=2Ei= sAntimatter =3D isDelete; } + + @Override + protected int getAdj= ustedFieldIdx(int fieldIdx) { + // 2 bits when update-aware: 1 for a= ntimatter and 1 for the update, otherwise, only 1 bit for the antimatter + = return updateAware ? fieldIdx + 2 : fieldIdx + 1; + } } diff --g= it a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/o= rg/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTupleWriterFactory=2E= java b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java= /org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTupleWriterFactory= =2Ejava index ad4f6eb=2E=2E03bce23 100644 --- a/hyracks-fullstack/hyracks/h= yracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm= /btree/tuples/LSMBTreeTupleWriterFactory=2Ejava +++ b/hyracks-fullstack/hyr= acks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/= am/lsm/btree/tuples/LSMBTreeTupleWriterFactory=2Ejava @@ -22,6 +22,7 @@ im= port org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits; import= org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTup= leWriter; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Etuples=2E= BTreeTypeAwareTupleWriterFactory; +import org=2Eapache=2Ehyracks=2Estorage= =2Eam=2Ecommon=2Eapi=2EINullIntrospector; public class LSMBTreeTupleWrit= erFactory extends BTreeTypeAwareTupleWriterFactory { @@ -30,15 +31,16 @@ = private final boolean isAntimatter; public LSMBTreeTupleWriterF= actory(ITypeTraits[] typeTraits, int numKeyFields, boolean isAntimatter, - = boolean updateAware) { - super(typeTraits, updateAware); = + boolean updateAware, ITypeTraits nullTypeTraits, INullIntrospe= ctor nullIntrospector) { + super(typeTraits, updateAware, nullTypeTr= aits, nullIntrospector); this=2EnumKeyFields =3D numKeyFields; = this=2EisAntimatter =3D isAntimatter; } @Override pu= blic BTreeTypeAwareTupleWriter createTupleWriter() { - return new LS= MBTreeTupleWriter(typeTraits, numKeyFields, isAntimatter, updateAware); + = return new LSMBTreeTupleWriter(typeTraits, numKeyFields, isAntimatter= , updateAware, nullTypeTraits, + nullIntrospector); } = } diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src= /main/java/org/apache/hyracks/storage/am/lsm/btree/utils/LSMBTreeUtil=2Ejav= a b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/or= g/apache/hyracks/storage/am/lsm/btree/utils/LSMBTreeUtil=2Ejava index 021f4= 16=2E=2E018c0d3 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-l= sm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/utils/LSMBTr= eeUtil=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/= src/main/java/org/apache/hyracks/storage/am/lsm/btree/utils/LSMBTreeUtil=2E= java @@ -34,6 +34,7 @@ import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtre= e=2Eimpls=2EDiskBTree; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtre= e=2Etuples=2EBTreeTypeAwareTupleWriterFactory; import org=2Eapache=2Ehyrac= ks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMetadataPageManagerFactory; +import org= =2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; imp= ort org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EITreeIndexFrameF= actory; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Etuples=2ET= ypeAwareTupleWriterFactory; import org=2Eapache=2Ehyracks=2Estorage=2Eam= =2Elsm=2Ebtree=2Eimpls=2EExternalBTree; @@ -76,15 +77,16 @@ bo= olean needKeyDupCheck, ITypeTraits[] filterTypeTraits, IBinaryComparatorFac= tory[] filterCmpFactories, int[] btreeFields, int[] filterFiel= ds, boolean durable, IMetadataPageManagerFactory freePageManagerFactory, = boolean updateAware, ITracer tracer, ICompressorDecompressorFact= ory compressorDecompressorFactory, - boolean hasBloomFilter) thr= ows HyracksDataException { - LSMBTreeTupleWriterFactory insertTupleW= riterFactory =3D - new LSMBTreeTupleWriterFactory(typeTraits= , cmpFactories=2Elength, false, updateAware); - LSMBTreeTupleWriterF= actory deleteTupleWriterFactory =3D - new LSMBTreeTupleWrite= rFactory(typeTraits, cmpFactories=2Elength, true, updateAware); - LS= MBTreeCopyTupleWriterFactory copyTupleWriterFactory =3D - ne= w LSMBTreeCopyTupleWriterFactory(typeTraits, cmpFactories=2Elength, updateA= ware); - LSMBTreeTupleWriterFactory bulkLoadTupleWriterFactory =3D -= new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories=2El= ength, false, updateAware); + boolean hasBloomFilter, ITypeTrait= s nullTypeTraits, INullIntrospector nullIntrospector) + throws H= yracksDataException { + LSMBTreeTupleWriterFactory insertTupleWriter= Factory =3D new LSMBTreeTupleWriterFactory(typeTraits, + cmp= Factories=2Elength, false, updateAware, nullTypeTraits, nullIntrospector); = + LSMBTreeTupleWriterFactory deleteTupleWriterFactory =3D new LSMBTr= eeTupleWriterFactory(typeTraits, + cmpFactories=2Elength, tr= ue, updateAware, nullTypeTraits, nullIntrospector); + LSMBTreeCopyTu= pleWriterFactory copyTupleWriterFactory =3D new LSMBTreeCopyTupleWriterFact= ory(typeTraits, + cmpFactories=2Elength, updateAware, nullTy= peTraits, nullIntrospector); + LSMBTreeTupleWriterFactory bulkLoadTu= pleWriterFactory =3D new LSMBTreeTupleWriterFactory(typeTraits, + = cmpFactories=2Elength, false, updateAware, nullTypeTraits, nullIntros= pector); ITreeIndexFrameFactory insertLeafFrameFactory =3D new B= TreeNSMLeafFrameFactory(insertTupleWriterFactory); ITreeIndexFrame= Factory copyTupleLeafFrameFactory =3D new BTreeNSMLeafFrameFactory(copyTupl= eWriterFactory); @@ -103,7 +105,8 @@ LSMComponentFilterFrameFactor= y filterFrameFactory =3D null; LSMComponentFilterManager filterMan= ager =3D null; if (filterCmpFactories !=3D null) { - Ty= peAwareTupleWriterFactory filterTupleWriterFactory =3D new TypeAwareTupleWr= iterFactory(filterTypeTraits); + TypeAwareTupleWriterFactory fil= terTupleWriterFactory =3D + new TypeAwareTupleWriterFact= ory(filterTypeTraits, nullTypeTraits, nullIntrospector); filte= rHelper =3D new ComponentFilterHelper(filterTupleWriterFactory, filterCmpFa= ctories); filterFrameFactory =3D new LSMComponentFilterFrameFa= ctory(filterTupleWriterFactory); filterManager =3D new LSMComp= onentFilterManager(filterFrameFactory); @@ -136,21 +139,21 @@ = int[] bloomFilterKeyFields, double bloomFilterFalsePositiveRate, ILSMMergeP= olicy mergePolicy, ILSMOperationTracker opTracker, ILSMIOOpera= tionScheduler ioScheduler, ILSMIOOperationCallbackFactory ioOp= CallbackFactory, ILSMPageWriteCallbackFactory pageWriteCallbackFactory, - = boolean durable, IMetadataPageManagerFactory freePageManagerFacto= ry, ITracer tracer) - throws HyracksDataException { - LSM= BTreeTupleWriterFactory insertTupleWriterFactory =3D - new L= SMBTreeTupleWriterFactory(typeTraits, cmpFactories=2Elength, false, false);= - LSMBTreeTupleWriterFactory deleteTupleWriterFactory =3D - = new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories=2Elength, t= rue, false); - LSMBTreeCopyTupleWriterFactory copyTupleWriterFactory= =3D - new LSMBTreeCopyTupleWriterFactory(typeTraits, cmpFac= tories=2Elength, false); + boolean durable, IMetadataPageManager= Factory freePageManagerFactory, ITracer tracer, + ITypeTraits nu= llTypeTraits, INullIntrospector nullIntrospector) throws HyracksDataExcepti= on { + LSMBTreeTupleWriterFactory insertTupleWriterFactory =3D new L= SMBTreeTupleWriterFactory(typeTraits, + cmpFactories=2Elengt= h, false, false, nullTypeTraits, nullIntrospector); + LSMBTreeTupleW= riterFactory deleteTupleWriterFactory =3D new LSMBTreeTupleWriterFactory(ty= peTraits, + cmpFactories=2Elength, true, false, nullTypeTrai= ts, nullIntrospector); + LSMBTreeCopyTupleWriterFactory copyTupleWri= terFactory =3D new LSMBTreeCopyTupleWriterFactory(typeTraits, + = cmpFactories=2Elength, false, nullTypeTraits, nullIntrospector); = ITreeIndexFrameFactory insertLeafFrameFactory =3D new BTreeNSMLeafFrameF= actory(insertTupleWriterFactory); ITreeIndexFrameFactory copyTuple= LeafFrameFactory =3D new BTreeNSMLeafFrameFactory(copyTupleWriterFactory); = ITreeIndexFrameFactory deleteLeafFrameFactory =3D new BTreeNSMLeaf= FrameFactory(deleteTupleWriterFactory); ITreeIndexFrameFactory int= eriorFrameFactory =3D new BTreeNSMInteriorFrameFactory(insertTupleWriterFac= tory); // This is the tuple writer that can do both inserts and de= letes - LSMBTreeTupleWriterFactory transactionTupleWriterFactory =3D= - new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories= =2Elength, false, false); + LSMBTreeTupleWriterFactory transactionTu= pleWriterFactory =3D new LSMBTreeTupleWriterFactory(typeTraits, + = cmpFactories=2Elength, false, false, nullTypeTraits, nullIntrospector= ); // This is the leaf frame factory for transaction components si= nce it // can be used for both inserts and deletes ITreeI= ndexFrameFactory transactionLeafFrameFactory =3D @@ -185,7 +188,8 @@ = double bloomFilterFalsePositiveRate, ILSMMergePolicy mergePolicy, IL= SMOperationTracker opTracker, ILSMIOOperationScheduler ioSched= uler, ILSMIOOperationCallbackFactory ioOpCallbackFactory, ILSM= PageWriteCallbackFactory pageWriteCallbackFactory, int[] buddyBTreeFields, = boolean durable, - IMetadataPageManagerFactory freePageManagerFa= ctory, ITracer tracer) throws HyracksDataException { + IMetadata= PageManagerFactory freePageManagerFactory, ITracer tracer, ITypeTraits null= TypeTraits, + INullIntrospector nullIntrospector) throws Hyracks= DataException { ITypeTraits[] buddyBtreeTypeTraits =3D new ITypeTr= aits[buddyBTreeFields=2Elength]; IBinaryComparatorFactory[] buddyB= treeCmpFactories =3D new IBinaryComparatorFactory[buddyBTreeFields=2Elength= ]; for (int i =3D 0; i < buddyBtreeTypeTraits=2Elength; i++) { @@ = -193,15 +197,15 @@ buddyBtreeCmpFactories[i] =3D cmpFactories[= buddyBTreeFields[i]]; } BTreeTypeAwareTupleWriterFactory = buddyBtreeTupleWriterFactory =3D - new BTreeTypeAwareTupleWr= iterFactory(buddyBtreeTypeTraits, false); + new BTreeTypeAwa= reTupleWriterFactory(buddyBtreeTypeTraits, false, nullTypeTraits, nullIntro= spector); ITreeIndexFrameFactory buddyBtreeInteriorFrameFactory = =3D new BTreeNSMInteriorFrameFactory(buddyBtreeTupleWriter= Factory); ITreeIndexFrameFactory buddyBtreeLeafFrameFactory =3D ne= w BTreeNSMLeafFrameFactory(buddyBtreeTupleWriterFactory); - LSMBTr= eeTupleWriterFactory insertTupleWriterFactory =3D - new LSMB= TreeTupleWriterFactory(typeTraits, cmpFactories=2Elength, false, false); - = LSMBTreeCopyTupleWriterFactory copyTupleWriterFactory =3D - = new LSMBTreeCopyTupleWriterFactory(typeTraits, cmpFactories=2Elength= , false); + LSMBTreeTupleWriterFactory insertTupleWriterFactory =3D = new LSMBTreeTupleWriterFactory(typeTraits, + cmpFactories=2E= length, false, false, nullTypeTraits, nullIntrospector); + LSMBTreeC= opyTupleWriterFactory copyTupleWriterFactory =3D new LSMBTreeCopyTupleWrite= rFactory(typeTraits, + cmpFactories=2Elength, false, nullTyp= eTraits, nullIntrospector); ITreeIndexFrameFactory insertLeafFrame= Factory =3D new BTreeNSMLeafFrameFactory(insertTupleWriterFactory); = ITreeIndexFrameFactory copyTupleLeafFrameFactory =3D new BTreeNSMLeafFram= eFactory(copyTupleWriterFactory); ITreeIndexFrameFactory interiorF= rameFactory =3D new BTreeNSMInteriorFrameFactory(insertTupleWriterFactory);= diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/m= ain/java/org/apache/hyracks/storage/am/lsm/common/dataflow/LsmResource=2Eja= va b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/= org/apache/hyracks/storage/am/lsm/common/dataflow/LsmResource=2Ejava index = fbd74dc=2E=2Ed905a0a 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage= -am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/dataf= low/LsmResource=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-l= sm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/dataflow/L= smResource=2Ejava @@ -27,6 +27,7 @@ import org=2Eapache=2Ehyracks=2Eapi=2E= exceptions=2EHyracksDataException; import org=2Eapache=2Ehyracks=2Eapi=2Ei= o=2EIPersistedResourceRegistry; import org=2Eapache=2Ehyracks=2Estorage=2E= am=2Ecommon=2Eapi=2EIMetadataPageManagerFactory; +import org=2Eapache=2Ehyr= acks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; import org=2Eapach= e=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallbackFa= ctory; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi= =2EILSMIOOperationSchedulerProvider; import org=2Eapache=2Ehyracks=2Estora= ge=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMMergePolicyFactory; @@ -49,7 +50,7 @@ = */ public abstract class LsmResource implements IResource { - private= static final long serialVersionUID =3D 1L; + private static final long = serialVersionUID =3D 2L; protected static final ObjectMapper OBJECT_MA= PPER =3D new ObjectMapper(); protected String path; protected fin= al IStorageManager storageManager; @@ -67,6 +68,8 @@ protected final I= LSMMergePolicyFactory mergePolicyFactory; protected final Map mergePolicyProperties; protected final boolean durable; + p= rotected final ITypeTraits nullTypeTraits; + protected final INullIntros= pector nullIntrospector; public LsmResource(String path, IStorageMan= ager storageManager, ITypeTraits[] typeTraits, IBinaryComparat= orFactory[] cmpFactories, ITypeTraits[] filterTypeTraits, @@ -75,7 +78,8 @@= ILSMPageWriteCallbackFactory pageWriteCallbackFactory, = IMetadataPageManagerFactory metadataPageManagerFactory, IVirtualBuff= erCacheProvider vbcProvider, ILSMIOOperationSchedulerProvider = ioSchedulerProvider, ILSMMergePolicyFactory mergePolicyFactory, - = Map mergePolicyProperties, boolean durable) { + = Map mergePolicyProperties, boolean durable, ITypeTraits n= ullTypeTraits, + INullIntrospector nullIntrospector) { = this=2Epath =3D path; this=2EstorageManager =3D storageManager; = this=2EtypeTraits =3D typeTraits; @@ -92,6 +96,8 @@ this=2E= mergePolicyFactory =3D mergePolicyFactory; this=2EmergePolicyPrope= rties =3D mergePolicyProperties; this=2Edurable =3D durable; + = this=2EnullTypeTraits =3D nullTypeTraits; + this=2EnullIntrospec= tor =3D nullIntrospector; } protected LsmResource(IPersistedRes= ourceRegistry registry, JsonNode json) throws HyracksDataException { @@ -15= 6,6 +162,16 @@ mergePolicyFactory =3D (ILSMMergePolicyFactory) reg= istry=2Edeserialize(json=2Eget("mergePolicyFactory")); mergePolicy= Properties =3D OBJECT_MAPPER=2EconvertValue(json=2Eget("mergePolicyProperti= es"), Map=2Eclass); durable =3D json=2Eget("durable")=2EasBoolean(= ); + if (json=2EhasNonNull("nullTypeTraits")) { + nullTyp= eTraits =3D (ITypeTraits) registry=2Edeserialize(json=2Eget("nullTypeTraits= ")); + } else { + nullTypeTraits =3D null; + } + = if (json=2EhasNonNull("nullIntrospector")) { + nullIntrosp= ector =3D (INullIntrospector) registry=2Edeserialize(json=2Eget("nullIntros= pector")); + } else { + nullIntrospector =3D null; + = } } protected void appendToJson(final ObjectNode json, IPers= istedResourceRegistry registry) @@ -206,6 +222,16 @@ json=2Eset("m= ergePolicyFactory", mergePolicyFactory=2EtoJson(registry)); json= =2EputPOJO("mergePolicyProperties", mergePolicyProperties); json= =2Eput("durable", durable); + if (nullTypeTraits !=3D null) { + = json=2Eset("nullTypeTraits", nullTypeTraits=2EtoJson(registry)); + = } else { + json=2Eset("nullTypeTraits", null); + } = + if (nullIntrospector !=3D null) { + json=2Eset("nullInt= rospector", nullIntrospector=2EtoJson(registry)); + } else { + = json=2Eset("nullIntrospector", null); + } } @Overr= ide diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/sr= c/main/java/org/apache/hyracks/storage/am/lsm/common/dataflow/LsmResourceFa= ctory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/= main/java/org/apache/hyracks/storage/am/lsm/common/dataflow/LsmResourceFact= ory=2Ejava index a138f01=2E=2Ee97a779 100644 --- a/hyracks-fullstack/hyrack= s/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am= /lsm/common/dataflow/LsmResourceFactory=2Ejava +++ b/hyracks-fullstack/hyra= cks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/= am/lsm/common/dataflow/LsmResourceFactory=2Ejava @@ -23,6 +23,7 @@ import = org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EIBinaryComparatorFactory;= import org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits; im= port org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMetadataPageMa= nagerFactory; +import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi= =2EINullIntrospector; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm= =2Ecommon=2Eapi=2EILSMIOOperationCallbackFactory; import org=2Eapache=2Ehy= racks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationSchedulerProvide= r; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILS= MMergePolicyFactory; @@ -34,7 +35,7 @@ public abstract class LsmResource= Factory implements IResourceFactory { - private static final long seri= alVersionUID =3D 1L; + private static final long serialVersionUID =3D 2L= ; protected final IStorageManager storageManager; protected final= ITypeTraits[] typeTraits; protected final IBinaryComparatorFactory[] = cmpFactories; @@ -50,6 +51,8 @@ protected final ILSMMergePolicyFactory= mergePolicyFactory; protected final Map mergePolicyPr= operties; protected final boolean durable; + protected final ITypeT= raits nullTypeTraits; + protected final INullIntrospector nullIntrospect= or; public LsmResourceFactory(IStorageManager storageManager, ITypeT= raits[] typeTraits, IBinaryComparatorFactory[] cmpFactories, I= TypeTraits[] filterTypeTraits, @@ -58,7 +61,8 @@ ILSMPageWrite= CallbackFactory pageWriteCallbackFactory, IMetadataPageManager= Factory metadataPageManagerFactory, IVirtualBufferCacheProvider vbcProvider= , ILSMIOOperationSchedulerProvider ioSchedulerProvider, ILSMMe= rgePolicyFactory mergePolicyFactory, - Map merge= PolicyProperties, boolean durable) { + Map merge= PolicyProperties, boolean durable, ITypeTraits nullTypeTraits, + = INullIntrospector nullIntrospector) { this=2EstorageManager =3D s= torageManager; this=2EtypeTraits =3D typeTraits; this=2Ec= mpFactories =3D cmpFactories; @@ -74,5 +78,7 @@ this=2EmergePolicy= Factory =3D mergePolicyFactory; this=2EmergePolicyProperties =3D m= ergePolicyProperties; this=2Edurable =3D durable; + this=2E= nullTypeTraits =3D nullTypeTraits; + this=2EnullIntrospector =3D nul= lIntrospector; } } diff --git a/hyracks-fullstack/hyracks/hyracks-sto= rage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/i= nvertedindex/dataflow/LSMInvertedIndexLocalResource=2Ejava b/hyracks-fullst= ack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/h= yracks/storage/am/lsm/invertedindex/dataflow/LSMInvertedIndexLocalResource= =2Ejava index 0af1f3f=2E=2E8550385 100644 --- a/hyracks-fullstack/hyracks/h= yracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storag= e/am/lsm/invertedindex/dataflow/LSMInvertedIndexLocalResource=2Ejava +++ b/= hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/jav= a/org/apache/hyracks/storage/am/lsm/invertedindex/dataflow/LSMInvertedIndex= LocalResource=2Ejava @@ -94,8 +94,7 @@ super(path, storageManage= r, typeTraits, cmpFactories, filterTypeTraits, filterCmpFactories, filterFi= elds, opTrackerProvider, ioOpCallbackFactory, pageWriteCal= lbackFactory, metadataPageManagerFactory, - vbcProvider, ioS= chedulerProvider, mergePolicyFactory, mergePolicyProperties, durable); - + = vbcProvider, ioSchedulerProvider, mergePolicyFactory, mergeP= olicyProperties, durable, null, null); this=2EtokenTypeTraits =3D = tokenTypeTraits; this=2EtokenCmpFactories =3D tokenCmpFactories; = this=2EtokenizerFactory =3D tokenizerFactory; diff --git a/hyracks-= fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/ap= ache/hyracks/storage/am/lsm/invertedindex/dataflow/LSMInvertedIndexLocalRes= ourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inve= rtedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/dat= aflow/LSMInvertedIndexLocalResourceFactory=2Ejava index 09c9699=2E=2E79c612= 3 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedind= ex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/dataflow/L= SMInvertedIndexLocalResourceFactory=2Ejava +++ b/hyracks-fullstack/hyracks/= hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/stora= ge/am/lsm/invertedindex/dataflow/LSMInvertedIndexLocalResourceFactory=2Ejav= a @@ -65,7 +65,7 @@ int[] invertedIndexFieldsForNonBulkLoadOps= , double bloomFilterFalsePositiveRate) { super(storageManager, typ= eTraits, cmpFactories, filterTypeTraits, filterCmpFactories, filterFields, = opTrackerFactory, ioOpCallbackFactory, pageWriteCallbackFa= ctory, metadataPageManagerFactory, - vbcProvider, ioSchedule= rProvider, mergePolicyFactory, mergePolicyProperties, durable); + = vbcProvider, ioSchedulerProvider, mergePolicyFactory, mergePolicyProp= erties, durable, null, null); this=2EtokenTypeTraits =3D tokenType= Traits; this=2EtokenCmpFactories =3D tokenCmpFactories; /= / ToDo: totally replace tokenizerFactory with full-text config diff --git a= /hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/ja= va/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndex= =2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/sr= c/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvert= edIndex=2Ejava index de65e3e=2E=2Ef2d5a37 100644 --- a/hyracks-fullstack/hy= racks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks= /storage/am/lsm/invertedindex/impls/LSMInvertedIndex=2Ejava +++ b/hyracks-f= ullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apa= che/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndex=2Ejava @@ -= 120,10 +120,10 @@ for (IVirtualBufferCache virtualBufferCache : vi= rtualBufferCaches) { InMemoryInvertedIndex memInvIndex =3D = createInMemoryInvertedIndex(virtualBufferCache, new Virtu= alFreePageManager(virtualBufferCache), i); - BTree deleteKeysBTr= ee =3D - BTreeUtils=2EcreateBTree(virtualBufferCache, ne= w VirtualFreePageManager(virtualBufferCache), - = invListTypeTraits, invListCmpFactories, BTreeLeafFrameType=2EREGULAR_NSM, -= ioManager=2EresolveAbsolutePath(fileManager=2Eg= etBaseDir() + "_virtual_del_" + i), false); + BTree deleteKeysBT= ree =3D BTreeUtils=2EcreateBTree(virtualBufferCache, + n= ew VirtualFreePageManager(virtualBufferCache), invListTypeTraits, invListCm= pFactories, + BTreeLeafFrameType=2EREGULAR_NSM, + = ioManager=2EresolveAbsolutePath(fileManager=2EgetBaseDir() + "= _virtual_del_" + i), false, null, null); LSMInvertedIndexMemor= yComponent mutableComponent =3D new LSMInvertedIndexMemoryComponent(this, m= emInvIndex, deleteKeysBTree, virtualBufferCache, filte= rHelper =3D=3D null ? null : filterHelper=2EcreateFilter()); m= emoryComponents=2Eadd(mutableComponent); diff --git a/hyracks-fullstack/hyr= acks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/= storage/am/lsm/invertedindex/inmemory/InMemoryInvertedIndex=2Ejava b/hyrack= s-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/= apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedIndex= =2Ejava index 6998861=2E=2Ec2e07b3 100644 --- a/hyracks-fullstack/hyracks/h= yracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storag= e/am/lsm/invertedindex/inmemory/InMemoryInvertedIndex=2Ejava +++ b/hyracks-= fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/ap= ache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedIndex=2E= java @@ -80,7 +80,7 @@ btreeCmpFactories[tokenTypeTraits=2Elen= gth + i] =3D invListCmpFactories[i]; } this=2Ebtree =3D B= TreeUtils=2EcreateBTree(virtualBufferCache, virtualFreePageManager, btreeTy= peTraits, - btreeCmpFactories, BTreeLeafFrameType=2EREGULAR_= NSM, btreeFileRef, false); + btreeCmpFactories, BTreeLeafFra= meType=2EREGULAR_NSM, btreeFileRef, false, null, null); } @Over= ride diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inverted= index/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/= OnDiskInvertedIndex=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-l= sm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedin= dex/ondisk/OnDiskInvertedIndex=2Ejava index 0f26435=2E=2E7a45c5d 100644 ---= a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/= java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedI= ndex=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inverted= index/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/= OnDiskInvertedIndex=2Ejava @@ -127,7 +127,8 @@ this=2EtokenTypeTra= its =3D tokenTypeTraits; this=2EtokenCmpFactories =3D tokenCmpFact= ories; this=2Ebtree =3D BTreeUtils=2EcreateDiskBTree(bufferCache, = getBTreeTypeTraits(tokenTypeTraits), tokenCmpFactories, - BT= reeLeafFrameType=2EREGULAR_NSM, btreeFile, pageManagerFactory=2EcreatePageM= anager(bufferCache), false); + BTreeLeafFrameType=2EREGULAR_= NSM, btreeFile, pageManagerFactory=2EcreatePageManager(bufferCache), false,= + null, null); this=2EnumTokenFields =3D btree=2Eg= etComparatorFactories()=2Elength; this=2EnumInvListKeys =3D invLis= tCmpFactories=2Elength; this=2EinvListsFile =3D invListsFile; diff= --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src= /main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/variables= ize/VariableSizeElementInvertedListBuilder=2Ejava b/hyracks-fullstack/hyrac= ks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/st= orage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeElementInvertedL= istBuilder=2Ejava index cfbb779=2E=2E3ce26c5 100644 --- a/hyracks-fullstack= /hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyra= cks/storage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeElementInv= ertedListBuilder=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-= lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedi= ndex/ondisk/variablesize/VariableSizeElementInvertedListBuilder=2Ejava @@ -= 46,7 +46,7 @@ for (int i =3D 0; i < invListFields=2Elength; i++) {= allFields[i + tokenTypeTraits=2Elength] =3D invListFields[i];= } - this=2Ewriter =3D new TypeAwareTupleWriter(allFields);= + this=2Ewriter =3D new TypeAwareTupleWriter(allFields, null, null)= ; InvertedIndexUtils=2EverifyHasVarSizeTypeTrait(invListFields);= } diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inver= tedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondi= sk/variablesize/VariableSizeElementOnDiskInvertedListCursor=2Ejava b/hyrack= s-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/= apache/hyracks/storage/am/lsm/invertedindex/ondisk/variablesize/VariableSiz= eElementOnDiskInvertedListCursor=2Ejava index ee2bc58=2E=2Ea5515b3 100644 -= -- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/mai= n/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/variablesize/= VariableSizeElementOnDiskInvertedListCursor=2Ejava +++ b/hyracks-fullstack/= hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyrac= ks/storage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeElementOnDi= skInvertedListCursor=2Ejava @@ -53,7 +53,7 @@ super(bufferCache, f= ileId, invListFields, stats); this=2EisInit =3D true; thi= s=2EtupleReference =3D new VariableSizeInvertedListTupleReference(invListFi= elds); - this=2EtupleWriter =3D new TypeAwareTupleWriter(invListFiel= ds); + this=2EtupleWriter =3D new TypeAwareTupleWriter(invListFields= , null, null); } public VariableSizeElementOnDiskInvertedListCu= rsor(IBufferCache bufferCache, int fileId, @@ -70,7 +70,7 @@ curre= ntElementIxForScan =3D 0; isInit =3D true; this=2EtupleRe= ference =3D new VariableSizeInvertedListTupleReference(invListFields); - = this=2EtupleWriter =3D new TypeAwareTupleWriter(invListFields); + = this=2EtupleWriter =3D new TypeAwareTupleWriter(invListFields, null, nul= l); } /** diff --git a/hyracks-fullstack/hyracks/hyracks-storag= e-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/inve= rtedindex/ondisk/variablesize/VariableSizeInvertedListSearchResultFrameTupl= eAccessor=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inverte= dindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk= /variablesize/VariableSizeInvertedListSearchResultFrameTupleAccessor=2Ejava= index ab71c0a=2E=2E75d0cfe 100644 --- a/hyracks-fullstack/hyracks/hyracks-= storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/ls= m/invertedindex/ondisk/variablesize/VariableSizeInvertedListSearchResultFra= meTupleAccessor=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-l= sm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedin= dex/ondisk/variablesize/VariableSizeInvertedListSearchResultFrameTupleAcces= sor=2Ejava @@ -58,7 +58,7 @@ throws HyracksDataException { = super(frameSize, fields); - this=2EtupleWriter =3D new TypeA= wareTupleWriter(fields); + this=2EtupleWriter =3D new TypeAwareTuple= Writer(fields, null, null); this=2EtupleReference =3D new Variable= SizeInvertedListTupleReference(fields); } diff --git a/hyracks-fulls= tack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/= hyracks/storage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeInvert= edListTupleReference=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-= lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedi= ndex/ondisk/variablesize/VariableSizeInvertedListTupleReference=2Ejava inde= x 8e1db66=2E=2E1e69076 100644 --- a/hyracks-fullstack/hyracks/hyracks-stora= ge-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/inv= ertedindex/ondisk/variablesize/VariableSizeInvertedListTupleReference=2Ejav= a +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/= main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/variablesi= ze/VariableSizeInvertedListTupleReference=2Ejava @@ -38,7 +38,7 @@ pub= lic VariableSizeInvertedListTupleReference(ITypeTraits[] typeTraits) throws= HyracksDataException { super(typeTraits); - this=2Etuple= Reference =3D new TypeAwareTupleReference(typeTraits); + this=2Etupl= eReference =3D new TypeAwareTupleReference(typeTraits, null); } = @Override diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-in= vertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/s= earch/InvertedIndexSearchResult=2Ejava b/hyracks-fullstack/hyracks/hyracks-= storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/ls= m/invertedindex/search/InvertedIndexSearchResult=2Ejava index 707d85d=2E=2E= 6958fd5 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inver= tedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/sear= ch/InvertedIndexSearchResult=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks= -storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/l= sm/invertedindex/search/InvertedIndexSearchResult=2Ejava @@ -86,7 +86,7 @@ = public InvertedIndexSearchResult(ITypeTraits[] invListFields, IHyracks= TaskContext ctx, ISimpleFrameBufferManager bufferManager) thro= ws HyracksDataException { this=2EinvListFields =3D invListFields; = - this=2EtupleWriter =3D new TypeAwareTupleWriter(invListFields); + = this=2EtupleWriter =3D new TypeAwareTupleWriter(invListFields, null,= null); initTypeTraits(invListFields); this=2Ectx =3D ctx= ; appender =3D new InvertedListSearchResultFrameTupleAppender(ctx= =2EgetInitialFrameSize()); diff --git a/hyracks-fullstack/hyracks/hyracks-s= torage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm= /invertedindex/util/InvertedIndexUtils=2Ejava b/hyracks-fullstack/hyracks/h= yracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storag= e/am/lsm/invertedindex/util/InvertedIndexUtils=2Ejava index dcde832=2E=2Ebe= e6be6 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inverte= dindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/util/I= nvertedIndexUtils=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am= -lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/inverted= index/util/InvertedIndexUtils=2Ejava @@ -130,7 +130,7 @@ IBina= ryComparatorFactory[] invListCmpFactories, IBufferCache diskBufferCache, = IPageManagerFactory freePageManagerFactory) throws HyracksDataEx= ception { BTreeTypeAwareTupleWriterFactory tupleWriterFactory =3D = - new BTreeTypeAwareTupleWriterFactory(invListTypeTraits, fa= lse); + new BTreeTypeAwareTupleWriterFactory(invListTypeTrai= ts, false, null, null); ITreeIndexFrameFactory leafFrameFactory = =3D BTreeUtils=2EgetLeafFrameFactory(tupleWriterFactory, B= TreeLeafFrameType=2EREGULAR_NSM); ITreeIndexFrameFactory interiorF= rameFactory =3D new BTreeNSMInteriorFrameFactory(tupleWriterFactory); @@ -1= 74,7 +174,8 @@ LSMComponentFilterFrameFactory filterFrameFactory = =3D null; LSMComponentFilterManager filterManager =3D null; = if (filterCmpFactories !=3D null) { - TypeAwareTupleWriterFac= tory filterTupleWriterFactory =3D new TypeAwareTupleWriterFactory(filterTyp= eTraits); + TypeAwareTupleWriterFactory filterTupleWriterFactory= =3D + new TypeAwareTupleWriterFactory(filterTypeTraits,= null, null); filterHelper =3D new ComponentFilterHelper(filte= rTupleWriterFactory, filterCmpFactories); filterFrameFactory = =3D new LSMComponentFilterFrameFactory(filterTupleWriterFactory); = filterManager =3D new LSMComponentFilterManager(filterFrameFactory); @@= -225,7 +226,8 @@ LSMComponentFilterFrameFactory filterFrameFactor= y =3D null; LSMComponentFilterManager filterManager =3D null; = if (filterCmpFactories !=3D null) { - TypeAwareTupleWriterF= actory filterTupleWriterFactory =3D new TypeAwareTupleWriterFactory(filterT= ypeTraits); + TypeAwareTupleWriterFactory filterTupleWriterFacto= ry =3D + new TypeAwareTupleWriterFactory(filterTypeTrait= s, null, null); filterHelper =3D new ComponentFilterHelper(fil= terTupleWriterFactory, filterCmpFactories); filterFrameFactory= =3D new LSMComponentFilterFrameFactory(filterTupleWriterFactory); = filterManager =3D new LSMComponentFilterManager(filterFrameFactory); d= iff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main= /java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeLocalResourc= e=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/= java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeLocalResource= =2Ejava index bfae1cd=2E=2Ec7a59f4 100644 --- a/hyracks-fullstack/hyracks/h= yracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm= /rtree/dataflow/LSMRTreeLocalResource=2Ejava +++ b/hyracks-fullstack/hyrack= s/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/= lsm/rtree/dataflow/LSMRTreeLocalResource=2Ejava @@ -76,7 +76,7 @@ = boolean isPointMBR, double bloomFilterFalsePositiveRate) { sup= er(path, storageManager, typeTraits, rtreeCmpFactories, filterTypeTraits, f= ilterCmpFactories, filterFields, opTrackerProvider, ioOpCa= llbackFactory, pageWriteCallbackFactory, metadataPageManagerFactory, - = vbcProvider, ioSchedulerProvider, mergePolicyFactory, mergePolic= yProperties, durable); + vbcProvider, ioSchedulerProvider, m= ergePolicyFactory, mergePolicyProperties, durable, null, null); th= is=2EbtreeCmpFactories =3D btreeCmpFactories; this=2EvalueProvider= Factories =3D valueProviderFactories; this=2ErtreePolicyType =3D r= treePolicyType; diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-l= sm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSM= RTreeLocalResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storag= e-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/datafl= ow/LSMRTreeLocalResourceFactory=2Ejava index 635f1ac=2E=2E31f78d8 100644 --= - a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/or= g/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeLocalResourceFactory= =2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/ma= in/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeLocalResou= rceFactory=2Ejava @@ -63,7 +63,7 @@ boolean isPointMBR, double= bloomFilterFalsePositiveRate) { super(storageManager, typeTraits,= cmpFactories, filterTypeTraits, filterCmpFactories, filterFields, = opTrackerProvider, ioOpCallbackFactory, pageWriteCallbackFactory, = metadataPageManagerFactory, - vbcProvider, ioSchedulerProvid= er, mergePolicyFactory, mergePolicyProperties, durable); + v= bcProvider, ioSchedulerProvider, mergePolicyFactory, mergePolicyProperties,= durable, null, null); this=2EbtreeCmpFactories =3D btreeCmpFactor= ies; this=2EvalueProviderFactories =3D valueProviderFactories; = this=2ErtreePolicyType =3D rtreePolicyType; diff --git a/hyracks-full= stack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks= /storage/am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLocalResource=2Ejava b= /hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/a= pache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLocalReso= urce=2Ejava index e4a8287=2E=2Ea0d0220 100644 --- a/hyracks-fullstack/hyrac= ks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am= /lsm/rtree/dataflow/LSMRTreeWithAntiMatterLocalResource=2Ejava +++ b/hyrack= s-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/h= yracks/storage/am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLocalResource=2E= java @@ -73,7 +73,7 @@ ILinearizeComparatorFactory linearizeCm= pFactory, int[] rtreeFields, boolean isPointMBR, boolean durable) { = super(path, storageManager, typeTraits, rtreeCmpFactories, filterTypeTrai= ts, filterCmpFactories, filterFields, opTrackerProvider, i= oOpCallbackFactory, pageWriteCallbackFactory, metadataPageManagerFactory, -= vbcProvider, ioSchedulerProvider, mergePolicyFactory, merge= PolicyProperties, durable); + vbcProvider, ioSchedulerProvid= er, mergePolicyFactory, mergePolicyProperties, durable, null, null); = this=2EbtreeCmpFactories =3D btreeCmpFactories; this=2EvaluePro= viderFactories =3D valueProviderFactories; this=2ErtreePolicyType = =3D rtreePolicyType; diff --git a/hyracks-fullstack/hyracks/hyracks-storage= -am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflo= w/LSMRTreeWithAntiMatterLocalResourceFactory=2Ejava b/hyracks-fullstack/hyr= acks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/= am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLocalResourceFactory=2Ejava ind= ex 6edf877=2E=2Eed4c5b5 100644 --- a/hyracks-fullstack/hyracks/hyracks-stor= age-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/data= flow/LSMRTreeWithAntiMatterLocalResourceFactory=2Ejava +++ b/hyracks-fullst= ack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/s= torage/am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLocalResourceFactory=2Ej= ava @@ -60,7 +60,7 @@ IBinaryComparatorFactory[] btreeComparat= orFactories) { super(storageManager, rteeTypeTraits, rtreeCmpFacto= ries, filterTypeTraits, filterCmpFactories, filterFields, = opTrackerFactory, ioOpCallbackFactory, pageWriteCallbackFactory, metadataPa= geManagerFactory, - vbcProvider, ioSchedulerProvider, mergeP= olicyFactory, mergePolicyProperties, durable); + vbcProvider= , ioSchedulerProvider, mergePolicyFactory, mergePolicyProperties, durable, = null, null); this=2EvalueProviderFactories =3D valueProviderFactor= ies; this=2ErtreePolicyType =3D rtreePolicyType; this=2El= inearizeCmpFactory =3D linearizeCmpFactory; diff --git a/hyracks-fullstack/= hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/stora= ge/am/lsm/rtree/tuples/LSMRTreeTupleReferenceForPointMBR=2Ejava b/hyracks-f= ullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyra= cks/storage/am/lsm/rtree/tuples/LSMRTreeTupleReferenceForPointMBR=2Ejava in= dex e28d5a2=2E=2Eef9eb47 100644 --- a/hyracks-fullstack/hyracks/hyracks-sto= rage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/tup= les/LSMRTreeTupleReferenceForPointMBR=2Ejava +++ b/hyracks-fullstack/hyrack= s/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/= lsm/rtree/tuples/LSMRTreeTupleReferenceForPointMBR=2Ejava @@ -26,6 +26,7 @@= import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ertree=2Etuples=2ERTreeType= AwareTupleReference; public class LSMRTreeTupleReferenceForPointMBR exte= nds RTreeTypeAwareTupleReference implements ILSMTreeTupleReference { + = private final int inputKeyFieldCount; //double field count for mbr seconda= ry key of an input tuple private final int inputTotalFieldCount; //tot= al field count (key + value fields) of an input tuple=2E private final= int storedKeyFieldCount; //double field count to be stored for the mbr sec= ondary key @@ -140,6 +141,7 @@ @Override public boolean isAntim= atter() { + // TODO(ali): antimatterAware should be checked? = // Check antimatter bit=2E return BitOperationUtils=2EgetBit(buf= , tupleStartOff, ANTIMATTER_BIT_OFFSET); } diff --git a/hyracks-fullst= ack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/s= torage/am/lsm/rtree/tuples/LSMRTreeTupleWriterForPointMBR=2Ejava b/hyracks-= fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyr= acks/storage/am/lsm/rtree/tuples/LSMRTreeTupleWriterForPointMBR=2Ejava inde= x ce6bdeb=2E=2Eae2d642 100644 --- a/hyracks-fullstack/hyracks/hyracks-stora= ge-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/tuple= s/LSMRTreeTupleWriterForPointMBR=2Ejava +++ b/hyracks-fullstack/hyracks/hyr= acks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/r= tree/tuples/LSMRTreeTupleWriterForPointMBR=2Ejava @@ -44,6 +44,7 @@ */ = public class LSMRTreeTupleWriterForPointMBR extends RTreeTypeAwareTupleWri= ter implements ILSMTreeTupleWriter { + private final int inputKeyField= Count; //double field count for mbr secondary key of an input tuple pr= ivate final int valueFieldCount; //value(or payload or primary key) field c= ount (same for an input tuple and a stored tuple) private final int in= putTotalFieldCount; //total field count (key + value fields) of an input tu= ple=2E @@ -157,6 +158,7 @@ } protected void setAntimatterBit(by= te[] targetBuf, int targetOff) { + // TODO(ali): antimatterAware sho= uld be checked? // Set antimatter bit to 1=2E BitOperatio= nUtils=2EsetBit(targetBuf, targetOff, ANTIMATTER_BIT_OFFSET); } diff -= -git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java= /org/apache/hyracks/storage/am/lsm/rtree/utils/LSMRTreeUtils=2Ejava b/hyrac= ks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/= hyracks/storage/am/lsm/rtree/utils/LSMRTreeUtils=2Ejava index ba6015d=2E=2E= cf44b2f 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree= /src/main/java/org/apache/hyracks/storage/am/lsm/rtree/utils/LSMRTreeUtils= =2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/ma= in/java/org/apache/hyracks/storage/am/lsm/rtree/utils/LSMRTreeUtils=2Ejava = @@ -102,7 +102,7 @@ rtreeLeafFrameTupleWriterFactory =3D rtree= InteriorFrameTupleWriterFactory; } BTreeTypeAwareTupleWri= terFactory btreeTupleWriterFactory =3D - new BTreeTypeAwareT= upleWriterFactory(btreeTypeTraits, false); + new BTreeTypeAw= areTupleWriterFactory(btreeTypeTraits, false, null, null); RTreeFr= ameFactory rtreeInteriorFrameFactory =3D new RTreeNSMInteriorFrameFactory( = rtreeInteriorFrameTupleWriterFactory, valueProviderFactori= es, rtreePolicyType, isPointMBR); RTreeFrameFactory rtreeLeafFrame= Factory =3D new RTreeNSMLeafFrameFactory(rtreeLeafFrameTupleWriterFactory, = @@ -127,7 +127,8 @@ LSMComponentFilterFrameFactory filterFrameFact= ory =3D null; LSMComponentFilterManager filterManager =3D null; = if (filterCmpFactories !=3D null) { - TypeAwareTupleWrite= rFactory filterTupleWriterFactory =3D new TypeAwareTupleWriterFactory(filte= rTypeTraits); + TypeAwareTupleWriterFactory filterTupleWriterFac= tory =3D + new TypeAwareTupleWriterFactory(filterTypeTra= its, null, null); filterHelper =3D new ComponentFilterHelper(f= ilterTupleWriterFactory, filterCmpFactories); filterFrameFacto= ry =3D new LSMComponentFilterFrameFactory(filterTupleWriterFactory); = filterManager =3D new LSMComponentFilterManager(filterFrameFactory);= @@ -217,7 +218,8 @@ LSMComponentFilterFrameFactory filterFrameFac= tory =3D null; LSMComponentFilterManager filterManager =3D null; = if (filterCmpFactories !=3D null) { - TypeAwareTupleWrit= erFactory filterTupleWriterFactory =3D new TypeAwareTupleWriterFactory(filt= erTypeTraits); + TypeAwareTupleWriterFactory filterTupleWriterFa= ctory =3D + new TypeAwareTupleWriterFactory(filterTypeTr= aits, null, null); filterHelper =3D new ComponentFilterHelper(= filterTupleWriterFactory, filterCmpFactories); filterFrameFact= ory =3D new LSMComponentFilterFrameFactory(filterTupleWriterFactory); = filterManager =3D new LSMComponentFilterManager(filterFrameFactory)= ; @@ -264,7 +266,7 @@ rtreeLeafFrameTupleWriterFactory =3D rtr= eeInteriorFrameTupleWriterFactory; } BTreeTypeAwareTupleW= riterFactory btreeTupleWriterFactory =3D - new BTreeTypeAwar= eTupleWriterFactory(btreeTypeTraits, false); + new BTreeType= AwareTupleWriterFactory(btreeTypeTraits, false, null, null); RTree= FrameFactory rtreeInteriorFrameFactory =3D new RTreeNSMInteriorFrameFactory= ( rtreeInteriorFrameTupleWriterFactory, valueProviderFacto= ries, rtreePolicyType, isPointMBR); RTreeFrameFactory rtreeLeafFra= meFactory =3D new RTreeNSMLeafFrameFactory(rtreeLeafFrameTupleWriterFactory= , diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/= java/org/apache/hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleReferenc= e=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java= /org/apache/hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleReference=2E= java index 61e2326=2E=2E8f7f3f0 100644 --- a/hyracks-fullstack/hyracks/hyra= cks-storage-am-rtree/src/main/java/org/apache/hyracks/storage/am/rtree/tupl= es/RTreeTypeAwareTupleReference=2Ejava +++ b/hyracks-fullstack/hyracks/hyra= cks-storage-am-rtree/src/main/java/org/apache/hyracks/storage/am/rtree/tupl= es/RTreeTypeAwareTupleReference=2Ejava @@ -27,7 +27,7 @@ public class RTre= eTypeAwareTupleReference extends TypeAwareTupleReference implements ITreeIn= dexTupleReference { public RTreeTypeAwareTupleReference(ITypeTraits[= ] typeTraits) { - super(typeTraits); + super(typeTraits, null= ); } } diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-rt= ree/src/main/java/org/apache/hyracks/storage/am/rtree/tuples/RTreeTypeAware= TupleWriter=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src= /main/java/org/apache/hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleWr= iter=2Ejava index 40b0481=2E=2E42a9662 100644 --- a/hyracks-fullstack/hyrac= ks/hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/storage/am/rtr= ee/tuples/RTreeTypeAwareTupleWriter=2Ejava +++ b/hyracks-fullstack/hyracks/= hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/storage/am/rtree/= tuples/RTreeTypeAwareTupleWriter=2Ejava @@ -29,7 +29,7 @@ public class RTr= eeTypeAwareTupleWriter extends TypeAwareTupleWriter { public RTreeTy= peAwareTupleWriter(ITypeTraits[] typeTraits) { - super(typeTraits); = + super(typeTraits, null, null); } public int writeTuple= Fields(ITreeIndexTupleReference[] refs, int startField, ByteBuffer targetBu= f, int targetOff) { diff --git a/hyracks-fullstack/hyracks/hyracks-storage-= am-rtree/src/main/java/org/apache/hyracks/storage/am/rtree/tuples/RTreeType= AwareTupleWriterFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-= am-rtree/src/main/java/org/apache/hyracks/storage/am/rtree/tuples/RTreeType= AwareTupleWriterFactory=2Ejava index 4536a0c=2E=2E0069cc3 100644 --- a/hyra= cks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyr= acks/storage/am/rtree/tuples/RTreeTypeAwareTupleWriterFactory=2Ejava +++ b/= hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache= /hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleWriterFactory=2Ejava @@= -27,7 +27,7 @@ private static final long serialVersionUID =3D 1L; = public RTreeTypeAwareTupleWriterFactory(ITypeTraits[] typeTraits) { - = super(typeTraits); + super(typeTraits, null, null); } = @Override diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-= storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/BTr= eeExamplesTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-sto= rage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/BTreeE= xamplesTest=2Ejava index 8c0f5a5=2E=2E9d91e14 100644 --- a/hyracks-fullstac= k/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apa= che/hyracks/storage/am/btree/BTreeExamplesTest=2Ejava +++ b/hyracks-fullsta= ck/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/ap= ache/hyracks/storage/am/btree/BTreeExamplesTest=2Ejava @@ -53,7 +53,7 @@ = LinkedMetaDataPageManager freePageManager =3D new L= inkedMetaDataPageManager(harness=2EgetBufferCache(), metaFrameFactory); = return BTreeUtils=2EcreateBTree(harness=2EgetBufferCache(), typeTrait= s, cmpFactories, - BTreeLeafFrameType=2EREGULAR_NSM, harness= =2EgetFileReference(), freePageManager, false); + BTreeLeafF= rameType=2EREGULAR_NSM, harness=2EgetFileReference(), freePageManager, fals= e, null, null); } } diff --git a/hyracks-fullstack/hyracks/hyracks-= tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/storag= e/am/btree/BTreeModificationOperationCallbackTest=2Ejava b/hyracks-fullstac= k/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apa= che/hyracks/storage/am/btree/BTreeModificationOperationCallbackTest=2Ejava = index ad9198c=2E=2E786a6af 100644 --- a/hyracks-fullstack/hyracks/hyracks-t= ests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage= /am/btree/BTreeModificationOperationCallbackTest=2Ejava +++ b/hyracks-fulls= tack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/= apache/hyracks/storage/am/btree/BTreeModificationOperationCallbackTest=2Eja= va @@ -39,7 +39,7 @@ IMetadataPageManager freePageManager =3D free= PageManagerFactory=2EcreatePageManager(harness=2EgetBufferCache()); = index =3D BTreeUtils=2EcreateBTree(harness=2EgetBufferCache(), SerdeUtils= =2EserdesToTypeTraits(keySerdes), SerdeUtils=2EserdesToCom= paratorFactories(keySerdes, keySerdes=2Elength), BTreeLeafFrameType=2EREGUL= AR_NSM, - harness=2EgetFileReference(), freePageManager, fal= se); + harness=2EgetFileReference(), freePageManager, false,= null, null); } @Override diff --git a/hyracks-fullstack/hyrack= s/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyra= cks/storage/am/btree/BTreeSearchCursorTest=2Ejava b/hyracks-fullstack/hyrac= ks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyr= acks/storage/am/btree/BTreeSearchCursorTest=2Ejava index 2e2f306=2E=2E82cd9= 09 100644 --- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-= btree-test/src/test/java/org/apache/hyracks/storage/am/btree/BTreeSearchCur= sorTest=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storag= e-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/BTreeSear= chCursorTest=2Ejava @@ -66,7 +66,7 @@ public static final int FIELD_CO= UNT =3D 2; public static final ITypeTraits[] TYPE_TRAITS =3D { Integer= Pointable=2ETYPE_TRAITS, IntegerPointable=2ETYPE_TRAITS }; public stat= ic final BTreeTypeAwareTupleWriterFactory TUPLE_WRITER_FACTORY =3D - = new BTreeTypeAwareTupleWriterFactory(TYPE_TRAITS, false); + = new BTreeTypeAwareTupleWriterFactory(TYPE_TRAITS, false, null, null); = public static final ITreeIndexMetadataFrameFactory META_FRAME_FACTORY =3D = new LIFOMetaDataFrameFactory(); public static final int KEY_FIELDS_COU= NT =3D 1; public static final IBinaryComparatorFactory[] CMP_FACTORIES= =3D { IntegerBinaryComparatorFactory=2EINSTANCE }; diff --git a/hyracks-fu= llstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/o= rg/apache/hyracks/storage/am/btree/BTreeSearchOperationCallbackTest=2Ejava = b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src= /test/java/org/apache/hyracks/storage/am/btree/BTreeSearchOperationCallback= Test=2Ejava index 89399bd=2E=2Ea7b4c4d 100644 --- a/hyracks-fullstack/hyrac= ks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyr= acks/storage/am/btree/BTreeSearchOperationCallbackTest=2Ejava +++ b/hyracks= -fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/jav= a/org/apache/hyracks/storage/am/btree/BTreeSearchOperationCallbackTest=2Eja= va @@ -40,7 +40,7 @@ new LinkedMetaDataPageManager(harness= =2EgetBufferCache(), metaFrameFactory); index =3D BTreeUtils=2Ecre= ateBTree(harness=2EgetBufferCache(), SerdeUtils=2EserdesToTypeTraits(keySer= des), SerdeUtils=2EserdesToComparatorFactories(keySerdes, = keySerdes=2Elength), BTreeLeafFrameType=2EREGULAR_NSM, - har= ness=2EgetFileReference(), freePageManager, false); + harnes= s=2EgetFileReference(), freePageManager, false, null, null); } = @Override diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-stor= age-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/BTreeSt= atsTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am= -btree-test/src/test/java/org/apache/hyracks/storage/am/btree/BTreeStatsTes= t=2Ejava index 7a24727=2E=2E97e5e46 100644 --- a/hyracks-fullstack/hyracks/= hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyrack= s/storage/am/btree/BTreeStatsTest=2Ejava +++ b/hyracks-fullstack/hyracks/hy= racks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/= storage/am/btree/BTreeStatsTest=2Ejava @@ -84,7 +84,8 @@ int keyFi= eldCount =3D 1; IBinaryComparatorFactory[] cmpFactories =3D new IB= inaryComparatorFactory[keyFieldCount]; cmpFactories[0] =3D Integer= BinaryComparatorFactory=2EINSTANCE; - BTreeTypeAwareTupleWriterFacto= ry tupleWriterFactory =3D new BTreeTypeAwareTupleWriterFactory(typeTraits, = false); + BTreeTypeAwareTupleWriterFactory tupleWriterFactory =3D + = new BTreeTypeAwareTupleWriterFactory(typeTraits, false, null= , null); ITreeIndexFrameFactory leafFrameFactory =3D new BTreeNSML= eafFrameFactory(tupleWriterFactory); ITreeIndexFrameFactory interi= orFrameFactory =3D new BTreeNSMInteriorFrameFactory(tupleWriterFactory); = ITreeIndexMetadataFrameFactory metaFrameFactory =3D new LIFOMetaData= FrameFactory(); diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyrack= s-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/B= TreeUpdateSearchTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyrac= ks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/= BTreeUpdateSearchTest=2Ejava index 5b86b4f=2E=2E720dd59 100644 --- a/hyrack= s-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/ja= va/org/apache/hyracks/storage/am/btree/BTreeUpdateSearchTest=2Ejava +++ b/h= yracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/te= st/java/org/apache/hyracks/storage/am/btree/BTreeUpdateSearchTest=2Ejava @@= -74,7 +74,8 @@ ISerializerDeserializer[] recDescSers =3D = { IntegerSerializerDeserializer=2EINSTANCE, IntegerSerializerDeser= ializer=2EINSTANCE }; - BTreeTypeAwareTupleWriterFactory tupleWrit= erFactory =3D new BTreeTypeAwareTupleWriterFactory(typeTraits, false); + = BTreeTypeAwareTupleWriterFactory tupleWriterFactory =3D + = new BTreeTypeAwareTupleWriterFactory(typeTraits, false, null, null); = ITreeIndexFrameFactory leafFrameFactory =3D new BTreeNSMLeafFrameFact= ory(tupleWriterFactory); ITreeIndexFrameFactory interiorFrameFacto= ry =3D new BTreeNSMInteriorFrameFactory(tupleWriterFactory); ITree= IndexMetadataFrameFactory metaFrameFactory =3D new LIFOMetaDataFrameFactory= (); diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am= -btree-test/src/test/java/org/apache/hyracks/storage/am/btree/FieldPrefixNS= MTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-b= tree-test/src/test/java/org/apache/hyracks/storage/am/btree/FieldPrefixNSMT= est=2Ejava index fe05828=2E=2Ee9b9e7f 100644 --- a/hyracks-fullstack/hyrack= s/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyra= cks/storage/am/btree/FieldPrefixNSMTest=2Ejava +++ b/hyracks-fullstack/hyra= cks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hy= racks/storage/am/btree/FieldPrefixNSMTest=2Ejava @@ -131,7 +131,7 @@ = ICachedPage page =3D bufferCache=2Epin(BufferedFileHandle=2EgetDiskPageI= d(btreeFileId, 0), true); try { - BTreeTypeAwareTuple= Writer tupleWriter =3D new BTreeTypeAwareTupleWriter(typeTraits, false); + = BTreeTypeAwareTupleWriter tupleWriter =3D new BTreeTypeAwareTupl= eWriter(typeTraits, false, null, null); BTreeFieldPrefixNSMLea= fFrame frame =3D new BTreeFieldPrefixNSMLeafFrame(tupleWriter); = frame=2EsetPage(page); frame=2EinitBuffer((byte) 0); diff --= git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test= /src/test/java/org/apache/hyracks/storage/am/btree/multithread/BTreeMultiTh= readTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-a= m-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/multithread/= BTreeMultiThreadTest=2Ejava index dcdbdc6=2E=2E6b94b03 100644 --- a/hyracks= -fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/jav= a/org/apache/hyracks/storage/am/btree/multithread/BTreeMultiThreadTest=2Eja= va +++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-t= est/src/test/java/org/apache/hyracks/storage/am/btree/multithread/BTreeMult= iThreadTest=2Ejava @@ -54,7 +54,7 @@ int[] bloomFilterKeyField= s) throws HyracksDataException { return BTreeUtils=2EcreateBTree(h= arness=2EgetBufferCache(), typeTraits, cmpFactories, BTree= LeafFrameType=2EREGULAR_NSM, harness=2EgetFileReference(), - = harness=2EgetPageManagerFactory()=2EcreatePageManager(harness=2EgetBufferC= ache()), false); + harness=2EgetPageManagerFactory()=2Ecreat= ePageManager(harness=2EgetBufferCache()), false, null, null); } = @Override diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-sto= rage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/util/B= TreeTestContext=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-st= orage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/util/= BTreeTestContext=2Ejava index 9d66b64=2E=2Eeaa1c28 100644 --- a/hyracks-ful= lstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/or= g/apache/hyracks/storage/am/btree/util/BTreeTestContext=2Ejava +++ b/hyrack= s-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/ja= va/org/apache/hyracks/storage/am/btree/util/BTreeTestContext=2Ejava @@ -56,= 7 +56,8 @@ IPageManager pageManager) throws Exception { = ITypeTraits[] typeTraits =3D SerdeUtils=2EserdesToTypeTraits(fieldSerdes= ); IBinaryComparatorFactory[] cmpFactories =3D SerdeUtils=2Eserdes= ToComparatorFactories(fieldSerdes, numKeyFields); - BTree btree =3D = BTreeUtils=2EcreateBTree(bufferCache, typeTraits, cmpFactories, leafType, f= ile, pageManager, false); + BTree btree =3D BTreeUtils=2EcreateBTree= (bufferCache, typeTraits, cmpFactories, leafType, file, pageManager, false,= + null, null); BTreeTestContext testCtx =3D new BT= reeTestContext(fieldSerdes, btree); return testCtx; } diff --= git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-= test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeExamples= Test=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-ls= m-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTree= ExamplesTest=2Ejava index 1d41280=2E=2E3313443 100644 --- a/hyracks-fullsta= ck/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/or= g/apache/hyracks/storage/am/lsm/btree/LSMBTreeExamplesTest=2Ejava +++ b/hyr= acks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/= test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeExamplesTest=2Eja= va @@ -64,7 +64,7 @@ harness=2EgetOperationTracker(), harn= ess=2EgetIOScheduler(), harness=2EgetIOOperationCallbackFactory(), = harness=2EgetPageWriteCallbackFactory(), true, filterTypeTraits, f= ilterCmpFactories, btreeFields, filterFields, true, harnes= s=2EgetMetadataPageManagerFactory(), false, ITracer=2ENONE, - = NoOpCompressorDecompressorFactory=2EINSTANCE, bloomFilterKeyFields !=3D n= ull); + NoOpCompressorDecompressorFactory=2EINSTANCE, bloomF= ilterKeyFields !=3D null, null, null); } @Before diff --git a/h= yracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/sr= c/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeModificationOpe= rationCallbackTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks= -storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/= btree/LSMBTreeModificationOperationCallbackTest=2Ejava index 8d8ec23=2E=2E2= 476761 100644 --- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage= -am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LS= MBTreeModificationOperationCallbackTest=2Ejava +++ b/hyracks-fullstack/hyra= cks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apach= e/hyracks/storage/am/lsm/btree/LSMBTreeModificationOperationCallbackTest=2E= java @@ -54,7 +54,7 @@ NoOpOperationTrackerFactory=2EINSTA= NCE=2EgetOperationTracker(null, null), harness=2EgetIOScheduler(), = harness=2EgetIOOperationCallbackFactory(), harness=2EgetPageWriteC= allbackFactory(), true, null, null, null, null, true, harn= ess=2EgetMetadataPageManagerFactory(), false, ITracer=2ENONE, - = NoOpCompressorDecompressorFactory=2EINSTANCE, HAS_BLOOM_FILTER); + = NoOpCompressorDecompressorFactory=2EINSTANCE, HAS_BLOOM_FILTER, = null, null); } @Override diff --git a/hyracks-fullstack/hyracks= /hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/h= yracks/storage/am/lsm/btree/LSMBTreeSearchOperationCallbackTest=2Ejava b/hy= racks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src= /test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeSearchOperationC= allbackTest=2Ejava index ab068ae=2E=2E6dec9f4 100644 --- a/hyracks-fullstac= k/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org= /apache/hyracks/storage/am/lsm/btree/LSMBTreeSearchOperationCallbackTest=2E= java +++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-b= tree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeSea= rchOperationCallbackTest=2Ejava @@ -63,7 +63,7 @@ NoOpOper= ationTrackerFactory=2EINSTANCE=2EgetOperationTracker(null, null), harness= =2EgetIOScheduler(), harness=2EgetIOOperationCallbackFacto= ry(), harness=2EgetPageWriteCallbackFactory(), true, null, null, null, = null, true, harness=2EgetMetadataPageManagerFactory(), false, = ITracer=2ENONE, - NoOpCompressorDecompressorFactory=2EINSTAN= CE, HAS_BLOOM_FILTER); + NoOpCompressorDecompressorFactory= =2EINSTANCE, HAS_BLOOM_FILTER, null, null); } @Override diff --= git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-= test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeUpdateIn= PlaceTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-= am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSM= BTreeUpdateInPlaceTest=2Ejava index 2954854=2E=2E0a4b37b 100644 --- a/hyrac= ks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/te= st/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeUpdateInPlaceTest= =2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-ls= m-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTree= UpdateInPlaceTest=2Ejava @@ -76,7 +76,7 @@ NoOpOperationTr= ackerFactory=2EINSTANCE=2EgetOperationTracker(null, null), harness=2EgetIOS= cheduler(), harness=2EgetIOOperationCallbackFactory(), har= ness=2EgetPageWriteCallbackFactory(), true, null, null, null, = null, true, harness=2EgetMetadataPageManagerFactory(), true, ITracer=2E= NONE, - NoOpCompressorDecompressorFactory=2EINSTANCE, HAS_BL= OOM_FILTER); + NoOpCompressorDecompressorFactory=2EINSTANCE,= HAS_BLOOM_FILTER, null, null); } @Override diff --git a/hyrack= s-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/tes= t/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeLocalResour= ce=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-= btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestL= smBtreeLocalResource=2Ejava index 3450531=2E=2E8e2d6dd 100644 --- a/hyracks= -fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test= /java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeLocalResourc= e=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-l= sm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/Te= stLsmBtreeLocalResource=2Ejava @@ -61,7 +61,7 @@ storageMa= nager, mergePolicyFactory, mergePolicyProperties, filterTypeTraits, filterC= mpFactories, btreeFields, filterFields, opTrackerProvider,= ioOpCallbackFactory, pageWriteCallbackFactory, metadataPa= geManagerFactory, vbcProvider, ioSchedulerProvider, durable, - = NoOpCompressorDecompressorFactory=2EINSTANCE, hasBloomFilter); + = NoOpCompressorDecompressorFactory=2EINSTANCE, hasBloomFilter, null,= null); } protected TestLsmBtreeLocalResource(IPersistedResourc= eRegistry registry, JsonNode json, int[] bloomFilterKeyFields, diff --git a= /hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/= src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeLoca= lResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-s= torage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/bt= ree/impl/TestLsmBtreeLocalResourceFactory=2Ejava index 229b027=2E=2E7d9b24a= 100644 --- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-ls= m-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/Tes= tLsmBtreeLocalResourceFactory=2Ejava +++ b/hyracks-fullstack/hyracks/hyrack= s-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/= storage/am/lsm/btree/impl/TestLsmBtreeLocalResourceFactory=2Ejava @@ -51,7 = +51,7 @@ opTrackerFactory, ioOpCallbackFactory, pageWriteC= allbackFactory, metadataPageManagerFactory, vbcProvider, i= oSchedulerProvider, mergePolicyFactory, mergePolicyProperties, durable, = bloomFilterKeyFields, bloomFilterFalsePositiveRate, isPrimary= , btreeFields, - NoOpCompressorDecompressorFactory=2EINSTANC= E, hasBloomFilter); + NoOpCompressorDecompressorFactory=2EIN= STANCE, hasBloomFilter, null, null); } @Override diff --git a/h= yracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/sr= c/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeUtil= =2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-bt= ree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsm= BtreeUtil=2Ejava index eebfb2b=2E=2Edd42838 100644 --- a/hyracks-fullstack/= hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/a= pache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeUtil=2Ejava +++ b/hyrac= ks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/te= st/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeUtil=2Ejav= a @@ -68,11 +68,11 @@ int[] btreeFields, int[] filterFields, b= oolean durable, IMetadataPageManagerFactory freePageManagerFactory, = boolean updateAware, ITracer tracer) throws HyracksDataException { = LSMBTreeTupleWriterFactory insertTupleWriterFactory =3D - = new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories=2Elength, fals= e, updateAware); + new LSMBTreeTupleWriterFactory(typeTraits= , cmpFactories=2Elength, false, updateAware, null, null); LSMBTree= TupleWriterFactory deleteTupleWriterFactory =3D - new LSMBTr= eeTupleWriterFactory(typeTraits, cmpFactories=2Elength, true, updateAware);= + new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories= =2Elength, true, updateAware, null, null); LSMBTreeCopyTupleWriter= Factory copyTupleWriterFactory =3D - new LSMBTreeCopyTupleWr= iterFactory(typeTraits, cmpFactories=2Elength, updateAware); + = new LSMBTreeCopyTupleWriterFactory(typeTraits, cmpFactories=2Elength, up= dateAware, null, null); ITreeIndexFrameFactory insertLeafFrameFact= ory =3D new BTreeNSMLeafFrameFactory(insertTupleWriterFactory); IT= reeIndexFrameFactory copyTupleLeafFrameFactory =3D new BTreeNSMLeafFrameFac= tory(copyTupleWriterFactory); ITreeIndexFrameFactory deleteLeafFra= meFactory =3D new BTreeNSMLeafFrameFactory(deleteTupleWriterFactory); @@ -8= 8,7 +88,8 @@ LSMComponentFilterFrameFactory filterFrameFactory =3D= null; LSMComponentFilterManager filterManager =3D null; = if (filterCmpFactories !=3D null) { - TypeAwareTupleWriterFactor= y filterTupleWriterFactory =3D new TypeAwareTupleWriterFactory(filterTypeTr= aits); + TypeAwareTupleWriterFactory filterTupleWriterFactory = =3D + new TypeAwareTupleWriterFactory(filterTypeTraits, = null, null); filterHelper =3D new ComponentFilterHelper(filter= TupleWriterFactory, filterCmpFactories); filterFrameFactory = =3D new LSMComponentFilterFrameFactory(filterTupleWriterFactory); = filterManager =3D new LSMComponentFilterManager(filterFrameFactory); di= ff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-b= tree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/multithread= /LSMBTreeMultiThreadTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/h= yracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/a= m/lsm/btree/multithread/LSMBTreeMultiThreadTest=2Ejava index fb1244f=2E=2E6= 6af4b2 100644 --- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage= -am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/mu= ltithread/LSMBTreeMultiThreadTest=2Ejava +++ b/hyracks-fullstack/hyracks/hy= racks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyra= cks/storage/am/lsm/btree/multithread/LSMBTreeMultiThreadTest=2Ejava @@ -60,= 7 +60,7 @@ harness=2EgetOperationTracker(), harness=2EgetI= OScheduler(), harness=2EgetIOOperationCallbackFactory(), h= arness=2EgetPageWriteCallbackFactory(), true, null, null, null, null, true,= harness=2EgetMetadataPageManagerFactory(), false, ITracer= =2ENONE, - NoOpCompressorDecompressorFactory=2EINSTANCE, blo= omFilterKeyFields !=3D null); + NoOpCompressorDecompressorFa= ctory=2EINSTANCE, bloomFilterKeyFields !=3D null, null, null); } = @Override diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-st= orage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btr= ee/perf/BTreeRunner=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyrack= s-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm= /btree/perf/BTreeRunner=2Ejava index 60a054e=2E=2Ebaa5508 100644 --- a/hyra= cks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/t= est/java/org/apache/hyracks/storage/am/lsm/btree/perf/BTreeRunner=2Ejava ++= + b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-te= st/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/BTreeRunner= =2Ejava @@ -49,6 +49,6 @@ ITreeIndexMetadataFrameFactory metaFrame= Factory =3D new LIFOMetaDataFrameFactory(); LinkedMetaDataPageMana= ger freePageManager =3D new LinkedMetaDataPageManager(bufferCache, metaFram= eFactory); btree =3D BTreeUtils=2EcreateBTree(bufferCache, typeTra= its, cmpFactories, BTreeLeafFrameType=2EREGULAR_NSM, file, - = freePageManager, false); + freePageManager, false, null, nu= ll); } } diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks= -storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/= btree/perf/ConcurrentSkipListRunner=2Ejava b/hyracks-fullstack/hyracks/hyra= cks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyrack= s/storage/am/lsm/btree/perf/ConcurrentSkipListRunner=2Ejava index c476a8c= =2E=2Ec57badc 100644 --- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-= storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/b= tree/perf/ConcurrentSkipListRunner=2Ejava +++ b/hyracks-fullstack/hyracks/h= yracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyr= acks/storage/am/lsm/btree/perf/ConcurrentSkipListRunner=2Ejava @@ -116,7 +1= 16,7 @@ this=2EdataGen =3D dataGen; this=2EnumBat= ches =3D numBatches; this=2EskipList =3D skipList; - = tupleWriterFactory =3D new TypeAwareTupleWriterFactory(typeTraits); + = tupleWriterFactory =3D new TypeAwareTupleWriterFactory(typeTraits, = null, null); tupleWriter =3D tupleWriterFactory=2EcreateTupleW= riter(); int numTuples =3D numBatches * batchSize; = tuples =3D new TypeAwareTupleReference[numTuples]; diff --git a/hyracks-f= ullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/j= ava/org/apache/hyracks/storage/am/lsm/btree/perf/InMemoryBTreeRunner=2Ejava= b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-tes= t/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/InMemoryBTreeR= unner=2Ejava index 3307093=2E=2E1eb4646 100644 --- a/hyracks-fullstack/hyra= cks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apach= e/hyracks/storage/am/lsm/btree/perf/InMemoryBTreeRunner=2Ejava +++ b/hyrack= s-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/tes= t/java/org/apache/hyracks/storage/am/lsm/btree/perf/InMemoryBTreeRunner=2Ej= ava @@ -68,7 +68,8 @@ protected void init(int pageSize, int numPages, = ITypeTraits[] typeTraits, IBinaryComparatorFactory[] cmpFactories) = throws HyracksDataException { bufferCache =3D new VirtualBuff= erCache(new HeapBufferAllocator(), pageSize, numPages); - BTreeTypeA= wareTupleWriterFactory tupleWriterFactory =3D new BTreeTypeAwareTupleWriter= Factory(typeTraits, false); + BTreeTypeAwareTupleWriterFactory tuple= WriterFactory =3D + new BTreeTypeAwareTupleWriterFactory(typ= eTraits, false, null, null); ITreeIndexFrameFactory leafFrameFacto= ry =3D new BTreeNSMLeafFrameFactory(tupleWriterFactory); ITreeInde= xFrameFactory interiorFrameFactory =3D new BTreeNSMInteriorFrameFactory(tup= leWriterFactory); IPageManager freePageManager =3D new VirtualFree= PageManager(bufferCache); diff --git a/hyracks-fullstack/hyracks/hyracks-te= sts/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/stor= age/am/lsm/btree/perf/InMemorySortRunner=2Ejava b/hyracks-fullstack/hyracks= /hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/h= yracks/storage/am/lsm/btree/perf/InMemorySortRunner=2Ejava index 4501485=2E= =2E1455fb3 100644 --- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-sto= rage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btre= e/perf/InMemorySortRunner=2Ejava +++ b/hyracks-fullstack/hyracks/hyracks-te= sts/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/stor= age/am/lsm/btree/perf/InMemorySortRunner=2Ejava @@ -71,7 +71,7 @@ = this=2EbatchSize =3D batchSize; this=2EtypeTraits =3D typeTraits; = tupleCmp =3D new TupleComparator(cmp); - tupleWriterFactory= =3D new TypeAwareTupleWriterFactory(typeTraits); + tupleWriterFacto= ry =3D new TypeAwareTupleWriterFactory(typeTraits, null, null); tu= pleWriter =3D tupleWriterFactory=2EcreateTupleWriter(); int numTup= les =3D numBatches * batchSize; tuples =3D new ArrayList(); @@ -131,7 +131,7 @@ this=2EdataGen =3D data= Gen; this=2EnumBatches =3D numBatches; this=2Eski= pList =3D skipList; - tupleWriterFactory =3D new TypeAwareTupleW= riterFactory(typeTraits); + tupleWriterFactory =3D new TypeAware= TupleWriterFactory(typeTraits, null, null); tupleWriter =3D tu= pleWriterFactory=2EcreateTupleWriter(); int numTuples =3D numB= atches * batchSize; tuples =3D new TypeAwareTupleReference[num= Tuples]; diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-stora= ge-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/= perf/LSMTreeRunner=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks= -storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/= btree/perf/LSMTreeRunner=2Ejava index f487bf1=2E=2E1b46fd7 100644 --- a/hyr= acks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/= test/java/org/apache/hyracks/storage/am/lsm/btree/perf/LSMTreeRunner=2Ejava= +++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree= -test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/LSMTreeRun= ner=2Ejava @@ -128,7 +128,7 @@ new ThreadCountingTracker()= , ioScheduler, NoOpIOOperationCallbackFactory=2EINSTANCE, = NoOpPageWriteCallbackFactory=2EINSTANCE, true, null, null, null, null, true= , TestStorageManagerComponentHolder=2EgetMetadataPageManag= erFactory(), false, ITracer=2ENONE, - NoOpCompressorDecompre= ssorFactory=2EINSTANCE, bloomFilterKeyFields !=3D null); + N= oOpCompressorDecompressorFactory=2EINSTANCE, bloomFilterKeyFields !=3D null= , null, null); } @Override diff --git a/hyracks-fullstack/hyrac= ks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache= /hyracks/storage/am/lsm/btree/tuples/LSMBTreeTuplesTest=2Ejava b/hyracks-fu= llstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/ja= va/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTuplesTest=2Ejava= index b81167b=2E=2E0a34c45 100644 --- a/hyracks-fullstack/hyracks/hyracks-= tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/st= orage/am/lsm/btree/tuples/LSMBTreeTuplesTest=2Ejava +++ b/hyracks-fullstack= /hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/= apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTuplesTest=2Ejava @@ -68= ,7 +68,7 @@ for (int numFields =3D numKeyFields; numFields <= =3D maxFieldSerdes=2Elength; numFields++) { // Create and = write tuple to bytes using an LSMBTreeTupleWriter=2E LSMBT= reeTupleWriter maxMatterTupleWriter =3D - new LSMBTr= eeTupleWriter(maxTypeTraits, numKeyFields, false, false); + = new LSMBTreeTupleWriter(maxTypeTraits, numKeyFields, false, false, = null, null); ITupleReference maxTuple =3D TupleUtils=2Ecre= ateTuple(maxFieldSerdes, (Object[]) maxFields); ByteBuffer= maxMatterBuf =3D writeTuple(maxTuple, maxMatterTupleWriter); = // Tuple reference should work for both matter and antimatter tuples (d= oesn't matter which factory creates it)=2E @@ -84,10 +84,12 @@ = } // Create and write tuple to bytes using an LSMBTre= eTupleWriter=2E ITupleReference tuple =3D TupleUtils=2Ecre= ateTuple(fieldSerdes, (Object[]) fields); - LSMBTreeTupleWri= ter matterTupleWriter =3D new LSMBTreeTupleWriter(typeTraits, numKeyFields,= false, false); + LSMBTreeTupleWriter matterTupleWriter =3D = + new LSMBTreeTupleWriter(typeTraits, numKeyFields, = false, false, null, null); LSMBTreeTupleWriter antimatterT= upleWriter =3D - new LSMBTreeTupleWriter(typeTraits,= numKeyFields, true, false); - LSMBTreeCopyTupleWriter copyT= upleWriter =3D new LSMBTreeCopyTupleWriter(typeTraits, numKeyFields, false)= ; + new LSMBTreeTupleWriter(typeTraits, numKeyFields= , true, false, null, null); + LSMBTreeCopyTupleWriter copyTu= pleWriter =3D + new LSMBTreeCopyTupleWriter(typeTrai= ts, numKeyFields, false, null, null); ByteBuffer matterBuf= =3D writeTuple(tuple, matterTupleWriter); ByteBuffer anti= matterBuf =3D writeTuple(tuple, antimatterTupleWriter); diff --git a/hyra= cks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/t= est/java/org/apache/hyracks/storage/am/lsm/btree/util/LSMBTreeTestContext= =2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-bt= ree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/util/LSMBTre= eTestContext=2Ejava index 6da04e7=2E=2E928f60c 100644 --- a/hyracks-fullsta= ck/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/or= g/apache/hyracks/storage/am/lsm/btree/util/LSMBTreeTestContext=2Ejava +++ b= /hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/= src/test/java/org/apache/hyracks/storage/am/lsm/btree/util/LSMBTreeTestCont= ext=2Ejava @@ -103,7 +103,7 @@ cmpFactories, bloomFilt= erKeyFields, bloomFilterFalsePositiveRate, mergePolicy, opTracker, = ioScheduler, ioOpCallbackFactory, pageWriteCallbackFactory, ne= edKeyDupCheck, filterTypeTraits, filterCmp, btreefield= s, filterfields, true, metadataPageManagerFactory, updateAware, ITracer=2EN= ONE, - NoOpCompressorDecompressorFactory=2EINSTANCE, tru= e); + NoOpCompressorDecompressorFactory=2EINSTANCE, true= , null, null); } else { lsmTree =3D LSMBTreeUtil=2Ecr= eateLSMTree(ioManager, virtualBufferCaches, file, diskBufferCache, typeTrai= ts, cmpFactories, bloomFilterKeyFields, bloomFilterFal= sePositiveRate, mergePolicy, opTracker, @@ -111,7 +111,7 @@ = true, metadataPageManagerFactory, updateAware, n= ew Tracer(LSMBTreeTestContext=2Eclass=2EgetSimpleName(), = ITraceCategoryRegistry=2ECATEGORIES_ALL, new TraceCategoryRegist= ry()), - NoOpCompressorDecompressorFactory=2EINSTANCE, t= rue); + NoOpCompressorDecompressorFactory=2EINSTANCE, tr= ue, null, null); } LSMBTreeTestContext testCtx =3D new LS= MBTreeTestContext(fieldSerdes, lsmTree, filtered); return testCtx;= diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-ls= m-common-test/src/test/java/org/apache/hyracks/storage/am/lsm/common/test/L= SMComponentFilterReferenceTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-t= ests/hyracks-storage-am-lsm-common-test/src/test/java/org/apache/hyracks/st= orage/am/lsm/common/test/LSMComponentFilterReferenceTest=2Ejava index 14320= 4f=2E=2E612ca4b 100644 --- a/hyracks-fullstack/hyracks/hyracks-tests/hyrack= s-storage-am-lsm-common-test/src/test/java/org/apache/hyracks/storage/am/ls= m/common/test/LSMComponentFilterReferenceTest=2Ejava +++ b/hyracks-fullstac= k/hyracks/hyracks-tests/hyracks-storage-am-lsm-common-test/src/test/java/or= g/apache/hyracks/storage/am/lsm/common/test/LSMComponentFilterReferenceTest= =2Ejava @@ -34,7 +34,7 @@ @Test public void test() throws Hyracks= DataException { LSMComponentFilterReference filter =3D new LSMComp= onentFilterReference( - new TypeAwareTupleWriter(new ITypeTr= aits[] { IntegerPointable=2ETYPE_TRAITS })); + new TypeAware= TupleWriter(new ITypeTraits[] { IntegerPointable=2ETYPE_TRAITS }, null, nul= l)); Assert=2EassertEquals(filter=2EgetLength(), 0); Asse= rt=2EassertFalse(filter=2EisMaxTupleSet() || filter=2EisMinTupleSet()); = filter=2EwriteMaxTuple(TupleUtils=2EcreateIntegerTuple(false, Integer= =2EMAX_VALUE)); @@ -46,7 +46,7 @@ Assert=2EassertTrue(filter=2Eget= Length() =3D=3D 20); byte[] serFilter =3D filter=2EgetByteArray();= LSMComponentFilterReference deserFilter =3D new LSMComponentFilte= rReference( - new TypeAwareTupleWriter((new ITypeTraits[] { = IntegerPointable=2ETYPE_TRAITS }))); + new TypeAwareTupleWri= ter(new ITypeTraits[] { IntegerPointable=2ETYPE_TRAITS }, null, null)); = deserFilter=2Eset(serFilter, 0, 20); Assert=2EassertTrue(des= erFilter=2EisMaxTupleSet() && deserFilter=2EisMinTupleSet()); Asse= rt=2EassertEquals( -- To view, visit https://asterix-gerrit=2Eics=2Euci= =2Eedu/c/asterixdb/+/11944 To unsubscribe, or for help writing mail filters= , visit https://asterix-gerrit=2Eics=2Euci=2Eedu/settings Gerrit-Project: = asterixdb Gerrit-Branch: master Gerrit-Change-Id: I5fc9c40af68c7b6aaa54ecb4= 40c8fe1e90046231 Gerrit-Change-Number: 11944 Gerrit-PatchSet: 1 Gerrit-Owne= r: Ali Alsuliman Gerrit-MessageType: newc= hange --nYDT6NJUqDk= Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Ali Alsuliman has uploaded this change for review=2E

View Change

[ASTERIXDB-2791][IDX] Set null flags when writing an inde=
x entry

- user model changes: no
- storage format changes: no
=
- interface changes: yes

Details:
When writing an index entry, se=
t the corresponding null flags
for keys that are considered null=2E When=
 reading an index entry,
examine the null flags to read the index entry =
properly=2E

 - add INullIntrospector interface that can be used by H=
yracks
   to know if a key is null or not=2E

Change-Id: I5fc9c40a=
f68c7b6aaa54ecb440c8fe1e90046231
---
M asterixdb/asterix-app/src/main=
/java/org/apache/asterix/app/io/PersistedResourceRegistry=2Ejava
M aster=
ixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNod=
e=2Ejava
M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/m=
etadata/bootstrap/MetadataBootstrap=2Ejava
M asterixdb/asterix-metadata/=
src/main/java/org/apache/asterix/metadata/declared/ArrayBTreeResourceFactor=
yProvider=2Ejava
M asterixdb/asterix-metadata/src/main/java/org/apache/a=
sterix/metadata/declared/BTreeResourceFactoryProvider=2Ejava
M asterixdb=
/asterix-metadata/src/main/java/org/apache/asterix/metadata/valueextractors=
/TupleCopyValueExtractor=2Ejava
A asterixdb/asterix-om/src/main/java/org=
/apache/asterix/formats/nontagged/NullIntrospector=2Ejava
M asterixdb/as=
terix-om/src/main/java/org/apache/asterix/formats/nontagged/TypeTraitProvid=
er=2Ejava
M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integrati=
on-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexS=
canOperatorTest=2Ejava
M hyracks-fullstack/hyracks/hyracks-examples/hyra=
cks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTree=
PrimaryIndexSearchOperatorTest=2Ejava
M hyracks-fullstack/hyracks/hyrack=
s-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests=
/am/btree/BTreePrimaryIndexStatsOperatorTest=2Ejava
M hyracks-fullstack/=
hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache=
/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest=2Ejava
M h=
yracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/tes=
t/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorT=
est=2Ejava
M hyracks-fullstack/hyracks/hyracks-examples/hyracks-integrat=
ion-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryInd=
exUpsertOperatorTest=2Ejava
M hyracks-fullstack/hyracks/hyracks-examples=
/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/lsm/bt=
ree/LSMBTreeOperatorTestHelper=2Ejava
M hyracks-fullstack/hyracks/hyrack=
s-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests=
/am/rtree/AbstractRTreeOperatorTest=2Ejava
M hyracks-fullstack/hyracks/h=
yracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/c=
ompressors/FieldPrefixCompressor=2Ejava
M hyracks-fullstack/hyracks/hyra=
cks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/data=
flow/BTreeResource=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am=
-btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeReso=
urceFactory=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-btree/=
src/main/java/org/apache/hyracks/storage/am/btree/frames/BTreeFieldPrefixNS=
MLeafFrame=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-btree/s=
rc/main/java/org/apache/hyracks/storage/am/btree/impls/FieldPrefixPrefixTup=
leReference=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-btree/=
src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupl=
eReference=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-btree/s=
rc/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTuple=
Writer=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/m=
ain/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupleWrit=
erFactory=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-btree/sr=
c/main/java/org/apache/hyracks/storage/am/btree/util/BTreeUtils=2Ejava
A=
 hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apac=
he/hyracks/storage/am/common/api/INullIntrospector=2Ejava
M hyracks-full=
stack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/st=
orage/am/common/tuples/SimpleTupleWriter=2Ejava
M hyracks-fullstack/hyra=
cks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/c=
ommon/tuples/TypeAwareTupleReference=2Ejava
M hyracks-fullstack/hyracks/=
hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/commo=
n/tuples/TypeAwareTupleWriter=2Ejava
M hyracks-fullstack/hyracks/hyracks=
-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/tuple=
s/TypeAwareTupleWriterFactory=2Ejava
M hyracks-fullstack/hyracks/hyracks=
-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree=
/dataflow/ExternalBTreeLocalResource=2Ejava
M hyracks-fullstack/hyracks/=
hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/ls=
m/btree/dataflow/ExternalBTreeLocalResourceFactory=2Ejava
M hyracks-full=
stack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks=
/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResource=2EjavaM hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/or=
g/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalR=
esourceFactory=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm=
-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSMBT=
reeLocalResource=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-l=
sm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSM=
BTreeLocalResourceFactory=2Ejava
M hyracks-fullstack/hyracks/hyracks-sto=
rage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tup=
les/LSMBTreeCopyTupleWriter=2Ejava
M hyracks-fullstack/hyracks/hyracks-s=
torage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/t=
uples/LSMBTreeCopyTupleWriterFactory=2Ejava
M hyracks-fullstack/hyracks/=
hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/ls=
m/btree/tuples/LSMBTreeTupleReference=2Ejava
M hyracks-fullstack/hyracks=
/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/l=
sm/btree/tuples/LSMBTreeTupleWriter=2Ejava
M hyracks-fullstack/hyracks/h=
yracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm=
/btree/tuples/LSMBTreeTupleWriterFactory=2Ejava
M hyracks-fullstack/hyra=
cks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/a=
m/lsm/btree/utils/LSMBTreeUtil=2Ejava
M hyracks-fullstack/hyracks/hyrack=
s-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/com=
mon/dataflow/LsmResource=2Ejava
M hyracks-fullstack/hyracks/hyracks-stor=
age-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/da=
taflow/LsmResourceFactory=2Ejava
M hyracks-fullstack/hyracks/hyracks-sto=
rage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/i=
nvertedindex/dataflow/LSMInvertedIndexLocalResource=2Ejava
M hyracks-ful=
lstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apach=
e/hyracks/storage/am/lsm/invertedindex/dataflow/LSMInvertedIndexLocalResour=
ceFactory=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inve=
rtedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/imp=
ls/LSMInvertedIndex=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-a=
m-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/inverte=
dindex/inmemory/InMemoryInvertedIndex=2Ejava
M hyracks-fullstack/hyracks=
/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/stor=
age/am/lsm/invertedindex/ondisk/OnDiskInvertedIndex=2Ejava
M hyracks-ful=
lstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apach=
e/hyracks/storage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeElem=
entInvertedListBuilder=2Ejava
M hyracks-fullstack/hyracks/hyracks-storag=
e-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/inve=
rtedindex/ondisk/variablesize/VariableSizeElementOnDiskInvertedListCursor=
=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex=
/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/varia=
blesize/VariableSizeInvertedListSearchResultFrameTupleAccessor=2Ejava
M =
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/jav=
a/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/variablesize/Varia=
bleSizeInvertedListTupleReference=2Ejava
M hyracks-fullstack/hyracks/hyr=
acks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/=
am/lsm/invertedindex/search/InvertedIndexSearchResult=2Ejava
M hyracks-f=
ullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apa=
che/hyracks/storage/am/lsm/invertedindex/util/InvertedIndexUtils=2Ejava
=
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/=
apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeLocalResource=2EjavaM hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/or=
g/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeLocalResourceFactory=
=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/mai=
n/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeWithAntiMat=
terLocalResource=2Ejava
M hyracks-fullstack/hyracks/hyracks-storage-am-l=
sm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSM=
RTreeWithAntiMatterLocalResourceFactory=2Ejava
M hyracks-fullstack/hyrac=
ks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am=
/lsm/rtree/tuples/LSMRTreeTupleReferenceForPointMBR=2Ejava
M hyracks-ful=
lstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyrack=
s/storage/am/lsm/rtree/tuples/LSMRTreeTupleWriterForPointMBR=2Ejava
M hy=
racks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apac=
he/hyracks/storage/am/lsm/rtree/utils/LSMRTreeUtils=2Ejava
M hyracks-ful=
lstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/st=
orage/am/rtree/tuples/RTreeTypeAwareTupleReference=2Ejava
M hyracks-full=
stack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/sto=
rage/am/rtree/tuples/RTreeTypeAwareTupleWriter=2Ejava
M hyracks-fullstac=
k/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/storage=
/am/rtree/tuples/RTreeTypeAwareTupleWriterFactory=2Ejava
M hyracks-fulls=
tack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/=
apache/hyracks/storage/am/btree/BTreeExamplesTest=2Ejava
M hyracks-fulls=
tack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/=
apache/hyracks/storage/am/btree/BTreeModificationOperationCallbackTest=2Eja=
va
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-te=
st/src/test/java/org/apache/hyracks/storage/am/btree/BTreeSearchCursorTest=
=2Ejava
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btr=
ee-test/src/test/java/org/apache/hyracks/storage/am/btree/BTreeSearchOperat=
ionCallbackTest=2Ejava
M hyracks-fullstack/hyracks/hyracks-tests/hyracks=
-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/BT=
reeStatsTest=2Ejava
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-st=
orage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/BTree=
UpdateSearchTest=2Ejava
M hyracks-fullstack/hyracks/hyracks-tests/hyrack=
s-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/F=
ieldPrefixNSMTest=2Ejava
M hyracks-fullstack/hyracks/hyracks-tests/hyrac=
ks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/=
multithread/BTreeMultiThreadTest=2Ejava
M hyracks-fullstack/hyracks/hyra=
cks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/st=
orage/am/btree/util/BTreeTestContext=2Ejava
M hyracks-fullstack/hyracks/=
hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hy=
racks/storage/am/lsm/btree/LSMBTreeExamplesTest=2Ejava
M hyracks-fullsta=
ck/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/or=
g/apache/hyracks/storage/am/lsm/btree/LSMBTreeModificationOperationCallback=
Test=2Ejava
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am=
-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBT=
reeSearchOperationCallbackTest=2Ejava
M hyracks-fullstack/hyracks/hyrack=
s-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/=
storage/am/lsm/btree/LSMBTreeUpdateInPlaceTest=2Ejava
M hyracks-fullstac=
k/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org=
/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeLocalResource=2Ejava<=
br>M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-t=
est/src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtree=
LocalResourceFactory=2Ejava
M hyracks-fullstack/hyracks/hyracks-tests/hy=
racks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am=
/lsm/btree/impl/TestLsmBtreeUtil=2Ejava
M hyracks-fullstack/hyracks/hyra=
cks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyrack=
s/storage/am/lsm/btree/multithread/LSMBTreeMultiThreadTest=2Ejava
M hyra=
cks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/t=
est/java/org/apache/hyracks/storage/am/lsm/btree/perf/BTreeRunner=2EjavaM hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-tes=
t/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/ConcurrentSkip=
ListRunner=2Ejava
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-stor=
age-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree=
/perf/InMemoryBTreeRunner=2Ejava
M hyracks-fullstack/hyracks/hyracks-tes=
ts/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/stora=
ge/am/lsm/btree/perf/InMemorySortRunner=2Ejava
M hyracks-fullstack/hyrac=
ks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache=
/hyracks/storage/am/lsm/btree/perf/LSMTreeRunner=2Ejava
M hyracks-fullst=
ack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/o=
rg/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTuplesTest=2Ejava
=
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test=
/src/test/java/org/apache/hyracks/storage/am/lsm/btree/util/LSMBTreeTestCon=
text=2Ejava
M hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am=
-lsm-common-test/src/test/java/org/apache/hyracks/storage/am/lsm/common/tes=
t/LSMComponentFilterReferenceTest=2Ejava
90 files changed, 506 insertion=
s(+), 194 deletions(-)

git pull ssh://asterix-gerrit=2Eics=2Euci=
=2Eedu:29418/asterixdb refs/changes/44/11944/1
diff --git a/asterix=
db/asterix-app/src/main/java/org/apache/asterix/app/io/PersistedResourceReg=
istry=2Ejava b/asterixdb/asterix-app/src/main/java/org/apache/asterix/app/i=
o/PersistedResourceRegistry=2Ejava
index 4cc6e43=2E=2E35ee6=
dd 100644
--- a/asterixdb/asterix-app/src/main/java/org/apa=
che/asterix/app/io/PersistedResourceRegistry=2Ejava
+++ b/a=
sterixdb/asterix-app/src/main/java/org/apache/asterix/app/io/PersistedResou=
rceRegistry=2Ejava
@@ -53,6 +53,7 @@
 impor=
t org=2Eapache=2Easterix=2Edataflow=2Edata=2Enontagged=2Ecomparators=2EList=
ItemBinaryComparatorFactory;
 import org=2Eapache=2Easterix=
=2Edataflow=2Edata=2Enontagged=2Evalueproviders=2EPrimitiveValueProviderFac=
tory;
 import org=2Eapache=2Easterix=2Eformats=2Enontagged=
=2EAnyBinaryComparatorFactory;
+import org=2Eapache=2Easterix=2Eformats=2Enontagged=2ENullIntros=
pector;
 import org=2Eapache=2Easterix=2Eformats=2Enontagge=
d=2EOrderedBinaryComparatorFactory;
 import org=2Eapache=2E=
asterix=2Eformats=2Enontagged=2EOrderedLinearizeComparatorFactory; import org=2Eapache=2Easterix=2Emetadata=2Eutils=2ESecondaryCorrel=
atedTreeIndexOperationsHelper;
@@ -175,6 +176,9 @@         registeredClasses=2Eput("FixedLengthTypeTrait", =
FixedLengthTypeTrait=2Eclass);
         registeredClasses=
=2Eput("VarLengthTypeTrait", VarLengthTypeTrait=2Eclass);<=
br> 
+        /=
/ INullIntrospector
+=
        registeredClasses=2Eput("NullIntrospector", NullIntrospec=
tor=2Eclass);
+
         // ILSMOperationTrackerFactory
        =
 registeredClasses=2Eput("PrimaryIndexOperationTrackerFactory", P=
rimaryIndexOperationTrackerFactory=2Eclass);
         regis=
teredClasses=2Eput("SecondaryIndexOperationTrackerFactory", Secon=
daryIndexOperationTrackerFactory=2Eclass);
diff --git a/ast=
erixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataN=
ode=2Ejava b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/me=
tadata/MetadataNode=2Ejava
index f8ba574=2E=2E9975f4c 10064=
4
--- a/asterixdb/asterix-metadata/src/main/java/org/apache=
/asterix/metadata/MetadataNode=2Ejava
+++ b/asterixdb/aster=
ix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode=2Ejava
@@ -50,7 +50,9 @@
 import org=2Eapache=2East=
erix=2Ecommon=2Etransactions=2ETxnId;
 import org=2Eapache=
=2Easterix=2Ecommon=2Eutils=2EStoragePathUtil;
 import org=
=2Eapache=2Easterix=2Eexternal=2Eindexing=2EExternalFile;
+import org=2Eapache=2Easterix=2Eforma=
ts=2Enontagged=2ENullIntrospector;
 import org=2Eapache=2Ea=
sterix=2Eformats=2Enontagged=2ESerializerDeserializerProvider;
+import org=2Eapache=2Easterix=2E=
formats=2Enontagged=2ETypeTraitProvider;
 import org=2Eapac=
he=2Easterix=2Emetadata=2Eapi=2EExtensionMetadataDataset;
 =
import org=2Eapache=2Easterix=2Emetadata=2Eapi=2EExtensionMetadataDatasetId=
;
 import org=2Eapache=2Easterix=2Emetadata=2Eapi=2EIExtens=
ionMetadataEntity;
@@ -1539,7 +1541,8 @@
 <=
/span>
     private ITupleReference getTupleToBeDeleted(TxnId txnI=
d, IMetadataIndex metadataIndex, ITupleReference searchKey)
             throws AlgebricksException, HyracksDataException {
=
-        IValueExtractor<ITupl=
eReference> valueExtractor =3D new TupleCopyValueExtractor(metadataIndex=
=2EgetTypeTraits());
=
+        IValueExtractor<ITupleReference> valueExtractor =3D new Tupl=
eCopyValueExtractor(metadataIndex=2EgetTypeTraits(),
+                TypeTraitProvider=2EINSTA=
NCE=2EgetTypeTrait(BuiltinType=2EANULL), NullIntrospector=2EINSTANCE);
         List<ITupleReference> results =3D new ArrayList&=
lt;>();
         searchIndex(txnId, metadataIndex, searc=
hKey, valueExtractor, results);
         if (results=2EisEm=
pty()) {
diff --git a/asterixdb/asterix-metadata/src/main/j=
ava/org/apache/asterix/metadata/bootstrap/MetadataBootstrap=2Ejava b/asteri=
xdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/Me=
tadataBootstrap=2Ejava
index 047b823=2E=2E125db21 100644
--- a/asterixdb/asterix-metadata/src/main/java/org/apache/ast=
erix/metadata/bootstrap/MetadataBootstrap=2Ejava
+++ b/aste=
rixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/=
MetadataBootstrap=2Ejava
@@ -45,6 +45,8 @@
=
 import org=2Eapache=2Easterix=2Eexternal=2Eapi=2EITypedAdapterFactory;
 import org=2Eapache=2Easterix=2Eexternal=2Edataset=2Eadapter=
=2EAdapterIdentifier;
 import org=2Eapache=2Easterix=2Eexte=
rnal=2Eindexing=2EExternalFile;
+import org=2Eapache=2Easterix=2Eformats=2Enontagged=2ENullIntro=
spector;
+import org=
=2Eapache=2Easterix=2Eformats=2Enontagged=2ETypeTraitProvider;
 import org=2Eapache=2Easterix=2Emetadata=2EIDatasetDetails;
=
 import org=2Eapache=2Easterix=2Emetadata=2EMetadataManager; import org=2Eapache=2Easterix=2Emetadata=2EMetadataNode;@@ -66,6 +68,7 @@
 import org=2Eapache=2Easterix=2Em=
etadata=2Efeeds=2EBuiltinFeedPolicies;
 import org=2Eapache=
=2Easterix=2Emetadata=2Eutils=2EMetadataConstants;
 import =
org=2Eapache=2Easterix=2Emetadata=2Eutils=2EMetadataUtil;
+import org=2Eapache=2Easterix=2Eom=2E=
types=2EBuiltinType;
 import org=2Eapache=2Easterix=2Eom=2E=
types=2EIAType;
 import org=2Eapache=2Easterix=2Eruntime=2E=
formats=2ENonTaggedDataFormat;
 import org=2Eapache=2Easter=
ix=2Etransaction=2Emanagement=2Eopcallbacks=2EPrimaryIndexOperationTrackerF=
actory;
@@ -428,7 +431,8 @@
               =
              storageComponentProvider=2EgetIoOperationSchedulerProvider(),=

                             appContext=2EgetMetadataMerge=
PolicyFactory(),
                             StorageConsta=
nts=2EDEFAULT_COMPACTION_POLICY_PROPERTIES, true, bloomFilterKeyFields,
-                         =
   bloomFilterFalsePositiveRate, true, null, NoOpCompressorDecompressorFact=
ory=2EINSTANCE, true);
+                            TypeTraitProvider=
=2EINSTANCE=2EgetTypeTrait(BuiltinType=2EANULL), NullIntrospector=2EINSTANC=
E);
             DatasetLocalResourceFactory dsLocalResourc=
eFactory =3D
                     new DatasetLocalResourceF=
actory(datasetId, lsmBtreeFactory);
             // TODO(am=
oudi) Creating the index should be done through the same code path as
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apache=
/asterix/metadata/declared/ArrayBTreeResourceFactoryProvider=2Ejava b/aster=
ixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/Ar=
rayBTreeResourceFactoryProvider=2Ejava
index 7c19282=2E=2E5=
654d16 100644
--- a/asterixdb/asterix-metadata/src/main/jav=
a/org/apache/asterix/metadata/declared/ArrayBTreeResourceFactoryProvider=2E=
java
+++ b/asterixdb/asterix-metadata/src/main/java/org/apa=
che/asterix/metadata/declared/ArrayBTreeResourceFactoryProvider=2Ejava
@@ -26,12 +26,14 @@
 import org=2Eapache=2Easte=
rix=2Ecommon=2Econtext=2EIStorageComponentProvider;
 import=
 org=2Eapache=2Easterix=2Ecommon=2Eexceptions=2ECompilationException;
 import org=2Eapache=2Easterix=2Ecommon=2Eexceptions=2EErrorCode=
;
+import org=2Eapach=
e=2Easterix=2Eformats=2Enontagged=2ENullIntrospector;
 impo=
rt org=2Eapache=2Easterix=2Emetadata=2Eapi=2EIResourceFactoryProvider;
 import org=2Eapache=2Easterix=2Emetadata=2Eentities=2EDataset;=

 import org=2Eapache=2Easterix=2Emetadata=2Eentities=2EInd=
ex;
 import org=2Eapache=2Easterix=2Emetadata=2Eutils=2EArr=
ayIndexUtil;
 import org=2Eapache=2Easterix=2Emetadata=2Eut=
ils=2EIndexUtil;
 import org=2Eapache=2Easterix=2Eom=2Etype=
s=2EARecordType;
+imp=
ort org=2Eapache=2Easterix=2Eom=2Etypes=2EBuiltinType;
 imp=
ort org=2Eapache=2Easterix=2Eom=2Etypes=2EIAType;
 import o=
rg=2Eapache=2Ehyracks=2Ealgebricks=2Ecommon=2Eexceptions=2EAlgebricksExcept=
ion;
 import org=2Eapache=2Ehyracks=2Ealgebricks=2Ecommon=
=2Eutils=2EPair;
@@ -66,6 +68,7 @@
        =
 int[] filterFields =3D IndexUtil=2EgetFilterFields(dataset, index, filterT=
ypeTraits);
         int[] btreeFields =3D IndexUtil=2EgetB=
treeFieldsIfFiltered(dataset, index);
         IStorageComp=
onentProvider storageComponentProvider =3D mdProvider=2EgetStorageComponent=
Provider();
+        =
ITypeTraitProvider typeTraitProvider =3D storageComponentProvider=2EgetType=
TraitProvider();
         ITypeTraits[] typeTraits =3D getT=
ypeTraits(mdProvider, dataset, index, recordType, metaType);
         IBinaryComparatorFactory[] cmpFactories =3D getCmpFactories(mdPr=
ovider, dataset, index, recordType, metaType);
         dou=
ble bloomFilterFalsePositiveRate =3D mdProvider=2EgetStorageProperties()=2E=
getBloomFilterFalsePositiveRate();
@@ -97,7 +100,8 @@
                         filterCmpFactories, filterFields, opTra=
ckerFactory, ioOpCallbackFactory,
                         =
pageWriteCallbackFactory, metadataPageManagerFactory, vbcProvider, ioSchedu=
lerProvider,
                         mergePolicyFactory, m=
ergePolicyProperties, true, null, bloomFilterFalsePositiveRate,
<=
span style=3D"color: hsl(0, 100%, 40%);">-                        index=2Ei=
sPrimaryIndex(), btreeFields, compDecompFactory, false);
+                        index=2EisPrim=
aryIndex(), btreeFields, compDecompFactory, false,
+                        typeTraitProvider=2E=
getTypeTrait(BuiltinType=2EANULL), NullIntrospector=2EINSTANCE);
=
             default:
                 throw new Comp=
ilationException(ErrorCode=2ECOMPILATION_UNKNOWN_DATASET_TYPE,
                         dataset=2EgetDatasetType()=2EtoString());
diff --git a/asterixdb/asterix-metadata/src/main/java/org/apach=
e/asterix/metadata/declared/BTreeResourceFactoryProvider=2Ejava b/asterixdb=
/asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/BTreeR=
esourceFactoryProvider=2Ejava
index 7f56f2c=2E=2E6276044 10=
0644
--- a/asterixdb/asterix-metadata/src/main/java/org/apa=
che/asterix/metadata/declared/BTreeResourceFactoryProvider=2Ejava+++ b/asterixdb/asterix-metadata/src/main/java/org/apache/asterix/me=
tadata/declared/BTreeResourceFactoryProvider=2Ejava
@@ -28,=
11 +28,13 @@
 import org=2Eapache=2Easterix=2Ecommon=2Eexce=
ptions=2EErrorCode;
 import org=2Eapache=2Easterix=2Eextern=
al=2Eindexing=2EFilesIndexDescription;
 import org=2Eapache=
=2Easterix=2Eexternal=2Eindexing=2EIndexingConstants;
+import org=2Eapache=2Easterix=2Eformats=
=2Enontagged=2ENullIntrospector;
 import org=2Eapache=2East=
erix=2Emetadata=2Eapi=2EIResourceFactoryProvider;
 import o=
rg=2Eapache=2Easterix=2Emetadata=2Eentities=2EDataset;
 imp=
ort org=2Eapache=2Easterix=2Emetadata=2Eentities=2EIndex;
 =
import org=2Eapache=2Easterix=2Emetadata=2Eutils=2EIndexUtil;
 import org=2Eapache=2Easterix=2Eom=2Etypes=2EARecordType;
+import org=2Eapache=2Easterix=2Eo=
m=2Etypes=2EBuiltinType;
 import org=2Eapache=2Easterix=2Eo=
m=2Etypes=2EIAType;
 import org=2Eapache=2Ehyracks=2Ealgebr=
icks=2Ecommon=2Eexceptions=2EAlgebricksException;
 import o=
rg=2Eapache=2Ehyracks=2Ealgebricks=2Ecommon=2Eutils=2EPair;
@@ -82,6 +84,7 @@
         ILSMIOOperationSchedulerProvide=
r ioSchedulerProvider =3D
                 storageComponent=
Provider=2EgetIoOperationSchedulerProvider();
         bool=
ean hasBloomFilter =3D bloomFilterFields !=3D null;
+        ITypeTraitProvider typeTraitProvi=
der =3D mdProvider=2EgetDataFormat()=2EgetTypeTraitProvider();
         switch (dataset=2EgetDatasetType()) {
        =
     case EXTERNAL:
                 return index=2EgetInde=
xName()=2Eequals(IndexingConstants=2EgetFilesIndexName(dataset=2EgetDataset=
Name()))
@@ -89,12 +92,14 @@
              =
                   filterTypeTraits, filterCmpFactories, filterFields, opTr=
ackerFactory,
                                 ioOpCallback=
Factory, pageWriteCallbackFactory, metadataPageManagerFactory,
                                 ioSchedulerProvider, mergePolicyFactor=
y, mergePolicyProperties, true, bloomFilterFields,
-                                bloomFilterFal=
sePositiveRate, false, btreeFields, hasBloomFilter)
+                                bloomFilt=
erFalsePositiveRate, false, btreeFields, hasBloomFilter,
+                                typeTr=
aitProvider=2EgetTypeTrait(BuiltinType=2EANULL), NullIntrospector=2EINSTANC=
E)
                         : new ExternalBTreeWithBuddyLoc=
alResourceFactory(storageManager, typeTraits, cmpFactories,
                                 filterTypeTraits, filterCmpFactories, fil=
terFields, opTrackerFactory,
                              =
   ioOpCallbackFactory, pageWriteCallbackFactory, metadataPageManagerFactor=
y,
                                 ioSchedulerProvider, me=
rgePolicyFactory, mergePolicyProperties, true, bloomFilterFields,-                               =
 bloomFilterFalsePositiveRate, false, btreeFields, hasBloomFilter);<=
br>+                           =
     bloomFilterFalsePositiveRate, false, btreeFields, hasBloomFilter,
+                        =
        typeTraitProvider=2EgetTypeTrait(BuiltinType=2EANULL), NullIntrospe=
ctor=2EINSTANCE);
             case INTERNAL:
                 AsterixVirtualBufferCacheProvider vbcProvider =3D
                         new AsterixVirtualBufferCacheProvider(d=
ataset=2EgetDatasetId());
@@ -112,7 +117,7 @@
                         pageWriteCallbackFactory, metadataPageManagerFa=
ctory, vbcProvider, ioSchedulerProvider,
                  =
       mergePolicyFactory, mergePolicyProperties, true, bloomFilterFields,<=
/span>
                         bloomFilterFalsePositiveRate, inde=
x=2EisPrimaryIndex(), btreeFields, compDecompFactory,
-                        hasBloomFilter);
+                      =
  hasBloomFilter, typeTraitProvider=2EgetTypeTrait(BuiltinType=2EANULL), Nu=
llIntrospector=2EINSTANCE);
             default:                 throw new CompilationException(ErrorCode=2ECOMPILAT=
ION_UNKNOWN_DATASET_TYPE,
                         dataset=
=2EgetDatasetType()=2EtoString());
diff --git a/asterixdb/a=
sterix-metadata/src/main/java/org/apache/asterix/metadata/valueextractors/T=
upleCopyValueExtractor=2Ejava b/asterixdb/asterix-metadata/src/main/java/or=
g/apache/asterix/metadata/valueextractors/TupleCopyValueExtractor=2Ejava
index c9fb39c=2E=2E9b19bb5 100644
--- a/aster=
ixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/valueextrac=
tors/TupleCopyValueExtractor=2Ejava
+++ b/asterixdb/asterix=
-metadata/src/main/java/org/apache/asterix/metadata/valueextractors/TupleCo=
pyValueExtractor=2Ejava
@@ -27,6 +27,7 @@
 =
import org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits;
 import org=2Eapache=2Ehyracks=2Eapi=2Eexceptions=2EHyracksDataE=
xception;
 import org=2Eapache=2Ehyracks=2Edataflow=2Ecommo=
n=2Edata=2Eaccessors=2EITupleReference;
+import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=
=2Eapi=2EINullIntrospector;
 import org=2Eapache=2Ehyracks=
=2Estorage=2Eam=2Ecommon=2Etuples=2ETypeAwareTupleReference;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Etuples=2ETypeAwa=
reTupleWriter;
 
@@ -42,8 +43,9 @@     private byte[] tupleBytes;
     private ByteBu=
ffer buf;
 
-    public TupleCopyValueExtractor(ITypeTraits[] typeTraits) {=

-        this=2EtupleWriter =
=3D new TypeAwareTupleWriter(typeTraits);
+    public TupleCopyValueExtractor(ITypeTraits[] type=
Traits, ITypeTraits nullTypeTraits,
+            INullIntrospector nullIntrospector) {+        this=2EtupleWriter =
=3D new TypeAwareTupleWriter(typeTraits, nullTypeTraits, nullIntrospector);=

         this=2EtupleReference =3D tupleWriter=2EcreateTup=
leReference();
     }
 
dif=
f --git a/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/non=
tagged/NullIntrospector=2Ejava b/asterixdb/asterix-om/src/main/java/org/apa=
che/asterix/formats/nontagged/NullIntrospector=2Ejava
new f=
ile mode 100644
index 0000000=2E=2Ef496efe
=
--- /dev/null
+++ b/asterixdb/asterix-om/src/main/java/org/=
apache/asterix/formats/nontagged/NullIntrospector=2Ejava
@@=
 -0,0 +1,55 @@
+/*
+ * Licensed to the Apa=
che Software Foundation (ASF) under one
+ * or more contributor license agreements=2E  See the N=
OTICE file
+ * distri=
buted with this work for additional information
+ * regarding copyright ownership=2E  The ASF li=
censes this file
+ * =
to you under the Apache License, Version 2=2E0 (the
+ * "License"); you may not use =
this file except in compliance
+ * with the License=2E  You may obtain a copy of the License at<=
/span>
+ *
+ *   http://www=2Eapache=2Eorg/licens=
es/LICENSE-2=2E0
+ *<=
/span>
+ * Unless required b=
y applicable law or agreed to in writing,
+ * software distributed under the License is distribu=
ted on an
+ * "A=
S IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied=2E=
  See the License for the
+ * specific language governing permissions and limitations
=
+ * under the License=2E=

+ */
+package org=2Eapache=2Easterix=2Eformats=
=2Enontagged;
+
+import static org=2Eapach=
e=2Easterix=2Eom=2Etypes=2EATypeTag=2ESERIALIZED_MISSING_TYPE_TAG;+import static org=2Eapache=
=2Easterix=2Eom=2Etypes=2EATypeTag=2ESERIALIZED_NULL_TYPE_TAG;
+
+import org=2Eapache=2Ehyracks=2Eapi=2Eexceptions=2E=
HyracksDataException;
+import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIJsonSerializable;
<=
span style=3D"color: hsl(120, 100%, 40%);">+import org=2Eapache=2Ehyracks=
=2Eapi=2Eio=2EIPersistedResourceRegistry;
+import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=
=2Eapi=2EINullIntrospector;
+
+import com=
=2Efasterxml=2Ejackson=2Edatabind=2EJsonNode;
+
+public class NullIntrospector implements INullIntrospector {<=
br>+
+    private static final long serialVersionUID=
 =3D 1L;
+
=
+    public static final INullI=
ntrospector INSTANCE =3D new NullIntrospector();
+
+    private NullIntrospector() {
+    }
+
+    @Overr=
ide
+    public boole=
an isNull(byte[] bytes, int offset, int length) {
+        return (bytes[offset] =3D=3D SERIALIZ=
ED_MISSING_TYPE_TAG || bytes[offset] =3D=3D SERIALIZED_NULL_TYPE_TAG)
+                &&=
; length =3D=3D 1;
+ =
   }
+
+    @Override
+    public JsonNode toJson(IPersistedRes=
ourceRegistry registry) throws HyracksDataException {
+        return registry=2EgetClassIdentif=
ier(getClass(), serialVersionUID);
+    }
=
+
+    @SuppressWarni=
ngs("squid:S1172") // unused parameter
+    public static IJsonSerializable fromJson(I=
PersistedResourceRegistry registry, JsonNode json) {
+        return INSTANCE;
+    }
+}
diff --git a/asterixdb/asterix-o=
m/src/main/java/org/apache/asterix/formats/nontagged/TypeTraitProvider=2Eja=
va b/asterixdb/asterix-om/src/main/java/org/apache/asterix/formats/nontagge=
d/TypeTraitProvider=2Ejava
index 1788445=2E=2E97fe9c1 10064=
4
--- a/asterixdb/asterix-om/src/main/java/org/apache/aster=
ix/formats/nontagged/TypeTraitProvider=2Ejava
+++ b/asterix=
db/asterix-om/src/main/java/org/apache/asterix/formats/nontagged/TypeTraitP=
rovider=2Ejava
@@ -28,6 +28,7 @@
      // WARNING: the byte sizes depend on the serializer!     // currently assuming a serializer that adds a 1-byte type indi=
cator before the data
+    private static final ITypeTraits ZERO_BYTE_TYPE_TRAIT =3D new FixedLe=
ngthTypeTrait(1);
     private static final ITypeTraits ONE=
_BYTE_TYPE_TRAIT =3D new FixedLengthTypeTrait(1 + 1);
     =
private static final ITypeTraits TWO_BYTE_TYPE_TRAIT =3D new FixedLengthTyp=
eTrait(2 + 1);
     private static final ITypeTraits FOUR_B=
YTE_TYPE_TRAIT =3D new FixedLengthTypeTrait(4 + 1);
@@ -48,=
6 +49,9 @@
             return null;
      =
   }
         switch (type=2EgetTypeTag()) {
+            case NULL:
<=
span style=3D"color: hsl(120, 100%, 40%);">+            case MISSING:
+                return ZE=
RO_BYTE_TYPE_TRAIT;
             case BOOLEAN:
             case TINYINT:
                 return ONE_=
BYTE_TYPE_TRAIT;
diff --git a/hyracks-fullstack/hyracks/hyr=
acks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/te=
sts/am/btree/BTreePrimaryIndexScanOperatorTest=2Ejava b/hyracks-fullstack/h=
yracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/=
hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest=2Ejava
<=
span>index 336204c=2E=2Ee4214cc 100644
--- a/hyracks-fullst=
ack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/ap=
ache/hyracks/tests/am/btree/BTreePrimaryIndexScanOperatorTest=2Ejava=

+++ b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integrat=
ion-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndex=
ScanOperatorTest=2Ejava
@@ -98,12 +98,12 @@
     @Override
     protected IResourceFactory createPrima=
ryResourceFactory() {
         return new BTreeResourceFact=
ory(storageManager, DataSetConstants=2EprimaryTypeTraits,
-                DataSetConstants=2Eprim=
aryComparatorFactories, pageManagerFactory);
+                DataSetConstants=2EprimaryComparat=
orFactories, pageManagerFactory, null, null);
     }=

 
     @Override
     protected I=
ResourceFactory createSecondaryResourceFactory() {
        =
 return new BTreeResourceFactory(storageManager, DataSetConstants=2Eseconda=
ryTypeTraits,
-        =
        DataSetConstants=2EsecondaryComparatorFactories, pageManagerFactory=
);
+                D=
ataSetConstants=2EsecondaryComparatorFactories, pageManagerFactory, null, n=
ull);
     }
 }
diff --git =
a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/=
test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSearchOperator=
Test=2Ejava b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integratio=
n-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndexSe=
archOperatorTest=2Ejava
index 870cfe3=2E=2E65bb2f0 100644
--- a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-int=
egration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimary=
IndexSearchOperatorTest=2Ejava
+++ b/hyracks-fullstack/hyra=
cks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyr=
acks/tests/am/btree/BTreePrimaryIndexSearchOperatorTest=2Ejava
@@ -103,12 +103,12 @@
     @Override
  =
   protected IResourceFactory createPrimaryResourceFactory() {
         return new BTreeResourceFactory(storageManager, DataSetConstan=
ts=2EprimaryTypeTraits,
-                DataSetConstants=2EprimaryComparatorFactories, pageManage=
rFactory);
+         =
       DataSetConstants=2EprimaryComparatorFactories, pageManagerFactory, n=
ull, null);
     }
 
     @=
Override
     protected IResourceFactory createSecondaryRes=
ourceFactory() {
         return new BTreeResourceFactory(s=
torageManager, DataSetConstants=2EsecondaryTypeTraits,
-                DataSetConstants=2Eseconda=
ryComparatorFactories, pageManagerFactory);
+                DataSetConstants=2EsecondaryCompara=
torFactories, pageManagerFactory, null, null);
     }
 }
diff --git a/hyracks-fullstack/hyracks/hyrack=
s-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests=
/am/btree/BTreePrimaryIndexStatsOperatorTest=2Ejava b/hyracks-fullstack/hyr=
acks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hy=
racks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest=2Ejava
index 5cffd95=2E=2Ea1f53bd 100644
--- a/hyracks-fullsta=
ck/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apa=
che/hyracks/tests/am/btree/BTreePrimaryIndexStatsOperatorTest=2Ejava=

+++ b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integrat=
ion-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreePrimaryIndex=
StatsOperatorTest=2Ejava
@@ -67,12 +67,12 @@
     @Override
     protected IResourceFactory createPrim=
aryResourceFactory() {
         return new BTreeResourceFac=
tory(storageManager, DataSetConstants=2EprimaryTypeTraits,
-                DataSetConstants=2Epri=
maryComparatorFactories, pageManagerFactory);
+                DataSetConstants=2EprimaryCompara=
torFactories, pageManagerFactory, null, null);
     }
 
     @Override
     protected =
IResourceFactory createSecondaryResourceFactory() {
       =
  return new BTreeResourceFactory(storageManager, DataSetConstants=2Esecond=
aryTypeTraits,
-       =
         DataSetConstants=2EsecondaryComparatorFactories, pageManagerFactor=
y);
+                =
DataSetConstants=2EsecondaryComparatorFactories, pageManagerFactory, null, =
null);
     }
 }
diff --git=
 a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src=
/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOpera=
torTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integra=
tion-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIn=
dexInsertOperatorTest=2Ejava
index b2060b6=2E=2Eb489474 100=
644
--- a/hyracks-fullstack/hyracks/hyracks-examples/hyrack=
s-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSe=
condaryIndexInsertOperatorTest=2Ejava
+++ b/hyracks-fullsta=
ck/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apa=
che/hyracks/tests/am/btree/BTreeSecondaryIndexInsertOperatorTest=2Ejava
@@ -121,12 +121,12 @@
     @Override     protected IResourceFactory createPrimaryResourceFactory() {
         return new BTreeResourceFactory(storageManager, DataS=
etConstants=2EprimaryTypeTraits,
-                DataSetConstants=2EprimaryComparatorFactories, p=
ageManagerFactory);
+=
                DataSetConstants=2EprimaryComparatorFactories, pageManagerF=
actory, null, null);
     }
 
     @Override
     protected IResourceFactory createSec=
ondaryResourceFactory() {
         return new BTreeResource=
Factory(storageManager, DataSetConstants=2EsecondaryTypeTraits,
<=
span style=3D"color: hsl(0, 100%, 40%);">-                DataSetConstants=
=2EsecondaryComparatorFactories, pageManagerFactory);
+                DataSetConstants=2Esecond=
aryComparatorFactories, pageManagerFactory, null, null);
  =
   }
 }
diff --git a/hyracks-fullstack/hyra=
cks/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyr=
acks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest=2Ejava b/hyracks-=
fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/java/=
org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperatorTest=2Ej=
ava
index e73a11e=2E=2Ea1d1502 100644
--- a=
/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/t=
est/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexSearchOperato=
rTest=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-exampl=
es/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btre=
e/BTreeSecondaryIndexSearchOperatorTest=2Ejava
@@ -117,12 +=
117,12 @@
     @Override
     protected IRe=
sourceFactory createPrimaryResourceFactory() {
         ret=
urn new BTreeResourceFactory(storageManager, DataSetConstants=2EprimaryType=
Traits,
-              =
  DataSetConstants=2EprimaryComparatorFactories, pageManagerFactory);
+                DataSetCo=
nstants=2EprimaryComparatorFactories, pageManagerFactory, null, null);
     }
 
     @Override<=
br>     protected IResourceFactory createSecondaryResourceFactory() {=

         return new BTreeResourceFactory(storageManager, D=
ataSetConstants=2EsecondaryTypeTraits,
-                DataSetConstants=2EsecondaryComparatorFact=
ories, pageManagerFactory);
+                DataSetConstants=2EsecondaryComparatorFactories, pa=
geManagerFactory, null, null);
     }
 }
diff --git a/hyracks-fullstack/hyracks/hyracks-examples/hyrac=
ks-integration-tests/src/test/java/org/apache/hyracks/tests/am/btree/BTreeS=
econdaryIndexUpsertOperatorTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-=
examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/a=
m/btree/BTreeSecondaryIndexUpsertOperatorTest=2Ejava
index =
a333dc4=2E=2Ef9cdfcf 100644
--- a/hyracks-fullstack/hyracks=
/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyrack=
s/tests/am/btree/BTreeSecondaryIndexUpsertOperatorTest=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tes=
ts/src/test/java/org/apache/hyracks/tests/am/btree/BTreeSecondaryIndexUpser=
tOperatorTest=2Ejava
@@ -120,12 +120,12 @@
=
     @Override
     protected IResourceFactory createPrimar=
yResourceFactory() {
         return new BTreeResourceFacto=
ry(storageManager, DataSetConstants=2EprimaryTypeTraits,
-                DataSetConstants=2Eprima=
ryComparatorFactories, pageManagerFactory);
+                DataSetConstants=2EprimaryComparato=
rFactories, pageManagerFactory, null, null);
     }<=
br> 
     @Override
     protected IR=
esourceFactory createSecondaryResourceFactory() {
         =
return new BTreeResourceFactory(storageManager, DataSetConstants=2Esecondar=
yTypeTraits,
-         =
       DataSetConstants=2EsecondaryComparatorFactories, pageManagerFactory)=
;
+                Da=
taSetConstants=2EsecondaryComparatorFactories, pageManagerFactory, null, nu=
ll);
     }
 }
diff --git a=
/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/t=
est/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTestHelper=
=2Ejava b/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-te=
sts/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorTes=
tHelper=2Ejava
index d46f5f5=2E=2Ebcd71a3 100644
=
--- a/hyracks-fullstack/hyracks/hyracks-examples/hyracks-integration-=
tests/src/test/java/org/apache/hyracks/tests/am/lsm/btree/LSMBTreeOperatorT=
estHelper=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-ex=
amples/hyracks-integration-tests/src/test/java/org/apache/hyracks/tests/am/=
lsm/btree/LSMBTreeOperatorTestHelper=2Ejava
@@ -49,6 +49,6 =
@@
                 getVirtualBufferCacheProvider(), Synchr=
onousSchedulerProvider=2EINSTANCE, MERGE_POLICY_FACTORY,
  =
               MERGE_POLICY_PROPERTIES, DURABLE, bloomFilterKeyFields,
                 LSMTreeOperatorTestHelper=2EDEFAULT_BLOOM_FILT=
ER_FALSE_POSITIVE_RATE, true, btreefields,
-                NoOpCompressorDecompressorFactory=2EIN=
STANCE, bloomFilterKeyFields !=3D null);
+                NoOpCompressorDecompressorFactory=2EIN=
STANCE, bloomFilterKeyFields !=3D null, null, null);
     }=

 }
diff --git a/hyracks-fullstack/hyracks/=
hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks=
/tests/am/rtree/AbstractRTreeOperatorTest=2Ejava b/hyracks-fullstack/hyrack=
s/hyracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyrac=
ks/tests/am/rtree/AbstractRTreeOperatorTest=2Ejava
index 2d=
092f1=2E=2E872502b 100644
--- a/hyracks-fullstack/hyracks/h=
yracks-examples/hyracks-integration-tests/src/test/java/org/apache/hyracks/=
tests/am/rtree/AbstractRTreeOperatorTest=2Ejava
+++ b/hyrac=
ks-fullstack/hyracks/hyracks-examples/hyracks-integration-tests/src/test/ja=
va/org/apache/hyracks/tests/am/rtree/AbstractRTreeOperatorTest=2Ejava
@@ -228,7 +228,7 @@
     protected void createPr=
imaryIndex() throws Exception {
         JobSpecification s=
pec =3D new JobSpecification();
         btreeFactory =3D n=
ew BTreeResourceFactory(storageManager, primaryTypeTraits, primaryComparato=
rFactories,
-          =
      pageManagerFactory);
+                pageManagerFactory, null, null);
   =
      IIndexBuilderFactory indexBuilderFactory =3D
        =
         new IndexBuilderFactory(storageManager, primarySplitProvider, btre=
eFactory, false);
         IndexCreateOperatorDescriptor pr=
imaryCreateOp =3D new IndexCreateOperatorDescriptor(spec, indexBuilderFacto=
ry);
diff --git a/hyracks-fullstack/hyracks/hyracks-storage=
-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/compressors/Fie=
ldPrefixCompressor=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-bt=
ree/src/main/java/org/apache/hyracks/storage/am/btree/compressors/FieldPref=
ixCompressor=2Ejava
index 8c327f7=2E=2E17fe593 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/mai=
n/java/org/apache/hyracks/storage/am/btree/compressors/FieldPrefixCompresso=
r=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage-am=
-btree/src/main/java/org/apache/hyracks/storage/am/btree/compressors/FieldP=
refixCompressor=2Ejava
@@ -32,6 +32,7 @@
 i=
mport org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Eimpls=2EBTreeFieldPre=
fixTupleReference;
 import org=2Eapache=2Ehyracks=2Estorage=
=2Eam=2Ebtree=2Eimpls=2EFieldPrefixSlotManager;
 import org=
=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWr=
iter;
+import org=2Ea=
pache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EITre=
eIndexFrame;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=
=2Ecommon=2Eapi=2EITreeIndexFrameCompressor;
 import org=2E=
apache=2Ehyracks=2Estorage=2Ecommon=2EMultiComparator;
@@ -=
46,10 +47,17 @@
 
     private final ITypeT=
raits[] typeTraits;
 
-    public FieldPrefixCompressor(ITypeTraits[] typeTraits, =
float ratioThreshold, int occurrenceThreshold) {
+    private final ITypeTraits nullTypeTraits;<=
/span>
+
+    private final INullIntrospector nul=
lIntrospector;
+
+    public FieldPrefixCo=
mpressor(ITypeTraits[] typeTraits, float ratioThreshold, int occurrenceThre=
shold,
+            I=
TypeTraits nullTypeTraits, INullIntrospector nullIntrospector) {
=
         this=2EtypeTraits =3D typeTraits;
         t=
his=2EratioThreshold =3D ratioThreshold;
         this=2Eoc=
currenceThreshold =3D occurrenceThreshold;
+        this=2EnullTypeTraits =3D nullTypeTraits;
+        this=2EnullInt=
rospector =3D nullIntrospector;
     }
 
     @Override
@@ -163,7 +171,8 @@
=
         int prefixTupleIndex =3D 0;
         uncompr=
essedTupleCount =3D 0;
 
-        BTreeTypeAwareTupleWriter tupleWriter =3D new BT=
reeTypeAwareTupleWriter(typeTraits, false);
+        BTreeTypeAwareTupleWriter tupleWriter =3D
+                new B=
TreeTypeAwareTupleWriter(typeTraits, false, nullTypeTraits, nullIntrospecto=
r);
         BTreeFieldPrefixTupleReference tupleToWrite =
=3D
                 new BTreeFieldPrefixTupleReference(tup=
leWriter=2EcreateTupleReference());
         tupleToWrite=
=2EsetFieldCount(fieldCount);
@@ -340,7 +349,8 @@         KeyPartition kp =3D new KeyPartition(maxCmps);
         keyPartitions=2Eadd(kp);
 
-        BTreeTypeAwareTupleWriter tupleWr=
iter =3D new BTreeTypeAwareTupleWriter(typeTraits, false);
+        BTreeTypeAwareTupleWriter tu=
pleWriter =3D
+      =
          new BTreeTypeAwareTupleWriter(typeTraits, false, nullTypeTraits, =
nullIntrospector);
 
         BTreeFieldPre=
fixTupleReference prevTuple =3D
                 new BTreeF=
ieldPrefixTupleReference(tupleWriter=2EcreateTupleReference());
<=
span>diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/ma=
in/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeResource=2Ejava b=
/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apach=
e/hyracks/storage/am/btree/dataflow/BTreeResource=2Ejava
in=
dex 4cf145b=2E=2E4589bed1 100644
--- a/hyracks-fullstack/hy=
racks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/=
btree/dataflow/BTreeResource=2Ejava
+++ b/hyracks-fullstack=
/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/=
am/btree/dataflow/BTreeResource=2Ejava
@@ -26,6 +26,7 @@
 import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIIOManager;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Eframes=
=2EBTreeLeafFrameType;
 import org=2Eapache=2Ehyracks=2Esto=
rage=2Eam=2Ebtree=2Eutil=2EBTreeUtils;
+import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2E=
api=2EINullIntrospector;
 import org=2Eapache=2Ehyracks=2Es=
torage=2Eam=2Ecommon=2Eapi=2EIPageManagerFactory;
 import o=
rg=2Eapache=2Ehyracks=2Estorage=2Ecommon=2EIIndex;
 import =
org=2Eapache=2Ehyracks=2Estorage=2Ecommon=2EIResource;
@@ -=
34,20 +35,25 @@
 
 public class BTreeResour=
ce implements IResource {
 
-    private static final long serialVersionUID =3D 1L=
;
+    private static=
 final long serialVersionUID =3D 2L;
     private String pa=
th;
     private final IStorageManager storageManager;
     private final ITypeTraits[] typeTraits;
  =
   private final IBinaryComparatorFactory[] comparatorFactories;
=
     private final IPageManagerFactory pageManagerFactory;
=
+    private final ITypeTraits =
nullTypeTraits;
+    =
private final INullIntrospector nullIntrospector;
 <=
br>     public BTreeResource(String path, IStorageManager storageMana=
ger, ITypeTraits[] typeTraits,
-            IBinaryComparatorFactory[] comparatorFactories, IPageM=
anagerFactory pageManagerFactory) {
+            IBinaryComparatorFactory[] comparatorFactories,=
 IPageManagerFactory pageManagerFactory,
+            ITypeTraits nullTypeTraits, INullIntrospec=
tor nullIntrospector) {
         this=2Epath =3D path;
         this=2EstorageManager =3D storageManager;
         this=2EtypeTraits =3D typeTraits;
         thi=
s=2EcomparatorFactories =3D comparatorFactories;
         t=
his=2EpageManagerFactory =3D pageManagerFactory;
+        this=2EnullTypeTraits =3D nullTypeTrai=
ts;
+        this=2En=
ullIntrospector =3D nullIntrospector;
     }
 
     @Override
@@ -56,7 +62,8 @@=

         IIOManager ioManager =3D ctx=2EgetIoManager();         FileReference resourceRef =3D ioManager=2Eresolve(path);
         return BTreeUtils=2EcreateBTree(bufferCache, typeTra=
its, comparatorFactories, BTreeLeafFrameType=2EREGULAR_NSM,
-                resourceRef, pageMana=
gerFactory=2EcreatePageManager(bufferCache), false);
+                resourceRef, pageManagerF=
actory=2EcreatePageManager(bufferCache), false, nullTypeTraits,
<=
span style=3D"color: hsl(120, 100%, 40%);">+                nullIntrospecto=
r);
     }
 
     @Override=

diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-=
btree/src/main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeResou=
rceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/=
main/java/org/apache/hyracks/storage/am/btree/dataflow/BTreeResourceFactory=
=2Ejava
index 7a21495=2E=2Ed817327 100644
-=
-- a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/a=
pache/hyracks/storage/am/btree/dataflow/BTreeResourceFactory=2Ejava<=
br>+++ b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/=
java/org/apache/hyracks/storage/am/btree/dataflow/BTreeResourceFactory=2Eja=
va
@@ -21,6 +21,7 @@
 import org=2Eapache=
=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EIBinaryComparatorFactory;
<=
span> import org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits;=

 import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EFileReference;=

+import org=2Eapache=
=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIPageMana=
gerFactory;
 import org=2Eapache=2Ehyracks=2Estorage=2Ecomm=
on=2EIResource;
 import org=2Eapache=2Ehyracks=2Estorage=2E=
common=2EIResourceFactory;
@@ -28,24 +29,29 @@
 
 public class BTreeResourceFactory implements IResour=
ceFactory {
 
-    private static final long serialVersionUID =3D 1L;
+    private static final long se=
rialVersionUID =3D 2L;
     private final IStorageManager s=
torageManager;
     private final ITypeTraits[] typeTraits;=

     private final IBinaryComparatorFactory[] comparatorFa=
ctories;
     private final IPageManagerFactory pageManager=
Factory;
+    private=
 final ITypeTraits nullTypeTraits;
+    private final INullIntrospector nullIntrospector;=

 
     public BTreeResourceFactory(IStorageManage=
r storageManager, ITypeTraits[] typeTraits,
-            IBinaryComparatorFactory[] comparatorFact=
ories, IPageManagerFactory pageManagerFactory) {
+            IBinaryComparatorFactory[] compara=
torFactories, IPageManagerFactory pageManagerFactory,
+            ITypeTraits nullTypeTraits, I=
NullIntrospector nullIntrospector) {
         this=2Estorag=
eManager =3D storageManager;
         this=2EtypeTraits =3D=
 typeTraits;
         this=2EcomparatorFactories =3D compar=
atorFactories;
         this=2EpageManagerFactory =3D pageM=
anagerFactory;
+     =
   this=2EnullTypeTraits =3D nullTypeTraits;
+        this=2EnullIntrospector =3D nullIntrospect=
or;
     }
 
     @Override=

     public IResource createResource(FileReference fileRef=
) {
         return new BTreeResource(fileRef=2EgetRelative=
Path(), storageManager, typeTraits, comparatorFactories,
-                pageManagerFactory);
+                pageMan=
agerFactory, nullTypeTraits, nullIntrospector);
     }
 
 }
diff --git a/hyracks-fulls=
tack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/stor=
age/am/btree/frames/BTreeFieldPrefixNSMLeafFrame=2Ejava b/hyracks-fullstack=
/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/=
am/btree/frames/BTreeFieldPrefixNSMLeafFrame=2Ejava
index 6=
584df3=2E=2E10eab69 100644
--- a/hyracks-fullstack/hyracks/=
hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/=
frames/BTreeFieldPrefixNSMLeafFrame=2Ejava
+++ b/hyracks-fu=
llstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/s=
torage/am/btree/frames/BTreeFieldPrefixNSMLeafFrame=2Ejava
=
@@ -75,8 +75,10 @@
         this=2EslotManager =3D new Fiel=
dPrefixSlotManager();
 
         ITypeTrait=
s[] typeTraits =3D tupleWriter=2EgetTypeTraits();
-        this=2EframePrefixTuple =3D new FieldPr=
efixPrefixTupleReference(typeTraits);
-        this=2Ecompressor =3D new FieldPrefixCompressor(typ=
eTraits, 0=2E001f, 2);
+        this=2Efra=
mePrefixTuple =3D new FieldPrefixPrefixTupleReference(typeTraits, nullTypeT=
raits);
+        this=
=2Ecompressor =3D
+  =
              new FieldPrefixCompressor(typeTraits, 0=2E001f, 2, nullTypeTr=
aits, tupleWriter=2EgetNullIntrospector());
     } 
     @Override
diff --git a/hyrac=
ks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyra=
cks/storage/am/btree/impls/FieldPrefixPrefixTupleReference=2Ejava b/hyracks=
-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyrack=
s/storage/am/btree/impls/FieldPrefixPrefixTupleReference=2Ejava
<=
span>index b767537=2E=2E2c728a7 100644
--- a/hyracks-fullst=
ack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/stora=
ge/am/btree/impls/FieldPrefixPrefixTupleReference=2Ejava
++=
+ b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/ap=
ache/hyracks/storage/am/btree/impls/FieldPrefixPrefixTupleReference=2Ejava<=
/span>
@@ -27,8 +27,8 @@
 
 public=
 class FieldPrefixPrefixTupleReference extends TypeAwareTupleReference {
 
-    pu=
blic FieldPrefixPrefixTupleReference(ITypeTraits[] typeTraits) {
=
-        super(typeTraits);
+    public FieldPrefixPre=
fixTupleReference(ITypeTraits[] typeTraits, ITypeTraits nullTypeTraits) {
+        super(typeTra=
its, nullTypeTraits);
     }
 
     // assumes tuple index refers to prefix tuples
dif=
f --git a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/=
org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupleReference=2Ej=
ava b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/=
apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupleReference=2Ejava<=
/span>
index 5b89a19=2E=2Ef4b1948 100644
--- a/hyr=
acks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hy=
racks/storage/am/btree/tuples/BTreeTypeAwareTupleReference=2Ejava+++ b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/ja=
va/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupleReference=
=2Ejava
@@ -23,14 +23,15 @@
 import org=2Ea=
pache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Etuples=2ETypeAwareTupleReference;=

 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eu=
til=2EBitOperationUtils;
 
+// TODO(ali): this is broken if updateAware is enabl=
ed
 public class BTreeTypeAwareTupleReference extends TypeA=
wareTupleReference implements IBTreeIndexTupleReference {
 =

     public static final byte UPDATE_BIT_OFFSET =3D 6;
 
     protected final boolean updateAware;
 
-    pu=
blic BTreeTypeAwareTupleReference(ITypeTraits[] typeTraits, boolean updateA=
ware) {
-        super(=
typeTraits);
+    pub=
lic BTreeTypeAwareTupleReference(ITypeTraits[] typeTraits, boolean updateAw=
are, ITypeTraits nullTypeTraits) {
+        super(typeTraits, nullTypeTraits);
 =
        this=2EupdateAware =3D updateAware;
     } 
diff --git a/hyracks-fullstack/hyracks/hyracks-st=
orage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTr=
eeTypeAwareTupleWriter=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-a=
m-btree/src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeA=
wareTupleWriter=2Ejava
index 083462b=2E=2E87f46b2 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/=
main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTupleWri=
ter=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage-=
am-btree/src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeType=
AwareTupleWriter=2Ejava
@@ -19,21 +19,23 @@
 package org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Etuples; 
 import org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2E=
value=2EITypeTraits;
-i=
mport org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EITreeIndexTupl=
eWriter;
+import org=
=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Etuples=
=2ETypeAwareTupleWriter;
 
-public class BTreeTypeAwareTupleWriter extends TypeAwa=
reTupleWriter implements ITreeIndexTupleWriter {
+// TODO(ali): this is broken if updateAware is=
 enabled
+public clas=
s BTreeTypeAwareTupleWriter extends TypeAwareTupleWriter {
=
     protected final boolean updateAware;
     protected bo=
olean isUpdated;
 
-    public BTreeTypeAwareTupleWriter(ITypeTraits[] typeTraits,=
 boolean updateAware) {
-        super(typeTraits);
+    public BTreeTypeAwareTupleWriter(ITypeTraits[] typeTraits, boo=
lean updateAware, ITypeTraits nullTypeTraits,
+            INullIntrospector nullIntrospector) {=

+        super(typeT=
raits, nullTypeTraits, nullIntrospector);
         this=2Eu=
pdateAware =3D updateAware;
     }
 =

     @Override
     public BTreeTypeAwareTupleRef=
erence createTupleReference() {
-        return new BTreeTypeAwareTupleReference(typeTraits, updat=
eAware);
+        ret=
urn new BTreeTypeAwareTupleReference(typeTraits, updateAware, nullTypeTrait=
s);
     }
 
     @Override=

diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-=
btree/src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwa=
reTupleWriterFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-=
btree/src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwa=
reTupleWriterFactory=2Ejava
index 7089e79=2E=2Ee2aa759 1006=
44
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-btree=
/src/main/java/org/apache/hyracks/storage/am/btree/tuples/BTreeTypeAwareTup=
leWriterFactory=2Ejava
+++ b/hyracks-fullstack/hyracks/hyra=
cks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/tupl=
es/BTreeTypeAwareTupleWriterFactory=2Ejava
@@ -19,6 +19,7 @=
@
 package org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2E=
tuples;
 
 import org=2Eapache=2Ehyracks=2E=
api=2Edataflow=2Evalue=2EITypeTraits;
+import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Ea=
pi=2EINullIntrospector;
 import org=2Eapache=2Ehyracks=2Est=
orage=2Eam=2Ecommon=2Etuples=2ETypeAwareTupleWriterFactory;
 
 public class BTreeTypeAwareTupleWriterFactory extends T=
ypeAwareTupleWriterFactory {
@@ -26,13 +27,14 @@
=
     private static final long serialVersionUID =3D 1L;
     protected final boolean updateAware;
 
-    public BTreeTypeAwareTupleWrit=
erFactory(ITypeTraits[] typeTraits, boolean updateAware) {
-        super(typeTraits);
<=
span style=3D"color: hsl(120, 100%, 40%);">+    public BTreeTypeAwareTupleW=
riterFactory(ITypeTraits[] typeTraits, boolean updateAware, ITypeTraits nul=
lTypeTraits,
+       =
     INullIntrospector nullIntrospector) {
+        super(typeTraits, nullTypeTraits, nullIntros=
pector);
         this=2EupdateAware =3D updateAware;
     }
 
     @Override     public BTreeTypeAwareTupleWriter createTupleWriter() {<=
br>-        return new BTreeTypeA=
wareTupleWriter(typeTraits, updateAware);
+        return new BTreeTypeAwareTupleWriter(typeTrai=
ts, updateAware, nullTypeTraits, nullIntrospector);
     }<=
/span>
 }
diff --git a/hyracks-fullstack/hyracks/h=
yracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/u=
til/BTreeUtils=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/=
src/main/java/org/apache/hyracks/storage/am/btree/util/BTreeUtils=2Ejava
index 0c06bc3=2E=2E09340b6 100644
--- a/hyrac=
ks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyra=
cks/storage/am/btree/util/BTreeUtils=2Ejava
+++ b/hyracks-f=
ullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/=
storage/am/btree/util/BTreeUtils=2Ejava
@@ -31,6 +31,7 @@
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Eimpl=
s=2EBTree;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2E=
btree=2Eimpls=2EDiskBTree;
 import org=2Eapache=2Ehyracks=
=2Estorage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriterFactory;=

+import org=2Eapache=2Ehyra=
cks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 imp=
ort org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIPageManager;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=
=2EITreeIndexFrameFactory;
 import org=2Eapache=2Ehyracks=
=2Estorage=2Ecommon=2EMultiComparator;
@@ -43,9 +44,10 @@
 
     public static BTree createBTree(IBuff=
erCache bufferCache, ITypeTraits[] typeTraits,
            =
 IBinaryComparatorFactory[] cmpFactories, BTreeLeafFrameType leafType, File=
Reference file,
-      =
      IPageManager freePageManager, boolean updateAware) throws HyracksData=
Exception {
+        =
    IPageManager freePageManager, boolean updateAware, ITypeTraits nullType=
Traits,
+            =
INullIntrospector nullIntrospector) throws HyracksDataException {         BTreeTypeAwareTupleWriterFactory tupleWriterFactory =3D
-                new BTree=
TypeAwareTupleWriterFactory(typeTraits, updateAware);
+                new BTreeTypeAwareTupleWr=
iterFactory(typeTraits, updateAware, nullTypeTraits, nullIntrospector);
         ITreeIndexFrameFactory leafFrameFactory =3D getLeafFr=
ameFactory(tupleWriterFactory, leafType);
         ITreeInd=
exFrameFactory interiorFrameFactory =3D new BTreeNSMInteriorFrameFactory(tu=
pleWriterFactory);
         return new BTree(bufferCache, f=
reePageManager, interiorFrameFactory, leafFrameFactory, cmpFactories,
@@ -54,9 +56,10 @@
 
     public=
 static DiskBTree createDiskBTree(IBufferCache bufferCache, ITypeTraits[] t=
ypeTraits,
             IBinaryComparatorFactory[] cmpFacto=
ries, BTreeLeafFrameType leafType, FileReference file,
-            IPageManager freePageManager, =
boolean updateAware) throws HyracksDataException {
+            IPageManager freePageManager, bo=
olean updateAware, ITypeTraits nullTypeTraits,
+            INullIntrospector nullIntrospector) =
throws HyracksDataException {
         BTreeTypeAwareTupleW=
riterFactory tupleWriterFactory =3D
-                new BTreeTypeAwareTupleWriterFactory(typeTrai=
ts, updateAware);
+  =
              new BTreeTypeAwareTupleWriterFactory(typeTraits, updateAware,=
 nullTypeTraits, nullIntrospector);
         ITreeIndexFram=
eFactory leafFrameFactory =3D getLeafFrameFactory(tupleWriterFactory, leafT=
ype);
         ITreeIndexFrameFactory interiorFrameFactory =
=3D new BTreeNSMInteriorFrameFactory(tupleWriterFactory);
 =
        return new DiskBTree(bufferCache, freePageManager, interiorFrameFac=
tory, leafFrameFactory, cmpFactories,
@@ -65,9 +68,10 @@
 
     public static BTree createBTree(IBuffe=
rCache bufferCache, IPageManager freePageManager, ITypeTraits[] typeTraits,=

             IBinaryComparatorFactory[] cmpFactories, BTre=
eLeafFrameType leafType, FileReference file,
-            boolean updateAware) throws HyracksDataE=
xception {
+         =
   boolean updateAware, ITypeTraits nullTypeTraits, INullIntrospector nullI=
ntrospector)
+       =
     throws HyracksDataException {
         BTreeTypeAwareT=
upleWriterFactory tupleWriterFactory =3D
-                new BTreeTypeAwareTupleWriterFactory(typ=
eTraits, updateAware);
+/*
+ * Lic=
ensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreemen=
ts=2E  See the NOTICE file
+ * distributed with this work for additional information
<=
span style=3D"color: hsl(120, 100%, 40%);">+ * regarding copyright ownershi=
p=2E  The ASF licenses this file
+ * to you under the Apache License, Version 2=2E0 (the<=
br>+ * "License"); yo=
u may not use this file except in compliance
+ * with the License=2E  You may obtain a copy of t=
he License at
+ *
+ *   http://www=2Eapach=
e=2Eorg/licenses/LICENSE-2=2E0
+ *
+ * Unl=
ess required by applicable law or agreed to in writing,
+ * software distributed under the Licen=
se is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express=
 or implied=2E  See the License for the
+ * specific language governing permissions and limitati=
ons
+ * under the Lic=
ense=2E
+ */+package org=2Eapache=2Ehyrac=
ks=2Estorage=2Eam=2Ecommon=2Eapi;
+
+impor=
t java=2Eio=2ESerializable;
+
+import org=
=2Eapache=2Ehyracks=2Eapi=2Eio=2EIJsonSerializable;
+
+/**
+ =
* An introspector that tells whether data is {@code NULL} or not=2E<=
br>+ */
+@FunctionalInterface
+public interface INullIntrospector exten=
ds Serializable, IJsonSerializable {
+
+  =
  boolean isNull(byte[] bytes, int offset, int length);
+
+}
diff --git a/hyracks-fullstack/hyracks/h=
yracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common=
/tuples/SimpleTupleWriter=2Ejava b/hyracks-fullstack/hyracks/hyracks-storag=
e-am-common/src/main/java/org/apache/hyracks/storage/am/common/tuples/Simpl=
eTupleWriter=2Ejava
index ca7217e=2E=2E6273b68 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/ma=
in/java/org/apache/hyracks/storage/am/common/tuples/SimpleTupleWriter=2Ejav=
a
+++ b/hyracks-fullstack/hyracks/hyracks-storage-am-common=
/src/main/java/org/apache/hyracks/storage/am/common/tuples/SimpleTupleWrite=
r=2Ejava
@@ -69,6 +69,7 @@
         int run=
ner =3D targetOff;
         int nullFlagsBytes =3D getNullF=
lagsBytes(tuple);
         int fieldSlotsBytes =3D getField=
SlotsBytes(tuple);
+ =
       // TODO(ali): fix if needed
         for (int i =3D =
0; i < nullFlagsBytes; i++) {
             targetBuf[run=
ner++] =3D (byte) 0;
         }
diff --git =
a/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apa=
che/hyracks/storage/am/common/tuples/TypeAwareTupleReference=2Ejava b/hyrac=
ks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyr=
acks/storage/am/common/tuples/TypeAwareTupleReference=2Ejava
index 11f86c9=2E=2E8eff435 100644
--- a/hyracks-fullstack=
/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage=
/am/common/tuples/TypeAwareTupleReference=2Ejava
+++ b/hyra=
cks-fullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hy=
racks/storage/am/common/tuples/TypeAwareTupleReference=2Ejava
@@ -35,11 +35,13 @@
     protected int dataStartOff;
 
     protected final ITypeTraits[] typeTrait=
s;
-    protected VarLe=
nIntDecoder encDec =3D VarLenIntEncoderDecoder=2EcreateDecoder();+    protected final ITypeTrai=
ts nullTypeTraits; // can be null
+    protected final VarLenIntDecoder encDec =3D VarLenIntEnco=
derDecoder=2EcreateDecoder();
     protected int[] decodedF=
ieldSlots;
 
-    public TypeAwareTupleReference(ITypeTraits[] typeTraits) {
+    public TypeAwareTuple=
Reference(ITypeTraits[] typeTraits, ITypeTraits nullTypeTraits) {         this=2EtypeTraits =3D typeTraits;
+        this=2EnullTypeTraits =3D nullTypeTra=
its;
         this=2EfieldStartIndex =3D 0;
         setFieldCount(typeTraits=2Elength);
     }=

@@ -57,11 +59,12 @@
         for (int i =3D field=
StartIndex; i < end; i++) {
             if (!typeTraits=
[i]=2EisFixedLength()) {
                 cumul +=3D encDec=
=2Edecode();
-         =
       decodedFieldSlots[field++] =3D cumul;
+            } else if (nullTypeTraits !=3D null &a=
mp;& isNull(buf, tupleStartOff, i)) {
+                cumul +=3D nullTypeTraits=2EgetFixedL=
ength();
             } else {
            =
     cumul +=3D typeTraits[i]=2EgetFixedLength();
-                decodedFieldSlots[field++] =3D =
cumul;
             }
+            decodedFieldSlots[field++] =3D cumul;<=
br>         }
         dataStartOff =3D encDec=2EgetP=
os();
     }
@@ -127,4 +130,21 @@     public int getTupleSize() {
         return dat=
aStartOff - tupleStartOff + decodedFieldSlots[fieldCount - 1];
     }
++    private boolean isNull(b=
yte[] flags, int flagsOffset, int fieldIdx) {
+        int adjustedFieldIdx =3D getAdjustedField=
Idx(fieldIdx);
+     =
   int flagByteIdx =3D adjustedFieldIdx / 8;
+        int flagBitIdx =3D 7 - (adjustedFieldIdx %=
 8);
+        return =
BitOperationUtils=2EgetBit(flags, flagsOffset + flagByteIdx, (byte) flagBit=
Idx);
+    }+
+    /**
+     * Adjusts the field index in case the null flags secti=
on starts with some other special-purpose fields=2E
+     *
+     * @param fieldIdx logical field index
<=
span style=3D"color: hsl(120, 100%, 40%);">+     * @return adjusted field i=
ndex
+     */<=
br>+    protected int getAdjust=
edFieldIdx(int fieldIdx) {
+        return fieldIdx;
+    }
 }
diff --git a/hyracks-f=
ullstack/hyracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks=
/storage/am/common/tuples/TypeAwareTupleWriter=2Ejava b/hyracks-fullstack/h=
yracks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/a=
m/common/tuples/TypeAwareTupleWriter=2Ejava
index eb6fe2a=
=2E=2E2528518 100644
--- a/hyracks-fullstack/hyracks/hyrack=
s-storage-am-common/src/main/java/org/apache/hyracks/storage/am/common/tupl=
es/TypeAwareTupleWriter=2Ejava
+++ b/hyracks-fullstack/hyra=
cks/hyracks-storage-am-common/src/main/java/org/apache/hyracks/storage/am/c=
ommon/tuples/TypeAwareTupleWriter=2Ejava
@@ -23,6 +23,7 @@<=
/span>
 
 import org=2Eapache=2Ehyracks=2Eapi=2Eda=
taflow=2Evalue=2EITypeTraits;
 import org=2Eapache=2Ehyrack=
s=2Edataflow=2Ecommon=2Edata=2Eaccessors=2EITupleReference;
+import org=2Eapache=2Ehyracks=2Esto=
rage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 import org=
=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EITreeIndexTupleWriter;<=
/span>
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eut=
il=2EBitOperationUtils;
 import org=2Eapache=2Ehyracks=2Eut=
il=2Eencoding=2EVarLenIntEncoderDecoder;
@@ -30,10 +31,14 @=
@
 public class TypeAwareTupleWriter implements ITreeIndexT=
upleWriter {
 
     protected final ITypeTr=
aits[] typeTraits;
-   =
 protected VarLenIntEncoderDecoder=2EVarLenIntDecoder decoder =3D VarLenInt=
EncoderDecoder=2EcreateDecoder();
+    protected final ITypeTraits nullTypeTraits; // can be nul=
l
+    protected fina=
l INullIntrospector nullIntrospector; // can be null
 
-    public TypeAwareTupleW=
riter(ITypeTraits[] typeTraits) {
+    public TypeAwareTupleWriter(ITypeTraits[] typeTraits, ITy=
peTraits nullTypeTraits,
+            INullIntrospector nullIntrospector) {
    =
     this=2EtypeTraits =3D typeTraits;
+        this=2EnullTypeTraits =3D nullTypeTraits;=

+        this=2EnullIntrosp=
ector =3D nullIntrospector;
     }
 =

     @Override
@@ -56,7 +61,7 @@
=
 
     @Override
     public TypeAwareTuple=
Reference createTupleReference() {
-        return new TypeAwareTupleReference(typeTraits);=

+        return new TypeAwa=
reTupleReference(typeTraits, nullTypeTraits);
     }=

 
     @Override
@@ -68,7 +73,7 @=
@
     public int writeTuple(ITupleReference tuple, byte[] =
targetBuf, int targetOff) {
         int runner =3D targetO=
ff;
         int nullFlagsBytes =3D getNullFlagsBytes(tuple=
);
-        // write nu=
ll indicator bits
+  =
      // reset null indicator bits
         for (int i =3D =
0; i < nullFlagsBytes; i++) {
             targetBuf[run=
ner++] =3D (byte) 0;
         }
@@ -80,10 +=
85,16 @@
             }
         } 
-        // w=
rite data fields
+   =
     // write data fields and set null indicator bits
     =
    for (int i =3D 0; i < tuple=2EgetFieldCount(); i++) {
-            System=2Earraycopy(tuple=
=2EgetFieldData(i), tuple=2EgetFieldStart(i), targetBuf, runner, tuple=2Ege=
tFieldLength(i));
-    =
        runner +=3D tuple=2EgetFieldLength(i);
+            byte[] fieldData =3D tuple=2EgetFiel=
dData(i);
+          =
  int fieldOffset =3D tuple=2EgetFieldStart(i);
+            int fieldLength =3D tuple=2EgetFiel=
dLength(i);
+        =
    if (nullIntrospector !=3D null && nullIntrospector=2EisNull(fie=
ldData, fieldOffset, fieldLength)) {
+                setNullFlag(targetBuf, targetOff, i);
+            }
=
+            System=2Earraycopy=
(fieldData, fieldOffset, targetBuf, runner, fieldLength);
+            runner +=3D fieldLength;<=
/span>
         }
 
         retur=
n runner - targetOff;
@@ -93,7 +104,7 @@
  =
   public int writeTupleFields(ITupleReference tuple, int startField, int n=
umFields, byte[] targetBuf, int targetOff) {
         int r=
unner =3D targetOff;
         int nullFlagsBytes =3D getNul=
lFlagsBytes(numFields);
-        // write null indicator bits
+        // reset null indicator bits
   =
      for (int i =3D 0; i < nullFlagsBytes; i++) {
     =
        targetBuf[runner++] =3D (byte) 0;
         }=

@@ -105,9 +116,15 @@
             }
         }
 
-        for (int i =3D startField; i < startField + numFields;=
 i++) {
-            Sy=
stem=2Earraycopy(tuple=2EgetFieldData(i), tuple=2EgetFieldStart(i), targetB=
uf, runner, tuple=2EgetFieldLength(i));
-            runner +=3D tuple=2EgetFieldLength(i);=

+        for (int i =3D sta=
rtField, targetField =3D 0; i < startField + numFields; i++, targetField=
++) {
+            by=
te[] fieldData =3D tuple=2EgetFieldData(i);
+            int fieldOffset =3D tuple=2EgetFieldSta=
rt(i);
+            i=
nt fieldLength =3D tuple=2EgetFieldLength(i);
+            if (nullIntrospector !=3D null &&=
amp; nullIntrospector=2EisNull(fieldData, fieldOffset, fieldLength)) {
+                setNullF=
lag(targetBuf, targetOff, targetField);
+            }
+            System=2Earraycopy(fieldData, fieldOffset, target=
Buf, runner, fieldLength);
+            runner +=3D fieldLength;
         }
 
         return runner - targetOff;@@ -145,8 +162,40 @@
         return typeTraits;
     }
 
+    public ITypeTraits getNullTypeTraits() {
+        return nullTypeTraits;
+    }
+
+    public INullIntrospector getNullIntrospector() {
+        return nullIn=
trospector;
+    }
+
     =
@Override
     public int getCopySpaceRequired(ITupleRefere=
nce tuple) {
         return bytesRequired(tuple);     }
+
+    /**
+     * Given a field index, this met=
hod finds its corresponding bit in the null flags section and sets it=2E
+     *
+     * @param flags data
=
+     * @param flagsOffset star=
t of the null flags data
+     * @param fieldIdx logical field index
+     */
+    private void setNullFlag(byte[] flags, int flagsOffset,=
 int fieldIdx) {
+   =
     int adjustedFieldIdx =3D getAdjustedFieldIdx(fieldIdx);
+        int flagByteIdx =3D adjust=
edFieldIdx / 8;
+    =
    int flagBitIdx =3D 7 - (adjustedFieldIdx % 8);
+        BitOperationUtils=2EsetBit(flags, fl=
agsOffset + flagByteIdx, (byte) flagBitIdx);
+    }
+
+    /**<=
/span>
+     * Adjusts the f=
ield index in case the null flags section starts with some other special-pu=
rpose fields=2E
+    =
 *
+     * @param fie=
ldIdx logical field index
+     * @return adjusted field index
+     */
+    protected int getAdjustedFieldIdx(int fieldIdx) {
<=
span style=3D"color: hsl(120, 100%, 40%);">+        return fieldIdx;=

+    }
 }
diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-co=
mmon/src/main/java/org/apache/hyracks/storage/am/common/tuples/TypeAwareTup=
leWriterFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-commo=
n/src/main/java/org/apache/hyracks/storage/am/common/tuples/TypeAwareTupleW=
riterFactory=2Ejava
index a0fa505=2E=2E586f243 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-common/src/ma=
in/java/org/apache/hyracks/storage/am/common/tuples/TypeAwareTupleWriterFac=
tory=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage=
-am-common/src/main/java/org/apache/hyracks/storage/am/common/tuples/TypeAw=
areTupleWriterFactory=2Ejava
@@ -20,20 +20,26 @@
=
 package org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Etuples;
 
 import org=2Eapache=2Ehyracks=2Eapi=2Edataf=
low=2Evalue=2EITypeTraits;
+import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullI=
ntrospector;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=
=2Ecommon=2Eapi=2EITreeIndexTupleWriterFactory;
  public class TypeAwareTupleWriterFactory implements ITreeIndexTuple=
WriterFactory {
 
-    private static final long serialVersionUID =3D 1L;-    protected ITypeTraits[] ty=
peTraits;
+    privat=
e static final long serialVersionUID =3D 2L;
+    protected final ITypeTraits[] typeTraits;
+    protected final ITyp=
eTraits nullTypeTraits;
+    protected final INullIntrospector nullIntrospector;
 
-    public TypeAwar=
eTupleWriterFactory(ITypeTraits[] typeTraits) {
+    public TypeAwareTupleWriterFactory(ITypeTra=
its[] typeTraits, ITypeTraits nullTypeTraits,
+            INullIntrospector nullIntrospector) {=

         this=2EtypeTraits =3D typeTraits;
+        this=2EnullTypeTraits =3D n=
ullTypeTraits;
+     =
   this=2EnullIntrospector =3D nullIntrospector;
     }
 
     @Override
     public T=
ypeAwareTupleWriter createTupleWriter() {
-        return new TypeAwareTupleWriter(typeTraits);
+        return new Typ=
eAwareTupleWriter(typeTraits, nullTypeTraits, nullIntrospector);
=
     }
 
 }
diff --gi=
t a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/or=
g/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeLocalResource=
=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/j=
ava/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeLocalReso=
urce=2Ejava
index d94a5e1=2E=2Ee16baf9 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/ja=
va/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeLocalResou=
rce=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage-=
am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow=
/ExternalBTreeLocalResource=2Ejava
@@ -29,6 +29,7 @@=

 import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIJsonSerializable;
 import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIPersistedResourc=
eRegistry;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2E=
common=2Eapi=2EIMetadataPageManagerFactory;
+import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecomm=
on=2Eapi=2EINullIntrospector;
 import org=2Eapache=2Ehyrack=
s=2Estorage=2Eam=2Elsm=2Ebtree=2Eutils=2ELSMBTreeUtil;
 imp=
ort org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOper=
ationCallbackFactory;
 import org=2Eapache=2Ehyracks=2Estor=
age=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationSchedulerProvider;@@ -54,12 +55,13 @@
             ILSMOperationTracke=
rFactory opTrackerProvider, ILSMIOOperationCallbackFactory ioOpCallbackFact=
ory,
             ILSMPageWriteCallbackFactory pageWriteCal=
lbackFactory,
             IMetadataPageManagerFactory meta=
dataPageManagerFactory,
-            ILSMIOOperationSchedulerProvider ioSchedulerProvider, boolean=
 durable, boolean hasBloomFilter) {
+            ILSMIOOperationSchedulerProvider ioSchedulerPro=
vider, boolean durable, boolean hasBloomFilter,
+            ITypeTraits nullTypeTraits, INullIn=
trospector nullIntrospector) {
         super(typeTraits, c=
mpFactories, bloomFilterKeyFields, bloomFilterFalsePositiveRate, isPrimary,=
 path,
                 storageManager, mergePolicyFactory,=
 mergePolicyProperties, filterTypeTraits, filterCmpFactories,
                 btreeFields, filterFields, opTrackerProvider, ioOpCallb=
ackFactory, pageWriteCallbackFactory,
                 meta=
dataPageManagerFactory, null, ioSchedulerProvider, durable,
-                NoOpCompressorDecompr=
essorFactory=2EINSTANCE, hasBloomFilter);
+                NoOpCompressorDecompressorFactory=2EI=
NSTANCE, hasBloomFilter, nullTypeTraits, nullIntrospector);
     }
 
     private ExternalBTreeLocalRe=
source(IPersistedResourceRegistry registry, JsonNode json, int[] bloomFilte=
rKeyFields,
@@ -80,7 +82,7 @@
             =
    mergePolicyFactory=2EcreateMergePolicy(mergePolicyProperties, serviceCt=
x),
                 opTrackerProvider=2EgetOperationTracke=
r(serviceCtx, this), ioSchedulerProvider=2EgetIoScheduler(serviceCtx),
                 ioOpCallbackFactory, pageWriteCallbackFactory,=
 durable, metadataPageManagerFactory,
-                serviceCtx=2EgetTracer());
+                serviceCtx=2EgetTrac=
er(), nullTypeTraits, nullIntrospector);
     }
<=
span> 
     @Override
diff --git a/hyracks-=
fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyr=
acks/storage/am/lsm/btree/dataflow/ExternalBTreeLocalResourceFactory=2Ejava=
 b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org=
/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeLocalResourceFac=
tory=2Ejava
index f398370=2E=2Ee4c29ba 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/ja=
va/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeLocalResou=
rceFactory=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-s=
torage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/d=
ataflow/ExternalBTreeLocalResourceFactory=2Ejava
@@ -24,6 +=
24,7 @@
 import org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Ev=
alue=2EITypeTraits;
 import org=2Eapache=2Ehyracks=2Eapi=2E=
io=2EFileReference;
 import org=2Eapache=2Ehyracks=2Estorag=
e=2Eam=2Ecommon=2Eapi=2EIMetadataPageManagerFactory;
+import org=2Eapache=2Ehyracks=2Estorage=
=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 import org=2Eapa=
che=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallback=
Factory;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Els=
m=2Ecommon=2Eapi=2EILSMIOOperationSchedulerProvider;
 impor=
t org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMMergePol=
icyFactory;
@@ -45,12 +46,13 @@
           =
  IMetadataPageManagerFactory metadataPageManagerFactory,
 =
            ILSMIOOperationSchedulerProvider ioSchedulerProvider, ILSMMerge=
PolicyFactory mergePolicyFactory,
             Map<Strin=
g, String> mergePolicyProperties, boolean durable, int[] bloomFilterKeyF=
ields,
-            dou=
ble bloomFilterFalsePositiveRate, boolean isPrimary, int[] btreeFields, boo=
lean hasBloomFilter) {
+            ITypeTraits nullTypeTraits, INullIntrospecto=
r nullIntrospector) {
         super(storageManager, typeTr=
aits, cmpFactories, filterTypeTraits, filterCmpFactories, filterFields,
                 opTrackerFactory, ioOpCallbackFactory, pageWr=
iteCallbackFactory, metadataPageManagerFactory, null,
     =
            ioSchedulerProvider, mergePolicyFactory, mergePolicyProperties,=
 durable, bloomFilterKeyFields,
                 bloomFilte=
rFalsePositiveRate, isPrimary, btreeFields, NoOpCompressorDecompressorFacto=
ry=2EINSTANCE,
-       =
         hasBloomFilter);
+                hasBloomFilter, nullTypeTraits, nullIntrospector);
     }
 
     @Override
@@ -59,6 +61,6 @@
                 bloomFilterFa=
lsePositiveRate, isPrimary, fileRef=2EgetRelativePath(), storageManager, me=
rgePolicyFactory,
                 mergePolicyProperties, f=
ilterTypeTraits, filterCmpFactories, btreeFields, bloomFilterKeyFields,
                 opTrackerProvider, ioOpCallbackFactory, pageW=
riteCallbackFactory, metadataPageManagerFactory,
-                ioSchedulerProvider, durable, ha=
sBloomFilter);
+     =
           ioSchedulerProvider, durable, hasBloomFilter, nullTypeTraits, nu=
llIntrospector);
     }
 }
=
diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/mai=
n/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBu=
ddyLocalResource=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-=
btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/Extern=
alBTreeWithBuddyLocalResource=2Ejava
index bf4b06c=2E=2E863=
ae56 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage=
-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflo=
w/ExternalBTreeWithBuddyLocalResource=2Ejava
+++ b/hyracks-=
fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyr=
acks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResource=2Eja=
va
@@ -29,6 +29,7 @@
 import org=2Eapache=
=2Ehyracks=2Eapi=2Eio=2EIJsonSerializable;
 import org=2Eap=
ache=2Ehyracks=2Eapi=2Eio=2EIPersistedResourceRegistry;
 im=
port org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMetadataPageMa=
nagerFactory;
+import=
 org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;<=
/span>
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ebtree=
=2Eutils=2ELSMBTreeUtil;
 import org=2Eapache=2Ehyracks=2Es=
torage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallbackFactory; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=
=2EILSMIOOperationSchedulerProvider;
@@ -57,12 +58,13 @@
             ILSMOperationTrackerFactory opTrackerProvider, I=
LSMIOOperationCallbackFactory ioOpCallbackFactory,
        =
     ILSMPageWriteCallbackFactory pageWriteCallbackFactory,
             IMetadataPageManagerFactory metadataPageManagerFactory,
-            ILSMIOOperation=
SchedulerProvider ioSchedulerProvider, boolean durable, boolean hasBloomFil=
ter) {
+            I=
LSMIOOperationSchedulerProvider ioSchedulerProvider, boolean durable, boole=
an hasBloomFilter,
+ =
           ITypeTraits nullTypeTraits, INullIntrospector nullIntrospector) =
{
         super(typeTraits, cmpFactories, buddyBtreeFields=
, bloomFilterFalsePositiveRate, isPrimary, path, storageManager,
=
                 mergePolicyFactory, mergePolicyProperties, filterTyp=
eTraits, filterCmpFactories, btreeFields,
                 =
filterFields, opTrackerProvider, ioOpCallbackFactory, pageWriteCallbackFact=
ory,
                 metadataPageManagerFactory, null, ioS=
chedulerProvider, durable,
-                NoOpCompressorDecompressorFactory=2EINSTANCE, hasBloom=
Filter);
+           =
     NoOpCompressorDecompressorFactory=2EINSTANCE, hasBloomFilter, nullType=
Traits, nullIntrospector);
     }
 <=
br>     private ExternalBTreeWithBuddyLocalResource(IPersistedResourc=
eRegistry registry, JsonNode json,
@@ -83,7 +85,7 @@=

                 mergePolicyFactory=2EcreateMergePolicy(mergePoli=
cyProperties, serviceCtx),
                 opTrackerProvid=
er=2EgetOperationTracker(serviceCtx, this), ioSchedulerProvider=2EgetIoSche=
duler(serviceCtx),
                 ioOpCallbackFactory, pa=
geWriteCallbackFactory, bloomFilterKeyFields, durable,
-                metadataPageManagerFactory=
, serviceCtx=2EgetTracer());
+                metadataPageManagerFactory, serviceCtx=2EgetTracer=
(), nullTypeTraits, nullIntrospector);
     }
 
     @Override
diff --git a/hyracks-fu=
llstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyrac=
ks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResourceFactory=
=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/j=
ava/org/apache/hyracks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddy=
LocalResourceFactory=2Ejava
index 91ebee1=2E=2E39c8aed 1006=
44
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-b=
tree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/Externa=
lBTreeWithBuddyLocalResourceFactory=2Ejava
+++ b/hyracks-fu=
llstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyrac=
ks/storage/am/lsm/btree/dataflow/ExternalBTreeWithBuddyLocalResourceFactory=
=2Ejava
@@ -24,6 +24,7 @@
 import org=2Eapa=
che=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits;
 impo=
rt org=2Eapache=2Ehyracks=2Eapi=2Eio=2EFileReference;
 impo=
rt org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMetadataPageMana=
gerFactory;
+import o=
rg=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=
=2Eapi=2EILSMIOOperationCallbackFactory;
 import org=2Eapac=
he=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationScheduler=
Provider;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2El=
sm=2Ecommon=2Eapi=2EILSMMergePolicyFactory;
@@ -45,12 +46,1=
3 @@
             IMetadataPageManagerFactory metadataPageM=
anagerFactory,
             ILSMIOOperationSchedulerProvide=
r ioSchedulerProvider, ILSMMergePolicyFactory mergePolicyFactory,             Map<String, String> mergePolicyProperties, boolea=
n durable, int[] buddyBtreeFields,
-            double bloomFilterFalsePositiveRate, boolean isPri=
mary, int[] btreeFields, boolean hasBloomFilter) {
+            double bloomFilterFalsePositiveR=
ate, boolean isPrimary, int[] btreeFields, boolean hasBloomFilter,+            ITypeTraits null=
TypeTraits, INullIntrospector nullIntrospector) {
         =
super(storageManager, typeTraits, cmpFactories, filterTypeTraits, filterCmp=
Factories, filterFields,
                 opTrackerProvider=
, ioOpCallbackFactory, pageWriteCallbackFactory, metadataPageManagerFactory=
, null,
                 ioSchedulerProvider, mergePolicyFa=
ctory, mergePolicyProperties, durable, buddyBtreeFields,
  =
               bloomFilterFalsePositiveRate, isPrimary, btreeFields, NoOpCo=
mpressorDecompressorFactory=2EINSTANCE,
-                hasBloomFilter);
+                hasBloomFilter, nullTypeTrai=
ts, nullIntrospector);
     }
 
<=
span>     @Override
@@ -59,6 +61,6 @@
     =
            bloomFilterFalsePositiveRate, isPrimary, fileRef=2EgetRelativeP=
ath(), storageManager, mergePolicyFactory,
                =
 mergePolicyProperties, filterTypeTraits, filterCmpFactories, btreeFields, =
filterFields,
                 opTrackerProvider, ioOpCallb=
ackFactory, pageWriteCallbackFactory, metadataPageManagerFactory,-                ioSchedulerProv=
ider, durable, hasBloomFilter);
+                ioSchedulerProvider, durable, hasBloomFilter, n=
ullTypeTraits, nullIntrospector);
     }
 }=

diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-=
lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LS=
MBTreeLocalResource=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-l=
sm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSM=
BTreeLocalResource=2Ejava
index 6098a1d=2E=2E656ae02 100644=

--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btr=
ee/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSMBTreeL=
ocalResource=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks=
-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree=
/dataflow/LSMBTreeLocalResource=2Ejava
@@ -31,6 +31,7 @@
 import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIJsonSerializable=
;
 import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIPersistedRes=
ourceRegistry;
 import org=2Eapache=2Ehyracks=2Estorage=2Ea=
m=2Ecommon=2Eapi=2EIMetadataPageManagerFactory;
+import org=2Eapache=2Ehyracks=2Estorage=2Eam=2E=
common=2Eapi=2EINullIntrospector;
 import org=2Eapache=2Ehy=
racks=2Estorage=2Eam=2Elsm=2Ebtree=2Eutils=2ELSMBTreeUtil;
=
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIO=
OperationCallbackFactory;
 import org=2Eapache=2Ehyracks=2E=
storage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationSchedulerProvider;
@@ -68,10 +69,12 @@
             ILSMPageWriteCa=
llbackFactory pageWriteCallbackFactory,
             IMetad=
ataPageManagerFactory metadataPageManagerFactory, IVirtualBufferCacheProvid=
er vbcProvider,
             ILSMIOOperationSchedulerProvid=
er ioSchedulerProvider, boolean durable,
-            ICompressorDecompressorFactory compressorDec=
ompressorFactory, boolean hasBloomFilter) {
+            ICompressorDecompressorFactory compress=
orDecompressorFactory, boolean hasBloomFilter,
+            ITypeTraits nullTypeTraits, INullInt=
rospector nullIntrospector) {
         super(path, storageM=
anager, typeTraits, cmpFactories, filterTypeTraits, filterCmpFactories, fil=
terFields,
                 opTrackerProvider, ioOpCallback=
Factory, pageWriteCallbackFactory, metadataPageManagerFactory,
-                vbcProvider, ioSch=
edulerProvider, mergePolicyFactory, mergePolicyProperties, durable);=

+                vbcProvide=
r, ioSchedulerProvider, mergePolicyFactory, mergePolicyProperties, durable,=
 nullTypeTraits,
+   =
             nullIntrospector);
         this=2EbloomFilter=
KeyFields =3D bloomFilterKeyFields;
         this=2EbloomFi=
lterFalsePositiveRate =3D bloomFilterFalsePositiveRate;
   =
      this=2EisPrimary =3D isPrimary;
@@ -108,7 +111,7 @@
                 opTrackerProvider=2EgetOperationTracker(ser=
viceCtx, this), ioSchedulerProvider=2EgetIoScheduler(serviceCtx),                 ioOpCallbackFactory, pageWriteCallbackFactory, isPr=
imary, filterTypeTraits, filterCmpFactories,
              =
   btreeFields, filterFields, durable, metadataPageManagerFactory, updateAw=
are, serviceCtx=2EgetTracer(),
-                compressorDecompressorFactory, hasBloomFilter);
+                compre=
ssorDecompressorFactory, hasBloomFilter, nullTypeTraits, nullIntrospector);=

     }
 
     @Override
diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm=
-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow/LSMBT=
reeLocalResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-=
am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflow=
/LSMBTreeLocalResourceFactory=2Ejava
index a2c16b8=2E=2Ee92=
6a48 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage=
-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/dataflo=
w/LSMBTreeLocalResourceFactory=2Ejava
+++ b/hyracks-fullsta=
ck/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/st=
orage/am/lsm/btree/dataflow/LSMBTreeLocalResourceFactory=2Ejava
<=
span>@@ -26,6 +26,7 @@
 import org=2Eapache=2Ehyracks=2Eapi=
=2Edataflow=2Evalue=2EITypeTraits;
 import org=2Eapache=2Eh=
yracks=2Eapi=2Eio=2EFileReference;
 import org=2Eapache=2Eh=
yracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMetadataPageManagerFactory;<=
br>+import org=2Eapache=2Ehyrac=
ks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 impo=
rt org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOpera=
tionCallbackFactory;
 import org=2Eapache=2Ehyracks=2Estora=
ge=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationSchedulerProvider;
=
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2E=
ILSMMergePolicyFactory;
@@ -57,10 +58,12 @@
             ILSMIOOperationSchedulerProvider ioSchedulerProvider, ILSMMer=
gePolicyFactory mergePolicyFactory,
             Map<Str=
ing, String> mergePolicyProperties, boolean durable, int[] bloomFilterKe=
yFields,
             double bloomFilterFalsePositiveRate, =
boolean isPrimary, int[] btreeFields,
-            ICompressorDecompressorFactory compressorDecomp=
ressorFactory, boolean hasBloomFilter) {
+            ICompressorDecompressorFactory compressorD=
ecompressorFactory, boolean hasBloomFilter,
+            ITypeTraits nullTypeTraits, INullIntros=
pector nullIntrospector) {
         super(storageManager, t=
ypeTraits, cmpFactories, filterTypeTraits, filterCmpFactories, filterFields=
,
                 opTrackerFactory, ioOpCallbackFactory, p=
ageWriteCallbackFactory, metadataPageManagerFactory,
-                vbcProvider, ioSchedulerPro=
vider, mergePolicyFactory, mergePolicyProperties, durable);
+                vbcProvider, ioSche=
dulerProvider, mergePolicyFactory, mergePolicyProperties, durable, nullType=
Traits,
+            =
    nullIntrospector);
         this=2EhasBloomFilter =3D h=
asBloomFilter;
         this=2EbloomFilterKeyFields =3D blo=
omFilterKeyFields;
         this=2EbloomFilterFalsePositive=
Rate =3D bloomFilterFalsePositiveRate;
@@ -75,7 +78,7 @@
                 isPrimary, fileRef=2EgetRelativePath(), stor=
ageManager, mergePolicyFactory, mergePolicyProperties,
    =
             filterTypeTraits, filterCmpFactories, btreeFields, filterField=
s, opTrackerProvider, ioOpCallbackFactory,
                =
 pageWriteCallbackFactory, metadataPageManagerFactory, vbcProvider, ioSched=
ulerProvider, durable,
=
-                compressorDecompressorFactory, hasBloomFilter);
=
+                compressorDeco=
mpressorFactory, hasBloomFilter, nullTypeTraits, nullIntrospector);<=
br>     }
 
     private void readObj=
ect(java=2Eio=2EObjectInputStream in) throws IOException, ClassNotFoundExce=
ption {
diff --git a/hyracks-fullstack/hyracks/hyracks-stor=
age-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tupl=
es/LSMBTreeCopyTupleWriter=2Ejava b/hyracks-fullstack/hyracks/hyracks-stora=
ge-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuple=
s/LSMBTreeCopyTupleWriter=2Ejava
index 7c924b5=2E=2E76916ec=
 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-=
lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMB=
TreeCopyTupleWriter=2Ejava
+++ b/hyracks-fullstack/hyracks/=
hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/ls=
m/btree/tuples/LSMBTreeCopyTupleWriter=2Ejava
@@ -21,11 +21=
,14 @@
 
 import org=2Eapache=2Ehyracks=2Ea=
pi=2Edataflow=2Evalue=2EITypeTraits;
 import org=2Eapache=
=2Ehyracks=2Edataflow=2Ecommon=2Edata=2Eaccessors=2EITupleReference;=

+import org=2Eapache=2Ehyra=
cks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 
 public class LSMBTreeCopyTupleWriter extends LSMBTreeTupleWr=
iter {
-    public LSMB=
TreeCopyTupleWriter(ITypeTraits[] typeTraits, int numKeyFields, boolean upd=
ateAware) {
+<=
br>+    public LSMBTreeCopyTupl=
eWriter(ITypeTraits[] typeTraits, int numKeyFields, boolean updateAware,
+            ITypeTrait=
s nullTypeTraits, INullIntrospector nullIntrospector) {
   =
      // Third parameter is never used locally, just give false=2E-        super(typeTraits, numK=
eyFields, false, updateAware);
+        super(typeTraits, numKeyFields, false, updateAware, null=
TypeTraits, nullIntrospector);
     }
 
     @Override
diff --git a/hyracks-fullstack/=
hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/stora=
ge/am/lsm/btree/tuples/LSMBTreeCopyTupleWriterFactory=2Ejava b/hyracks-full=
stack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks=
/storage/am/lsm/btree/tuples/LSMBTreeCopyTupleWriterFactory=2Ejavaindex 02cd900=2E=2Edd43ef4 100644
--- a/hyracks-ful=
lstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyrack=
s/storage/am/lsm/btree/tuples/LSMBTreeCopyTupleWriterFactory=2Ejava<=
br>+++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/m=
ain/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeCopyTupleWr=
iterFactory=2Ejava
@@ -22,18 +22,20 @@
 imp=
ort org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits; import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Etuples=2EBT=
reeTypeAwareTupleWriter;
 import org=2Eapache=2Ehyracks=2Es=
torage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriterFactory;
=
+import org=2Eapache=2Ehyracks=
=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 
 public class LSMBTreeCopyTupleWriterFactory extends BTreeTypeAw=
areTupleWriterFactory {
     private static final long seri=
alVersionUID =3D 1L;
     private final int numKeyFields;
 
-    p=
ublic LSMBTreeCopyTupleWriterFactory(ITypeTraits[] typeTraits, int numKeyFi=
elds, boolean updateAware) {
-        super(typeTraits, updateAware);
+    public LSMBTreeCopyTupleWriterFactory(ITypeT=
raits[] typeTraits, int numKeyFields, boolean updateAware,
+            ITypeTraits nullTypeTrai=
ts, INullIntrospector nullIntrospector) {
+        super(typeTraits, updateAware, nullTypeTraits=
, nullIntrospector);
         this=2EnumKeyFields =3D numKe=
yFields;
     }
 
     @Ove=
rride
     public BTreeTypeAwareTupleWriter createTupleWrit=
er() {
-        return =
new LSMBTreeCopyTupleWriter(typeTraits, numKeyFields, updateAware);<=
br>+        return new LSMBTree=
CopyTupleWriter(typeTraits, numKeyFields, updateAware, nullTypeTraits, null=
Introspector);
     }
 }
di=
ff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/=
java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTupleReference=
=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/j=
ava/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTupleReference=
=2Ejava
index 2a5e232=2E=2Ee5c365a 100644
-=
-- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/o=
rg/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTupleReference=2Ejava=

+++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btr=
ee/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTup=
leReference=2Ejava
@@ -33,8 +33,9 @@
     p=
rivate boolean resetFieldCount =3D false;
     private fina=
l int numKeyFields;
 
-    public LSMBTreeTupleReference(ITypeTraits[] typeTraits,=
 int numKeyFields, boolean updateAware) {
-        super(typeTraits, updateAware);
+    public LSMBTreeTupleReference(I=
TypeTraits[] typeTraits, int numKeyFields, boolean updateAware,
<=
span style=3D"color: hsl(120, 100%, 40%);">+            ITypeTraits nullTyp=
eTraits) {
+        s=
uper(typeTraits, updateAware, nullTypeTraits);
         thi=
s=2EnumKeyFields =3D numKeyFields;
     }
 =

@@ -94,4 +95,10 @@
     public int getTupl=
eStart() {
         return tupleStartOff;
 =
    }
+
+    @Override
+    protected int getAdjustedFieldIdx(i=
nt fieldIdx) {
+     =
   // 2 bits when update-aware: 1 for antimatter and 1 for the update, othe=
rwise, only 1 bit for the antimatter
+        return updateAware ? fieldIdx + 2 : fieldIdx + 1;<=
/span>
+    }
 }
diff --git a/hyracks-fullstack/hyracks/hyracks-storage=
-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/=
LSMBTreeTupleWriter=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-l=
sm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBT=
reeTupleWriter=2Ejava
index 09ced10=2E=2Ebcbf93f 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/s=
rc/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTupleWr=
iter=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage=
-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/=
LSMBTreeTupleWriter=2Ejava
@@ -25,6 +25,7 @@
 import org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2EITypeTraits;
 import org=2Eapache=2Ehyracks=2Edataflow=2Ecommon=2Edata=2Ea=
ccessors=2EITupleReference;
 import org=2Eapache=2Ehyracks=
=2Estorage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriter;
+import org=2Eapache=2Ehyracks=2Es=
torage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 import org=
=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eutil=2EBitOperationUtils;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=
=2Eapi=2EILSMTreeTupleWriter;
 
@@ -33,8 +3=
4,9 @@
     private boolean isAntimatter;
 =
    private final int numKeyFields;
 
-    public LSMBTreeTupleWriter(ITypeTraits[=
] typeTraits, int numKeyFields, boolean isAntimatter, boolean updateAware) =
{
-        super(typeTr=
aits, updateAware);
+=
    public LSMBTreeTupleWriter(ITypeTraits[] typeTraits, int numKeyFields, =
boolean isAntimatter, boolean updateAware,
+            ITypeTraits nullTypeTraits, INullIntrosp=
ector nullIntrospector) {
+        super(typeTraits, updateAware, nullTypeTraits, nullIntrospect=
or);
         this=2EnumKeyFields =3D numKeyFields;<=
br>         this=2EisAntimatter =3D isAntimatter;
   =
  }
@@ -56,7 +58,7 @@
 
   =
  @Override
     public LSMBTreeTupleReference createTupleR=
eference() {
-        r=
eturn new LSMBTreeTupleReference(typeTraits, numKeyFields, updateAware);
+        return new LSM=
BTreeTupleReference(typeTraits, numKeyFields, updateAware, nullTypeTraits);=

     }
 
     @Override
@@ -83,6 +85,7 @@
     public int writeTuple(=
ITupleReference tuple, byte[] targetBuf, int targetOff) {
 =
        int bytesWritten =3D -1;
         if (isAntimatter)=
 {
+            // TO=
DO(ali): check this with secondary primary-key index
      =
       bytesWritten =3D super=2EwriteTupleFields(tuple, 0, numKeyFields, ta=
rgetBuf, targetOff);
             // Set antimatter bit to =
1=2E
             BitOperationUtils=2EsetBit(targetBuf, tar=
getOff, ANTIMATTER_BIT_OFFSET);
@@ -100,4 +103,10 @@=

     public void setAntimatter(boolean isDelete) {
         this=2EisAntimatter =3D isDelete;
     }<=
br>+
+    @Override
+    protected int getAdjustedFieldIdx(int fieldIdx) =
{
+        // 2 bits =
when update-aware: 1 for antimatter and 1 for the update, otherwise, only 1=
 bit for the antimatter
+        return updateAware ? fieldIdx + 2 : fieldIdx + 1;
+    }
 }diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree=
/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTuple=
WriterFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btr=
ee/src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTup=
leWriterFactory=2Ejava
index ad4f6eb=2E=2E03bce23 100644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/=
src/main/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTupleW=
riterFactory=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks=
-storage-am-lsm-btree/src/main/java/org/apache/hyracks/storage/am/lsm/btree=
/tuples/LSMBTreeTupleWriterFactory=2Ejava
@@ -22,6 +22,7 @@=

 import org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Evalue=2E=
ITypeTraits;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=
=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriter;
 import org=
=2Eapache=2Ehyracks=2Estorage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWr=
iterFactory;
+import =
org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
 
 public class LSMBTreeTupleWriterFactory e=
xtends BTreeTypeAwareTupleWriterFactory {
 
@@ -30,15 +31,16 @@
     private final boolean isAntimatte=
r;
 
     public LSMBTreeTupleWriterFactory=
(ITypeTraits[] typeTraits, int numKeyFields, boolean isAntimatter,-            boolean updateAwar=
e) {
-        super(typ=
eTraits, updateAware);
+        super(typeTraits, updateAware, nullTypeTraits, nullIntrospec=
tor);
         this=2EnumKeyFields =3D numKeyFields;=

         this=2EisAntimatter =3D isAntimatter;
  =
   }
 
     @Override
     =
public BTreeTypeAwareTupleWriter createTupleWriter() {
-        return new LSMBTreeTupleWriter(typ=
eTraits, numKeyFields, isAntimatter, updateAware);
+        return new LSMBTreeTupleWriter(typeT=
raits, numKeyFields, isAntimatter, updateAware, nullTypeTraits,
<=
span style=3D"color: hsl(120, 100%, 40%);">+                nullIntrospecto=
r);
     }
 
 }
<=
span>diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/sr=
c/main/java/org/apache/hyracks/storage/am/lsm/btree/utils/LSMBTreeUtil=2Eja=
va b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/o=
rg/apache/hyracks/storage/am/lsm/btree/utils/LSMBTreeUtil=2Ejava
=
index 021f416=2E=2E018c0d3 100644
--- a/hyracks-fulls=
tack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/hyracks/=
storage/am/lsm/btree/utils/LSMBTreeUtil=2Ejava
+++ b/hyrack=
s-fullstack/hyracks/hyracks-storage-am-lsm-btree/src/main/java/org/apache/h=
yracks/storage/am/lsm/btree/utils/LSMBTreeUtil=2Ejava
@@ -3=
4,6 +34,7 @@
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=
=2Ebtree=2Eimpls=2EDiskBTree;
 import org=2Eapache=2Ehyrack=
s=2Estorage=2Eam=2Ebtree=2Etuples=2EBTreeTypeAwareTupleWriterFactory;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EI=
MetadataPageManagerFactory;
+import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINull=
Introspector;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=
=2Ecommon=2Eapi=2EITreeIndexFrameFactory;
 import org=2Eapa=
che=2Ehyracks=2Estorage=2Eam=2Ecommon=2Etuples=2ETypeAwareTupleWriterFactor=
y;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ebt=
ree=2Eimpls=2EExternalBTree;
@@ -76,15 +77,16 @@
=
             boolean needKeyDupCheck, ITypeTraits[] filterTypeTraits,=
 IBinaryComparatorFactory[] filterCmpFactories,
           =
  int[] btreeFields, int[] filterFields, boolean durable, IMetadataPageMana=
gerFactory freePageManagerFactory,
             boolean upd=
ateAware, ITracer tracer, ICompressorDecompressorFactory compressorDecompre=
ssorFactory,
-         =
   boolean hasBloomFilter) throws HyracksDataException {
-        LSMBTreeTupleWriterFactory inser=
tTupleWriterFactory =3D
-                new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories=
=2Elength, false, updateAware);
-        LSMBTreeTupleWriterFactory deleteTupleWriterFactory =3D
-                new LSM=
BTreeTupleWriterFactory(typeTraits, cmpFactories=2Elength, true, updateAwar=
e);
-        LSMBTreeCo=
pyTupleWriterFactory copyTupleWriterFactory =3D
-                new LSMBTreeCopyTupleWriterFactor=
y(typeTraits, cmpFactories=2Elength, updateAware);
-        LSMBTreeTupleWriterFactory bulkLoadTup=
leWriterFactory =3D
-  =
              new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories=2Elen=
gth, false, updateAware);
+            boolean hasBloomFilter, ITypeTraits nullTypeTraits, INull=
Introspector nullIntrospector)
+            throws HyracksDataException {
+        LSMBTreeTupleWriterFactory insert=
TupleWriterFactory =3D new LSMBTreeTupleWriterFactory(typeTraits,+                cmpFactories=
=2Elength, false, updateAware, nullTypeTraits, nullIntrospector);+        LSMBTreeTupleWriterFa=
ctory deleteTupleWriterFactory =3D new LSMBTreeTupleWriterFactory(typeTrait=
s,
+                c=
mpFactories=2Elength, true, updateAware, nullTypeTraits, nullIntrospector);=

+        LSMBTreeCop=
yTupleWriterFactory copyTupleWriterFactory =3D new LSMBTreeCopyTupleWriterF=
actory(typeTraits,
+ =
               cmpFactories=2Elength, updateAware, nullTypeTraits, nullIntr=
ospector);
+        L=
SMBTreeTupleWriterFactory bulkLoadTupleWriterFactory =3D new LSMBTreeTupleW=
riterFactory(typeTraits,
+                cmpFactories=2Elength, false, updateAware, nullTypeTra=
its, nullIntrospector);
 
         ITreeInd=
exFrameFactory insertLeafFrameFactory =3D new BTreeNSMLeafFrameFactory(inse=
rtTupleWriterFactory);
         ITreeIndexFrameFactory copy=
TupleLeafFrameFactory =3D new BTreeNSMLeafFrameFactory(copyTupleWriterFacto=
ry);
@@ -103,7 +105,8 @@
         LSMCompon=
entFilterFrameFactory filterFrameFactory =3D null;
        =
 LSMComponentFilterManager filterManager =3D null;
        =
 if (filterCmpFactories !=3D null) {
-            TypeAwareTupleWriterFactory filterTupleWriterFac=
tory =3D new TypeAwareTupleWriterFactory(filterTypeTraits);
+            TypeAwareTupleWriterFac=
tory filterTupleWriterFactory =3D
+                    new TypeAwareTupleWriterFactory(filterTyp=
eTraits, nullTypeTraits, nullIntrospector);
             fi=
lterHelper =3D new ComponentFilterHelper(filterTupleWriterFactory, filterCm=
pFactories);
             filterFrameFactory =3D new LSMCom=
ponentFilterFrameFactory(filterTupleWriterFactory);
       =
      filterManager =3D new LSMComponentFilterManager(filterFrameFactory);<=
/span>
@@ -136,21 +139,21 @@
             int[] bl=
oomFilterKeyFields, double bloomFilterFalsePositiveRate, ILSMMergePolicy me=
rgePolicy,
             ILSMOperationTracker opTracker, ILS=
MIOOperationScheduler ioScheduler,
             ILSMIOOpera=
tionCallbackFactory ioOpCallbackFactory, ILSMPageWriteCallbackFactory pageW=
riteCallbackFactory,
- =
           boolean durable, IMetadataPageManagerFactory freePageManagerFact=
ory, ITracer tracer)
- =
           throws HyracksDataException {
-        LSMBTreeTupleWriterFactory insertTupleWriterFact=
ory =3D
-              =
  new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories=2Elength, false, =
false);
-        LSMBTr=
eeTupleWriterFactory deleteTupleWriterFactory =3D
-                new LSMBTreeTupleWriterFactory(=
typeTraits, cmpFactories=2Elength, true, false);
-        LSMBTreeCopyTupleWriterFactory copyTuple=
WriterFactory =3D
-    =
            new LSMBTreeCopyTupleWriterFactory(typeTraits, cmpFactories=2El=
ength, false);
+     =
       boolean durable, IMetadataPageManagerFactory freePageManagerFactory,=
 ITracer tracer,
+   =
         ITypeTraits nullTypeTraits, INullIntrospector nullIntrospector) th=
rows HyracksDataException {
+        LSMBTreeTupleWriterFactory insertTupleWriterFactory =3D new=
 LSMBTreeTupleWriterFactory(typeTraits,
+                cmpFactories=2Elength, false, false, nu=
llTypeTraits, nullIntrospector);
+        LSMBTreeTupleWriterFactory deleteTupleWriterFactory =
=3D new LSMBTreeTupleWriterFactory(typeTraits,
+                cmpFactories=2Elength, true, fal=
se, nullTypeTraits, nullIntrospector);
+        LSMBTreeCopyTupleWriterFactory copyTupleWriterFa=
ctory =3D new LSMBTreeCopyTupleWriterFactory(typeTraits,
+                cmpFactories=2Elength,=
 false, nullTypeTraits, nullIntrospector);
         ITreeIn=
dexFrameFactory insertLeafFrameFactory =3D new BTreeNSMLeafFrameFactory(ins=
ertTupleWriterFactory);
         ITreeIndexFrameFactory cop=
yTupleLeafFrameFactory =3D new BTreeNSMLeafFrameFactory(copyTupleWriterFact=
ory);
         ITreeIndexFrameFactory deleteLeafFrameFactor=
y =3D new BTreeNSMLeafFrameFactory(deleteTupleWriterFactory);
         ITreeIndexFrameFactory interiorFrameFactory =3D new BTreeNSMInt=
eriorFrameFactory(insertTupleWriterFactory);
         // Th=
is is the tuple writer that can do both inserts and deletes
-        LSMBTreeTupleWriterFactory tr=
ansactionTupleWriterFactory =3D
-                new LSMBTreeTupleWriterFactory(typeTraits, cmpFac=
tories=2Elength, false, false);
+        LSMBTreeTupleWriterFactory transactionTupleWriterFactor=
y =3D new LSMBTreeTupleWriterFactory(typeTraits,
+                cmpFactories=2Elength, false, =
false, nullTypeTraits, nullIntrospector);
         // This =
is the leaf frame factory for transaction components since it
         // can be used for both inserts and deletes
   =
      ITreeIndexFrameFactory transactionLeafFrameFactory =3D
@@ -185,7 +188,8 @@
             double bloomFilterFalseP=
ositiveRate, ILSMMergePolicy mergePolicy, ILSMOperationTracker opTracker,
             ILSMIOOperationScheduler ioScheduler, ILSMIOOpe=
rationCallbackFactory ioOpCallbackFactory,
             ILS=
MPageWriteCallbackFactory pageWriteCallbackFactory, int[] buddyBTreeFields,=
 boolean durable,
-    =
        IMetadataPageManagerFactory freePageManagerFactory, ITracer tracer)=
 throws HyracksDataException {
+            IMetadataPageManagerFactory freePageManagerFactory, =
ITracer tracer, ITypeTraits nullTypeTraits,
+            INullIntrospector nullIntrospector) thr=
ows HyracksDataException {
         ITypeTraits[] buddyBtre=
eTypeTraits =3D new ITypeTraits[buddyBTreeFields=2Elength];
         IBinaryComparatorFactory[] buddyBtreeCmpFactories =3D new IBinary=
ComparatorFactory[buddyBTreeFields=2Elength];
         for =
(int i =3D 0; i < buddyBtreeTypeTraits=2Elength; i++) {
=
@@ -193,15 +197,15 @@
             buddyBtreeCmpFactories[i=
] =3D cmpFactories[buddyBTreeFields[i]];
         }<=
br>         BTreeTypeAwareTupleWriterFactory buddyBtreeTupleWriterFac=
tory =3D
-             =
   new BTreeTypeAwareTupleWriterFactory(buddyBtreeTypeTraits, false);
+                new BTree=
TypeAwareTupleWriterFactory(buddyBtreeTypeTraits, false, nullTypeTraits, nu=
llIntrospector);
         ITreeIndexFrameFactory buddyBtree=
InteriorFrameFactory =3D
                 new BTreeNSMInter=
iorFrameFactory(buddyBtreeTupleWriterFactory);
         ITr=
eeIndexFrameFactory buddyBtreeLeafFrameFactory =3D new BTreeNSMLeafFrameFac=
tory(buddyBtreeTupleWriterFactory);
 
-        LSMBTreeTupleWriterFactory insertTu=
pleWriterFactory =3D
- =
               new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories=2Ele=
ngth, false, false);
- =
       LSMBTreeCopyTupleWriterFactory copyTupleWriterFactory =3D
=
-                new LSMBTreeCopy=
TupleWriterFactory(typeTraits, cmpFactories=2Elength, false);
+        LSMBTreeTupleWriterFactor=
y insertTupleWriterFactory =3D new LSMBTreeTupleWriterFactory(typeTraits,
+                cmpFa=
ctories=2Elength, false, false, nullTypeTraits, nullIntrospector);+        LSMBTreeCopyTupleWri=
terFactory copyTupleWriterFactory =3D new LSMBTreeCopyTupleWriterFactory(ty=
peTraits,
+          =
      cmpFactories=2Elength, false, nullTypeTraits, nullIntrospector);
         ITreeIndexFrameFactory insertLeafFrameFactory =3D new =
BTreeNSMLeafFrameFactory(insertTupleWriterFactory);
       =
  ITreeIndexFrameFactory copyTupleLeafFrameFactory =3D new BTreeNSMLeafFram=
eFactory(copyTupleWriterFactory);
         ITreeIndexFrameF=
actory interiorFrameFactory =3D new BTreeNSMInteriorFrameFactory(insertTupl=
eWriterFactory);
diff --git a/hyracks-fullstack/hyracks/hyr=
acks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/=
common/dataflow/LsmResource=2Ejava b/hyracks-fullstack/hyracks/hyracks-stor=
age-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/da=
taflow/LsmResource=2Ejava
index fbd74dc=2E=2Ed905a0a 100644=

--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-com=
mon/src/main/java/org/apache/hyracks/storage/am/lsm/common/dataflow/LsmReso=
urce=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage=
-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/dataf=
low/LsmResource=2Ejava
@@ -27,6 +27,7 @@
 i=
mport org=2Eapache=2Ehyracks=2Eapi=2Eexceptions=2EHyracksDataException;
 import org=2Eapache=2Ehyracks=2Eapi=2Eio=2EIPersistedResource=
Registry;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ec=
ommon=2Eapi=2EIMetadataPageManagerFactory;
+import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Ecommo=
n=2Eapi=2EINullIntrospector;
 import org=2Eapache=2Ehyracks=
=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationCallbackFactory;
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2E=
api=2EILSMIOOperationSchedulerProvider;
 import org=2Eapach=
e=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMMergePolicyFactory;
@@ -49,7 +50,7 @@
  */
 publ=
ic abstract class LsmResource implements IResource {
 
-    private static final l=
ong serialVersionUID =3D 1L;
+    private static final long serialVersionUID =3D 2L;
<=
span>     protected static final ObjectMapper OBJECT_MAPPER =3D new ObjectM=
apper();
     protected String path;
     p=
rotected final IStorageManager storageManager;
@@ -67,6 +68=
,8 @@
     protected final ILSMMergePolicyFactory mergePoli=
cyFactory;
     protected final Map<String, String> m=
ergePolicyProperties;
     protected final boolean durable;=

+    protected final=
 ITypeTraits nullTypeTraits;
+    protected final INullIntrospector nullIntrospector;
=
 
     public LsmResource(String path, IStorageManage=
r storageManager, ITypeTraits[] typeTraits,
             IB=
inaryComparatorFactory[] cmpFactories, ITypeTraits[] filterTypeTraits,
@@ -75,7 +78,8 @@
             ILSMPageWriteCal=
lbackFactory pageWriteCallbackFactory,
             IMetada=
taPageManagerFactory metadataPageManagerFactory, IVirtualBufferCacheProvide=
r vbcProvider,
             ILSMIOOperationSchedulerProvide=
r ioSchedulerProvider, ILSMMergePolicyFactory mergePolicyFactory,-            Map<String, Stri=
ng> mergePolicyProperties, boolean durable) {
+            Map<String, String> mergePol=
icyProperties, boolean durable, ITypeTraits nullTypeTraits,
+            INullIntrospector nullI=
ntrospector) {
         this=2Epath =3D path;
         this=2EstorageManager =3D storageManager;
     =
    this=2EtypeTraits =3D typeTraits;
@@ -92,6 +96,8 @@
         this=2EmergePolicyFactory =3D mergePolicyFactory;
         this=2EmergePolicyProperties =3D mergePolicyPropertie=
s;
         this=2Edurable =3D durable;
+        this=2EnullTypeTraits =3D nullT=
ypeTraits;
+        t=
his=2EnullIntrospector =3D nullIntrospector;
     }<=
br> 
     protected LsmResource(IPersistedResourceReg=
istry registry, JsonNode json) throws HyracksDataException {
@@ -156,6 +162,16 @@
         mergePolicyFactory =3D (ILS=
MMergePolicyFactory) registry=2Edeserialize(json=2Eget("mergePolicyFac=
tory"));
         mergePolicyProperties =3D OBJECT_MAP=
PER=2EconvertValue(json=2Eget("mergePolicyProperties"), Map=2Ecla=
ss);
         durable =3D json=2Eget("durable")=
=2EasBoolean();
+    =
    if (json=2EhasNonNull("nullTypeTraits")) {
+            nullTypeTraits =3D (ITypeT=
raits) registry=2Edeserialize(json=2Eget("nullTypeTraits"));
+        } else {<=
br>+            nullTypeTraits =
=3D null;
+        }<=
/span>
+        if (json=2Eh=
asNonNull("nullIntrospector")) {
+            nullIntrospector =3D (INullIntrospector)=
 registry=2Edeserialize(json=2Eget("nullIntrospector"));+        } else {
<=
span style=3D"color: hsl(120, 100%, 40%);">+            nullIntrospector =
=3D null;
+        }<=
/span>
     }
 
     protected voi=
d appendToJson(final ObjectNode json, IPersistedResourceRegistry registry)<=
/span>
@@ -206,6 +222,16 @@
         json=2Eset(&q=
uot;mergePolicyFactory", mergePolicyFactory=2EtoJson(registry));
         json=2EputPOJO("mergePolicyProperties", merge=
PolicyProperties);
         json=2Eput("durable",=
 durable);
+        i=
f (nullTypeTraits !=3D null) {
+            json=2Eset("nullTypeTraits", nullTypeTrait=
s=2EtoJson(registry));
+=
            json=2Eset("nullTypeTraits", null);
+        }
+        if (nullIntrospector !=3D null) {
+            json=2Eset(&q=
uot;nullIntrospector", nullIntrospector=2EtoJson(registry));+        } else {
+            json=2Eset("nul=
lIntrospector", null);
+        }
     }
 
=
     @Override
diff --git a/hyracks-fullstack/hyracks/hyrac=
ks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/co=
mmon/dataflow/LsmResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks=
-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/comm=
on/dataflow/LsmResourceFactory=2Ejava
index a138f01=2E=2Ee9=
7a779 100644
--- a/hyracks-fullstack/hyracks/hyracks-storag=
e-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/data=
flow/LsmResourceFactory=2Ejava
+++ b/hyracks-fullstack/hyra=
cks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/=
am/lsm/common/dataflow/LsmResourceFactory=2Ejava
@@ -23,6 +=
23,7 @@
 import org=2Eapache=2Ehyracks=2Eapi=2Edataflow=2Ev=
alue=2EIBinaryComparatorFactory;
 import org=2Eapache=2Ehyr=
acks=2Eapi=2Edataflow=2Evalue=2EITypeTraits;
 import org=2E=
apache=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EIMetadataPageManagerFactor=
y;
+import org=2Eapac=
he=2Ehyracks=2Estorage=2Eam=2Ecommon=2Eapi=2EINullIntrospector;
<=
span> import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EI=
LSMIOOperationCallbackFactory;
 import org=2Eapache=2Ehyrac=
ks=2Estorage=2Eam=2Elsm=2Ecommon=2Eapi=2EILSMIOOperationSchedulerProvider;<=
/span>
 import org=2Eapache=2Ehyracks=2Estorage=2Eam=2Elsm=2Ecommo=
n=2Eapi=2EILSMMergePolicyFactory;
@@ -34,7 +35,7 @@<=
br> 
 public abstract class LsmResourceFactory implem=
ents IResourceFactory {
 
-    private static final long serialVersionUID =3D 1L;<=
/span>
+    private static f=
inal long serialVersionUID =3D 2L;
     protected final ISt=
orageManager storageManager;
     protected final ITypeTrai=
ts[] typeTraits;
     protected final IBinaryComparatorFact=
ory[] cmpFactories;
@@ -50,6 +51,8 @@
     =
protected final ILSMMergePolicyFactory mergePolicyFactory;
=
     protected final Map<String, String> mergePolicyProperties;
     protected final boolean durable;
+    protected final ITypeTraits nullTypeTraits=
;
+    protected fina=
l INullIntrospector nullIntrospector;
 
   =
  public LsmResourceFactory(IStorageManager storageManager, ITypeTraits[] t=
ypeTraits,
             IBinaryComparatorFactory[] cmpFacto=
ries, ITypeTraits[] filterTypeTraits,
@@ -58,7 +61,8 @@
             ILSMPageWriteCallbackFactory pageWriteCallbackFac=
tory,
             IMetadataPageManagerFactory metadataPage=
ManagerFactory, IVirtualBufferCacheProvider vbcProvider,
  =
           ILSMIOOperationSchedulerProvider ioSchedulerProvider, ILSMMergeP=
olicyFactory mergePolicyFactory,
-            Map<String, String> mergePolicyProperties, boo=
lean durable) {
+    =
        Map<String, String> mergePolicyProperties, boolean durable, I=
TypeTraits nullTypeTraits,
+            INullIntrospector nullIntrospector) {
  =
       this=2EstorageManager =3D storageManager;
         t=
his=2EtypeTraits =3D typeTraits;
         this=2EcmpFactori=
es =3D cmpFactories;
@@ -74,5 +78,7 @@
    =
     this=2EmergePolicyFactory =3D mergePolicyFactory;
    =
     this=2EmergePolicyProperties =3D mergePolicyProperties;
         this=2Edurable =3D durable;
+        this=2EnullTypeTraits =3D nullTypeTraits;=

+        this=2EnullIntrosp=
ector =3D nullIntrospector;
     }
 }
diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-in=
vertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/d=
ataflow/LSMInvertedIndexLocalResource=2Ejava b/hyracks-fullstack/hyracks/hy=
racks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage=
/am/lsm/invertedindex/dataflow/LSMInvertedIndexLocalResource=2Ejava<=
br>index 0af1f3f=2E=2E8550385 100644
--- a/hyracks-fu=
llstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apac=
he/hyracks/storage/am/lsm/invertedindex/dataflow/LSMInvertedIndexLocalResou=
rce=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage-=
am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invert=
edindex/dataflow/LSMInvertedIndexLocalResource=2Ejava
@@ -9=
4,8 +94,7 @@
 
         super(path, storage=
Manager, typeTraits, cmpFactories, filterTypeTraits, filterCmpFactories, fi=
lterFields,
                 opTrackerProvider, ioOpCallbac=
kFactory, pageWriteCallbackFactory, metadataPageManagerFactory,
<=
span style=3D"color: hsl(0, 100%, 40%);">-                vbcProvider, ioSc=
hedulerProvider, mergePolicyFactory, mergePolicyProperties, durable);
-
+                vbcProvider, ioSchedulerProvid=
er, mergePolicyFactory, mergePolicyProperties, durable, null, null);=

         this=2EtokenTypeTraits =3D tokenTypeTraits;
         this=2EtokenCmpFactories =3D tokenCmpFactories;
         this=2EtokenizerFactory =3D tokenizerFactory;
di=
ff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/s=
rc/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/dataflow/LSMIn=
vertedIndexLocalResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-=
storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/ls=
m/invertedindex/dataflow/LSMInvertedIndexLocalResourceFactory=2Ejava=

index 09c9699=2E=2E79c6123 100644
--- a/hyracks-f=
ullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apa=
che/hyracks/storage/am/lsm/invertedindex/dataflow/LSMInvertedIndexLocalReso=
urceFactory=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-=
storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/ls=
m/invertedindex/dataflow/LSMInvertedIndexLocalResourceFactory=2Ejava=

@@ -65,7 +65,7 @@
             int[] invertedInde=
xFieldsForNonBulkLoadOps, double bloomFilterFalsePositiveRate) {
=
         super(storageManager, typeTraits, cmpFactories, filterTypeTr=
aits, filterCmpFactories, filterFields,
                 op=
TrackerFactory, ioOpCallbackFactory, pageWriteCallbackFactory, metadataPage=
ManagerFactory,
-      =
          vbcProvider, ioSchedulerProvider, mergePolicyFactory, mergePolicy=
Properties, durable);
+                vbcProvider, ioSchedulerProvider, mergePolicyFactory, mer=
gePolicyProperties, durable, null, null);
         this=2Et=
okenTypeTraits =3D tokenTypeTraits;
         this=2EtokenCm=
pFactories =3D tokenCmpFactories;
         // ToDo: totally=
 replace tokenizerFactory with full-text config
diff --git =
a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/j=
ava/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvertedIndex=
=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/sr=
c/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInvert=
edIndex=2Ejava
index de65e3e=2E=2Ef2d5a37 100644
=
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/=
src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/impls/LSMInve=
rtedIndex=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-st=
orage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/=
invertedindex/impls/LSMInvertedIndex=2Ejava
@@ -120,10 +120=
,10 @@
         for (IVirtualBufferCache virtualBufferCache=
 : virtualBufferCaches) {
             InMemoryInvertedInde=
x memInvIndex =3D
                     createInMemoryInvert=
edIndex(virtualBufferCache, new VirtualFreePageManager(virtualBufferCache),=
 i);
-            BTree=
 deleteKeysBTree =3D
- =
                   BTreeUtils=2EcreateBTree(virtualBufferCache, new Virtual=
FreePageManager(virtualBufferCache),
-                            invListTypeTraits, invListCmpFac=
tories, BTreeLeafFrameType=2EREGULAR_NSM,
-                            ioManager=2EresolveAbsolute=
Path(fileManager=2EgetBaseDir() + "_virtual_del_" + i), false);
+            BTree del=
eteKeysBTree =3D BTreeUtils=2EcreateBTree(virtualBufferCache,
+                    new VirtualFr=
eePageManager(virtualBufferCache), invListTypeTraits, invListCmpFactories,<=
/span>
+                    =
BTreeLeafFrameType=2EREGULAR_NSM,
+                    ioManager=2EresolveAbsolutePath(fileManag=
er=2EgetBaseDir() + "_virtual_del_" + i), false, null, null);
             LSMInvertedIndexMemoryComponent mutableComponent =
=3D new LSMInvertedIndexMemoryComponent(this, memInvIndex,
=
                     deleteKeysBTree, virtualBufferCache, filterHelper =3D=
=3D null ? null : filterHelper=2EcreateFilter());
         =
    memoryComponents=2Eadd(mutableComponent);
diff --git a/=
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/jav=
a/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/InMemoryInverted=
Index=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedind=
ex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/inmemory/I=
nMemoryInvertedIndex=2Ejava
index 6998861=2E=2Ec2e07b3 1006=
44
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-i=
nvertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/=
inmemory/InMemoryInvertedIndex=2Ejava
+++ b/hyracks-fullsta=
ck/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hy=
racks/storage/am/lsm/invertedindex/inmemory/InMemoryInvertedIndex=2Ejava
@@ -80,7 +80,7 @@
             btreeCmpFactor=
ies[tokenTypeTraits=2Elength + i] =3D invListCmpFactories[i];
         }
         this=2Ebtree =3D BTreeUtils=2Ecreate=
BTree(virtualBufferCache, virtualFreePageManager, btreeTypeTraits,-                btreeCmpFactor=
ies, BTreeLeafFrameType=2EREGULAR_NSM, btreeFileRef, false);
+                btreeCmpFactories,=
 BTreeLeafFrameType=2EREGULAR_NSM, btreeFileRef, false, null, null);=

     }
 
     @Overridediff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invert=
edindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondis=
k/OnDiskInvertedIndex=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am=
-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/inverted=
index/ondisk/OnDiskInvertedIndex=2Ejava
index 0f26435=2E=2E=
7a45c5d 100644
--- a/hyracks-fullstack/hyracks/hyracks-stor=
age-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/in=
vertedindex/ondisk/OnDiskInvertedIndex=2Ejava
+++ b/hyracks=
-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/a=
pache/hyracks/storage/am/lsm/invertedindex/ondisk/OnDiskInvertedIndex=2Ejav=
a
@@ -127,7 +127,8 @@
         this=2Etoken=
TypeTraits =3D tokenTypeTraits;
         this=2EtokenCmpFac=
tories =3D tokenCmpFactories;
         this=2Ebtree =3D BTr=
eeUtils=2EcreateDiskBTree(bufferCache, getBTreeTypeTraits(tokenTypeTraits),=
 tokenCmpFactories,
-  =
              BTreeLeafFrameType=2EREGULAR_NSM, btreeFile, pageManagerFacto=
ry=2EcreatePageManager(bufferCache), false);
+                BTreeLeafFrameType=2EREGULAR_NSM, =
btreeFile, pageManagerFactory=2EcreatePageManager(bufferCache), false,
+                null, nu=
ll);
         this=2EnumTokenFields =3D btree=2EgetComparat=
orFactories()=2Elength;
         this=2EnumInvListKeys =3D =
invListCmpFactories=2Elength;
         this=2EinvListsFile =
=3D invListsFile;
diff --git a/hyracks-fullstack/hyracks/hy=
racks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage=
/am/lsm/invertedindex/ondisk/variablesize/VariableSizeElementInvertedListBu=
ilder=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedind=
ex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/var=
iablesize/VariableSizeElementInvertedListBuilder=2Ejava
ind=
ex cfbb779=2E=2E3ce26c5 100644
--- a/hyracks-fullstack/hyra=
cks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/s=
torage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeElementInverted=
ListBuilder=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-=
storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/ls=
m/invertedindex/ondisk/variablesize/VariableSizeElementInvertedListBuilder=
=2Ejava
@@ -46,7 +46,7 @@
         for (int=
 i =3D 0; i < invListFields=2Elength; i++) {
           =
  allFields[i + tokenTypeTraits=2Elength] =3D invListFields[i];
<=
span>         }
-      =
  this=2Ewriter =3D new TypeAwareTupleWriter(allFields);
+        this=2Ewriter =3D new TypeAwar=
eTupleWriter(allFields, null, null);
 
    =
     InvertedIndexUtils=2EverifyHasVarSizeTypeTrait(invListFields);<=
br>     }
diff --git a/hyracks-fullstack/hyracks/hyra=
cks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/a=
m/lsm/invertedindex/ondisk/variablesize/VariableSizeElementOnDiskInvertedLi=
stCursor=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inverted=
index/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/=
variablesize/VariableSizeElementOnDiskInvertedListCursor=2Ejava
<=
span>index ee2bc58=2E=2Ea5515b3 100644
--- a/hyracks-fullst=
ack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/h=
yracks/storage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeElement=
OnDiskInvertedListCursor=2Ejava
+++ b/hyracks-fullstack/hyr=
acks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/=
storage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeElementOnDiskI=
nvertedListCursor=2Ejava
@@ -53,7 +53,7 @@
=
         super(bufferCache, fileId, invListFields, stats);
=
         this=2EisInit =3D true;
         this=2EtupleRefer=
ence =3D new VariableSizeInvertedListTupleReference(invListFields);<=
br>-        this=2EtupleWriter =
=3D new TypeAwareTupleWriter(invListFields);
+        this=2EtupleWriter =3D new TypeAwareTupleW=
riter(invListFields, null, null);
     }
 <=
/span>
     public VariableSizeElementOnDiskInvertedListCursor(IBu=
fferCache bufferCache, int fileId,
@@ -70,7 +70,7 @@=

         currentElementIxForScan =3D 0;
         =
isInit =3D true;
         this=2EtupleReference =3D new Var=
iableSizeInvertedListTupleReference(invListFields);
-        this=2EtupleWriter =3D new TypeAwar=
eTupleWriter(invListFields);
+        this=2EtupleWriter =3D new TypeAwareTupleWriter(invListFie=
lds, null, null);
     }
 
=
     /**
diff --git a/hyracks-fullstack/hyracks/hyracks-sto=
rage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/i=
nvertedindex/ondisk/variablesize/VariableSizeInvertedListSearchResultFrameT=
upleAccessor=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inve=
rtedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ond=
isk/variablesize/VariableSizeInvertedListSearchResultFrameTupleAccessor=2Ej=
ava
index ab71c0a=2E=2E75d0cfe 100644
--- a=
/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/ja=
va/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/variablesize/Vari=
ableSizeInvertedListSearchResultFrameTupleAccessor=2Ejava
+=
++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/mai=
n/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/variablesize/=
VariableSizeInvertedListSearchResultFrameTupleAccessor=2Ejava
@@ -58,7 +58,7 @@
             throws HyracksDataExcepti=
on {
         super(frameSize, fields);
 
-        this=2EtupleWri=
ter =3D new TypeAwareTupleWriter(fields);
+        this=2EtupleWriter =3D new TypeAwareTupleWrit=
er(fields, null, null);
         this=2EtupleReference =3D =
new VariableSizeInvertedListTupleReference(fields);
     }<=
/span>
 
diff --git a/hyracks-fullstack/hyracks/hy=
racks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage=
/am/lsm/invertedindex/ondisk/variablesize/VariableSizeInvertedListTupleRefe=
rence=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedind=
ex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/ondisk/var=
iablesize/VariableSizeInvertedListTupleReference=2Ejava
ind=
ex 8e1db66=2E=2E1e69076 100644
--- a/hyracks-fullstack/hyra=
cks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/s=
torage/am/lsm/invertedindex/ondisk/variablesize/VariableSizeInvertedListTup=
leReference=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-=
storage-am-lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/ls=
m/invertedindex/ondisk/variablesize/VariableSizeInvertedListTupleReference=
=2Ejava
@@ -38,7 +38,7 @@
     public Varia=
bleSizeInvertedListTupleReference(ITypeTraits[] typeTraits) throws HyracksD=
ataException {
         super(typeTraits);
=
 
-        this=2Etuple=
Reference =3D new TypeAwareTupleReference(typeTraits);
+        this=2EtupleReference =3D new Ty=
peAwareTupleReference(typeTraits, null);
     }
<=
span> 
     @Override
diff --git a/hyracks-=
fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/ap=
ache/hyracks/storage/am/lsm/invertedindex/search/InvertedIndexSearchResult=
=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/sr=
c/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/search/Inverted=
IndexSearchResult=2Ejava
index 707d85d=2E=2E6958fd5 100644<=
/span>
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-inve=
rtedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedindex/sea=
rch/InvertedIndexSearchResult=2Ejava
+++ b/hyracks-fullstac=
k/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apache/hyr=
acks/storage/am/lsm/invertedindex/search/InvertedIndexSearchResult=2Ejava
@@ -86,7 +86,7 @@
     public InvertedIndexS=
earchResult(ITypeTraits[] invListFields, IHyracksTaskContext ctx,             ISimpleFrameBufferManager bufferManager) throws Hyracks=
DataException {
         this=2EinvListFields =3D invListFi=
elds;
-        this=2Et=
upleWriter =3D new TypeAwareTupleWriter(invListFields);
+        this=2EtupleWriter =3D new Type=
AwareTupleWriter(invListFields, null, null);
         initT=
ypeTraits(invListFields);
         this=2Ectx =3D ctx;
         appender =3D new InvertedListSearchResultFrameTupleApp=
ender(ctx=2EgetInitialFrameSize());
diff --git a/hyracks-fu=
llstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/org/apac=
he/hyracks/storage/am/lsm/invertedindex/util/InvertedIndexUtils=2Ejava b/hy=
racks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/java/=
org/apache/hyracks/storage/am/lsm/invertedindex/util/InvertedIndexUtils=2Ej=
ava
index dcde832=2E=2Ebee6be6 100644
--- a=
/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-invertedindex/src/main/ja=
va/org/apache/hyracks/storage/am/lsm/invertedindex/util/InvertedIndexUtils=
=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage-am-=
lsm-invertedindex/src/main/java/org/apache/hyracks/storage/am/lsm/invertedi=
ndex/util/InvertedIndexUtils=2Ejava
@@ -130,7 +130,7 @@
             IBinaryComparatorFactory[] invListCmpFactories, I=
BufferCache diskBufferCache,
             IPageManagerFacto=
ry freePageManagerFactory) throws HyracksDataException {
  =
       BTreeTypeAwareTupleWriterFactory tupleWriterFactory =3D
-                new BTreeTypeAware=
TupleWriterFactory(invListTypeTraits, false);
+                new BTreeTypeAwareTupleWriterFact=
ory(invListTypeTraits, false, null, null);
         ITreeIn=
dexFrameFactory leafFrameFactory =3D
                 BTree=
Utils=2EgetLeafFrameFactory(tupleWriterFactory, BTreeLeafFrameType=2EREGULA=
R_NSM);
         ITreeIndexFrameFactory interiorFrameFactor=
y =3D new BTreeNSMInteriorFrameFactory(tupleWriterFactory);
@@ -174,7 +174,8 @@
         LSMComponentFilterFrameFactor=
y filterFrameFactory =3D null;
         LSMComponentFilterM=
anager filterManager =3D null;
         if (filterCmpFactor=
ies !=3D null) {
-     =
       TypeAwareTupleWriterFactory filterTupleWriterFactory =3D new TypeAwa=
reTupleWriterFactory(filterTypeTraits);
+            TypeAwareTupleWriterFactory filterTupleWrit=
erFactory =3D
+      =
              new TypeAwareTupleWriterFactory(filterTypeTraits, null, null)=
;
             filterHelper =3D new ComponentFilterHelper(f=
ilterTupleWriterFactory, filterCmpFactories);
             =
filterFrameFactory =3D new LSMComponentFilterFrameFactory(filterTupleWriter=
Factory);
             filterManager =3D new LSMComponentFi=
lterManager(filterFrameFactory);
@@ -225,7 +226,8 @@=

         LSMComponentFilterFrameFactory filterFrameFactory =3D nu=
ll;
         LSMComponentFilterManager filterManager =3D nu=
ll;
         if (filterCmpFactories !=3D null) {
=
-            TypeAwareTupleWriter=
Factory filterTupleWriterFactory =3D new TypeAwareTupleWriterFactory(filter=
TypeTraits);
+       =
     TypeAwareTupleWriterFactory filterTupleWriterFactory =3D
+                    new TypeAware=
TupleWriterFactory(filterTypeTraits, null, null);
         =
    filterHelper =3D new ComponentFilterHelper(filterTupleWriterFactory, fi=
lterCmpFactories);
             filterFrameFactory =3D new =
LSMComponentFilterFrameFactory(filterTupleWriterFactory);
 =
            filterManager =3D new LSMComponentFilterManager(filterFrameFact=
ory);
diff --git a/hyracks-fullstack/hyracks/hyracks-storag=
e-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/datafl=
ow/LSMRTreeLocalResource=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage=
-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflo=
w/LSMRTreeLocalResource=2Ejava
index bfae1cd=2E=2Ec7a59f4 1=
00644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-ls=
m-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMR=
TreeLocalResource=2Ejava
+++ b/hyracks-fullstack/hyracks/hy=
racks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/=
rtree/dataflow/LSMRTreeLocalResource=2Ejava
@@ -76,7 +76,7 =
@@
             boolean isPointMBR, double bloomFilterFalse=
PositiveRate) {
         super(path, storageManager, typeTr=
aits, rtreeCmpFactories, filterTypeTraits, filterCmpFactories, filterFields=
,
                 opTrackerProvider, ioOpCallbackFactory, =
pageWriteCallbackFactory, metadataPageManagerFactory,
-                vbcProvider, ioSchedulerPro=
vider, mergePolicyFactory, mergePolicyProperties, durable);
+                vbcProvider, ioSche=
dulerProvider, mergePolicyFactory, mergePolicyProperties, durable, null, nu=
ll);
         this=2EbtreeCmpFactories =3D btreeCmpFactorie=
s;
         this=2EvalueProviderFactories =3D valueProvider=
Factories;
         this=2ErtreePolicyType =3D rtreePolicyT=
ype;
diff --git a/hyracks-fullstack/hyracks/hyracks-storage=
-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflo=
w/LSMRTreeLocalResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-s=
torage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/d=
ataflow/LSMRTreeLocalResourceFactory=2Ejava
index 635f1ac=
=2E=2E31f78d8 100644
--- a/hyracks-fullstack/hyracks/hyrack=
s-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtre=
e/dataflow/LSMRTreeLocalResourceFactory=2Ejava
+++ b/hyrack=
s-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/h=
yracks/storage/am/lsm/rtree/dataflow/LSMRTreeLocalResourceFactory=2Ejava
@@ -63,7 +63,7 @@
             boolean isPoin=
tMBR, double bloomFilterFalsePositiveRate) {
         super=
(storageManager, typeTraits, cmpFactories, filterTypeTraits, filterCmpFacto=
ries, filterFields,
                 opTrackerProvider, ioO=
pCallbackFactory, pageWriteCallbackFactory, metadataPageManagerFactory,
-                vbcProvid=
er, ioSchedulerProvider, mergePolicyFactory, mergePolicyProperties, durable=
);
+                v=
bcProvider, ioSchedulerProvider, mergePolicyFactory, mergePolicyProperties,=
 durable, null, null);
         this=2EbtreeCmpFactories =
=3D btreeCmpFactories;
         this=2EvalueProviderFactori=
es =3D valueProviderFactories;
         this=2ErtreePolicyT=
ype =3D rtreePolicyType;
diff --git a/hyracks-fullstack/hyr=
acks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/=
am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLocalResource=2Ejava b/hyracks-=
fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyr=
acks/storage/am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLocalResource=2Eja=
va
index e4a8287=2E=2Ea0d0220 100644
--- a/=
hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/ap=
ache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLocalResou=
rce=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage-=
am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow=
/LSMRTreeWithAntiMatterLocalResource=2Ejava
@@ -73,7 +73,7 =
@@
             ILinearizeComparatorFactory linearizeCmpFac=
tory, int[] rtreeFields, boolean isPointMBR, boolean durable) {
<=
span>         super(path, storageManager, typeTraits, rtreeCmpFactories, fi=
lterTypeTraits, filterCmpFactories, filterFields,
         =
        opTrackerProvider, ioOpCallbackFactory, pageWriteCallbackFactory, m=
etadataPageManagerFactory,
-                vbcProvider, ioSchedulerProvider, mergePolicyFactory, =
mergePolicyProperties, durable);
+                vbcProvider, ioSchedulerProvider, mergePolicyF=
actory, mergePolicyProperties, durable, null, null);
      =
   this=2EbtreeCmpFactories =3D btreeCmpFactories;
        =
 this=2EvalueProviderFactories =3D valueProviderFactories;
=
         this=2ErtreePolicyType =3D rtreePolicyType;
diff -=
-git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java=
/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLoc=
alResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm=
-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRT=
reeWithAntiMatterLocalResourceFactory=2Ejava
index 6edf877=
=2E=2Eed4c5b5 100644
--- a/hyracks-fullstack/hyracks/hyrack=
s-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtre=
e/dataflow/LSMRTreeWithAntiMatterLocalResourceFactory=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/jav=
a/org/apache/hyracks/storage/am/lsm/rtree/dataflow/LSMRTreeWithAntiMatterLo=
calResourceFactory=2Ejava
@@ -60,7 +60,7 @@
             IBinaryComparatorFactory[] btreeComparatorFactories) {=

         super(storageManager, rteeTypeTraits, rtreeCmpFactories,=
 filterTypeTraits, filterCmpFactories, filterFields,
      =
           opTrackerFactory, ioOpCallbackFactory, pageWriteCallbackFactory,=
 metadataPageManagerFactory,
-                vbcProvider, ioSchedulerProvider, mergePolicyFactory=
, mergePolicyProperties, durable);
+                vbcProvider, ioSchedulerProvider, mergePolic=
yFactory, mergePolicyProperties, durable, null, null);
    =
     this=2EvalueProviderFactories =3D valueProviderFactories;
         this=2ErtreePolicyType =3D rtreePolicyType;
  =
       this=2ElinearizeCmpFactory =3D linearizeCmpFactory;
=
diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/mai=
n/java/org/apache/hyracks/storage/am/lsm/rtree/tuples/LSMRTreeTupleReferenc=
eForPointMBR=2Ejava b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtre=
e/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/tuples/LSMRTreeTupl=
eReferenceForPointMBR=2Ejava
index e28d5a2=2E=2Eef9eb47 100=
644
--- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-=
rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/tuples/LSMRTree=
TupleReferenceForPointMBR=2Ejava
+++ b/hyracks-fullstack/hy=
racks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage=
/am/lsm/rtree/tuples/LSMRTreeTupleReferenceForPointMBR=2Ejava
@@ -26,6 +26,7 @@
 import org=2Eapache=2Ehyracks=2Estora=
ge=2Eam=2Ertree=2Etuples=2ERTreeTypeAwareTupleReference;
 <=
/span>
 public class LSMRTreeTupleReferenceForPointMBR extends RTr=
eeTypeAwareTupleReference implements ILSMTreeTupleReference {
+
     private fin=
al int inputKeyFieldCount; //double field count for mbr secondary key of an=
 input tuple
     private final int inputTotalFieldCount; /=
/total field count (key + value fields) of an input tuple=2E
     private final int storedKeyFieldCount; //double field count to be st=
ored for the mbr secondary key
@@ -140,6 +141,7 @@ 
     @Override
     public boolea=
n isAntimatter() {
+ =
       // TODO(ali): antimatterAware should be checked?
   =
      // Check antimatter bit=2E
         return BitOperati=
onUtils=2EgetBit(buf, tupleStartOff, ANTIMATTER_BIT_OFFSET);
     }
diff --git a/hyracks-fullstack/hyracks/hyracks-sto=
rage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtree/tup=
les/LSMRTreeTupleWriterForPointMBR=2Ejava b/hyracks-fullstack/hyracks/hyrac=
ks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rtr=
ee/tuples/LSMRTreeTupleWriterForPointMBR=2Ejava
index ce6bd=
eb=2E=2Eae2d642 100644
--- a/hyracks-fullstack/hyracks/hyra=
cks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/lsm/rt=
ree/tuples/LSMRTreeTupleWriterForPointMBR=2Ejava
+++ b/hyra=
cks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache=
/hyracks/storage/am/lsm/rtree/tuples/LSMRTreeTupleWriterForPointMBR=2Ejava<=
/span>
@@ -44,6 +44,7 @@
  */
 
 public class LSMRTreeTupleWriterForPointMBR extends RTreeTyp=
eAwareTupleWriter implements ILSMTreeTupleWriter {
+
     private final int inpu=
tKeyFieldCount; //double field count for mbr secondary key of an input tupl=
e
     private final int valueFieldCount; //value(or payloa=
d or primary key) field count (same for an input tuple and a stored tuple)<=
/span>
     private final int inputTotalFieldCount; //total field =
count (key + value fields) of an input tuple=2E
@@ -157,6 +=
158,7 @@
     }
 
     prot=
ected void setAntimatterBit(byte[] targetBuf, int targetOff) {
+        // TODO(ali): antimatter=
Aware should be checked?
         // Set antimatter bit to =
1=2E
         BitOperationUtils=2EsetBit(targetBuf, targetO=
ff, ANTIMATTER_BIT_OFFSET);
     }
diff --g=
it a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/o=
rg/apache/hyracks/storage/am/lsm/rtree/utils/LSMRTreeUtils=2Ejava b/hyracks=
-fullstack/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hy=
racks/storage/am/lsm/rtree/utils/LSMRTreeUtils=2Ejava
index=
 ba6015d=2E=2Ecf44b2f 100644
--- a/hyracks-fullstack/hyrack=
s/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/storage/am/=
lsm/rtree/utils/LSMRTreeUtils=2Ejava
+++ b/hyracks-fullstac=
k/hyracks/hyracks-storage-am-lsm-rtree/src/main/java/org/apache/hyracks/sto=
rage/am/lsm/rtree/utils/LSMRTreeUtils=2Ejava
@@ -102,7 +102=
,7 @@
             rtreeLeafFrameTupleWriterFactory =3D rtr=
eeInteriorFrameTupleWriterFactory;
         }
         BTreeTypeAwareTupleWriterFactory btreeTupleWriterFactory =3D
-                new BTre=
eTypeAwareTupleWriterFactory(btreeTypeTraits, false);
+                new BTreeTypeAwareTupleWr=
iterFactory(btreeTypeTraits, false, null, null);
         R=
TreeFrameFactory rtreeInteriorFrameFactory =3D new RTreeNSMInteriorFrameFac=
tory(
                 rtreeInteriorFrameTupleWriterFactory=
, valueProviderFactories, rtreePolicyType, isPointMBR);
   =
      RTreeFrameFactory rtreeLeafFrameFactory =3D new RTreeNSMLeafFrameFact=
ory(rtreeLeafFrameTupleWriterFactory,
@@ -127,7 +127,8 @@
         LSMComponentFilterFrameFactory filterFrameFactory =
=3D null;
         LSMComponentFilterManager filterManager =
=3D null;
         if (filterCmpFactories !=3D null) {
-            TypeAwareTuple=
WriterFactory filterTupleWriterFactory =3D new TypeAwareTupleWriterFactory(=
filterTypeTraits);
+ =
           TypeAwareTupleWriterFactory filterTupleWriterFactory =3D<=
br>+                    new Typ=
eAwareTupleWriterFactory(filterTypeTraits, null, null);
   =
          filterHelper =3D new ComponentFilterHelper(filterTupleWriterFacto=
ry, filterCmpFactories);
             filterFrameFactory =
=3D new LSMComponentFilterFrameFactory(filterTupleWriterFactory);             filterManager =3D new LSMComponentFilterManager(filterF=
rameFactory);
@@ -217,7 +218,8 @@
         =
LSMComponentFilterFrameFactory filterFrameFactory =3D null;
         LSMComponentFilterManager filterManager =3D null;
         if (filterCmpFactories !=3D null) {
-            TypeAwareTupleWriterFactory filterTuple=
WriterFactory =3D new TypeAwareTupleWriterFactory(filterTypeTraits);=

+            TypeAwareTuple=
WriterFactory filterTupleWriterFactory =3D
+                    new TypeAwareTupleWriterFactory(=
filterTypeTraits, null, null);
             filterHelper =
=3D new ComponentFilterHelper(filterTupleWriterFactory, filterCmpFactories)=
;
             filterFrameFactory =3D new LSMComponentFilte=
rFrameFactory(filterTupleWriterFactory);
             filte=
rManager =3D new LSMComponentFilterManager(filterFrameFactory);
<=
span>@@ -264,7 +266,7 @@
             rtreeLeafFrameTupleWr=
iterFactory =3D rtreeInteriorFrameTupleWriterFactory;
     =
    }
         BTreeTypeAwareTupleWriterFactory btreeTupleW=
riterFactory =3D
-     =
           new BTreeTypeAwareTupleWriterFactory(btreeTypeTraits, false);
+                new BT=
reeTypeAwareTupleWriterFactory(btreeTypeTraits, false, null, null);<=
br>         RTreeFrameFactory rtreeInteriorFrameFactory =3D new RTree=
NSMInteriorFrameFactory(
                 rtreeInteriorFram=
eTupleWriterFactory, valueProviderFactories, rtreePolicyType, isPointMBR);<=
/span>
         RTreeFrameFactory rtreeLeafFrameFactory =3D new RT=
reeNSMLeafFrameFactory(rtreeLeafFrameTupleWriterFactory,
di=
ff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java=
/org/apache/hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleReference=2E=
java b/hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org=
/apache/hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleReference=2Ejava=

index 61e2326=2E=2E8f7f3f0 100644
--- a/hy=
racks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/h=
yracks/storage/am/rtree/tuples/RTreeTypeAwareTupleReference=2Ejava+++ b/hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/j=
ava/org/apache/hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleReference=
=2Ejava
@@ -27,7 +27,7 @@
 public class RTr=
eeTypeAwareTupleReference extends TypeAwareTupleReference implements ITreeI=
ndexTupleReference {
 
     public RTreeTyp=
eAwareTupleReference(ITypeTraits[] typeTraits) {
-        super(typeTraits);
+        super(typeTraits, null);     }
 
 }
diff --=
git a/hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/=
apache/hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleWriter=2Ejava b/h=
yracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/=
hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleWriter=2Ejava
=
index 40b0481=2E=2E42a9662 100644
--- a/hyracks-fulls=
tack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/stor=
age/am/rtree/tuples/RTreeTypeAwareTupleWriter=2Ejava
+++ b/=
hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache=
/hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleWriter=2Ejava@@ -29,7 +29,7 @@
 public class RTreeTypeAwareTupleW=
riter extends TypeAwareTupleWriter {
 
    =
 public RTreeTypeAwareTupleWriter(ITypeTraits[] typeTraits) {
-        super(typeTraits);+        super(typeTraits, nu=
ll, null);
     }
 
     pu=
blic int writeTupleFields(ITreeIndexTupleReference[] refs, int startField, =
ByteBuffer targetBuf, int targetOff) {
diff --git a/hyracks=
-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyrack=
s/storage/am/rtree/tuples/RTreeTypeAwareTupleWriterFactory=2Ejava b/hyracks=
-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyrack=
s/storage/am/rtree/tuples/RTreeTypeAwareTupleWriterFactory=2Ejavaindex 4536a0c=2E=2E0069cc3 100644
--- a/hyracks-full=
stack/hyracks/hyracks-storage-am-rtree/src/main/java/org/apache/hyracks/sto=
rage/am/rtree/tuples/RTreeTypeAwareTupleWriterFactory=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-storage-am-rtree/src/main/java/or=
g/apache/hyracks/storage/am/rtree/tuples/RTreeTypeAwareTupleWriterFactory=
=2Ejava
@@ -27,7 +27,7 @@
     private stat=
ic final long serialVersionUID =3D 1L;
 
  =
   public RTreeTypeAwareTupleWriterFactory(ITypeTraits[] typeTraits) {
-        super(typeTraits);=

+        super(typeT=
raits, null, null);
     }
 
     @Override
diff --git a/hyracks-fullstack/hyracks/hyr=
acks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/s=
torage/am/btree/BTreeExamplesTest=2Ejava b/hyracks-fullstack/hyracks/hyrack=
s-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/stor=
age/am/btree/BTreeExamplesTest=2Ejava
index 8c0f5a5=2E=2E9d=
91e14 100644
--- a/hyracks-fullstack/hyracks/hyracks-tests/=
hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/b=
tree/BTreeExamplesTest=2Ejava
+++ b/hyracks-fullstack/hyrac=
ks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyr=
acks/storage/am/btree/BTreeExamplesTest=2Ejava
@@ -53,7 +53=
,7 @@
         LinkedMetaDataPageManager freePageManager =
=3D
                 new LinkedMetaDataPageManager(harness=
=2EgetBufferCache(), metaFrameFactory);
         return BTr=
eeUtils=2EcreateBTree(harness=2EgetBufferCache(), typeTraits, cmpFactories,=

-                BTree=
LeafFrameType=2EREGULAR_NSM, harness=2EgetFileReference(), freePageManager,=
 false);
+           =
     BTreeLeafFrameType=2EREGULAR_NSM, harness=2EgetFileReference(), freePa=
geManager, false, null, null);
     }
 
 }
diff --git a/hyracks-fullstack/hyracks/hyra=
cks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/st=
orage/am/btree/BTreeModificationOperationCallbackTest=2Ejava b/hyracks-full=
stack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org=
/apache/hyracks/storage/am/btree/BTreeModificationOperationCallbackTest=2Ej=
ava
index ad9198c=2E=2E786a6af 100644
--- a=
/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/=
test/java/org/apache/hyracks/storage/am/btree/BTreeModificationOperationCal=
lbackTest=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-te=
sts/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/=
am/btree/BTreeModificationOperationCallbackTest=2Ejava
@@ -=
39,7 +39,7 @@
         IMetadataPageManager freePageManager=
 =3D freePageManagerFactory=2EcreatePageManager(harness=2EgetBufferCache())=
;
         index =3D BTreeUtils=2EcreateBTree(harness=2Eget=
BufferCache(), SerdeUtils=2EserdesToTypeTraits(keySerdes),
=
                 SerdeUtils=2EserdesToComparatorFactories(keySerdes, keySer=
des=2Elength), BTreeLeafFrameType=2EREGULAR_NSM,
-                harness=2EgetFileReference(), fr=
eePageManager, false);
-       =
     new BTreeTypeAwareTupleWriterFactory(TYPE_TRAITS, false);
+            new BTreeTypeAwareTu=
pleWriterFactory(TYPE_TRAITS, false, null, null);
     publ=
ic static final ITreeIndexMetadataFrameFactory META_FRAME_FACTORY =3D new L=
IFOMetaDataFrameFactory();
     public static final int KEY=
_FIELDS_COUNT =3D 1;
     public static final IBinaryCompar=
atorFactory[] CMP_FACTORIES =3D { IntegerBinaryComparatorFactory=2EINSTANCE=
 };
diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hy=
racks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btr=
ee/BTreeSearchOperationCallbackTest=2Ejava b/hyracks-fullstack/hyracks/hyra=
cks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/st=
orage/am/btree/BTreeSearchOperationCallbackTest=2Ejava
inde=
x 89399bd=2E=2Ea7b4c4d 100644
--- a/hyracks-fullstack/hyrac=
ks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyr=
acks/storage/am/btree/BTreeSearchOperationCallbackTest=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-t=
est/src/test/java/org/apache/hyracks/storage/am/btree/BTreeSearchOperationC=
allbackTest=2Ejava
@@ -40,7 +40,7 @@
      =
           new LinkedMetaDataPageManager(harness=2EgetBufferCache(), metaFr=
ameFactory);
         index =3D BTreeUtils=2EcreateBTree(ha=
rness=2EgetBufferCache(), SerdeUtils=2EserdesToTypeTraits(keySerdes),
                 SerdeUtils=2EserdesToComparatorFactories(keySer=
des, keySerdes=2Elength), BTreeLeafFrameType=2EREGULAR_NSM,
-                harness=2EgetFileRefe=
rence(), freePageManager, false);
+                harness=2EgetFileReference(), freePageManager=
, false, null, null);
     }
 
     @Override
diff --git a/hyracks-fullstack/hyracks/h=
yracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks=
/storage/am/btree/BTreeStatsTest=2Ejava b/hyracks-fullstack/hyracks/hyracks=
-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/stora=
ge/am/btree/BTreeStatsTest=2Ejava
index 7a24727=2E=2E97e5e4=
6 100644
--- a/hyracks-fullstack/hyracks/hyracks-tests/hyra=
cks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree=
/BTreeStatsTest=2Ejava
+++ b/hyracks-fullstack/hyracks/hyra=
cks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/st=
orage/am/btree/BTreeStatsTest=2Ejava
@@ -84,7 +84,8 @@
         int keyFieldCount =3D 1;
         IBin=
aryComparatorFactory[] cmpFactories =3D new IBinaryComparatorFactory[keyFie=
ldCount];
         cmpFactories[0] =3D IntegerBinaryCompara=
torFactory=2EINSTANCE;
=
-        BTreeTypeAwareTupleWriterFactory tupleWriterFactory =3D new BTreeT=
ypeAwareTupleWriterFactory(typeTraits, false);
+        BTreeTypeAwareTupleWriterFactory tupleWr=
iterFactory =3D
+    =
            new BTreeTypeAwareTupleWriterFactory(typeTraits, false, null, n=
ull);
         ITreeIndexFrameFactory leafFrameFactory =3D =
new BTreeNSMLeafFrameFactory(tupleWriterFactory);
         =
ITreeIndexFrameFactory interiorFrameFactory =3D new BTreeNSMInteriorFrameFa=
ctory(tupleWriterFactory);
         ITreeIndexMetadataFrame=
Factory metaFrameFactory =3D new LIFOMetaDataFrameFactory();
diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-b=
tree-test/src/test/java/org/apache/hyracks/storage/am/btree/BTreeUpdateSear=
chTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-=
btree-test/src/test/java/org/apache/hyracks/storage/am/btree/BTreeUpdateSea=
rchTest=2Ejava
index 5b86b4f=2E=2E720dd59 100644
=
--- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btre=
e-test/src/test/java/org/apache/hyracks/storage/am/btree/BTreeUpdateSearchT=
est=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-tests/hy=
racks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btr=
ee/BTreeUpdateSearchTest=2Ejava
@@ -74,7 +74,8 @@         ISerializerDeserializer[] recDescSers =3D
 =
                { IntegerSerializerDeserializer=2EINSTANCE, IntegerSerializ=
erDeserializer=2EINSTANCE };
 
-        BTreeTypeAwareTupleWriterFactory tupleWrit=
erFactory =3D new BTreeTypeAwareTupleWriterFactory(typeTraits, false);
+        BTreeTypeAwareTu=
pleWriterFactory tupleWriterFactory =3D
+                new BTreeTypeAwareTupleWriterFactory(ty=
peTraits, false, null, null);
         ITreeIndexFrameFacto=
ry leafFrameFactory =3D new BTreeNSMLeafFrameFactory(tupleWriterFactory);
         ITreeIndexFrameFactory interiorFrameFactory =3D new=
 BTreeNSMInteriorFrameFactory(tupleWriterFactory);
        =
 ITreeIndexMetadataFrameFactory metaFrameFactory =3D new LIFOMetaDataFrameF=
actory();
diff --git a/hyracks-fullstack/hyracks/hyracks-te=
sts/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/=
am/btree/FieldPrefixNSMTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-test=
s/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am=
/btree/FieldPrefixNSMTest=2Ejava
index fe05828=2E=2Ee9b9e7f=
 100644
--- a/hyracks-fullstack/hyracks/hyracks-tests/hyrac=
ks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/=
FieldPrefixNSMTest=2Ejava
+++ b/hyracks-fullstack/hyracks/h=
yracks-tests/hyracks-storage-am-btree-test/src/test/java/org/apache/hyracks=
/storage/am/btree/FieldPrefixNSMTest=2Ejava
@@ -131,7 +131,=
7 @@
         ICachedPage page =3D bufferCache=2Epin(Buffer=
edFileHandle=2EgetDiskPageId(btreeFileId, 0), true);
      =
   try {
 
-            BTreeTypeAwareTupleWriter tupleWriter =3D new BTreeTypeAwa=
reTupleWriter(typeTraits, false);
+            BTreeTypeAwareTupleWriter tupleWriter =3D new BTr=
eeTypeAwareTupleWriter(typeTraits, false, null, null);
    =
         BTreeFieldPrefixNSMLeafFrame frame =3D new BTreeFieldPrefixNSMLeaf=
Frame(tupleWriter);
             frame=2EsetPage(page);
             frame=2EinitBuffer((byte) 0);
dif=
f --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-=
test/src/test/java/org/apache/hyracks/storage/am/btree/multithread/BTreeMul=
tiThreadTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-stora=
ge-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/multithr=
ead/BTreeMultiThreadTest=2Ejava
index dcdbdc6=2E=2E6b94b03 =
100644
--- a/hyracks-fullstack/hyracks/hyracks-tests/hyrack=
s-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/m=
ultithread/BTreeMultiThreadTest=2Ejava
+++ b/hyracks-fullst=
ack/hyracks/hyracks-tests/hyracks-storage-am-btree-test/src/test/java/org/a=
pache/hyracks/storage/am/btree/multithread/BTreeMultiThreadTest=2Ejava
@@ -54,7 +54,7 @@
             int[] bloomFilte=
rKeyFields) throws HyracksDataException {
         return B=
TreeUtils=2EcreateBTree(harness=2EgetBufferCache(), typeTraits, cmpFactorie=
s,
                 BTreeLeafFrameType=2EREGULAR_NSM, harne=
ss=2EgetFileReference(),
+                harness=2EgetPageManagerFactory()=2EcreatePa=
geManager(harness=2EgetBufferCache()), false, null, null);
=
     }
 
     @Override
dif=
f --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-=
test/src/test/java/org/apache/hyracks/storage/am/btree/util/BTreeTestContex=
t=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree=
-test/src/test/java/org/apache/hyracks/storage/am/btree/util/BTreeTestConte=
xt=2Ejava
index 9d66b64=2E=2Eeaa1c28 100644
--- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-btree-tes=
t/src/test/java/org/apache/hyracks/storage/am/btree/util/BTreeTestContext=
=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-tests/hyrac=
ks-storage-am-btree-test/src/test/java/org/apache/hyracks/storage/am/btree/=
util/BTreeTestContext=2Ejava
@@ -56,7 +56,8 @@
             IPageManager pageManager) throws Exception {
         ITypeTraits[] typeTraits =3D SerdeUtils=2EserdesToTypeTraits(fi=
eldSerdes);
         IBinaryComparatorFactory[] cmpFactorie=
s =3D SerdeUtils=2EserdesToComparatorFactories(fieldSerdes, numKeyFields);<=
/span>
-        BTree btree =
=3D BTreeUtils=2EcreateBTree(bufferCache, typeTraits, cmpFactories, leafTyp=
e, file, pageManager, false);
+        BTree btree =3D BTreeUtils=2EcreateBTree(bufferCache, typ=
eTraits, cmpFactories, leafType, file, pageManager, false,
+                null, null);<=
br>         BTreeTestContext testCtx =3D new BTreeTestContext(fieldSe=
rdes, btree);
         return testCtx;
    =
 }
diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyr=
acks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/=
lsm/btree/LSMBTreeExamplesTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-t=
ests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/sto=
rage/am/lsm/btree/LSMBTreeExamplesTest=2Ejava
index 1d41280=
=2E=2E3313443 100644
--- a/hyracks-fullstack/hyracks/hyrack=
s-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/=
storage/am/lsm/btree/LSMBTreeExamplesTest=2Ejava
+++ b/hyra=
cks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/t=
est/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeExamplesTest=2Ejav=
a
@@ -64,7 +64,7 @@
                 harnes=
s=2EgetOperationTracker(), harness=2EgetIOScheduler(), harness=2EgetIOOpera=
tionCallbackFactory(),
                 harness=2EgetPageWr=
iteCallbackFactory(), true, filterTypeTraits, filterCmpFactories, btreeFiel=
ds,
                 filterFields, true, harness=2EgetMetad=
ataPageManagerFactory(), false, ITracer=2ENONE,
-                NoOpCompressorDecompressorFactory=
=2EINSTANCE, bloomFilterKeyFields !=3D null);
+                NoOpCompressorDecompressorFactory=
=2EINSTANCE, bloomFilterKeyFields !=3D null, null, null);
 =
    }
 
     @Before
diff -=
-git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree=
-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeModific=
ationOperationCallbackTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests=
/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage=
/am/lsm/btree/LSMBTreeModificationOperationCallbackTest=2Ejava
index 8d8ec23=2E=2E2476761 100644
--- a/hyracks-fullsta=
ck/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/or=
g/apache/hyracks/storage/am/lsm/btree/LSMBTreeModificationOperationCallback=
Test=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-tests/h=
yracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/a=
m/lsm/btree/LSMBTreeModificationOperationCallbackTest=2Ejava
@@ -54,7 +54,7 @@
                 NoOpOperationTrackerFa=
ctory=2EINSTANCE=2EgetOperationTracker(null, null), harness=2EgetIOSchedule=
r(),
                 harness=2EgetIOOperationCallbackFacto=
ry(), harness=2EgetPageWriteCallbackFactory(), true, null, null, null,
                 null, true, harness=2EgetMetadataPageManagerFa=
ctory(), false, ITracer=2ENONE,
-                NoOpCompressorDecompressorFactory=2EINSTANCE, HAS=
_BLOOM_FILTER);
+    =
            NoOpCompressorDecompressorFactory=2EINSTANCE, HAS_BLOOM_FILTER,=
 null, null);
     }
 
    =
 @Override
diff --git a/hyracks-fullstack/hyracks/hyracks-t=
ests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/sto=
rage/am/lsm/btree/LSMBTreeSearchOperationCallbackTest=2Ejava b/hyracks-full=
stack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java=
/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeSearchOperationCallbackTes=
t=2Ejava
index ab068ae=2E=2E6dec9f4 100644
=
--- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-=
test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeSearchOp=
erationCallbackTest=2Ejava
+++ b/hyracks-fullstack/hyracks/=
hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hy=
racks/storage/am/lsm/btree/LSMBTreeSearchOperationCallbackTest=2Ejava
@@ -63,7 +63,7 @@
                 NoOpOperation=
TrackerFactory=2EINSTANCE=2EgetOperationTracker(null, null), harness=2EgetI=
OScheduler(),
                 harness=2EgetIOOperationCall=
backFactory(), harness=2EgetPageWriteCallbackFactory(), true, null, null, n=
ull,
                 null, true, harness=2EgetMetadataPage=
ManagerFactory(), false, ITracer=2ENONE,
-                NoOpCompressorDecompressorFactory=2EINST=
ANCE, HAS_BLOOM_FILTER);
+                NoOpCompressorDecompressorFactory=2EINSTANCE, HAS_BLOO=
M_FILTER, null, null);
     }
 
<=
span>     @Override
diff --git a/hyracks-fullstack/hyracks/=
hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hy=
racks/storage/am/lsm/btree/LSMBTreeUpdateInPlaceTest=2Ejava b/hyracks-fulls=
tack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/=
org/apache/hyracks/storage/am/lsm/btree/LSMBTreeUpdateInPlaceTest=2Ejava
index 2954854=2E=2E0a4b37b 100644
--- a/hyrac=
ks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/te=
st/java/org/apache/hyracks/storage/am/lsm/btree/LSMBTreeUpdateInPlaceTest=
=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-tests/hyrac=
ks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/ls=
m/btree/LSMBTreeUpdateInPlaceTest=2Ejava
@@ -76,7 +76,7 @@<=
/span>
                 NoOpOperationTrackerFactory=2EINSTANCE=2Eg=
etOperationTracker(null, null), harness=2EgetIOScheduler(),
                 harness=2EgetIOOperationCallbackFactory(), harness=2EgetP=
ageWriteCallbackFactory(), true, null, null, null,
        =
         null, true, harness=2EgetMetadataPageManagerFactory(), true, ITrac=
er=2ENONE,
-           =
     NoOpCompressorDecompressorFactory=2EINSTANCE, HAS_BLOOM_FILTER);
+                NoOpCompr=
essorDecompressorFactory=2EINSTANCE, HAS_BLOOM_FILTER, null, null);<=
br>     }
 
     @Override
=
diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-=
am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/imp=
l/TestLsmBtreeLocalResource=2Ejava b/hyracks-fullstack/hyracks/hyracks-test=
s/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storag=
e/am/lsm/btree/impl/TestLsmBtreeLocalResource=2Ejava
index =
3450531=2E=2E8e2d6dd 100644
--- a/hyracks-fullstack/hyracks=
/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/h=
yracks/storage/am/lsm/btree/impl/TestLsmBtreeLocalResource=2Ejava+++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm=
-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/Test=
LsmBtreeLocalResource=2Ejava
@@ -61,7 +61,7 @@
                 storageManager, mergePolicyFactory, mergePolicyPropert=
ies, filterTypeTraits, filterCmpFactories,
                =
 btreeFields, filterFields, opTrackerProvider, ioOpCallbackFactory, pageWri=
teCallbackFactory,
                 metadataPageManagerFact=
ory, vbcProvider, ioSchedulerProvider, durable,
-                NoOpCompressorDecompressorFactory=
=2EINSTANCE, hasBloomFilter);
+                NoOpCompressorDecompressorFactory=2EINSTANCE, has=
BloomFilter, null, null);
     }
      protected TestLsmBtreeLocalResource(IPersistedResourceRegistry=
 registry, JsonNode json, int[] bloomFilterKeyFields,
diff =
--git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btre=
e-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBt=
reeLocalResourceFactory=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hy=
racks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am=
/lsm/btree/impl/TestLsmBtreeLocalResourceFactory=2Ejava
ind=
ex 229b027=2E=2E7d9b24a 100644
--- a/hyracks-fullstack/hyra=
cks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apach=
e/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeLocalResourceFactory=2Ejava=

+++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-stor=
age-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree=
/impl/TestLsmBtreeLocalResourceFactory=2Ejava
@@ -51,7 +51,=
7 @@
                 opTrackerFactory, ioOpCallbackFactory=
, pageWriteCallbackFactory, metadataPageManagerFactory,
   =
              vbcProvider, ioSchedulerProvider, mergePolicyFactory, mergePo=
licyProperties, durable,
                 bloomFilterKeyFie=
lds, bloomFilterFalsePositiveRate, isPrimary, btreeFields,
-                NoOpCompressorDecompre=
ssorFactory=2EINSTANCE, hasBloomFilter);
+                NoOpCompressorDecompressorFactory=2EIN=
STANCE, hasBloomFilter, null, null);
     }
 
     @Override
diff --git a/hyracks-full=
stack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java=
/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeUtil=2Ejava b/hyr=
acks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/=
test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeUtil=2Ej=
ava
index eebfb2b=2E=2Edd42838 100644
--- a=
/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/=
src/test/java/org/apache/hyracks/storage/am/lsm/btree/impl/TestLsmBtreeUtil=
=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-tests/hyrac=
ks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/ls=
m/btree/impl/TestLsmBtreeUtil=2Ejava
@@ -68,11 +68,11 @@
             int[] btreeFields, int[] filterFields, boolean d=
urable, IMetadataPageManagerFactory freePageManagerFactory,
             boolean updateAware, ITracer tracer) throws HyracksDataExcept=
ion {
         LSMBTreeTupleWriterFactory insertTupleWriter=
Factory =3D
-          =
      new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories=2Elength, fal=
se, updateAware);
+  =
              new LSMBTreeTupleWriterFactory(typeTraits, cmpFactories=2Elen=
gth, false, updateAware, null, null);
         LSMBTreeTupl=
eWriterFactory deleteTupleWriterFactory =3D
-                new LSMBTreeTupleWriterFactory(typeTr=
aits, cmpFactories=2Elength, true, updateAware);
+                new LSMBTreeTupleWriterFactory=
(typeTraits, cmpFactories=2Elength, true, updateAware, null, null);<=
br>         LSMBTreeCopyTupleWriterFactory copyTupleWriterFactory =3D=

-                new L=
SMBTreeCopyTupleWriterFactory(typeTraits, cmpFactories=2Elength, updateAwar=
e);
+                =
new LSMBTreeCopyTupleWriterFactory(typeTraits, cmpFactories=2Elength, updat=
eAware, null, null);
         ITreeIndexFrameFactory insert=
LeafFrameFactory =3D new BTreeNSMLeafFrameFactory(insertTupleWriterFactory)=
;
         ITreeIndexFrameFactory copyTupleLeafFrameFactory=
 =3D new BTreeNSMLeafFrameFactory(copyTupleWriterFactory);
=
         ITreeIndexFrameFactory deleteLeafFrameFactory =3D new BTreeNSMLeaf=
FrameFactory(deleteTupleWriterFactory);
@@ -88,7 +88,8 @@
         LSMComponentFilterFrameFactory filterFrameFactory =
=3D null;
         LSMComponentFilterManager filterManager =
=3D null;
         if (filterCmpFactories !=3D null) {
-            TypeAwareTuple=
WriterFactory filterTupleWriterFactory =3D new TypeAwareTupleWriterFactory(=
filterTypeTraits);
+ =
           TypeAwareTupleWriterFactory filterTupleWriterFactory =3D<=
br>+                    new Typ=
eAwareTupleWriterFactory(filterTypeTraits, null, null);
   =
          filterHelper =3D new ComponentFilterHelper(filterTupleWriterFacto=
ry, filterCmpFactories);
             filterFrameFactory =
=3D new LSMComponentFilterFrameFactory(filterTupleWriterFactory);             filterManager =3D new LSMComponentFilterManager(filterF=
rameFactory);
diff --git a/hyracks-fullstack/hyracks/hyrack=
s-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/=
storage/am/lsm/btree/multithread/LSMBTreeMultiThreadTest=2Ejava b/hyracks-f=
ullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/j=
ava/org/apache/hyracks/storage/am/lsm/btree/multithread/LSMBTreeMultiThread=
Test=2Ejava
index fb1244f=2E=2E66af4b2 100644
--- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btr=
ee-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/multithread/L=
SMBTreeMultiThreadTest=2Ejava
+++ b/hyracks-fullstack/hyrac=
ks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache=
/hyracks/storage/am/lsm/btree/multithread/LSMBTreeMultiThreadTest=2Ejava
@@ -60,7 +60,7 @@
                 harness=2E=
getOperationTracker(), harness=2EgetIOScheduler(), harness=2EgetIOOperation=
CallbackFactory(),
                 harness=2EgetPageWriteC=
allbackFactory(), true, null, null, null, null, true,
     =
            harness=2EgetMetadataPageManagerFactory(), false, ITracer=2ENON=
E,
-                NoO=
pCompressorDecompressorFactory=2EINSTANCE, bloomFilterKeyFields !=3D null);=

+                NoO=
pCompressorDecompressorFactory=2EINSTANCE, bloomFilterKeyFields !=3D null, =
null, null);
     }
 
     =
@Override
diff --git a/hyracks-fullstack/hyracks/hyracks-te=
sts/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/stor=
age/am/lsm/btree/perf/BTreeRunner=2Ejava b/hyracks-fullstack/hyracks/hyrack=
s-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/=
storage/am/lsm/btree/perf/BTreeRunner=2Ejava
index 60a054e=
=2E=2Ebaa5508 100644
--- a/hyracks-fullstack/hyracks/hyrack=
s-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/=
storage/am/lsm/btree/perf/BTreeRunner=2Ejava
+++ b/hyracks-=
fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/=
java/org/apache/hyracks/storage/am/lsm/btree/perf/BTreeRunner=2Ejava=

@@ -49,6 +49,6 @@
         ITreeIndexMetadataFram=
eFactory metaFrameFactory =3D new LIFOMetaDataFrameFactory();
         LinkedMetaDataPageManager freePageManager =3D new LinkedMetaDat=
aPageManager(bufferCache, metaFrameFactory);
         btree=
 =3D BTreeUtils=2EcreateBTree(bufferCache, typeTraits, cmpFactories, BTreeL=
eafFrameType=2EREGULAR_NSM, file,
-                freePageManager, false);
+                freePageManager, false, n=
ull, null);
     }
 }
diff =
--git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btre=
e-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/Concurren=
tSkipListRunner=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-st=
orage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btr=
ee/perf/ConcurrentSkipListRunner=2Ejava
index c476a8c=2E=2E=
c57badc 100644
--- a/hyracks-fullstack/hyracks/hyracks-test=
s/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storag=
e/am/lsm/btree/perf/ConcurrentSkipListRunner=2Ejava
+++ b/h=
yracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/sr=
c/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/ConcurrentSkipList=
Runner=2Ejava
@@ -116,7 +116,7 @@
         =
    this=2EdataGen =3D dataGen;
             this=2EnumBatc=
hes =3D numBatches;
             this=2EskipList =3D skipLi=
st;
-            tupleW=
riterFactory =3D new TypeAwareTupleWriterFactory(typeTraits);
+            tupleWriterFactory =
=3D new TypeAwareTupleWriterFactory(typeTraits, null, null);
             tupleWriter =3D tupleWriterFactory=2EcreateTupleWriter();
             int numTuples =3D numBatches * batchSize;=

             tuples =3D new TypeAwareTupleReference[numTuples];
diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks=
-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/=
btree/perf/InMemoryBTreeRunner=2Ejava b/hyracks-fullstack/hyracks/hyracks-t=
ests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/sto=
rage/am/lsm/btree/perf/InMemoryBTreeRunner=2Ejava
index 330=
7093=2E=2E1eb4646 100644
--- a/hyracks-fullstack/hyracks/hy=
racks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyra=
cks/storage/am/lsm/btree/perf/InMemoryBTreeRunner=2Ejava
++=
+ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-te=
st/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/InMemoryBTree=
Runner=2Ejava
@@ -68,7 +68,8 @@
     protec=
ted void init(int pageSize, int numPages, ITypeTraits[] typeTraits, IBinary=
ComparatorFactory[] cmpFactories)
             throws Hyrac=
ksDataException {
         bufferCache =3D new VirtualBuffe=
rCache(new HeapBufferAllocator(), pageSize, numPages);
-        BTreeTypeAwareTupleWriterFactory t=
upleWriterFactory =3D new BTreeTypeAwareTupleWriterFactory(typeTraits, fals=
e);
+        BTreeTyp=
eAwareTupleWriterFactory tupleWriterFactory =3D
+                new BTreeTypeAwareTupleWriterFa=
ctory(typeTraits, false, null, null);
         ITreeIndexFr=
ameFactory leafFrameFactory =3D new BTreeNSMLeafFrameFactory(tupleWriterFac=
tory);
         ITreeIndexFrameFactory interiorFrameFactory=
 =3D new BTreeNSMInteriorFrameFactory(tupleWriterFactory);
=
         IPageManager freePageManager =3D new VirtualFreePageManager(buffer=
Cache);
diff --git a/hyracks-fullstack/hyracks/hyracks-test=
s/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storag=
e/am/lsm/btree/perf/InMemorySortRunner=2Ejava b/hyracks-fullstack/hyracks/h=
yracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyr=
acks/storage/am/lsm/btree/perf/InMemorySortRunner=2Ejava
in=
dex 4501485=2E=2E1455fb3 100644
--- a/hyracks-fullstack/hyr=
acks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apac=
he/hyracks/storage/am/lsm/btree/perf/InMemorySortRunner=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-bt=
ree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/InMemor=
ySortRunner=2Ejava
@@ -71,7 +71,7 @@
      =
   this=2EbatchSize =3D batchSize;
         this=2EtypeTrai=
ts =3D typeTraits;
         tupleCmp =3D new TupleComparato=
r(cmp);
-        tupleW=
riterFactory =3D new TypeAwareTupleWriterFactory(typeTraits);
+        tupleWriterFactory =3D ne=
w TypeAwareTupleWriterFactory(typeTraits, null, null);
    =
     tupleWriter =3D tupleWriterFactory=2EcreateTupleWriter();
         int numTuples =3D numBatches * batchSize;
    =
     tuples =3D new ArrayList<TypeAwareTupleReference>();
<=
span>@@ -131,7 +131,7 @@
             this=2EdataGen =3D da=
taGen;
             this=2EnumBatches =3D numBatches;
             this=2EskipList =3D skipList;
-            tupleWriterFactory =3D new Type=
AwareTupleWriterFactory(typeTraits);
+            tupleWriterFactory =3D new TypeAwareTupleWrite=
rFactory(typeTraits, null, null);
             tupleWriter =
=3D tupleWriterFactory=2EcreateTupleWriter();
             =
int numTuples =3D numBatches * batchSize;
             tupl=
es =3D new TypeAwareTupleReference[numTuples];
diff --git a=
/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/=
src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/LSMTreeRunner=2E=
java b/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree=
-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/LSMTreeRun=
ner=2Ejava
index f487bf1=2E=2E1b46fd7 100644
--- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btre=
e-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/perf/LSMTreeRu=
nner=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-tests/h=
yracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/a=
m/lsm/btree/perf/LSMTreeRunner=2Ejava
@@ -128,7 +128,7 @@
                 new ThreadCountingTracker(), ioScheduler, N=
oOpIOOperationCallbackFactory=2EINSTANCE,
                 =
NoOpPageWriteCallbackFactory=2EINSTANCE, true, null, null, null, null, true=
,
                 TestStorageManagerComponentHolder=2EgetM=
etadataPageManagerFactory(), false, ITracer=2ENONE,
-                NoOpCompressorDecompressorF=
actory=2EINSTANCE, bloomFilterKeyFields !=3D null);
+                NoOpCompressorDecompresso=
rFactory=2EINSTANCE, bloomFilterKeyFields !=3D null, null, null);     }
 
     @Override
diff --git a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am=
-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/lsm/btree/tuple=
s/LSMBTreeTuplesTest=2Ejava b/hyracks-fullstack/hyracks/hyracks-tests/hyrac=
ks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/am/ls=
m/btree/tuples/LSMBTreeTuplesTest=2Ejava
index b81167b=2E=
=2E0a34c45 100644
--- a/hyracks-fullstack/hyracks/hyracks-t=
ests/hyracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/sto=
rage/am/lsm/btree/tuples/LSMBTreeTuplesTest=2Ejava
+++ b/hy=
racks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src=
/test/java/org/apache/hyracks/storage/am/lsm/btree/tuples/LSMBTreeTuplesTes=
t=2Ejava
@@ -68,7 +68,7 @@
             for=
 (int numFields =3D numKeyFields; numFields <=3D maxFieldSerdes=2Elength=
; numFields++) {
                 // Create and write tuple=
 to bytes using an LSMBTreeTupleWriter=2E
                 =
LSMBTreeTupleWriter maxMatterTupleWriter =3D
-                        new LSMBTreeTupleWriter(maxT=
ypeTraits, numKeyFields, false, false);
+                        new LSMBTreeTupleWriter(maxType=
Traits, numKeyFields, false, false, null, null);
          =
       ITupleReference maxTuple =3D TupleUtils=2EcreateTuple(maxFieldSerdes=
, (Object[]) maxFields);
                 ByteBuffer maxMat=
terBuf =3D writeTuple(maxTuple, maxMatterTupleWriter);
    =
             // Tuple reference should work for both matter and antimatter =
tuples (doesn't matter which factory creates it)=2E
@@ =
-84,10 +84,12 @@
                 }
       =
          // Create and write tuple to bytes using an LSMBTreeTupleWriter=
=2E
                 ITupleReference tuple =3D TupleUtils=
=2EcreateTuple(fieldSerdes, (Object[]) fields);
-                LSMBTreeTupleWriter matterTupleWr=
iter =3D new LSMBTreeTupleWriter(typeTraits, numKeyFields, false, false);
+                LSMBT=
reeTupleWriter matterTupleWriter =3D
+                        new LSMBTreeTupleWriter(typeTraits=
, numKeyFields, false, false, null, null);
                =
 LSMBTreeTupleWriter antimatterTupleWriter =3D
-                        new LSMBTreeTupleWriter(ty=
peTraits, numKeyFields, true, false);
-                LSMBTreeCopyTupleWriter copyTupleWriter =3D=
 new LSMBTreeCopyTupleWriter(typeTraits, numKeyFields, false);
+                        new LSMB=
TreeTupleWriter(typeTraits, numKeyFields, true, false, null, null);<=
br>+                LSMBTreeCop=
yTupleWriter copyTupleWriter =3D
+                        new LSMBTreeCopyTupleWriter(typeTraits=
, numKeyFields, false, null, null);
                 ByteBu=
ffer matterBuf =3D writeTuple(tuple, matterTupleWriter);
  =
               ByteBuffer antimatterBuf =3D writeTuple(tuple, antimatterTup=
leWriter);
 
diff --git a/hyracks-fullstack=
/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/test/java/org/=
apache/hyracks/storage/am/lsm/btree/util/LSMBTreeTestContext=2Ejava b/hyrac=
ks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test/src/te=
st/java/org/apache/hyracks/storage/am/lsm/btree/util/LSMBTreeTestContext=2E=
java
index 6da04e7=2E=2E928f60c 100644
--- =
a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-btree-test=
/src/test/java/org/apache/hyracks/storage/am/lsm/btree/util/LSMBTreeTestCon=
text=2Ejava
+++ b/hyracks-fullstack/hyracks/hyracks-tests/h=
yracks-storage-am-lsm-btree-test/src/test/java/org/apache/hyracks/storage/a=
m/lsm/btree/util/LSMBTreeTestContext=2Ejava
@@ -103,7 +103,=
7 @@
                     cmpFactories, bloomFilterKeyField=
s, bloomFilterFalsePositiveRate, mergePolicy, opTracker,
  =
                   ioScheduler, ioOpCallbackFactory, pageWriteCallbackFacto=
ry, needKeyDupCheck, filterTypeTraits,
                    =
 filterCmp, btreefields, filterfields, true, metadataPageManagerFactory, up=
dateAware, ITracer=2ENONE,
-                    NoOpCompressorDecompressorFactory=2EINSTANCE, true=
);
+                 =
   NoOpCompressorDecompressorFactory=2EINSTANCE, true, null, null);<=
br>         } else {
             lsmTree =3D LSMBTre=
eUtil=2EcreateLSMTree(ioManager, virtualBufferCaches, file, diskBufferCache=
, typeTraits,
                     cmpFactories, bloomFilte=
rKeyFields, bloomFilterFalsePositiveRate, mergePolicy, opTracker,@@ -111,7 +111,7 @@
                     true, metad=
ataPageManagerFactory,
                     updateAware, ne=
w Tracer(LSMBTreeTestContext=2Eclass=2EgetSimpleName(),
   =
                          ITraceCategoryRegistry=2ECATEGORIES_ALL, new Trac=
eCategoryRegistry()),
-=
                    NoOpCompressorDecompressorFactory=2EINSTANCE, true);
+                    No=
OpCompressorDecompressorFactory=2EINSTANCE, true, null, null);
         }
         LSMBTreeTestContext testCtx =3D new=
 LSMBTreeTestContext(fieldSerdes, lsmTree, filtered);
     =
    return testCtx;
diff --git a/hyracks-fullstack/hyracks/=
hyracks-tests/hyracks-storage-am-lsm-common-test/src/test/java/org/apache/h=
yracks/storage/am/lsm/common/test/LSMComponentFilterReferenceTest=2Ejava b/=
hyracks-fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-common-test/=
src/test/java/org/apache/hyracks/storage/am/lsm/common/test/LSMComponentFil=
terReferenceTest=2Ejava
index 143204f=2E=2E612ca4b 100644
--- a/hyracks-fullstack/hyracks/hyracks-tests/hyracks-storag=
e-am-lsm-common-test/src/test/java/org/apache/hyracks/storage/am/lsm/common=
/test/LSMComponentFilterReferenceTest=2Ejava
+++ b/hyracks-=
fullstack/hyracks/hyracks-tests/hyracks-storage-am-lsm-common-test/src/test=
/java/org/apache/hyracks/storage/am/lsm/common/test/LSMComponentFilterRefer=
enceTest=2Ejava
@@ -34,7 +34,7 @@
     @Tes=
t
     public void test() throws HyracksDataException {
         LSMComponentFilterReference filter =3D new LSMCompone=
ntFilterReference(
-   =
             new TypeAwareTupleWriter(new ITypeTraits[] { IntegerPointable=
=2ETYPE_TRAITS }));
+=
                new TypeAwareTupleWriter(new ITypeTraits[] { IntegerPointab=
le=2ETYPE_TRAITS }, null, null));
         Assert=2EassertE=
quals(filter=2EgetLength(), 0);
         Assert=2EassertFal=
se(filter=2EisMaxTupleSet() || filter=2EisMinTupleSet());
 =
        filter=2EwriteMaxTuple(TupleUtils=2EcreateIntegerTuple(false, Integ=
er=2EMAX_VALUE));
@@ -46,7 +46,7 @@
       =
  Assert=2EassertTrue(filter=2EgetLength() =3D=3D 20);
    =
     byte[] serFilter =3D filter=2EgetByteArray();
        =
 LSMComponentFilterReference deserFilter =3D new LSMComponentFilterReferenc=
e(
-                new=
 TypeAwareTupleWriter((new ITypeTraits[] { IntegerPointable=2ETYPE_TRAITS }=
)));
+               =
 new TypeAwareTupleWriter(new ITypeTraits[] { IntegerPointable=2ETYPE_TRAIT=
S }, null, null));
         deserFilter=2Eset(serFilter, 0,=
 20);
         Assert=2EassertTrue(deserFilter=2EisMaxTuple=
Set() && deserFilter=2EisMinTupleSet());
         A=
ssert=2EassertEquals(

To view, visit chan= ge 11944=2E To unsubscribe, or for help writing mail filters, visit settings=2E<= /p>

Gerrit-Proje= ct: asterixdb
Gerrit-Branch: master
Gerrit-Change-Id: I5fc9c40af68c7b6aaa54ecb= 440c8fe1e90046231
Gerrit-Change-Number:= 11944
Gerrit-PatchSet: 1
Gerrit-Owner: Ali Alsuliman <ali=2Eal=2Esolaiman@= gmail=2Ecom>
Gerrit-MessageType: new= change
--nYDT6NJUqDk=--