Mailing-List: contact user-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@giraph.apache.org
MIME-Version: 1.0
Date: Wed, 9 Sep 2015 10:44:08 +0400
Message-ID: 
 <CAPoux5ppFaTLbMDyKW8QiKVGtQW5j8jqA6SkZ8pAa7F5XuPw7A@mail.gmail.com>
Subject: How to specify filtering in hbase "query" during input superstep
From: Vitaly Tsvetkoff <vi.v.tsvetkoff@gmail.com>
To: user@giraph.apache.org
Content-Type: multipart/alternative; boundary=001a114daa86389fc7051f4acf25

--001a114daa86389fc7051f4acf25
Content-Type: text/plain; charset=UTF-8

Hello!
I use giraph-hbase and write custom CustomHBaseTableInputFormat.
I want to apply some filters (like o.a.h.hbase.filter.RowFilter,
FamilyFilter etc) to get clear data after the "query". For example, I want
to get only vertex with specifying rowkey id. Is it possible?
I try to do it like this:
public class CustomHBaseTableInputFormat extends HBaseVertexInputFormat {
    @Override
    public VertexReader<Text, FloatWritable, FloatWritable>
createVertexReader(InputSplit split, TaskAttemptContext context) throws
IOException {
        return new CustomHBaseReader (split, context);
    }
    // other methods to impliment

    public static class CustomHBaseReader extends HBaseVertexReader {
        public HBaseTableReader(InputSplit split, TaskAttemptContext
context) throws IOException {
            super(split, context);
        }

        @Override
        public void initialize(InputSplit inputSplit, TaskAttemptContext
context) throws IOException, InterruptedException {
            super.initialize(inputSplit, context);
            String startIdsRegexp = getStartVertexRegexp();
            System.err.println("set row filter with regexp=" +
startIdsRegexp);
            Filter rowFilter = new RowFilter(CompareFilter.CompareOp.EQUAL,
new RegexStringComparator(startIdsRegexp));
            Scan scan =
HBaseVertexInputFormat.BASE_FORMAT.getScan().setFilter(rowFilter);
            System.err.println("scan=" + scan);
          //super.initialize(inputSplit, context);
        }
    }
    // other methods to impliment
}

Log says what scan contains my filter but all of dataset is read (without
applying any filters).

I know about vertexInputFilterClass property, but it filters after query
with a lot of unusable data.
What is a way to set filters correctly? Can I use o.a.h.hbase.filter
package for this? If yes, what do I wrong?

--001a114daa86389fc7051f4acf25
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hello!<div>I use giraph-hbase and write custom CustomHBase=
TableInputFormat.</div><div>I want to apply some filters (like o.a.h.hbase.=
filter.RowFilter, FamilyFilter etc) to get clear data after the &quot;query=
&quot;. For example, I want to get only vertex with specifying rowkey id. I=
s it possible?</div><div>I try to do it like this:</div><div><div>public cl=
ass=C2=A0CustomHBaseTableInputFormat=C2=A0extends HBaseVertexInputFormat {<=
br></div><div>=C2=A0 =C2=A0 @Override</div><div>=C2=A0 =C2=A0 public Vertex=
Reader&lt;Text, FloatWritable, FloatWritable&gt; createVertexReader(InputSp=
lit split, TaskAttemptContext context) throws IOException {</div><div>=C2=
=A0 =C2=A0 =C2=A0 =C2=A0 return new=C2=A0CustomHBaseReader=C2=A0(split, con=
text);</div><div>=C2=A0 =C2=A0 }</div></div><div>=C2=A0 =C2=A0 // other met=
hods to impliment</div><div><br></div><div><div>=C2=A0 =C2=A0 public static=
 class=C2=A0CustomHBaseReader=C2=A0extends HBaseVertexReader {</div><div>=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 public HBaseTableReader(InputSplit split, TaskA=
ttemptContext context) throws IOException {<br></div><div>=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 super(split, context);</div><div>=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 }</div><div><br></div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 @Override=
</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 public void initialize(InputSplit in=
putSplit, TaskAttemptContext context) throws IOException, InterruptedExcept=
ion {</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 super.initialize(=
inputSplit, context);<br></div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 String startIdsRegexp =3D getStartVertexRegexp();</div><div>=C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 System.err.println(&quot;set row filter wit=
h regexp=3D&quot; + startIdsRegexp);</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 Filter rowFilter =3D new RowFilter(CompareFilter.CompareOp.EQ=
UAL, new RegexStringComparator(startIdsRegexp));</div><div>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Scan scan =3D HBaseVertexInputFormat.BASE_FORMA=
T.getScan().setFilter(rowFilter);</div><div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 System.err.println(&quot;scan=3D&quot; + scan);</div><div>=C2=A0=
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 //super.initialize(inputSplit, context);</div>=
<div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 }<br></div></div><div>=C2=A0 =C2=A0 }</div=
><div>=C2=A0 =C2=A0 // other methods to impliment</div><div>}</div><div><br=
></div><div>Log says what scan contains my filter but all of dataset is rea=
d (without applying any filters).</div><div><br></div><div>I know about ver=
texInputFilterClass property, but it filters after query with a lot of unus=
able data.<br></div><div>What is a way to set filters correctly? Can I use =
o.a.h.hbase.filter package for this? If yes, what do I wrong?</div></div>

--001a114daa86389fc7051f4acf25--