hivemall-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From myui <...@git.apache.org>
Subject [GitHub] incubator-hivemall pull request #97: [HIVEMALL-130] Support user-defined dic...
Date Fri, 14 Jul 2017 03:48:28 GMT
Github user myui commented on a diff in the pull request:

    https://github.com/apache/incubator-hivemall/pull/97#discussion_r127379441
  
    --- Diff: core/src/main/java/hivemall/utils/io/LimitedInputStream.java ---
    @@ -0,0 +1,89 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + *   http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing,
    + * software distributed under the License is distributed on an
    + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    + * KIND, either express or implied.  See the License for the
    + * specific language governing permissions and limitations
    + * under the License.
    + */
    +package hivemall.utils.io;
    +
    +import javax.annotation.Nonnegative;
    +import java.io.FilterInputStream;
    +import java.io.IOException;
    +import java.io.InputStream;
    +
    +/**
    + * Input stream which is limited to a certain length
    + * Implementation is based on BoundedInputStream in Apache Commons IO and LimitedInputStream
in Apache Commons FileUpload
    + *
    + * @link https://commons.apache.org/proper/commons-io/javadocs/api-2.5/org/apache/commons/io/input/BoundedInputStream.html
    + * @link https://commons.apache.org/proper/commons-fileupload/apidocs/org/apache/commons/fileupload/util/LimitedInputStream.html
    + */
    +public class LimitedInputStream extends FilterInputStream {
    +
    +    protected final long max;
    +    protected long pos = 0L;
    +
    +    public LimitedInputStream(final InputStream in, @Nonnegative final long maxSize)
{
    +        super(in);
    +        this.max = maxSize;
    +    }
    +
    +    protected void raiseError() throws IOException {
    +        throw new IOException("Exceeded maximum size of input stream: limit = " + max
+ " bytes, but pos = " + pos);
    +    }
    +
    +    private void proceed(@Nonnegative long bytes) throws IOException {
    +        this.pos += bytes;
    +        if (pos > max) {
    +            raiseError();
    +        }
    +    }
    +
    +    @Override
    +    public int read() throws IOException {
    +        int res = super.read();
    +        if (res != -1) {
    +            proceed(1L);
    +        }
    +        return res;
    +    }
    +
    +    @Override
    +    public int read(byte[] b, int off, int len) throws IOException {
    +        int res = super.read(b, off, len);
    +        if (res > 0) {
    +            proceed(res);
    +        }
    +        return res;
    +    }
    +
    +    @Override
    +    public long skip(long n) throws IOException {
    +        long res = super.skip(n);
    +        if (res > 0) {
    +            proceed(res);
    +        }
    +        return res;
    +    }
    +
    +    @Override
    +    public int available() throws IOException {
    +        if (pos >= max) {
    --- End diff --
    
    `available()` itself is not need to override.
    `raiseError()` should be called when the max threshold is exceeded.
    
    Some clients tried to sleep while input stream is filled as follows:
    
    ```java
    while(true) {
       if(is.available() == 0) {// could return 0 where pos = max      
          if(numtry++ >= 2)  throw new IllegalStateException(xxxx);
          Time.sleep(1000); // sleep until filled
          continue;
       }
       // never reached here in the current code
       int bytes = in.read(buf); // expected to call raiseError where pos > max
       ...   
    }
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message