hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Panda Song (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-7209) getting error "Can not create a Path from an empty string" when running mapreduce job
Date Fri, 31 May 2019 10:44:00 GMT
Panda Song created MAPREDUCE-7209:
-------------------------------------

             Summary: getting error "Can not create a Path from an empty string" when running
mapreduce job
                 Key: MAPREDUCE-7209
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7209
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Panda Song


hdfs:3.1.1

Mapreduce came into the error "Can not create a Path from an empty string" ,It just happened
with the specific input file

The input file  is too large to upload,can somebody help me ? 
{code:java}
#!/usr/bin/python3

from mrjob.job import MRJob
from mrjob.step import MRStep
import mrjob.protocol as protocol
import sys
import json


class QM(MRJob):

    FILES = ['utils.py', 'models.py', 'stats.py']
    LIBJARS = ['nicknack-1.0.1.jar']

    JOBCONF = {
        'mapred.reduce.tasks': 4,
        'mapreduce.output.fileoutputformat.compress': "true",
        'mapreduce.output.fileoutputformat.compress.codec': 'org.apache.hadoop.io.compress.GzipCodec'
    }

    INPUT_PROTOCOL = protocol.TextValueProtocol
    INTERNAL_PROTOCOL = protocol.JSONProtocol
    OUTPUT_PROTOCOL = protocol.RawProtocol

    HADOOP_OUTPUT_FORMAT = 'nicknack.MultipleValueOutputFormat'


    def mapper(self, _, value):
        from models import OpenApiOpen, ThirdPart_UserOpenApp, filter_baidu_pan_android, filter_vivo
        from stats import Stats
        from utils import get_mr_input_file_name, DATA
        try:
            input_file_name = get_mr_input_file_name()
            if 'ThirdPart_UserOpenApp' in input_file_name:
                r = ThirdPart_UserOpenApp.parse_qm(value)
            elif 'OpenApiOpen' in input_file_name:
                r = OpenApiOpen.parse_qm(value)
            else:
                r = None

            if r is None:
                return Stats.on_error(self)

            ok, client_id, did, imei, v = r
                yield DATA, v
            else:
                return Stats.on_error(self)
        except Exception as e:
            return Stats.on_exception(self)

if __name__ == '__main__':
    QM.run()
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message