pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (PIG-2956) Invalid cache specification for some streaming statement
Date Wed, 15 May 2013 20:45:17 GMT

     [ https://issues.apache.org/jira/browse/PIG-2956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Dai updated PIG-2956:
----------------------------

    Attachment: PIG-2956-2.patch

Dive deeper into the issue. Here is what I find:
1. The problem with "new URI(src.toString())": It is not compatible with Windows path style,
colon and space will result an exception

2. The problem with "src.toUri()": It encodes "#" character. Hadoop will use URI.getFragment()
to get the symlink from a uri, getFragment only search for "#" character not the encoded one

I attach a new patch, which takes out the symlink part, use "src.toUri()" to encode the rest,
then append the symlink. Tested both Windows and Linux works.
                
> Invalid cache specification for some streaming statement
> --------------------------------------------------------
>
>                 Key: PIG-2956
>                 URL: https://issues.apache.org/jira/browse/PIG-2956
>             Project: Pig
>          Issue Type: Sub-task
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 0.12
>
>         Attachments: PIG-2956-1_0.10.patch, PIG-2956-1.patch, PIG-2956-2.patch
>
>
> Another category of failure in e2e tests, such as ComputeSpec_1, ComputeSpec_2, ComputeSpec_3,
RaceConditions_1, RaceConditions_3, RaceConditions_4, RaceConditions_7, RaceConditions_8.
> Here is stack:
> ERROR 6003: Invalid cache specification. File doesn't exist: C:/Program Files (x86)/GnuWin32/bin/head.exe
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobCreationException: ERROR
2017: Internal error creating job configuration.
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:723)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:258)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:151)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1318)
> at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1303)
> at org.apache.pig.PigServer.execute(PigServer.java:1293)
> at org.apache.pig.PigServer.executeBatch(PigServer.java:364)
> at org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:133)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
> at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:166)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:84)
> at org.apache.pig.Main.run(Main.java:561)
> at org.apache.pig.Main.main(Main.java:111)
> Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR 6003: Invalid
cache specification. File doesn't exist: C:/Program Files (x86)/GnuWin32/bin/head.exe
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.setupDistributedCache(JobControlCompiler.java:1151)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.setupDistributedCache(JobControlCompiler.java:1129)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:447)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message