I am not sure about Pig, but its easily achievable in MapReduce. We had a similar requirement, we had to convert logs from RFC syslog format (5424) into JSON. We have a MR job which does this for us. The reason why we chose MR was mainly for Error Handling - like missing fields in some records, removing some blacklisted fields (like SSN etc) which we thought was easier to do it in MR than pig.


On Sat, Jul 4, 2015 at 10:50 AM, Divya Gehlot <divya.htconex@gmail.com> wrote:
I am new to pig and I have a log file in below formatĀ 

for which I need to write pig script and store in below JSON formatĀ 
{Message1:Message,date:2015-07-01,Time:22:58:53.66,E:E,machine :xxxxxxxxxx.xxx.xxxxx.xxx,data:{key_1:value,key_2:value,key_3:value,key_3:value,key_3:value,key_5:value.....} }

Can somebody help me in writing custom loader .

would really appreciate your help.