Bogdan Bogdan - 3 months ago 7
Linux Question

bash find and replace "enter" with comma before given pattern

I have a big CSV file that contains paragraphs like this:

first line1
second line1
third line1
fourth line1
first line2
second line2
third line2
fourth line2


After processing I would like to be translated into this:

first line1,second line1,third line1,fourth line1
first line2,second line2,third line2,fourth line2


Note: first line, second line, etc contains special characters like . , " :

I'm thinking that an option could be to find "second" word from second line1 and replace the "enter" before it with a comma, this way the second line1 will be on the right side of the first line1.

How can I do this?

Indeed, it's very possible that above example is not the real ACTUAL data, here it is:

137822118,user,User,192.168.100.20,2016-07-25 23:19:05 DST,iScript,iScript send MML command,B12345-BXL_ABCD_BD,Succeeded,"NE Name:B12345-BXL_ABCD_BD
MML Command:LST DEVIP:OPONEMS=""user"", IPOFEMSWS=""192.168.100.20"";
MML Result:Successful.
",2016-07-25 23:19:05 DST
137821234,user,User,192.168.100.21,2016-07-25 23:19:05 DST,iScript,iScript send MML command,B12345-BXL_ASDF_BD,Succeeded,"NE Name:B12345-BXL_ASDF_BD
MML Command:LST DEVIP:OPONEMS=""user"", IPOFEMSWS=""192.168.100.21"";
MML Result:Successful.
",2016-07-25 22:18:05 DST


The CSV file contains a lot of this kind of paragraphs.

The output should be (one paragraph in one line):

137822118,user,User,192.168.100.20,2016-07-25 23:19:05 DST,iScript,iScript send MML command,B12345-BXL_ABCD_BD,Succeeded,"NE Name:B12345-BXL_ABCD_BD,MML Command:LST DEVIP:OPONEMS=""user"", IPOFEMSWS=""192.168.100.20""; MML Result:Successful. ",2016-07-25 23:19:05 DST
137821234,user,User,192.168.100.21,2016-07-25 23:19:05 DST,iScript,iScript send MML command,B12345-BXL_ASDF_BD,Succeeded,"NE Name:B12345-BXL_ASDF_BD,MML Command:LST DEVIP:OPONEMS=""user"", IPOFEMSWS=""192.168.100.21""; MML Result:Successful. ",2016-07-25 22:18:05 DST





Thanks a lot for your help!

I tried your solutions and it almost works, but is not the expected result.
It seems that the example I give you is slight different from the source file because of the posting format here.

Please find below the real source CSV file (only a couple of lines as the complete file contains more than one million)

https://www.wetransfer.com/downloads/637b36b2148550ad090c22c9e8297a9c20160804081835/48b90b

Sorry for the misunderstanding and thank again!

Answer

You can use paste for this, for example:

$ paste -d, - - - - < file
first line1,second line1,third line1,fourth line1
first line2,second line2,third line2,fourth line2

- means standard input, when you specify N of them (N=4 in this example), paste will form a line from N lines from standard input.

-d is to specify the column separator, in this example a comma.

Comments