Python Question

Appending 5 text files columnwise

I want to add all lines of 4 text files and have an output file with 5 columns. one of my files has two columns. I have tryed to do it with csv.reader, but I am not able to get the correct result. At this moment I am working on this code:

from os import walk
import csv
mypath = 'C:\\Users\\files to append'
o_data = []
files = []
for (dirpath, dirnames, filenames) in walk(mypath):
files.extend(filenames)
break
print(files)
for afile in files:
file_h=open(afile)
a_list = []
a_list.append(file_h.read())
csv_reader = csv.reader(file_h, delimiter = ' ')
for row in csv_reader:
a_list.append(row[0])
o_data.append((n for n in a_list))
file_h.close()

with open('output.dat', 'w') as op_file:
csv_writer = csv.writer(op_file, delimiter = ' ')
for row in list(zip(*o_data)):
csv_writer.writerow(row)


my five text files look like this with differnt values:

SCALAR
ND 9418
ST 0
TS 45000.34
0.0000
100.02


the result should be like this ( 4 headers and 5 number of columns):

SCALAR SCALAR SCALAR SCALAR
ND 9418 ND 9418 ND 9418 ND 9418
ST 0 ST 0 ST 0 ST 0 ST 0
TS 45000.34 TS 45000.34 TS 45000.34 TS 45000.34
0.0000 1.0000 2.4344 4.5656 81.2123
100.02 123.32 333.85 435.33 987.11


I would appreciate any suggustion.

Attempt 2

I tryed to rewrite it in other way. So this is my solution, but it does not work properly. I cannot understand why it cannot rename the "output1.out" to "output.out"

This is the code:

import os

"""
Please put all the bnecessary data into the directory
"""

f = []
for file in os.listdir('C:\\Users\\Append'):
if file.endswith(".dat"):
f.append(file)
print(file)
os.rename(file,"input.dat")
file = file.rsplit('.', 1)[0]
print(file)
with open("output.out", "r") as textfile1, open("input.dat", "r") as textfile2,\
open("output1.out", "w") as out:
for x, y in zip(textfile1, textfile2):
x = x.strip()
y = y.strip()
print("{0} {1}".format(x, y), file = out)
print(fname)
os.rename("input.dat", file+".txt")
os.rename("output1.out", "output.out" )
print(f) # just for checking

Answer

The following should do what you are looking for. Rather than using os.walk() it uses glob.glob() to get a suitable list of files, e.g. *.dat for all dat files, or you could use i*.dat depending on your file names.

It reads each file into a data list and then uses the zip(*data) trick to read the rows of columns as columns of rows. With these it then combines each of the lists for each row into a single list using chain.from_iterable() and writes this out to the output CSV file with spaces as delimiters.

from itertools import chain
import glob
import csv

with open('output.csv', 'w', newline='') as f_output:
    csv_output = csv.writer(f_output, delimiter=' ')
    data = []

    for filename in glob.glob('c*.txt'):
        with open(filename, newline='') as f_input:
            csv_input = csv.reader(f_input, delimiter=' ', skipinitialspace=True)
            data.append(list(csv_input))

    for row in zip(*data):
        csv_output.writerow(chain.from_iterable(row))

Giving you something like:

SCALAR SCALAR SCALAR SCALAR SCALAR
ND 9418 ND 9419 ND 9420 ND 9421 ND 9422
ST 0 ST 1 ST 2 ST 3 ST 4
TS 45000.34 TS 45000.35 TS 45000.36 TS 45000.37 TS 45000.38
0.0000 0.0001 0.0002 0.0003 0.0004
100.02 100.03 100.04 100.05 100.06