user2725742 user2725742 - 1 month ago 19
Python Question

Generate 3D array from my data

I am struggling to get the data below into a 3D array. A little help would be appreciated. If the outline below is what you would do, feel free to change it.

with open("data.txt","r") as f:
info = []
for line in f:
if line[0] == "#":
continue
freq, delay, reading = line.split(',')

if not info:
# Enter first entry into info
else:
# Insert freq, delay and reading into 3D array

# Print data.txt in something like the format at the end of question
# (not necessarily in order)
for freq in info:
print freq[0]
for delay in freq:
print "\t%d" % delay[0]
for reading in delay:
print "\t\t%f" % reading


Below is a sample data.txt

#freq,delay,reading
291,1000,-29.22
320,1000,-29.33
270,2000,-29.11
240,1500,-29.04
220,1000,-28.89
272,1000,-29.11
291,1500,-29.21
320,1000,-29.34
270,2000,-29.1
240,1000,-29.02
220,1500,-28.89
272,1500,-29.12
291,1000,-29.19
320,1000,-29.32
270,2000,-29.1
240,1000,-29.02
220,1000,-28.88
272,1000,-29.1


Roughly desired output below.

220
1000
-28.89
-28.88
1500
-28.89
240
1000
-29.02
-29.02
1500
-29.04
270
2000
-29.11
-29.1
-29.1
272
1000
-29.11
-29.1
1500
-29.12
291
1000
-29.22
-29.19
1500
-29.21
320
1000
-29.34
-29.33
-29.32

Answer

I do not know how big your text file is. But the easiest way to do that would be to order the lines before splitting them. However, that requires to read them into memory (or create a new file if you do not have enough memory - unlikely).

lines = []
with open("data.txt") as file:
   for line in file:
      lines.append(line)
sorted_lines = sorted(lines)

last_freq = None
last_delay = None

for line in sorted_lines:
    freq, delay, reading = line.strip('\n').split(',')
    if last_freq_column != freq:
         print freq,
    last_freq = freq
    if last_delay_column != delay:
         print delay,
    last_delay = delay
    print reading

Consider it pseudo code since I did not test it and their might be typos. But you will get the idea.

By the way your output is not really a 3D array. But if you were to generate a list of lists of lists (as you have to do it in Python) you could easily modify the code.