Pakox.Wang Pakox.Wang - 22 days ago 8
Python Question

Python - Reading and Writing Structured Binary Files

Currently I'm trying to manipulate a kind of binary file,
which structure is like this:

FileHeader + [SectionHeader + Binary(1D-array)] * NumSetions


After searching on the internet, I came up with the following code to read it:

import numpy as np

# Always BIG Endian
#file_header bytes: 12
file_header_dtype = np.dtype([
('nsection', '>i4'), # number of sections
('nsample', '>i4'), # number of samples for each section
('dt', '>f4'), # sampling rate
])

#section_header bytes: 28
section_header_dtype = np.dtype([
('rec', '>i4'), # record number
('x', '>f8'), # x, in meter
('y', '>f8'), # y, in meter
('z', '>f8'), # z, in meter
])

def dtype_section(nt):
return np.dtype(section_header_dtype.descr + [('binary', ('>f4',nt))])

def read_file_header(_file):
with open(_file, 'rb') as f:
file_header = np.fromstring(f.read(12), dtype=file_header_dtype)
return file_header

def readFile(filename):
raw = open(filename, 'rb')
file_header = np.fromstring(raw.read(12), dtype=file_header_dtype)
nt = file_header['nsample'][0]
dtype_file_sections = dtype_section(nt)
data = np.fromstring(raw.read(), dtype=dtype_file_sections)
return (file_header, data)


With this way, I can easily call the header-striped binary part and perform
plt.imshow
or anything else.

data = readFile('site1.bin')
data_arr = data[1]['binary']
#plt.imshow(data_arr)


The problem is, I cannot find a way to output the data while maintaining same data structure

ndarray.tofile()
only works for one array per time

And
np.array((data[0],data[1])).tofile('test')
would cause IOError


IOError: cannot write object arrays to a file in binary mode


I'm pretty new to Python and I'm not sure if I made any mistake.
Or should I consider another way to read this kind of file, rather than using
numpy.dtype
?
Please help me.

Answer Source

The straightforward way would be to simply write to a binary file:

with open('test','wb') as f:
    f.write(data[0].tobytes())
    f.write(data[1].tobytes())