tharen tharen - 2 months ago 12
Python Question

Handle Fortran Character Arrays from Python with F2PY

I have a legacy Fortran library I've wrapped with F2PY. However, I'm at a loss for how to properly read character arrays declared as module data, from Python. The data data comes through, but the array is transposed in such a way that it is indiscernible. How can I get Numpy to correctly handle my array? I'd be satisfied with a 2 dimensional array of characters if they were in an intelligible order.

The character arrays are declared and populated in Fortran like so:

module plot_mod
implicit none

CHARACTER*4, JSP(39)

...

JSP = (/ &
'SF ', 'WF ', 'GF ', 'AF ', 'RF ', 'SS ', 'NF ', &
'YC ', 'IC ', 'ES ', 'LP ', 'JP ', 'SP ', 'WP ', &
'PP ', 'DF ', 'RW ', 'RC ', 'WH ', 'MH ', 'BM ', &
'RA ', 'WA ', 'PB ', 'GC ', 'AS ', 'CW ', 'WO ', &
'WJ ', 'LL ', 'WB ', 'KP ', 'PY ', 'DG ', 'HT ', &
'CH ', 'WI ', ' ', 'OT '/)

end module plot_mod


In Python 2.7 (previous version of numpy) I could do this:

x = numpy.frombuffer(fvslib.plot_mod.jsp.data, numpy.dtype('a4'))


But now Python (3.4.4) and Numpy (1.10.4) raises an error,
BufferError: memoryview: underlying buffer is not C-contiguous
.

I know I should be able to get Numpy to handle this for me by reshaping, or using stride tricks, but I can't seem to figure it out. The array is reported as F-contiguous, so at least that seems correct.

If I simply print the array it looks like this:

array([[b'S', b' ', b' ', b'L'],
[b'F', b'L', b' ', b' '],
[b' ', b'P', b'B', b' '],
[b' ', b' ', b'M', b'W'],
[b'W', b' ', b' ', b'B'],
[b'F', b'J', b' ', b' '],
[b' ', b'P', b'R', b' '],
[b' ', b' ', b'A', b'K'],
[b'G', b' ', b' ', b'P'],
[b'F', b'S', b' ', b' '],
[b' ', b'P', b'W', b' '],
[b' ', b' ', b'A', b'P'],
[b'A', b' ', b' ', b'Y'],
[b'F', b'W', b' ', b' '],
[b' ', b'P', b'P', b' '],
[b' ', b' ', b'B', b'D'],
[b'R', b' ', b' ', b'G'],
[b'F', b'P', b' ', b' '],
[b' ', b'P', b'G', b' '],
[b' ', b' ', b'C', b'H'],
[b'S', b' ', b' ', b'T'],
[b'S', b'D', b' ', b' '],
[b' ', b'F', b'A', b' '],
[b' ', b' ', b'S', b'C'],
[b'N', b' ', b' ', b'H'],
[b'F', b'R', b' ', b' '],
[b' ', b'W', b'C', b' '],
[b' ', b' ', b'W', b'W'],
[b'Y', b' ', b' ', b'I'],
[b'C', b'R', b' ', b' '],
[b' ', b'C', b'W', b' '],
[b' ', b' ', b'O', b' '],
[b'I', b' ', b' ', b' '],
[b'C', b'W', b' ', b' '],
[b' ', b'H', b'W', b' '],
[b' ', b' ', b'J', b'O'],
[b'E', b' ', b' ', b'T'],
[b'S', b'M', b' ', b' '],
[b' ', b'H', b'L', b' ']],
dtype='|S1')


What I would like an array like this:

[['SF ']
, ['WF ']
, ['GF ']
, ['AF ']
, ['RF ']
, ['SS ']
, ['NF ']
, ['YC ']
, ['IC ']
, ['ES ']
, ['LP ']
, ['JP ']
, ['SP ']
, ['WP ']
, ['PP ']
, ['DF ']
, ['RW ']
, ['RC ']
, ['WH ']
, ['MH ']
, ['BM ']
, ['RA ']
, ['WA ']
, ['PB ']
, ['GC ']
, ['AS ']
, ['CW ']
, ['WO ']
, ['WJ ']
, ['LL ']
, ['WB ']
, ['KP ']
, ['PY ']
, ['DG ']
, ['HT ']
, ['CH ']
, ['WI ']
, [' ']
, ['OT ']]

Answer

I haven't tried running f2py on your module, but if I define the array you show as:

In [11]: s = array([[b'S', b' ', b' ', b'L'],
    ...:    [b'F', b'L', b' ', b' '],
    ...:    [b' ', b'P', b'B', b' '],
    ...:    [b' ', b' ', b'M', b'W'],
    ...:    [b'W', b' ', b' ', b'B'],
    ...:    [b'F', b'J', b' ', b' '],
    ...:    [b' ', b'P', b'R', b' '],
    ...:    [b' ', b' ', b'A', b'K'],
    ...:    [b'G', b' ', b' ', b'P'],
    ...:    [b'F', b'S', b' ', b' '],
    ...:    [b' ', b'P', b'W', b' '],
    ...:    [b' ', b' ', b'A', b'P'],
    ...:    [b'A', b' ', b' ', b'Y'],
    ...:    [b'F', b'W', b' ', b' '],
    ...:    [b' ', b'P', b'P', b' '],
    ...:    [b' ', b' ', b'B', b'D'],
    ...:    [b'R', b' ', b' ', b'G'],
    ...:    [b'F', b'P', b' ', b' '],
    ...:    [b' ', b'P', b'G', b' '],
    ...:    [b' ', b' ', b'C', b'H'],
    ...:    [b'S', b' ', b' ', b'T'],
    ...:    [b'S', b'D', b' ', b' '],
    ...:    [b' ', b'F', b'A', b' '],
    ...:    [b' ', b' ', b'S', b'C'],
    ...:    [b'N', b' ', b' ', b'H'],
    ...:    [b'F', b'R', b' ', b' '],
    ...:    [b' ', b'W', b'C', b' '],
    ...:    [b' ', b' ', b'W', b'W'],
    ...:    [b'Y', b' ', b' ', b'I'],
    ...:    [b'C', b'R', b' ', b' '],
    ...:    [b' ', b'C', b'W', b' '],
    ...:    [b' ', b' ', b'O', b' '],
    ...:    [b'I', b' ', b' ', b' '],
    ...:    [b'C', b'W', b' ', b' '],
    ...:    [b' ', b'H', b'W', b' '],
    ...:    [b' ', b' ', b'J', b'O'],
    ...:    [b'E', b' ', b' ', b'T'],
    ...:    [b'S', b'M', b' ', b' '],
    ...:    [b' ', b'H', b'L', b' ']],
    ...:   dtype='|S1')

I can get an array that looks like what you want with:

In [12]: s.T.reshape(-1, 4).view('S4')
Out[12]: 
array([[b'SF  '],
       [b'WF  '],
       [b'GF  '],
       [b'AF  '],
       [b'RF  '],
       [b'SS  '],
       [b'NF  '],
       [b'YC  '],
       [b'IC  '],
       [b'ES  '],
       [b'LP  '],
       [b'JP  '],
       [b'SP  '],
       [b'WP  '],
       [b'PP  '],
       [b'DF  '],
       [b'RW  '],
       [b'RC  '],
       [b'WH  '],
       [b'MH  '],
       [b'BM  '],
       [b'RA  '],
       [b'WA  '],
       [b'PB  '],
       [b'GC  '],
       [b'AS  '],
       [b'CW  '],
       [b'WO  '],
       [b'WJ  '],
       [b'LL  '],
       [b'WB  '],
       [b'KP  '],
       [b'PY  '],
       [b'DG  '],
       [b'HT  '],
       [b'CH  '],
       [b'WI  '],
       [b'    '],
       [b'OT  ']], 
      dtype='|S4')

Note that the data type is 'S4', to match the declared size of the Fortran array.

That result leaves a trivial second dimension, so you might want to convert it to a one-dimensional array, e.g.

In [22]: s.T.reshape(-1, 4).view('S4')[:,0]
Out[22]: 
array([b'SF  ', b'WF  ', b'GF  ', b'AF  ', b'RF  ', b'SS  ', b'NF  ',
       b'YC  ', b'IC  ', b'ES  ', b'LP  ', b'JP  ', b'SP  ', b'WP  ',
       b'PP  ', b'DF  ', b'RW  ', b'RC  ', b'WH  ', b'MH  ', b'BM  ',
       b'RA  ', b'WA  ', b'PB  ', b'GC  ', b'AS  ', b'CW  ', b'WO  ',
       b'WJ  ', b'LL  ', b'WB  ', b'KP  ', b'PY  ', b'DG  ', b'HT  ',
       b'CH  ', b'WI  ', b'    ', b'OT  '], 
      dtype='|S4')