Numpy savetxt do not handle 18 ditgits long int correctly

Question

I need to save some long int to text file. I use Numpy savetxt to do it as below:

np.savetxt('test.txt', np.c_[objID,z,e,PA,n],fmt='%i    %1.4f    %1.4f    %1.4f    %1.4f')

I tried either

%i

or

%f

for

objID

, but the item in the

text.txt

is not correct. I list a few

objID

from the source as below:

587722952230174996

587722952230175035

587722952230175138

However, the same item in the

test.txt

becomes:

587722952230174976

587722952230174976

587722952230175104

Anyone knows how to fix that? Thanks.

Recommended for you: Get network issues from WhatsUp Gold. Not end users.

Answer 1

I don't have numpy installed in Py2, but I can recreate your numbers with plain Python operations:

>>> ll=[587722952230174996,587722952230175035,587722952230175138]
>>> ll
[587722952230174996L, 587722952230175035L, 587722952230175138L]
>>> '%.0f, %.0f, %.0f'%tuple(ll)
'587722952230174976, 587722952230174976, 587722952230175104'
>>> '%i, %i, %i'%tuple(ll)
'587722952230174996, 587722952230175035, 587722952230175138'

The problem display is consistent with rounding to float, but %i formatting handles them without problem.

With numpy on py3

In [243]: '%i, %i, %i'%tuple(np.array(ll))
Out[243]: '587722952230174996, 587722952230175035, 587722952230175138'

In [245]: '%i, %i, %i'%tuple(np.array(ll, np.float))
Out[245]: '587722952230174976, 587722952230174976, 587722952230175104'

np.array(ll) has dtype int64.

==================

My guess is that:

`np.c_[objID,z,e,PA,n]`

is concatenating the values, and creating a float64 array. The other numbers appear to be floats, so even if objID is long, the whole thing is float. So the %i format still sees the float 'rounding'.

What's the shape and type of objID',z', etc? You may have to join them in a structured array if you want to keep the full long value.

In [249]: z=[.1,.2,.3]
In [250]: np.c_[ll,z]
Out[250]: 
array([[  5.87722952e+17,   1.00000000e-01],
       [  5.87722952e+17,   2.00000000e-01],
       [  5.87722952e+17,   3.00000000e-01]])

In [255]: np.savetxt('test.csv', data,fmt='%i, %f')
In [256]: cat test.csv
587722952230174976, 0.100000
587722952230174976, 0.200000
587722952230175104, 0.300000

Taking the structured array route:

In [274]: ds = np.empty((3,),dtype=[('f0',np.int64),('f1',np.float64)])
In [275]: ds['f0']=ll
In [276]: ds['f1']=z
In [277]: ds
Out[277]: 
array([(587722952230174996, 0.1), (587722952230175035, 0.2),
       (587722952230175138, 0.3)], 
      dtype=[('f0', '<i8'), ('f1', '<f8')])
In [278]: np.savetxt('test.csv', ds,fmt='%i,   %f')
In [279]: cat test.csv
587722952230174996,   0.100000
587722952230175035,   0.200000
587722952230175138,   0.300000

Numpy savetxt do not handle 18 ditgits long int correctly

Sign up