Simplicity Simplicity -4 years ago 178
Python Question

Python - Retrieving data and labels from a pickle file

I have a pickle file that looks as follows:

[array([[[148, 124, 115],
[150, 127, 116],
[154, 129, 121],
...,
[159, 142, 133],
[159, 142, 133],
[161, 145, 142]],

[[165, 136, 145],
[176, 137, 141],
[178, 138, 144],
...,
[199, 163, 171],
[202, 163, 167],
[200, 158, 163]]]), array([1, 1])]


In a previous question, we were able to retrieve the data and label by doing that separately. But, when I have many images that approach would not be suitable.

My script now looks as follows:

data, labels = [], []
for i in range(0, 1):

filename = 'data.pickle'
batch_data = unpickle(filename)
if len(data) > 0:
data = np.vstack((data, batch_data[0][i]))
labels = np.hstack((labels, batch_data[1][i]))
else:
data = batch_data[0][0]
labels = batch_data[1][0]

data = data.astype(np.float32)
return data, labels


When I run the code and
print
the labels for instance, I always get
1
, while I was expecting to get two labels,
[1 1]
(I'm not sure if they should be displayed as an array?)

What am I doing wrong here?

Thanks.

Answer Source

I was able to get labels the way you expect it to be. I used

# Create batch data that represents what you are asking, I created three labels and data
batch_data = np.array([[np.random.random((5,5)), np.random.random((5,5)), np.random.random((5,5))], np.array([1,1,1])])

#pickle the data
import pickle
pickle.dump( batch_data, open( "test.pickle", "wb" ) )

# create data and labels seperately

def test_func(batch_data):
    data, labels = [], []
    for i in range(0, batch_data.shape[1]):
        if len(data) > 0:
            data = np.vstack((data, batch_data[0][i]))
            labels = np.hstack((labels, batch_data[1][i]))
        else:
            data = batch_data[0][0]
            labels = batch_data[1][0]
        data = data.astype(np.float32)
    return data, labels

# unpickle
unpickled_batch_data = pickle.load(open( "test.pickle", "rb" ))

# get stacked data and labels
data, labels =  test_func(unpickled_batch_data)
print labels

returns

[1 1 1]
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download