Lucas Lucas - 19 days ago 8
Python Question

python - Transform data to numpy array for sklearn

I have a dataset formed by some text columns (with limited possibilities) and some numeric columns in a csv format. Is there any way to automatically transform the text columns to numbers (for example: A will be 0, B will be 1 and so on) to transform the dataset to

np.array
?

This will be later used on
scikit-learn
, so it needs to be
np.array
at the end of all the processing.

EDIT: Adding one line of the dataset:

ENABLED;ENABLED;10;MANUAL;ENABLED;ENABLED;1800000;OFF;0.175;5.0;0.13;OFF;NEITHER;ENABLED;-65;2417;"wifi01";65;-75;DISCONNECTED;NO;NO;2621454;432477;3759;2.2436838539123705E-6;

Answer

You can apply sklearn.preprocessing.labelEncoder() to each text column. Here is an example:

import pandas as pd
df = pd.DataFrame({'col1': [1,2,3,4,5],
                  'col2': ['ON','ON','OFF','OFF','ON']})
from sklearn.preprocessing import LabelEncoder 
lb = LabelEncoder()
df['encoded'] = lb.fit_transform(df.col2)
df

  col1  col2  encoded
0   1    ON     1
1   2    ON     1
2   3    OFF    0
3   4    OFF    0
4   5    ON     1

I just added the numerical values in another column but you can replace them. Also, you can convert them into numpy array:

df.as_matrix()
array([[1, 'ON', 1],
       [2, 'ON', 1],
       [3, 'OFF', 0],
       [4, 'OFF', 0],
       [5, 'ON', 1]], dtype=object)

Here is how you may encode with numpy. In this example I am just passing a python list:

alist = ['ON','ON','OFF','OFF','ON']
uniqe_values , y = np.unique(alist, return_inverse=True)
print uniqe_values
print y

The results are:

['OFF' 'ON']
[1 1 0 0 1]