O.rka O.rka - 15 days ago 5
Python Question

How to "melt" `pandas.DataFrame` objects in Python 3?

I'm trying to

melt
certain columns of a
pd.DataFrame
while preserving columns of the other. In this case, I want to
melt
sine
and
cosine
columns into
values
and then which column they came from (i.e.
sine
or
cosine
) into a new columns entitled
data_type
then preserving the original
desc
column.

How can I use
pd.melt
to achieve this without melting and concatenating each component manually?


# Data
a = np.linspace(0,2*np.pi,100)
DF_data = pd.DataFrame([a, np.sin(np.pi*a), np.cos(np.pi*a)], index=["t", "sine", "cosine"], columns=["t_%d"%_ for _ in range(100)]).T
DF_data["desc"] = ["info about this" for _ in DF_data.index]


enter image description here

The round about way I did it:

# Melt each part
DF_melt_A = pd.DataFrame([DF_data["t"],
DF_data["sine"],
pd.Series(DF_data.shape[0]*["sine"], index=DF_data.index, name="data_type"),
DF_data["desc"]]).T.reset_index()
DF_melt_A.columns = ["idx","t","values","data_type","desc"]
DF_melt_B = pd.DataFrame([DF_data["t"],
DF_data["cosine"],
pd.Series(DF_data.shape[0]*["cosine"], index=DF_data.index, name="data_type"),
DF_data["desc"]]).T.reset_index()
DF_melt_B.columns = ["idx","t","values","data_type","desc"]

# Merge
pd.concat([DF_melt_A, DF_melt_B], axis=0, ignore_index=True)


enter image description here

If I do
pd.melt(DF_data
I get a complete meltdown

enter image description here

In response to the comments:
enter image description here

Answer

allright so I had to create a similar df because I did not have access to your a variable. I change your a variable for a list from 0 to 99... so t will be 0 to 99

you could do this :

a = range(0, 100)
DF_data = pd.DataFrame([a, [np.sin(x)for x in a], [np.cos(x)for x in a]], index=["t", "sine", "cosine"], columns=["t_%d"%_ for _ in range(100)]).T
DF_data["desc"] = ["info about this" for _ in DF_data.index]

df = pd.melt(DF_data, id_vars=['t','desc'])
df.head(5)

this should return what you are looking for.

     t             desc variable     value
0  0.0  info about this     sine  0.000000
1  1.0  info about this     sine  0.841471
2  2.0  info about this     sine  0.909297
3  3.0  info about this     sine  0.141120
4  4.0  info about this     sine -0.756802