python的drop duplicates_python – Pandas drop_duplicates方法不起作用

drop_duplicates不能用于数据框中的列表,如错误消息所示.但是,您可以删除作为str的数据帧上的重复项,然后使用结果中的索引从原始df中提取行.

建立

df = pd.DataFrame({‘Keyword’: {0: ‘apply’, 1: ‘apply’, 2: ‘apply’, 3: ‘terms’, 4: ‘terms’},

‘X’: {0: [1, 2], 1: [1, 2], 2: ‘xy’, 3: ‘xx’, 4: ‘yy’},

‘Y’: {0: ‘yy’, 1: ‘yy’, 2: ‘yx’, 3: ‘ix’, 4: ‘xi’}})

#Drop directly causes the same error

df.drop_duplicates()

Traceback (most recent call last):

…

TypeError: unhashable type: ‘list’

解

#convert hte df to str type, drop duplicates and then select the rows from original df.

df.iloc[df.astype(str).drop_duplicates().index]

Out[205]:

Keyword X Y

0 apply [1, 2] yy

2 apply xy yx

3 terms xx ix

4 terms yy xi

#the list elements are still list in the final results.

df.iloc[df.astype(str).drop_duplicates().index].loc[0,’X’]

Out[207]: [1, 2]