背景:有一个list,里面的每一个元素都是dict,根据某一个key进行去重,在这里,key代表question
#!/usr/bin/env python # -*- coding: utf-8 -*- # created by fhqplzj on 2017/12/07 上午11:38 from itertools import groupby from operator import itemgetter import pandas as pd def distinct(items): questions = map(itemgetter('question'), items) df = pd.DataFrame({ 'items': items, 'questions': questions }) return df.drop_duplicates(['questions'])['items'].tolist() def distinct2(items): exist_questions = set() result = [] for item in items: question = item['question'] if question not in exist_questions: exist_questions.add(question) result.append(item) return result def distinct3(items): key = itemgetter('question') items = sorted(items, key=key) return [next(v) for _, v in groupby(items, key=key)] def distinct4(items): from itertools import compress mask = (~pd.Series(map(itemgetter('question'), items)).duplicated()).tolist() return list(compress(items, mask)) if __name__ == '__main__': data = [ {'question': 'a', 'ans': 'b'}, {'question': 'b', 'ans': 'd'}, {'question': 'a', 'ans': 'p'}, {'question': 'b', 'ans': 'e'} ] print distinct4(data)
以上这篇python去重,一个由dict组成的list的去重示例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持菜鸟教程(cainiaojc.com)。
声明:本文内容来源于网络,版权归原作者所有,内容由互联网用户自发贡献自行上传,本网站不拥有所有权,未作人工编辑处理,也不承担相关法律责任。如果您发现有涉嫌版权的内容,欢迎发送邮件至:notice#cainiaojc.com(发邮件时,请将#更换为@)进行举报,并提供相关证据,一经查实,本站将立刻删除涉嫌侵权内容。