示例代码:
import pandas as pd
import numpy as np
df = pd.DataFrame({
'key1':['a', 'a', 'b', 'b', 'a'],
'key2':['one', 'two', 'one', 'two', 'one'],
'data1':np.random.randn(5),
'data2':np.random.randn(5)
})
grouped = df.groupby(df['key1'])
print(grouped.mean())
------------------------------------------------------------------------------------------------------------------
data1 data2
key1
a -0.464027 -0.992397
b -0.629515 -0.391474
- 可以使用字符串形式的列名:
df.groupby('key1')
- grouped返回的是一个DataFrameGroupBy对象,他没有做任何运算,所以速度非常快。
- groupby可以接受的参数可以有很多种,比如一个list:
print df.groupby([1,2,2,3,3]).mean()
------------------------------------------------------------------------------------------------------------------
data1 data2
1 -0.664111 0.964810
2 -0.074053 0.802726
3 0.122837 -0.035785
- 如果只需要data1的数据,以下几种写法是一样的。
- grouped = df['data1'].groupby(df['key1']).mean()
- grouped = df.groupby(df['key1'])['data1'].mean()
- grouped = df.groupby(df['key1']).mean()['data1']
- 返回一个层次化的结果:
print grouped = df.groupby([df['key1'],df['key2']]).mean()
------------------------------------------------------------------------------------------------------------------
data1 data2
key1 key2
a one 0.190639 -1.077724
two 1.523810 -0.753498
b one -1.026170 -0.893146
two -0.051379 1.461553
- 可以用grouped.size()查看分组的大小,比如:
grouped = df.groupby([df['key1'],df['key2']])
print(grouped.size()) #grouped.size()是一个拥有MultiIndex的Series
print(type(grouped.size()))
print(type(grouped.size().index))
------------------------------------------------------------------------------------------------------------------
key1 key2
a one 2
two 1
b one 1
two 1
dtype: int64
<class 'pandas.core.series.Series'>
<class 'pandas.core.indexes.multi.MultiIndex'>
- 可以对group对象进行迭代
for i,j in df.groupby([df['key1'],df['key2']]):
print(i) # i其实是个tuple
print('-----------')
print(j) # j是个DataFrame
------------------------------------------------------------------------------------------------------------------
('a', 'one')
-----------
data1 data2 key1 key2
0 0.815046 1.269757 a one
4 -0.604281 -0.956418 a one
('a', 'two')
-----------
data1 data2 key1 key2
1 -0.938286 2.636096 a two
('b', 'one')
-----------
data1 data2 key1 key2
2 -0.454884 0.141963 b one
('b', 'two')
-----------
data1 data2 key1 key2
3 -1.042242 0.618984 b two
- 以函数作为groupby的参数:
print df.groupby(lambda x:'even' if x%2==0 else 'odd').mean()
------------------------------------------------------------------------------------------------------------------
data1 data2
even 0.645358 -0.642165
odd 0.160585 -0.429005
这个不难理解,单数行和双数行分别作为两组进行聚合。当然,如果给df加一个字符串形式的index,这样的写法就有问题了,因为传进来的x就不能进行对2取余操作了。
- 可以根据索引进行分组
index = pd.MultiIndex.from_arrays([['even','odd','even','odd','even'],
[0,1,2,3,4]],names=['a','b'])
df.index = index
print(df.groupby(level='a').mean())
------------------------------------------------------------------------------------------------------------------
data1 data2
a
even -0.113491 0.730719
odd 0.076897 0.016876
- 除了内置的函数比如mean(),sum()等,还可以自定义聚合函数:
df.groupby('key1')['data1','data2'].agg(lambda arr:arr.max()-arr.min())
------------------------------------------------------------------------------------------------------------------
data1 data2
key1
a 2.508309 2.334477
b 0.107973 0.203492
2.508309其实就是取出'key1==a'的所有data1的值,组成一个数组。然后用最大值减去最小值。其他3项同理。
- agg可以接受一个函数列表:
print df.groupby('key1')['data1','data2'].agg(['min','max'])
print df.groupby('key1')['data1','data2'].agg(['min','max']).columns
------------------------------------------------------------------------------------------------------------------
data1 data2
min max min max
key1
a -1.586040 0.922269 -1.312042 1.022435
b 0.527926 0.635899 0.279316 0.482807
MultiIndex(levels=[[u'data1', u'data2'], [u'min', u'max']],labels=[[0, 0, 1, 1], [0, 1, 0, 1]])
- 或者可以提供一个从列名到函数的映射:
print df.groupby('key1').agg({'data1':'min','data2':'max'})
------------------------------------------------------------------------------------------------------------------
data1 data2
key1
a -1.586040 1.022435
b 0.527926 0.482807
- 除了聚合,还可以进行transform()
>>> df
data1 data2 key1 key2
a b
even 0 0.922269 0.110285 a one
odd 1 -0.181773 1.022435 a two
even 2 0.635899 0.279316 b one
odd 3 0.527926 0.482807 b two
even 4 -1.586040 -1.312042 a one
[5 rows x 4 columns]
>>> df.groupby('key1').transform('mean')
data1 data2
a b
even 0 -0.281848 -0.059774
odd 1 -0.281848 -0.059774
even 2 0.581912 0.381061
odd 3 0.581912 0.381061
even 4 -0.281848 -0.059774
其实就是.mean()以后又把结果反向传播到df