《Scikit-Learn与TensorFlow机器学习实用指南》 第07章中介绍了对多个分类器集成的一些方法，在特征重要性一节中在mnist数据集中使用随机森林分类器进行训练并绘制特征的重要程度。原书中这部分只是一笔带过，直接展示了一张结果图片，在这里将过程代码记录一下以备不时之需。
# -*- coding:utf-8 -*-
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
import matplotlib.pyplot as plt
mnist = np.load("mnist.npz") # minist数据集保存在本地
###############################训练部分###################################
rnd_clf = RandomForestClassifier(n_estimators=500, n_jobs=-1) # 使用500棵树
rnd_clf.fit(mnist['x_train'].reshape(60000, -1), mnist['y_train'])
y_pred = rnd_clf.predict(mnist['x_test'].reshape(10000,-1))
print("RandomForestClassifier's score = {:.2f}".format(accuracy_score(mnist['y_test'],y_pred)))
print(rnd_clf.feature_importances_.reshape(28, 28))
###############################结果绘制部分###################################
ax = plt.gca()
# Plot the heatmap
im = ax.imshow(rnd_clf.feature_importances_.reshape(28, 28), cmap='hot')
# Create colorbar
cbar = ax.figure.colorbar(im, ax=ax)
cbar.ax.set_ylabel("feature_importance", rotation=-90, va='bottom')
ax.set_title("Feature importance of rnf on mnist")
plt.savefig("mnist_feature_importance.jpg")
对于这台办公电脑来说，训练数据集及特征都有点大，在我这儿运行一次大概需要十分钟左右。以下便是输出的结果:
RandomForestClassifier's score = 0.97
[[0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00
0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00
0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00
6.72187144e-08 0.00000000e+00 7.05320940e-08 0.00000000e+00
0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00
0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00
0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00
0.00000000e+00 7.39360616e-08 7.23927153e-07 2.26719830e-06
2.23493533e-06 2.03704818e-06 2.09066722e-06 3.76107944e-06
3.06812052e-06 2.58693243e-06 3.60986162e-06 4.57920099e-06
4.68058582e-06 3.67373548e-06 1.33552823e-06 8.43976749e-07
9.10643628e-07 5.54599141e-07 6.07265291e-07 8.72811389e-07
0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 0.00000000e+00 1.94444808e-07
1.45884599e-07 4.52881485e-07 2.42784390e-06 1.16766910e-05
2.20257759e-05 2.98827127e-05 7.05722248e-05 1.52262744e-04
9.24315445e-05 2.36582435e-04 1.90520296e-04 1.57711311e-04
1.34269807e-04 1.29530630e-04 9.82215580e-05 4.88907992e-05
2.82908301e-05 1.23145819e-05 1.02558709e-05 3.27337022e-06
4.73702697e-07 1.94819814e-07 0.00000000e+00 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 2.76304171e-07 1.12135479e-06
9.00284828e-07 1.61649339e-06 1.19471951e-05 2.59706742e-05
7.05762213e-05 1.37033204e-04 3.07515149e-04 4.25079441e-04
7.79016029e-04 8.50674145e-04 1.34366013e-03 1.62881978e-03
2.51871450e-03 1.93833501e-03 1.21358337e-03 8.21835453e-04
3.55522864e-04 1.61982266e-04 5.95584866e-05 2.42254033e-05
1.22613102e-05 4.51435500e-06 4.95874527e-07 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 1.95193427e-07 1.47428670e-06
3.95559760e-06 1.38919782e-05 3.62863207e-05 7.09076160e-05
1.54253122e-04 3.29906731e-04 6.97046225e-04 9.67326983e-04
1.48656756e-03 1.88461559e-03 2.09312539e-03 2.13763408e-03
1.64360651e-03 1.16606077e-03 8.04435444e-04 6.31830052e-04
3.63080406e-04 2.20666244e-04 1.33016305e-04 8.04852147e-05
3.98133032e-05 1.85783469e-05 1.88352763e-06 2.89270784e-07]
[0.00000000e+00 0.00000000e+00 3.84533682e-07 2.44946140e-06
1.29899459e-05 5.15719072e-05 1.19611554e-04 2.45163883e-04
5.08438394e-04 1.25569964e-03 2.21606743e-03 2.69962612e-03
3.54370818e-03 4.28630755e-03 5.89175571e-03 7.13127169e-03
4.88390688e-03 3.32250464e-03 2.48584783e-03 1.44656261e-03
1.05670994e-03 7.64144520e-04 5.42288871e-04 3.72029242e-04
1.10726725e-04 2.77619228e-05 4.69172481e-06 1.15589953e-06]
[0.00000000e+00 2.78370886e-07 9.55331196e-07 8.32725905e-06
3.01224648e-05 9.50907701e-05 2.26386686e-04 5.07952131e-04
9.91298167e-04 1.77526391e-03 2.63567068e-03 2.67991069e-03
2.44873618e-03 3.05916544e-03 4.17432258e-03 4.22327580e-03
3.56994779e-03 2.88404796e-03 2.09588836e-03 1.63266454e-03
1.45328899e-03 1.38202690e-03 1.29180654e-03 8.62266531e-04
3.55113503e-04 6.57410774e-05 1.18211266e-05 2.47323407e-06]
[0.00000000e+00 5.14361427e-07 2.24354877e-06 1.40884452e-05
6.20754027e-05 1.87303695e-04 3.27890754e-04 6.36167690e-04
1.01215415e-03 1.74577923e-03 2.16138295e-03 2.28335792e-03
2.48369352e-03 3.90579732e-03 6.02817896e-03 6.45252730e-03
4.62868714e-03 3.02825872e-03 2.53389111e-03 2.20846801e-03
1.83896767e-03 1.51141958e-03 1.54608260e-03 1.16860823e-03
8.49292325e-04 1.19268313e-04 1.55474435e-05 3.20764924e-06]
[6.17531755e-08 7.38635664e-08 4.10303647e-06 1.87676648e-05
8.51605028e-05 1.99337002e-04 3.91957537e-04 6.85090089e-04
1.16877581e-03 1.54487380e-03 2.64381077e-03 3.12570362e-03
2.93835828e-03 3.97668856e-03 5.05140972e-03 5.88400982e-03
4.45087455e-03 3.15098514e-03 3.20671602e-03 3.00716422e-03
2.24023543e-03 1.60382282e-03 1.27847593e-03 1.02477039e-03
8.62156845e-04 1.40472700e-04 1.73068234e-05 1.39771472e-06]
[0.00000000e+00 6.65223879e-07 5.71982821e-06 3.12497152e-05
9.61934675e-05 2.42152294e-04 4.65062162e-04 8.32157649e-04
1.26378949e-03 2.00980115e-03 3.93206449e-03 4.65364264e-03
3.33566908e-03 3.18175617e-03 3.63681606e-03 4.16293304e-03
3.55544790e-03 3.79045995e-03 3.99677389e-03 3.42347091e-03
2.62385283e-03 1.79605945e-03 9.96434705e-04 6.33857483e-04
4.56679013e-04 1.24874257e-04 2.06209216e-05 2.00263818e-06]
[6.80949726e-08 3.45858136e-07 5.68729950e-06 2.84428099e-05
9.01500762e-05 2.08376465e-04 4.83340375e-04 9.89919898e-04
1.61786201e-03 3.08705423e-03 5.18482267e-03 5.77449421e-03
3.50145987e-03 2.89416282e-03 3.32467327e-03 3.58658713e-03
3.93005922e-03 3.83905449e-03 4.54133475e-03 3.67491374e-03
2.70386999e-03 2.24833291e-03 9.83920715e-04 5.16371227e-04
2.05544557e-04 6.44081007e-05 1.29052400e-05 2.07789453e-06]
[0.00000000e+00 6.25700572e-07 4.02965733e-06 2.66854553e-05
9.57816171e-05 2.22806706e-04 6.07724259e-04 1.23338241e-03
2.40826906e-03 3.76842721e-03 5.10132713e-03 5.08339103e-03
3.90752153e-03 3.17736478e-03 4.49254136e-03 4.41962319e-03
3.33349554e-03 3.51120338e-03 4.20688223e-03 3.28288235e-03
2.13962497e-03 2.18927803e-03 1.92566562e-03 5.89868981e-04
1.54939905e-04 3.70419139e-05 8.25124777e-06 9.21301750e-07]
[0.00000000e+00 1.95349787e-07 3.50259572e-06 1.73590460e-05
1.00214234e-04 2.88051414e-04 8.41321922e-04 1.70180490e-03
2.89144599e-03 4.60583401e-03 5.53223911e-03 6.09251702e-03
4.43053970e-03 3.92102201e-03 8.21176102e-03 5.10886437e-03
3.39743301e-03 3.87948149e-03 4.79443130e-03 2.54054145e-03
1.68360892e-03 2.09222645e-03 2.10872701e-03 9.05097890e-04
1.33519986e-04 2.08074020e-05 6.84403619e-06 9.62519982e-07]
[0.00000000e+00 4.11843746e-07 1.70573540e-06 1.89144061e-05
9.53995572e-05 3.40248441e-04 9.65245985e-04 1.82390263e-03
3.48478116e-03 5.25478128e-03 5.93900413e-03 5.65968730e-03
5.21747625e-03 7.49875761e-03 1.04929185e-02 4.18284840e-03
3.93560267e-03 5.29003124e-03 3.65807389e-03 1.64064559e-03
1.31732771e-03 1.47755902e-03 2.95416372e-03 1.14348784e-03
1.54278291e-04 2.32616725e-05 9.42778161e-06 9.53768412e-07]
[0.00000000e+00 2.16919960e-07 1.87684611e-06 1.66823203e-05
1.01336275e-04 4.06141739e-04 1.18494164e-03 2.19404352e-03
3.27486638e-03 4.75211257e-03 5.14966150e-03 4.32160639e-03
4.21926094e-03 7.00699768e-03 7.45584175e-03 3.98168226e-03
3.73621371e-03 7.86925558e-03 3.42493735e-03 1.56305660e-03
1.19955567e-03 1.38890232e-03 1.58180372e-03 6.79229281e-04
1.82927506e-04 3.43768580e-05 1.13170774e-05 4.94179840e-07]
[0.00000000e+00 6.10297632e-07 3.26550580e-06 1.95400759e-05
9.01600951e-05 4.88849478e-04 1.47498418e-03 2.66141597e-03
3.77390489e-03 5.53884978e-03 5.41101511e-03 3.94245123e-03
5.29733345e-03 7.48583180e-03 6.05219504e-03 3.52508859e-03
3.66556277e-03 6.23357751e-03 3.15939551e-03 1.69404369e-03
1.46901170e-03 1.64698237e-03 1.05149525e-03 4.73186185e-04
1.56611158e-04 3.91799040e-05 1.31872568e-05 1.30982902e-06]
[0.00000000e+00 6.05007699e-07 4.84738322e-06 2.99204713e-05
8.87986967e-05 5.13587389e-04 1.38498319e-03 2.74800289e-03
3.93203063e-03 3.79730930e-03 4.13275574e-03 4.36567026e-03
5.58477857e-03 8.46334421e-03 6.27433648e-03 3.48133876e-03
3.32536773e-03 3.28254666e-03 2.90574178e-03 2.10027036e-03
1.57491111e-03 1.02558368e-03 8.68113066e-04 3.60603706e-04
1.59144930e-04 5.44194337e-05 1.62781425e-05 1.92088965e-06]
[0.00000000e+00 5.96488214e-07 4.29558987e-06 3.78808400e-05
1.37500543e-04 5.69404636e-04 1.16842596e-03 2.80124538e-03
2.70493826e-03 3.88268007e-03 4.70424136e-03 5.94736451e-03
5.58264798e-03 6.36820020e-03 4.04944861e-03 2.72819045e-03
1.86943337e-03 2.81590692e-03 2.10194358e-03 1.79660488e-03
1.47575256e-03 1.00342428e-03 5.82998811e-04 3.74202953e-04
2.15010748e-04 6.82355387e-05 1.02034011e-05 2.25183695e-06]
[1.59132931e-07 2.18323892e-07 7.47193888e-06 5.18935272e-05
1.85496919e-04 6.16417534e-04 1.58202754e-03 2.53209913e-03
2.88458605e-03 4.06909014e-03 5.89155009e-03 5.52259323e-03
4.15017262e-03 4.08216618e-03 2.57869744e-03 1.76068182e-03
1.43484425e-03 2.32732329e-03 2.56368458e-03 1.85669314e-03
1.54820775e-03 9.51704639e-04 6.31848854e-04 4.06238699e-04
2.41776275e-04 5.23712600e-05 6.89421579e-06 8.08261300e-07]
[0.00000000e+00 1.03306784e-06 9.46506566e-06 6.57920582e-05
2.34014989e-04 6.01772108e-04 1.73833855e-03 2.68418862e-03
3.75626370e-03 4.83302033e-03 7.45449755e-03 5.75898868e-03
3.49346649e-03 2.44668942e-03 1.91117278e-03 1.41691918e-03
1.32749950e-03 2.13568490e-03 2.97290074e-03 2.51560388e-03
1.62697986e-03 1.04970472e-03 6.96288203e-04 4.37929548e-04
1.74622541e-04 3.37683100e-05 3.70699528e-06 5.81941041e-07]
[0.00000000e+00 1.71947892e-07 5.75648201e-06 6.85173791e-05
2.20129752e-04 5.74939269e-04 1.37530939e-03 3.37473529e-03
4.90837475e-03 5.50406058e-03 4.23855151e-03 3.53108567e-03
2.80768522e-03 2.33336124e-03 2.01399045e-03 1.72195302e-03
1.42113863e-03 1.86763027e-03 2.06731210e-03 1.60929471e-03
1.13822772e-03 9.22022211e-04 4.70347915e-04 2.89641774e-04
9.76731759e-05 1.92482818e-05 3.69647281e-06 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 6.00705070e-06 5.17819816e-05
1.84264872e-04 3.95112726e-04 1.22321853e-03 3.06914776e-03
5.06122048e-03 4.31827452e-03 2.66928013e-03 1.91927171e-03
1.76701474e-03 1.35817970e-03 1.43762306e-03 1.27971918e-03
1.25435638e-03 1.62406791e-03 1.43814035e-03 1.21872754e-03
9.54961609e-04 7.25464410e-04 3.45333913e-04 1.52563227e-04
4.62422300e-05 9.08798237e-06 1.98181877e-06 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 3.12952503e-06 3.26032250e-05
1.14078995e-04 2.76311041e-04 6.92257517e-04 1.46932728e-03
3.04322467e-03 3.27487262e-03 2.88669258e-03 2.27432824e-03
1.80038034e-03 1.70994393e-03 1.72785874e-03 1.57696846e-03
1.28938166e-03 1.10501128e-03 8.82434545e-04 6.77015983e-04
4.29709044e-04 2.96546891e-04 1.57349635e-04 7.06803303e-05
2.25848051e-05 5.69141949e-06 1.66140369e-06 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 1.28287131e-06 1.45424881e-05
6.70197016e-05 1.35259890e-04 3.65164942e-04 8.35628784e-04
1.50173191e-03 2.11570970e-03 3.29011742e-03 3.77187760e-03
4.77697503e-03 4.88625405e-03 3.65409141e-03 2.66413359e-03
1.56741601e-03 1.02426444e-03 6.69553733e-04 4.48176217e-04
2.38302486e-04 1.40050516e-04 7.35784041e-05 3.18077961e-05
1.30254366e-05 3.56116459e-06 2.70688199e-06 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 1.15015557e-06 1.02485482e-05
4.05145856e-05 9.94495968e-05 2.30671794e-04 3.75498173e-04
6.29952578e-04 8.21734290e-04 1.15695527e-03 1.70445467e-03
1.61932776e-03 1.60365202e-03 9.91779736e-04 8.23030897e-04
5.62179926e-04 4.42830420e-04 3.35572641e-04 2.35001177e-04
1.39734593e-04 8.51518422e-05 3.99410193e-05 2.06783168e-05
5.86447323e-06 1.70492198e-06 3.93087896e-07 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 1.77498371e-07 1.14701854e-06
1.83187927e-05 5.60170058e-05 1.20623352e-04 2.93300448e-04
5.24931746e-04 6.79296388e-04 6.63063438e-04 1.19144118e-03
1.12532491e-03 7.92959601e-04 6.38070863e-04 5.14056128e-04
4.39221589e-04 4.62898228e-04 3.15400377e-04 2.13560932e-04
1.09951416e-04 5.41428607e-05 1.93086868e-05 5.12521408e-06
2.46171028e-06 3.92717996e-07 2.58069073e-07 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 0.00000000e+00 2.55546801e-07
3.26141321e-06 1.05406017e-05 1.32237364e-05 2.38815606e-05
6.80552863e-05 1.13438182e-04 1.67493434e-04 1.76727822e-04
3.12118301e-04 3.03342222e-04 1.61497128e-04 1.66683392e-04
1.91547770e-04 1.22019359e-04 5.94146070e-05 4.02129761e-05
2.64222928e-05 1.09962790e-05 3.13662823e-06 8.33629926e-07
4.40591561e-07 0.00000000e+00 0.00000000e+00 0.00000000e+00]
[0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00
0.00000000e+00 0.00000000e+00 4.68704103e-07 5.75936011e-07
1.01689957e-06 2.77302117e-06 2.74288558e-06 2.95320659e-06
3.54539267e-06 4.77772329e-06 5.01587705e-06 4.60820386e-06
4.29632580e-06 4.03611814e-06 2.32871382e-06 2.30845495e-06
1.01360802e-06 1.44524809e-06 7.79101538e-07 1.37828141e-07
0.00000000e+00 0.00000000e+00 0.00000000e+00 0.00000000e+00]]
还没怎么调整参数，随机森林就在训练集上得到了不错的分数0.97。当然还有提升空间，这篇文章姑且不讨论这个。特征重要性图片保存在当前目录下，文件名为 mnist_feature_importance.jpg。和书中的示例还是略有不同，可以通过调整参数实现：
res.jpg
差不多就是这样了。如果只想练习画图的话，可以把文中的结果复制过去，然后写个循环将字符列表处理成np.array即可，还是比较简单的。最后再留一个小tips，科学计数法文本转浮点型可以直接使用
float()方法:
>>> float('4.29632580e-06')
>>> 4.2963258e-06
