<p><span style="font-size:16px"/><span style="font-size:15px">本文分享2020年相关顶会对于DeepGNN Over-Smoothing的解法，共汇报五篇顶会论文，汇报完整版</span><span style="font-size:15px">ppt</span><span style="font-size:15px">可通过关注公众号【AI机器学习与知识图谱】后回复关键词：</span><span style="font-size:15px"><strong>Over-Smoothing</strong></span><span style="font-size:15px"> 来获得，供学习者使用！</span></p><p>
</p><p><span style="font-size:20px"><strong>Motivation</strong></span></p><p>
</p><p><span style="font-size:16px"><span>在计算机视觉中，模型</span><span>CNN</span><span>随着其层次加深可以学习到更深层次的特征信息，叠加</span><span>64</span><span>层或</span><span>128</span><span>层是十分正常的现象，且能较浅层取得更优的效果；</span></span></p><p>
</p><p><span style="font-size:16px"><span>图卷积神经网络</span><span>GCNs</span><span>是一种针对图结构数据的深度学习方法，但目前大多数的</span><span>GCN</span><span>模型都是浅层的，如</span><span>GCN</span><span>，</span><span>GAT</span><span>模型都是在</span><span>2</span><span>层时取得最优效果，随着加深模型效果就会大幅度下降；</span></span></p><p>
</p><p><span style="font-size:16px"><span>GCN</span><span>随着模型层次加深会出现</span><span>Over-Smoothing</span><span>问题，</span><span>Over-Smoothing</span><span>既相邻的节点随着网络变深就会越来越相似，最后学习到的</span><span>nodeembedding</span><span>便无法区分，模型效果下降。</span></span></p><p>
</p><p><span style="font-size:16px"><span>为什么要将</span><span>GNN</span><span>做深，</span><span>DeeperGNN</span><span>适用于解决什么问题：</span></span></p><p><span style="font-size:16px"><span>
</span></span></p><p><span style="font-size:16px"><span>（</span><span>1</span><span>）少标签半监督节点分类</span></span></p><p><span style="font-size:16px"><span>（</span><span>2</span><span>）少特征半监督节点分类</span></span></p><p><span style="font-size:16px">
</span></p><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-47abc128677fa9bc.jpeg" img-data="{"format":"jpeg","size":4941,"height":92,"width":414}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><p>
</p><p><span style="font-size:17px"><span>1</span><span>、邻接矩阵</span><span>A</span><span>：归一化方式</span><span>P ̃</span><span>，采样等；</span></span></p><p><span style="font-size:17px"><span>2</span><span>、节点特征</span><span>H</span><span>：</span><span>LocalInformation, Global Information</span><span>如何选择；</span></span></p><p><span style="font-size:17px"><span>3</span><span>、参数</span><span>W</span><span>：</span><span>Transformation</span><span>添加的位置（相对于</span><span>Propagation</span><span>）等；</span></span></p><p><span style="font-size:17px"><span style="font-size:17px">4</span><span style="font-size:17px">、损失函数</span><span style="font-size:17px">Loss/Normalization</span><span style="font-size:17px">：通过</span><span style="font-size:17px">loss</span><span style="font-size:17px">函数控制模型对于</span><span style="font-size:17px">Node Embedding</span><span style="font-size:17px">学习</span></span></p><p><span style="font-size:17px"><span style="font-size:17px">
</span></span></p><p><span style="font-size:17px">接下来，我们将通过上述</span><span style="font-size:17px">一个公式</span><span style="font-size:17px">，结合以上</span><span style="font-size:17px">四点观点</span><span style="font-size:17px">，分享下面</span><span style="font-size:17px">五篇论文</span></p><p>
</p><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-3640a831ff5c9e61.jpeg" img-data="{"format":"jpeg","size":43330,"height":240,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><p><strong style="font-size: 20px;">
</strong></p><p><strong style="font-size: 20px;">Papers</strong></p><p>
</p><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-c564ed904447d5ef.jpeg" img-data="{"format":"jpeg","size":61781,"height":620,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-c282c604902519a3.jpeg" img-data="{"format":"jpeg","size":51672,"height":610,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-4b68baee76250060.jpeg" img-data="{"format":"jpeg","size":54624,"height":594,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-fbcfe850a7f988c0.jpeg" img-data="{"format":"jpeg","size":90310,"height":597,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-742698e96353d5e8.jpeg" img-data="{"format":"jpeg","size":89021,"height":631,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-674f8ff707e8bc00.jpeg" img-data="{"format":"jpeg","size":87297,"height":603,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-0acc9e103b887039.jpeg" img-data="{"format":"jpeg","size":89541,"height":628,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-7b7a1139fa5efc5e.jpeg" img-data="{"format":"jpeg","size":88058,"height":621,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-b8b06c0cb0996abc.jpeg" img-data="{"format":"jpeg","size":50470,"height":554,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-4f387a188d3e3edd.jpeg" img-data="{"format":"jpeg","size":58781,"height":605,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-64973d075d4aabd5.jpeg" img-data="{"format":"jpeg","size":63448,"height":579,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-93fdef83f8a45d12.jpeg" img-data="{"format":"jpeg","size":82517,"height":623,"width":1080}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><p>
</p><p><span style="font-size:15px">五篇顶会论文，汇报完整版</span><span style="font-size:15px">ppt</span><span style="font-size:15px">可通过关注公众号【AI机器学习与知识图谱】后回复关键词：</span><span style="font-size:15px"><strong>Over-Smoothing</strong></span><span style="font-size:15px"> 来获得</span>，<span style="font-size:15px">之前详细介绍过GCNII<span style="font-size:15px">这篇论文</span>，后续会详细介绍其他几篇论文。有用就点个再看呗</span></p><div class="image-package"><img src="https://upload-images.jianshu.io/upload_images/26011021-a5f7ef6fe8c88877.jpeg" img-data="{"format":"jpeg","size":1759,"height":64,"width":64}" class="uploaded-img" style="min-height:200px;min-width:200px;" width="auto" height="auto"/>
</div><p/><p>
</p><p><span style="font-size:18px"><strong>往期精彩</strong></span></p><p>
</p><p><span style="font-size:14px">干货 | Attention注意力机制超全综述</span></p><p>【面经系列】八位硕博大佬的字节之旅</p><p>【机器学习系列】机器学习中的两大学派
</p><p/><p><span style="font-size:14px">机器学习中优化相关理论知识简述</span></p><p><span style="font-size:14px">Transformer 模型的 PyTorch 实现</span></p><p><span style="font-size:14px"/></p><p><span style="font-size:14px">干货|一文弄懂机器学习中偏差和方差</span></p><p><span style="font-size:14px">Transformer模型细节理解及Tensorflow实现</span>
</p><p><span style="font-size:14px"/></p><p><span style="font-size:14px">机器学习算法篇：最大似然估计证明最小二乘法合理性</span></p>