【Sentence-bert】
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
论文: https://arxiv.org/abs/1908.10084
代码: https://github.com/UKPLab/sentence-transformers
【Bert-flow】
On the Sentence Embeddings from Pre-trained Language Models
论文: https://arxiv.org/abs/2011.05864
代码: https://github.com/bohanli/BERT-flow
动机: 论文作者认为BERT生成的句向量具有的语义相似度信息无法通过简单的余弦相似度体现出来，论文通过一个基于流(flow)的生成模型训练一个标准的高斯分布，对 BERT句向量空间中的点进行一一映射，再在标准高斯空间中进行相似度计算
【Bert-whitening】
Whitening Sentence Representations for Better Semantics and Faster Retrieval
论文: https://arxiv.org/abs/2103.15316
代码: https://github.com/bojone/BERT-whitening
作者认为Bert-flow模型的关键不在于flow模型，基于Bert模型输出的[CLS]向量在文本语义计算任务中无法取得好的效果的原因可能是句向量所属的坐标系并非标准正交基的推论，通过一个简单的线性变换实现了和Bert-flow相近的效果。
参考:
细说Bert-whitening的原理
https://zhuanlan.zhihu.com/p/380874824
你可能不需要BERT-flow：一个线性变换媲美BERT-flow
https://kexue.fm/archives/8069
【ConSERT】
ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer
论文: https://arxiv.org/pdf/2105.11741.pdf
代码: https://github.com/yym6472/ConSERT
参考:
论文阅读“ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer”
//www.greatytc.com/p/de79f4e166da
【SimCSE】
SimCSE: Simple Contrastive Learning of Sentence Embeddings
论文: https://arxiv.org/pdf/2104.08821.pdf
代码: https://github.com/princeton-nlp/SimCSE
参考:
【论文复现】SimCSE对比学习: 文本增广是什么牛马，我只需要简单Dropout两下
https://blog.csdn.net/weixin_45839693/article/details/116302914
【Trans-Encoder】
TRANS-ENCODER: UNSUPERVISED SENTENCE-PAIR MODELLING THROUGH SELF- AND MUTUAL-DISTILLATIONS
论文://arxiv.org/pdf/2109.13059v3.pdf
代码 ：https://github.com/amzn/trans-encoder
参考：
文本匹配——【ICLR 2022】Trans-Encoder
https://cloud.tencent.com/developer/article/2041362
TRANS-ENCODER：自蒸馏和互蒸馏的无监督句对模型
https://zhuanlan.zhihu.com/p/437729611
【Poly-encoders】
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
论文: https://arxiv.org/pdf/1905.01969.pdf
参考:
论文笔记 | Poly-encoders
https://zhuanlan.zhihu.com/p/380867813
【PairSupCon】
Pairwise Supervised Contrastive Learning of Sentence Representations
论文: https://arxiv.org/abs/2109.05424
代码: https://github.com/amazon-research/sentence-representations/PairSupCon
参考:
Pairwise Supervised Contrastive Learning of Sentence Representations
https://zhuanlan.zhihu.com/p/421196130
