Sentence Representations via Gaussian Embedding

Shohei Yoda, Hayato Tsukagoshi, Ryohei Sasano, Koichi Takeda


Abstract
Recent progress in sentence embedding, which represents a sentence’s meaning as a point in a vector space, has achieved high performance on several tasks such as the semantic textual similarity (STS) task.However, a sentence representation cannot adequately express the diverse information that sentences contain: for example, such representations cannot naturally handle asymmetric relationships between sentences.This paper proposes GaussCSE, a Gaussian-distribution-based contrastive learning framework for sentence embedding that can handle asymmetric inter-sentential relations, as well as a similarity measure for identifying entailment relations.Our experiments show that GaussCSE achieves performance comparable to that of previous methods on natural language inference (NLI) tasks, and that it can estimate the direction of entailment relations, which is difficult with point representations.
Anthology ID:
2024.eacl-short.36
Volume:
Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
418–425
Language:
URL:
https://aclanthology.org/2024.eacl-short.36
DOI:
Bibkey:
Cite (ACL):
Shohei Yoda, Hayato Tsukagoshi, Ryohei Sasano, and Koichi Takeda. 2024. Sentence Representations via Gaussian Embedding. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 418–425, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
Sentence Representations via Gaussian Embedding (Yoda et al., EACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eacl-short.36.pdf
Video:
 https://aclanthology.org/2024.eacl-short.36.mp4