Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs

Haohai Sun, Shangyi Geng, Jialun Zhong, Han Hu, Kun He


Abstract
Temporal Knowledge Graph (TKG) reasoning has attracted increasing attention due to its enormous potential value, and the critical issue is how to model the complex temporal structure information effectively. Recent studies use the method of encoding graph snapshots into hidden vector space and then performing heuristic deductions, which perform well on the task of entity prediction. However, these approaches cannot predict when an event will occur and have the following limitations: 1) there are many facts not related to the query that can confuse the model; 2) there exists information forgetting caused by long-term evolutionary processes. To this end, we propose a Graph Hawkes Transformer (GHT) for both TKG entity prediction and time prediction tasks in the future time. In GHT, there are two variants of Transformer, which capture the instantaneous structural information and temporal evolution information, respectively, and a new relational continuous-time encoding function to facilitate feature evolution with the Hawkes process. Extensive experiments on four public datasets demonstrate its superior performance, especially on long-term evolutionary tasks.
Anthology ID:
2022.emnlp-main.507
Volume:
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
7481–7493
Language:
URL:
https://aclanthology.org/2022.emnlp-main.507
DOI:
10.18653/v1/2022.emnlp-main.507
Bibkey:
Cite (ACL):
Haohai Sun, Shangyi Geng, Jialun Zhong, Han Hu, and Kun He. 2022. Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7481–7493, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Graph Hawkes Transformer for Extrapolated Reasoning on Temporal Knowledge Graphs (Sun et al., EMNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.emnlp-main.507.pdf
Software:
 2022.emnlp-main.507.software.zip
Dataset:
 2022.emnlp-main.507.dataset.zip