Abstract

In order to understand a coherent text, humans infer semantic or logical relations between textual units. For example, in ``I am hungry. I did not have lunch today.'' the reader infers a ``causality'' relation even if it is not explicitly stated via a term such as ``because''. The linguistic device used to link textual units without the use of such explicit terms is called an ``implicit discourse relation''. Recognising implicit relations automatically is a much more challenging task than in the explicit case. Previous methods to address this problem relied heavily on conventional machine learning techniques such as CRFs and SVMs which require many hand-engineered features.

In this thesis, we investigate the use of various convolutional neural networks and sequence-to-sequence models to address the automatic recognition of implicit discourse relations. We demonstrate how our sequence-to-sequence model can achieve state-of-the-art performance with the use of an attention mechanism. In addition, we investigate the automatic representation learning of discourse relations in high capacity neural networks and show that for certain discourse relations such a network does learn discourse relations in only a few neurons.