Knowledge graphs represent information via entities and their relationships. This form of relational knowledge representation has a long history in logic and artificial intelligence. More recently, it has also been the basis of the Semantic Web to create a “web of data” that is readable by machines.

In this project, we explore embedding methods for learning from knowledge graphs. These methods combine multiple advantages:

- State-oft-the-art results for complex tasks like link prediction and entity resolution
- Scalable to knowledge graphs with millions of entities and billions of facts
- Provide access to relational information for deep learning methods

Relational embedding methods are therefore not only interesting from a knowledge graph perspective, but can also be an important step towards relational reasoning in modern AI systems.

One of the first knowledge graph embedding methods is
RESCAL which computes a three-way factorization
of an adjacency tensor that represents the knowledge graph. Alternatively, it can
be interpreted as a *compositional model*, where pairs of entities are
represented via the tensor product of their embeddings. RESCAL is a very
powerful model that can capture complex relational patterns over multiple hops
in a graph. Figure 1 shows an illustration of the factorization model.

However, RESCAL can be hard to scale to very large knowledge-graphs because its
has a quadratic runtime and memory complexity with regard to the embedding
dimension. For this reason, we also explored compositonal operators that are
more efficient than the tensor product. One outcome of this research direction
are Holographic embeddings of knowledge graphs
(HolE) which use *circular convolution* as
the compositional operator. Due to this, it can also be interpreted as a
multi-relational *associative memory*, where the relation embeddings store which
latent pairs (or pairs of prototypes) are true for a certain relation type. HolE
retains the excellent performance of RESCAL while being far more scalable, as it
only has a linear dependency on the embedding dimension.

An extensive review of statistical machine learning for knowledge graphs is available here. Moreover, a python library that implements many state-of-the-art knowledge graph embeddings methods (e.g., RESCAL, HolE, TransE, ER-MLP) is available on Github.