Knowledge Graph Completion (KGC) methods serve as a valuable tool to identify missing information in a knowledge graph (KG), such as predicting a missing relation between two entities or inferring properties about an entity which does not currently exist in the KG; the results of such KGC methods can be used to enable knowledge-driven down stream tasks. To further enhance the capabilities of KGC methods and to help understand their predictions, context can play an important role -- however, our understanding and use of "context" as it relates to KGC methods has been limited in existing works, often relying on vague or ad-hoc definitions in "context-aware" KGC methods. In this thesis, we explore how to incorporate context into KGC methods from the perspectives of three use case domains (cooking recipes, event forecasting, and tabular data management) and KGC subtasks through the development of novel KGC methods. Additionally, we investigate how we can capture "context" as it relates to KGC methods in a more explicit manner through the development of an ontology model. Through this thesis' contributions, we demonstrate how context can be incorporated through a variety of different methods and tasks to achieve greater performance in difficult experimental settings, as well as how such context can be represented in our model.