This paper provides an interpretation of Wittgenstein’s remarks in On Certainty.  On Certainty was intended as a response to the sceptic.  Despite this, it seems that Wittgenstein’s remarks, in particular the ones referencing the “groundlessness” of our epistemic practices, imply a kind of relativism.  One of the interpretive challenges of this text is to figure out exactly how his view succeeds at being anti-sceptical despite this groundlessness.  I argue that rational incommensurability on its own does not imply relativism.  I critique a contextualist interpretation of On Certainty by Michael Williams.  Williams argues that our epistemic practices are not system dependent, and that this provides a reason for Wittgenstein not being a relativist.  I will argue that for Wittgenstein, our epistemic practices are system-dependent, but that we can avoid relativism by demonstrating that the charge of relativism falls outside of the bounds of what can be intelligibly thought.