This thesis brings two disparate fields of research together; the fields of artificial neural networks and pseudorandom number generation. In it, we answer variations on the following question: can recurrent neural networks generate pseudorandom numbers? In doing so, we provide a new construction of an $n$-dimensional neural network that has period $2^n$, for all $n$. We also provide a technique for constructing neural networks based on the theory of shift register sequences. The randomness capabilities of these networks is then measured via the theoretical notion of computational indistinguishability and a battery of statistical tests. In particular, we show that neural networks cannot be pseudorandom number generators according to the theoretical definition of computational indistinguishability. We contrast this result by providing some neural networks that pass all of the tests in the SmallCrush battery of tests in the TestU01 testing suite.