During the last 15 years, many efforts have been directed towards the development of digital detectors for X-ray imaging. Direct conversion stabilized amorphous Selenium (a-Se)-based X-ray image detection with an active matrix array is one of the most widely used approaches which can provide excellent X-ray images, and is commercially available for mammography and general radiography. However, the X-ray sensitivity of a-Se detectors used in these systems changes as a result of previous X-ray exposures. This change in sensitivity which creates ghosting is recoverable by resting the detector for several hours. In this work, the physics of ghosting and its recovery mechanisms in multilayer a-Se detectors are experimentally and theoretically investigated.  A numerical model is developed to study the time and exposure dependent X-ray sensitivity of multilayer a-Se X-ray imaging detectors on repeated X-ray exposures. This model considers accumulated trapped charges and their effects (trap filling, recombination, electric field profile, electric field dependent electron-hole pair creation energy), the carrier transport in the blocking layers, X-ray induced meta-stable deep trap center generations, and the effects of charge injection. The time dependent carrier detrapping and structural relaxation (recovery of meta-stable trap centers) are also considered. The continuity equations for both holes and electrons, trapping rate equations, and the Poisson's equation across the photoconductor for a step X-ray exposure are simultaneously solved by the Backward Euler finite difference method. It is found that the sensitivity in a rested sample is recovered mainly by the carrier detrapping and the recombination of the injected carriers with the existing trapped carriers. The sensitivity is expected to recover almost fully by resting the sample longer than the recovery time constant of the meta-stable trap centers. The theoretical model agrees well with the experimental results