Abstract

The use of multimedia system have moved beyond the studio barriers and into the homes. As computers become more powerful, multimedia systems become more realizable on the PC. As these multimedia systems become more complicated, the need to provide complex integration systems and synchronization arises. To develop a multimedia system, one must ensure that a synchronization approach is in place to solve the timing issues related to the media types involved. Temporal information in multimedia systems must be maintained such that no loss of coherency is endured. The system must ensure that no matter how much processing is performed on the signals, the output has to maintain the temporal integrity of the signals as they were when they were inputted. This thesis develops a multimedia system that processes two media streams. Audio and video streams are fed to the system. The system produces an object segmented output, (silhouettes of the object) along with the recognized speech from the audio. The speech that is to be recognized by the system is spoken by the objects/speakers. The challenge lies in maintaining the synchronization and integrating the video and the recognized speech at the output. Note that the system is a stream based system by that the video and audio are continuously captured and processed. This thesis presents a solution to the problem of synchronization in the temporal domain and the overall integration of the multimedia system. The thesis presents a time-stamp approach to solve the synchronization problem between audio and video signals. This approach is adaptive to the cases where the video processing delay is larger or smaller than the audio processing delay. The contributions include the verification of using time-stamps in the synchronization process and that it is possible to synchronize heavily delayed signals. The system requires an integration process such that the audio and video signals are integrated with one another at the output