Lip sync.
Posted: Mon May 22, 2023 2:15 pm
A week or so ago on the QO-100 chat, someone (I think it was that station who's a really good singer) asked if it was possible to change lip sync in real time. You shouldn't really be able to take liberties like changing the timings in a transport stream, but it does seem to be practical.
I've written a test program which accepts a UDP TS, modifies the presentation time stamps (PTS) in the audio stream and sends the TS onwards. The time difference applied to the audio PTS's can be changed on the fly from the keyboard.
The video stream also has PTS's, which are not changed. The PTS's are relative to the program clock reference (PCR) which is the master clock embedded in the TS. VLC uses the PCR and the audio and video PTS's to know when to render them.
The question is, is the program practical to use in this form? I'm not familiar with OBS / DATVEasy / Pluto setups. Is there a point in the system where a UDP TS can be directed to an IP address? I believe that different setups can send IQ data or TS to the Pluto, so only the latter would be practical.
VLC copes quite well. Whenever a change is made to the audio timing, it keeps the video going and the audio re-appears in a second or two. MPV is less elegant and there can be a second or two of video stop,slow,quick.
Brian
.
I've written a test program which accepts a UDP TS, modifies the presentation time stamps (PTS) in the audio stream and sends the TS onwards. The time difference applied to the audio PTS's can be changed on the fly from the keyboard.
The video stream also has PTS's, which are not changed. The PTS's are relative to the program clock reference (PCR) which is the master clock embedded in the TS. VLC uses the PCR and the audio and video PTS's to know when to render them.
The question is, is the program practical to use in this form? I'm not familiar with OBS / DATVEasy / Pluto setups. Is there a point in the system where a UDP TS can be directed to an IP address? I believe that different setups can send IQ data or TS to the Pluto, so only the latter would be practical.
VLC copes quite well. Whenever a change is made to the audio timing, it keeps the video going and the audio re-appears in a second or two. MPV is less elegant and there can be a second or two of video stop,slow,quick.
Brian
.