Gents
I have read quite a lot about the PCR problem induced by the use of Hauppauge MPEG encoders, but there does not seem to be a single knowledge-base about the issue. Given that it is holding up DATV-Express at present, and that MK808-based Digilite encoders also suffer, I thought it might be useful to discuss it here. My initial questions:
- Does it only affect Linux-based implementations, or are Windows-based solutions also affected?
- Are there any Hauppauge encoders that do not exhibit the problem?
- Is the code that is being used to attempt to correct the problem available for inspection?
- Is there anything that a fresh (but ageing!) mind can contribute to the solution (such as analysis of TS files?)
- Would it be useful to build a database of Receivers that tolerate/don't tolerate the problem?
Happy to try to help
Dave
G8GKQ
The Hauppauge PCR Problem
Forum rules
This forum is run by the BATC (British Amateur Television Club), it is service made freely available to all interested parties, please do not abuse this privilege.
Thank you
This forum is run by the BATC (British Amateur Television Club), it is service made freely available to all interested parties, please do not abuse this privilege.
Thank you
Re: The Hauppauge PCR Problem
Dave,
One problem is that the Linux drivers have been reverse engineered. My understanding is
that they took the Hauppauge Windows code then sniffed the I2C communications to reverse
engineer the Linux driver. Consequently not everything is supported.
The Conexiant chip they are using does support a transport stream but the Linux driver
appears only to support program streams. When I send it a command to select a transport
stream it comes back with an error code saying it is not supported.
Therefore I am having to use the program SCR clock to synthesise a PCR. The problem is
the SCR clock seems to have large jumps in it which means I can't use it directly. I am currently
using a similar technique to the one used by Brian G4EWJ. I keep a running PCR clock that is updated
each time a transport packet is sent. The running clock is then 'conditioned' by the SCR updates from
the Hauppauge (a bit like a PLL). Unfortunately the jumps in the SCR are large and fairly unpredictable.
However over long periods of time they do smooth out.
From speaking to Brian G4EWJ it seems the SCR clock he is using under Windows is much better behaved
than the Linux one which seems very strange to me.
One solution is to use the approach that Jean-Pierre F6DZP is using and that is to use software video
encoders. His work is on the Windows platform. Similar encoders are available for Linux but it is
a patent minefield.
At the end of the day the Hauppauge devices were not designed for broadcasting but for recording a
program stream to a disk file or DVD recorder.
P.C based DVB-S dongle receivers don't seem to mind about the inaccurate PCR, I suspect they simply use the
DTS and PTS timestamps and a lot of buffering to get around the problem. STBs on the other hand have a very
small amount of buffering and use the PCR to manage the queues and if the PCR/DTS difference is too
large they overflow.
- Charles G4GUO
One problem is that the Linux drivers have been reverse engineered. My understanding is
that they took the Hauppauge Windows code then sniffed the I2C communications to reverse
engineer the Linux driver. Consequently not everything is supported.
The Conexiant chip they are using does support a transport stream but the Linux driver
appears only to support program streams. When I send it a command to select a transport
stream it comes back with an error code saying it is not supported.
Therefore I am having to use the program SCR clock to synthesise a PCR. The problem is
the SCR clock seems to have large jumps in it which means I can't use it directly. I am currently
using a similar technique to the one used by Brian G4EWJ. I keep a running PCR clock that is updated
each time a transport packet is sent. The running clock is then 'conditioned' by the SCR updates from
the Hauppauge (a bit like a PLL). Unfortunately the jumps in the SCR are large and fairly unpredictable.
However over long periods of time they do smooth out.
From speaking to Brian G4EWJ it seems the SCR clock he is using under Windows is much better behaved
than the Linux one which seems very strange to me.
One solution is to use the approach that Jean-Pierre F6DZP is using and that is to use software video
encoders. His work is on the Windows platform. Similar encoders are available for Linux but it is
a patent minefield.
At the end of the day the Hauppauge devices were not designed for broadcasting but for recording a
program stream to a disk file or DVD recorder.
P.C based DVB-S dongle receivers don't seem to mind about the inaccurate PCR, I suspect they simply use the
DTS and PTS timestamps and a lot of buffering to get around the problem. STBs on the other hand have a very
small amount of buffering and use the PCR to manage the queues and if the PCR/DTS difference is too
large they overflow.
- Charles G4GUO
Re: The Hauppauge PCR Problem
Charles,
When you say 'the PCR is conditioned by the SCR' what is that process exactly? The PCR is effectively a real time clock in the transport stream and once you've started it, it should only be affected by the number of bytes transmitted in the transport stream. The SCR in the program stream should not affect the PCR at all after the PCR has been given an initial value relative to the SCR. DigiLite subtracts the 'TS Delay' time in DL Config from the first SCR in the program stream to set the PCR, so the data in the first program stream frame arrives at the decoder 'TS Delay' before the decoder needs to display it.
The SCR in Windows is the same for several blocks of program stream. I think it just means that 'this data relates to the frame at xxx milliseconds' and it goes up by 40ms for data for the next frame.
DL uses the difference between the SCR and the PCR to pace the output and insert padding packets depending on that difference and the fullness of the USB output buffers.
I had a look at the 15 second program stream you sent me a while back and apart from the very last frame, the SCR increases with time as I would expect. I couldn't see any jumps. The SCR in Linux doesn't go up in 40ms increments, so I assume it's the time that that block of program stream data was produced. That shouldn't be a drawback.
What sort of jumps are you seeing in the program stream? Maybe the 15s file you sent me is too short for it to occur. Perhaps you could send me a much longer file to have a look at.
An alternate to the SCR is to use the PTS (presentation time stamp) or DTS (decode time stamp) in the program stream. In the Linux program stream file you sent, these go up in 40ms increments for every frame as you would expect, assuming they aren't jumping around as well.
Brian
When you say 'the PCR is conditioned by the SCR' what is that process exactly? The PCR is effectively a real time clock in the transport stream and once you've started it, it should only be affected by the number of bytes transmitted in the transport stream. The SCR in the program stream should not affect the PCR at all after the PCR has been given an initial value relative to the SCR. DigiLite subtracts the 'TS Delay' time in DL Config from the first SCR in the program stream to set the PCR, so the data in the first program stream frame arrives at the decoder 'TS Delay' before the decoder needs to display it.
The SCR in Windows is the same for several blocks of program stream. I think it just means that 'this data relates to the frame at xxx milliseconds' and it goes up by 40ms for data for the next frame.
DL uses the difference between the SCR and the PCR to pace the output and insert padding packets depending on that difference and the fullness of the USB output buffers.
I had a look at the 15 second program stream you sent me a while back and apart from the very last frame, the SCR increases with time as I would expect. I couldn't see any jumps. The SCR in Linux doesn't go up in 40ms increments, so I assume it's the time that that block of program stream data was produced. That shouldn't be a drawback.
What sort of jumps are you seeing in the program stream? Maybe the 15s file you sent me is too short for it to occur. Perhaps you could send me a much longer file to have a look at.
An alternate to the SCR is to use the PTS (presentation time stamp) or DTS (decode time stamp) in the program stream. In the Linux program stream file you sent, these go up in 40ms increments for every frame as you would expect, assuming they aren't jumping around as well.
Brian
Re: The Hauppauge PCR Problem
Hello Brian,
The PCR I am generating is running at a slightly different frequency than the SCR from the Hauppauge.
I suspect the clock on the DATV-Express board is off frequency as that is ultimately what is clocking the
PCR. That is why I need to modify it using the SCR as reference.
If I let my code free run and ignore the SCR the 27 MHz oscillator in my Tandberg receiver reports a PLL lock to
the received PCR but of course the PTS/DTS errors causes buffer over and under runs.
From memory the SCR increments in about 4ms intervals but occasionally the increment is 12 or more ms.
They are always multiples of 4ms. Maybe when it has nothing to send it simply sends nothing (including an SCR)
and then catches up later. I don't get any stuttering of the picture or sound so it does not look as if I am dropping
packets.
- Charles
The PCR I am generating is running at a slightly different frequency than the SCR from the Hauppauge.
I suspect the clock on the DATV-Express board is off frequency as that is ultimately what is clocking the
PCR. That is why I need to modify it using the SCR as reference.
If I let my code free run and ignore the SCR the 27 MHz oscillator in my Tandberg receiver reports a PLL lock to
the received PCR but of course the PTS/DTS errors causes buffer over and under runs.
From memory the SCR increments in about 4ms intervals but occasionally the increment is 12 or more ms.
They are always multiples of 4ms. Maybe when it has nothing to send it simply sends nothing (including an SCR)
and then catches up later. I don't get any stuttering of the picture or sound so it does not look as if I am dropping
packets.
- Charles
Re: The Hauppauge PCR Problem
It'll always be a potential problem when the transmission side reference oscillator differs from the video capture side reference oscillator. If there's a 1kHz difference between the nominal 27MHz oscillators on which the PCR and other timestamps are based, then you would expect a one second difference after 27,000 seconds - 7.5 hours. DigiLite would re-sync at this point.
I don't think it matters about the SCR increments, as long the SCR tracks the PTS/DTS. It's probably more useful having it that way rather than the 40ms jumps on Windows. What do you actually do with the SCR when you get it?
I don't normally use Linux, but I've had to set up a CentOS system for work, so I can try some long video captures and see what the analysers make of it. At least I can when I work out what to do. I'm still trying to work out whether I have v4l-utils. YUM doesn't find it on the internet when I ask it.
Which PS analyser do you use? I use this one:
http://www.codeproject.com/Articles/667 ... ream-files
Brian
I don't think it matters about the SCR increments, as long the SCR tracks the PTS/DTS. It's probably more useful having it that way rather than the 40ms jumps on Windows. What do you actually do with the SCR when you get it?
I don't normally use Linux, but I've had to set up a CentOS system for work, so I can try some long video captures and see what the analysers make of it. At least I can when I work out what to do. I'm still trying to work out whether I have v4l-utils. YUM doesn't find it on the internet when I ask it.
Which PS analyser do you use? I use this one:
http://www.codeproject.com/Articles/667 ... ream-files
Brian
Re: The Hauppauge PCR Problem
I have changed my code to use the SCR to determine whether extra NULLs have to be inserted.
This approach is working better but the DATV-Express symbol clock seems to be running at a
different frequency to the 27 MHz in the PVR. (of course it maybe a maths error on my part).
I have a number of analyser programs but none of them specifically look at the program stream.
- Charles
This approach is working better but the DATV-Express symbol clock seems to be running at a
different frequency to the 27 MHz in the PVR. (of course it maybe a maths error on my part).
I have a number of analyser programs but none of them specifically look at the program stream.
- Charles
Re: The Hauppauge PCR Problem
I think the symbol rate clock is an important one to get right. I suppose on DATV-Express you don't have a pin with the SR clock on it to measure the frequency, because you're outputting analogue data directly.
Brian
Brian
Re: The Hauppauge PCR Problem
Hello Brian,
The SR clock is generated internally from the 48 MHz USB2 clock on the board.
The 48 MHz is multiplied by the FPGA's PLL and then divided to produce a 400 MHz
clock. The 400 MHz is used to clock a 64 bit counter. On each clock cycle the
counter is increment by an amount dependent on the wanted output frequency.
The MSB of the 64 bit counter is used as the SR clock. There will be a slight
inaccuracy and some jitter on the clock but it is hardly noticeable. I can bring the
SR clock out to an FPGA I/O pin if needed. The USB2 clock is based on a cheap
24 MHz crystal.
It is also possible to use the 20 MHz oscillator that the ARDF6755 chip uses as a reference
but that requires adding a zero ohm resistor to the board.
There is also the possibility of adding an Si570 clock generator chip to the board but we didn't want
to do that as those chips are quite expensive so we just left the PCB footprint for it.
Those chips work in a similar fashion to my FPGA code except they multiply up into the GHz range
before doing the division. Internally the FPGA is limited to around 400 MHz.
On my home system I eventually plan to remove the 20 Mhz reference on the board and feed it with
a 20 MHz GPS locked reference so I can derive both the RF frequency and the SR from GPS time.
Then no one will be able to accuse me of being off frequency
Sorry I can't help you with your v4l-utils problem as I use Ubuntu and the package manager makes it
easy to install all this stuff! I have used Fedora in the past which uses RPM and with that I seem to
remember it will install all the dependencies for you. When ever I have a problem like that I Google it.
There is bound to be someone else that has had exactly the same problem.
- Charles
The SR clock is generated internally from the 48 MHz USB2 clock on the board.
The 48 MHz is multiplied by the FPGA's PLL and then divided to produce a 400 MHz
clock. The 400 MHz is used to clock a 64 bit counter. On each clock cycle the
counter is increment by an amount dependent on the wanted output frequency.
The MSB of the 64 bit counter is used as the SR clock. There will be a slight
inaccuracy and some jitter on the clock but it is hardly noticeable. I can bring the
SR clock out to an FPGA I/O pin if needed. The USB2 clock is based on a cheap
24 MHz crystal.
It is also possible to use the 20 MHz oscillator that the ARDF6755 chip uses as a reference
but that requires adding a zero ohm resistor to the board.
There is also the possibility of adding an Si570 clock generator chip to the board but we didn't want
to do that as those chips are quite expensive so we just left the PCB footprint for it.
Those chips work in a similar fashion to my FPGA code except they multiply up into the GHz range
before doing the division. Internally the FPGA is limited to around 400 MHz.
On my home system I eventually plan to remove the 20 Mhz reference on the board and feed it with
a 20 MHz GPS locked reference so I can derive both the RF frequency and the SR from GPS time.
Then no one will be able to accuse me of being off frequency
Sorry I can't help you with your v4l-utils problem as I use Ubuntu and the package manager makes it
easy to install all this stuff! I have used Fedora in the past which uses RPM and with that I seem to
remember it will install all the dependencies for you. When ever I have a problem like that I Google it.
There is bound to be someone else that has had exactly the same problem.
- Charles
Re: The Hauppauge PCR Problem
Looks like I have found the problem.
The PVR's CBR mode is not very 'constant' and was momentarily exceeding the
capacity of the channel and hence the PCR clock was overtaking the SCR / PTS /DTS.
By reducing the calculated PVR bitrate by 5/8 the timing is now looking much better.
For the first time I have been able to receive good audio on my Tandberg receiver
with its lip sync mode in operation. There is still some timing jitter but it is in the
ns range rather than the ms range.
- Charles G4GUO
The PVR's CBR mode is not very 'constant' and was momentarily exceeding the
capacity of the channel and hence the PCR clock was overtaking the SCR / PTS /DTS.
By reducing the calculated PVR bitrate by 5/8 the timing is now looking much better.
For the first time I have been able to receive good audio on my Tandberg receiver
with its lip sync mode in operation. There is still some timing jitter but it is in the
ns range rather than the ms range.
- Charles G4GUO