Saturday, July 11, 2020

VHDL Teletext Decoder Part 3: Data recovery, processing and page extraction

In Part 2, I designed a display controller which can display a 40 character by 25 line screen of text and graphics as used by Teletext (and Mode 7 on the BBC Micro). The display controller reads from a dual-port RAM, which means it can be used in any FPGA project where data display and basic graphics are required. In the case of a teletext decoder, the VBI data needs to be extracted from the composite video signal, error correction needs to be applied, and then the data relating to the currently selected page number needs to be loaded into the dual-port RAM so that it is displayed by the display controller.

RTL block diagram

The teletext decoder section of the design has been broken up into four modules:

  • TXT_DATA_RECOVERY: Detects the start of a packet (line) of teletext data and extracts the serial data
  • TXT_DESERIALISER: Converts the serial data to parallel
  • TXT_DATA_PROCESSOR: Performs error correction on the data and determines the page number and subcode of each packet received
  • TXT_MEMORY_CONTROLLER: Writes packets from the selected page number to the dual-port RAM

Parallel vs serial transmission schemes

Not to be confused with serial and parallel data, these terms refer to the order in which pages are transmitted in the VBI (Vertical Blanking Interval).

In a parallel mode scheme, a few lines from one page in a particular magazine are transmitted followed by a few lines from a page from another magazine and so on. This approach was favoured by UK broadcasters when teletext was broadcast here and is still used by TEEFAX. This scheme can be identified from the display of page numbers only from the current magazine in the rolling header when waiting for a page.

In a serial mode scheme, all packets transmitted belong to the page number and magazine of the last header packet transmitted. This approach is favoured by international broadcasters, and is identified by page numbers from every magazine being displayed in the rolling header.

More detailed information about these schemes is described in Annex B of ETSI EN 300 706.

TXT_DATA_RECOVERY

This module detects the start sequence of the packet then forwards the serial data onto the next module. The start of the packet is shown below:

To simplify the design and avoid the need for a high-speed ADC, the packet is simply digitally sampled, which means the module only has a single digital input. A high-speed comparator is the only active semiconductor required, although the user may have to adjust the trimmer pot RV1 when switching sources.

Composite video input and sampling circuit

The comparator converts the analogue signal into a digital signal suitable for the FPGA:

The data recovery module, along with the rest of the teletext decoder section, runs at 27.750MHz, four times the data bit rate. A SI5351 programmable clock source is used to generate this unusual frequency with great accuracy along with the unique video clock required by the VGA controller for each resolution.

There are four sampled bits per data bit (known as 4x oversampling), and the bit rate has a very tight tolerance so there is no need to worry about clock recovery, which simplifies the design a lot. A shift register is continuously fed with the digital data provided by the comparator and a packet is detected once the start sequence is detected in the shift register. One out of every four bits will be sampled during the inter-bit transition, so the comparison is performed using only three sampled bits in every data bit, then the data sent to the next module is recovered from the middle sampled bit of each subsequent data bit.

Start pattern in teletext waveform

The colour burst is not sampled, nor is the video line number counted. These are not necessary for decoding, and the only consequence is the decoder will be capable of receiving anything transmitted otherwise in accordance with the specification which may be rejected by a commercial decoder.

TXT_DESERIALISER

This module converts the recovered data from serial to parallel. The FRAME_VALID_OUT signal indicates the start and end of each packet received to the next module.

TXT_DATA_PROCESSOR

This is the most complex module in the teletext decoder.

Page number detection

The two main types of packet are page header and normal packets. There's also non-displayable packets, but those will be explained later.

Teletext packet types

Every packet contains the magazine number, which is the leftmost digit of the three-digit page number, however, only the header packet contains the remaining two digits of the page number and the subcode. When a parallel transmission scheme is used, pages from different magazines may be transmitted on different lines in the same video frame, which means that the data processor must remember the page number and subcode of the last transmitted header packet for each of the 8 magazines. This is stored in the PAGE_AND_SUBCODE_CACHE whenever a header packet is received and retrieved whenever a packet other than a header packet is received so that the correct page and subcode is always sent to the next module along with the decoded data.

Packet processing

Hamming coding is used to protect certain data such as the page number and subcode to improve the chances of the header packet being decoded correctly in cases of poor reception. This reduces the chance of an incorrect page being loaded on top of the displayed page when a subsequent header packet is incorrectly determined to have the selected page number. Unfortunately, Hamming code dramatically increases the number of bits taken up by each piece of data, so it is only possible to use it for the most important data, and the display data is only protected with a simple parity check with no error correction capability because it is less critical that every display byte is correctly decoded. It should be noted that correct decoding of every single display byte is expected when the signal quality is good and there is no interference.

The parity checking and Hamming decoding is carried out by sub-modules (VHDL components) in TXT_DATA_PROCESSOR.

TXT_MEMORY_CONTROLLER

This module selects the appropriate address in the dual-port RAM to write each packet belonging to the specified page number to, then forwards the packet data on to the dual-port RAM.

It also controls the display of the status line. The last 8 characters should be continuously updated regardless of the magazine of the header row, whilst the rest of the header row should only be updated when waiting for a newly-selected page number ("search mode"), in which case only header rows from the current magazine should be displayed when the pages are transmitted using the parallel transmission mode, and header rows from all magazines should be displayed when pages are transmitted using the serial transmission mode.

This module also extracts the editorial links (also known as Fastext) for the four coloured buttons and the index button from the relevant non-displayable packet.

The 8 characters in the top left corner of the display are called the status characters. These are controlled by the keypad controller, and are used to display the page number and subcode number. The memory controller monitors these characters and updates the dual-port RAM whenever they change. The UPCOMING_FRAME_IN signal ensures that page content updates (which must happen in real-time) are prioritised over updates to the status characters.

The REQ_SUBCODE_SPEC_IN signal is set when the user requires a specific sub-page to be displayed. When the Time-Text button is pressed, the keypad controller sets the REQ_SUBCODE_SPEC_IN signal and the memory controller will then only display pages where both the page number and subpage match the requested values, otherwise the subpage is ignored and all subpages from the current page will be displayed as they are received.

Screenshot of the project in operation

The project can decode pages generated by VBIT2 running on a Raspberry Pi.

Teletext decoder connected to a display, showing a teletext page

Next: Data recovery, processing and page extraction

Adding an aspect radio switch to CRT TVs

This post is about a simple modification that can be done to many CRT TVs to add an aspect ratio switch. The main uses for this nowadays wo...