AP - The Recognized Standard In Audio Test

By Bruce Hofer, Chairman & Co-Founder, Audio Precision

After our announcement last month introducing our new PDM option, this month I’m pleased to be able to follow up with another major feature to be released in APx500 v3.0: PESQ.

PESQ stands for ‘Perceptual Evaluation of Speech Quality’ and is an enhanced perceptual measurement for voice quality in telecommunications. If you want a Mean Opinion Score value (or MOS), then PESQ will give it to you.

The new PESQ option will be especially relevant for engineers working in telecom, handsets, and handsfree accessories. PESQ complements APx’s PDM and Bluetooth interfaces particularly well. Add two channels of analog, HDMI, and digital serial, and you have a single integrated audio analyzer for smartphone design and carrier handset evaluation.

Meanwhile, for those of you in the Los Angeles or Boston areas with a passion for analog circuit design, I invite you to attend the talk I’m giving at the AES LA section meeting on February 28, or to register for AP’s Boston College Master Class on March 8, where I’ll be joined by AP’s VP of Engineering, Tom Kite, and AP Field Applications Engineer Dan Foley. Details for both events are below.

Bruce

PESQ perceptual audio measurement will be a major new optional feature in APx500 version 3.0, due for release in April. Perceptual audio measurements are widely used in telecommunications for testing smartphone and VoIP networks and equipment. Other types of perceptual audio measurements will follow in later APx500 releases.

Perceptual audio tests measure how people perceive sound quality. While useful for evaluating both small differences in high-quality music, and larger differences in lower-quality voice material, this article focuses on doing the latter using PESQ (Perceptual Evaluation of Speech Quality).

Perceptual voice quality tests are especially valuable for devices that are bandwidth and/or bitrate limited, and that have codecs that significantly alter the sound. In these cases, many compromises must be made, and perceptual tests help to determine what balance of compromises will lead to the best voice intelligibility. In fact, it can be difficult with some of these devices to get consistent or comparable results using standard sine wave audio tests of electrical properties, such as frequency response and distortion. Ultimately, a combination of both perceptual and electrical audio tests can help to give the most complete picture of the performance of these devices.

Subjective vs. Objective Perceptual Tests

In subjective perceptual measurements, a group of people is assembled and asked to judge the sound quality of various audio clips, typically on a scale from 1 to 5. Subjective perceptual measurements require careful selection of a representative audience and careful control over the environment to achieve meaningful results. When all the individual scores are tallied, the result is called the Mean Opinion Score (MOS).

It’s expensive and time consuming to test with real people, and results will vary from group to group. We can overcome these limitations by instead using tools that incorporate algorithms based on psychoacoustic modeling. Psychoacoustic modeling seeks to correlate measurable impairments in the audio with users’ opinion scores. Testing with these tools imparts other advantages, like being able to make small adjustments to a design and quickly observe the results, or being able to do perceptual testing on a production line—things that would be essentially impossible to do with groups of people. These measurements are classified as objective because they are unaffected by human temperament or test conditions, and the results are repeatable.

PESQ

PESQ¹ is one such perceptual quality measurement tool. It was developed by OPTICOM GmbH in Germany and forms the basis of ITU-T Recommendation P.862. It is specifically designed for testing voice quality on low bandwidth devices, like mobile phones and smartphones. When used in this context with appropriate test signals, it can achieve a very high correlation with results obtainable using human subjects.

Voice quality testing may be done using a reference signal (known as full reference, or FR), or with no reference signal (no reference, or NR). In full reference testing, the measurement tool compares an original recording to one that has been degraded by the system under test. In no reference testing, there is no reference signal played and the measurement tool computes a score without using a comparison. No reference testing is strongly talker dependent and requires a large sampling of voices to achieve accuracy similar to full reference. PESQ uses the full reference method to evaluate voice quality, as shown in the block diagram below.

PESQ In APx500

AP’s PESQ implementation adds two new APx measurements (PESQ and PESQ Average) to deliver comprehensive results:

The PESQ measurement returns overall perceptual quality in MOS or PESQ units after playing each voice sample, along with a quality vs. time display to help pinpoint specific problems such as clipped words or dropouts. Additional results show average delay, delay vs. time, and the reference and the acquired waveforms.

The PESQ Average measurement allows you to run a collection of different voice samples (as recommended in ITU-T P.862) and then display the resulting overall score.

Both measurements let you choose to analyze the entire signal, or to look at only the active speech or silence. When 8 kHz sample rate voice clips are played, you will have the option to choose PESQ Narrowband (ITU-T P.862) or PESQ Wideband (ITU-T P.862.2) analysis.

PESQ is fully integrated into the APx500 measurement navigator in the same way as the existing electrical properties measurements, giving it access to all the same automation, statistical, and reporting features. PESQ measurement may also be combined with electrical measurements to make comprehensive automated test suites. All standard and optional I/O modules may be used, including unbalanced/balanced analog, unbalanced/balanced/optical digital, serial digital, Bluetooth, HDMI, and PDM (available in April).

PESQ will be available as a paid software option on all APx Series analyzers when APx500 v3.0 is released in April.

¹ PESQ® is a registered trademark of OPTICOM GmbH, Germany. Used under license.

APx500 v2.9 introduced the new Data Output node in the navigator. This month, we examine that feature in greater detail.

The new Data Output node provides a simple way to save measurement data from an automated APx project running on a production line to a CSV file. To make use of it, check the box beside Data Output at the bottom of the APx500 measurement navigator. If you wish, you can right-click on it to change the default file name.

Each time an automated sequence is run, all single-point measurement results will be appended to a new line in the file. Single point measurements include Level and Gain, THD+N, IMD, Signal to Noise Ratio, Maximum Output, Crosstalk, Phase, Bandpass, DC Level, Frequency, and Noise. If the file does not already exist at run time, a new one with column headers will be created. If you change the selected measurements or the number of channels in a project sequence after being run the first time, be sure to delete the file so that a new one with the correct headers is created.

The “Prompt User for Device ID” option in Project/Sequence Properties goes hand-in-hand with this feature. When checked, the user is prompted to enter a device ID, such as a serial number, each time the sequence is run (using the keyboard, or alternatively with a barcode scanner). The ID flows into reports and into a column in the Data Output file.

The CSV output file is not only useful for record keeping, but for quality control analysis. Below is a trendline chart made in Microsoft Excel, using the data shown in Figure 2 above.

We’ve looked at how the Data Output node provides a quick way to export production line data. Other ways to export data include the Export Result Data step in the navigator/sequencer (new in APx500 v2.9), the Show Graph Data icon above the graph area (click the Export button after opening), the default and customized reports, and the APx API.

Events:

		AP Chairman Bruce Hofer will speak on Analog Circuit Design in the 2010s at the AES LA section meeting on February 28.
		AP’s Boston College Master Class is March 8, featuring AP Chairman Bruce Hofer, VP of Engineering Tom Kite, and Field Applications Engineer Dan Foley. Registration required.
		Joe Begin, AP’s Director of Products, will give a technical session on Characterizing Class D Amplifiers at the AES Brazil Regional Convention, São Paulo, Brazil, May 8–10.