Woodward on Woodward on Bayes

Author: Julian Champkin

In the February 2012 issue of Significance, ‘Philip Woodward on Philip Woodward’ outlines how Philip Woodward – the senior of the two namesakes - applied Bayes Theorem to radar and other waveforms to extract the maximum information from a noise-infected signal. Students might like to know more. Here, as promised in the magazine, we present a more detailed example.

To illustrate how Woodward proposed that Bayesian methods could be used to determine the message contained in a noisy signal, we give an adapted version of one of his examples. It is taken from section 4.4 of his book Probability and Information Theory: With Applications to Radar.

Suppose that a message is to be sent. The message is the result of a sports match and it is transmitted as a waveform: a ‘positive’ half-wave of amplitude 20 if the match is won, a flat line if it is drawn, and a ‘negative’ half-wave, again of amplitude 20, if it is lost.

The receiving machine cannot read the whole wave but samples it at five places to give five discrete values. Were there no noise, the signals it would receive are as follows:

Message (θ) -  Signal (x)

Match won  - 0 10 20 10 0
Match drawn - 0 0 0 0 0
Match lost - 0 -10 -20 -10 0

However, what the waveform in fact receives is: y = x + e

where e is white Gaussian noise with a known variance of 100. It can be shown that a relatively simple formula connects the posterior distribution - that is, p(θ | y), the probability of each match result given the signal received - with the prior distribution p(θ), the prior belief of the probability of each result.

Suppose the match was drawn, so that (unknown to the observer who reads the message) the waveform at the receiver was pure noise, e.g.:

y = 10 -5 -9 7 -7

How can the observer decide which message had actually been sent?

If each match result is a priori equally likely, then it is easy to compute from the formula the posterior probabilities of each result:

Won 0.008
Drawn 0.796
Lost 0.196

The observer would conclude that the match was probably drawn, and would happen to be right.

Woodward showed how the Bayesian approach quantifies how a mistake in the assumed size of the noise or signal affects our interpretation of the message. An error in the magnitude of the noise does not alter the order of the probabilities but will influence their actual values; as the noise increases Bayes theorem gives the intuitive result that posterior beliefs tend to the prior beliefs, i.e. no information is transmitted. An error in the signal level is more serious as this will in general alter the order of the posterior probabilities.

Woodward also goes on to show the ease with which Bayesian methods deal with nuisance parameters (referred to as "stray parameters" in the book), extending the previous example by including an unknown time-origin of the signal. The solution is obtained by including a prior for the time-origin and integrating over this parameter; a good illustration of how the Bayesian approach matches the intuitive solution.