For a complete pattern , the weight update of a backpropagation step for weight is
Using the approximation of Equation 1, we obtain for an incomplete data point (compare Tresp, Ahmad and Neuneier, 1994)
Here, indicates the sum over complete patterns in the training set, and is the standard deviation of the output noise. Note that the gradient is a network of normalized Gaussian basis functions where the ``output-weight'' is now
The derivation of the last equation can be found in the Appendix. Figure 3 shows experimental results.
Figure: In the experiment, we used the Boston housing data set, which consists of 506 samples. The task is to predict the housing price from 13 variables which were thought to influence the housing price in a neighborhood. The network (multi-layer perceptron) was trained with 28 complete patterns plus an additional 225 incomplete samples. The horizontal axis indicates how many inputs were missing in these 225 samples. The vertical axis shows the generalization performance. The continuous line indicates the performance of our approach and the dash-dotted line indicates the performance, if the mean is substituted for a missing variable. The dashed line indicates the performance of a network only trained with the 28 complete patterns.