Training (Backpropagation)

next up previous
Next: Conclusions Up: Efficient Methods for Dealing Previous: Experimental Results

Training (Backpropagation)

For a complete pattern , the weight update of a backpropagation step for weight is

Using the approximation of Equation 1, we obtain for an incomplete data point (compare Tresp, Ahmad and Neuneier, 1994)


Here, indicates the sum over complete patterns in the training set, and is the standard deviation of the output noise. Note that the gradient is a network of normalized Gaussian basis functions where the ``output-weight'' is now

The derivation of the last equation can be found in the Appendix. Figure 3 shows experimental results.

Figure: In the experiment, we used the Boston housing data set, which consists of 506 samples. The task is to predict the housing price from 13 variables which were thought to influence the housing price in a neighborhood. The network (multi-layer perceptron) was trained with 28 complete patterns plus an additional 225 incomplete samples. The horizontal axis indicates how many inputs were missing in these 225 samples. The vertical axis shows the generalization performance. The continuous line indicates the performance of our approach and the dash-dotted line indicates the performance, if the mean is substituted for a missing variable. The dashed line indicates the performance of a network only trained with the 28 complete patterns.

Subutai Ahmad
Mon Mar 27 18:14:29 PST 1995