Merging of Two Artificial Neural Networks

This paper addresses the problem of merging two feedforward neural networks into one network. Merging is accomplished at the level of hidden layer. A new network selects its hidden layer's units from the two networks to be merged We uses information theoretic criterion (quadratic mutual information) in the selection process. The hidden unit's output and the target patterns are considers as random variables and the mutual information between them is calculated. The mutual information between hidden units are also considered to prevent the statistically dependent units from being selected. Because mutual information is invariant under linear transformation of the variables, it shows the property of the robust estimation.