How to learn multiple tasks.

The paper examines the question of how learning multiple tasks interacts with neural architectures and the flow of information through those architectures. It approaches the question by using the idealization of an artificial neural network where it is possible to ask more precise questions about the effects of modular versus nonmodular architectures as well as the effects of sequential vs. simultaneous learning of tasks. While prior work has shown a clear advantage of modular architectures when the two tasks must be learned at the same time from the start, this advantage may disappear when one task is first learned to a criterion before the second task is undertaken. Nonmodular networks, in some cases of sequential learning, achieve success levels comparable to those of modular networks. In particular, if a nonmodular network is to learn two tasks of different difficulty and the more difficult task is presented first and learned to a criterion, then the network will learn the second easier one without permanent degradation of the first one. In contrast, if the easier task is learned first, a nonmodular task may perform significantly less well than a modular one. It seems that the reason for these difference has to do with the fact that the sequential presentation of the more difficult task first minimizes interference between the two tasks. More broadly, the studies summarized in this paper seem to imply that no single learning architecture is optimal for all situations.