How to learn multiple tasks

This paper examines the question of how the presence of multiple tasks interacts with learning architectures and the flow of information through those architectures. It approaches the question by using the idealization of an artificial neural network where it is possible to ask more precisely about the effects of modular versus non-modular architectures as well as the effects of sequential vs. simultaneous learning of tasks. While prior work has shown a clear advantage of modular architectures when the two tasks must be learned at the same time from the start, this advantage may disappear when one task is first learned to a criterion before the second task is undertaken. Nonmodular networks, in some cases of sequential learning, achieve success levels comparable to those of modular networks. In particular, if a nonmodular network is to learn two tasks of different difficulties and the more difficult task is presented first and learned to a criterion, then the network will learn the second easier one without permanent degradation of the first one. In contrast, if the easier task is learned first, a nonmodular task may perform significantly less well than a modular one. It seems that the reason for these difference has to do with the fact that the sequential presentation of the more difficult task first minimizes interference between the two tasks. More broadly, the studies summarized in this paper make it clear no one learning architecture is optimal in all situations.