Representative Task Self-Selection for Flexible Clustered Lifelong Learning

Abstract:

The lifelong machine learning paradigm learns a sequence of tasks based on previous experiences, such as knowledge library or deep network weights. Most recent lifelong learning models have knowledge libraries or deep networks of a prescribed size, which can degrade performance for both learned and new tasks in a new task environment (cluster). To address this challenge, we propose a novel incremental clustered lifelong learning framework with two knowledge libraries, feature learning library and model knowledge library, called Flexible Clustered Lifelong Learning (FCL 3). The autoencoder-modeled feature learning library maintains a set of representations common to all observed tasks, and the model knowledge library can be self-selected by identifying and adding new representative models (clusters). Our FCL 3 model first encodes a new task by soft-assigning it to multiple representative models over feature learning library using knowledge from these libraries. Then: 1) the new task with a higher outlier probability will be judged as a new representative and used to redefine both feature learning library and representative models over time; or 2) the new task with a lower outlier probability will only refine the library. As new tasks arise, we model this lifelong learning problem as an alternating direction minimization problem for model optimization. Finally, we analyze several multitask data sets and find that our FCL 3 model outperforms most lifelong learning frameworks, even batch clustered multitask learning models.

Note: Please discuss with our team before submitting this abstract to the college. This Abstract or Synopsis varies based on student project requirements.