CS330 lecture 1&2 notes

Informal Problem Definitions

• The multi-task learning problem: Learn all of the tasks more quickly or more proficiently than learning them independently.
• The meta-learning problem: Given data/experience on previous tasks, learn a new task more quickly and/or more proficiently.

Multi-Task Learning Basics

Traditional single-task learning:

Typical loss: negative log likelihood

What’s a task?

A task: $\mathscr{T}_{i} \triangleq\left\{p_{i}(\mathbf{x}), p_{i}(\mathbf{y} | \mathbf{x}), \mathscr{L}_{i}\right\}$

data generating distributions

Here a task is defined as the distribution over data samples, the distribution over data labels, and a loss function.

Corresponding datasets: $\mathscr{D}_{i}^{tr}$ training set, $\mathscr{D}_{i}^{t s t}$ test set.

Usually $\mathscr{D}_{i}$ denotes the training set.

Multi-task classification: $\mathscr{L}_{i}$ same across all tasks. E.g., in handwritten character recognition across different languages, the form of the loss function may be the same.

Multi-label learning: $\mathscr{L}_{i}, {p}_{i}(x)$ same across all tasks. E.g., in the CelebA multi-label recognition task, the samples and the loss function are identical.

The loss function may vary across tasks in the following cases:

• mixed discrete, continuous labels across tasks
• caring more about one task than another (i.e., different weights for different tasks?)

Conditioning on the task

The multi-task learning problem requires introducing a task descriptor as a variable that describes the task; the question is how to design this variable.

Assume ${z}_{i}$ is the task index. The most straightforward approach is multiplicative gating, which effectively trains each task in the multi-task setting with its own separate network, without sharing parameters.

The other extreme is to directly concat $z_i$, in which case all parameters are shared except those that come after the input $z_i$.

Yet another idea is to split $\theta$ into shared parameters $\theta^{sh}$ and task-specific parameters $\theta^i$ — i.e., shared and non-shared parameters.

The optimization objective then becomes

The problem then becomes which parameters to share and when.

Common Choices

The common choices are mainly concatenation and addition — the figures make them clear at a glance.

1. Concatenation-based conditioning

2. Additive conditioning

In fact, the two are equivalent.

3. Multi-head architecture

4. Multiplicative conditioning

The multiplicative approach offers:

• stronger expressive power
• multiplication gating for regression tasks
• better generalization across independent networks and heads

Complex Choices

There are also many other more complex choices.

But where the design inspiration comes from is just like choosing the hyperparameters of a neural network:

• different problems are independent of one another
• for any specific problem, it mostly relies on the designer’s intuition and background knowledge
• current approaches are more art than science

Optimizing the objective

Objective: $\min _{\theta} \sum_{i=1}^{T} \mathscr{L}_{i}\left(\theta, \mathscr{D}_{i}\right)$

The typical procedure:

1. Sample a minibatch of tasks $\mathscr{B} \sim\left\{\mathscr{T}_{i}\right\}$
2. Sample a minibatch of data from each task $\mathscr{D}_{i}^{b} \sim \mathscr{D}_{i}$
3. Compute the loss on each minibatch-task: $\hat{\mathscr{L}}(\theta, \mathscr{B})=\sum_{\mathcal{T}_{k} \in \mathscr{B}} \mathscr{L}_{k}\left(\theta, \mathscr{D}_{k}^{b}\right)$
4. Backpropagate to compute gradients $\nabla_{\theta} \hat{\mathscr{L}}$
5. Update the gradients with your favorite optimizer

Note: this ensures that tasks are sampled uniformly regardless of their data size.

Tip: for regression tasks, make sure task labels are on the same scale.

Challenge

1. Negative transfer

Multi-task training on CIFAR-100 performs worse than training tasks independently.

Possible causes:

• optimization challenges
  • interference between different tasks
  • different learning rates across tasks
• limited expressive capacity
  • multi-task networks are large

Solution:

share less across tasks (soft parameter sharing)

• allows for more fluid degrees of parameter sharing (advantage)
• yet another set of design decisions/hyperparameters (drawback)

The latter term is soft parameter sharing: the difference between one task’s parameters and the previous one is used as a regularization term, which effectively makes each task’s parameters as similar as possible — i.e., the parameters are shared.

2. Overfitting

Overfitting is usually caused by not sharing enough parameters; the solution is to share more. Intuitively, insufficient sharing makes each task overfit, which resembles independent training.

Meta-Learning Basics

Two views of meta-learning:

• Mechanistic view
  • a deep neural network that can take in an entire dataset and make predictions on new data
  • the network is trained on a meta-dataset that contains different datasets for different tasks
  • this view makes it easy to implement a meta-learning algorithm
• Probabilistic view
  • extract prior knowledge from a series of meta-learning tasks
  • use a small amount of data plus prior information to infer a relatively effective posterior
  • this view leads to a better understanding of meta-learning algorithms

Problem definitions

First, recall supervised learning:

Existing issues:

• requires a large amount of labeled data
• labels are very limited for some tasks nowadays

To be continued