CS330 Lecture 1&2 Study Notes (Incomplete)
CS330 lecture 1&2 notes Informal Problem Definitions The multi-task learning problem: Learn all of the tasks more quickly or more proficiently than learning them independently. The meta-learning problem: Given data/experience on previous tasks, learn a new task more quickly and/or more proficiently. Multi-Task Learning Basics Traditional single-task learning: D={(x,y)k}minθL(θ,D) \begin{array}{l}{\mathscr{D}=\left\{(\mathbf{x}, \mathbf{y})_{k}\right\}} \\ {\min _{\theta} \mathscr{L}(\theta, \mathscr{D})}\end{array} Typical loss: negative log likelihood L(θ,D)=−E(x,y)∼D[logfθ(y∣x)] \mathscr{L}(\theta, \mathscr{D})=-\mathbb{E}_{(x, y) \sim \mathscr{D}}\left[\log f_{\theta}(\mathbf{y} | \mathbf{x})\right] What’s a task? A task: Ti≜{pi(x),pi(y∣x),Li}\mathscr{T}_{i} \triangleq\left\{p_{i}(\mathbf{x}), p_{i}(\mathbf{y} | \mathbf{x}), \mathscr{L}_{i}\right\} ...