Watermelon Book

Introduction Preamble Machine learning: suppose we use P to evaluate the performance of a computer program on a certain class of tasks T; if a program improves its performance on the tasks in T through experience E, then we say that, with respect to T and P, the program has learned from E. Basic Terminology Data-related: A dataset contains a series of records, and each record is an instance or a sample. That which reflects certain properties is an attribute or a feature. The value an attribute takes is called the attribute value, and the space spanned by the attributes is called the attribute space, the sample space, or the input space. Every point in this space is called a feature vector. ...

Model Evaluation and Selection Empirical Error and Overfitting Error rate: the overall proportion of misclassified samples Accuracy: the proportion of correctly classified samples Error: the difference between the actual output and the true output; on training samples it is the empirical error or training error, and on new samples it is the generalization error Overfitting: generalization performance degrades Underfitting: performs poorly even on the training samples Evaluation Methods A testing set is typically used to measure a model’s performance, with the testing error serving as an approximation of the generalization error ...