GI01/M055/GI20, Supervised Learning. CW#2

Please hand in your assignment to CS Reception, Room 5.25, Malet Place Engineering Building. Aim: To get familiarity with kernels, SVM's and regularisation. Presentation, clarity, and synthesis of exposition will be taken into account in the assessment of these exercises. 1. [60 pts] (kernel properties) Are the following functions K : IRd IRd ! IR valid kernels? Explain your observation. When K is a valid kernel provide a feature map representation for it. (a) K(x; t) = f(x)f(t), where f : IRd ! IR. (b) K(x; t) = x>Dt, where D is a diagonal matrix with non-negative elements. (c) K(x; t) = x>t ?? (x>t)2. (d) K(x; t) = Qd i=1 xiti (Note: we used the notation xi for the i{th component of the vector x 2 IRd). (e) K(x; t) = cos(angle(x; t)). (f) K(x; t) = min(x; t); x; t 0. 2. [30 pts] (SVM's) Assume that the set S = f(xi; yi)gm i=1 IR2 f??1; 1g of binary examples is strictly linearly separable by a line going through the origin, that is, there exists w 2 IR2 such that the linear function f(x) = w>x, x 2 IR2 has the property that yif(xi) > 0 for every i = 1; : : : ;m. In this case, a linear separable SVM computes the parameters w by solving the optimisation problem: P1 : minw2IR2 1 2 w>w : yiw>xi 1; i = 1; : : : ;m : (a) Show that the vector w solving problem P1 has the form w = Pm i=1 ciyixi where c1; : : : ; cm are some nonnegative coecients. (b) Show that the coecients c1; : : : ; cm in the above formula solve the optimisation problem P2 : max 8< :?? 1 2 Xm i;j=1 cicjyiyjx> i xj + Xm i=1 ci : cj 0; j = 1; : : : ;m 9= ;: (c) Argue that, if (^c1; : : : ; ^cm) solves problem P2 and ^w solves problem P1, then ^w> ^w = Pm i=1 ^ci. 3. [10 pts] (kernels) Let x; t 2 (??1; 1) and dene the kernel K(x; t) = 1 1 ?? xt : (a) Show that K is a valid kernel. (b) Given any distinct inputs x1; : : : ; xm 2 (??1; 1) show that the kernel matrix K = (K(xi; xj) : i; j = 1; : : : ;m) is invertible. 1 9 Prior learning assessment portofoilio Homework Ch. 1.4-1.6 Name: __________________________ Date: _____________ 1. Does the table describe a function? A) no B) yes 2. Does the table describe a function? A) yes B) no 3. Which set of ordered pairs represents a function from P to Q? P = {3, 6, 9, 12} Q = {–4, –2, 0} A) {(3, –4), (6, –2), (6, 0), (9, –2), (12, –4)} B) {(9, –4), (9, –2), (9, 0)} C) {(9, –2), (6, –4), (3, –2), (6, 0), (9, –4)} D) {(6, –2), (9, 0), (12, –2)} E) {(3, 0), (9, –2), (3, –4), (9, 0)} 4. Evaluate the function at the specified value of the independent variable and simplify. g (y) = 6y + 2 g (–0.9) A) –5.4y + 12 B) –7.4 C) –3.4 D) –0.9y + 2 E) –0.9y – 2 5. Evaluate the function at the specified value of the independent variable and simplify. f (x – 1) A) B) C) D) E) 6. Find all real values of x such that f (x) = 0. A) B) C) D) E) 7. Find the value(s) of x for which f (x) = g (x). f (x) = x2 + 5x – 18 g (x) = 4x – 6 A) –18, –23, B) –18, 5, C) 3, –4 D) –3, 4 E) 13, 8. Find the domain of the function. A) all real numbers B) all real numbers , C) all real numbers D) s = –6, s = 0 E) s = –6 9. Find the difference quotient and simplify your answer. f (x) = 4x2 + 3x, , h? 0A) 5 + h B) C) D) 3 + 4h E) 11 + 4h 10. A rectangle is bounded by the x-axis and the semicircle (see figure). Write the area A of the rectangle as a function of x and determine the domain of the function. –9 9 A) , –9?x?9B) , x??0C) , –9?x?9D) , all real numbersE) , x??0 11. Use the Vertical Line Test to determine in which of the graphs y is not a function of x. A) x B) x C) x D) x E) All of the choices (A, B, C, and D) represent functions. 12. Find the zeroes of the functions algebraically. A) x = 3, x = –7, B) x = 3, x = –7 C) D) x = –3, x = 7 E) x = –3, x = 7, 13. Determine the intervals over which the function is increasing, decreasing, or constant. A) B) C) D) E) 14. Graph the function and determine the interval(s) for which f (x) ? 0. f (x) = –x2 – 2x A) B) C) D) E) {–2} 15. Determine whether the function is even, odd, or neither. A) B) C) 16. Evaluate the function for the indicated values. (i) f (1) (ii) f (–5.5) (iii) f A) (i) 24 (ii) –5 (iii) 19 B) (i) 24 (ii) –5 (iii) 23 C) (i) 23 (ii) –1 (iii) 23 D) (i) 23 (ii) –1 (iii) 19 E) (i) 23 (ii) –5 (iii) 19 17. Which function does the graph represent? A) B) C) D) E) 18. Which graph represents the function? A) B) C) D) E) Answer Key 1. A 2. A 3. D 4. C 5. A 6. B 7. C 8. A 9. E 10. A 11. D 12. D 13. B 14. B 15. C 16. E 17. B 18. C