Skip to main content

CS 6601- AI - Machine Learning

Machine Leraning​

Text​

CH. 19 Learning from Examples

CH. 20 Learning Probabilistic Models

CH. 21 Deep Learning

Topics​

In theory.

  • Machine Learning

  • Pattern Recognition (not necessary learning)

  • Data Mining (big data)

Applications:

  • Recognition
    • Optical
    • Face
    • Text
    • handwriting
    • Speech
    • activity
  • Web Search
  • Spam filtering

Weka and HTK

Supervised Learning​

label examples from learning.

Decision Trees

K-Nearest Neighbours(KNN)​

Try different k, check the best result, label it with majority of k nearest neighbours.

Cross validations​

Assumption:

  • The data collected is a good representation for the problem
  • The data spans the space of unknown examples

Steps:

  1. Take 10% randomly for reserved final prediction test.
  2. The remaining 90%: 20% for testing(independent test set), 80% for training.

ML Competition: Kaggle, codalab, chalearn

Leave one out, Cross Validation(LOOCV)

Normal distribution​

Central Limit Theorem

Bayes Classifier​

AIMA: Chapter 20.1-20.2

Maximum Likelihood

Decision Trees​

Entropy

H(x)=−Σin(P(xi)log2(P(xi)))H(x) = -\Sigma_i^n(P(x_i)log_2(P(x_i)))

Information gain

Remainder(A)=Σkn(pk+nk)(p+n)B(pkpk+nk)Remainder(A) = \Sigma_k^n {(p_k+n_k)\over(p+n)} B({p_k \over p_k+n_k}) Gain(A)=B(pp+n)−Remainder(A)Gain(A) = B({p\over p+n}) - Remainder(A)

Random forest​

Boosting​

Neural Nets​

Input -> Activation function -> output

Feed forward network has no internal states, has connections in one direction.

Recurrent networks have directed cycles with delays.

Unsupervised Learning​

discover repeated patterns in the data.

Deep Learning(Neural Networks)​

"Deep" refers to circuits are typically in a layered structure and computations are steps by steps.