Intro Ml

Two Ways to Categorize ML algorithms

References:

Two ways: Group them by learning style, and by their similarity in form or function.

By Learning Styles

cite: https://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/

  • Supervised learning: all training data with known labels.

    • e.g. Logistic Regression, Back propagation neural network.

  • Unsupervised learning: Input data is not labeled.

    • e.g. Apriori algorithm, K-Means.

  • Semi-Supervised learning: input data is a mixture of labeled and unlabelled examples.

    • e.g.

By Forms or Functions

cite: https://machinelearningmastery.com/a-tour-of-machine-learning-algorithms/

Algorithms grouped by similarities of how they work:

Regression Algorithms. Modeling the relationship between variables that is iteratively refined using a measure of error in the predictions made by the model. used in statistical machine learning.

  • Ordinary Least Squares Regression (OLSR)
  • Linear Regression
  • Logistic Regression
  • Stepwise Regression
  • Multivariate Adaptive Regression Splines (MARS)
  • Locally Estimated Scaterplot Smoothing (LOESS)

Instance-based Algorithms. Winner-take-all methods, memory-based learning. Find the best match between new data and existing data. Focus is on the representation of the stored instances and similarity measures used between instances.

  • k-Nearest Neighbor (KNN)
  • Learning Vector Quantization (LVQ)
  • Self-Organizing Map (SOM)
  • Locally Weighted Learning (LWL)
  • Support Vector Machines (SVM)

Regularization Algorithms.

  • Ridge regression.
  • Least Absolute Shrinkage and Selection Operator (LASSO)
  • Elastic Net
  • Least-Angle Regression (LARS)

Decision Tree Algorithms.

  • Classification and Regression Tree (CART)
  • Iterative Dichotomiser 3 (ID3)
  • C4.5. Using concept of information entropy. Choose best attribute to split data into subsets according to normalized information gain (difference in entropy).
  • C5.0.
  • Chi-squared Automatic Interaction Detection (CHAID)
  • Decision Stump
  • M5
  • Conditional Decision Trees

Clustering Algorithms.

  • k-Means
  • k-Medians
  • Expectation Maximisation (EM)
  • Hierachical Clustering

Association Rule Learning Algorithms. extract rules that best explain observed relationships between variables in the data.

  • Apriori Algorithm
  • Eclat Algorithm

Aritificial Neural Network ALgorithm. inspired by the structure/function of biological neural networks. Enormous field comprised of hundreds of algorithms and variants for all manner of problem types.

  • Perceptron.
  • Multilayer Perceptrons (MLP),
  • Back-Propagation.
  • Stochastic Gradient Descent.
  • Hopfield Network.
  • Radial Basis Function Network (RBFN).
  • Deep Learning algorithms. exploit abundant cheap computation.
    • Convolutional Neural Network (CNN)
    • Recurrent Neural Networks (RNNs)
    • Long Short-Term Memory Networks (LSTMs)
    • Stacked Auto-Encoders
    • Deep Boltzmann Machine (DBM)
    • Deep Belief Networks (DBN)

Dimensionality Reduction Algorithms. Unsupervised manner. Describe data using less information. Useful to visualize dimensional data or to simplify data then to be used by supervised learning. Classification, regression.

  • Principle Component Analysis (PCA)
  • Principle Component Regression (PCR)
  • Partial Least Squares Regression (PLSR)
  • Sammon Mapping
  • Multidimensional Scaling (MDS)
  • Projection Pursuit
  • Linear Discriminant Analysis (LDA)
  • Mixture Discriminant Analysis (MDA)
  • Quadratic Discriminant Analysis (QDA)
  • Flexible Discriminant Analysis (FDA)

Ensemble Algorithms. combination of weaker models. Focus on what types of weak learners to combine and the ways in which to combine them.

  • Boosting
  • Bootstrapped Aggregation (Bagging)
  • AdaBoost
  • Weighted Average (Blending)
  • Stacked Generalization (Stacking)
  • Gradient Boosting Machines (GBM)
  • Gradient Boosted Regression Trees (GBRT)
  • Random Forest. Combining uncorrelated decision trees.

Evolutionary Algorithms (EA).

wiki

Algorithms of subfields in ML.

  • Computational intelligence (evolutionary algorithms, etc.)
  • Computer Vision (CV)
  • Natural Language Processing (NLP)
  • Recommender Systems
  • Reinforcement Learning
  • Graphical Models
  • …

Algorithms for special tasks in ML.

  • Feature selection algorithms
  • Algorithm accuracy evaluation
  • Performance measures
  • Optimization algorithms

More

  • Kernel Functions

    • References: Kernel Functions Machine Learning is the field of study that gives computers the ability to learn without being explicitly programmed. – Arthur Samuel in 1959. A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. – Tom Michell, CMU. Support Vector Machines:

Created Nov 10, 2020 // Last Updated Aug 31, 2021

If you could revise
the fundmental principles of
computer system design
to improve security...

... what would you change?