Outline of machine learning

photo src: www-personal.umd.umich.edu

The following outline is provided as an overview of and topical guide to machine learning:

Machine learning - subfield of computer science (more particularly soft computing) that evolved from the study of pattern recognition and computational learning theory in artificial intelligence. In 1959, Arthur Samuel defined machine learning as a "Field of study that gives computers the ability to learn without being explicitly programmed". Machine learning explores the study and construction of algorithms that can learn from and make predictions on data. Such algorithms operate by building a model from an example training set of input observations in order to make data-driven predictions or decisions expressed as outputs, rather than following strictly static program instructions.

photo src: chdoig.github.io

Maps, Directions, and Place Reviews

What type of thing is machine learning?

• An academic discipline
• A branch of science
  • An applied science
    • A subfield of computer science
      • A branch of artificial intelligence
      • A subfield of soft computing

Machine Learning Topics Video

Branches of machine learning

Subfields

• Computational learning theory - studying the design and analysis of machine learning algorithms.
• Grammar induction
• Meta learning

Cross-disciplinary fields

• Adversarial machine learning
• Quantum machine learning
• Robot learning

photo src: www.oreilly.com

Machine learning hardware

• Graphics processing unit
• Tensor processing unit
• Vision processing unit

photo src: chdoig.github.io

Machine learning tools

• Comparison of deep learning software
  • Comparison of deep learning software/Resources

Machine learning frameworks

• Marvin - minimalist GPU-only N-dimensional ConvNet framework, developed by the Princeton Vision Group.

Proprietary frameworks

• Amazon Machine Learning
• Apache Spark MLlib
• DistBelief - replaced by TenserFlow
• Microsoft Azure ML Studio
• Microsoft Computational Network Toolkit
• Microsoft Distributed Machine Learning Toolkit
• TensorFlow
• Veles (deep learning framework)

Open source frameworks

• Apache Singa
• Brainstorm (deep learning framework)
• Caffe
• MLPACK
• Neon
• WikiBrain - includes Java library of machine learning algorithms for use on Wikipedia data.

Machine learning libraries

photo src: www.sbs.com.au

Machine learning methods

• Dimensionality reduction
  • CCA
  • Factor analysis
  • Flexible discriminant analysis (FDA)
  • Independent component analysis (ICA)
  • Linear discriminant analysis (LDA)
  • Mixture discriminant analysis (MDA)
  • Multidimensional scaling (MDS)
  • Non-negative matrix factorization (NMF)
  • Partial least squares regression (PLSR)
  • Principal component analysis (PCA)
  • Principal component regression (PCR)
  • Projection pursuit
  • Q­ratic discriminant analysis (QDA)
  • Sammon mapping
  • t-distributed stochastic neighbor embedding (t-SNE)
• Ensemble learning algorithm? - use of multiple learning algorithms to obtain better predictive performance than could be obtained from any of the constituent learning algorithms alone.
  • Boosting
  • Bootstrap aggregating (Bagging)
  • AdaBoost
  • Stacked Generalization (blending)
  • Gradient boosting machine (GBM)
  • Gradient boosted decision tree (GBRT)
  • Random Forest
• Instance-based algorithm
  • K-nearest neighbors algorithm (KNN)
  • Learning vector quantization (LVQ)
  • Self-organizing map (SOM)
  • Locally weighted learning (LWL)
• Regression analysis
  • Logistic regression
  • Ordinary least squares regression (OLSR)
  • Linear regression
  • Stepwise regression
  • Multivariate adaptive regression splines (MARS)
  • Locally estimated scatterplot smoothing (LOESS)
• Regularization algorithm
  • Ridge regression
  • Least Absolute Shrinkage and Selection Operator (LASSO)
  • Elastic net
  • Least-angle regression (LARS)
• Classifiers
  • Probabilistic classifier
    • Naive Bayes classifier
  • Binary classifier
  • Linear classifier
  • Hierarchical classifier

Supervised learning

• Supervised learning
  • AODE
  • Association rule learning algorithms
    • Apriori algorithm
    • Eclat algorithm
  • Case-based reasoning
  • Gaussian process regression
  • Gene expression programming
  • Group method of data handling (GMDH)
  • Inductive logic programming
  • Instance-based learning
  • Lazy learning
  • Learning Automata
  • Learning Vector Quantization
  • Logistic Model Tree
  • Minimum message length (decision trees, decision graphs, etc.)
    • Nearest Neighbor Algorithm
    • Analogical modeling
  • Probably approximately correct learning (PAC) learning
  • Ripple down rules, a knowledge acquisition methodology
  • Symbolic machine learning algorithms
  • Support vector machines
  • Random Forests
  • Ensembles of classifiers
    • Bootstrap aggregating (bagging)
    • Boosting (meta-algorithm)
  • Ordinal classification
  • Information fuzzy networks (IFN)
  • Conditional Random Field
  • ANOVA
  • Quadratic classifiers
  • k-nearest neighbor
  • Boosting
    • SPRINT
  • Bayesian networks
    • Naive Bayes
  • Hidden Markov models

Artificial neural network

• Artificial neural network
  • Autoencoder
  • Backpropagation
  • Boltzmann machine
  • Convolutional neural network
  • Deep learning
  • Hopfield network
  • Multilayer perceptron
  • Perceptron
  • Radial basis function network (RBFN)
  • Restricted Boltzmann machine
  • Recurrent neural network (RNN)
  • Self-organizing map (SOM)
  • Spiking neural network

Bayesian

• Bayesian statistics
  • Bayesian knowledge base
  • Naive Bayes
  • Gaussian Naive Bayes
  • Multinomial Naive Bayes
  • Averaged One-Dependence Estimators (AODE)
  • Bayesian Belief Network (BBN)
  • Bayesian Network (BN)

Decision tree

• Decision tree algorithm
  • Classification and regression tree (CART)
  • Iterative Dichotomiser 3 (ID3)
  • C4.5 algorithm
  • C5.0 algorithm
  • Chi-squared Automatic Interaction Detection (CHAID)
  • Decision stump
  • Conditional decision tree
  • ID3 algorithm
  • Random forest
  • SLIQ

Linear classifier

• Linear classifier
  • Fisher's linear discriminant
  • Linear regression
  • Logistic regression
  • Multinomial logistic regression
  • Naive Bayes classifier
  • Perceptron
  • Support vector machine

Unsupervised learning

• Unsupervised learning
  • Expectation-maximization algorithm
  • Vector Quantization
  • Generative topographic map
  • Information bottleneck method

Artificial neural network

• Artificial neural network
  • Feedforward neural network
    • Extreme learning machine
  • Logic learning machine
  • Self-organizing map

Association rule learning

• Association rule learning
  • Apriori algorithm
  • Eclat algorithm
  • FP-growth algorithm

Hierarchical clustering

• Hierarchical clustering
  • Single-linkage clustering
  • Conceptual clustering

Cluster analysis

• Cluster analysis
  • BIRCH
  • DBSCAN
  • Expectation-maximization (EM)
  • Fuzzy clustering
  • Hierarchical Clustering
  • K-means algorithm
  • K-means clustering
  • K-medians
  • Mean-shift
  • OPTICS algorithm

Anomaly detection

• Anomaly detection
• k-nearest neighbors classification (k-NN)
• Local outlier factor

Semi-supervised learning

• Semi-supervised learning
  • Generative models
  • Low-density separation
  • Graph-based methods
  • Co-training

Reinforcement learning

• Reinforcement learning
  • Temporal difference learning
  • Q-learning
  • Learning Automata
  • State-Action-Reward-State-Action (SARSA)

Deep learning

• Deep learning
  • Deep belief networks
  • Deep Boltzmann machines
  • Deep Convolutional neural networks
  • Deep Recurrent neural networks
  • Hierarchical temporal memory
  • Deep Boltzmann Machine (DBM)
  • Stacked Auto-Encoders

Others

• Data Pre-processing
• Online machine learning

photo src: www.theodysseyonline.com

Applications of machine learning

• Biomedical informatics
• Computer vision
• Customer relationship management -
• Data mining
• Email filtering
• Inverted pendulum - balance and equilibrium system.
• Natural language processing
  • Automatic annotation
  • Automatic summarization
  • Automatic taxonomy construction
  • Dialog system
  • Grammar checker
  • Language recognition
    • Handwriting recognition
    • Optical character recognition
    • Speech recognition
  • Machine translation
  • Question answering
  • Speech synthesis
  • Text simplification
• Pattern recognition
  • Facial recognition system
  • Handwriting recognition
  • Image recognition
  • Optical character recognition
  • Speech recognition
• Recommendation system
• Search engine

photo src: www.math.sjsu.edu

Machine learning problems and tasks

• Anomaly detection
• Association rules
• Bias-variance dilemma
• Classification
• Clustering
• Empirical risk minimization
• Feature engineering
• Feature learning
• Learning to rank
• Occam learning
• Online learning
• PAC learning
• Regression
• Reinforcement Learning
• Semi-supervised learning
• Statistical learning
• Structured prediction
  • Graphical models
    • Bayesian network
    • Conditional random field (CRF)
    • Hidden Markov model (HMM)
• Unsupervised learning
• VC theory

photo src: sdtimes.com

Machine learning research

• List of artificial intelligence projects
• List of datasets for machine learning research

History of machine learning

• Timeline of machine learning

Machine learning projects

• Google Brain

Machine learning organizations

• Knowledge Engineering and Machine Learning Group

Machine learning venues

Machine learning conferences and workshops

• Artificial Intelligence and Security (AISec) (co-located workshop with CCS)
• Conference on Neural Information Processing Systems (NIPS)
• ECML PKDD
• International Conference on Machine Learning (ICML)

Machine learning journals

• Machine Learning
• Journal of Machine Learning Research (JMLR)
• Neural Computation

Persons influential in machine learning

• Alberto Broggi
• Andrei Knyazev
• Andrew McCallum
• Andrew Ng
• Armin B. Cremers
• Ayanna Howard
• Barney Pell
• Ben Goertzel
• Ben Taskar
• Bernhard Schölkopf
• Brian D. Ripley
• Christopher G. Atkeson
• Corinna Cortes
• Demis Hassabis
• Douglas Lenat
• Eric Xing
• Ernst Dickmanns
• Geoffrey Hinton - co-inventor of the backpropagation and contrastive divergence training algorithms
• Hans-Peter Kriegel
• Hartmut Neven
• Heikki Mannila
• Jacek M. Zurada
• Jaime Carbonell
• Jerome H. Friedman
• John D. Lafferty
• John Platt - invented SMO and Platt scaling
• Julie Beth Lovins
• Jürgen Schmidhuber
• Karl Steinbuch
• Katia Sycara
• Leo Breiman - invented bagging and random forests
• Lise Getoor
• Luca Maria Gambardella
• Léon Bottou
• Marcus Hutter
• Mehryar Mohri
• Michael Collins
• Michael I. Jordan
• Michael L. Littman
• Nando de Freitas
• Ofer Dekel
• Oren Etzioni
• Pedro Domingos
• Peter Flach
• Pierre Baldi
• Pushmeet Kohli
• Ray Kurzweil
• Rayid Ghani
• Ross Quinlan
• Salvatore J. Stolfo
• Sebastian Thrun
• Selmer Bringsjord
• Sepp Hochreiter
• Shane Legg
• Stephen Muggleton
• Steve Omohundro
• Tom M. Mitchell
• Trevor Hastie
• Vasant Honavar
• Vladimir Vapnik - co-inventor of the SVM and VC theory
• Yann LeCun - invented convolutional neural networks
• Yasuo Matsuyama
• Yoshua Bengio
• Zoubin Ghahramani

Source of the article : Wikipedia