Machine Learning - Algorithms

2016

Introduction to Machine Learning Algorithms Applied Research Partners

An algorithm will do as you say, for as long as you tell your algorithm to do as you say. Otherwise, it will do nothing!

Algorithms | 1 2 An Introduction to Algorithms An algorithm will do as you say. For as long as you tell your algorithm to do as you say, it will do as you say. Otherwise, it will do nothing!

Al-go-rhythm For as long as you know what you want your machine to do then finding the correct algorithm to complete the task is simply a matter of skill, luck and/or time. But if you have no idea of what you want to do with your data, you have no starting point. Essentially, what this comes down to is regardless of however smart a machine learning task may appear to be, you must be at least smart enough to know what algorithm to select or build. Without this basic foundation, you have nothing, just data and a very bored machine.

Algorithms act on instructions to provide predictions and classifications from observations which you feed to them. At the base of any ‘smart behaviour’, algorithms just predict & classify observations and to do this the user must instruct the machine what they want as an output. As an input, we simply code the step by step approach which our machine will take in order to arrive at its judgements and decisions and having coded our step-by-step process (the algorithm) we feed it.

What do you feed a hungry algorithm?

The ‘observations’ that we refer to can also simply be called data and this data can be of any form and these days they are taking a wide range of forms. Data can be images as in Image recognition used on Google Images or traffic cameras; data can be audio input or text documents as in language classification algorithms like Deep Mind, Siri or IBM Watson; data can also be Numbers such as stock prices as in electronic stock exchange systems which use order matching systems to match buy and sell orders for assets around the world. A machine learning algorithm is a set of user defined inputs to produce prediction or classification outputs, that’s all. The only thing we humans have to do is select the correct algorithm and feed the algorithm some data, yummy data of course!

So far, we know one thing for certain: Algorithms for machine learning predict and

classify observations.

The second thing for us to know which is essential for machine learning concerns the observations themselves. We first ask “what type of data are we dealing with?” This initial step is essential because analysing wonderful observations whilst using the wrong tools can lead your machine to output wonderfully wrong decisions. Google once called a lady a monkey! Not cool.

Algorithms | 1 So, whether its image, text, audio or numbers there are only ever two kinds of observations: Numerical & Categorical. Numerical as logic suggests is data whose value can be understood with the use of numbers and there are two forms of numerical data: Continuous and Discrete.

Categorical is data whose value can be understood with the use of category labels such as questionnaires which ask us to tick either ‘Agree, Don’t Mind, Disagree’ and again, there are two forms of categorical data: Nominal and Ordinal. By knowing your data types, you can extract the full potential out of your algorithm.

Knife, Fork or Spoon feed?

How exactly to feed these observations into your algorithm can vary and the entire variety of ways will not be discussed in any great detail here in this introductory section. The most common ways to feed your algorithm is via a database of stored historical observations or in the absence of storing old cold data your machine can be fed hot data straight from a network of connected streams or with neural networks where data can be fed into layers of your machine with each layer having different permutations of the initial input data.

A few quick examples Let’s say that you’ve identified some interesting observations and now you want to test drive an algorithm or two. If your scenario required the forecasting of continuous data such as sales data or financial market data you may come across Decision Tree Algorithms, Time Series Algorithms, Regression Algorithms or some others. Or perhaps your scenario requires you to analyse the page view statistics for a website project of an e-commerce business who wants to optimise their website layout and so your observations may require you to forecast discrete data leading you to come across Neural Network Algorithms and Naïve Bayes Algorithms or furthermore perhaps you just want to classify nominal data because you owe your local grocer a favour and she wants to know the shopping habits of her male and female customers, so you decide to employ a Clustering Analysis Algorithm. The point is, for every scenario you come across there will be more than one algorithm that can do the job so choosing the ‘right’ algorithm depends largely on the particular output you are looking for.

This takes us back to the initial paragraph. Knowing what you want from your data will help you decide what algorithm you will use. If you don’t know what you want to achieve with your data, you will have no algorithm and you will be data rich and algorithm poor.

The limitations of your algorithm are in the confidence of its judgements. A note on Heuristics As humans we use likelihood and probability judgements in our daily lives but rarely do we sit down before every judgement and make a statistical assessment of our decisions and write down these statistics in a handy pocket book to refer to before proceeding. It wouldn’t be cool, it wouldn’t be efficient and it would simply be weird. As psychologists what we call Heuristics is the use of simple quick mental judgements which we employ to arrive at a good approximation for a decision. An

Algorithms | 1 example of a heuristic can be seen when we quickly judge whether or not we can jump over an obstacle based on a quick approximation of its height compared to our height. On most occasions we get it right, but there are times where we get it quite wrong and end up face down usually thinking ‘Why did I do that?’. When heuristics fail us it’s commonly because we have not fully taken into account all relevant parameters and probabilities carefully enough. Thankfully, machines don’t have to be cool but they do have to be efficient, they do have to sit down and assess the validity of their decisions, they do have to take into account all parameters and probabilities and in most circumstances they do have to be accurate.

In areas of machine learning where an algorithms’ accuracy is a precious commodity, knowing how perfect or imperfect the output of its decision is can be assessed by using likelihood and probability statistics. As it is the nature of machine learning algorithms to make judgements and decision we will naturally want to know how well we can trust these judgements and decisions and for this reason a note of caution is made for the user of any algorithm to have at least a basic understanding of probability and likelihood statistics as a useful tool when designing and ultimately assessing an algorithm, no matter how simple or complex. As a rule-of-thumb, if accuracy is important for your algorithm, stay away from rule-of-thumb judgements and get familiar with confidence intervals.