The Machine Learning Canvas plays a crucial role in the solution design process. It’s a remarkably simple technique for clearly defining the objectives, scope and outcomes of a project.
Why use the Machine Learning Canvas?
A major reason for failure with any machine learning project is that they start without a clearly defined value or without a proper measurement of success in place. This can cause problems further along in the project. If targets are unclear, then the client’s expectations are not aligned with the end result. This is because, as a relatively new field, it is not always clear where to integrate machine learning within a business. Enter, the Machine Learning Canvas. The use of the canvas is the best way of starting a project off on the right foot as it allows realistic, attainable goals to be set. That was, everybody is clear on what needs to be done and when success has been achieved.
So, what is a Machine Learning Canvas?
Put simply, the machine learning canvas is a visual grid composed of ten individual blocks. Each block represents a decision to be made; a key area which requires definition during a project. It’s used as a consultancy tool that allows for ease of interaction between company and client. It looks like this:
It starts with the Value Proposition in the centre. The value proposition clearly states the client’s needs: the objectives and end-goals of the project. The left side of the model is dedicated to predictions, while the right focuses on defining the data sources and data requirements to achieve our prediction goals. The block at the bottom of the canvas is devoted to measuring how well the system works.
So, to use a real-world example, imagine that a client comes along who wants to predict the winner of each match of the 2018 World Cup for betting purposes. Bear with me here.
You’d start in the centre with the value proposition. The value proposition is central to clearly stating the client’s objectives and goals. Scyfer defines this by first completing a Value Proposition Canvas. So, in this case, the objective of the project is to predict the outcome of each match with accuracy. The client’s pains are losing money in bets and the gains would be detailed knowledge of who to bet on and therefore winning money from an accurate bet.
Then, you’d move on to the left of the canvas, with the Machine Learning task.
The ML task block aims to define the inputs and outputs of each project as well as the type of problem that the client wishes to solve.
In this case, the inputs would be the history of each team in terms of wins and losses and how much they earn; the players that each team will use and the history of each player in terms of: goals; penalties; injuries; salaries; teams played for and leagues played in. The output should be a prediction of who will win each match.
The Decisions block aims to define how predictions will be used to make decisions that provide value to the client. So here, the predictions made would be which team out of each match pair would win with a certain amount of probability. The decisions would then be how much risk to put into betting; whether to bet on a team or not.
Moving onto the Making Predictions block, the objective here is to decide when predictions will be made on new inputs and how long it will take to featurize a new input and make a prediction. In this case, the client and company might agree to make these predictions daily or before every new match, due to the changing odds of victory for each team as the cup goes on, based on changing scores. Therefore, for this client it makes more sense to make predictions often.
The Offline Evaluation is designed to assess the performance of a model before it goes into production. Therefore, because here you have trained your model on the corpus of matches from former world cups, you can test its performance against some of these matches as an ‘offline’ setting.
The Online Evaluation will test the results while they are happening. Here, it would be a comparison with the actual results as the cups progresses.
It is key to agree with our client on the metric used, as well as to agree on a satisfaction level based on that metric.
Let’s say we define our metric as accuracy, namely the number of matches predicted correctly out of the total number of matches. We could state that a satisfactory level is 60%, which is an improvement upon a random result.
It is important to set realistic, achievable goals so that the client is clear upon what constitutes positive results; that a 99% accuracy goal is unreachable.
Next, you’d move onto the right side of the canvas. The Data Sources block aims to state how raw data will be gathered. Here your data sources would be databases of teams and individual players, for example from the football manager game. The goal of the collecting data block is to determine how new data to learn from will be gathered. So here, you could say that you’d collect the live scores of each team published online after every world cup match.
The intention of the Features block is to input representations extracted from raw data sources. Therefore, here you could consider the percentages of free kicks that scored, based on the number of free kicks that a player kicked and the total number that reached the goal. For example, if 20 free kicks are played and 10 result in a goal, you can say that 50% of free kicks will result in goals. This is a way of simplifying as well as making data more meaningful.
Here is your completed canvas:
Therefore, through completing each block in a logical, step-by-step process, the decisions regarding scope, predictions and data are already made.
The best solution can be provided for our client’s problem of losing bets.
These blocks are intertwined and the way they will be filled in is dependent upon the client and the type of problem they have. For example, in some cases, the machine learning task will be clear, while the decisions that need to be made may be less clear. In other cases, the decisions may be clear while the machine learning task may be less so. It is this that will determine in which order the MLC will be completed.
What’s cool about the MLC?
First of all, the machine learning canvas is super visual. Which is perfect if you’re a visual learner, since everything is clearly laid out in a logical, illustrative format. It allows both client and company to see exactly what decisions are being made throughout the process.
This means that the project is clearly defined from the outset. The client can see the scope and parameters of the project and what is and isn’t achievable in the given time frame. In other words, the project has distinct goals which gives it an end-date.
The MLC embodies the spirit of collaboration. Different specialists within a company can work together, pooling their wide-ranging areas of expertise to solve a single problem. The relationship between the client and the company is also heightened. The client is involved throughout the decision-making process, meaning that the company knows exactly what it needs to do to meet their needs. Both client and company are clear on the details which keeps issues and disputes to a minimum.
The great thing about the machine learning canvas is that it can be tailored for every client.
For some projects, some blocks will be more relevant than others, while some might not be used at all.
This enables it to fit the customer’s needs and therefore the best product that serves the customer can be made.
Finally, as the canvas is gradually filled, it is easy to spot when hurdles are going to arise. This means that difficulties can be planned for and solved at an early stage, thereby enhancing the machine learning technology that will be used.
How does Scyfer use the MLC?
Scyfer uses the machine learning canvas as a means of strengthening its relationship with its clients, as it aligns the clients need to solve a problem with Scyfer’s willingness and ability to provide a solution. The canvas is the perfect platform to allow Scyfer to fulfil the client’s needs with a quality-orientated product, particularly with a machine learning as a field, since not all clients will have a solid grasp of what is and isn’t possible with machine learning. The canvas improves the overall quality of a project and ensures that the end goal fits the client’s specifications. This firm client focus is the true goal of Scyfer’s use of the machine learning canvas.