Skip main navigation

Using your model in Scratch

Using your model in scratch
2.5
How do you deploy your new model into Scratch so you can actually use it? I open the custom version of Scratch found at rpf.io/ml-scratch. You’ll need the specialised blocks to interact with your model. I have also included this starter project for you to use, a rock paper scissors game that only uses text commands. But we’re going to change them. First, you’ll need the Teachable Machine blocks from this left-hand side menu. And you’ll need this Use model block right at the start of your programme. So I’m going to slot it in here just underneath the When green flag clicked command. There is a space here for the model you created in the last step.
49.4
You just need to paste in the URL. To finish the import, you’ll need to run the programme. At the moment, I don’t have any of the classes I need. You can tell that from the blocks on the side, they just show Class 1. We changed that in the previous step, so I’m going to click the green flag. You’ll see that those classes have now changed to the ones that we created in our Teachable Machine. You’ll know it has worked because the classes appear in this dropdown menu. We’re going to change the user choice subroutine so it turns the camera on. So using my Teachable Machine blocks, I’m going to add Turn video on to this subroutine.
96.1
I want it to wait for a second, so I’m going to use this control block. At the moment, it’s using text commands, which I don’t need anymore. So I’m just going to get rid of that text command I don’t need and this text answer that I no longer need. Instead, I’m going to use my model and slot that in there. And then finally, I just want to turn off my video after the second. Let’s turn that Off. So now you’ve got a scratch game that uses real hand gestures to play a rock paper scissors. Let’s give it a go.
140
My little person says, Hello! Welcome to rock paper scissors. First, how many rounds would you like to play? Let’s do three rounds. It’s asking me to make my gesture in three seconds’ time. So what shall I do? Scissors.
159.5
And it’s recognised that I chose scissors. It’s chosen paper, so I win– yes. Let’s give it another go. So in three seconds time, I think I will go rock.
172.8
It recognised I chose rock. The game chose scissors. And I won again. So I’m the winner.
182.1
Follow the instructions in the article below to see how you can improve on this already awesome game. How did you do in your games against the computer? What other games could you make using Teachable Machine and Scratch? Share your experiences and ideas in the comments below.

In this step, you are going to use your classification model to play a game of rock, paper, scissors against the computer. I will show you how you can use Scratch (a free block-based programming language) to achieve this.

Step 1: Getting started

To get started, open this Scratch program that will play a game of rock, paper, scissors.

  • Run the program by clicking on the green flag
  • Play the game to get a feel for how it works without using your classification model
  • Select See inside to look at the code

Step 2: Create your Scratch program

You’re now going to open this same program using a version of Scratch that allows you to use customised blocks that interact with Google’s Teachable Machine.

Step 3: Add your image classification model

Find the custom set of blocks for Teachable Machine from the menu on the left-hand side of your window. Using the left mouse button, grab the first block labelled use model and drag it so that it clicks into place directly under the when green flag clicked block.

You will see a space in the use model block where you can paste in the URL of your machine learning model that you created in the previous step.

Go ahead and add your URL. Click on the green flag to run your program, allow a second for Scratch to execute that new block of code, and stop the program. You can now test to see if it has successfully imported your model by looking for the when model detects block on the left-hand menu. If the drop-down menu contains your class labels (“Rock”, “Paper”, and “Scissors”), then Scratch has successfully imported your model. If the the class labels don’t appear, try again to export and upload your model as you did in the previous step.

GIF showing the teachable machine model being imported into the program

Step 4: Edit the user_choice subroutine

Now, you need to edit the user_choice subroutine so that instead of asking the user to enter rock, paper, or scissors via the keyboard, you will instead activate the camera on their computer so that they can use a gesture instead. To achieve this, replicate the code below:

Image showing the following blocks: Define user_choice, turn video on, say "make your choice in three seconds time for two seconds, say rock for 1 second, say paper for 1 second, say scissors for 1 second, wait for 1 second, set player_choice to model prediction, turn video off

You will need to insert the appropriate blocks in order to:

  • Turn the video on
  • Capture the image or gesture
  • Set the variable player_choice to the prediction that has been made
  • Turn the video off

All that is left to do now is to click the green flag and play the game!

Improve the model

If you find that your game isn’t recognising your gesture or image, try clicking the tick box next to model prediction from the Teachable Machine blocks menu on the left-hand side. This will make a box appear on the program window showing the prediction your model is currently making. If you find the predictions from your model are inaccurate, then go back to Teachable Machine and add more samples to your classes.

A teachable machine block: model prediction. The tick box next to it is checked.

You may also notice that even when you don’t hold anything up to the camera, your model still makes a prediction. My model, for example, thinks that the closest match for no gesture is “scissors”. Try using the say block combined with the confidence block to get your sprite, called Abby, to tell us how confident the model is in classifying no gesture as scissors. You might then want to find a way to replay the round if the confidence score is below a certain threshold.

The say scratch block combined with the confidence score teachable machine block: say confidence for Scissors for two seconds.

Here’s my completed version of the Scratch program. To use it, you’ll need to change the URL for the machine learning model to the URL of your own Teachable Machine model.

Discussion

What other programs do you think you could make using Scratch combined with Teachable Machine? Share your ideas in the comments section.

This article is from the free online

Introduction to Machine Learning and AI

Created by
FutureLearn - Learning For Life

Our purpose is to transform access to education.

We offer a diverse selection of courses from leading universities and cultural institutions from around the world. These are delivered one step at a time, and are accessible on mobile, tablet and desktop, so you can fit learning around your life.

We believe learning should be an enjoyable, social experience, so our courses offer the opportunity to discuss what you’re learning with others as you go, helping you make fresh discoveries and form new ideas.
You can unlock new opportunities with unlimited access to hundreds of online short courses for a year by subscribing to our Unlimited package. Build your knowledge with top universities and organisations.

Learn more about how FutureLearn is transforming access to education