Judging FLL Robot Design (part 2 - the programming)

We continue the discussion of the Robot Design category from the FIRST LEGO League Competition. This time we stop on the programming - how to evaluate the programming of the team during its preparation for the competition.

  • #52
  • 21 Dec 2013
  • 10:57


Statistics designed by Scott Lewis from the Noun Project


In the first part of this video we did a quick overview of the FLL Robot design category. And the judging process. Find the link below this video. In part two I would like to go into details about the referee sheets and especially about programming. First thing you should know is that the referee sheets are public. Everyone could go to the FLL website and download them. Just go to the website, visit FLL events.

Judging and awards and then download the rubrics.

Here you could see I combined rubrics for all the three categories.

Core values.


And Robot design.

Using the sheet is quite straight forward. There are three sections: Mechanical design. Programming. Strategy and innovation. For each section you have three rows and five columns.

As the team enters you should mark their achievements in the given section. And this could be NOT demonstrated, beginning, developing, accomplished, exemplary. If you think that the team should receive an award for Mechanical design. Or for Programming. Or for Strategy and innovation. Mark them at the bottom.

Take your time to know all the values in the different cells before the tournament. It could take you about 10 to 20 minutes. Also be sure to sync with the other judges in your team that you have the same understanding about these values. One of the most important things about this scoring sheet is to judge team relatively. Compare them to each other and NOT to a absolute scale. I have been to a competition where most of the teams are really exemplary. Which is great! Then you come to a situation where two or three teams are obviously better. But there isn't a second column for exemplary. So you should somehow adjust your judging. Leave the best two or three team in the competition as exemplary. And you can mark the others as accomplished. But that is again just a proposal. In the next videos there are a number of pictures I want to show you of different robots. But a lot start from the Programming section. Teams are allowed to use both NXT and Ev3 software. These product are developed for young students that are just starting with programming. The first great challenge for us as judges is to evaluate the complexity of the program. Theoretically it is possible to solve all the missions on every FLL field using only Move Blocks. So Move here with motor B and C then lift something with motor A then move again with motor B and C. Then lift something again and move. But this is not a good program for a number of reasons. As students grow and are at the age of 14-16 When they are no longer allowed to participate in FLL. They should have basic understanding about programming autonomous robots. So considering the following scenario: Team A collects 400 points. Team B again collects 400 points. Team A does it with one big program with just a sequence of Blocks for moving.

Team B has developed an autonomous robot with the use of sensors, variables, states, transitions, threads and other even more reliable design patterns .

So which team is better?

My personal understanding is that Team B has learned a lot more than Team A. They were more open minded and would have greater potential of continuing studding in the field of technical science. Let's now look at the first row of the scoring sheet.

Exemplary states: It should achieve purpose every time. And begging would be: Would not achieve purpose and would be inconsistent. From my experience programs that are relining only on Blocks like this one here for moving. They depend on gaps, on battery, on the initial starting position and on many other parameters. So they would hardly achieve the purpose and most of the time be very inconsistent. On the other hand appropriate use of sensors , decision making, logic operations, variables loops, switches, parallel using of different Blocks can greatly increase the average score of the team. Next we come to how well is the program organised and is it understandable. Where for beginning we have: excessive code, difficult to understand. And for exemplary we have extreme light code and easy to understand for everybody. Many judges will now ask for comments.

Like this one here. I have personally over a decade of experience in software development. Comments from my point of view are the great enemy of every good software developer. Because they introduce redundancy in your code. This means that the next time you change your code you should also change the comment. And if you forget to change the comment which you will serenely do. Then looking at the code two months from now rises the question which is correct the comment or the code. Should I now change the code to reflect the comment or should I change the comment to reflect the code. What I'm personally looking for in the programs is for them to be understandable. And if you cannot understand a simple program developed for the FLL competition. In a matter of minutes. Then you could consider this program as "Not understandable" Modules or MyBlocks could be used here it's much easier to look at a program with the following structure. House and Animals. Humans return to base. And it is much more understandable to see the sequence of the Blocks. Instead of Move, Move, Move, Move and again Move. There is a saying in the software development word: "Everybody could write a program that a computer can understand. The goal is to write a program that a human can understand." Looking at the last row. Automation, navigation. You should have one thing in mind: Is it possible to develop a robot that the team could just start and without any intervention receive the highest score. So - no changing programs.

No changing attachments.

No strategic touching.

And returning back to base. A robot that runs completely on it's own. This should be the goal of every team. What I like to ask the team here for example is how many missions do they have?

How many sensors do they use?

When asking about the programming try to understand the extend to which the coach was involved. I was once judging a team that developed a state machine. Check the links bellow for more information about state machines. It was obvious that the coach had done all the programming. After a couple of questions we found out that the students cannot write even a simple program for following lines. The coach was completely honestly not aware that it should be the children doing the programming to level they can. So to keep the competition fare we didn't allow this team to participate in the Robot Game. Now I finish this video and in the next parts I will continue judging mechanical design and strategy and innovation. I have prepared some very interesting photos of robots. And I have gathered them trough my experience. Find the link bellow to the other parts of the video.