Pattern detection with the camera | Detección de patrones con la cámara

(— Español al final —)

Multi pattern detection project

This time I’m presenting a software developed with my team for the Butiá project, which is a plugin for TurtleBlocks for pattern recognition using the XO camera (for example, traffic signs). The program was developed detached from TurtleBlocks, so it can be used as a library from any other software developed in C or python.

The project was developed in C and ported to python using a binding. The plugin adds a palette to TurtleBlocks for pattern recognition with blocks such as “Is the pattern Visible?” and all the blocks of the possible marks (Traffic signs).

The wiki of the project (in spanish) can be found here, in the Butiá wiki, where you can read more about the internals and how to use it:
http://www.fing.edu.uy/inco/proyectos/butia/mediawiki/index.php/Butia_reconocimiento_marcas

At the end of this article there are videos showing the project and the video of the presentation in the Sumo.UY event.

Proyecto Reconocimiento de marcas

Esta vez estoy presentando un software desarrollado con mi equipo para el proyecto Butiá, que es un plugin para TurtleBlocks para el reconocimiento de marcas con la cámara de la computadora XO (por ejemplo, señales de tráfico). El programa fue desarrollado independiente de TurtleBlocks, así que puede ser utilizado como una biblioteca de cualquier otro software desarrollado en C o python.

El proyecto fue desarrollado en C y se creó un binding para Python. El plugin añade una paleta para TurtleBlocks para el reconocimiento de patrones con bloques tales como “¿Está el patrón visible?” y todos los bloques de las posibles marcas (señales de tráfico) que sirven como parámetros para el bloque anterior.

La wiki del proyecto se puede encontrar aquí, en la wiki del proyecto Butiá. En ella podrán leer más sobre como utilizarlo y el funcionamiento interno:
http://www.fing.edu.uy/inco/proyectos/butia/mediawiki/index.php/Butia_reconocimiento_marcas

Al final de este artículo hay videos mostrando el proyecto y el video de la presentación en el evento del Sumo.UY edición 2012.



Experience at the LARC 2012 – Fortaleza, Brazil.

(— Español más abajo —)

English

Last week we participated with my team (MonsterButiá) at the LARC (Latin American Robotics Competicion) 2012, in Fortaleza Brazil.

We participated in the IEEE Open category, which involves the creation of a Sand cleaning Robot. The objective is to collect black painted soda cans from the sand, and deposit them in the trash (a red cylinder), always avoiding running into a maniqui and a chair, and avoiding stepping into the water.

There were 31 teams from all Latin America. We started devoloping our robot in June of 2012 and the competition was from 17 to 22 october 2012.

The experience was really good, and we managed to get the Third place!

We had 2 best rounds, one at the qualifier rounds and one at the final rounds.
In the last qualifier round we managed to pick up 5 cans, and the final round we managed to pick up 4 cans.

There are a lot of things to improve, but this competition helped us to test our knowledge in Computer Vision and object recognition, interaction with kinect devices, robot behaviors, etc.

The MonsterButia in the news

http://www.cromo.com.uy/2012/11/monster-butia-subio-al-podio-en-brasil/
http://www.lr21.com.uy/enredados/1069737-uruguay-obtuvo-el-tercer-puesto-en-el-concurso-latinoamericano-de-robotica

Project presentation

Slides presenting the work (spanish)

At the end of the posts you can watch the videos of the 2 best rounds and some pictures.

—————————————

Español

La semana pasada participamos con mi equipo MonsterButiá en el LARC (Latin American Robotics Competition) 2012, en Fortaleza, Brasil.

Participamos en la categoría IEEE Open, que implica la creación de un robot de limpieza de playas. El objetivo es recoger las latas de refrescos pintadas de negro de la arena, para finalmente depositarlas en la basura (un cilindro rojo), siempre evitando toparse con un maniqui y una silla, además de evitar entrar en el agua.

Hubo 31 equipos inscriptos de América Latina. Empezamos a desarrollar nuestro robot en junio de 2012 y la competencia fue 17-22 octubre de 2012.

La experiencia fue muy buena, y logramos un excelente resultado, consiguiendo el tercer puesto!

Teníamos dos mejores rondas, una en las rondas eliminatorias y una en las rondas finales.
En la última ronda de clasificación nos las arreglamos para recoger latas de 5 y la ronda final, nos las arreglamos para recoger 4 latas.

Hay muchas cosas que mejorar, pero este concurso nos ayudó a poner a prueba nuestros conocimientos en Visión por computador y reconocimiento de objetos, la interacción con los dispositivos Kinect, comportamientos de robot, etc.

El MonsterButiá en las noticias

http://www.cromo.com.uy/2012/11/monster-butia-subio-al-podio-en-brasil/
http://www.lr21.com.uy/enredados/1069737-uruguay-obtuvo-el-tercer-puesto-en-el-concurso-latinoamericano-de-robotica

Presentación del proyecto

Diapositivas de presentación del proyecto

Pictures and videos

This slideshow requires JavaScript.

Second qualifier round / Segunda ronda de clasificación

First Final Round / Primer ronda final

Robotic Football player

This time I’m going to present the Football player we had to developed for the embedded robotics course, as a part of my MsC in Computer science.

The objective was to create a two wheeled robot with a single camera which would work autonomously to find the ball and score a goal.

Here are the hardware and software specifications, a link to the source and some documentation if you want to learn a bit more.
Hardware:

Software:

  • Ubuntu 11.04 installed in the Beagleboard
  • video4linux for video capturing from the USB camera
  • source code (developed using lua 5.1 and some c from the v4l binding for lua)

The whole source code can be found here: https://github.com/rsisto/luaRoboEmb. The lua file to run to start the robot is penaltyKicker/subsumptionKicker.lua

The robot behavior was developed as a reactive paradigm, using a subsumption architecture .
The whole documentation can be found here (they are in Spanish, so if you have trouble reading using a translator, just ask):
Part 1, Building the wheeled robot and an ax12 API for lua
Part 2, Developing the kicker behavior

Some images and videos

This slideshow requires JavaScript.

Here is a video of the robot scoring a goal 🙂

Access EJB 3.0 in JBoss 5, from JBoss 4 – java.lang.ClassCastException

If you are in a process of migrating JBoss servers you may sometimes need to deploy an EAR in a newer server, while having othres in the older servers, and keep the integration between the apps. I faced this issue when migrating from JBoss 4.3 to JBoss 5, but you may face a similar issue when migrating different versions.

One problem that may arrive when an application in the older server (JBoss 4) tries to access an EJB in the newer (JBoss 5) is this exception when doing a lookup:
java.lang.ClassCastException: javax.naming.Reference cannot be cast to XXX
Being XXX the interface you want to lookup in the JBoss 5 server.

This problem occurs because the application is using by default the client libraries of the JBoss 4 server. To solve this problem, you need to add the client jar files of the JBoss 5 server to the application in JBoss 4.
In order to do this, you need to copy the jars from JBoss 5 jbossall-client.jar and all its dependencies (listed in the META-INF, excluding jnp-client.jar) inside the application EAR file in JBoss 4 (in the lib folder inside the EAR).

This is the full list of jar files that need to be copied from /jboss-as/client inside /lib:
common-annotations.jar
commons-beanutils.jar
commons-collections.jar
commons-digester.jar
commons-logging.jar
concurrent.jar
ejb3-persistence.jar
hibernate-annotations.jar
hibernate-commons-annotations.jar
hibernate-core.jar
hibernate-validator.jar
jboss-aop-client.jar
jboss-appclient.jar
jboss-aspect-jdk50-client.jar
jboss-client.jar
jboss-common-core.jar
jboss-deployers-client-spi.jar
jboss-deployers-client.jar
jboss-deployers-core-spi.jar
jboss-deployers-core.jar
jboss-deployment.jar
jboss-ejb3-common-client.jar
jboss-ejb3-core-client.jar
jboss-ejb3-ext-api-impl.jar
jboss-ejb3-ext-api.jar
jboss-ejb3-proxy-clustered-client.jar
jboss-ejb3-proxy-impl-client.jar
jboss-ejb3-proxy-spi-client.jar
jboss-ejb3-security-client.jar
jboss-ha-client.jar
jboss-ha-legacy-client.jar
jboss-iiop-client.jar
jboss-integration.jar
jboss-j2se.jar
jboss-javaee.jar
jboss-jsr77-client.jar
jboss-logging-jdk.jar
jboss-logging-log4j.jar
jboss-logging-spi.jar
jboss-mdr.jar
jboss-messaging-client.jar
jboss-profileservice-client.jar
jboss-remoting.jar
jboss-security-spi.jar
jboss-serialization.jar
jboss-srp-client.jar
jboss-system-client.jar
jboss-system-jmx-client.jar
jbossall-client.jar
jbosscx-client.jar
jbossjts.jar
jbosssx-as-client.jar
jbosssx-client.jar
jmx-client.jar
jmx-invoker-adaptor-client.jar
jsf-facelets.jar
jsf-tlds.jar
jstl.jar
PagosWebFachada.jar
richfaces-api-3.2.1.GA.jar
richfaces-impl-3.2.1.GA.jar
richfaces-ui-3.2.1.GA.jar
slf4j-api.jar
slf4j-jboss-logging.jar
standard.jar
xmlsec.jar

NOTE: If you fail to copy this list and add also the jnp-client.jar you will get this error when deploying the application:
java.lang.IncompatibleClassChangeError: Class org.jnp.server.NamingServer does not implement the requested interface org.jnp.interfaces.Naming

Lua real multithreading using Lanes and Linda

Introduction

In this post I’ll share my first impression and thoughts about developing mulithreaded applications in lua.
When developing in lua you may have realized that there is no such thing as multithreading, but there is a concept that is propposed as an alternative, called coroutines, which is useful for developing single-threaded applications and sharing the processor between several routines.
Coroutines may be the desired approach for several programs, but sometimes real multithreading is required, as in the case of 2 routines that must execute simultaneously and at a different frequency or when you need a thread to block waiting for user input while doing some background processing. There are lots of examples about this, but I think I made my point.
In my case I wanted to implement a keyboard controlled wheeled robot, so I needed 2 threads, one that would block waiting for user input (arrow keys), and another which would use the input values to control my robot.

Lanes

In order to develop the keyboard controlled robot I found a multithreading library for lua called Lua Lanes (you can find more information in the LuaLanes github project), which allows running multiple lua states in parallel.
Installing Lua Lanes is very easy, just run from the terminal:

luarocks install lanes

After installing I tried the lua scripts proposed as examples in the page, but for my surprise, they didn’t work!
I am using lua version 5.1 and Lua Lanes dated december 2010, and in my case, looking at examples and by trial and error I got to a very simple and working example, which runs 2 functions in parallel, here is the script:

–import extension and initialize
lanes = require "lanes".configure()

–thread 1 function
local function loop1( )
while true do
print("loop1")
os.execute("sleep 0.1")
end
end

–thread 2 function
function loop2()
while true do
print("loop2")
os.execute("sleep 0.1")
end
end

–create and start the threads, notice the ‘()’ to start the thread. you can parameters to the function.
thread1= lanes.gen("*",loop1)()
thread2= lanes.gen("*",loop2)()

This script works fine for me, it should work also for you. You could even define each loop in different lua files, decoupling the function from the threads script.

Linda

After successfully running lua lanes, the only thing left is to pass data from one lane (thread) to another. The api to do this is called Linda, which offers two ways of sending data between lanes:

  • Data queue, as a producer/consumer way (send/receive functions)
  • Single Data slot, which is overriden every time it’s written to (set/get functions)

There is some example using send and receive functions in the lanes documentation, and here is a working example using set and get.

lanes = require "lanes".configure()

local linda= lanes.linda()

local function loop( )
for i=1,10 do
print( "sending: "..i )
linda:set( "x", i )
end
print("end sender")
end

function receiver()
while true do
print("receiving")
local val= linda:get( "x" )
if val==nil then
print( "nil received" )
–break
else
print( "received: "..val )
break
end

end
print("end receiver")
end

a= lanes.gen("*",loop)()
b= lanes.gen("*",receiver)()

Conclusions and considerations

Lanes and Lindas are a nice alternative to multithreading in lua, since there are no built in functionality to do this.
It may be seem not trivial to configure it at the beginning, but once you have a working example it works fine.
Last but not least, it’s not trivial to find errors on a specific lane; in case one lane stops running due to a failed assertion or some other error, that lane won’t print an error to the console and will stop running. This can be avoided by not using assert, printing errors and exit the program in case an error is detected to signal other lanes to stop.

Simon says game in BeagleBoard SBC

Introduction

In my last post about GPIO and LEDs in lua I wrote about a nice feature in Linux when working on embedded systems, such as the possibility to access hardware devices using the file system.
In this post I’ll be using this feature to build the game ‘Simon says’ in lua. I’ll be using a BeagleBoard rev C4 with ubuntu 11.04 installed.
We made this project with my colleagues for an embedded robotics course, so the credit is also shared with them 🙂

GPIOs in the Beagleboard Note

According to the Beagleboard datasheet, the processor can be configured to use all the pins in the expansion port as GPIOs, but this is a more advanced task which requires changing the MUX registers in the processor, which can be achieved by recompiling the kernel changing the kernel headers.
The Ubuntu image for Beagleboard already has several pins configured as GPIO. Thanks to Michael Shiloh, he found the pins that can be used as GPIO by default. This is the list of GPIOs you can use (and the corresponding pin in the expansion port):

  • 157 (pin 22)
  • 159 (pin 18)
  • 161 (pin 16)
  • 162 (pin 14)
  • 136 (pin 9)
  • 137 (pin 7)
  • 138 (pin 5)
  • 139 (pin 3)

For a reference on the pin number for each of these GPIOs read the Beagleboard datasheet.

Development

For this game I only used 3 LEDS, because at that time I didn’t have more of these 🙂 but it’s very easy to add more.
I used 3 GPIO ports to connect the LEDs and 3 for the buttons. Using the default ubuntu installation I could plug 4 leds and 4 buttons. If I wanted more, I should tune the kernel as said earlier in the post.

Connecting the leds and buttons

The picture below shows how everything is connected. It works as a proof of concept, but I wouldn’t give this to anyone to play 😛

Since the picture does not explain this clearly
I used pins 3,5,7 (GPIO 139,138,137 respectively) as input for the buttons and pins 12,14,16 (GPIO 158,162,161 respectively) as output GPIOs for the LEDs.
The diagram below shows how to connect the buttons and leds.

Source code

The source code is available in Github https://github.com/rsisto/simonSaysLuaGpio
The source code is quite simple, The GPIO library explained in the GPIO post was used to interface with the leds and buttons.
The rest of the code is just a main loop implementing the game logic. When the user loses, the leds blink several times and the application quits.

That’s it for now. Let me know if you have any comments or if the code was any use for you! Please share! 😀

GPIO and LEDs in Linux and lua

Hi all,

In my previous posts I wrote about my development in Java, but this time I’ll talk about other area I’m working on, robotics. This time I’ll talk about some basics, managing General Purpose Input Output (GPIO) and LEDs in Linux using the terminal, and also using lua, a very lightweight and portable language.

Basics

When working with embedded robotics or electronics interfacing with computers, it is (often) a good idea to work with linux for several reasons, just to mention a few:

  • Open source: It’s easy to customize the distro, so you can leave out parts of the OS you won’t use, making it more lightweight to run in an embedded system.
  • Portability: Linux kernel is portable to a huge amount of architectures, making it very attractive for use in embedded robotics.
  • Community: There is a big community out there using linux to develop embedded systems, developing new features to linux, testing, discussing and so on.

GPIO and LED filesystem in Linux

When working in linux there other great advantage is that you can control de LEDs and GPIO just by accessing the file system, without the need of any drivers. For example, if you want to control the built in LEDs you would use the /sys/class/leds/ portion of the file system. For GPIOs you’ll use /sys/class/gpio/
Using the file system it’s possible to control these hardware components from any linux terminal and easily create a program that uses them.
In the next sections I’ll explain how to use them from both alternatives.

managing built-in LEDs with the terminal

It’s very easy to control the built-in LEDs, I’ll show you how in this section.
Open a terminal and go to superuser (superuser permissions are needed in order to work with GPIOs and LEDs by default, although it is possible to change this with some OS rules):

sudo su

when listing the directory /sys/class/leds on my laptop I see these items:

root@rafael-vaio:/sys/class/leds# ls
ath9k-phy0  mmc0::  mmc1::

I’ll show you how to control the LED mmc1:: (on my laptop it’s the one near the memory card slot). First, when listing the contents of the mmc1:: folder I see these items:

root@rafael-vaio:/sys/class/leds# ls /sys/class/leds/mmc1\:\:/
brightness  device  max_brightness  power  subsystem  trigger  uevent

The important files here are brightness and trigger. brightness, as you’ll guess, it’s the file that stores the status of the led. trigger is which trigger will change the status of the led, you can see this using:

root@rafael-vaio:/sys/class/leds# cat /sys/class/leds/mmc1\:\:/trigger 
none ADP1-online BAT0-charging-or-full BAT0-charging BAT0-full BAT0-charging-blink-full-solid mmc0 [mmc1] rfkill0 rfkill1 rfkill2 rfkill3 phy0rx phy0tx phy0assoc phy0radio phy0tpt 

The value between the square brackets is the current trigger for this led (in this case, mmc1). To change this (remember the previous value since you should change it back when you finish), just run

root@rafael-vaio:/sys/class/leds# echo none > /sys/class/leds/mmc1\:\:/trigger 

From now on, there is no system trigger that’ll change this led, so it’s going to be only you changing the value.
To change the value and read it:

root@rafael-vaio:/sys/class/leds# echo 1 > /sys/class/leds/mmc1\:\:/brightness 
root@rafael-vaio:/sys/class/leds# cat /sys/class/leds/mmc1\:\:/brightness 
1
root@rafael-vaio:/sys/class/leds# echo 0 > /sys/class/leds/mmc1\:\:/brightness 
root@rafael-vaio:/sys/class/leds# cat /sys/class/leds/mmc1\:\:/brightness 
0

And voila, as simple as that you can start playing with the leds 🙂

managing GPIO using the terminal

The GPIOs are pins that are input/output pins that are connected to the processor, and in Linux it is very straightforward to configure and use them.
This is a bit more complex to test from your pc, since you need to physically access the GPIOs of the processor. In my case, I’ll be using these from a Single Board Computer with linux to read the value of sensors and send commands to actuators on my robotics projects. If you already own one of these, you probably already know how to control them, but if not, there are several options to start with. You can look at the List of single-board computers in wikipedia. In my experiments I used a BeagleBoard rev C4 which has several accessible GPIOs in the expansion bus.

So, shall we?
The steps to control a GPIO in linux are these:

  1. export the GPIO
  2. configure the GPIO as input or output
  3. set the value / read the value

The first step is to export the GPIO, this will tell linux that we’ll be handling the GPIO from the file system. We’ll work in the /sys/class/gpio portion of the fs.
When listing the contents of /sys/class/gpio/ we’ll see only two files: export and unexport. In order to initialize the GPIO with id 56 run:

echo 56 > /sys/class/gpio/export

This will create a folder gpio56 with the following content:

ls /sys/class/gpio/gpio56
direction  edge  power  subsystem  uevent  value

To set the direction (in/out), echo the value ‘in’ or ‘out’ to the direction file:

echo out /sys/class/gpio/gpio56/direction

To set the value, just change the value file, similar as we did with the leds (values 1 or 0):

echo 1 /sys/class/gpio/gpio56/value

That’s it. If you want to use the pin as an input, you should echo ‘in’ to the direction file and read the value file with ‘cat’

LEDs and GPIO using lua

lua is a high level language which has a very small footprint, and runs as fast as a C program. This is desirable when working with embedded systems, since we need to save every last bit of memory.

It is very simple to write a program that uses the led or the gpio filesystem, you just need to execute the commands seen previously and that’s it.
You can check out the code of the simple GPIO lua library I uploaded to Github.

That’s it for now, I’ll be posting more on embedded robotics next week. Let’s talk in the comments!