If you’re going to catch a thief, a terrorist, or a political malcontent; you have to detect what’s odd about their behavior. Especially in public. This requires a number of cameras. It will also require special analytical software, facial recognition, gesture recognition, movement analysis – that sort of thing. The system will need to be fully integrated, real-time, and able to interface usefully with those few humans who operate it, and with those in authority who make decisions based on the analysis. The system is called HERMES, and was just announced by the Computer Vision Centre (CVC) of Universitat Autònoma de Barcelona (UAB, Spain). It’s another sign that the lower cost, greater sophistication, and actionable results of advanced sensor/surveillance techniques are reaching new levels of application.

HERMES is an acronym: Human Expressive Graphic Representation of Motion and their Evaluation in Sequences, which means that if you walk into a town square: Cameras capture your presence at a distance; other cameras capture your motion and body movements; still other cameras zoom in on the face for close-ups of expressions. All images are then correlated and put to analysis by artificial intelligence recognition routines. The system is sophisticated enough to learn, recognize, and store movement and expression patterns.

Then HERMES must communicate. This is done by an avatar (some kind of computer representation, presumably of a person) that uses a newly created ‘natural language’ to describe movements and expressions. This approach makes it easier for people to understand and act upon what the HERMES system is finding, particularly in the case where it finds something alarming, unusual, or suspicious.

The list of innovations and improvement in camera technique, visual analysis software, natural language, and system integration is interesting:

1. Cameras system: static cameras were used to supply a full scene, and high resolution active cameras – pan-tilt-zoom sensors (horizontal and vertical inclinations and zoom) – were used for the automatic tracking and close-ups of individuals. To do this, optimisation techniques were applied to the information contained in the images.
2. Movement analysis of objects and individuals in the images. The information obtained is used to guide the active cameras towards where the action is taking place. These tasks were carried out using different tracking techniques.
3. Movement analysis of a person’s body in order to extract information from different parts of the body, analyse these actions and describe or predict behaviours. In this case, techniques based on pattern and silhouette recognition were used.
4. Analysis of facial movements to understand emotional states of an individual, attitudes and possible reactions. In this sub-project new techniques were created and used for the tracking and aligning of 2D and 3D faces.
5. Integration of software and natural language with the aim of describing what is happening in the scenes recorded using a conceptual representation scheme.
6. Full integration of system, software and hardware to work in real environments and in real time. The system was designed and put to use in real life situations to test its functioning.
7. The generation of virtual sequences based on the description of behaviours in natural language and the interaction of real and virtual worlds in the same sequence, using increased reality techniques.

The application advantages of HERMES are obvious, mainly in the fields of intelligent surveillance and the prevention of accidents or crimes. However, researchers consider that there is much to be gained with the use of this tool in sectors such as marketing or psychology.

[Source: EurekAlert]

This is today’s technology. Let your imagination run with it…