Sensors and user input

Pondering interactivity

As we experiment with VRML, it becomes clear that this is a powerful way to generate interesting 3-dimensional representations of things, but there are other ways to do the same thing. Rendering tools, and even physical modeling environments can do the same thing. In order for our work to truly approach the illusion of reality, it MUST respond to user interaction in interesting ways. By this definition, VRML 1.0 was barely a VR tool. The user could look at a world from a user-controlled perspective, might have moved the object through something like the EXAMINE mode, and could move around inside a 'virtual world', but there was very little interactivity with it. So far, we have used VRML as a modelling tool, but we have not yet explored the factors that allow VRML 2.0 to be an interactive and dynamic environment. In this unit, we will begin to do so.

Sensing user input

Real Audio

The first thing we will consider is how we might sense input from the user. The user should be considered part of the world, and ought to have some ability to interact with components in the world. There should be some way to tell when the user has done certain things, like moved to a certain area, touched an object, dragged an object, or other actions.

VRML uses special nodes called sensors to gather exactly this kind of information. We will examine a number of different kinds of sensors, and experiment with how they work.

Let's start with an example of a world that does something interesting, and examine how it works...
ts.wrl

#VRML V2.0 utf8
#ts.wrl
#demonstrates touch sensor

Group{ 
  children [
    Shape {
      appearance Appearance {
        material Material {
          diffuseColor 1 1 1
        } # end material
      } # end appearance
    
      geometry Sphere { }
    } # end shape
  
    DEF ts TouchSensor {}
  ] # end children
} # end ball group

DEF blue DirectionalLight {
  ambientIntensity 0
  color 0 0 1
  direction 0 0 -1
  intensity 1
  on FALSE
} # end light

#turn off the headlight.
NavigationInfo {
  headlight FALSE
} # end NavInfo

#add some ambient light (to compensate for no headlight)
DirectionalLight {
  ambientIntensity .5
  color 1 1 1
  direction 0 0 -1
  intensity .2
  on TRUE
} # end light

ROUTE ts.isActive TO blue.on

How it works

The user will see a dimly lit sphere. When the user clicks on the sphere, it will appear to turn blue (oooooh!) This world has one primary object, the sphere itself. The sphere is pretty normal. It's white, even though it appears gray or blue. The sphere belongs in a group, which it shares with a touch sensor. TouchSensors are one example of a special kind of node called sensors. Sensors are not visible to the user. They are intended to capture user events, and send out signals. The TouchSensor in particular is designed to interpret whether the elements in its group were touched. Whenever any element in the same group as the TouchSensor is touched by the user, a number of signals are emitted by the sensor. Here is the spec of the TouchSensor node:

TouchSensor {
  exposedField SFBool  enabled TRUE
  eventOut     SFVec3f hitNormal_changed
  eventOut     SFVec3f hitPoint_changed
  eventOut     SFVec2f hitTexCoord_changed
  eventOut     SFBool  isActive
  eventOut     SFBool  isOver
  eventOut     SFTime  touchTime
}

As you can see, it has a bunch of eventOuts (whatever those are). The one we are interested in is isActive. This thing sends out an SFBool value (TRUE or FALSE) when the user has clicked on the group. Our TouchSensor is named ts (get it? ts for 'T'ouch'S'ensor.) The touch sensor allows any object in its group to respond to "touches" with the mouse. Since the sphere and the touch sensor are in the same group, clicking on the sphere will trigger the 'isActive' action.

The ball will never actually change color. Instead, I'll turn off the normal lighting and add a small ambient light to the scene so you can see the ball dimly lit. (The sphere is white, but it appears gray because there is very little light in the scene. You'll learn much more about lighting in chapter 10). To make the ball seem blue when it is clicked on, I pointed a bright blue spotlight on the ball, which will be turned on when the user clicks on the sphere.

The spotlight is a new object, but you can probably guess at its purpose. We don't actually see the spotlight, but we will see its effects. As a default, the spotlight is between the viewer and the sphere, and is pointed directly at the sphere. We currently have the spotlight turned off (the on field is set to FALSE), and when turned on, it will shine a blue light (set by the color field.) The spotlight is named blueLight. Here is the official spec of the SpotLight node:

SpotLight {
  exposedField SFFloat ambientIntensity  0 
  exposedField SFVec3f attenuation       1 0 0
  exposedField SFFloat beamWidth         1.570796
  exposedField SFColor color             1 1 1 
  exposedField SFFloat cutOffAngle       0.785398
  exposedField SFVec3f direction         0 0 -1
  exposedField SFFloat intensity         1  
  exposedField SFVec3f location          0 0 0  
  exposedField SFBool  on                TRUE
  exposedField SFFloat radius            100 
}

As usual, we will not need all the fields. For this example, we'll just deal with on, direction, and color. The meaning of all these fields is reasonably obvious. Note that the on field is of SFBOOL type. That will be important in a minute or two.

Routing

The natural question is "OK, the sensor is emitting this information, but who or what is hearing it?" So far, nothing. In order to do anything interesting, we need to channel the information somewhere. The last line of the code is the following:

ROUTE theButton.isActive TO blueLight.on

What the ROUTE statement does is channel these events that are generated by the sensor to appropriate receptacles. In other words, the value of the first thing will be transferred to the second. theButton.isActive is FALSE whenever the user is NOT clicking on the button. The value FALSE is transferred over to the blueLight.on field, so our sphere is it's default white color. When the user clicks on the button, the value of theButton.isActive becomes TRUE. The ROUTE statement sends that TRUE value to blueLight.on, turning on the spotlight. The ball then appears to turn blue. (whew!) Now that you know what's going on, look at the world again, and make sure you can visualize the process.

Plane Sensor

Real Audio

The above example was pretty easy to understand, but boolean data is not all that interesting. Often we are interested in values that would be more complicated, such as locations (3Dvec3fs) or colors. There are a variety of sensors, and they each specialize in receiving certain kinds of messages. Each also sends out specific kinds of data. Take a look at the following world for an interesting example:
planeSense.wrl

#VRML V2.0 utf8

Group{
  children [
    DEF cone Transform {
      children [
        Shape {
          appearance Appearance{
            material Material {
              diffuseColor 1 .5 .5
            } # end material
          } # end appearance
    
          geometry Cone{
            height 2
            bottomRadius 1
          } # end geometry
        } # end Shape
      ] # end children
    } # end cone

    DEF ps PlaneSensor{
      enabled TRUE
    } # end planeSensor

  ] # end children


} # end group

ROUTE ps.translation_changed TO cone.translation

How it works

As you can see, this is a very simple world. However, when you click on the cone and drag (using the mouse, wand, or whatever pointing device your system provides), you will see that the cone moves!! Of course, we needed to make a few modifications to the basic cone to allow all this to happen, but it is worth the effort. First, we built a rather ordinary cone shape, then placed it into a Transform. I named that Transform node 'cone' (lower case c cone is not the same as the Cone geometry type, so this is legal. In a moment, you will see why we named the Transform node rather than the Cone geometry. The Transform is inside a Group, which also contains a PlaneSensor. The PlaneSensor is named ps. So, we have a basic geometry with a sensor attached.

The ROUTE statement is where the real magic happens:

ROUTE ps.translation_changed TO cone.translation

The PlaneSensor exposes a special output field called translation_changed. This field sends out a 3Dvec3f corresponding to how much the mouse has moved since it was clicked over the group containing the sensor. We route that coordinate to the translation field of the cone. As you can see, the cone SHAPE does not have any translation field, but the transform that contains the shape does have translation, because that is an exposed field of the Transform node. The net effect is that when the user clicks and drags on an object, she will be able to move it around on the screen. Note that the PlaneSensor will move the object in the XY plane only. We will talk later about how to make it appear to move things in other planes.

The cylinder sensor

Take a look at the following world:
cylSense.wrl

#VRML V2.0 utf8

Group{
  children [
    DEF cone Transform {
      children [
        Shape {
          appearance Appearance{
            material Material {
              diffuseColor 1 .5 .5
            } # end material
          } # end appearance
    
          geometry Box{

          } # end geometry
        } # end Shape
      ] # end children
    } # end cone

    DEF cs CylinderSensor{
      enabled TRUE
    } # end Sensor
  ] # end children
} # end group

ROUTE cs.rotation_changed TO cone.rotation

As you can see, it appears that the user can 'spin' the object. As usual, there is a sensor attached to a group. The sensor sends out a rotation, which can be fed to the rotation field of a Transform. Of course, that is exactly what happens. The Cylinder sensor always rotates around the Y axis. There's a lot of hidden math going on to translate the 2-dimensional motions of the mouse into a rotation, but all you need worry about most of the time is that it does pretty much what you would expect. Again, we'll describe later how to deal with it if we want to rotate along a different axis.

The sphere sensor

The last of the 'shape' sensors is the sphere sensor. You can see its behaviour more easily than I can describe it:

sphereSense.wrl

#VRML V2.0 utf8

Group{
  children [
    DEF cone Transform {
      children [
        Shape {
          appearance Appearance{
            material Material {
              diffuseColor 1 .5 .5
            } # end material
          } # end appearance
    
          geometry Cone{
            height 2
            bottomRadius 1
          } # end geometry
        } # end Shape
      ] # end children
    } # end cone

    DEF ss SphereSensor{
      enabled TRUE
    } # end planeSensor
  ] # end children
} # end group

ROUTE ss.rotation_changed TO cone.rotation

Like the cylinder sensor, the sphere sensor has a rotation_changed out value, but this one is much more liberal. The translation will go along any axis, and is very much like the feel we get when using the examine mode.

The proximity sensor

Real Audio

Another very interesting sensor keeps track of whether the user's avatar (virtual prescence) is close to a certain spot. The proximity sensor has a size value and a center value. If the user is within the virtual box described by the size field and centered on the center field, the sensor will emit an SFBool called isActive. Here's an example. Note that this is a familiar world, but the button is gone. Now, to turn on the light, just move close to the sphere. Here is an example:
ps.wrl

#VRML V2.0 utf8
#ps.wrl
#demonstrates proximity sensor

Group{ 
  children [
    Shape {
      appearance Appearance {
        material Material {
          diffuseColor 1 1 1
        } # end material
      } # end appearance
    
      geometry Sphere { }
    } # end shape
  
    DEF ps ProximitySensor {
      size 7 7 7
    } # end 
  ] # end children
} # end ball group

DEF blue DirectionalLight {
  ambientIntensity 0
  color 0 0 1
  direction 0 0 -1
  intensity 1
  on FALSE
} # end light

#turn off the headlight.
NavigationInfo {
  headlight FALSE
} # end NavInfo

#add some ambient light (to compensate for no headlight)
DirectionalLight {
  ambientIntensity .5
  color 1 1 1
  direction 0 0 -1
  intensity .2
  on TRUE
} # end light

ROUTE ps.isActive TO blue.on

Some more involved examples:

multiple sensors

Real Audio

You can connect more than one sensor to a shape, and a sensor can connect to more than one shape. Here's a very simple example: twoSense.wrl

#VRML V2.0 utf8

Group{
  children [
    DEF cone Transform {
      children [
        Shape {
          appearance Appearance{
            material Material {
              diffuseColor 1 .5 .5
            } # end material
          } # end appearance
    
          geometry Cone{
            height 2
            bottomRadius 1
          } # end geometry
        } # end Shape
      ] # end children
    } # end cone

    DEF ps PlaneSensor{
      enabled TRUE
    } # end planeSensor

    DEF ss SphereSensor {
      enabled TRUE
    } # end sphereSensor
  ] # end children


} # end group

ROUTE ps.translation_changed TO cone.translation
ROUTE ss.rotation_changed TO cone.rotation

The block world

Here's an example of a repetitive file:

senseLeg.wrl

I will not show you the wrl file, because it is over 700 lines long, and quite repetitive. It was written in just under an hour, by using a javascript application to generate most of the code. Basically, here were the steps I used:

create a working prototype of one block in VRML
Write an HTML / Javascript page that can generate that code
Include variables in the code for generating multiple values
put the code in a for loop to generate all 15 shapes of one color
Copy the generated code back into a .wrl file
change values to make another color
repeat as needed.

Here's the javascript used to generate the code: makeLeg.html

<html>
<head>
<title>Make a leg wrl file</title>

<script>
function doIt(){
  var counter = 0;
  var code = "";
   for (counter = 0; counter < 15; counter++){ 
    //set x and y values for block placement  
    var xOffset = (counter * .25) ;
    var yOffset = -3.5;
  
    //write the code
    code += "#yellow" + counter + "\n";
    code += "Group {\n";
    code += "  children [\n";
    code += "    DEF yellow" + counter +" Leg4 {\n";
    code += "      color 1 1 0 \n";
    code += "      translation ";
    code += xOffset + " ";
    code += yOffset + " 0\n";
    code += "    } \n";
    code += "    DEF yellow";
    code += counter + "PS PlaneSensor { }\n";
    code += "  ] # end children \n";
    code += "} # end group \n";
    code += "ROUTE yellow";
    code += counter;
    code += "PS.translation_changed TO yellow";
    code += counter;
    code += ".translation \n";
    code += "\n";
  
  } // end for loop
  
  window.document.theForm.txtOutput.value = code;

} // end function
</script>



</head>

<body bgcolor = "white">
<center>
<h1>Make a leg wrl file<hr></h1>
</center>

<form name = theForm>
<textarea name = txtOutput
          rows = 30
          cols = 60>
</textarea>
<br>
<input type = button
       value = "do it!!"
       onClick = doIt()>

</form>
<hr>
</body>
</html>

The lesson you might learn here is that sometimes you can use your programming skills to great effect to automate the construction of repetitive pages.

Rotating sensors

The plane and cylinder sensors are great, but they are limited in nature. The plane sensor will only translate along the XY plane, and the cylinder sensor will only rotate around the Y axis. What if you want some other behavior? We will see a much cleaner answer to this problem when we work with scripts, but for now, here is a solution that will work without any scripting. Examine the following

planeSense2.wrl

#VRML V2.0 utf8

Transform{
  rotation 1 0 0 1.57
  children [
    DEF cone Transform {
      rotation 1 0 0 -1.57
      children [
        Shape {
          appearance Appearance{
            material Material {
              diffuseColor 1 .5 .5
            } # end material
          } # end appearance
    
          geometry Cone{
            height 2
            bottomRadius 1
          } # end geometry
        } # end Shape
      ] # end children
    } # end cone

    DEF ps PlaneSensor{
      enabled TRUE
    } # end planeSensor


  ] # end children


} # end group

ROUTE ps.translation_changed TO cone.translation

The main thing to notice here is that we have a PAIR of nested transforms. The first Transform is rotated 'back' 1/4 circle around the X axis. The second (named) transform is rotated 'forward' 1/4 circle. The planeSensor is tipped back, and the cone inside it is tipped forward. The cone is the part that receives the input. This is quite a sneaky trick.

The robot

Real Audio

Here is a more complex world that demonstrates the same kind of trick with rotations:

robot.wrl

#VRML V2.0 utf8
# robot.wrl
# illustrates use of proximity sensor
# to keep a control panel close.
# also illustrates cylinderSensor for motion


#robot
DEF base Transform {
  translation 0 -2 0
  children [

    #base
    Shape {
      appearance DEF robColor Appearance {
        material Material {
          diffuseColor .6 .6 .9
        } # end material
      } # end appearance
      geometry Cylinder {
        height .2
        radius 1.5
      } # end geometry
    } # end shape

    #lower arm
    Transform  {
      translation 0 0 1
      #tip back so sensor will work
      rotation 1 0 0 1.57
      children [
        #rotate this 'Y' which is really 'Z'
        DEF la Transform {
          children [
            Transform {
              #tip shape back up
              rotation 1 0 0 -1.57
              center 0 -1 0
              children [
                Shape {
                  appearance USE robColor
                  geometry Cylinder {
                    height 2
                    radius .3
                  } # end geometry
                } # end shape
 
                # upper arm
                DEF ua Transform {
                  translation 1 1 0
                  center -1 0 0
                  children [
                    Shape {
                      appearance USE robColor
                      geometry Box {
                        size 2 .1 .1
                      } # end geometry
                    } # end shape
                  ] # end children
                } # end transform
              ] # end children
            } # end transform
          ] # end children
        } # end tranform
      ] # end children
    } # end transform
  ] # end children
} # end lowerArm

#control panel
Transform {
  translation 0 -2. 4
  children [

    #base
    Shape {
      appearance DEF ctlColor Appearance {
        material Material {
          diffuseColor .6 .6 .6
        } # end material
      } # end ctlColor
      geometry Box {
        size 6 .1 1
      } # end geometry
    } # end shape

    #rotator
    DEF rotator Transform {
      translation -2.5 .125 0
      children [
        Shape {
          appearance USE ctlColor
          geometry Cylinder {
            height .25
            radius .3
          } # end geometry
        } # end shape
        DEF rotCS CylinderSensor { }
      ] # end children
    } # end transform

    #lower arm control
    Transform {
      translation 0 0 .5
      #rotate whole shape down
      rotation 1 0 0 1.57
      children [

        #this is the transform that will be turned
        DEF laControl Transform {
          children [
            Transform {
              #rotate arm back up
              rotation 1 0 0 -1.57
              center 0 -.5 0
              children [
                Shape {
                  appearance USE ctlColor
                  geometry Cylinder {
                    height 1
                    radius .1
                  } # end geometry
                } # end shape
              ] # end children
            } # end transform
          ] # end children
        } # end transform
        
        DEF laCS CylinderSensor {
          minAngle -1
          maxAngle 1
          offset -.5
        }

      ] # end children
    } # end laControl

    #upper arm control
    DEF uaControl Transform {
      translation 2.5 .125 0
      children [
        Shape {
          appearance USE ctlColor
          geometry Cylinder {
            height .25
            radius .3
          } # end geometry
        } # end shape
        DEF uaCS CylinderSensor { }
      ] # end children
    } # end transform

  ] # end children
} # end transform

Viewpoint {
  position 0 3 10
  orientation 1 0 0 -.5
} # end viewpoint


ROUTE rotCS.rotation_changed TO rotator.rotation
ROUTE rotCS.rotation_changed TO base.rotation
ROUTE laCS.rotation_changed TO laControl.rotation
ROUTE laControl.rotation_changed TO la.rotation
ROUTE uaCS.rotation_changed TO uaControl.rotation
ROUTE uaCS.rotation_changed TO ua.rotation

This one is pretty big, but it is mostly not too bad. There are two main structures: the control panel and the robot. All the movements are rotations. The first knob is very easy, as it is meant to rotate around the Y axis, which is the default behavior. The middle lever causes quite a bit more trouble, as it requires a number of new features. We will tip the assembly back so that it's Y axis is the Z axis of the larger world. We also need a named transform with NO default rotation to receive rotation values, and nested inside that, we need the transform which will tip the named transform back up. In all, we have three nested transformations. Note also that we set values for the minAngle and maxAngle fields of the cylinderSensor, which caused limits to be set in the model's rotation.

You might also note that the upper arm was considerably easier to work with, because it was a box, and very flexible size attribute eliminates the need for an extra rotation, so the box could be done with only two transforms.

© Andy Harris
Indiana University / Purdue University, Indianapolis
email: aharris@.cs.iupui.edu
homepage: www.cs.iupui.edu/~aharris