Virtually every 3D app or game will need to support mouse or touch interaction with the 3D scene. Consider a real-time strategy game like StarCraft where clicking on units is essential to the gameplay. Check out the Stage3D API and you’ll quickly realize that there’s zero functionality to handling mouse or touch events. So how do all of these games handle it? The answer is a technique called “picking” and this article will show you how to implement it.

The “picking” process is actually pretty simple and consists only of a couple steps:

  1. Construct a ray from the camera pointing “into” the world through the touch/click point. Think of the user’s click as occurring on the near plane of the camera.
  2. Find the 3D object in the scene that the ray hits first

The first step is done like so:

// Normalize the click
var xUnit:Number = (x * 2 / stage.stageWidth) - 1.0;
var yUnit:Number = -((y * 2 / stage.stageHeight) - 1.0);
 
// Make the picking ray
var rayOrigin:Vector3D = new Vector3D();
var rayDir:Vector3D = new Vector3D();
camera.getPickingRay(xUnit, yUnit, rayOrigin, rayDir);

For this article, I’m upgrading my Simple Stage3D Camera by adding the getPickingRay function to construct the ray from the camera’s position into the world through the touch/click point:

public function getPickingRay(xUnit:Number, yUnit:Number, intoOrigin:Vector3D, intoDir:Vector3D): void
{
	intoOrigin.x = __position.x;
	intoOrigin.y = __position.y;
	intoOrigin.z = __position.z;
	intoOrigin.w = 1;
 
	var nearPlaneHeight:Number = __near * Math.tan(__vFOV);
	var nearPlaneWidth:Number = nearPlaneHeight * __aspect;
 
	var rightOffset:Number = xUnit * nearPlaneWidth;
	var upOffset:Number = yUnit * nearPlaneHeight;
 
	// dir = viewDir*near + rightDir*rightOffset + realUpDir*upOffset
	intoDir.x = __viewDir.x*__near + __rightDir.x*rightOffset + __realUpDir.x*upOffset;
	intoDir.y = __viewDir.y*__near + __rightDir.y*rightOffset + __realUpDir.y*upOffset;
	intoDir.z = __viewDir.z*__near + __rightDir.z*rightOffset + __realUpDir.z*upOffset;
	intoDir.w = 0;
	intoDir.normalize();
}

Now we just need to fire that ray at all of the objects in the 3D scene. In this article, I’ve upgraded the Sphere3D class from Procedurally-Generated Sphere to get the intersection distance between the sphere and the ray:

public function intersectRay(origin:Vector3D, dir:Vector3D): Number
{
	var temp:Vector3D = new Vector3D();
 
	temp.x = origin.x - posX;
	temp.y = origin.y - posY;
	temp.z = origin.z - posZ;
 
	var a:Number = dir.dotProduct(dir);
	var b:Number = 2 * dir.dotProduct(temp);
	var c:Number = temp.dotProduct(temp) - radius*radius;
 
	var disc:Number = b*b - 4*a*c;
	return disc >= 0 ? (-b - Math.sqrt(disc))/(2*a) : NaN;
}

Lastly, we just need to use the above function on all of the spheres in the scene:

// Test the picking ray against all renderables
var intersectionDistance:Number = Infinity;
var picked:Sphere3D;
for each (var sphere:Sphere3D in spheres)
{
	// Compute intersection distance and reject non-intersections
	// and bounding volumes we're inside
	var distance:Number = sphere.intersectRay(rayOrigin, rayDir);
	if (!(distance > 0) || distance > intersectionDistance)
	{
		continue;
	}
 
	intersectionDistance = distance;
	picked = sphere;
}

And, of course, hook up a mouse or touch listener to call the above code.

Launch the Demo (click a sphere to stop its rotation, click off the spheres to resume)

Full source code and test texture

The above is a simple picking implementation and there’s much more you can do from here:

  • Add support many more types of bounding volumes
  • Use culling to reduce the number of objects you test against
  • Support picking against the triangles of the 3D object’s mesh

Perhaps there will be articles on those some day. For now, if you’ve spotted any bugs or have any questions or suggestions, feel free to leave a comment!