^{1}

^{1}

^{2}

^{1}

^{1}

^{2}

Mouse picking is the most commonly used intuitive operation to interact with 3D scenes in a variety of 3D graphics applications. High performance for such operation is necessary in order to provide users with fast responses. This paper proposes a fast and reliable mouse picking algorithm using graphics hardware for 3D triangular scenes. Our approach uses a multi-layer rendering algorithm to perform the picking operation in linear time complexity. The objectspace based ray-triangle intersection test is implemented in a highly parallelized geometry shader. After applying the hardware-supported occlusion queries, only a small number of objects (or sub-objects) are rendered in subsequent layers, which accelerates the picking efficiency. Experimental results demonstrate the high performance of our novel approach. Due to its simplicity, our algorithm can be easily integrated into existing real-time rendering systems.

Mouse picking, as the most intuitive way to interact with 3D scenes, is ubiquitous in many interactive 3D graphics applications, such as mesh editing, geometry painting and 3D games. In many Massive Multi-player Role Playing Games (MMRPGs), for instance, thousands of players compete against each other, and the picking operation is frequently applied. Such applications require picking to be performed as fast as possible in order to respond to players with a minimum time delay. In recent years, programmable graphics hardware is getting more and more powerful. How to make full use of the co-processors in the picking operation becomes important.

The WYSIWYG method, which takes advantage of graphics hardware to rerender scene objects into an auxiliary frame buffer, was first proposed by Robin Forrest in the mid-1980s and used in 3D painting by Hanrahan and Haeberli [

In this paper, we propose a simple, fast and reliable picking algorithm (FRMP) using graphics hardware for 3D triangular scenes. By combining the multi-layer culling approach of Govindaraju et al. [

It is fast—our approach is 2 to 14 times as fast as the traditional GPU-based picking one.

It is reliable—our approach performs the operation in object-space, and the exact intersection information can be computed.

It is parallel—the ray-triangle intersection detection is implemented as a geometry shader.

It is simple—our novel approach operates directly on triangular meshes and can be easily integrated into existing real-time rendering systems.

The rest of the paper is organized as follows. Section

Intersection detection is widely used in computer graphics. The mouse picking operation can be performed by an ordinary ray-object intersection test and accelerated by lots of schemes for high efficiency.

The methods for interference detection are typically based on bounding volume data structures and hierarchical spatial decomposition techniques. They are K-d trees [

Hardware occlusion queries are also used in collision detection for large environments to efficiently compute all the contacts at high frame rates by Govindaraju et al. [

Some acceleration techniques for real-time rendering need to be applied in our method. Triangle strips and view frustum culling were introduced by [

Our mouse picking operation takes the screen coordinate of the cursor and the scene to be rendered as input, and outputs the intersection information, such as object id, triangle id, and even the barycentric coordinate of the intersection point. In this section, we first present an overview of our algorithm and then we discuss it in detail.

Our FRMP method exploits the new features of the 4th generation of PC-class programmable graphics processing units [

Algorithm workflow.

Once the user clicks on the screen, compute the picking ray origin and direction in the view coordinate system.

Set the render target with one-pixel size.

Set the render states

After the view frustum culling, render the bounding boxes of the visible objects. We issue a boolean occlusion query for each object during this rendering pass.

Render the bounding boxes of all sub-objects whose corresponding occlusion query returns

Reset the states

Render the actual triangles whose corresponding occlusion query returns

If the occlusion query returns

The novel multi-layer rendering pass on programmable graphics shaders is outlined below:

Transform the per-vertex position to the view coordinate system in the vertex shader.

Perform the object-space-based ray-triangle intersection test in the geometry shader, output a point with picking information if the triangle is intersected. The

Output the picking information directly in the pixel shader.

The

The

In this section, we present the ray-intersection test introduced by Möller and Trumbore [

A ray,

When the user clicks the mouse, the screen coordinates of the cursor are transformed through the projection matrix into a view-space ray that goes from the eye-point through the point clicked on the screen and into the screen.A point,

(Left) a simple ray and its parameters. (Right) barycentric coordinate for a triangle, along with some example point values.

We use a multi-layer rendering algorithm to perform linear time intersection tests, taking advantage of the 4th generation of PC-class programmable graphics processing units. The overall approach makes no assumption about the object's motion and is directly applicable to all triangulated models.

First of all, we set a

Second, we render the bounding boxes of all sub-objects whose corresponding occlusion query returns

Next, the actual triangles of the unpruned sub-objects are rendered. We only issue one occlusion query for all the triangles during this step. We would like to get the exact intersection result after this step. Triangles outside the view frustum are discarded, and only the closest triangle is needed. Thus the render states

Lastly, if the occlusion query passes, the triangle with the minimal distance from the eye-point is picked and its intersection information can be retrieved from the

Our algorithm takes the screen coordinates of the cursor and the scene to be rendered as the input, and outputs intersection information, such as object id, triangle id, and even the barycentric coordinate of the intersection point. Now our algorithm can be used with platforms which support Direct3D 10 APIs. We have incorporated our FRMP method into a Direct3D 10-based scene graph library and tested it on four scenes in order to evaluate its efficiency for different scene types. All tests were conducted on a PC with a 1.83 GHz Intel Core 2 Duo 6320 CPU, 2 GB main memory, an NVIDIA Geforce 8800 GTS GPU, 320 MB graphics memory, and Windows Vista 64bit Operating System.

The four test scenes comprise of an arrangement of a toy elk model (3290 polygons), a Venus model (43 357 polygons), 2000 randomly rotated teapots (12.64 M polygons) and 10 000 randomly rotated tori (8 M polygons), all are in resolution of

The four test scenes: the toy elk (upper left), Venus (upper right) the teapots (lower left) and the tori (lower right). Note that the picked objects are shown in wireframe and the picked triangles are shown in black, whereas other objects are shaded normally.

The toy elk scene only has 3290 triangles, while the Venus scene consists of large number of triangles. Both are simple cases to handle for the picking operation as only one object is used and is not occlusion culled. These two scenes were tested in order to evaluate the efficiencies in simple cases. Such cases may occur in mesh editing or geometry painting applications.

The teapots scene with 12.64 M triangles and the tori scene with 8 M triangles are complex cases and are designed to rotate randomly from frame-to-frame. They can offer good occlusions as most of their objects are occluded in most instances.

For each test scene, we report the processing times of our fast and reliable mouse picking (FRMP) algorithm in comparison to the CPU implementation of our algorithm, and to the traditional GPU method (WYSIWYG) (see Figure

Processing time comparisons for the toy elk (upper left), the Venus (upper right), the teapots (lower left) and the tori (lower right). Note that the lower two scenes use a logarithmic scale to capture the high variations in processing times. Note that if no object is picked, the processing times of our method will even be faster because we picked one object on purpose to perform these tests.

As we can see from a number of scene statistics shown in Table

Statistics for the four test scenes. The processing times are in (miliseconds).

Model name | Triangles per model | Modelnumber | Method | Longest time(miliseconds) | Shortest time(miliseconds) | Average time(milisecond) | Speedup |
---|---|---|---|---|---|---|---|

Toy elk | 1 | CPU | 3.064 | 2.852 | 2.910 | 1.000 | |

WYSIWYG | 6.993 | 6.390 | 6.464 | 0.450 | |||

FRMP | 0.891 | 0.314 | 0.441 | 6.599 | |||

Venus | 1 | CPU | 42.497 | 37.824 | 38.859 | 1.000 | |

WYSIWYG | 11.974 | 10.010 | 10.948 | 3.549 | |||

FRMP | 2.249 | 1.521 | 1.702 | 22.831 | |||

Teapot | 2000 | CPU | 4500.598 | 4320.855 | 4387.013 | 1.000 | |

WYSIWYG | 165.392 | 160.398 | 163.254 | 26.872 | |||

FRMP | 83.334 | 78.293 | 80.959 | 54.188 | |||

Torus | 10 000 | CPU | 1720.887 | 1696.976 | 1706.358 | 1.000 | |

WYSIWYG | 47.918 | 38.016 | 42.411 | 40.234 | |||

FRMP | 19.636 | 14.120 | 17.651 | 96.672 |

We have presented a novel algorithm for intersection tests between a picking ray and multiple objects in an arbitrarily complex 3D environment using some new features of graphics hardware. The algorithm in this paper is fast, more reliable, parallelizable, and simple. Our algorithm is applicable to all triangulated models, making no assumptions about the input primitives and can compute the exact intersection information in object-space. Furthermore, our FRMP picking operation can achieve high efficiency as compared with traditional methods. Due to its simplicity, our algorithm can be easily integrated into existing real-time rendering applications. Our FRMP picking approach is of relevance to interactive graphics applications.The presented approach still leaves some room for improvement and for extensions. For instance, alternative acceleration techniques for real-time rendering may be applied to our FRMP method. Moreover, additional hardware features will be useful with the progress of the graphics hardware. In the future, we would like to extend and to apply our technique to the generic collision detection field.

(1)

(2) edge

(3) edge

(4) Picker.w =

(5)

(6)

(7)

(8)

(9)

(10)

(11)

(12)

(13)

(14)

(15)

(16)

(17)

(18) Picker.y =

(19)

(20)

(21)

(22)

(23) Picker.z =

(24) Picker.z = Picker.z / Picker.w

(25)

(26) output.Pos = float4(0, 0, Picker.z, 1)

(27)

(28) output.Pos.zw =

(29) output.Info = float4(Picker.xy/Picker.w, FacetID,

(30) outStream.

The authors would like to thank the Cybergames '08 conference and special issue reviewers for their dedicated help in improving the paper. Many thanks also to Xiaoyan Luo, Charlie C. L. Wang, and Feifei Wei for their help and their valuable advice. The models used for the test in our paper can be downloaded from