10 Clipping, Culling, and Visibility Testing

10.010 How do I tell if a vertex has been clipped or not?

You can use the OpenGL Feedback feature to determine if a vertex will be clipped or not. After you're in Feedback mode, simply send the vertex in question as a GL_POINTS primitive. Then switch back to GL_RENDER mode and check the size of the Feedback buffer. A size of zero indicates a clipped vertex.

Typically, OpenGL implementations don't provide a fast feedback mechanism. It might be faster to perform the clip test manually. To do so, construct six plane equations corresponding to the clip-coordinate view volume and transform them into object space by the current ModelView matrix. A point is clipped if it violates any of the six plane equations.

Here's a GLUT example that shows how to calculate the object-space view-volume planes and clip test bounding boxes against them.

10.020 How do I perform occlusion or visibility testing?

OpenGL provides no direct support for determining whether a given primitive will be visible in a scene for a given viewpoint. At worst, an application will need to perform these tests manually.  The previous question contains information on how to do this.

Higher-level APIs, such as Fahernheit Large Model, may provide this feature.

HP OpenGL platforms support an Occlusion Culling extension. To use this extension, enable the occlusion test, render some bounding geometry, and call glGetBooleanv() to obtain the visibility status of the geometry.

10.030 How do I render to a nonrectangular viewport?

OpenGL's stencil buffer can be used to mask the area outside of a non-rectangular viewport. With stencil enabled and stencil test appropriately set, rendering can then occur in the unmasked area. Typically an application will write the stencil mask once, and then render repeated frames into the unmasked area.

As with the depth buffer, an application must ask for a stencil buffer when the window and context are created.

An application will perform such a rendering as follows:

/* Enable stencil test and leave it enabled throughout */

/* Prepare to write a single bit into the stencil buffer in the area outside the viewport */
glStencilFunc (GL_ALWAYS, 0x1, 0x1);

/* Render a set of geometry corresponding to the area outside the viewport here */

/* The stencil buffer now has a single bit painted in the area outside the viewport */

/* Prepare to render the scene in the viewport */
glStencilFunc (GL_EQUAL, 0x0, 0x1);

/* Render the scene inside the viewport here */

/* …render the scene again as needed for animation purposes */

After a single bit is painted in the area outside the viewport, an application may render geometry to either the area inside or outside the viewport. To render to the inside area, use glStencilFunc(GL_EQUAL,0x0,0x1), as the code above shows. To render to the area outside the viewport, use glStencilFunc(GL_EQUAL,0x1,0x1).

You can obtain similar results using only the depth test. After rendering a 3D scene to a rectangular viewport, an app can clear the depth buffer and render the nonrectangular frame.

10.040 When an OpenGL primitive moves placing one vertex outside the window, suddenly the color or texture mapping is incorrect. What's going on?

There are two potential causes for this.

When a primitive lies partially outside the window, it often crosses the view volume boundary. OpenGL must clip any primitive that crosses the view volume boundary. To clip a primitive, OpenGL must interpolate the color values, so they're correct at the new clip vertex. This interpolation is perspective correct. However, when a primitive is rasterized, the color values are often generated using linear interpolation in window space, which isn't perspective correct. The difference in generated color values means that for any given barycentric coordinate location on a filled primitive, the color values may be different depending on whether the primitive is clipped. If the color values generated during rasterization were perspective correct, this problem wouldn't exist.

For some OpenGL implementations, texture coordinates generated during rasterization aren't perspective correct. However, you can usually make them perspective correct by calling glHint(GL_PERSPECTIVE_CORRECTION_HINT,GL_NICEST);. Colors generated at the rasterization stage aren't perspective correct in almost every OpenGL implementation, and can't be made so. For this reason, you're more likely to encounter this problem with colors than texture coordinates.

A second reason the color or texture mapping might be incorrect for a clipped primitive is because the color values or texture coordinates are nonplanar. Color values are nonplanar when the three color components at each vertex don't lie in a plane in 3D color space. 2D texture coordinates are always planar. However, in this context, the term nonplanar is used for texture coordinates that look up a texel area that isn't congruent in shape to the primitive being textured.

Nonplanar colors or texture coordinates aren't a problem for triangular primitives, but the problem may occur with GL_QUADS, GL_QUAD_STRIP and GL_POLYGON primitives. When using nonplanar color values or texture coordinates, there isn't a correct way to generate new values associated with clipped vertices. Even perspective-correct interpolation can create differences between clipped and nonclipped primitives. The solution to this problem is to not use nonplanar color values and texture coordinates.

10.050 I know my geometry is inside the view volume. How can I turn off OpenGL's view-volume clipping to maximize performance?

Standard OpenGL doesn't provide a mechanism to disable the view-volume clipping test; thus, it will occur for every primitive you send.

Some implementations of OpenGL support the GL_EXT_clip_volume_hint extension. If the extension is available, a call to glHint(GL_CLIP_VOLUME_CLIPPING_HINT_EXT,GL_FASTEST) will inform OpenGL that the geometry is entirely within the view volume and that view-volume clipping is unnecessary. Normal clipping can be resumed by setting this hint to GL_DONT_CARE. When clipping is disabled with this hint, results are undefined if geometry actually falls outside the view volume.

10.060 When I move the viewpoint close to an object, it starts to disappear. How can I disable OpenGL's zNear clipping plane?

You can't. If you think about it, it makes sense: What if the viewpoint is in the middle of a scene? Certainly some geometry is behind the viewer and needs to be clipped. Rendering it will produce undesirable results.

For correct perspective and depth buffer calculations to occur, setting the zNear clipping plane to 0.0 is also not an option. The zNear clipping plane must be set at a positive (nonzero) distance in front of the eye.

To avoid the clipping artifacts that can otherwise occur, an application must track the viewpoint location within the scene, and ensure it doesn't get too close to any geometry. You can usually do this with a simple form of collision detection. This FAQ contains more information on collision detection with OpenGL.

If you're certain that your geometry doesn't intersect any of the view-volume planes, you might be able to use an extension to disable clipping. See the previous question for more information.

10.070 How do I draw glBitmap() or glDrawPixels() primitives that have an initial glRasterPos() outside the window's left or bottom edge?

When the raster position is set outside the window, it's often outside the view volume and subsequently marked as invalid. Rendering the glBitmap and glDrawPixels primitives won't occur with an invalid raster position. Because glBitmap/glDrawPixels produce pixels up and to the right of the raster position, it appears impossible to render this type of primitive clipped by the left and/or bottom edges of the window.

However, here's an often-used trick: Set the raster position to a valid value inside the view volume. Then make the following call:

glBitmap (0, 0, 0, 0, xMove, yMove, NULL);

This tells OpenGL to render a no-op bitmap, but move the current raster position by (xMove,yMove). Your application will supply (xMove,yMove) values that place the raster position outside the view volume. Follow this call with the glBitmap() or glDrawPixels() to do the rendering you desire.

10.080 Why doesn't glClear() work for areas outside the scissor rectangle?

The OpenGL Specification states that glClear() only clears the scissor rectangle when the scissor test is enabled. If you want to clear the entire window, use the code:

glDisable (GL_SCISSOR_TEST);
glClear (…);

10.090 How does face culling work? Why doesn't it use the surface normal?

OpenGL face culling calculates the signed area of the filled primitive in window coordinate space. The signed area is positive when the window coordinates are in a counter-clockwise order and negative when clockwise. An app can use glFrontFace() to specify the ordering, counter-clockwise or clockwise, to be interpreted as a front-facing or back-facing primitive. An application can specify culling either front or back faces by calling glCullFace(). Finally, face culling must be enabled with a call to glEnable(GL_CULL_FACE); .

OpenGL uses your primitive's window space projection to determine face culling for two reasons. To create interesting lighting effects, it's often desirable to specify normals that aren't orthogonal to the surface being approximated. If these normals were used for face culling, it might cause some primitives to be culled erroneously. Also, a dot-product culling scheme could require a matrix inversion, which isn't always possible (i.e., in the case where the matrix is singular), whereas the signed area in DC space is always defined.

However, some OpenGL implementations support the GL_EXT_ cull_vertex extension. If this extension is present, an application may specify a homogeneous eye position in object space. Vertices are flagged as culled, based on the dot product of the current normal with a vector from the vertex to the eye. If all vertices of a primitive are culled, the primitive isn't rendered. In many circumstances, using this extension results in faster rendering, because it culls faces at an earlier stage of the rendering pipeline.