|Publication number||US7733352 B2|
|Application number||US 11/228,876|
|Publication date||8 Jun 2010|
|Filing date||16 Sep 2005|
|Priority date||15 Apr 2003|
|Also published as||EP1616303A2, US20040207631, US20060109277, WO2004095376A2, WO2004095376A3|
|Publication number||11228876, 228876, US 7733352 B2, US 7733352B2, US-B2-7733352, US7733352 B2, US7733352B2|
|Inventors||Simon Fenney, Paolo Giuseppe Fazzini|
|Original Assignee||Imagination Technologies Limited|
|Export Citation||BiBTeX, EndNote, RefMan|
|Patent Citations (5), Non-Patent Citations (19), Referenced by (2), Classifications (14), Legal Events (1)|
|External Links: USPTO, USPTO Assignment, Espacenet|
This is a continuation of Ser. No. 10/611,719, filed Jul. 1, 2003 now abandoned.
This invention relates to a method and apparatus for generating bump map data for use in a 3 dimensional computer graphics system.
In the field of 3D computer graphics, detail is often added to otherwise smooth objects though the use of Bump Mapping, which was introduced by Blinn in his paper “Simulation of Wrinkled Surfaces” (SIGGRAPH 1978, pp 286-292). This operates by perturbing, on a pixel-by-pixel basis, an object's otherwise ‘smoothly’ varying surface normal vector. Because the surface's normal vector is used when computing the shading of that surface, its modification can give the appearance of bumps.
In Blinn's technique, each perturbation is computed by first taking derivates of a bump displacement texture or ‘height map’ and subsequently applying it to the original surface normal and surface tangent vectors. The height map is a simple array of scalar values that gives the ‘vertical’ displacement or ‘height’ of a surface at regular grid points relative to that surface. Typically these are represented by monochromatic image data, e.g. a bitmap, with the brightness of any pixel being representative of the ‘height’ at that point. Standard texture mapping practices are used to access the height data. The normal perturbations and lighting calculations are done in global or model space.
A more ‘hardware friendly’ method was later developed by Peercy et al (“Efficient Bump Mapping Hardware”, SIGGRAPH 1997, pp 303-306, (also U.S. Pat. No. 5,949,424)). This directly stores perturbed surface normals in a texture map, often called a normal map. Unlike Blinn's method, these normals are defined in a local tangential coordinate space, which can be likened to the representation of parts of the earth's surface on a page in an atlas. In Peercy's technique, the lights used for shading are also transformed into this tangential space and thus the shading calculations are also computed locally. This process significantly reduces the number of calculations required when using bump mapping. It has become popular in recent 3D hardware systems and is sometimes known as ‘Dot3 bump mapping’.
To minimize the texture memory and, more importantly, memory bandwidth required by this procedure, it is desirable to compress the normal maps. Unfortunately many of the commonly used texture compression schemes are not suitable as they cause a loss of information that, when applied to the special case of normal maps, can cause an unacceptable degradation in image quality. Two methods that are specifically tailored to normal maps, however, are described in our International patent application No. WO9909523—these typically still use 16 bits to represent each surface normal.
This then leaves the task of generating the normal map. One popular method again uses an initial height map, as originally described by Blinn. From that height map, a normal map can then be pre-computed, prior to rendering, by taking the cross product of the local derivative vectors of the height function sampled at regular positions. For cases where texture filtering is required, e.g. those based on the well-known MIP mapping techniques, the height map should be repeatedly down-sampled and the associated normal map regenerated to produce the multiple MIP map levels. Problems can arise, however, when applying the texture filtering techniques, e.g. bilinear or trilinear filtering, to normal maps.
It should be noted that whereas the height map could be of relatively low precision—often as low as, say 8 or even 4 bits per pixel—the normal map may require 16 to 32 bits per pixel. The pre-processing steps of generating and compressing the normal map and the process of using the compressed normal map in 3D rendering are shown in
Also well known in the art is the aspect of texture filtering, primarily the application of bilinear or trilinear filtering, the latter as invented by Williams (“Pyramidal Parametrics”, Lance Williams, Computer Graphics, Vol. 7, No. 3, July 1983, pp 1-11). Bilinear filtering is briefly discussed below, since trilinear filtering is just the blending of two bilinear operations.
A 2D texture can be considered to be a vector function of 2 variables (U, V). For simplicity in this discussion, we will assume that, for an N×N pixel texture, the values of U and V range from 0 to N. When bilinear filtering is applied, the pixel, or ‘texel’, values stored in the texture can be considered to be representative of the points in the centres of the respective texels, i.e. at coordinates (i+0.5, j+0.5), where i and j are integers and represent the texel coordinate of the particular texel. This is illustrated in
In particular, the process used in the art will be some simple variation of the following:
It should be noted that the colours in 3D computer graphics are usually 4-D entities, having Red, Green, Blue, and Alpha (i.e. transparency) components. When the bilinear blending described above is performed, all four components of the various colour values are operated on in parallel. This is shown in the second stage of the bilinear operation in
Another known aspect of 3D computer graphics is that of fitting smooth surfaces through or near a set of control points. In particular we are interested in two types known as uniform B-spline and Bezier splines, as described in literature such as “Computer Graphics. Principles and Practice” (Foley et al) or “Curves and Surfaces for CAGD. A practical guide” (Farin).
Of particular interest to this application is the case of a bi-quadratic B-spline which has C1 continuity (i.e. continuous first derivative). A bi-quadratic B-spline also has the property that, for any point on the surface, a sub-grid of 3×3 control points is needed to evaluate that point and/or derivatives at that point. A one-dimensional slice though a section of a quadratic B-spline is shown in
One popular way of evaluating such a curve is to first convert it to the equivalent Bezier representation, i.e. a different set of 3 control points, and then apply the de Casteljau algorithm which uses repeated linear interpolation (see Farin). For the simple case of quadratic curves, this amounts to using a new set of control points which are ‘88’, ‘81’ (i.e., it is re-used), and ‘89’. Points ‘88’ and ‘89’ are just the mid points of the connecting line segments and could be found by simple averaging.
For the conversion of a bi-quadratic B-spline surface, the 3×3 grid of control points can be replaced by an equivalent set of 3×3 Bezier control points. An example showing the situation for a bi-quadratic surface is shown, in plan form, in
then the equivalent set of Bezier points are computed from:
Referring again to
The height map defines ‘height’ values only at certain sample locations and so a means of computing the height surface at other points is required. In particular, bump mapping requires the surface normal which, in turn, usually implies the need for surface tangents. Blinn points out that the surface height is not actually required and proposes a function that only computes tangents. He notes that in order to avoid discontinuities in the shading, his tangent functions are continuous. Using the 3×3 grid of height samples shown in 6, Blinn's function performs 3 bilinear blends respectively of the top left, top right, and bottom left neighbours, and then computes the differences of the top left and top right result and the top left and bottom left result as part of the tangent generation.
Although Blinn's function results in a continuous normal, its derivative can have discontinuities. Unfortunately, the human visual system is very sensitive to changes in the derivative of image intensity, and so ‘artefacts’ can be seen. The method also tends to emphasise the underlying grid of the height map, which can be seen in
Although the introduction of Peercy et al's pre-perturbed normal map method makes bump mapping more practical in real-time hardware, it still requires ‘large’ texture formats as well as the separate pre-processing step to convert a height map to normal map. The ‘large’ texture formats consume valuable bandwidth as well as memory and cache storage and, although special normal map compression techniques exist, these formats are still often larger than the original source height map. Also filtering of the normal map may also be problematic.
A further limitation of Peercy et al's technique is that dynamic bump mapping, i.e. where the bump heights are computed frame-by-frame, is far more difficult to achieve. For example, the height values may be generated as the result of a separate rendering pass. The pre-processing step, including generation of the various MIP map levels, may take too much time to allow real-time rendering.
Finally, it is beneficial to use a height function with C2 (or higher) continuity so that the normal interpolation is C1 (or higher). In particular, it is important to have an inexpensive means of producing this function.
We have appreciated that it is possible to implement, in hardware, an additional set of functions that provides an efficient means for direct transformation of a height map into filtered perturbed surface normals that have C1 continuity. These normals can subsequently be used for various rendering purposes such as per-pixel lighting. In particular, we have devised a method which, by re-using colour texture filtering hardware that is ubiquitous in today's graphics systems in a new way with the addition of some small processing units, achieves the functions needed to compute the normal from a smooth surface controlled by a set of heights. Thus the data can be generated substantially in real time.
The filtered surface normals are created ‘on demand’ and are not stored. This provides the joint benefits of reducing the amount of texture data and bandwidth needed for bump mapping, as well as overcoming some of the issues with the filtering of normal maps. This feature is also important when using dynamic height maps in real-time rendering since a pre-processing step may be prohibitive.
Embodiments of the invention keep the advantages of computing bump map-based shading in local tangent space as described by Peercy et al, (although it is not restricted to doing so), with the convenience of directly using Blinn's height map but with the option of using a function with higher continuity.
Preferred embodiments of the invention will now be described in detail by way of example with reference to the accompanying diagrams in which:
The preferred embodiment will now be described. Access is provided to height map textures, which store an array of height values, using preferably 4 or 8 bits per texel. Each value will encode a fixed-point number with some number of fraction bits—preferably ź of the bits will be assigned to the fractional part.
The embodiment fits a bi-quadratic B-spline through this set of points, thus giving the virtual height map texture C1 continuity (i.e. continuous first derivatives). In
The manner in which the normal is computed is now described with reference to
A modified texel fetch unit, ‘151’, which in
For brevity, these have be renumbered a, b, etc.
It will be apparent to those skilled in the art that, with application of the address-bit interleaved texture storage format described in our British patent number GB2297886, such a height-map can be packed into the ‘equivalent’, in terms of storage, of a colour texture of ˝×˝ resolution of the height map. Each 2×2 group of scalar height data would occupy the space of a single four-dimensional colour. With such a format, the height map data can then be accessed using a very simple modification of exactly the same fetch mechanism used by units ‘60’ thru ‘63’ in
The 3×3 grid of samples is then fed into the ‘Replicate’ unit, 152, which outputs values to the Red, Green, Blue, and Alpha bilinear units. In particular, the Red channel receives the top left grid of 2×2 scalar values, i.e. those fetched from . . .
. . . while similarly the green channel receives the top right set, the blue, the bottom left, and the alpha receives the bottom right. Clearly some values, such as b′ or e′, will be used more than once, thus the grids supplied to each unit overlap at least partially.
Unit 153 takes the blend factors, ‘55’, and computes new sets of U and V blends as follows:
As Ublend and Vblend are typically fixed point numbers, it should be appreciated that these ‘calculations’ are completely trivial and incur no cost at all in hardware.
These new blend values are distributed to bilinear units, ‘65’ thru ‘68’ as follows:
Red: (Ublend0, Vblend0)
Green: (Ublend1, Vblend0)
Blue: (Ublend0, Vblend1)
Alpha: (Ublend1, Vblend1)
This manipulation of the blend factors eliminates the need to convert from the quadratic B-spline control points to the Bezier control points, as described previously in Equation 1. These bilinear units therefore effectively produce data which will enable surface normals with C1 continuity to subsequently be derived.
The results of the 4 bilinear interpolations are fed to the tangent construction unit, 155. This generates two tangent vectors, Tang1 and Tang2, which are functionally equivalent to using the following calculations:
Tang1 [X]:=1 Texturesize;
Tang1 [Y]:=LinearInterpolate(VBlend, GreenResult−RedResult, AlphaResult−BlueResult);
Tang2 [Y]:=LinearInterpolate(UBlend, BlueResult−RedResult, AlphaResult−GreenResult);
Tang2 [Z]:=1 Texturesize
. . . where
LinearInterpolate (x, A, B) :=A+x*(B−A);
For reasons that will soon be apparent, unit 155 actually only outputs three values: Tang1[y], Tang2[y] and 1 Texturesize.
Finally, in unit ‘156’, the cross product of these tangents is computed. It should be noted that if the preferred embodiment is chosen, the presence of zeros in the tangent components simplifies the cross product to the following calculation:
N [x]:=Tang1 [y];
N [y]:=1 Texturesize
N [z]:=Tang2 [y];
This vector is then normalised, preferably by squaring the N vector, computing the inverse of the square root of the result, and multiplying that scalar by the original components. The normalisation step may appear expensive, but it would be a requirement of any system that supported compressed normal maps, such as that described in WO9909523 or British patent application No. 0216668.4. Thus, if such texture formats were already supported, the re-normalisation hardware would be reused. An example of the output of this embodiment is shown in
In an alternative embodiment, trilinear filtering can be adapted to support normal generation—the only difference in procedure will be that the values fed into tangent construction unit ‘155’ will be a ‘blend’ of the values computed from two adjacent MIP map levels chosen. Other embodiments supporting improved anisotropic filtering are also feasible.
In another embodiment, an interpolated scale factor may be applied to the deltas/tangents before normalisation so that a height map can be scaled differently for different models or different parts of the same model.
In another embodiment, the blend factor adjust unit, 153, is not used and the B-spline control points are converted to the equivalent Bezier representations according to Equation 1 in a modified ‘152’ unit.
In another embodiment, the actual interpolated height value would be computed by including a third linear blending operation.
In another embodiment, Blinn's height interpolation function could be employed. In this embodiment, the blend factor adjust unit, 153, is not used and it is unnecessary to use the bilinear ‘alpha’ channel. That also implies that it is unnecessary to fetch source texel ‘k’. The tangent unit, 155, then simplifies to compute the difference of ‘green’ and ‘red’ and the difference of ‘blue’ and ‘red’.
In another alternative embodiment, dedicated sampling hardware could be included that takes numerous texture samples and applies an alternative derivative filter such as 4 taps, Sobel, Prewitt, Parks-McClellan derivatives filters as represented in
In another embodiment, colour textures are also filtered using bi-quadratic B-splines, either through the addition of bilinear filtering units, or by iterations through the colour channels, whereby the individual weights to the bilinear units are adjusted according to the previously described embodiments.
|Cited Patent||Filing date||Publication date||Applicant||Title|
|US5949424||28 Feb 1997||7 Sep 1999||Silicon Graphics, Inc.||Method, system, and computer program product for bump mapping in tangent space|
|US6765584 *||14 Mar 2002||20 Jul 2004||Nvidia Corporation||System and method for creating a vector map in a hardware graphics pipeline|
|US6850244 *||11 Jan 2001||1 Feb 2005||Micron Techology, Inc.||Apparatus and method for gradient mapping in a graphics processing system|
|JPH09231402A||Title not available|
|WO1999009523A1||19 Aug 1998||25 Feb 1999||Simon James Fenney||Shading three dimensional computer graphics images|
|1||*||"What is texel?-a definition from WhatIs.com", http://whatis.techtarget.com/definition/0,,sid9-gci214519,00.html#. Last updated on: Apr. 5, 2005. Accessed on: Nov. 10, 2008.|
|2||*||"What is texel?—a definition from WhatIs.com", http://whatis.techtarget.com/definition/0,,sid9—gci214519,00.html#. Last updated on: Apr. 5, 2005. Accessed on: Nov. 10, 2008.|
|3||*||Bikker, Jacco. "Bilinear Filtering (Interpolation)." Jan. 13, 1999. http://www.flipcode.com/articles/article-bilinearfiltering.shtml. Accessed Mar. 10, 2005.|
|4||*||Bikker, Jacco. "Bilinear Filtering (Interpolation)." Jan. 13, 1999. http://www.flipcode.com/articles/article—bilinearfiltering.shtml. Accessed Mar. 10, 2005.|
|5||Blinn, James F., Simulation of Wrinkled Surfaces, Caltech/JPL, SIGGRAPH 1978, pp. 286-292.|
|6||Ernst, I. et al., "Hardware-Supported Bump Mapping", Computers and Graphics,, Pergamom Press Ltd., Oxford, Great Britain, vol. 20, No. 4, (Jul. 1, 1996), pp. 515-521.|
|7||Farin, Curves and Surfaces for CAGD, A practical guide, pp. 33-35.|
|8||*||Hast et al. "Reconstruction Filters for Bump Mapping" Journal of WSCG Univ. West Bohemia Czech Republic, Feb. 4, 2002-Feb. 8, 2002, pp. 9-12, XP002299248, ISSN: 1213-6972.|
|9||Hast, A. et al. Reconstruction Filters for Bump Mapping, Journal of WSCG Univ. West Bohemia Czech Republic, Feb. 4, 2002-Feb. 8, 2002, pp. 9-12.|
|10||International Search Report, with Written Opinion of the International Searching Authority, mailed Oct. 20, 2004 in PCT International Application No. PCT/GB2004/001315 (11 pages).|
|11||Loe, K.F., "A sinusoidal polynomial spline and its Bezier blended interpolant", Journal of Computational and Applied Mathematics, vol. 71 (Jul. 27, 1996), pp. 383-393.|
|12||*||Loop et al., "Generalized B-spline surfaces of arbitrary topology", Proceedings of the 17th annual conference on Computer graphics and interactive techniques, ACM 1990, pp. 347-356.|
|13||Malzblender, T. et al., "Polynomial Texture Maps", Computer Graphics Proceedings, Siggraph, New York, New York: ACM, U.S., Aug. 12, 2001, pp. 519-528.|
|14||Mann, Stephen, et al, Computing Values and Derivatives of Bézier and B-spline Tensor Products, CAGD, vol. 12, No. 1, Feb. 1995.|
|15||Peercy, Mark, at al, Efficient Bump Mapping Hardware, Silicon Graphics Computer Systems, SIGGRAPH 1997, pp. 303-306.|
|16||*||Peters, Jörg. "Biquartic Cl-surface splines over irregular meshes." Computer-Aided Design. vol. 27, No. 12, p. 895-903. Jan. 26, 1995. Date accessed Aug. 4, 2006.|
|17||U.S. Appl. No. 10/621,111, filed Jul. 2003.|
|18||United Kingdom Patent Office Application No. 0308737.6, Search Report, Sep. 2003.|
|19||Williams, Lance, Pyramidal Parametrics, Computer Graphics, vol. 7, No. 3, Jul. 1983, pp. 1-11.|
|Citing Patent||Filing date||Publication date||Applicant||Title|
|US7973705 *||17 Jul 2009||5 Jul 2011||Garmin Switzerland Gmbh||Marine bump map display|
|US20110012773 *||20 Jan 2011||Garmin Ltd.||Marine bump map display|
|U.S. Classification||345/584, 382/263, 345/582, 345/586, 345/609, 345/428, 382/264, 382/260, 345/426, 345/643|
|International Classification||G06T15/04, G09G5/00|