Flash makes it pretty easy to use any DisplayObject as a Stage3D texture. This is a great feature since you can use powerful, traditional classes like MovieClip, Sprite, TextField, and Shape to build a texture—often with vector graphics—and then use Stage3D‘s GPU hardware acceleration to render them with maximum performance. But this path is fraught with subtle problems, any one of which could result in poor rendering quality that’s quite hard to debug. Today’s article takes you through the process step by step to make sure you end up with great results.

First off, here’s a high level view of the steps you must take to transform your DisplayObject into a Stage3D texture:

  1. Create vector graphics (e.g. explicit Shape objects, assets from Flash Pro)
  2. Convert to BitmapData using BitmapData.draw
  3. Create and upload to a Scene3D Texture
  4. Render triangles with a fragment shader that samples the texture

As you know, vector graphics are mathematically defined and, for all intents and purposes, of “perfect” quality at any rotation or scale. So you won’t have quality problems in step #1.

In step #2, BitmapData.draw has one parameter affecting quality: smoothing. It defaults to false, so setting it to true can really help in some cases. There is also a non-parameter that affects quality: Stage.quality. So if your Stage quality is low, you’ll get a lower quality rendering to the BitmapData. An alternative to changing the entire Stage‘s quality is to use BitmapData.drawWithQuality. This has been around since Flash Player 11.3 and AIR 3.3, so it may require you to target a higher version of Flash Player than you’re already using.

Step #3 involves creating a texture which can also cause the quality to be lower. The format parameter to Context3D.createTexture can be Context3DTextureFormat.BGRA for full quality, but the other formats all involve lower quality. The BGRA_PACKED and BGR_PACKED formats only use 16 bits per pixel and the COMPRESSED and COMPRESSED_ALPHA use lossy texture formats (PVR, ETC, and DXT) via the ATF container format.

Lastly, step #4 is where your fragment shader samples the texture on a per-pixel basis. There are two possible problems in this step. First, the point you’re sampling on the texture could be imprecise if, for example, integer clamping was applied at some phase. Second, textures are sampled using a variety of texture flags. The most important flags in this case are the ones controlling the filtering mode: nearest and linear. The nearest flag will result in a much blockier output since there is no smoothing applied, but you can guarantee that only colors in the texture will be used.

If you’re not sure at which step your problem is occurring, I’d recommend a guess-and-check approach until you’ve found the culprit. Here are the parameters:

  1. Turn on smoothing and increase the Stage.quality for your BitmapData.draw. If you can require Flash Player 11.3, switch to BitmapData.drawWithQuality instead and set smoothing to true and quality to BEST.
  2. Ensure your Context3D.createTexture is using the BGRA format instead of any of the others.
  3. Ensure your fragment shader texture sampling flags include linear and not nearest

You may need to apply more than one of these techniques in order to solve the quality issue, but you want to apply as few of them as possible in order to keep performance high. Technique #1 will only affect the time required to create the textures, which is hopefully an infrequent process. Technique #2 will require more VRAM and more upload time than the other formats. The former will become an issue if you’re using lots of textures but the latter is only a concern at texture creation time. Finally, technique #3 will affect the drawing of every single pixel on every single frame. Luckily, most modern GPUs are so fast that the difference between linear and nearest-neighbor sampling is essentially non-existent.