Using ATF Is Harder Than You Might Think
Using Adobe’s new compressed texture format should be as simple as replacing some PNG and JPEG images with ATFs their tools created, but it’s not. If you don’t know what you’re doing, the process can be pretty confusing. Today’s article walks you through the steps to upgrade a Stage3D
-using app to make use of ATF textures.
As mentioned in the intro, the first steps are pretty straightforward:
- Convert some PNG files to ATF
- Replace your
Texture.uploadFromBitmapData
calls toTexture.uploadCompressedTextureFromByteArray
- Target Flash Player 11.4’s playerglobal.swc
- Replace your old version of Adobe’s
AGALMiniAssembler
class with the new one
If you get this error, you forgot to target Flash 11.4:
ArgumentError: Error #3677: Texture decoding failed. Internal error. at flash.display3D.textures::Texture/uploadCompressedTextureFromByteArray() at YourApp/yourFunction()
But even after correctly following all those steps you’ll get this error:
Error: Error #3606: Sampler 0 format does not match texture format. at flash.display3D::Context3D/drawTriangles() at YourApp/yourFunction()
This means that we missed a step which has no example in Adobe’s introductory post:
- Modify your AGAL fragment shader code to specify what kind of texture you’re sampling
You must specify “dxt1” for opaque ATF textures and “dxt5” for transparent ATF textures. Don’t worry about the specific compression algorithm being called out here since these are really just synonyms; the correct texture compression algorithm will still be used depending on the device your app is running on. Here’s what a simple fragment shader used to look like:
tex oc, v0, fs0 <2d,linear,mipnone,clamp>
And here’s what it needs to look like when we use ATF textures (in this case an opaque one):
tex oc, v0, fs0 <2d,linear,mipnone,clamp,dxt1>
You don’t need to specify anything if you’re still using uncompressed BitmapData
-sourced textures. However, it seems that you can specify “dxt1” and still sample from an uncompressed texture. Will this work on every possible device? I don’t know, but it sure does work on my Mac OS X 10.8 running an Intel HD Graphics 4000 or an NVIDIA GeForce GT 650M. If you can can get away with always specifying it, that may simplify your shaders somewhat. You won’t need to create two versions of each shader: one for ATF textures and one for uncompressed textures.
To demonstrate the final product, I’ve made a small (<200 lines) test app that simply displays a texture flat on the stage:
package { import flash.display3D.Context3DTextureFormat; import flash.display.BitmapData; import flash.display3D.IndexBuffer3D; import flash.display3D.VertexBuffer3D; import flash.display3D.Program3D; import flash.display3D.textures.Texture; import flash.utils.ByteArray; import com.adobe.utils.AGALMiniAssembler; import flash.display3D.Context3DProgramType; import flash.display3D.Context3DVertexBufferFormat; import flash.display.Bitmap; import flash.display.Sprite; import flash.display.Stage3D; import flash.display.StageAlign; import flash.display.StageScaleMode; import flash.display3D.Context3D; import flash.display3D.Context3DRenderMode; import flash.events.Event; import flash.events.MouseEvent; import flash.text.TextField; import flash.text.TextFieldAutoSize; import flash.text.TextFormat; public class UsingATF extends Sprite { private static const PAD:Number = 5; [Embed(source="earth.jpg")] private static const TEXTURE_BITMAPDATA:Class; [Embed(source="earth.atf", mimeType="application/octet-stream")] private static const TEXTURE_ATF:Class; private var program:Program3D; private var posUV:VertexBuffer3D; private var tris:IndexBuffer3D; private var textureBitmapData:Texture; private var textureATF:Texture; private var curTexture:Texture; private var context3D:Context3D; private var modeDisplay:TextField; public function UsingATF() { stage.align = StageAlign.TOP_LEFT; stage.scaleMode = StageScaleMode.NO_SCALE; stage.frameRate = 60; var stage3D:Stage3D = stage.stage3Ds[0]; stage3D.addEventListener(Event.CONTEXT3D_CREATE, onContextCreated); stage3D.requestContext3D(Context3DRenderMode.AUTO); } protected function onContextCreated(ev:Event): void { var stage3D:Stage3D = stage.stage3Ds[0]; stage3D.removeEventListener(Event.CONTEXT3D_CREATE, onContextCreated); context3D = stage3D.context3D; context3D.configureBackBuffer(stage.stageWidth, stage.stageHeight, 0); context3D.enableErrorChecking = true; makeButtons("BitmapData", "ATF"); var assembler:AGALMiniAssembler = new AGALMiniAssembler(); assembler.assemble( Context3DProgramType.VERTEX, "mov op, va0\n" + "mov v0, va1" ); var vertexProgram:ByteArray = assembler.agalcode; assembler.assemble( Context3DProgramType.FRAGMENT, "tex oc, v0, fs0 <2d,linear,mipnone,clamp,dxt1>" ); var fragmentProgram:ByteArray = assembler.agalcode; program = context3D.createProgram(); program.upload(vertexProgram, fragmentProgram); posUV = context3D.createVertexBuffer(4, 5); posUV.uploadFromVector( new <Number>[ // X, Y, Z, U, V -1, -1, 0, 0, 1, -1, 1, 0, 0, 0, 1, 1, 0, 1, 0, 1, -1, 0, 1, 1 ], 0, 4 ); // Create the triangles index buffer tris = context3D.createIndexBuffer(6); tris.uploadFromVector( new <uint>[ 0, 1, 2, 2, 3, 0 ], 0, 6 ); var bmd:BitmapData = (new TEXTURE_BITMAPDATA() as Bitmap).bitmapData; textureBitmapData = context3D.createTexture( bmd.width, bmd.height, Context3DTextureFormat.BGRA, false ); textureBitmapData.uploadFromBitmapData(bmd); var atfBytes:ByteArray = new TEXTURE_ATF() as ByteArray; textureATF = context3D.createTexture( bmd.width, bmd.height, Context3DTextureFormat.COMPRESSED, false ); textureATF.uploadCompressedTextureFromByteArray(atfBytes, 0); curTexture = textureBitmapData; modeDisplay = new TextField(); modeDisplay.autoSize = TextFieldAutoSize.LEFT; modeDisplay.defaultTextFormat = new TextFormat("_sans", 36); modeDisplay.text = "BitmapData"; addChild(modeDisplay); addEventListener(Event.ENTER_FRAME, onEnterFrame); } private function makeButtons(...labels): Number { var curX:Number = PAD; var curY:Number = stage.stageHeight - PAD; for each (var label:String in labels) { var tf:TextField = new TextField(); tf.mouseEnabled = false; tf.selectable = false; tf.defaultTextFormat = new TextFormat("_sans"); tf.autoSize = TextFieldAutoSize.LEFT; tf.text = label; tf.name = "lbl"; var button:Sprite = new Sprite(); button.buttonMode = true; button.graphics.beginFill(0xF5F5F5); button.graphics.drawRect(0, 0, tf.width+PAD, tf.height+PAD); button.graphics.endFill(); button.graphics.lineStyle(1); button.graphics.drawRect(0, 0, tf.width+PAD, tf.height+PAD); button.addChild(tf); button.addEventListener(MouseEvent.CLICK, onButton); if (curX + button.width > stage.stageWidth - PAD) { curX = PAD; curY -= button.height + PAD; } button.x = curX; button.y = curY - button.height; addChild(button); curX += button.width + PAD; } return curY - button.height; } private function onButton(ev:MouseEvent): void { var mode:String = ev.target.getChildByName("lbl").text; switch (mode) { case "BitmapData": curTexture = textureBitmapData; modeDisplay.text = "BitmapData"; break; case "ATF": curTexture = textureATF; modeDisplay.text = "ATF"; break; } } private function onEnterFrame(ev:Event): void { context3D.clear(0.5, 0.5, 0.5); context3D.setProgram(program); context3D.setTextureAt(0, curTexture); context3D.setVertexBufferAt(0, posUV, 0, Context3DVertexBufferFormat.FLOAT_3); context3D.setVertexBufferAt(1, posUV, 3, Context3DVertexBufferFormat.FLOAT_2); context3D.drawTriangles(tris); context3D.present(); } } }
Launch the test app
Download the test app (source code, textures, binaries)
Spot a bug in the article? Have a question or suggestion? Post a comment!
#1 by i.o. on December 3rd, 2012 ·
“…,dxt1>” or “…,dxt5>” — it was helpful. Thank you ;)
#2 by heweitykc on December 5th, 2012 ·
is 11.4 needed?
#3 by jackson on December 5th, 2012 ·
Yes, that’s step #3 in the article and the fourth limitation listed by Adobe.
#4 by Mark on December 7th, 2012 ·
Im getting an error..
I’m using WIN 11,4,402,265 (Debug player)
#5 by jackson on December 7th, 2012 ·
Hmm, I haven’t seen that one. It would seem to indicate that the texture creation or uploading failed. Is it only with the compressed one or with uncompressed too? Any other errors?
#6 by ben w on December 7th, 2012 ·
Error: Error #3663: Sampler 0 binds an undefined texture.
at flash.display3D::Context3D/drawTriangles()
at UsingATF/onEnterFrame()
Same thing :(
11.5 here
#7 by jackson on December 11th, 2012 ·
I tried it again and was getting that same error for some reason. I rebuilt the SWF and everything seemed OK. I’m not sure why the problem was happening, but I’ve updated the demo app and ZIP file. If you try it again and it still doesn’t work, let me know.
#8 by Mark on December 14th, 2012 ·
Ha, it works, nice!
#9 by benjamin guihaire on June 22nd, 2013 ·
using ATF textures was definitely harder than I thought!
*atf multi platform
Another difficulty with ATF.. if you target multiple platforms, such as Android, iOS, and Browser, you don’t want to package ATF texutre containing all 3 formats (too big!), which means you need to package differently for the 3 platforms to only provide what is needed , and create 3 ATF , one for each platform, which makes the tooling side a bit more complex.
*hit test transparency:
in the game I was working on, we need hit test detection when we click on a 3d model, and to do that, we didn’t want to detect hit test on transparent pixels… so we have to find where in the triangle the mouse click happened, then find the UVs, and finally do a lookup in the texture to know if we hit a transparent pixel or not… which made the use of ATF texture at first impossible. okay, ATF are internally DXT on a browser, so I guess it would be possible to know if we hit a transparent pixel or not by decoding the byteArray, but on iOS or Android, the formats are different, making it a very difficult problem to solve.
The solution I took, is to add at the end of the ByteArray containing the ATF, additionnal bytes containing a bit-array of the transparent pixels info. So now we can use ATF and still do precise hit test… so when we need to do hit test detection, we look in the ATF byteArray , skip the ATF bytes to find our transparent info.. Luckly, ATF format contains the size, so we can easily skip that part and look just after it.
ATF file format used to be described in that post: http://www.bytearray.org/?p=4472
So yes, definitely switching to ATF was harder than planned!
Benjamin Guihaire.
#10 by jackson on June 22nd, 2013 ·
Excellent notes the process. And the bit-array sounds like a tidy solution for the hit-testing problem.
#11 by zeke on June 26th, 2013 ·
i got an error in debug mode:
Error: Error #3606: Sampler 0 format does not match texture format.
bug when i use the run mode the eath pic is show
#12 by Jeff on June 26th, 2013 ·
FYI, I got Error #3677 (even using -swf-version=17, aka targetting Flash 11.4) when I accidentally encoded a non-square PNG using png2atf. None of the tools in the workflow warned me, go figure.
Actually, as a “decoding failure”, I could imagine there could be a number of failure modes (incorrect encoding parameters, wrong compression types for the given device, etc).
ArgumentError: Error #3677: Texture decoding failed. Internal error.
at flash.display3D.textures::Texture/uploadCompressedTextureFromByteArray()
at YourApp/yourFunction()
Anyway, thanks for this useful post.
#13 by moosefetcher on January 13th, 2014 ·
Hello there. Thanks for the atf tips. I’ve just recently decided to switch to atf textures. Do you know if there’s a way of loading in the atfs as opposed to embedding them? I’m loading them with a URLLoader and that SEEMS to be working; I just can’t find how to pass the resulting .data property of the URLLoader into a ByteArray. Casting it ‘ByteArray(_loader.data)’ causes a type coercion error and using _loader.data as ByteArray causes the result to be null. Do you know what the solution is to this? Thanks.
#14 by jackson on January 14th, 2014 ·
You should definitely be able to load ATF textures from a URL. Flash doesn’t really care where you get the
ByteArray
. You should basically be able to follow this example (at the bottom of the page) with one tweak: setloader.dataFormat = URLLoaderDataFormat.BINARY
to ensure the file you load is treated as binary, not text.