Warp3D: 2048x2048 texture / W3D_DrawArray problem

A forum for general AmigaOS 4.x support questions that are not platform-specific
User avatar
Daytona675x
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 34
Joined: Wed Jan 22, 2014 5:18 pm
Location: Cologne, Germany
Contact:

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by Daytona675x »

@Karlos
If P96/Graphics AllocBitMap() functions returned VRAM resident allocations that were already texture-friendly aligned for the target GPU, *none* of this complex allocation strategy would be necessary
Yes, although I still don't see the reason for the "complex" part here.
Allocating a BitMap doesn't even guarantee that the area you allocate is even in VRAM, let alone aligned appropriately. If you just allocate a BitMap with these functions, you'll get a buffer in system memory that is copied to a VRAM allocation if/when P96 deems it appropriate to do so.
So it may be that all your realign fun is useless because at the end what you got was normal RAM?
So, a VRAM allocator that sat "on top" of all this was written and it is generally quite efficient. It allocates as few BitMaps as possible and re-uses memory within them for many textures.
Efficient maybe, but something in this card-house is obviously flawed.
In your case, allocating large (1024x1024 and higher) textures is *always* going to push the allocator into requesting a new BitMap.
Which should be no problem, that's its job :-)
1) The allocated BitMap is in VRAM
So do you have control over that or not? From your sentence above it sounded like it's a matter of luck: sometimes your mem is in VRAM, sometimes not.
2) The allocated BitMap is not excessively rectangular (in order not to upset the graphics subsystem which has width/height restrictions too)
So what's the concrete limits here?
3) The allocated bitmap's total linear size is *at least* as large as the requested allocation plus whatever padding is required for texture alignment requirements which P96/graphics generally know nothing about.
Yes. Which means: a few extra bytes.
Ideally, there should be some "allocate VRAM aligned to my exact requirements " function exposed by the graphics sub system that W3D could use, but instead we have BitMaps.
Any why not add such a function?
When allocating your 2048x2048x32 bit texture I expect that it should fail almost always, if not actually always because the padding alone will probably cause BitMap dimensions larger than 2048x2048.
Yes, yes. If you have to realign such a 2048x2048 texture and if the only way to allocated RAM is by using a bitmap-function which has a limit at 2048x2048, well...
Then you could try to request buffers from it multiple times until you get a well-aligned buffer ;-) Hm, actually: this doesn't sound much worse than other strategys...

But how do you explain 2048x1024 / 1024x2048 failures? As being said: those fail in pretty much exactly the same way. Your explanations don't cover those. From what I see I'd guess the resize-algorithm is flawed, at least that could explain it. What's your thought on that? Where will you start digging?
Warp3D driver code-basher and bug-smasher - btw.: driver writing is nothing mysterious
User avatar
Karlos
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 84
Joined: Sun Jun 19, 2011 11:42 am
Location: United Kingdom of England and anybody else that wishes to remain.

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by Karlos »

Daytona675x wrote:@Karlos

So it may be that all your realign fun is useless because at the end what you got was normal RAM?
No. Please re-read the above posts. The allocator function takes your Context's display bitmap and tells P96/Graphics to use it as a friend bitmap when allocating. This ensures the bitmap is created in VRAM (otherwise it fails) *but* causes you to have no control *at all* on the pixel format/alignment. Typically the alignment it has is not sufficient for texture usage, hence the "alignment fun" is required. The allocator takes into account the anticipated depth of the BitMap we will receive and computes "as square as possible but still aligned to a sensible width" dimensions for it in order to get a BitMap that will have our requested size within it at the appropriate hardware alignment for texture mapping.

All this is basically a hacky work around for the fact that there's not (or at least wasn't at the time of writing) a way to do the following:
Ideally, there should be some "allocate VRAM aligned to my exact requirements " function exposed by the graphics sub system that W3D could use, but instead we have BitMaps.
Any why not add such a function?
Aside from the Permedia2 RTG driver, I've not worked on any aspect of P96 or graphics. That's a question for the respective maintainers.
User avatar
Daytona675x
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 34
Joined: Wed Jan 22, 2014 5:18 pm
Location: Cologne, Germany
Contact:

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by Daytona675x »

@Karlos
No. Please re-read the above posts...
Yes, I know, my question came from the fact that you said this:
If you just allocate a BitMap with these functions, you'll get a buffer in system memory that is copied to a VRAM allocation if/when P96 deems it appropriate to do so.
which should be of no real interest to the topic if you make sure to get VRAM always.
Aside from the Permedia2 RTG driver, I've not worked on any aspect of P96 or graphics. That's a question for the respective maintainers.
Yes, of course. Realistic chances that somebody will take care?

But you skipped the most interesting part. What about those two questions?
But how do you explain 2048x1024 / 1024x2048 failures? As being said: those fail in pretty much exactly the same way. Your explanations don't cover those. From what I see I'd guess the resize-algorithm is flawed, at least that could explain it. What's your thought on that? Where will you start digging?
Warp3D driver code-basher and bug-smasher - btw.: driver writing is nothing mysterious
User avatar
Hans
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 703
Joined: Tue Dec 21, 2010 9:25 pm
Location: New Zealand
Contact:

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by Hans »

Daytona675x wrote:
If only things were so simple. Picasso96 doesn't like to share VRAM with anything else, so Warp3D has to pull a few tricks. Without going into too much detail, Warp3D usually allocates blocks of VRAM at a time, and then stores multiple smaller textures in one block. Due to the way that things are set up, it has to lock these allocations in place, or the texture base pointers will become invalid if Picasso96 chooses to perform paging/defragging (which could happen at any time). Unfortunately, this also means that Picasso96's defragging becomes less effective, because there are large blocks of VRAM locked in place.
This issue happens any time. I'd not expect 128 MB VRAM / 2GB RAM to be fragmented that much, especially right after boot, that this allocation wouldn't be possible anymore. As further tests revealed it even happens with 16bit textures too. There are some interesting test results below that reveal that it sometimes happens even if only about 4 MB tex-data is used for a texture. That's such a low amount compared to what's available, it just shouldn't fail.
Ooh, but Picasso96 loves to fragment memory, and makes no attempt to defrag until it a VRAM allocation fails. Also, don't forget that your Workbench screen & icons also take up memory; a high resolution screen with backdrop image, compositing effects enabled, and a bunch of dockies enabled can easily chew through 50 MB or so of VRAM, allocated goodness knows where. I'm not saying that it's definitely fragmentation, but when you're attempting to allocate 12.5% of total VRAM as one block then it's a definite possibility (and that rises to 20.5% if 50 MB is already allocated).

Back when I still used an A1-XE with a Radeon 9000 pro, I sometimes had to disable compositing to be able to play certain games at an acceptable speed. Otherwise the texture paging would kick in, and drag the speed right down.
Daytona675x wrote: 4. it also happens with 2048x1024 and 1024x2048 at both 32bit and 16bit (! so it fails even at about low 4 MB tex-data).

5. it works with 2048x512 and 512x2048 at both 16bit and 32bit (! so it can also work if about 4 MB tex-data).
This behaviour doesn't make sense. I'm nowhere near as familiar with the Warp3D internals as Karlos, but my impression was that the allocator doesn't care in the slightest what the width and height are; only the total allocation size matters. So, one 4MB texture should have as much chance of successful allocation as any other 4MB texture, regardless of the dimensions (subject to texture sizing constraints, of course).

Hans
http://hdrlab.org.nz/ - Amiga OS 4 projects, programming articles and more. Home of the RadeonHD driver for Amiga OS 4.x project.
User avatar
Karlos
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 84
Joined: Sun Jun 19, 2011 11:42 am
Location: United Kingdom of England and anybody else that wishes to remain.

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by Karlos »

Hans wrote:
Daytona675x wrote: 4. it also happens with 2048x1024 and 1024x2048 at both 32bit and 16bit (! so it fails even at about low 4 MB tex-data).

5. it works with 2048x512 and 512x2048 at both 16bit and 32bit (! so it can also work if about 4 MB tex-data).
This behaviour doesn't make sense. I'm nowhere near as familiar with the Warp3D internals as Karlos, but my impression was that the allocator doesn't care in the slightest what the width and height are; only the total allocation size matters. So, one 4MB texture should have as much chance of successful allocation as any other 4MB texture, regardless of the dimensions (subject to texture sizing constraints, of course).

Hans
Correct. It does seem like there is a bug based on Daytona's tests but it it's hard to rule out vram fragmentation, especially if this is in a compositing environment. I didn't write the bitmap allocator code (above process descriptions are from memory) but did tune it to minimise fragmentation on the lowest end systems. It really shouldn't matter what a given allocation is for. Two 4 Meg allocations should work, whether one is for a 32 bit texture or the other a 16 bit one for a texture of twice the area.

The allocator has no idea what the dimensions of the texture are as all it is given is the final byte size requirement. If larger allocations are failing regularly but sometimes work, I would suspect fragmented vram. Lots of paging of bitmaps that still doesn't give a large enough contiguous block.

As I don't presently have access to an OS 4 system, much less a working R200 based one, all I can do is conjecture.
kas1e
Beta Tester
Beta Tester
Posts: 542
Joined: Sat Jun 18, 2011 7:56 am
Contact:

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by kas1e »

@Karlos
As I don't presently have access to an OS 4 system, much less a working R200 based one, all I can do is conjecture.
Are you still want to have only classic amigaos4 system, or, for the sake of development can go for sam or something ? I mean, we for sure can help with it if you want to spend some hours on all this.
User avatar
Karlos
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 84
Joined: Sun Jun 19, 2011 11:42 am
Location: United Kingdom of England and anybody else that wishes to remain.

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by Karlos »

kas1e wrote:@Karlos
As I don't presently have access to an OS 4 system, much less a working R200 based one, all I can do is conjecture.
Are you still want to have only classic amigaos4 system, or, for the sake of development can go for sam or something ? I mean, we for sure can help with it if you want to spend some hours on all this.
I must confess, I might be getting the itch again. My problem now is that I have so many more important responsibilities that there's no time to use even my classic. Assuming I can even find it in what was supposed to be my "office" :(
User avatar
Karlos
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 84
Joined: Sun Jun 19, 2011 11:42 am
Location: United Kingdom of England and anybody else that wishes to remain.

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by Karlos »

@Daytona

I might have found something. Please check your PM.
User avatar
Karlos
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 84
Joined: Sun Jun 19, 2011 11:42 am
Location: United Kingdom of England and anybody else that wishes to remain.

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by Karlos »

Apologies for the glacial delay. Today is the first time in ages I've had any time work on personal stuff.

There is a debug build of the W3D_Picasso96.library (53.11) available at the following location:

http://extropia.co.uk/warp3d/

Username: warp3dbeta

Password by PM to anybody that wants it for testing texture allocation issues in Warp3D/MiniGL.

@Daytona - already PM'd you.
User avatar
Karlos
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 84
Joined: Sun Jun 19, 2011 11:42 am
Location: United Kingdom of England and anybody else that wishes to remain.

Re: Warp3D: 2048x2048 texture / W3D_DrawArray problem

Post by Karlos »

Just a note to add, a few people have contacted me using X1000 systems.

If you are using a Warp3D supported RadeonHD card, this updated W3D_Picasso96.library will not be of any use to you as the memory management routines it provides are not used.

This update is strictly for R100, R200, Voodoo and Permedia based systems.
Post Reply