Memory allocation

[xenic]

A question that has not yet been asked is *why* can't OS4 automatically unlock memory when it is freed? This would at least avoid a nasty "locked memory" leak. I can imagine that implementing this might increase memory overhead slightly, but that seems like a small price to pay.

Doing that would *not* remove the need for an improved API, but it would reduce the urgency of it.

I think maybe our concerns are overblown. I wrote a test program that checks the total available memory, allocates 3/4 of the available memory (without AVT_Lock, FALSE), frees it without unlocking the memory and then allocates half of the available memory as unlocked memory (AVT_Lock, FALSE). Apparently, failing to Unlock memory doesn't affect it's availability to the system. I'm guessing that it just affects virtual memory and swapping.

[chris]

A question that has not yet been asked is *why* can't OS4 automatically unlock memory when it is freed? This would at least avoid a nasty "locked memory" leak. I can imagine that implementing this might increase memory overhead slightly, but that seems like a small price to pay.

Doing that would *not* remove the need for an improved API, but it would reduce the urgency of it.

I think maybe our concerns are overblown. I wrote a test program that checks the total available memory, allocates 3/4 of the available memory (without AVT_Lock, FALSE), frees it without unlocking the memory and then allocates half of the available memory as unlocked memory (AVT_Lock, FALSE). Apparently, failing to Unlock memory doesn't affect it's availability to the system. I'm guessing that it just affects virtual memory and swapping.

Did you allocate it as MEMF_SHARED though? Earlier it was said that only MEMF_SHARED memory would be locked by default. (most people won't be using this)

[ssolie]

Apparently, failing to Unlock memory doesn't affect it's availability to the system. I'm guessing that it just affects virtual memory and swapping.

I don't think you understand what page locking actually is.

Is the wiki really that bad or is this just an understanding problem?

[nbache]

Is the wiki really that bad or is this just an understanding problem?

It's not *that* bad, but I'd still suggest some resequencing. For instance:

Locking System Memory
The LockMem() function is used to explicitly lock a memory block. This function will make sure your memory block is not unmapped, swapped out or somehow made inaccessible. Use this function wisely. If there is no good reason to lock memory then do not do it. It will prevent the memory system from optimizing memory layout and may lead to poor performance.

Code: Select all

IExec->LockMem(mem, 1000);

Before deallocating the memory is must be unlocked as well.

Code: Select all

IExec->UnLockMem(mem, 1000);

By default, MEMF_SHARED and MEMF_EXECUTABLE memory is locked when allocated and must be explicitly unlocked before freeing. Always refer to the autodocs of any memory allocation functions to determine whether explicit unlocking is required.
Do not forget to UnlockMem()
Failure to unlock memory will have adverse affects on the memory system. Locked memory pages cannot be moved so the system cannot optimize the layout of locked memory pages.

(styles/layout reproduced as closely as possible).

All I did was move the first paragraph ("By default, ...") further down. I think this presents the concepts in a more logical sequence.

Best regards,

Niels

[xenic]

Did you allocate it as MEMF_SHARED though? Earlier it was said that only MEMF_SHARED memory would be locked by default. (most people won't be using this)

Yes it was MEMF_SHARED. You can try the same experiment yourself; it only takes a few minutes to write some code that allocates/deallocates memory.

[jaokim]

I'm still not very comfortable with this "definition":

This function will make sure your memory block is not unmapped, swapped out *or somehow made inaccessible*.

This is not a definition. I would call "somehow" a random event, which a computer system really can't do.

The memory is always *accessible*, just perhaps not at this precise clock cycle, which is why you want to use locked memory in interrupts and likewise. Just strike the "somehow made inaccessible" bit. Any driver developers should be assumed to have some more knowledge, and the ones that don't are not helped by the explanation.

My suggestion:

This function will make sure your memory block is not unmapped or swapped out. This is only useful during interrupts, device driver code or similar.

I don't mean that the wiki needs to be a newbie manual, but one shouldn't give novice developers the room to make any false guesses.

Furthermore, I do think there are some of us that don't exactly know what "page locking" means (especially since we can never lock or unlock *pages*, only memory, at least according to the name of the functions: Lock/UnlockMem).

This is how I understand it (I hope I'm conceptually correct, I haven't bothered with page size and stuff):
If we allocate and lock 16 Mbyte of memory, and then free it without unlocking, the system will get its 16 Mbyte back. However, the system cannot swap or move these 16 Mbyte. Nonetheless, these 16 Mbyte are available for any new allocations. Any 1 Mbyte allocations will eventually get a piece of these 16 Mbyte. But, now imagine the system running out of memory. The system tries to swap out memory, but the big 16 Mbyte chunk of memory cannot be swapped since it's locked, even though not all of the 16 Mbyte of it are used.

Did I get it right?

I think I made some misplaced quotes in the beginning of the thread which lead some to believe that locked memory can't be freed -- which it can.

Furthermore, I also think nbache's change suggestion is good.

[ChrisH]

I'm still not very comfortable with this "definition":

This function will make sure your memory block is not unmapped, swapped out *or somehow made inaccessible*.

This is not a definition. I would call "somehow" a random event, which a computer system really can't do.

I woul suggest replacing "somehow" with "otherwise". Then it would read:

This function will make sure your memory block is not unmapped, swapped out *or otherwise made inaccessible*.

That seems clearer to me, and still achieves the intended goal (which removing it entirely does not).