AmigaOneXE:
System: PPCMotorola MPC 7447/7457 Apollo/AltiVec(tm) (emulated MC68020/FPU) (INST: Cache) (External Cache)
Level 1 Cache size: 32768, Level 2 Cache size: 524288
L2CR: 80000000, enabled, partity check disabled, instruction & data, replacement algorithm: pseudo-random
MSSCR0: 00000000, L2 prefetch disabled (0 engines), 8 maximum outstanding data-bus transactions.
512MB RAM
Radeon9200 - 128MB RAM
I'll come up with a bold statement and would love if one of the devs can prove me wrong :-/
SWAP is definitely broken on OS4upd3 on at least an AmigaOneXE (my machine) and a Sam440ep
It doesn't matter what settings i use for SWAP...it simply don't work as it should or i understand, but maybe i'm expecting too much from a new feature?
I tried the settings described in the release notes (see also here for more info on my SWAP).
Just for fun i played around with SWAP and used 512, 1024, 2048, 4096 blocks...it simply doesn't matter, whenever SWAP is called it's bombing my system...ideally first bringing down USB (with the cursed "Unsolvable errors in UHCI..." requester) and then locking up Workbench
The first linked thread is originally about a game (Aquaria) and together with this game i have found a reproduceable way to crash my Workbench...again :-/
I have 512MB RAM, the game is ~222MB in size.
I copy the game into RAM Disk, leaving me with ~120MB free space available.
Normally the game would lock up randomly while playing, but now if one starts the game from RAM (or HD, doesn't matter) it will try to load in it's data and hopelessly lockup as soon as SWAP should kick in (for me it's about 50 to 75% of the games progress bar.
Please someone explain to me what do i have to change to make SWAP work and/or confirm this misbehavour.
I'm gladly sharing any more needed info
Thank you very much
EDIT: Tried from FFS, SFS and JXFS partitions
SWAP broken
SWAP broken
People are dying.
Entire ecosystems are collapsing.
We are in the beginning of a mass extinction.
And all you can talk about is money and fairytales of eternal economic growth.
How dare you!
– Greta Thunberg
Entire ecosystems are collapsing.
We are in the beginning of a mass extinction.
And all you can talk about is money and fairytales of eternal economic growth.
How dare you!
– Greta Thunberg
Re: [BUG] SWAP broken
To be honest I'm not too sure why people insist on testing SWAP by forcing a low memory situation (eg filling ram up with files so to reduce free memory - perhaps storage allocation is different from program utilisation?)
That out of the way, to clarify, you have setup SWAP as described in the manual (anything else has just been experimentation on your part trying to find a solution to your predictament) when you run into a low memory situation (as far as I can tell its forced by filling your RAM disk up?) and the software you are trying to actually run doesn't but crashes or causes problems in your OS?
Maybe SWAP doesn't work this way, perhaps you need enough memory to execute the main file first THEN any further memory is allocated properly... so if there isn't enough memory to setup the initial program it just fails...
For example is this game WB friendly? What if you didn't fill the ram disk first, ran the game then switched back to Workbench and filled the ram disk up, maybe the game would run then because the paging etc is allocating correctly... a game probably isn't a good example anyway since it could be doing funny stuff in the background...
Of course I could be completely wrong but I always try and look at the logical overview
But like Rigo commented in the other thread, something is coming out soon in relation to this?
p.s. also, when's the last time you checked your memory? and sorry if I my answer isn't exactly specific to everything you're saying - I'm just throwing a few ideas around...
That out of the way, to clarify, you have setup SWAP as described in the manual (anything else has just been experimentation on your part trying to find a solution to your predictament) when you run into a low memory situation (as far as I can tell its forced by filling your RAM disk up?) and the software you are trying to actually run doesn't but crashes or causes problems in your OS?
Maybe SWAP doesn't work this way, perhaps you need enough memory to execute the main file first THEN any further memory is allocated properly... so if there isn't enough memory to setup the initial program it just fails...
For example is this game WB friendly? What if you didn't fill the ram disk first, ran the game then switched back to Workbench and filled the ram disk up, maybe the game would run then because the paging etc is allocating correctly... a game probably isn't a good example anyway since it could be doing funny stuff in the background...
Of course I could be completely wrong but I always try and look at the logical overview
But like Rigo commented in the other thread, something is coming out soon in relation to this?
p.s. also, when's the last time you checked your memory? and sorry if I my answer isn't exactly specific to everything you're saying - I'm just throwing a few ideas around...
~Yes I am a Kiwi, No, I did not appear as an extra in 'Lord of the Rings'~
1x AmigaOne X5000 2.0GHz 2gM RadeonR9280X AOS4.x
3x AmigaOne X1000 1.8GHz 2gM RadeonHD7970 AOS4.x
1x AmigaOne X5000 2.0GHz 2gM RadeonR9280X AOS4.x
3x AmigaOne X1000 1.8GHz 2gM RadeonHD7970 AOS4.x
Re: [BUG] SWAP broken
Yes, it was pure exeperimentation on my part to probably find differences...there weren't any, sadly
All of which you described matches that game (WB friendliness)...as said in the other thread at amigans i was able to play quite a way into the game, but now it's only partly possible to play the game due to random crashes in read/write situations *when RAM is already filled to the top* (i checked by switching to WB) so it's qiute possibly paged memory acting up
But as you said there is something in the works...maybe we even get a short notice on what went wrong?
I do not want to stir things up any further as a bug has been confirmed as i understand it
But thanks for taking part in my thread
All of which you described matches that game (WB friendliness)...as said in the other thread at amigans i was able to play quite a way into the game, but now it's only partly possible to play the game due to random crashes in read/write situations *when RAM is already filled to the top* (i checked by switching to WB) so it's qiute possibly paged memory acting up
But as you said there is something in the works...maybe we even get a short notice on what went wrong?
I do not want to stir things up any further as a bug has been confirmed as i understand it
But thanks for taking part in my thread
People are dying.
Entire ecosystems are collapsing.
We are in the beginning of a mass extinction.
And all you can talk about is money and fairytales of eternal economic growth.
How dare you!
– Greta Thunberg
Entire ecosystems are collapsing.
We are in the beginning of a mass extinction.
And all you can talk about is money and fairytales of eternal economic growth.
How dare you!
– Greta Thunberg
Re: [BUG] SWAP broken
I just made a very simple CLI tool to test memory allocations using MEMF_VIRTUAL.
http://hem.bredband.net/deniil/files/swaptest.lha
It allows you to allocate 20MB chunks and fill them with data and free them.
Some initial tests shows that swapping simply doesn't work. First I had my system up normally with 277 or 512MB free and run the program and allocated until I had 5MB left. I then made one more allocation (of 20MB). That went fine. It took a while, like 2 seconds. It swapped something out and gave me those 20 extra MB. Free memory was now about 500k. Then I tried yet another allocation and the system locked up. No HD activity.
Ok then, I though. Let's boot WITHOUT startup-sequence! So I did and run the program. Now I had 454/512MB free. I allocated up until I had 7MB left. Then I made one more allocation and it failed to get the memory, i.e. AllocMem returned NULL, the program said. Then the system lockup up! WTF??
Looks like even a plain and simple out-of-memory is not even handled properly but causes a problems.
NOTE: I call AvailMem(MEMF_ANY) after any action in the program. WB also calls that periodically. Could it be that it is really AvailMem that has got some bitrot in it causing all these problems with update 3, not the actual swapping??!? In the test without s-s it locked up after returning from AllocMem and after a printf line, but before the AvailMem and a second printf had returned.
My SWAP partition uses 4096 blocksize and has 87382 cylinders which seems to add up to a couple of kilobytes below 2GB. Would a too large SWAP partition cause instant death?
http://hem.bredband.net/deniil/files/swaptest.lha
It allows you to allocate 20MB chunks and fill them with data and free them.
Some initial tests shows that swapping simply doesn't work. First I had my system up normally with 277 or 512MB free and run the program and allocated until I had 5MB left. I then made one more allocation (of 20MB). That went fine. It took a while, like 2 seconds. It swapped something out and gave me those 20 extra MB. Free memory was now about 500k. Then I tried yet another allocation and the system locked up. No HD activity.
Ok then, I though. Let's boot WITHOUT startup-sequence! So I did and run the program. Now I had 454/512MB free. I allocated up until I had 7MB left. Then I made one more allocation and it failed to get the memory, i.e. AllocMem returned NULL, the program said. Then the system lockup up! WTF??
Looks like even a plain and simple out-of-memory is not even handled properly but causes a problems.
NOTE: I call AvailMem(MEMF_ANY) after any action in the program. WB also calls that periodically. Could it be that it is really AvailMem that has got some bitrot in it causing all these problems with update 3, not the actual swapping??!? In the test without s-s it locked up after returning from AllocMem and after a printf line, but before the AvailMem and a second printf had returned.
My SWAP partition uses 4096 blocksize and has 87382 cylinders which seems to add up to a couple of kilobytes below 2GB. Would a too large SWAP partition cause instant death?
- Thomas Frieden
- AmigaOS Core Developer
- Posts: 147
- Joined: Fri Dec 10, 2010 3:21 pm
Re: [BUG] SWAP broken
Why MEMF_VIRTUAL ? Have you actually checked the autodocs ? If yes, where does it even mention MEMF_VIRTUAL ?Deniil wrote:I just made a very simple CLI tool to test memory allocations using MEMF_VIRTUAL.
http://hem.bredband.net/deniil/files/swaptest.lha
MEMF_VIRTUAL is NOT for memory allocations. Use either MEMF_SHARED or MEMF_PRIVATE. Using MEMF_VIRTUAL will make use of the default memory type MEMF_SHARED.
And there's your problem. MEMF_SHARED allocates memory that is automatically locked in memory. This can NOT be paged out. In other words, you basically fill the whole memory and lock it in place. Yes, this will lead to a lockup at one point because there simply is no way to page anything in or out if all memory ends up being locked. Linux "solves" this by killing off random processes and re-using their memory. AmigaOS does not have memory ownership, so there is NOTHING that can be done once this situation arises.
See above. Your tests are flawed because you are making assumptions that are actually not true.Deniil wrote:Some initial tests shows that swapping simply doesn't work.
Do you honestly think we would have put this out without tests ? I can assure you that it works, but your test only showed the one single case (all memory is locked) that can not be handled, and normally doesn't even occur in normal use.Deniil wrote:Looks like even a plain and simple out-of-memory is not even handled properly but causes a problems.
NoDeniil wrote: Would a too large SWAP partition cause instant death?
Re: [BUG] SWAP broken
... which is really a pain and does not work very well. More than once, I had to witness that it kills important services (like the sshd on a server) such that there is no way but pressing the reset button (instead of killing the task which consumes the biggest memory chunk)Thomas Frieden wrote:Linux "solves" this by killing off random processes and re-using their memory.
So, AmigaOS4 isn't worse in this regard
nexus
Re: [BUG] SWAP broken
The SWAP problem hase been fixed. My settings follow;
SWAP
Blocksize=4096
Buffers=10
Maxtransfer=7FFFFFFF (7 Fs)
Mask=FFFFFC
Not sure why the buffer setting is low, guessing default, but it does work. I can copy a 1.4GB file to RAM without problems now, crashed the system before the fix.
SWAP
Blocksize=4096
Buffers=10
Maxtransfer=7FFFFFFF (7 Fs)
Mask=FFFFFC
Not sure why the buffer setting is low, guessing default, but it does work. I can copy a 1.4GB file to RAM without problems now, crashed the system before the fix.
X1000 - Antec Solo case, 4GB DDR2 mem, XFX R9 270 gfx card, Plextor DVD, 2 Samsung 1TB F3 HDD, RTL-8139 network card, Cooler Master Silent Pro 600W PSU
Re: [BUG] SWAP broken
Yay
Thanks to all
Thanks to all
People are dying.
Entire ecosystems are collapsing.
We are in the beginning of a mass extinction.
And all you can talk about is money and fairytales of eternal economic growth.
How dare you!
– Greta Thunberg
Entire ecosystems are collapsing.
We are in the beginning of a mass extinction.
And all you can talk about is money and fairytales of eternal economic growth.
How dare you!
– Greta Thunberg
Re: [BUG] SWAP broken
I did read somewhere (apparently not in the latest autodocs. May have been the migration guide) that VIRTUAL is required to make it swappable, and also that VIRTUAL automatically implies PRIVATE (not the other way around). That's why.Why MEMF_VIRTUAL ? Have you actually checked the autodocs ? If yes, where does it even mention MEMF_VIRTUAL ?
MEMF_VIRTUAL is NOT for memory allocations. Use either MEMF_SHARED or MEMF_PRIVATE. Using MEMF_VIRTUAL will make use of the default memory type MEMF_SHARED.
But as a side note: Why would SHARED be locked when it can't be used in interrupts anyway? I don't see the point of that since the memory subsystem should be below the tasks anyway and access' and page faults be handled by the MMU irregardless of what task accesses the address? Just asking.
Well that explains why it failed completely when running without startup-sequence and my test program was basically the only thing in RAM. However, running out of memory should NEVER EVER cause a lockup of the machine (even if the caller ignores the return code, which my test program obviously doesn't, it should then just DSI with a NULL page exception).And there's your problem. MEMF_SHARED allocates memory that is automatically locked in memory. This can NOT be paged out. In other words, you basically fill the whole memory and lock it in place. Yes, this will lead to a lockup at one point because there simply is no way to page anything in or out if all memory ends up being locked.
But I read elsewhere that update 3 does have a bug causing this.
Re: [BUG] SWAP broken
@Deniil
I checked the OS4 migration guide, and the only really relevant things it has to say are:
But the bolded part seems to suggest that it can be swapped out?!? This appears to contradict what Thomas said:
I checked the OS4 migration guide, and the only really relevant things it has to say are:
No mention of MEMF_VIRTUAL !On AmigaOS 3.9 and earlier you had to allocate e.g. Messages and Message Ports in public memory; this is no longer needed under AmigaOS 4.0 instead use the MEMF_SHARED flag. The MEMF_SHARED flag shares a lot of the properties that its public counterpart has, but is much more friendly to the system. Most notably, MEMF_SHARED memory
1. is unlimited in size as far as address space and swap space is sufficient
2. can be read by all tasks in the system, but doesn't have too many strings attached
3. can be locked by a special API call to keep in from being swapped out
The only place where MEMF_PUBLIC should be used if when your memory needs to be accessible by an interrupt. In all other cases use MEMF_SHARED. If you are allocating private data, like storage for strings or other data, use MEMF_ANY or MEMF_PRIVATE. This makes your memory inaccessible from other tasks, which is currently not enforced but will be so in the future.
But the bolded part seems to suggest that it can be swapped out?!? This appears to contradict what Thomas said:
MEMF_SHARED allocates memory that is automatically locked in memory. This can NOT be paged out.