As a possible solution for resource tracking that could be applied retroactively, has anyone ever considered implementation of a cleanup stack?
A cleanup stack is a thread (task) specific 'stack' structure containing a copy of the handles of all your dynamic allocations. Then, if your task exits unexpectedly, or doesn't exit at all, the kernel can still unload all the resource allocations, in the reverse order they were allocated, and get the memory back, close the windows, close the file handles etc.
Note that the cleanup stack is nothing to do with the regular stack, the stack being the last or one of the last things deallocated when the program itself exits.
Symbian OS used it extensively, as an app developer after each new object was allocated, you'd put the object (i.e. it's pointer) onto the stack, and then when the stack was unwound all the destructors of those objects could be run to free up the resources.
Since Amiga isn't an object oriented API you'd need something slightly different i think, you'd need to store the pointer to the resource, and also the type of the resource, and possibly the size for some types. Then the cleanup would pop each item one by one, calling the appropriate Free/Close function for each one in turn, in a task of its own.
Also Symbian apps put the onus on the app developer to push and pop items from the stack themselves during the normal execution of the app. That doesn't help us with adding it retroactively to add stability to AmigaOS point of view though.
What would be needed is for all OS functions that allocate resources upon request, to be modified so they would need to put a copy of the pointer to the resource, along with it's type and perhaps it's size, into a thread specific cleanup stack data structure. Likewise any call to free a resource would remove it from the cleanup stack so it didn't get deallocated twice later. They'd need to know what task the allocations belonged to, and this they do know because they run in the same thread - in fact, it'd best be done for Processes only, not tasks, because few if any programs have more than one Process but some might have multiple additional tasks. And few if any apps are only tasks and not processes.
The handle for the process/tasks's stack could itself be on the cleanup stack, as the very first item, so it too can be deallocated when the process is killed or ends. Some special permission might be needed by the cleanup process to access protected memory. Or it might make sense to only implement it for 68k apps, and assume that OS4 apps can be coded to use it manually.
Even a normally exiting application would still have it cleanup stack popped after exit - in theory there shouldn't be anything left on it but if there is, then the memory leak will be resolved.
Thoughts?
There's another complementary way to do it too - create servers that act as resource owners. For example, a file server or a window/gadget server. You then overload the open/close functions to point to these servers instead.
When you open a file, what happens is the file server opens the file you requested, and stores the pointer it's given in a cleanup table, logging which process/task was the callee. It then relays you the handle. Later, if you call to close it, it checks that it hasn't already been closed by looking it up in it's cleanup table, so that it doesn't accidentally try to close it twice, or close an invalid one. Periodically it checks if the task still exists, and if not it puts the resource into a 'possible cleanupable' state, eventually deallocating it completely if it goes unused for a long period of time. A new API would allow a task to tell it if it spawn a sub task or otherwise passed the handle to another task and therefore those other tasks might also use it.
Since the file server and window server and other resource servers would run for the lifetime of the session, they don't need to worry so much about their own cleanup.