[MLton-devel] Controlling memory usage for MPI MLton processes

[Stephen Weeks]

Hi Roland.  It's nice to hear from you again after a few years.

> First of all, I would like to congratulate you to an amzingly bug-free 
> compiler. I have previously tried many 110.x SML/NJ versions and have had 
> various problems with all of them.
> 
> However, MLton managed to compile my ADATE automatic programming system 
> without activating any detectable compiler /run-time system bug. That's 
> well done!

Great, we're glad to hear it.  Thanks for the compliment.

> My only problem is that I cannot use MLtons ram-slop option under MPI.  
> This option appears necessary to prevent swapping for my application and
> does its job perfectly in single process mode. But I have not succeeded in
> communicationg this option to MPI slave processes and wonder if it can be
> set in some other way than through a command line option?

There is no way in MLton 20030716 to set ram-slop other than the
command line.  In the next release, there will be a compiler flag,
-runtime, that will let you compile @MLton runtime options into the
executable.

I don't quite understand how your application works.   I would think
that if you set ram-slop at the command line and then the MLton
process forks to create other processes, then those would inherit the
ram-slop of the parent.  Is that failing to happen?  Or is the problem
that you would like the sum of the ram-slops for all the MLton
processes to be constant?

> Under SML/NJ, a process needs about 65MB, but the MLton processes insist
> on using about 210MB which triggers swapping on the 256MB Beowulf cluster
> nodes.
> 
> 
> Any suggestion welcome!

You could use the fixed-heap or max-heap options to limit processes to
a certain size.

You could use the (undocumented) live-ratio option, which controls how
agressive the runtime is in grabbing memory.  The default value is 8,
which when combined with two-space copying GC means that the actual
ratio of heap size to live data is 16.  You could try setting this
value to 4 or 6 and I wouldn't expect you to see much of a slowdown.

You could use MLton.GC.{pack,unpack}, which allow a process that knows
it will be dormant for a while to shrink the heap while it is dormant.

You could recompile the runtime with a different a ram-slop value.


Let us know if any (or none) of these work or if you find some other
workable solution.



-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
See the people who have HELPED US provide better services:
Click here: http://sourceforge.net/supporters.php
_______________________________________________
MLton-devel mailing list
MLton-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlton-devel