[MLton-devel] D2 problem
Stephen Weeks
MLton@mlton.org
Tue, 23 Apr 2002 20:40:15 -0700
Alain, I just checked in a fix for the out of memory problem you were
seeing. Now, MLton attempts to use as much of the address space as it
can on Cygwin. Unfortunately, I still have a problem when running
your test code on Cygwin. I have included a fragment of the log
below. The program dies when attempting to do a GC while a
512,000,000 byte array is live. The error is better :-) in that MLton
has used as large a portion of the address space as it can.
Unfortunately, we are limited by address space restrictions on
Windows/Cygwin. First, Windows only allows 2G of address space per
process [0, 2G). Second, before MLton starts running, various
portions of the address space have been reserved, leaving only about
1.1G of contiguous space available. Because MLton allocates 1.25
times the amount of live data space, which comes to 641 million bytes,
there is only about 480 million available for to space, which is not
enough.
I don't see any easy way to work around the address space limitation
and fragmentation problems on windows with MLton's current approach of
using two large semispaces. The only way out that I see is to
allocate semispaces as collections of smaller chunks (say 10M or
100M). There isn't too much code in MLton that relies on the
semispaces being contiguous, so this should be feasible, although it's
not a 1-day fix. Of course, this also only works if the data fits
into the smaller chunks. I'm not sure if your actual code uses such
large arrays. I don't see any solution for that on Windows without
breaking up the array.
Thoughts, anyone?
*** CHECKING PHASE 3, n=3000
***** Phase 2.7 (Allocating 512000000 bytes.)
/tmp/filemV690r.3.s 0: GC_doGC
Starting gc. bytesRequested = 512000008
Releasing to space.
fromSpace = 1bb40000 toSpace = 1d310000
fromSpace size = 6,205,440 toSpace size = 641,101,824
Shrinking new space at 1d310000 to 640794624 bytes.
Shrinking old space at 1bb40000 to 0 bytes.
Releasing to space.
Finished gc.
time(ms): 40
live(bytes): 633,928 (10.2%)
/tmp/filemV690r.3.s 0: GC_doGC
Starting gc. bytesRequested = 0
[Requested 611M cannot be satisfied, backing off by 30M (need = 0M).
[Requested 580M cannot be satisfied, backing off by 30M (need = 0M).
[Requested 549M cannot be satisfied, backing off by 30M (need = 0M).
[Requested 519M cannot be satisfied, backing off by 30M (need = 0M).
[Requested 488M cannot be satisfied, backing off by 30M (need = 0M).
fromSpace = 1d310000 toSpace = 43680000
fromSpace size = 640,794,624 toSpace size = 480,579,584
Total Phys. Mem: 427216896
Avail Phys. Mem: 3145728
Total Page File: 1180844032
Avail Page File: 28672
Total Virtual: 2147352576
Avail Virtual: 733569024
0x 0 65536 MEM_FREE PAGE_NOACCESS
0x 10000 4096 MEM_COMMIT PAGE_READWRITE
0x 11000 61440 MEM_FREE PAGE_NOACCESS
0x 20000 4096 MEM_COMMIT PAGE_READWRITE
0x 21000 61440 MEM_FREE PAGE_NOACCESS
0x 30000 2068480 MEM_RESERVE PAGE_NOACCESS
0x 229000 4096 MEM_COMMIT PAGE_NOACCESS
0x 22a000 24576 MEM_COMMIT PAGE_READWRITE
0x 230000 24576 MEM_COMMIT PAGE_READWRITE
0x 236000 1024000 MEM_RESERVE PAGE_READWRITE
0x 330000 4096 MEM_COMMIT PAGE_READWRITE
0x 331000 61440 MEM_RESERVE PAGE_READWRITE
0x 340000 90112 MEM_COMMIT PAGE_READONLY
0x 356000 40960 MEM_FREE PAGE_NOACCESS
0x 360000 147456 MEM_COMMIT PAGE_READONLY
0x 384000 49152 MEM_FREE PAGE_NOACCESS
0x 390000 266240 MEM_COMMIT PAGE_READONLY
0x 3d1000 61440 MEM_FREE PAGE_NOACCESS
0x 3e0000 12288 MEM_COMMIT PAGE_READONLY
0x 3e3000 118784 MEM_FREE PAGE_NOACCESS
0x 400000 4096 MEM_COMMIT PAGE_READONLY
0x 401000 397312 MEM_COMMIT PAGE_EXECUTE_READ
0x 462000 28672 MEM_COMMIT PAGE_WRITECOPY
0x 469000 12288 MEM_COMMIT PAGE_READWRITE
0x 46c000 1413120 MEM_COMMIT PAGE_READONLY
0x 5c5000 45056 MEM_FREE PAGE_NOACCESS
0x 5d0000 12288 MEM_COMMIT PAGE_EXECUTE_READ
0x 5d3000 774144 MEM_RESERVE PAGE_EXECUTE_READ
0x 690000 4096 MEM_COMMIT PAGE_EXECUTE_READ
0x 691000 28672 MEM_RESERVE PAGE_EXECUTE_READ
0x 698000 32768 MEM_FREE PAGE_NOACCESS
0x 6a0000 274432 MEM_COMMIT PAGE_READONLY
0x 6e3000 118784 MEM_FREE PAGE_NOACCESS
0x 700000 245760 MEM_COMMIT PAGE_EXECUTE_READ
0x 73c000 2899968 MEM_RESERVE PAGE_EXECUTE_READ
0x a00000 4096 MEM_COMMIT PAGE_READWRITE
0x a01000 61440 MEM_FREE PAGE_NOACCESS
0x a10000 4096 MEM_COMMIT PAGE_READWRITE
0x a11000 61440 MEM_FREE PAGE_NOACCESS
0x a20000 4096 MEM_COMMIT PAGE_READWRITE
0x a21000 126976 MEM_RESERVE PAGE_READWRITE
0x a40000 4096 MEM_COMMIT PAGE_READWRITE
0x a41000 157020160 MEM_FREE PAGE_NOACCESS
0x a000000 49152 MEM_COMMIT PAGE_READWRITE
0x a00c000 16384 MEM_FREE PAGE_NOACCESS
0x a010000 4096 MEM_COMMIT PAGE_READWRITE
0x a011000 268431360 MEM_RESERVE PAGE_READWRITE
0x1a010000 20480 MEM_COMMIT PAGE_READWRITE
0x1a015000 45056 MEM_FREE PAGE_NOACCESS
0x1a020000 2088960 MEM_RESERVE PAGE_NOACCESS
0x1a21e000 4096 MEM_COMMIT PAGE_NOACCESS
0x1a21f000 4096 MEM_COMMIT PAGE_READWRITE
0x1a220000 4096 MEM_COMMIT PAGE_READWRITE
0x1a221000 61440 MEM_FREE PAGE_NOACCESS
0x1a230000 1048576 MEM_COMMIT PAGE_READWRITE
0x1a330000 50200576 MEM_FREE PAGE_NOACCESS
0x1d310000 640794624 MEM_COMMIT PAGE_READWRITE
0x4362c000 307200 MEM_RESERVE PAGE_READWRITE
0x43677000 36864 MEM_FREE PAGE_NOACCESS
0x43680000 480579584 MEM_COMMIT PAGE_READWRITE
0x600d1000 15921152 MEM_FREE PAGE_NOACCESS
0x61000000 4096 MEM_COMMIT PAGE_READONLY
0x61001000 663552 MEM_COMMIT PAGE_EXECUTE_READ
0x610a3000 8192 MEM_COMMIT PAGE_READWRITE
0x610a5000 8192 MEM_COMMIT PAGE_WRITECOPY
0x610a7000 16384 MEM_COMMIT PAGE_READWRITE
0x610ab000 8192 MEM_COMMIT PAGE_WRITECOPY
0x610ad000 4096 MEM_COMMIT PAGE_READWRITE
0x610ae000 4096 MEM_COMMIT PAGE_EXECUTE_READWRITE
0x610af000 4096 MEM_COMMIT PAGE_WRITECOPY
0x610b0000 4096 MEM_COMMIT PAGE_EXECUTE_WRITECOPY
0x610b1000 4096 MEM_COMMIT PAGE_WRITECOPY
0x610b2000 4096 MEM_COMMIT PAGE_EXECUTE_WRITECOPY
0x610b3000 4096 MEM_COMMIT PAGE_READWRITE
0x610b4000 4096 MEM_COMMIT PAGE_EXECUTE_READWRITE
0x610b5000 4096 MEM_COMMIT PAGE_WRITECOPY
0x610b6000 4096 MEM_COMMIT PAGE_EXECUTE_WRITECOPY
0x610b7000 4096 MEM_COMMIT PAGE_WRITECOPY
0x610b8000 12288 MEM_COMMIT PAGE_EXECUTE_WRITECOPY
0x610bb000 4096 MEM_COMMIT PAGE_WRITECOPY
0x610bc000 4096 MEM_COMMIT PAGE_EXECUTE_WRITECOPY
0x610bd000 4096 MEM_COMMIT PAGE_WRITECOPY
0x610be000 4096 MEM_COMMIT PAGE_EXECUTE_WRITECOPY
0x610bf000 4096 MEM_COMMIT PAGE_WRITECOPY
0x610c0000 4096 MEM_COMMIT PAGE_EXECUTE_WRITECOPY
0x610c1000 4096 MEM_COMMIT PAGE_WRITECOPY
0x610c2000 4096 MEM_COMMIT PAGE_EXECUTE_WRITECOPY
0x610c3000 8192 MEM_COMMIT PAGE_READWRITE
0x610c5000 192512 MEM_COMMIT PAGE_WRITECOPY
0x610f4000 4096 MEM_COMMIT PAGE_READWRITE
0x610f5000 12288 MEM_COMMIT PAGE_WRITECOPY
0x610f8000 8192 MEM_COMMIT PAGE_READWRITE
0x610fa000 24576 MEM_COMMIT PAGE_WRITECOPY
0x61100000 4096 MEM_COMMIT PAGE_READWRITE
0x61101000 45056 MEM_COMMIT PAGE_WRITECOPY
0x6110c000 4931584 MEM_FREE PAGE_NOACCESS
0x615c0000 16384 MEM_COMMIT PAGE_READWRITE
0x615c4000 5013504 MEM_RESERVE PAGE_READWRITE
0x61a8c000 271794176 MEM_FREE PAGE_NOACCESS
0x71dc0000 4096 MEM_COMMIT PAGE_READONLY
0x71dc1000 12288 MEM_COMMIT PAGE_EXECUTE_READ
0x71dc4000 16384 MEM_COMMIT PAGE_WRITECOPY
0x71dc8000 8192 MEM_COMMIT PAGE_READONLY
0x71dca000 80699392 MEM_FREE PAGE_NOACCESS
0x76ac0000 4096 MEM_COMMIT PAGE_READONLY
0x76ac1000 86016 MEM_COMMIT PAGE_EXECUTE_READ
0x76ad6000 8192 MEM_COMMIT PAGE_READONLY
0x76ad8000 4096 MEM_COMMIT PAGE_READWRITE
0x76ad9000 4096 MEM_COMMIT PAGE_WRITECOPY
0x76ada000 12288 MEM_COMMIT PAGE_READONLY
0x76add000 17903616 MEM_FREE PAGE_NOACCESS
0x77bf0000 4096 MEM_COMMIT PAGE_READONLY
0x77bf1000 8192 MEM_COMMIT PAGE_EXECUTE_READ
0x77bf3000 4096 MEM_COMMIT PAGE_READONLY
0x77bf4000 4096 MEM_COMMIT PAGE_READWRITE
0x77bf5000 8192 MEM_COMMIT PAGE_READONLY
0x77bf7000 1871872 MEM_FREE PAGE_NOACCESS
0x77dc0000 4096 MEM_COMMIT PAGE_READONLY
0x77dc1000 172032 MEM_COMMIT PAGE_EXECUTE_READ
0x77deb000 40960 MEM_COMMIT PAGE_READONLY
0x77df5000 4096 MEM_COMMIT PAGE_READWRITE
0x77df6000 12288 MEM_COMMIT PAGE_WRITECOPY
0x77df9000 8192 MEM_COMMIT PAGE_READWRITE
0x77dfb000 16384 MEM_COMMIT PAGE_READONLY
0x77dff000 69632 MEM_FREE PAGE_NOACCESS
0x77e10000 4096 MEM_COMMIT PAGE_READONLY
0x77e11000 286720 MEM_COMMIT PAGE_EXECUTE_READ
0x77e57000 24576 MEM_COMMIT PAGE_READONLY
0x77e5d000 4096 MEM_COMMIT PAGE_READWRITE
0x77e5e000 36864 MEM_COMMIT PAGE_READONLY
0x77e67000 36864 MEM_FREE PAGE_NOACCESS
0x77e70000 4096 MEM_COMMIT PAGE_READONLY
0x77e71000 266240 MEM_COMMIT PAGE_EXECUTE_READ
0x77eb2000 36864 MEM_COMMIT PAGE_READONLY
0x77ebb000 4096 MEM_COMMIT PAGE_READWRITE
0x77ebc000 36864 MEM_COMMIT PAGE_READONLY
0x77ec5000 45056 MEM_FREE PAGE_NOACCESS
0x77ed0000 4096 MEM_COMMIT PAGE_READONLY
0x77ed1000 147456 MEM_COMMIT PAGE_EXECUTE_READ
0x77ef5000 4096 MEM_COMMIT PAGE_READONLY
0x77ef6000 4096 MEM_COMMIT PAGE_READWRITE
0x77ef7000 20480 MEM_COMMIT PAGE_READONLY
0x77efc000 16384 MEM_FREE PAGE_NOACCESS
0x77f00000 4096 MEM_COMMIT PAGE_READONLY
0x77f01000 241664 MEM_COMMIT PAGE_EXECUTE_READ
0x77f3c000 36864 MEM_COMMIT PAGE_READONLY
0x77f45000 8192 MEM_COMMIT PAGE_READWRITE
0x77f47000 94208 MEM_COMMIT PAGE_READONLY
0x77f5e000 8192 MEM_FREE PAGE_NOACCESS
0x77f60000 4096 MEM_COMMIT PAGE_READONLY
0x77f61000 245760 MEM_COMMIT PAGE_EXECUTE_READ
0x77f9d000 32768 MEM_COMMIT PAGE_READONLY
0x77fa5000 4096 MEM_COMMIT PAGE_READWRITE
0x77fa6000 4096 MEM_COMMIT PAGE_WRITECOPY
0x77fa7000 8192 MEM_COMMIT PAGE_READWRITE
0x77fa9000 4096 MEM_COMMIT PAGE_WRITECOPY
0x77faa000 81920 MEM_COMMIT PAGE_READONLY
0x77fbe000 123936768 MEM_FREE PAGE_NOACCESS
0x7f5f0000 1048576 MEM_RESERVE PAGE_NOACCESS
0x7f6f0000 24576 MEM_COMMIT PAGE_EXECUTE_READ
0x7f6f6000 1024000 MEM_RESERVE PAGE_EXECUTE_READ
0x7f7f0000 8126464 MEM_FREE PAGE_NOACCESS
0x7ffb0000 147456 MEM_COMMIT PAGE_READONLY
0x7ffd4000 36864 MEM_FREE PAGE_NOACCESS
0x7ffdd000 4096 MEM_COMMIT PAGE_EXECUTE_READWRITE
0x7ffde000 4096 MEM_COMMIT PAGE_EXECUTE_READWRITE
0x7ffdf000 4096 MEM_COMMIT PAGE_EXECUTE_READWRITE
0x7ffe0000 4096 MEM_COMMIT PAGE_READONLY
0x7ffe1000 61440 MEM_RESERVE PAGE_NOACCESS
0x7ffe1000 61440 MEM_RESERVE PAGE_NOACCESS
0x7ffe1000 61440 MEM_RESERVE PAGE_NOACCESS
size=512000008 skip=0 remaining=480547684 s->fromSize=640794624 s->toSize=480579584 headerBytes=8 objectBytes=512000000 header=4000
Out of memory (forward).
Diagnostic: probably a RAM problem.
_______________________________________________
MLton-devel mailing list
MLton-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/mlton-devel