[MLton] Crash fread(...) failed (only read 0) (Cannot allocate memory) during deepFlatten with MLton 20070826

[Nicolas Bertolotti]

It is easy to determine it (it is proportional to the size of the heap). The problem, as Matthew explained, is that there is an additional cost when the heap is resized (the whole card/cross map has to be moved and so on).

I actually made some experiments using the attached "ugly" patch which does not really allocate both areas but basically ensures that the card/cross map allocation succeeds after the heap is allocated.

It consists in allocating enough bytes for the heap and the card/cross map with mmap() each time a heap is allocated. Then, it immediately runs remap() in order to release the bytes that correspond to the card/cross map. The consequence is that, when it is necessary to allocate a new card/cross map after that, it always works.

I was wondering whether it would be possible to do something in the same "spirit" rather than merging the heap and the card/cross map.

Nicolas

> -----Original Message-----
> From: mlton-bounces at mlton.org [mailto:mlton-bounces at mlton.org] On Behalf
> Of Vesa Karvonen
> Sent: Wednesday, April 30, 2008 4:00 PM
> To: Matthew Fluet
> Subject: Re: [MLton] Crash fread(...) failed (only read 0) (Cannot
> allocate memory) during deepFlatten with MLton 20070826
>
> On Wed, Apr 30, 2008 at 5:44 PM, Matthew Fluet <fluet at tti-c.org> wrote:
> [...]
> >  That seems to be the case.  This would actually consume a little more
> > memory when using the copying collector, since both the from-heap and
> the
> > to-heap will have card/cross maps allocated within them.  And resizing
> heaps
> > would become a little more complicated, since the size of the card/cross
> map
> > is proportional to the size of the heap.
>
> Is it difficult/impossible to compute the required sizes and allocate
> (all) the heap(s) and card/cross map(s) in a single "transaction"
> (IOW, try to allocated all of them and if one allocation fails, free
> the others and try again with a smaller heap size)?
>
> -Vesa Karvonen
>
> _______________________________________________
> MLton mailing list
> MLton at mlton.org
> http://mlton.org/mailman/listinfo/mlton
-------------- next part --------------
A non-text attachment was scrubbed...
Name: heap.patch
Type: application/octet-stream
Size: 6260 bytes
Desc: heap.patch
Url : http://mlton.org/pipermail/mlton/attachments/20080430/37389dc6/heap-0001.obj