Re: es memory requirements

[Paul Haahr <haahr@mv.us.adobe.com>]

> I brought up es "in production" today on a fairly large system, and the
> good news is that it seems to be working fine - have not heard one complaint
> about anything breaking.  The application in question runs under 400 lines
> of es code, so we're not talking about a mammoth program.

still, it's the largest extant es program.  i'm glad to hear that it's working.

can you say anything about speed relative to rc?  i would expect performance to be  
comparable.

> The bad news is that it's using 3 to 4 times as much memory as rc did, to
> do pretty much the same thing.  That puts it almost in a class with perl.
> Most of the memory is allocated at run time - the compiled binary sizes are
> pretty comparable with rc.

we haven't yet profiled memory usage in es, and don't even have real mechanisms at  
our fingertips for giving back info on es's dynamic memory usage patterns.  my  
guess is that while we allocate a fair amount of memory, working sets are not much  
larger than working sets in rc.  more data is necessary.  (we should probably add  
an extra GCINFO compile-time option for reporting amounts of live data.)

> I'm probably going to have to pull the es code and restore rc, my colleagues
> here are getting pretty uptight over the memory (we apparently had some
> low on memory problems this morning at only around 300 users, and we expect
> more like 500 when things pick up.)

does sound bad.  when you say low on memory, do you mean low on swap space or do  
you mean lots of thrashing?  the former is not that surprising (300-500 users!!!),  
the latter is more disturbing.

> What I'm wondering, since I faintly recall that es uses garbage collection,
> would it be possible to influence the collection algorithm?  The application
> tends to create a lot of functions and lists and then delete them all at once,
> and I'm thinking it might be possible to do something, along with deleting,
> that would allow the space to be reclaimed for the next batch of functions.  


deleting them does free them up, the next time a collection is done.  the  
allocator requires free memory to be contiguous (it's a copying collector), so  
knowing that some things are free already really doesn't help.

could it be that some temporary variables (with global scope) still point at the  
dead values?

> (When I say "delete", I mean assign a value of (), as in "fn-oldthing = ()".)

as an aside, the easiest way to do a delete is
	var =
rather than the rc-influenced
	var = ()

back from vacation & ready to hack,
paul