Defect: with an old-space limit (--maxOldSpaceSize, or when the system
refuses a new segment), a process that allocates with no end stops the VM
with "Could not allocate new object in the old space". The image gets no
low-space signal, so the low space watcher of the image cannot stop the
process.
Cause: SpurMemoryManager>>sufficientSpaceAfterGC: reports low space only
when the free old space is below the threshold of the image
(lowSpaceThreshold, 400000 bytes in Pharo). One scavenge can tenure
more than that, and SpurGenerationScavenger>>copyToOldSpace:bytes:format:
then stops the VM.
Change (one method): when old space cannot grow, space is low as soon as the free old space is less than two new spaces. One new space is for the next scavenge. The other is for the low space watcher and for the unwind of the process that the watcher stops. Before that answer the method runs one full GC if old space has grown by more than a new space since the last one.
The change is in Slang, so the build runs the code generation again. On
this Mac: file the method into the code generation image
(build/vmmaker/image/VMMaker.image of the build tree), then build. The
generation runs because the image is newer than the generated sources.
Test (2026-11-17, macOS 27, arm64): scripts/cogdiff/lowspace_recursion.st
with an old space of 1024 MB (a recursion with no end, in an image with
the low space watcher).
| VM | Answer |
|---|---|
| stock Cog | none: the VM stops, and then gets a segmentation fault in the code that prints the report |
| Cog with the change | OutOfMemory (the recursion ends, and the image goes on) |
A full sweep of the Pharo 14 tests on the Cog with the change
(sweepcogp14-new7, 39461 tests) has the same result as the sweep before
the change: 39373 pass, and no batch stopped.
Not tested: Windows, Linux, and a run with no old-space limit where the system refuses memory.
A probe that prints the same lines on a correct VM. Run it with Pharo.image eval or file it in:
"lowspace_recursion.st -- a recursion with no end, in an image with the low
space watcher. The answer is the class and the text of the error that ends
the recursion.
scripts/vmrun.sh eval --old-space 1024 cog scripts/cogdiff/lowspace_recursion.st
scripts/vmrun.sh eval --old-space 1024 cogp scripts/cogdiff/lowspace_recursion.st
scripts/vmrun.sh eval --old-space 1024 ours scripts/cogdiff/lowspace_recursion.st
Measured 2026-11-17 with the three commands above:
stock Cog no answer: the VM stops with 'Could not allocate new object in
the old space' when the old space reaches its limit, and then
with a segmentation fault in the code that prints the report.
patched Cog 'OutOfMemory: ' (cogp with the change of
docs/upstream/cog-low-space-before-old-space-limit.patch;
before that change cogp did the same as stock Cog)
our VM before the change 'Error: stack overflow: recursion deeper than
56000 frames' (a limit that Cog does not have,
the iospharo defect list); after the change 'OutOfMemory: '
The answer that the image is made for: OutOfMemory, from the low space
watcher."
| recurse |
Smalltalk installLowSpaceWatcher.
recurse := nil.
recurse := [ :n | (recurse value: n + 1) + 1 ].
[ recurse value: 1. 'the recursion returned' ]
on: Error
do: [ :e | e class name , ': ' , ((e messageText ifNil: [ '' ]) asString first: (((e messageText ifNil: [ '' ]) asString size) min: 80)) ]
Names such as scripts/vmrun.sh and scripts/cogdiff/ are scripts of the iospharo repository, which runs one VM at a time. cog is stock Cog, cogp is Cog with the change, ours is iospharo.
A change of the Cog sources (pharo-project/pharo-vm, branch pharo-12), on the local branch fix/low-space-signal-before-old-space-limit. State: committed on the branch, tested. Not sent upstream.
From 39104295636e8ebdbc53dba7b2c3fd7e1355d8f1 Mon Sep 17 00:00:00 2001 From: Aaron Wohl Date: Tue, 17 Nov 2026 12:56:27 -0500 Subject: [PATCH] Spur: signal low space before old space reaches its limit With --maxOldSpaceSize (or when the OS refuses a new segment), a process that allocates with no end stops the VM with 'Could not allocate new object in the old space', and the image gets no low space signal. sufficientSpaceAfterGC: reported low space only when the free old space was below the threshold of the image (lowSpaceThreshold, 400000 bytes in Pharo). One scavenge can tenure more than that, and copyToOldSpace:bytes:format: then stops the VM. Now, when old space cannot grow, space is low as soon as the free old space is less than two new spaces: one for the next scavenge, and one for the low space watcher of the image and the unwind of the process that the watcher stops. Before that answer the method runs one full GC if old space has grown by more than a new space since the last one, so that garbage does not count as use. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> --- .../VMMaker/SpurMemoryManager.class.st | 27 ++++++++++++++----- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/smalltalksrc/VMMaker/SpurMemoryManager.class.st b/smalltalksrc/VMMaker/SpurMemoryManager.class.st index 3c5ab99d6..abe149566 100644 --- a/smalltalksrc/VMMaker/SpurMemoryManager.class.st +++ b/smalltalksrc/VMMaker/SpurMemoryManager.class.st @@ -13006,7 +13006,7 @@ SpurMemoryManager >> sufficientSpaceAfterGC: numBytes [ which is a stop-the-world a young generation reclaimer. In Spur we run the scavenger. Answer if space is not low." - | heapSizePostGC | + | heapSizePostGC cannotGrow lowMark | self assert: numBytes = 0. self scavengingGCTenuringIf: TenureByAge. heapSizePostGC := segmentManager totalOldSpaceCapacity @@ -13014,12 +13014,25 @@ SpurMemoryManager >> sufficientSpaceAfterGC: numBytes [ (heapSizePostGC - heapSizeAtPreviousGC) asFloat / heapSizeAtPreviousGC >= heapGrowthToSizeGCRatio ifTrue: [ self fullGC ]. - [ - totalFreeOldSpace < growHeadroom and: [ - (self growOldSpaceByAtLeast: 0 callingOperation: 'ensuring sufficient space after GC') - notNil ] ] whileTrue: [ - totalFreeOldSpace >= growHeadroom ifTrue: [ ^ true ] ]. - lowSpaceThreshold > totalFreeOldSpace ifTrue: [ "space is low" + "When old space cannot grow (maxOldSpaceSize, or the OS refuses a segment), the + scavenger stops the VM (copyToOldSpace:bytes:format:) as soon as one scavenge + tenures more than the free old space. So space is low when old space cannot + take two new spaces: one for the next scavenge, and one for the low space + watcher and the unwind of the process that the watcher stops." + lowMark := scavenger newSpaceCapacity * 2. + cannotGrow := false. + [ totalFreeOldSpace < (growHeadroom max: lowMark) and: [ cannotGrow not ] ] whileTrue: [ + (self growOldSpaceByAtLeast: 0 callingOperation: 'ensuring sufficient space after GC') isNil + ifTrue: [ cannotGrow := true ] ]. + (cannotGrow and: [ lowSpaceThreshold > 0 and: [ totalFreeOldSpace < lowMark ] ]) ifTrue: [ + "Old space can hold garbage that no full GC has seen: the test of the ratio + above is relative to the last GC. Collect one time before the answer that + space is low." + heapSizePostGC := segmentManager totalOldSpaceCapacity - totalFreeOldSpace. + heapSizePostGC > (heapSizeAtPreviousGC + scavenger newSpaceCapacity) ifTrue: [ + self fullGC ] ]. + (lowSpaceThreshold > totalFreeOldSpace or: [ + cannotGrow and: [ lowSpaceThreshold > 0 and: [ totalFreeOldSpace < lowMark ] ] ]) ifTrue: [ "space is low" lowSpaceThreshold := 0. "avoid signalling low space twice" ^ false ]. ^ true -- 2.54.0 (Apple Git-157)
State of 2026-11-20. Made from the test records of the iospharo project (a second Pharo VM). The fixes are proposals for review.