scripts/image-fixes/20-Fuel-widestring-globals.stFLGlobalCluster>>encodeGlobalName:on: (added), FLGlobalCluster>>nextGlobalNameFrom: (added), FLGlobalCluster>>serializeGlobalClass:on:, FLGlobalCluster>>serializeGlobalAssociationKeyed:on:, FLGlobalCluster>>materializeGlobalAssociationFrom:, FLGlobalCluster>>materializeGlobalClassFrom:, FLBasicSerializationTest>>testWideStringGlobal, FLSerializationTest>>expectedFailuresTwo defects. (1) Fuel (image code) wrote the name of a global (class name or global key) with encodeString:, which stores one byte per character, so the high bytes of a WideString/WideSymbol name were lost and the materializer looked up a wrong name (FLGlobalNotFound, FLClassNotFound). The fix writes global names as UTF-8 and reads them back as UTF-8, with a Latin-1 fallback for bytes that are not valid UTF-8; for ASCII names the bytes are identical to before, so existing files still load. (2) testWideStringGlobal was itself wrong: it used a WideString as a key of SystemEnvironment (which accepts only Symbols) and then asserted that the materialized value is identical to the KEY. The key is now a WideSymbol and the assertion compares with the global value that the key finds. The expectedFailures override in FLSerializationTest (the cause of the 'expected failure' mark) now answers #() because all three tests pass. Behaviour is the same on our VM and on Cog; no VM defect.
| Test | Defect in | State | Cause |
|---|---|---|---|
FLBinaryFileStreamBasicSerializationTest>>testConsiderCustomWideSymbolGlobal |
image-code | repaired | FLGlobalCluster encoded global names one byte per character; wide names lost high bytes |
FLBinaryFileStreamBasicSerializationTest>>testWideStringClassName |
image-code | repaired | same: class name written as bytes, FLClassNotFound |
FLBinaryFileStreamBasicSerializationTest>>testWideStringGlobal |
test | repaired | WideString key rejected by SystemEnvironment>>at:put: (Symbols only); assertion compared value with the key. Fixed in the test plus the name encoding |
FLByteArrayBasicSerializationTest>>testConsiderCustomWideSymbolGlobal |
image-code | repaired | same as above |
FLByteArrayBasicSerializationTest>>testWideStringClassName |
image-code | repaired | same as above |
FLByteArrayBasicSerializationTest>>testWideStringGlobal |
test | repaired | same as above |
FLFileReferenceStreamBasicSerializationTest>>testConsiderCustomWideSymbolGlobal |
image-code | repaired | same as above |
FLFileReferenceStreamBasicSerializationTest>>testWideStringClassName |
image-code | repaired | same as above |
FLFileReferenceStreamBasicSerializationTest>>testWideStringGlobal |
test | repaired | same as above |
FLFullBasicSerializationTest>>testConsiderCustomWideSymbolGlobal |
image-code | repaired | same as above |
FLFullBasicSerializationTest>>testWideStringClassName |
image-code | repaired | same as above |
FLFullBasicSerializationTest>>testWideStringGlobal |
test | repaired | same as above |
FLGZippedBasicSerializationTest>>testConsiderCustomWideSymbolGlobal |
image-code | repaired | same as above |
FLGZippedBasicSerializationTest>>testWideStringClassName |
image-code | repaired | same as above |
FLGZippedBasicSerializationTest>>testWideStringGlobal |
test | repaired | same as above |
Regression check: Whole classes run on both VMs, base image vs fixed image. Our VM and Cog gave the same counts. Base to fixed: FLBinaryFileStream, FLByteArray, FLFileReferenceStream BasicSerializationTest 75 pass + 2 fail + 1 error to 78 pass; FLGZippedBasicSerializationTest 78 pass + 2 fail + 1 error to 81 pass; FLFullBasicSerializationTest 82 pass + 2 fail + 1 error to 85 pass (a first base run of this class came back empty because of a parallel-run collision; I re-ran it alone to get the 82/2/1). Unchanged before and after: FLBlockClosureSerializationTest 18, FLCompiledMethodSerializationTest 6, FLContextSerializationTest 9, FLDecoderTest 8, FLEncoderDecoderTest 5, FLEncoderTest 16, FLFullBlockClosureSerializationTest 8, FLGlobalEnvironmentTest 12 pass + 1 skip, FLGlobalSendSerializationTest 2, FLMaterializerTest 12, FLSerializerTest 34. FLClassSerializationTest and FLTraitSerializationTest printed no result lines in any run, base or fixed, so they are not compared. I did not test the Latin-1 fallback for old files with bytes 128-255; I did not run the full suite.
Upstream note: Fuel: encode global names as UTF-8 (FLGlobalCluster). FLGlobalCluster wrote class names and global keys with FLEncoder>>encodeString:, one byte per character, so WideString/WideSymbol names lost their high bytes and materialization raised FLGlobalNotFound/FLClassNotFound. Names are now written as UTF-8 (identical bytes for ASCII) and decoded with utf8Decoded, with a Latin-1 fallback for older files. Also fix FLBasicSerializationTest>>testWideStringGlobal (WideString key in SystemEnvironment is rejected; the assertion compared the value to the key; use a WideSymbol key and compare with the global value) and remove the three entries of FLSerializationTest>>expectedFailures. Project: Fuel (pharo-project/fuel, in the image as FL*). Written for the owner to review; no PR opened.
Last full sweeps of 2026-11-17: iospharo (sweep14-new14) and stock Cog with the Cog changes (sweepcogp14-new10), with all image fixes. "Before" is the state of the test before the repairs. The Stock Cog column has a value only where the record measured stock Cog: "PASS 3 of 3 (repair run)" is the check of each repair on stock Cog v12.0.6 with the fix entry; "-" is not measured separately.
| Test | Before | State | iospharo VM | Patched Cog | Stock Cog |
|---|---|---|---|---|---|
FLBinaryFileStreamBasicSerializationTest>>testConsiderCustomWideSymbolGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLBinaryFileStreamBasicSerializationTest>>testWideStringClassName | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLBinaryFileStreamBasicSerializationTest>>testWideStringGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLByteArrayBasicSerializationTest>>testConsiderCustomWideSymbolGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLByteArrayBasicSerializationTest>>testWideStringClassName | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLByteArrayBasicSerializationTest>>testWideStringGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLFileReferenceStreamBasicSerializationTest>>testConsiderCustomWideSymbolGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLFileReferenceStreamBasicSerializationTest>>testWideStringClassName | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLFileReferenceStreamBasicSerializationTest>>testWideStringGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLFullBasicSerializationTest>>testConsiderCustomWideSymbolGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLFullBasicSerializationTest>>testWideStringClassName | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLFullBasicSerializationTest>>testWideStringGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLGZippedBasicSerializationTest>>testConsiderCustomWideSymbolGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLGZippedBasicSerializationTest>>testWideStringClassName | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
FLGZippedBasicSerializationTest>>testWideStringGlobal | XFAIL | repaired | PASS | PASS | PASS 3 of 3 (repair run) |
Run the tests in a Pharo 14 image (build 791). Before the fix they skip, fail or carry the expected-failure mark.
"Run in a Playground of a Pharo 14 image (Do it and print each line)." ((FLBinaryFileStreamBasicSerializationTest selector: #testConsiderCustomWideSymbolGlobal) run) printString. ((FLBinaryFileStreamBasicSerializationTest selector: #testWideStringClassName) run) printString. ((FLBinaryFileStreamBasicSerializationTest selector: #testWideStringGlobal) run) printString. ((FLByteArrayBasicSerializationTest selector: #testConsiderCustomWideSymbolGlobal) run) printString. ((FLByteArrayBasicSerializationTest selector: #testWideStringClassName) run) printString. ((FLByteArrayBasicSerializationTest selector: #testWideStringGlobal) run) printString. ((FLFileReferenceStreamBasicSerializationTest selector: #testConsiderCustomWideSymbolGlobal) run) printString. ((FLFileReferenceStreamBasicSerializationTest selector: #testWideStringClassName) run) printString. ((FLFileReferenceStreamBasicSerializationTest selector: #testWideStringGlobal) run) printString. ((FLFullBasicSerializationTest selector: #testConsiderCustomWideSymbolGlobal) run) printString. ((FLFullBasicSerializationTest selector: #testWideStringClassName) run) printString. ((FLFullBasicSerializationTest selector: #testWideStringGlobal) run) printString. ((FLGZippedBasicSerializationTest selector: #testConsiderCustomWideSymbolGlobal) run) printString. ((FLGZippedBasicSerializationTest selector: #testWideStringClassName) run) printString. ((FLGZippedBasicSerializationTest selector: #testWideStringGlobal) run) printString.
testConsiderCustomWideSymbolGlobal "The same than #testConsiderCustomGlobal but with a WideSymbol." | aWideSymbol aPerson | aWideSymbol := (WideString streamContents: [ :stream | 256 to: 280 do: [ :code | stream nextPut: code asCharacter ] ]) asSymbol. self serializer lookUpInGlobals: aWideSymbol. aPerson := FLPerson new. self environmentOfTest at: aWideSymbol put: aPerson. addedGlobals add: aWideSymbol. self shouldnt: [ self resultOfSerializeAndMaterialize: aPerson ] raise: FLGlobalNotFound
testWideStringClassName | class className | className := 'Foox' asWideString at: 4 put: (Character value: 265); yourself. class := self classFactory silentlyMake: [ :builder | builder name: className ]. self environmentOfTest at: class name put: class. self shouldnt: [ self resultOfSerializeAndMaterialize: class ] raise: FLClassNotFound
testWideStringGlobal | global globalValue | global := 'Foox' asWideString at: 4 put: (Character value: 265); yourself. globalValue := global , 'value'. self environmentOfTest at: global put: globalValue. addedGlobals add: global. self serializer lookUpInGlobals: global. self shouldnt: [ self serialize: 'bar' -> globalValue ] raise: Error. self assert: self materialized class equals: Association. self assert: self materialized value identicalTo: global
How to apply an entries file: file in 00-header.st and the entries file as one chunk (the iospharo script scripts/gen-image-fixes.py makes that chunk). Each entry checks for the defect first and prints FIXED, ADDED or SKIP. Download 00-header.st.
"Fuel-widestring-globals. Fuel wrote the name of a global (a class name, a
global key) with encodeString:, which stores one byte per character. A name
with a character above 255 (WideString, WideSymbol) lost its high bytes, so
the materializer looked up a wrong name (FLGlobalNotFound, FLClassNotFound).
Global names are now written as UTF-8 and read back as UTF-8. For an ASCII
name the bytes are the same as before, so old files still load. A name with
bytes 128-255 that is not valid UTF-8 (a file from before this change) is
read as Latin-1, as before."
add value: #FLGlobalCluster value: #encodeGlobalName:on: value: 'serialization'
value:
'encodeGlobalName: aName on: anEncoder
"Write the name of a global as UTF-8. Same bytes as encodeString: for an ASCII name."
anEncoder encodeString: aName asString utf8Encoded asString'.
add value: #FLGlobalCluster value: #nextGlobalNameFrom: value: 'serialization'
value:
'nextGlobalNameFrom: aDecoder
"Read a global name written by #encodeGlobalName:on:. Bytes that are not valid UTF-8 come from an older file and are Latin-1."
| string |
string := aDecoder nextEncodedString.
^ ([ string asByteArray utf8Decoded ]
on: Error
do: [ :error | string ]) asSymbol'.
fix value: #FLGlobalCluster value: #serializeGlobalClass:on:
value: [ :source | source includesSubstring: 'anEncoder encodeString: aClass name' ]
value:
'serializeGlobalClass: aClass on: anEncoder
self encodeGlobalName: aClass name on: anEncoder'.
fix value: #FLGlobalCluster value: #serializeGlobalAssociationKeyed:on:
value: [ :source | source includesSubstring: 'anEncoder encodeString: aSymbol' ]
value:
'serializeGlobalAssociationKeyed: aSymbol on: anEncoder
self encodeGlobalName: aSymbol on: anEncoder'.
fix value: #FLGlobalCluster value: #materializeGlobalAssociationFrom:
value: [ :source | source includesSubstring: 'nextEncodedString asSymbol' ]
value:
'materializeGlobalAssociationFrom: aDecoder
| globalName |
globalName := self nextGlobalNameFrom: aDecoder.
^ aDecoder globalEnvironment
associationAt: globalName
ifAbsent: [ FLGlobalNotFound signalWithName: globalName ]'.
fix value: #FLGlobalCluster value: #materializeGlobalClassFrom:
value: [ :source | source includesSubstring: 'nextEncodedString asSymbol' ]
value:
'materializeGlobalClassFrom: aDecoder
| className |
className := self nextGlobalNameFrom: aDecoder.
^ aDecoder classNamed: className'.
"testWideStringGlobal put a WideString into a SystemEnvironment as key, which
only accepts Symbols (SystemEnvironment>>at:put:), and then expected the
materialized VALUE to be the KEY. The key is now a WideSymbol and the value
expected is the global value, the object found by that key."
fix value: #FLBasicSerializationTest value: #testWideStringGlobal
value: [ :source | source includesSubstring: 'yourself.' ]
value:
'testWideStringGlobal
| global globalValue |
global := (''Foox'' asWideString
at: 4 put: (Character value: 265);
yourself) asSymbol.
globalValue := global asString , ''value''.
self environmentOfTest
at: global
put: globalValue.
addedGlobals add: global.
self serializer lookUpInGlobals: global.
self shouldnt: [ self serialize: ''bar'' -> globalValue ] raise: Error.
self assert: self materialized class equals: Association.
self assert: self materialized value identicalTo: globalValue'.
"The three tests pass now: no expected failures remain."
fix value: #FLSerializationTest value: #expectedFailures
value: [ :source | source includesSubstring: 'testWideStringGlobal' ]
value:
'expectedFailures
^ #()'.
State of 2026-11-20. Made from the test records of the iospharo project (a second Pharo VM). The fixes are proposals for review.