Pharo test defects

Cly-async-results

State
Fixed (image fix)
Defect in
the test
Tests
2
Fix
image fix 20-Cly-async-results.st
Upstream repository
pharo-project/pharo

Record

Both tests were an obsolete inverted expectation in the test, not a VM defect. ClyAsyncQueryResultTest>>testItemsChangedNotificationShouldResetItems was marked expectedFailure and called super, i.e. ClyQueryNavigationResultTest's test that itemsChanged resets items (deny isBuilt, items isNil). By design an async result never resets built items: ClyAsyncQueryResult>>initializeItems says so, and the sibling ClyAsyncQueryResultTest>>testItemsChangedNotificationShouldNotResetExistingItems asserts the opposite of the inherited test. The inherited test can never pass, so the mark was permanent. The fix removes the expectedFailure mark and the super call, and the test now checks the async contract (initializeItems builds, then itemsChanged leaves isBuilt true, items non-nil, needsRebuild true). ClySemiAsyncQueryResultTest inherits the test via shouldInheritSelectors, so one entry repairs both.

Test Defect in State Cause
ClyAsyncQueryResultTest>>testItemsChangedNotificationShouldResetItems test repaired Expected-failure override called the superclass test whose expectation (items reset) contradicts the documented async behaviour; changed expectation to the async contract
ClySemiAsyncQueryResultTest>>testItemsChangedNotificationShouldResetItems test repaired Inherited from ClyAsyncQueryResultTest (shouldInheritSelectors); same override, same repair

Regression check: Whole classes run before (base image) and after (fixed image). ClyAsyncQueryResultTest: ours and Cog 19 PASS + 1 FAIL before, 20 PASS after. ClySemiAsyncQueryResultTest: ours and Cog 20 PASS + 1 FAIL before, 21 PASS after. ClyQueryNavigationResultTest (abstract superclass, unchanged by the fix): 10 ERROR before and after on both VMs, identical. No non-test method was changed, so no other classes cover the change.

Upstream note: Pharo, package Calypso-NavigationModel-Tests: ClyAsyncQueryResultTest>>testItemsChangedNotificationShouldResetItems is an expectedFailure that calls super, whose expectation (itemsChanged resets items) contradicts ClyAsyncQueryResult>>initializeItems ("async query should never reset existing items") and the sibling test testItemsChangedNotificationShouldNotResetExistingItems. It can never pass. Replace the body with the async contract (items stay built, needsRebuild is true) and drop the expectedFailure pragma. ClySemiAsyncQueryResultTest inherits it. Not for Cog; the owner reviews any PR first.

Results

Last full sweeps of 2026-11-17: iospharo (sweep14-new14) and stock Cog with the Cog changes (sweepcogp14-new10), with all image fixes. "Before" is the state of the test before the repairs. The Stock Cog column has a value only where the record measured stock Cog: "PASS 3 of 3 (repair run)" is the check of each repair on stock Cog v12.0.6 with the fix entry; "-" is not measured separately.

TestBeforeStateiospharo VMPatched CogStock Cog
ClyAsyncQueryResultTest>>testItemsChangedNotificationShouldResetItemsXFAILrepairedPASSPASSPASS 3 of 3 (repair run)
ClySemiAsyncQueryResultTest>>testItemsChangedNotificationShouldResetItemsXFAILrepairedPASSPASSPASS 3 of 3 (repair run)

Test

Run the tests in a Pharo 14 image (build 791). Before the fix they skip, fail or carry the expected-failure mark.

"Run in a Playground of a Pharo 14 image (Do it and print each line)."
((ClyAsyncQueryResultTest selector: #testItemsChangedNotificationShouldResetItems) run) printString.
((ClySemiAsyncQueryResultTest selector: #testItemsChangedNotificationShouldResetItems) run) printString.

Source of the tests before the fix

ClyAsyncQueryResultTest>>testItemsChangedNotificationShouldResetItems before the fix (Pharo 14 build 791)
testItemsChangedNotificationShouldResetItems
	<expectedFailure>
	"Async query should not reset own items which was already built.
	Another test describes that it should only reset actual result"

	super testItemsChangedNotificationShouldResetItems

Fix

How to apply an entries file: file in 00-header.st and the entries file as one chunk (the iospharo script scripts/gen-image-fixes.py makes that chunk). Each entry checks for the defect first and prints FIXED, ADDED or SKIP. Download 00-header.st.

20-Cly-async-results.st (download)
fix value: #ClyAsyncQueryResultTest value: #testItemsChangedNotificationShouldResetItems
	value: [ :src | src includesSubstring: 'expectedFailure' ]
	value: 'testItemsChangedNotificationShouldResetItems
	"The inherited test expects itemsChanged to reset the items. An async result must not:
	ClyAsyncQueryResult>>initializeItems says it never resets existing items, so that the
	browser can swap the items when the new result is ready. itemsChanged only marks the
	result as needing a rebuild and keeps the items built so far."

	queryResult initializeItems.
	self assert: queryResult isBuilt.

	queryResult itemsChanged.

	self assert: queryResult isBuilt.
	self assert: queryResult items isNotNil.
	self assert: queryResult needsRebuild'.

State of 2026-11-20. Made from the test records of the iospharo project (a second Pharo VM). The fixes are proposals for review.