Hi Phil,

On Fri, Nov 7, 2014 at 3:42 AM, phil@highoctane.be <phil@highoctane.be> wrote:

Code (Transcript calls of mine)

open: fileName forWrite: writeMode��
"Open the file with the given name. If writeMode is true, allow writing, otherwise open the file in read-only mode."
| f |
f := fileName asVmPathName.
Transcript log: 'File:', f.

fileID := StandardFileStream retryWithGC:[self primOpen: f writable: writeMode]��
until:[:id| id notNil]��
forFileNamed: fileName.
fileID ifNil: [
' > Cannot get fileID' logCr.
^ nil]. ��"allows sender to detect failure"
(' > Got fileID: ', fileID printString) logCr.
name := fileName.
self register.
rwmode := writeMode.
buffer1 := String new: 1.
self enableReadBuffering

I have a ton of files to open. And some were slow to return a problem.

After digging, I got to the method above.

Now at one moment, I get nil fileIDs.

What is that retryWithGC: until: thing???

retryWithGC: execBlock until: testBlock forFileNamed: fullName "Re-implemented to only force GC if a file with the given name exists" | blockValue foundIt | blockValue := execBlock value. (testBlock value: blockValue) ifTrue:[^blockValue]. "See if we have a file with the given name" foundIt := self registry keys "hold on strongly for now" anySatisfy:[:file| file name sameAs: fullName]. foundIt ifFalse:[^blockValue]. Smalltalk garbageCollectMost. blockValue := execBlock value. (testBlock value: blockValue) ifTrue:[^blockValue]. Smalltalk garbageCollect. ^execBlock value.

I think its horrible personally but it /should/ work.�� The idea is that files shouldn't need to be closed explicitly; which is fine; we want to be able to rely on finalization to close them eventually.�� Hence retryWithGC: is called when an open fails, presumably because it failed because the process is out of file handles, presumably because several as-yet-unfinalized files have yet to be finalized.�� So running the GC /should/ run finalization.�� One issue could be that there are bugs meaning that finalization does /not/ happen.�� For example, Igor could take a look and find out quickly whether that's the case.�� Another issue could be that the finalization loop is at too low a priority, or has been terminated for some reason.

But finalization can not be relied upon to be timely.�� A better solution is to explicity close files where possible using a form such as��forceNewFileNamed:do: et al.�� Another solution is possible in Spur where we can simplify the finalization by using ephemerons so there are no proxies for files.�� But that's not relevant immediately.

Huhhhhh, yeah, some lookup going on in a registry...

and GCs? For file opens?

I am doing these opens in a webapp, what I want is to know if the file is there or not and fail fast, not have all of this thing (debugging kind of nightmarish...)

I have tested with 100 files and of course it all works since I get the IDs back from the registry.

But at one point, it all fails. Registry full?

What to do here?

TIA
Phil

best,

Eliot