[Pharo-project] StandardFileStream size limit?
Hey all! I have just added large file (chunkwise upload and download) support to Kom/Networking/Iliad in Pharo 1.4 and while testing it I tried uploading a 2.9Gb file. I haven't looked at the exact error message, and I haven't tried writing a trivial test - but the "half written" file on disk is almost EXACTLY 2Gb (2147483647) so I am wondering - do we still have a 2Gb file size limit? This is "one click 1.4" on Ubuntu 64 with ext4. regards, Göran
Hey! Ok, so the plot thickens: If I run this in a "oneclick 1.4" I will get a file write error on 2Gb: | f b | f := StandardFileStream newFileNamed: 'test'. b := ByteArray new: 1024*1024*100. "100Mb" [30 timesRepeat: [f nextPutAll: b]] ensure: [f close] "3Gb" ...but it worked fine using a self built "standard" VM from squeakvm.org (4.4.7.2357)! Same image btw. Now, does it work with the bleeding edge Cog? Tried r2556 and nope, same problem. So I "guess" it is a Cog thing? I haven't tried building Cog from source. regards, Göran
On Wed, Jun 13, 2012 at 04:08:14PM +0200, G?ran Krampe wrote:
Hey!
Ok, so the plot thickens:
If I run this in a "oneclick 1.4" I will get a file write error on 2Gb:
| f b | f := StandardFileStream newFileNamed: 'test'. b := ByteArray new: 1024*1024*100. "100Mb" [30 timesRepeat: [f nextPutAll: b]] ensure: [f close] "3Gb"
...but it worked fine using a self built "standard" VM from squeakvm.org (4.4.7.2357)! Same image btw.
Now, does it work with the bleeding edge Cog? Tried r2556 and nope, same problem.
So I "guess" it is a Cog thing? I haven't tried building Cog from source.
regards, G?ran
Background on large file support, see especially Bert's summary: http://forum.world.st/Re-squeak-dev-filesize-reporting-0-for-very-large-file... http://bugs.squeak.org/view.php?id=7522 http://www.suse.de/~aj/linux_lfs.html I'm not sure if the Cog VMs are being compiled with the the LFS option, although I expect that if you compile it yourself with the build options that Bert explains, then it should start working. Dave
On Wed, Jun 13, 2012 at 7:40 AM, David T. Lewis <lewis@mail.msen.com> wrote:
On Wed, Jun 13, 2012 at 04:08:14PM +0200, G?ran Krampe wrote:
Hey!
Ok, so the plot thickens:
If I run this in a "oneclick 1.4" I will get a file write error on 2Gb:
| f b | f := StandardFileStream newFileNamed: 'test'. b := ByteArray new: 1024*1024*100. "100Mb" [30 timesRepeat: [f nextPutAll: b]] ensure: [f close] "3Gb"
...but it worked fine using a self built "standard" VM from squeakvm.org (4.4.7.2357)! Same image btw.
Now, does it work with the bleeding edge Cog? Tried r2556 and nope, same problem.
So I "guess" it is a Cog thing? I haven't tried building Cog from source.
regards, G?ran
Background on large file support, see especially Bert's summary:
http://forum.world.st/Re-squeak-dev-filesize-reporting-0-for-very-large-file... http://bugs.squeak.org/view.php?id=7522 http://www.suse.de/~aj/linux_lfs.html
I'm not sure if the Cog VMs are being compiled with the the LFS option, although I expect that if you compile it yourself with the build options that Bert explains, then it should start working.
I just committed the necessary changes for my branch. Will rebuild soon. Göran, if you're in a hurry build your own? You'd need to be in http://www.squeakvm.org/svn/squeak/branches/Cog/unixbuild/bld and run ./mvm. -- best, Eliot
This still fails three and a half years later with a default Pharo 4 vm and image. :( I spent a few hours trying to diagnose why Pharo was telling me that I file I knew existed didn't exist. Very frustrating trying to debug this and then to find it is a vm and not an image issue. I then to search the archives to find this is a known issue for this long and still broken. The latest linux vm works fine. http://files.pharo.org/vm/pharo/linux/Pharo-VM-linux-latest.zip Is there a reason why the "latest" vm isn't the stable vm? Or at least this bugfix applied to the stable vm? Thanks. Jimmie On 06/13/2012 05:00 PM, eliot.miranda at gmail.com (Eliot Miranda) wrote:
On Wed, Jun 13, 2012 at 7:40 AM, David T. Lewis <lewis at mail.msen.com> wrote:
On Wed, Jun 13, 2012 at 04:08:14PM +0200, G?ran Krampe wrote:
Hey!
Ok, so the plot thickens:
If I run this in a "oneclick 1.4" I will get a file write error on 2Gb:
| f b | f := StandardFileStream newFileNamed: 'test'. b := ByteArray new: 1024*1024*100. "100Mb" [30 timesRepeat: [f nextPutAll: b]] ensure: [f close] "3Gb"
...but it worked fine using a self built "standard" VM from squeakvm.org (4.4.7.2357)! Same image btw.
Now, does it work with the bleeding edge Cog? Tried r2556 and nope, same problem.
So I "guess" it is a Cog thing? I haven't tried building Cog from source.
regards, G?ran Background on large file support, see especially Bert's summary:
http://forum.world.st/Re-squeak-dev-filesize-reporting-0-for-very-large-file... http://bugs.squeak.org/view.php?id=7522 http://www.suse.de/~aj/linux_lfs.html
I'm not sure if the Cog VMs are being compiled with the the LFS option, although I expect that if you compile it yourself with the build options that Bert explains, then it should start working.
I just committed the necessary changes for my branch. Will rebuild soon. G?ran, if you're in a hurry build your own? You'd need to be in http://www.squeakvm.org/svn/squeak/branches/Cog/unixbuild/bld and run ./mvm.
2015-11-12 6:14 GMT+01:00 Jimmie Houchin <jlhouchin@gmail.com>:
This still fails three and a half years later with a default Pharo 4 vm and image. :(
I spent a few hours trying to diagnose why Pharo was telling me that I file I knew existed didn't exist. Very frustrating trying to debug this and then to find it is a vm and not an image issue. I then to search the archives to find this is a known issue for this long and still broken.
The latest linux vm works fine. http://files.pharo.org/vm/pharo/linux/Pharo-VM-linux-latest.zip
Yes, the latest vm is pretty stable. But I think the problem is, there are to many changes for to many different projects: - I added a change to the windows keyevent handling and created a small regression (can not use ctrl+space anymore) - changes for OSWindow /SDL - spur (?) or any other changes from the main squeak vm - ... I think we should have create release candidate branch and make sure more people testing that version and all newer changes go to the master/development branch only. nicolai
Is there a reason why the "latest" vm isn't the stable vm? Or at least this bugfix applied to the stable vm?
Thanks.
Jimmie
On 06/13/2012 05:00 PM, eliot.miranda at gmail.com (Eliot Miranda) wrote:
On Wed, Jun 13, 2012 at 7:40 AM, David T. Lewis <lewis at mail.msen.com> wrote:
On Wed, Jun 13, 2012 at 04:08:14PM +0200, G?ran Krampe wrote:
Hey!
Ok, so the plot thickens:
If I run this in a "oneclick 1.4" I will get a file write error on 2Gb:
| f b | f := StandardFileStream newFileNamed: 'test'. b := ByteArray new: 1024*1024*100. "100Mb" [30 timesRepeat: [f nextPutAll: b]] ensure: [f close] "3Gb"
...but it worked fine using a self built "standard" VM from squeakvm.org (4.4.7.2357)! Same image btw.
Now, does it work with the bleeding edge Cog? Tried r2556 and nope, same problem.
So I "guess" it is a Cog thing? I haven't tried building Cog from source.
regards, G?ran
Background on large file support, see especially Bert's summary:
http://forum.world.st/Re-squeak-dev-filesize-reporting-0-for-very-large-file... http://bugs.squeak.org/view.php?id=7522 http://www.suse.de/~aj/linux_lfs.html
I'm not sure if the Cog VMs are being compiled with the the LFS option, although I expect that if you compile it yourself with the build options that Bert explains, then it should start working.
I just committed the necessary changes for my branch. Will rebuild soon.
G?ran, if you're in a hurry build your own? You'd need to be in http://www.squeakvm.org/svn/squeak/branches/Cog/unixbuild/bld and run ./mvm.
latest vm has issues with OSProcess stdout the last time I checked on Yosemite. I would not call it stable unless we talk only crashes here. On Thu, Nov 12, 2015 at 2:55 PM Nicolai Hess <nicolaihess@gmail.com> wrote:
2015-11-12 6:14 GMT+01:00 Jimmie Houchin <jlhouchin@gmail.com>:
This still fails three and a half years later with a default Pharo 4 vm and image. :(
I spent a few hours trying to diagnose why Pharo was telling me that I file I knew existed didn't exist. Very frustrating trying to debug this and then to find it is a vm and not an image issue. I then to search the archives to find this is a known issue for this long and still broken.
The latest linux vm works fine. http://files.pharo.org/vm/pharo/linux/Pharo-VM-linux-latest.zip
Yes, the latest vm is pretty stable. But I think the problem is, there are to many changes for to many different projects: - I added a change to the windows keyevent handling and created a small regression (can not use ctrl+space anymore) - changes for OSWindow /SDL - spur (?) or any other changes from the main squeak vm - ...
I think we should have create release candidate branch and make sure more people testing that version and all newer changes go to the master/development branch only.
nicolai
Is there a reason why the "latest" vm isn't the stable vm? Or at least this bugfix applied to the stable vm?
Thanks.
Jimmie
On 06/13/2012 05:00 PM, eliot.miranda at gmail.com (Eliot Miranda) wrote:
On Wed, Jun 13, 2012 at 7:40 AM, David T. Lewis <lewis at mail.msen.com> wrote:
On Wed, Jun 13, 2012 at 04:08:14PM +0200, G?ran Krampe wrote:
Hey!
Ok, so the plot thickens:
If I run this in a "oneclick 1.4" I will get a file write error on 2Gb:
| f b | f := StandardFileStream newFileNamed: 'test'. b := ByteArray new: 1024*1024*100. "100Mb" [30 timesRepeat: [f nextPutAll: b]] ensure: [f close] "3Gb"
...but it worked fine using a self built "standard" VM from squeakvm.org (4.4.7.2357)! Same image btw.
Now, does it work with the bleeding edge Cog? Tried r2556 and nope, same problem.
So I "guess" it is a Cog thing? I haven't tried building Cog from source.
regards, G?ran
Background on large file support, see especially Bert's summary:
http://forum.world.st/Re-squeak-dev-filesize-reporting-0-for-very-large-file... http://bugs.squeak.org/view.php?id=7522 http://www.suse.de/~aj/linux_lfs.html
I'm not sure if the Cog VMs are being compiled with the the LFS option, although I expect that if you compile it yourself with the build options that Bert explains, then it should start working.
I just committed the necessary changes for my branch. Will rebuild
soon. G?ran, if you're in a hurry build your own? You'd need to be in http://www.squeakvm.org/svn/squeak/branches/Cog/unixbuild/bld and run ./mvm.
2015-11-12 14:10 GMT+01:00 Dimitris Chloupis <kilon.alios@gmail.com>:
latest vm has issues with OSProcess stdout the last time I checked on Yosemite. I would not call it stable unless we talk only crashes here.
Ah, I didn't know this was a vm issue. Does this means it is working with the "official" stable vm ?
On Thu, Nov 12, 2015 at 2:55 PM Nicolai Hess <nicolaihess@gmail.com> wrote:
2015-11-12 6:14 GMT+01:00 Jimmie Houchin <jlhouchin@gmail.com>:
This still fails three and a half years later with a default Pharo 4 vm and image. :(
I spent a few hours trying to diagnose why Pharo was telling me that I file I knew existed didn't exist. Very frustrating trying to debug this and then to find it is a vm and not an image issue. I then to search the archives to find this is a known issue for this long and still broken.
The latest linux vm works fine. http://files.pharo.org/vm/pharo/linux/Pharo-VM-linux-latest.zip
Yes, the latest vm is pretty stable. But I think the problem is, there are to many changes for to many different projects: - I added a change to the windows keyevent handling and created a small regression (can not use ctrl+space anymore) - changes for OSWindow /SDL - spur (?) or any other changes from the main squeak vm - ...
I think we should have create release candidate branch and make sure more people testing that version and all newer changes go to the master/development branch only.
nicolai
Is there a reason why the "latest" vm isn't the stable vm? Or at least this bugfix applied to the stable vm?
Thanks.
Jimmie
On 06/13/2012 05:00 PM, eliot.miranda at gmail.com (Eliot Miranda) wrote:
On Wed, Jun 13, 2012 at 7:40 AM, David T. Lewis <lewis at mail.msen.com> wrote:
On Wed, Jun 13, 2012 at 04:08:14PM +0200, G?ran Krampe wrote:
Hey!
Ok, so the plot thickens:
If I run this in a "oneclick 1.4" I will get a file write error on 2Gb:
| f b | f := StandardFileStream newFileNamed: 'test'. b := ByteArray new: 1024*1024*100. "100Mb" [30 timesRepeat: [f nextPutAll: b]] ensure: [f close] "3Gb"
...but it worked fine using a self built "standard" VM from squeakvm.org (4.4.7.2357)! Same image btw.
Now, does it work with the bleeding edge Cog? Tried r2556 and nope, same problem.
So I "guess" it is a Cog thing? I haven't tried building Cog from source.
regards, G?ran
Background on large file support, see especially Bert's summary:
http://forum.world.st/Re-squeak-dev-filesize-reporting-0-for-very-large-file... http://bugs.squeak.org/view.php?id=7522 http://www.suse.de/~aj/linux_lfs.html
I'm not sure if the Cog VMs are being compiled with the the LFS option, although I expect that if you compile it yourself with the build options that Bert explains, then it should start working.
I just committed the necessary changes for my branch. Will rebuild
soon. G?ran, if you're in a hurry build your own? You'd need to be in http://www.squeakvm.org/svn/squeak/branches/Cog/unixbuild/bld and run ./mvm.
2015-11-12 14:15 GMT+01:00 Nicolai Hess <nicolaihess@gmail.com>:
2015-11-12 14:10 GMT+01:00 Dimitris Chloupis <kilon.alios@gmail.com>:
latest vm has issues with OSProcess stdout the last time I checked on Yosemite. I would not call it stable unless we talk only crashes here.
Ah, I didn't know this was a vm issue. Does this means it is working with the "official" stable vm ?
Can you check if OSProcess is working with the stable vm? This may help to fix this issue. Thanks nicolai
On Thu, Nov 12, 2015 at 2:55 PM Nicolai Hess <nicolaihess@gmail.com> wrote:
2015-11-12 6:14 GMT+01:00 Jimmie Houchin <jlhouchin@gmail.com>:
This still fails three and a half years later with a default Pharo 4 vm and image. :(
I spent a few hours trying to diagnose why Pharo was telling me that I file I knew existed didn't exist. Very frustrating trying to debug this and then to find it is a vm and not an image issue. I then to search the archives to find this is a known issue for this long and still broken.
The latest linux vm works fine. http://files.pharo.org/vm/pharo/linux/Pharo-VM-linux-latest.zip
Yes, the latest vm is pretty stable. But I think the problem is, there are to many changes for to many different projects: - I added a change to the windows keyevent handling and created a small regression (can not use ctrl+space anymore) - changes for OSWindow /SDL - spur (?) or any other changes from the main squeak vm - ...
I think we should have create release candidate branch and make sure more people testing that version and all newer changes go to the master/development branch only.
nicolai
Is there a reason why the "latest" vm isn't the stable vm? Or at least this bugfix applied to the stable vm?
Thanks.
Jimmie
On 06/13/2012 05:00 PM, eliot.miranda at gmail.com (Eliot Miranda) wrote:
On Wed, Jun 13, 2012 at 7:40 AM, David T. Lewis <lewis at mail.msen.com> wrote:
On Wed, Jun 13, 2012 at 04:08:14PM +0200, G?ran Krampe wrote:
Hey!
Ok, so the plot thickens:
If I run this in a "oneclick 1.4" I will get a file write error on 2Gb:
| f b | f := StandardFileStream newFileNamed: 'test'. b := ByteArray new: 1024*1024*100. "100Mb" [30 timesRepeat: [f nextPutAll: b]] ensure: [f close] "3Gb"
...but it worked fine using a self built "standard" VM from squeakvm.org (4.4.7.2357)! Same image btw.
Now, does it work with the bleeding edge Cog? Tried r2556 and nope, same problem.
So I "guess" it is a Cog thing? I haven't tried building Cog from source.
regards, G?ran
Background on large file support, see especially Bert's summary:
http://forum.world.st/Re-squeak-dev-filesize-reporting-0-for-very-large-file... http://bugs.squeak.org/view.php?id=7522 http://www.suse.de/~aj/linux_lfs.html
I'm not sure if the Cog VMs are being compiled with the the LFS option, although I expect that if you compile it yourself with the build options that Bert explains, then it should start working.
I just committed the necessary changes for my branch. Will rebuild
soon. G?ran, if you're in a hurry build your own? You'd need to be in http://www.squeakvm.org/svn/squeak/branches/Cog/unixbuild/bld and run ./mvm.
2015-11-13 16:24 GMT+01:00 Nicolai Hess <nicolaihess@gmail.com>:
2015-11-12 14:15 GMT+01:00 Nicolai Hess <nicolaihess@gmail.com>:
2015-11-12 14:10 GMT+01:00 Dimitris Chloupis <kilon.alios@gmail.com>:
latest vm has issues with OSProcess stdout the last time I checked on Yosemite. I would not call it stable unless we talk only crashes here.
Ah, I didn't know this was a vm issue. Does this means it is working with the "official" stable vm ?
Can you check if OSProcess is working with the stable vm? This may help to fix this issue.
OSProcess works on the stable vm on Mac OS X, and on the stable and on the latest vm on Linux. Thierry
Thanks nicolai
On Thu, Nov 12, 2015 at 2:55 PM Nicolai Hess <nicolaihess@gmail.com> wrote:
2015-11-12 6:14 GMT+01:00 Jimmie Houchin <jlhouchin@gmail.com>:
This still fails three and a half years later with a default Pharo 4 vm and image. :(
I spent a few hours trying to diagnose why Pharo was telling me that I file I knew existed didn't exist. Very frustrating trying to debug this and then to find it is a vm and not an image issue. I then to search the archives to find this is a known issue for this long and still broken.
The latest linux vm works fine. http://files.pharo.org/vm/pharo/linux/Pharo-VM-linux-latest.zip
Yes, the latest vm is pretty stable. But I think the problem is, there are to many changes for to many different projects: - I added a change to the windows keyevent handling and created a small regression (can not use ctrl+space anymore) - changes for OSWindow /SDL - spur (?) or any other changes from the main squeak vm - ...
I think we should have create release candidate branch and make sure more people testing that version and all newer changes go to the master/development branch only.
nicolai
Is there a reason why the "latest" vm isn't the stable vm? Or at least this bugfix applied to the stable vm?
Thanks.
Jimmie
On 06/13/2012 05:00 PM, eliot.miranda at gmail.com (Eliot Miranda) wrote:
On Wed, Jun 13, 2012 at 7:40 AM, David T. Lewis <lewis at mail.msen.com> wrote:
On Wed, Jun 13, 2012 at 04:08:14PM +0200, G?ran Krampe wrote:
Hey!
Ok, so the plot thickens:
If I run this in a "oneclick 1.4" I will get a file write error on 2Gb:
| f b | f := StandardFileStream newFileNamed: 'test'. b := ByteArray new: 1024*1024*100. "100Mb" [30 timesRepeat: [f nextPutAll: b]] ensure: [f close] "3Gb"
...but it worked fine using a self built "standard" VM from squeakvm.org (4.4.7.2357)! Same image btw.
Now, does it work with the bleeding edge Cog? Tried r2556 and nope, same problem.
So I "guess" it is a Cog thing? I haven't tried building Cog from source.
regards, G?ran
Background on large file support, see especially Bert's summary:
http://forum.world.st/Re-squeak-dev-filesize-reporting-0-for-very-large-file... http://bugs.squeak.org/view.php?id=7522 http://www.suse.de/~aj/linux_lfs.html
I'm not sure if the Cog VMs are being compiled with the the LFS option, although I expect that if you compile it yourself with the build options that Bert explains, then it should start working.
I just committed the necessary changes for my branch. Will rebuild
soon. G?ran, if you're in a hurry build your own? You'd need to be in http://www.squeakvm.org/svn/squeak/branches/Cog/unixbuild/bld and run ./mvm.
On Wed, Jun 13, 2012 at 12:35 AM, Göran Krampe <goran@krampe.se> wrote:
Hey all!
I have just added large file (chunkwise upload and download) support to Kom/Networking/Iliad in Pharo 1.4 and while testing it I tried uploading a 2.9Gb file.
I haven't looked at the exact error message, and I haven't tried writing a trivial test - but the "half written" file on disk is almost EXACTLY 2Gb (2147483647) so I am wondering - do we still have a 2Gb file size limit?
I think so. Still need to add the relevant define (LARGEFILE_SOURCE or some such) to the linux build to enable 64-bit file support.
This is "one click 1.4" on Ubuntu 64 with ext4.
regards, Göran
-- best, Eliot
participants (7)
-
David T. Lewis -
Dimitris Chloupis -
Eliot Miranda -
Göran Krampe -
Jimmie Houchin -
Nicolai Hess -
Thierry Goubier