Pavel's ChangeLog week of 2017-01-01
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the disk from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared. File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this: readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ]. However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative. Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea too so to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly. - The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its model. Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it. Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful. Cheers, -- Pavel
ehh, new year :-D so 2018-01-01 2018-01-08 10:30 GMT+01:00 Pavel Krivanek <pavel.krivanek@gmail.com>:
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the disk from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared.
File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this:
readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ].
However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative.
Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea too so to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly. - The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its model. Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it.
Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful.
Cheers, -- Pavel
On 8 Jan 2018, at 10:30, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the disk from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared.
File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this:
readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ].
Yes, that is correct.
However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative.
This is great work, Pavel, I know Marcus wanted this for a loooooong time ;-) Now, I thought (naively maybe) that in memory sources meant that each method object holds a single (compressed) string with its own individual source code. In such a system no positioning in a huge stream would (ever) be necessary, no ? AFAIK, the need for positioning is only to get to the right source bit in this huge pool.
Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea too so to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly. - The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its model. Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it.
Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful.
Cheers, -- Pavel
2018-01-08 10:39 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:
On 8 Jan 2018, at 10:30, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the disk from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared.
File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this:
readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ].
Yes, that is correct.
However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative.
This is great work, Pavel, I know Marcus wanted this for a loooooong time ;-)
Now, I thought (naively maybe) that in memory sources meant that each method object holds a single (compressed) string with its own individual source code. In such a system no positioning in a huge stream would (ever) be necessary, no ?
I use an implementation that should be more memory efficient and make much less stress for the GC. It uses the old concept of pointers to a sources file but the sources file is a set of compressed segments stored in memory. When all sources compressed as a single file take about 6MB, with compression by segments it takes about 7MB. I expect that for very small methods the compressed version can save no memory at all and you would need a next extra object for every method. -- Pavel
AFAIK, the need for positioning is only to get to the right source bit in this huge pool.
Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea too so to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly. - The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its model. Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it.
Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful.
Cheers, -- Pavel
On Mon, Jan 8, 2018 at 6:30 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the disk from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared.
File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this:
readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ].
However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative.
Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea too so to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly.
Great news. I didn't look at Ring 2 code recently but I say your esug talk and I imagine it will fit well for the change model and UI tool. What may need attention is serialization to text file. Right now, the Ring 1 object returned by asRingDefinition (+ some tweaks) is serialized and materialized via STON into the .ombu files. With Ring 2 serialized with STON, the file format will be different and in consequence the new epicea won't be able to open a .ombu file generated in Ring 1. Not sure if Pharo community find this acceptable. IMO it is. - If not-acceptable, there could be some workaround. - If acceptable, that's the easiest path, and also it could be a good opportunity to improve the file format: It could be more compact and even have a better extension such as .epicea or something that comunicates that tthose files have code changes or sessions. cheers, MartÃn
- The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its model. Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it.
Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful.
Cheers, -- Pavel
2018-01-08 16:27 GMT+01:00 Martin Dias <tinchodias@gmail.com>:
On Mon, Jan 8, 2018 at 6:30 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the disk from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared.
File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this:
readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ].
However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative.
Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea too so to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly.
Great news. I didn't look at Ring 2 code recently but I say your esug talk and I imagine it will fit well for the change model and UI tool.
What may need attention is serialization to text file. Right now, the Ring 1 object returned by asRingDefinition (+ some tweaks) is serialized and materialized via STON into the .ombu files. With Ring 2 serialized with STON, the file format will be different and in consequence the new epicea won't be able to open a .ombu file generated in Ring 1. Not sure if Pharo community find this acceptable. IMO it is. - If not-acceptable, there could be some workaround. - If acceptable, that's the easiest path, and also it could be a good opportunity to improve the file format: It could be more compact and even have a better extension such as .epicea or something that comunicates that tthose files have code changes or sessions.
I think that the change of the format is not a a practical issue because probably no-one will need to read the old Ombu files from a different Pharo version. But a change of the format will be handy because every Ring2 model is a standalone environment and Ombu thus saves all information that it includes. So a simple method change creates a record with almost 400 lines. It must be of course much slower too. -- Pavel
cheers, MartÃn
- The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its model. Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it.
Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful.
Cheers, -- Pavel
Hi Pavel I was thinking that it would be nice to have a simple DB build in Pharo (I thought that CDB could do the job) to manage source. Because I like the idea that when the system crashes I only lose at max the method I'm editing. Esteban did once a try to connect a tokyo tyrant db now I would prefer to avoid to have FFI in the middle. To me we should **********stabilise*********** pharo for real. Stef On Mon, Jan 8, 2018 at 4:50 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
2018-01-08 16:27 GMT+01:00 Martin Dias <tinchodias@gmail.com>:
On Mon, Jan 8, 2018 at 6:30 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the disk from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared.
File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this:
readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ].
However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative.
Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea too so to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly.
Great news. I didn't look at Ring 2 code recently but I say your esug talk and I imagine it will fit well for the change model and UI tool.
What may need attention is serialization to text file. Right now, the Ring 1 object returned by asRingDefinition (+ some tweaks) is serialized and materialized via STON into the .ombu files. With Ring 2 serialized with STON, the file format will be different and in consequence the new epicea won't be able to open a .ombu file generated in Ring 1. Not sure if Pharo community find this acceptable. IMO it is. - If not-acceptable, there could be some workaround. - If acceptable, that's the easiest path, and also it could be a good opportunity to improve the file format: It could be more compact and even have a better extension such as .epicea or something that comunicates that tthose files have code changes or sessions.
I think that the change of the format is not a a practical issue because probably no-one will need to read the old Ombu files from a different Pharo version. But a change of the format will be handy because every Ring2 model is a standalone environment and Ombu thus saves all information that it includes. So a simple method change creates a record with almost 400 lines. It must be of course much slower too.
-- Pavel
cheers, MartÃn
- The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its model. Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it.
Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful.
Cheers, -- Pavel
2018-01-08 22:42 GMT+01:00 Stephane Ducasse <stepharo.self@gmail.com>:
Hi Pavel
I was thinking that it would be nice to have a simple DB build in Pharo (I thought that CDB could do the job) to manage source. Because I like the idea that when the system crashes I only lose at max the method I'm editing. Esteban did once a try to connect a tokyo tyrant db now I would prefer to avoid to have FFI in the middle. To me we should **********stabilise*********** pharo for real.
There is a lot of things we want to change in the system. Just integration of Calypso and removal of the old FileStream users could cause a lot of issues. So we should really think what to do with Pharo 7 release. Of course I would like to see a lot of the cool new features in Pharo 7 but we will probably need to decide between some of them and stability. Compressed in-memory sources are of course not really important. -- Pavel
Stef
On Mon, Jan 8, 2018 at 4:50 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
2018-01-08 16:27 GMT+01:00 Martin Dias <tinchodias@gmail.com>:
On Mon, Jan 8, 2018 at 6:30 AM, Pavel Krivanek <
pavel.krivanek@gmail.com>
wrote:
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the
disk
from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared.
File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this:
readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ].
However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative.
Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea
too so
to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly.
Great news. I didn't look at Ring 2 code recently but I say your esug talk and I imagine it will fit well for the change model and UI tool.
What may need attention is serialization to text file. Right now, the Ring 1 object returned by asRingDefinition (+ some tweaks) is serialized and materialized via STON into the .ombu files. With Ring 2 serialized with STON, the file format will be different and in consequence the new epicea won't be able to open a .ombu file generated in Ring 1. Not sure if Pharo community find this acceptable. IMO it is. - If not-acceptable, there could be some workaround. - If acceptable, that's the easiest path, and also it could be a good opportunity to improve the file format: It could be more compact and even have a better extension such as .epicea or something that comunicates that tthose files have code changes or sessions.
I think that the change of the format is not a a practical issue because probably no-one will need to read the old Ombu files from a different Pharo version. But a change of the format will be handy because every Ring2 model is a standalone environment and Ombu thus saves all information that it includes. So a simple method change creates a record with almost 400 lines. It must be of course much slower too.
-- Pavel
cheers, MartÃn
- The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its
model.
Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it.
Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful.
Cheers, -- Pavel
On 8 Jan 2018, at 23:09, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
2018-01-08 22:42 GMT+01:00 Stephane Ducasse <stepharo.self@gmail.com <mailto:stepharo.self@gmail.com>>: Hi Pavel
I was thinking that it would be nice to have a simple DB build in Pharo (I thought that CDB could do the job) to manage source. Because I like the idea that when the system crashes I only lose at max the method I'm editing.
cdb cannot because it means âconstant databaseâ : once created, cannot be modified and we will need to install sources there. tyrantdb was just a PoC⦠it was cool, but just that⦠lot of work to make it a real replacement (and also, there is the FFI dependence problem).
Esteban did once a try to connect a tokyo tyrant db now I would prefer to avoid to have FFI in the middle. To me we should **********stabilise*********** pharo for real.
There is a lot of things we want to change in the system. Just integration of Calypso and removal of the old FileStream users could cause a lot of issues. So we should really think what to do with Pharo 7 release. Of course I would like to see a lot of the cool new features in Pharo 7 but we will probably need to decide between some of them and stability. Compressed in-memory sources are of course not really important.
yes. I still didnât do the âfreezeâ call because there are tons of things that still need a lot of work. and we still need to introduce SISTA (preview). but I would not introduce anything else right now, and that includes Calypso, image-based-sources (or DB sources) and any of the cool stuff we have in the pipeline⦠just because if we introduce it we will release super later. Esteban
-- Pavel
Stef
On Mon, Jan 8, 2018 at 4:50 PM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:
2018-01-08 16:27 GMT+01:00 Martin Dias <tinchodias@gmail.com <mailto:tinchodias@gmail.com>>:
On Mon, Jan 8, 2018 at 6:30 AM, Pavel Krivanek <pavel.krivanek@gmail.com <mailto:pavel.krivanek@gmail.com>> wrote:
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the disk from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared.
File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this:
readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ].
However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative.
Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea too so to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly.
Great news. I didn't look at Ring 2 code recently but I say your esug talk and I imagine it will fit well for the change model and UI tool.
What may need attention is serialization to text file. Right now, the Ring 1 object returned by asRingDefinition (+ some tweaks) is serialized and materialized via STON into the .ombu files. With Ring 2 serialized with STON, the file format will be different and in consequence the new epicea won't be able to open a .ombu file generated in Ring 1. Not sure if Pharo community find this acceptable. IMO it is. - If not-acceptable, there could be some workaround. - If acceptable, that's the easiest path, and also it could be a good opportunity to improve the file format: It could be more compact and even have a better extension such as .epicea or something that comunicates that tthose files have code changes or sessions.
I think that the change of the format is not a a practical issue because probably no-one will need to read the old Ombu files from a different Pharo version. But a change of the format will be handy because every Ring2 model is a standalone environment and Ombu thus saves all information that it includes. So a simple method change creates a record with almost 400 lines. It must be of course much slower too.
-- Pavel
cheers, MartÃn
- The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its model. Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it.
Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful.
Cheers, -- Pavel
Hi esteban I did not get your point on Calypso. To me we should remove nautilus and use Calypso since denis is around. Stef On Mon, Jan 8, 2018 at 11:27 PM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 8 Jan 2018, at 23:09, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
2018-01-08 22:42 GMT+01:00 Stephane Ducasse <stepharo.self@gmail.com>:
Hi Pavel
I was thinking that it would be nice to have a simple DB build in Pharo (I thought that CDB could do the job) to manage source. Because I like the idea that when the system crashes I only lose at max the method I'm editing.
cdb cannot because it means âconstant databaseâ : once created, cannot be modified and we will need to install sources there. tyrantdb was just a PoC⦠it was cool, but just that⦠lot of work to make it a real replacement (and also, there is the FFI dependence problem).
Esteban did once a try to connect a tokyo tyrant db now I would prefer to avoid to have FFI in the middle. To me we should **********stabilise*********** pharo for real.
There is a lot of things we want to change in the system. Just integration of Calypso and removal of the old FileStream users could cause a lot of issues. So we should really think what to do with Pharo 7 release. Of course I would like to see a lot of the cool new features in Pharo 7 but we will probably need to decide between some of them and stability. Compressed in-memory sources are of course not really important.
yes. I still didnât do the âfreezeâ call because there are tons of things that still need a lot of work. and we still need to introduce SISTA (preview).
but I would not introduce anything else right now, and that includes Calypso, image-based-sources (or DB sources) and any of the cool stuff we have in the pipeline⦠just because if we introduce it we will release super later.
Esteban
-- Pavel
Stef
On Mon, Jan 8, 2018 at 4:50 PM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
2018-01-08 16:27 GMT+01:00 Martin Dias <tinchodias@gmail.com>:
On Mon, Jan 8, 2018 at 6:30 AM, Pavel Krivanek <pavel.krivanek@gmail.com> wrote:
In memory sources file - I slightly improved the PR for integration of the sources file inside the object memory. Now it will be possible to extract them back to the disk from the menu so everything can work as before. I'm not sure if it will be integrated into Pharo 7 or we will wait for Pharo 8 but it seems to be prepared.
File API and caching - because the in-memory sources file do not use deprecated MultiByteFileStream, I played with the new file API and discovered that ZnCharacterReadStream on File is about 10 times slower than the old MultiByteFileStream. That's because ZnCharacterReadStream does not use any cache. For better speed the ZnBufferedReadStream needs to be used like this:
readStream := File openForReadFileNamed: 'file.txt'. zn := ZnCharacterReadStream on: readStream encoding: #UTF8. buffered := ZnBufferedReadStream on: zn. [ buffered atEnd ] whileFalse: [ buffered next: 20000 ].
However the ZnBufferedReadStream is by design only a linear stream and the position cannot be changed and thus it cannot be used for *.sources files etc. We will need to prepare a positionable alternative.
Ring 2 integration - I was working on replacing of the old Ring with the new reimplementation. The old Ring is used on many sensitive places in the system. Mainly in tools however it is used by Monticello and Epicea too so to try to remove it is like to cut a branch below yourself on an undermined tree. The approach of the old and new Ring is in many senses very different and the full compatibility will not be supported. I partly provide compatibility API, partly adopt the code directly.
Great news. I didn't look at Ring 2 code recently but I say your esug talk and I imagine it will fit well for the change model and UI tool.
What may need attention is serialization to text file. Right now, the Ring 1 object returned by asRingDefinition (+ some tweaks) is serialized and materialized via STON into the .ombu files. With Ring 2 serialized with STON, the file format will be different and in consequence the new epicea won't be able to open a .ombu file generated in Ring 1. Not sure if Pharo community find this acceptable. IMO it is. - If not-acceptable, there could be some workaround. - If acceptable, that's the easiest path, and also it could be a good opportunity to improve the file format: It could be more compact and even have a better extension such as .epicea or something that comunicates that tthose files have code changes or sessions.
I think that the change of the format is not a a practical issue because probably no-one will need to read the old Ombu files from a different Pharo version. But a change of the format will be handy because every Ring2 model is a standalone environment and Ombu thus saves all information that it includes. So a simple method change creates a record with almost 400 lines. It must be of course much slower too.
-- Pavel
cheers, MartÃn
- The real system and the model is mixed many times in the users of the old Ring. For example a model for a method is created but then when the package of the method is asked, it is already a real package, not its model. Such places need to be cleaned. - I already have an image that has no old Ring code at all but because of some memory leaks caused by the mixtures of the models and real system I was not able to bootstrap it successfully. More work is needed. - I discovered that the image has a serious memory leak caused by Iceberg/GT and Announcers but I still need to find more information about it.
Smalltalk archeology - during the free days on the beginning of the year I looked at Smalltalk-78. I extracted the data from Lively Kernel so the original image can read from external files, not only the recent updated ones. - Then I looked at SqueakJS again and tried to reproduce Craig's experiments with Pharo on it. Generally it is about 100 times slower than native Pharo which makes it hard to use for real-life tasks. However it would be pity to do not provide at least basic attention to it because in some cases it may be really useful.
Cheers, -- Pavel
participants (5)
-
Esteban Lorenzano -
Martin Dias -
Pavel Krivanek -
Stephane Ducasse -
Sven Van Caekenberghe