Fuel materialization on spur
We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel? thanks, Norbert primitive #value: in Character class failed Stacktrace Character class(Object)>>primitiveFailed: Character class(Object)>>primitiveFailed Character class>>value: Character class>>materializeFrom: FLHookPrimitiveCluster>>materializeInstanceWith: FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [ self clusterInstancesStep ] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [ :aDecoder | (FLMaterialization with: aDecoder) run; yourself ] in FLMaterializer>>setDefaultMaterialization FLMaterializer>>materializeFrom: FLMaterializer class>>materializeFromByteArray: MAPExampleModels class>>readModelFromSelector: MAPExampleModels class>>readModelNamed: [ self readModelNamed: aString ] in MAPExampleModels class>>named: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: MAPExampleModels class>>named: MAPTest>>model MAPTest>>tcapModule MAPTest>>testInsertSubscriberDataBitStringAccess [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: Given>>produceReturnValueAt: MAPTest(Phexample)>>performTest
On 12/15/2015 07:18 AM, Norbert Hartl wrote:
We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that...
+1
or will it just not be possible to exchange objects between spur and non-spur using fuel?
thanks,
Norbert
primitive #value: in Character class failed
Stacktrace
Character class(Object)>>primitiveFailed: Character class(Object)>>primitiveFailed Character class>>value: Character class>>materializeFrom: FLHookPrimitiveCluster>>materializeInstanceWith: FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [ self clusterInstancesStep ] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [ :aDecoder | (FLMaterialization with: aDecoder) run; yourself ] in FLMaterializer>>setDefaultMaterialization FLMaterializer>>materializeFrom: FLMaterializer class>>materializeFromByteArray: MAPExampleModels class>>readModelFromSelector: MAPExampleModels class>>readModelNamed: [ self readModelNamed: aString ] in MAPExampleModels class>>named: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: MAPExampleModels class>>named: MAPTest>>model MAPTest>>tcapModule MAPTest>>testInsertSubscriberDataBitStringAccess [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: Given>>produceReturnValueAt: MAPTest(Phexample)>>performTest
-- . .. .. ^,^ best, robert
On 15 Dec 2015, at 13:18, Norbert Hartl <norbert@hartl.name> wrote:
We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?
I think is not possible, but I will let fuel maintainers to have the last word :) Esteban
thanks,
Norbert
primitive #value: in Character class failed Stacktrace
Character class(Object)>>primitiveFailed: Character class(Object)>>primitiveFailed Character class>>value: Character class>>materializeFrom: FLHookPrimitiveCluster>>materializeInstanceWith: FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [ self clusterInstancesStep ] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [ :aDecoder | (FLMaterialization with: aDecoder) run; yourself ] in FLMaterializer>>setDefaultMaterialization FLMaterializer>>materializeFrom: FLMaterializer class>>materializeFromByteArray: MAPExampleModels class>>readModelFromSelector: MAPExampleModels class>>readModelNamed: [ self readModelNamed: aString ] in MAPExampleModels class>>named: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: MAPExampleModels class>>named: MAPTest>>model MAPTest>>tcapModule MAPTest>>testInsertSubscriberDataBitStringAccess [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: Given>>produceReturnValueAt: MAPTest(Phexample)>>performTest
On 15 Dec 2015, at 14:06, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 15 Dec 2015, at 13:18, Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>> wrote:
We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?
I think is not possible, but I will let fuel maintainers to have the last word :)
Thatâs right. In general itâs not possible (at the moment). If thereâs demand, we could try to come up with a conversion package which can be used to load non-spur files.
Esteban
thanks,
Norbert
primitive #value: in Character class failed Stacktrace
Character class(Object)>>primitiveFailed: Character class(Object)>>primitiveFailed Character class>>value: Character class>>materializeFrom: FLHookPrimitiveCluster>>materializeInstanceWith: FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [ self clusterInstancesStep ] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [ :aDecoder | (FLMaterialization with: aDecoder) run; yourself ] in FLMaterializer>>setDefaultMaterialization FLMaterializer>>materializeFrom: FLMaterializer class>>materializeFromByteArray: MAPExampleModels class>>readModelFromSelector: MAPExampleModels class>>readModelNamed: [ self readModelNamed: aString ] in MAPExampleModels class>>named: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: MAPExampleModels class>>named: MAPTest>>model MAPTest>>tcapModule MAPTest>>testInsertSubscriberDataBitStringAccess [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: Given>>produceReturnValueAt: MAPTest(Phexample)>>performTest
Hi Esteban,
On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 15 Dec 2015, at 13:18, Norbert Hartl <norbert@hartl.name> wrote:
We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?
I think is not possible, but I will let fuel maintainers to have the last word :)
Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate. In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not.
Esteban
thanks,
Norbert
primitive #value: in Character class failed Stacktrace
Character class(Object)>>primitiveFailed: Character class(Object)>>primitiveFailed Character class>>value: Character class>>materializeFrom: FLHookPrimitiveCluster>>materializeInstanceWith: FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [ self clusterInstancesStep ] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [ :aDecoder | (FLMaterialization with: aDecoder) run; yourself ] in FLMaterializer>>setDefaultMaterialization FLMaterializer>>materializeFrom: FLMaterializer class>>materializeFromByteArray: MAPExampleModels class>>readModelFromSelector: MAPExampleModels class>>readModelNamed: [ self readModelNamed: aString ] in MAPExampleModels class>>named: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: MAPExampleModels class>>named: MAPTest>>model MAPTest>>tcapModule MAPTest>>testInsertSubscriberDataBitStringAccess [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: Given>>produceReturnValueAt: MAPTest(Phexample)>>performTest
Hi,
On 16 Dec 2015, at 08:21, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Esteban,
On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <estebanlm@gmail.com <mailto:estebanlm@gmail.com>> wrote:
On 15 Dec 2015, at 13:18, Norbert Hartl <norbert@hartl.name <mailto:norbert@hartl.name>> wrote:
We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?
I think is not possible, but I will let fuel maintainers to have the last word :)
Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate.
Oh well⦠of course is possible. What I meant is that I think current implementation of Fuel does not support it, just that :) but again, Iâm not sure: Martin and Mariano can for sure say something more detailed about :) (problem is Martin is on vacations after getting his PhD and Mariano didnât followed in close the spur migration) cheers, Esteban
In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not.
Esteban
thanks,
Norbert
primitive #value: in Character class failed Stacktrace
Character class(Object)>>primitiveFailed: Character class(Object)>>primitiveFailed Character class>>value: Character class>>materializeFrom: FLHookPrimitiveCluster>>materializeInstanceWith: FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [ self clusterInstancesStep ] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [ :aDecoder | (FLMaterialization with: aDecoder) run; yourself ] in FLMaterializer>>setDefaultMaterialization FLMaterializer>>materializeFrom: FLMaterializer class>>materializeFromByteArray: MAPExampleModels class>>readModelFromSelector: MAPExampleModels class>>readModelNamed: [ self readModelNamed: aString ] in MAPExampleModels class>>named: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: MAPExampleModels class>>named: MAPTest>>model MAPTest>>tcapModule MAPTest>>testInsertSubscriberDataBitStringAccess [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: Given>>produceReturnValueAt: MAPTest(Phexample)>>performTest
I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images. 2015-12-16 10:34 GMT+01:00 Esteban Lorenzano <estebanlm@gmail.com>:
Hi,
On 16 Dec 2015, at 08:21, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Esteban,
On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 15 Dec 2015, at 13:18, Norbert Hartl <norbert@hartl.name> wrote:
We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?
I think is not possible, but I will let fuel maintainers to have the last word :)
Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate.
Oh well⦠of course is possible. What I meant is that I think current implementation of Fuel does not support it, just that :) but again, Iâm not sure: Martin and Mariano can for sure say something more detailed about :) (problem is Martin is on vacations after getting his PhD and Mariano didnât followed in close the spur migration)
cheers, Esteban
In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not.
Esteban
thanks,
Norbert
primitive #value: in Character class failed
Stacktrace
Character class(Object)>>primitiveFailed: Character class(Object)>>primitiveFailed Character class>>value: Character class>>materializeFrom: FLHookPrimitiveCluster>>materializeInstanceWith: FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [ self clusterInstancesStep ] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [ :aDecoder | (FLMaterialization with: aDecoder) run; yourself ] in FLMaterializer>>setDefaultMaterialization FLMaterializer>>materializeFrom: FLMaterializer class>>materializeFromByteArray: MAPExampleModels class>>readModelFromSelector: MAPExampleModels class>>readModelNamed: [ self readModelNamed: aString ] in MAPExampleModels class>>named: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: MAPExampleModels class>>named: MAPTest>>model MAPTest>>tcapModule MAPTest>>testInsertSubscriberDataBitStringAccess [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: Given>>produceReturnValueAt: MAPTest(Phexample)>>performTest
On 16 Dec 2015, at 13:24, Clément Bera <bera.clement@gmail.com> wrote:
I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
I don't know. Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
From my POV FUEL is only compatible with its own version, a sacrifice made to gain performance. IIUC it is not a stable external format for long lived data with backward/forward compatibility.
2015-12-16 10:34 GMT+01:00 Esteban Lorenzano <estebanlm@gmail.com>: Hi,
On 16 Dec 2015, at 08:21, Eliot Miranda <eliot.miranda@gmail.com> wrote:
Hi Esteban,
On Dec 15, 2015, at 5:06 AM, Esteban Lorenzano <estebanlm@gmail.com> wrote:
On 15 Dec 2015, at 13:18, Norbert Hartl <norbert@hartl.name> wrote:
We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?
I think is not possible, but I will let fuel maintainers to have the last word :)
Of course it's possible; in VW we exchanged objects between 32- & 64-but images using a format much like fuel and in 32-bits all floats are arrays of 32-bit bit objects whereas in 64-bits a range of floats are immediate.
Oh well⦠of course is possible. What I meant is that I think current implementation of Fuel does not support it, just that :) but again, Iâm not sure: Martin and Mariano can for sure say something more detailed about :) (problem is Martin is on vacations after getting his PhD and Mariano didnât followed in close the spur migration)
cheers, Esteban
In spur characters are immediate (Max, "primitive" is not the right term) whereas in v3 they're objects with a single isn't var. So at least fuel needs to be modified to read characters specially. One way to architect this is to ask the class (SmallInteger, Character, Float etc) to materialize, and then the class can decide how to represent its instances. The question is whether characters in spur should serialize themselves using the same wire format as v3 or whether fuel should modify its wire format to serialize immediate characters specially and have v3 materialize characters as serialized by spur. The issue is whether it's important to preserve object identity of characters with code >= 256 or not.
Esteban
thanks,
Norbert
primitive #value: in Character class failed Stacktrace
Character class(Object)>>primitiveFailed: Character class(Object)>>primitiveFailed Character class>>value: Character class>>materializeFrom: FLHookPrimitiveCluster>>materializeInstanceWith: FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [ self clusterInstancesStep ] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [ :aDecoder | (FLMaterialization with: aDecoder) run; yourself ] in FLMaterializer>>setDefaultMaterialization FLMaterializer>>materializeFrom: FLMaterializer class>>materializeFromByteArray: MAPExampleModels class>>readModelFromSelector: MAPExampleModels class>>readModelNamed: [ self readModelNamed: aString ] in MAPExampleModels class>>named: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: MAPExampleModels class>>named: MAPTest>>model MAPTest>>tcapModule MAPTest>>testInsertSubscriberDataBitStringAccess [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: Given>>produceReturnValueAt: MAPTest(Phexample)>>performTest
Hi. 2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:
On 16 Dec 2015, at 13:24, Clément Bera <bera.clement@gmail.com> wrote:
I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
I don't know.
Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.
On 16 Dec 2015, at 13:46, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:
On 16 Dec 2015, at 13:24, Clément Bera <bera.clement@gmail.com> wrote:
I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
I don't know.
Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.
What you want might not be the same to what it was designed for. I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another. Let's see what the Fuel guys have to say about this.
2015-12-16 13:49 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:
I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.
Will see what they answer. But supporting different versions of Fuel is not same as supporting single format for any platform
Hi guys, I would like to say a couple of things: 1) *Fuel format*: Sven is right in the sense that Fuel has problems of migrating to another Fuel version. However.... there is some explanation needed here. Previously (years ago), a given Fuel version was not compatible with previous versions because we were still changing/improving what we call Fuel format, that is, basically, how objects are written and read from the stream. We have stopped with those optimization/changes years ago. So the "Fuel format" has been stable so far for a couple of years. 2) *Fuel gets in the internals*: In Pharo 4.0 Fuel was not compatible with Fuel from 3.0. I don't remember the exact detail but it was a change in Pharo's Date (or DateAndTime I cannot remember). Now the version that should work in 5.0 changes the Characater serialization. What I want to make clear here is that it's NOT that we are changing the fuel format 1) as years ago when Fuel as still evolving from this point of view. It's Pharo changes that make our incompatibility. But that doesn't mean we cannot fix it. 3) *Yes, compromises for speed*: I think that both examples that break compatibility in 4.0 (Date / DateAndTime change) and in 5.0 with Spur (Character) is because we target speed. We do have special clusters for both of those cases (Date and Character), and so, we get into it's internals for performance reasons. If the internals change, then we break. This is why a text serializer for example may simply not break, because it continues using text representation that might not have been affected by the underlaying internal representation. We will try to do a Pharo 40 - Pharo 5.0 (or spur actually) compatibility. Best, On Wed, Dec 16, 2015 at 9:49 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 16 Dec 2015, at 13:46, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:
On 16 Dec 2015, at 13:24, Clément Bera <bera.clement@gmail.com> wrote:
I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
I don't know.
Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.
What you want might not be the same to what it was designed for.
I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.
Let's see what the Fuel guys have to say about this.
-- Mariano http://marianopeck.wordpress.com
OK, thanks for the explanation, good to hear it. (But, just nitpicking, it still does not say officially 'you can safely save your data in Fuel format and rest assured that you will always be able to read it back'). (Also, if there are compatibility problems between Pharo versions (understandably Pharo's doing), how can there be compatibility with other Smalltalk implementation, not that I expect any, but again, what is the official word ?)
On 16 Dec 2015, at 14:10, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
Hi guys,
I would like to say a couple of things:
1) Fuel format: Sven is right in the sense that Fuel has problems of migrating to another Fuel version. However.... there is some explanation needed here. Previously (years ago), a given Fuel version was not compatible with previous versions because we were still changing/improving what we call Fuel format, that is, basically, how objects are written and read from the stream. We have stopped with those optimization/changes years ago. So the "Fuel format" has been stable so far for a couple of years.
2) Fuel gets in the internals: In Pharo 4.0 Fuel was not compatible with Fuel from 3.0. I don't remember the exact detail but it was a change in Pharo's Date (or DateAndTime I cannot remember). Now the version that should work in 5.0 changes the Characater serialization. What I want to make clear here is that it's NOT that we are changing the fuel format 1) as years ago when Fuel as still evolving from this point of view. It's Pharo changes that make our incompatibility. But that doesn't mean we cannot fix it.
3) Yes, compromises for speed: I think that both examples that break compatibility in 4.0 (Date / DateAndTime change) and in 5.0 with Spur (Character) is because we target speed. We do have special clusters for both of those cases (Date and Character), and so, we get into it's internals for performance reasons. If the internals change, then we break. This is why a text serializer for example may simply not break, because it continues using text representation that might not have been affected by the underlaying internal representation.
We will try to do a Pharo 40 - Pharo 5.0 (or spur actually) compatibility.
Best,
On Wed, Dec 16, 2015 at 9:49 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 16 Dec 2015, at 13:46, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:
On 16 Dec 2015, at 13:24, Clément Bera <bera.clement@gmail.com> wrote:
I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
I don't know.
Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.
What you want might not be the same to what it was designed for.
I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.
Let's see what the Fuel guys have to say about this.
-- Mariano http://marianopeck.wordpress.com
On Wed, Dec 16, 2015 at 10:17 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
OK, thanks for the explanation, good to hear it.
(But, just nitpicking, it still does not say officially 'you can safely save your data in Fuel format and rest assured that you will always be able to read it back').
No, I can't say that (at least myself). It does mean it's not possible, it simply means that it will take some time to make it work and so if nobody does it, it won't work. However, note that the efforts seem to have been much smaller than years ago. For example, with very little changes, it's likely we could have manage automatically from Pharo 3.0 until now.
(Also, if there are compatibility problems between Pharo versions (understandably Pharo's doing), how can there be compatibility with other Smalltalk implementation, not that I expect any, but again, what is the official word ?)
There is none compatibility with other Smalltalk versions in the sense that a serialized stream in one st could be materialized in another st. However, note that what we DO support is that Fuel works in every possible version of Pharo and in a wide range of latest squeak images.
On 16 Dec 2015, at 14:10, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
Hi guys,
I would like to say a couple of things:
1) Fuel format: Sven is right in the sense that Fuel has problems of migrating to another Fuel version. However.... there is some explanation needed here. Previously (years ago), a given Fuel version was not compatible with previous versions because we were still changing/improving what we call Fuel format, that is, basically, how objects are written and read from the stream. We have stopped with those optimization/changes years ago. So the "Fuel format" has been stable so far for a couple of years.
2) Fuel gets in the internals: In Pharo 4.0 Fuel was not compatible with Fuel from 3.0. I don't remember the exact detail but it was a change in Pharo's Date (or DateAndTime I cannot remember). Now the version that should work in 5.0 changes the Characater serialization. What I want to make clear here is that it's NOT that we are changing the fuel format 1) as years ago when Fuel as still evolving from this point of view. It's Pharo changes that make our incompatibility. But that doesn't mean we cannot fix it.
3) Yes, compromises for speed: I think that both examples that break compatibility in 4.0 (Date / DateAndTime change) and in 5.0 with Spur (Character) is because we target speed. We do have special clusters for both of those cases (Date and Character), and so, we get into it's internals for performance reasons. If the internals change, then we break. This is why a text serializer for example may simply not break, because it continues using text representation that might not have been affected by the underlaying internal representation.
We will try to do a Pharo 40 - Pharo 5.0 (or spur actually) compatibility.
Best,
On Wed, Dec 16, 2015 at 9:49 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 16 Dec 2015, at 13:46, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:
On 16 Dec 2015, at 13:24, Clément Bera <bera.clement@gmail.com> wrote:
I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
I don't know.
Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.
What you want might not be the same to what it was designed for.
I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.
Let's see what the Fuel guys have to say about this.
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
It probably should be noted here as well that http://pharo.gemtalksystems.com/book/PharoTools/SIXX/ (Smalltalk Instance eXchange in XML) http://www.mars.dti.ne.jp/~umejava/smalltalk/sixx/index.html allows to transfer data from one image version to another image version / Smalltalk dialect. The format is useful for long-term storage. It is actively maintained. --Hannes On 12/16/15, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
On Wed, Dec 16, 2015 at 10:17 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
OK, thanks for the explanation, good to hear it.
(But, just nitpicking, it still does not say officially 'you can safely save your data in Fuel format and rest assured that you will always be able to read it back').
No, I can't say that (at least myself). It does mean it's not possible, it simply means that it will take some time to make it work and so if nobody does it, it won't work.
However, note that the efforts seem to have been much smaller than years ago. For example, with very little changes, it's likely we could have manage automatically from Pharo 3.0 until now.
(Also, if there are compatibility problems between Pharo versions (understandably Pharo's doing), how can there be compatibility with other Smalltalk implementation, not that I expect any, but again, what is the official word ?)
There is none compatibility with other Smalltalk versions in the sense that a serialized stream in one st could be materialized in another st.
However, note that what we DO support is that Fuel works in every possible version of Pharo and in a wide range of latest squeak images.
On 16 Dec 2015, at 14:10, Mariano Martinez Peck <marianopeck@gmail.com> wrote:
Hi guys,
I would like to say a couple of things:
1) Fuel format: Sven is right in the sense that Fuel has problems of migrating to another Fuel version. However.... there is some explanation needed here. Previously (years ago), a given Fuel version was not compatible with previous versions because we were still changing/improving what we call Fuel format, that is, basically, how objects are written and read from the stream. We have stopped with those optimization/changes years ago. So the "Fuel format" has been stable so far for a couple of years.
2) Fuel gets in the internals: In Pharo 4.0 Fuel was not compatible with Fuel from 3.0. I don't remember the exact detail but it was a change in Pharo's Date (or DateAndTime I cannot remember). Now the version that should work in 5.0 changes the Characater serialization. What I want to make clear here is that it's NOT that we are changing the fuel format 1) as years ago when Fuel as still evolving from this point of view. It's Pharo changes that make our incompatibility. But that doesn't mean we cannot fix it.
3) Yes, compromises for speed: I think that both examples that break compatibility in 4.0 (Date / DateAndTime change) and in 5.0 with Spur (Character) is because we target speed. We do have special clusters for both of those cases (Date and Character), and so, we get into it's internals for performance reasons. If the internals change, then we break. This is why a text serializer for example may simply not break, because it continues using text representation that might not have been affected by the underlaying internal representation.
We will try to do a Pharo 40 - Pharo 5.0 (or spur actually) compatibility.
Best,
On Wed, Dec 16, 2015 at 9:49 AM, Sven Van Caekenberghe <sven@stfx.eu> wrote:
On 16 Dec 2015, at 13:46, Denis Kudriashov <dionisiydk@gmail.com> wrote:
Hi.
2015-12-16 13:33 GMT+01:00 Sven Van Caekenberghe <sven@stfx.eu>:
On 16 Dec 2015, at 13:24, Clément Bera <bera.clement@gmail.com> wrote:
I don't know if SqueakV3 - Spur32 support is needed, however, I think it is really important to be able to serialize and materialize objects between 32 bits spur images and 64bits spur images.
I don't know.
Yes it would be nice, but since Fuel cannot even exchange data between different version itself (If I understand the situation correctly), why should that have to work ?
It is needed to use fuel as communication format between services which run on different platforms. For example you can exchange data between 64bits server and 32bits android phone.
What you want might not be the same to what it was designed for.
I might be wrong, but that is how I understood it from discussion in the past when moving data from one fuel version to another.
Let's see what the Fuel guys have to say about this.
-- Mariano http://marianopeck.wordpress.com
-- Mariano http://marianopeck.wordpress.com
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
No, it is a Smalltalk format. On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
If you want to move data to Java then you probably go for JSON or a particuar XML format. On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
2015-12-16 14:58 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
Of course. Or any other format you can find :) SIXX is XML. And maybe somebody implement it in different languages. I just interesting about it.
Please consider Avro. robert On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/ On 12/16/15, Robert Withers <robert.w.withers@gmail.com> wrote:
Please consider Avro.
robert
On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
I don't know; I'd be surprised. There are 2 parts: encoding and schema. Fuel would be great for it. It is binary. A Squeak/Pharo port would follow the usual get it to work, work right, work fast! :) It's perfect for BigData as it is the recommended Hadoop storage format. hint...hint.. cheers, robert On 12/16/2015 09:57 AM, H. Hirzel wrote:
Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/
On 12/16/15, Robert Withers <robert.w.withers@gmail.com> wrote:
Please consider Avro.
robert
On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
Seriously, I think of the I3 work from VW, so CORBA interface schemas are dynamically generated. The same could be done with AVRO schemas, then you could easily read and analyze BigData data, which is pretty happening in the market and still just getting started. Fuel would be a key part of that answer, I can't wait to do more with Fuel. regards, robert On 12/16/2015 10:06 AM, Robert Withers wrote:
I don't know; I'd be surprised.
There are 2 parts: encoding and schema. Fuel would be great for it. It is binary. A Squeak/Pharo port would follow the usual get it to work, work right, work fast! :)
It's perfect for BigData as it is the recommended Hadoop storage format. hint...hint..
cheers, robert
On 12/16/2015 09:57 AM, H. Hirzel wrote:
Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/
On 12/16/15, Robert Withers <robert.w.withers@gmail.com> wrote:
Please consider Avro.
robert
On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
Allow me to speak a little on my ideas of a distributed meta layer and control layer in cloud deployments with Squeak/Pharo, hopefully in bounds. A distributed system must have an metadata repsoitory describing the traffic and activity. If you look at deployment, code that works with a type and version of data must be present. With packaging and dynamic loading, chunks of code registered to a data type/version in the meta demand loads a achunk to do directed work in a particular deployed image. BigData event flows and data analysis like to get replicated and relocated, in realtime. Supporting this in the meta definitional layer and the 1/2 control layer starts to do dynamic, late-binding in the network. This is the advantage Squeak/Pharo image-based environments bring to the business table. Thank you and apologies, robert On 12/16/2015 09:57 AM, H. Hirzel wrote:
Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/
On 12/16/15, Robert Withers <robert.w.withers@gmail.com> wrote:
Please consider Avro.
robert
On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
I'd like to add that this is a real challenge to existing players in BigData, all of who offer solutions in this exact space. Whoever controls it's specification, controls it's cloud market. My back of the envelope proposal (my apologies for my abrasiveness with this approach) is such: We develop a dynamic cloud meta & control solution: replicated, eventually consistent and so on. We define the serialization, interfaces and package identification to this meta as open-source standards. With interfaces I think Avro. With serialization I think ASN1DER encoding, plus non-fixed dynamic structure extension. With package identification I think Gradle. Do we have a groovy compiler? I thought I would round out my thinking as I believe this to be an important opportunity. Given that the new internet is going to be governed by always running, maximally interactive execution sites, it will naturally move to Smalltalk, so I do not think there is anything for anyone to be concerned about. Patience and not reacting emotionally to misinformation in the marketplace will show true colors. Business is warfare. We have the advantage. peace, robert On 12/16/2015 12:38 PM, Robert Withers wrote:
Allow me to speak a little on my ideas of a distributed meta layer and control layer in cloud deployments with Squeak/Pharo, hopefully in bounds.
A distributed system must have an metadata repsoitory describing the traffic and activity. If you look at deployment, code that works with a type and version of data must be present. With packaging and dynamic loading, chunks of code registered to a data type/version in the meta demand loads a achunk to do directed work in a particular deployed image. BigData event flows and data analysis like to get replicated and relocated, in realtime. Supporting this in the meta definitional layer and the 1/2 control layer starts to do dynamic, late-binding in the network. This is the advantage Squeak/Pharo image-based environments bring to the business table.
Thank you and apologies, robert
On 12/16/2015 09:57 AM, H. Hirzel wrote:
Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/
On 12/16/15, Robert Withers <robert.w.withers@gmail.com> wrote:
Please consider Avro.
robert
On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
Excuse me: With interfaces I think groovy. With object serialization I think avro. With header encoding I think ASN1DER with dynamic extension, we need variable encoding of statistical objects. With packages I think gradle. this would be powerful, robert On 12/16/2015 01:00 PM, Robert Withers wrote:
I'd like to add that this is a real challenge to existing players in BigData, all of who offer solutions in this exact space. Whoever controls it's specification, controls it's cloud market.
My back of the envelope proposal (my apologies for my abrasiveness with this approach) is such:
We develop a dynamic cloud meta & control solution: replicated, eventually consistent and so on. We define the serialization, interfaces and package identification to this meta as open-source standards.
With interfaces I think Avro.
With serialization I think ASN1DER encoding, plus non-fixed dynamic structure extension.
With package identification I think Gradle. Do we have a groovy compiler?
I thought I would round out my thinking as I believe this to be an important opportunity. Given that the new internet is going to be governed by always running, maximally interactive execution sites, it will naturally move to Smalltalk, so I do not think there is anything for anyone to be concerned about.
Patience and not reacting emotionally to misinformation in the marketplace will show true colors. Business is warfare. We have the advantage.
peace, robert
On 12/16/2015 12:38 PM, Robert Withers wrote:
Allow me to speak a little on my ideas of a distributed meta layer and control layer in cloud deployments with Squeak/Pharo, hopefully in bounds.
A distributed system must have an metadata repsoitory describing the traffic and activity. If you look at deployment, code that works with a type and version of data must be present. With packaging and dynamic loading, chunks of code registered to a data type/version in the meta demand loads a achunk to do directed work in a particular deployed image. BigData event flows and data analysis like to get replicated and relocated, in realtime. Supporting this in the meta definitional layer and the 1/2 control layer starts to do dynamic, late-binding in the network. This is the advantage Squeak/Pharo image-based environments bring to the business table.
Thank you and apologies, robert
On 12/16/2015 09:57 AM, H. Hirzel wrote:
Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/
On 12/16/15, Robert Withers <robert.w.withers@gmail.com> wrote:
Please consider Avro.
robert
On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
Can you make a more detailed vision (with a picture) about this? As I have a cluster running and am paid to do some stuff with all you list, I can use some time exploring those avenues. Phil On Wed, Dec 16, 2015 at 7:07 PM, Robert Withers <robert.w.withers@gmail.com> wrote:
Excuse me:
With interfaces I think groovy. With object serialization I think avro. With header encoding I think ASN1DER with dynamic extension, we need variable encoding of statistical objects. With packages I think gradle.
this would be powerful, robert
On 12/16/2015 01:00 PM, Robert Withers wrote:
I'd like to add that this is a real challenge to existing players in BigData, all of who offer solutions in this exact space. Whoever controls it's specification, controls it's cloud market.
My back of the envelope proposal (my apologies for my abrasiveness with this approach) is such:
We develop a dynamic cloud meta & control solution: replicated, eventually consistent and so on. We define the serialization, interfaces and package identification to this meta as open-source standards.
With interfaces I think Avro.
With serialization I think ASN1DER encoding, plus non-fixed dynamic structure extension.
With package identification I think Gradle. Do we have a groovy compiler?
I thought I would round out my thinking as I believe this to be an important opportunity. Given that the new internet is going to be governed by always running, maximally interactive execution sites, it will naturally move to Smalltalk, so I do not think there is anything for anyone to be concerned about.
Patience and not reacting emotionally to misinformation in the marketplace will show true colors. Business is warfare. We have the advantage.
peace, robert
On 12/16/2015 12:38 PM, Robert Withers wrote:
Allow me to speak a little on my ideas of a distributed meta layer and control layer in cloud deployments with Squeak/Pharo, hopefully in bounds.
A distributed system must have an metadata repsoitory describing the traffic and activity. If you look at deployment, code that works with a type and version of data must be present. With packaging and dynamic loading, chunks of code registered to a data type/version in the meta demand loads a achunk to do directed work in a particular deployed image. BigData event flows and data analysis like to get replicated and relocated, in realtime. Supporting this in the meta definitional layer and the 1/2 control layer starts to do dynamic, late-binding in the network. This is the advantage Squeak/Pharo image-based environments bring to the business table.
Thank you and apologies, robert
On 12/16/2015 09:57 AM, H. Hirzel wrote:
Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/
On 12/16/15, Robert Withers <robert.w.withers@gmail.com> wrote:
Please consider Avro.
robert
On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com>:
It probably should be noted here as well that
http://pharo.gemtalksystems.com/book/PharoTools/SIXX/
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
I really would love to and I will probably do so to take my mind of the refactoring I am engaged in as I broke my SecureSession code. Note this basically includes a small outside header describing the contents of a FEC encoded msg. There will be an inside header coming later. I am not yet ASN1 encoding it. The picture I used to have, if you can imagine it is a 9 layer OSI stack, where layer 8 is the marketplace of running or invokable applications and the 9th, the meta, describes the work. One job where we did this we pushed event structure definitions and writer and reader interfaces for event flows. So all of this definition is stuffed into the meta, accessible anywhere in the distributed app. Furthermore, we injected decoders into the presentation layer and injected running readers and writers into the control layer (part of layer 8... 8 1/2) I hope this gives you a bit of a better picture what I am speaking about. When my world stabilizes I will try to get back to you soon, but I cannot promise you I will remember - my apologies for that. regards, robert On 12/16/2015 05:22 PM, phil@highoctane.be wrote:
Can you make a more detailed vision (with a picture) about this?
As I have a cluster running and am paid to do some stuff with all you list, I can use some time exploring those avenues.
Phil
On Wed, Dec 16, 2015 at 7:07 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Excuse me:
With interfaces I think groovy. With object serialization I think avro. With header encoding I think ASN1DER with dynamic extension, we need variable encoding of statistical objects. With packages I think gradle.
this would be powerful, robert
On 12/16/2015 01:00 PM, Robert Withers wrote:
I'd like to add that this is a real challenge to existing players in BigData, all of who offer solutions in this exact space. Whoever controls it's specification, controls it's cloud market.
My back of the envelope proposal (my apologies for my abrasiveness with this approach) is such:
We develop a dynamic cloud meta & control solution: replicated, eventually consistent and so on. We define the serialization, interfaces and package identification to this meta as open-source standards.
With interfaces I think Avro.
With serialization I think ASN1DER encoding, plus non-fixed dynamic structure extension.
With package identification I think Gradle. Do we have a groovy compiler?
I thought I would round out my thinking as I believe this to be an important opportunity. Given that the new internet is going to be governed by always running, maximally interactive execution sites, it will naturally move to Smalltalk, so I do not think there is anything for anyone to be concerned about.
Patience and not reacting emotionally to misinformation in the marketplace will show true colors. Business is warfare. We have the advantage.
peace, robert
On 12/16/2015 12:38 PM, Robert Withers wrote:
Allow me to speak a little on my ideas of a distributed meta layer and control layer in cloud deployments with Squeak/Pharo, hopefully in bounds.
A distributed system must have an metadata repsoitory describing the traffic and activity. If you look at deployment, code that works with a type and version of data must be present. With packaging and dynamic loading, chunks of code registered to a data type/version in the meta demand loads a achunk to do directed work in a particular deployed image. BigData event flows and data analysis like to get replicated and relocated, in realtime. Supporting this in the meta definitional layer and the 1/2 control layer starts to do dynamic, late-binding in the network. This is the advantage Squeak/Pharo image-based environments bring to the business table.
Thank you and apologies, robert
On 12/16/2015 09:57 AM, H. Hirzel wrote:
Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/
On 12/16/15, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Please consider Avro.
robert
On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com <mailto:hannes.hirzel@gmail.com>> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com <mailto:dionisiydk@gmail.com>> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com <mailto:hannes.hirzel@gmail.com>>:
It probably should be noted here as well that
http://pharo.gemtalksystems.com/book/PharoTools/SIXX/
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
Phil, I just posted a new diagram that starts to show the 8th and 9th layers. The specs for events, producers and consumers, bound to named type-structure definitions ("types"), are stored at the meta and injects apps (7) into cloud execution container (8) and into type decoders in layer 5. Hopefully this helps you see what I was saying. https://github.com/RabbitPhoneWarrenNetwork/Porcini/wiki/SecureSession robert On 12/16/2015 05:22 PM, phil@highoctane.be wrote:
Can you make a more detailed vision (with a picture) about this?
As I have a cluster running and am paid to do some stuff with all you list, I can use some time exploring those avenues.
Phil
On Wed, Dec 16, 2015 at 7:07 PM, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Excuse me:
With interfaces I think groovy. With object serialization I think avro. With header encoding I think ASN1DER with dynamic extension, we need variable encoding of statistical objects. With packages I think gradle.
this would be powerful, robert
On 12/16/2015 01:00 PM, Robert Withers wrote:
I'd like to add that this is a real challenge to existing players in BigData, all of who offer solutions in this exact space. Whoever controls it's specification, controls it's cloud market.
My back of the envelope proposal (my apologies for my abrasiveness with this approach) is such:
We develop a dynamic cloud meta & control solution: replicated, eventually consistent and so on. We define the serialization, interfaces and package identification to this meta as open-source standards.
With interfaces I think Avro.
With serialization I think ASN1DER encoding, plus non-fixed dynamic structure extension.
With package identification I think Gradle. Do we have a groovy compiler?
I thought I would round out my thinking as I believe this to be an important opportunity. Given that the new internet is going to be governed by always running, maximally interactive execution sites, it will naturally move to Smalltalk, so I do not think there is anything for anyone to be concerned about.
Patience and not reacting emotionally to misinformation in the marketplace will show true colors. Business is warfare. We have the advantage.
peace, robert
On 12/16/2015 12:38 PM, Robert Withers wrote:
Allow me to speak a little on my ideas of a distributed meta layer and control layer in cloud deployments with Squeak/Pharo, hopefully in bounds.
A distributed system must have an metadata repsoitory describing the traffic and activity. If you look at deployment, code that works with a type and version of data must be present. With packaging and dynamic loading, chunks of code registered to a data type/version in the meta demand loads a achunk to do directed work in a particular deployed image. BigData event flows and data analysis like to get replicated and relocated, in realtime. Supporting this in the meta definitional layer and the 1/2 control layer starts to do dynamic, late-binding in the network. This is the advantage Squeak/Pharo image-based environments bring to the business table.
Thank you and apologies, robert
On 12/16/2015 09:57 AM, H. Hirzel wrote:
Is there a Pharo implementation? https://avro.apache.org/docs/1.2.0/
On 12/16/15, Robert Withers <robert.w.withers@gmail.com <mailto:robert.w.withers@gmail.com>> wrote:
Please consider Avro.
robert
On 12/16/2015 08:58 AM, H. Hirzel wrote:
If you want to move data to Java then you probably go for JSON or a particuar XML format.
On 12/16/15, H. Hirzel <hannes.hirzel@gmail.com <mailto:hannes.hirzel@gmail.com>> wrote:
No, it is a Smalltalk format.
On 12/16/15, Denis Kudriashov <dionisiydk@gmail.com <mailto:dionisiydk@gmail.com>> wrote:
2015-12-16 14:41 GMT+01:00 H. Hirzel <hannes.hirzel@gmail.com <mailto:hannes.hirzel@gmail.com>>:
It probably should be noted here as well that
http://pharo.gemtalksystems.com/book/PharoTools/SIXX/
Just to know does java implementation exists?
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
-- . .. .. ^,^ best, robert
Characters are now primitive. There shouldnât be any need to change Character serialization though. I tried to reproduce your case and I canât. The only way I can get the primitive to fail is by supplying a negative value. I suspect that the problem happens beforehand, so that the byte being read is now at the wrong offset. Can you give me a runnable example? Cheers, Max
On 15 Dec 2015, at 13:18, Norbert Hartl <norbert@hartl.name> wrote:
We have a test build for pharo5 that checks if we need to do stuff for future migrations. Since yesterday it fails, of course. I have a fueled out object graph stored as byte array in a method. The fuel bytes are serialized using a non-spur image. Now when the spur image reads the bytes I get an exception. Has this something to do with immediate characters? Is there a way to fix that or will it just not be possible to exchange objects between spur and non-spur using fuel?
thanks,
Norbert
primitive #value: in Character class failed Stacktrace
Character class(Object)>>primitiveFailed: Character class(Object)>>primitiveFailed Character class>>value: Character class>>materializeFrom: FLHookPrimitiveCluster>>materializeInstanceWith: FLHookPrimitiveCluster(FLIteratingCluster)>>materializeInstancesStepWith: FLMaterialization>>clusterInstancesStep [ self clusterInstancesStep ] in FLMaterialization>>instancesStep SmallInteger(Integer)>>timesRepeat: FLMaterialization>>instancesStep FLMaterialization>>run [ :aDecoder | (FLMaterialization with: aDecoder) run; yourself ] in FLMaterializer>>setDefaultMaterialization FLMaterializer>>materializeFrom: FLMaterializer class>>materializeFromByteArray: MAPExampleModels class>>readModelFromSelector: MAPExampleModels class>>readModelNamed: [ self readModelNamed: aString ] in MAPExampleModels class>>named: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: MAPExampleModels class>>named: MAPTest>>model MAPTest>>tcapModule MAPTest>>testInsertSubscriberDataBitStringAccess [ testMethod perform: testMethod selector ] in Given>>produceReturnValueAt: [ self at: key put: aBlock value ] in Dictionary>>at:ifAbsentPut: Dictionary>>at:ifAbsent: Dictionary>>at:ifAbsentPut: Given>>produceReturnValueAt: MAPTest(Phexample)>>performTest
participants (11)
-
Clément Bera -
Denis Kudriashov -
Eliot Miranda -
Esteban Lorenzano -
H. Hirzel -
Mariano Martinez Peck -
Max Leske -
Norbert Hartl -
phil＠highoctane.be -
Robert Withers -
Sven Van Caekenberghe