To clarify, external type declarations are different that the type specification of a structure, such as an X509Certificate and its TBSCertificate. These are type applied, in similar fashion to NeoJSON, will forecite and predeterminization on both encoding and decoding sides. These type definitions are applied ad-hoc. In the case of an external type declaration, as understood so far, these declare new types recognizable by the internal type identification. Why is important to system? When presentation layer is decoding a message, arbitrary objects may be encoded inthe graph of arguments or receiver. As these are discovered in tracing these graphs, these arbitrary objects will have ASN.1 schema definitionModels from the declarations and should de/manifest appropriately. Hopefully. - HH
-------- Original Message -------- Subject: Re: [Pharo-dev] [squeak-dev] Serializers/Encoders Local Time: August 15, 2017 2:28 AM UTC Time: August 15, 2017 6:28 AM From: henry@callistohouse.club To: Squeak-dev <squeak-dev@lists.squeakfoundation.org>, Pharo Development List <pharo-dev@lists.pharo.org>
After more research, including TJSON, we realized the choice of ASN.1 at the lower layer is exactly what could continue at the higher layer, with more work to expand ASN.1 capability.
Sorry for the huge 1 MB image. The classes in Fuel are, beginning with serialization or demanifestation:
Serializer * Serialization * Analyzer * Encoder ** Analysis ***[] Clusterizations { handling source-substitutions }
onthe wire these clusterizations are encoded. On the receiving side, the process of materialization or manifestation is straightforwards, activate the clusterizations, with a decoder forming a proxy instance graph, then annealing them as references. I think these are the points of substitution, but I am honestly guessing from a faint memory.
Materializer * Materialization * Decoder (***[]) on the wire clusterizations...
I really want/need/desire to dive into where and how substitutions are occurring, on source-side and destination. This in order to add this ability to the ASN.1 system.
In ASN.1 specification, there is a mechanism to specify external structures and expand the definitional nature of ASN.1. ASN.1 carries a notation and we have the start of a DefinitionModel. Fitting this into Fuel would be sweet and it may be straightforward: break the graph walking of ASN.1 just have the target encoding, in nesting: so pre and post, and change the Fuel Encoder/Decoder. I believe that since clusterizations will not pass across, a decode analyzer will be needed.
Getting III (Interactive Invocation Interface) support to generate ASN.1 definition models and classes to model them, bi-directionally, will enable the analyzers to setup clusterizations that en/decode properly. Here's to hope that this will be so.
Among the varied requirements, the core requirement is passing extended substitutable types/schemas between environments: Squeak - Pharo - Java. I think ASN.1 with its types is the best choice. I have no idea if Bouncy Castle can support external types. I am happy it can do CHOICE, though with manual coding to implement this. Separating the Types in Squeak/Pharo Cryptograghy turns out to be a good choice and interchangeable we hope.
I hope this clarifies and illuminates, I am driven to share these thoughts.
Warmly, - HH
-------- Original Message -------- Subject: Re: [squeak-dev] [Pharo-dev] Serializers/Encoders Local Time: August 14, 2017 6:27 PM UTC Time: August 14, 2017 10:27 PM From: henry@callistohouse.club To: Pharo Development List <pharo-dev@lists.pharo.org>, Squeak-dev <squeak-dev@lists.squeakfoundation.org>
In further consideration, what seems missing from JSON encoding is an ANY type and a CHOICE type. With these and proper type encoding extension to JSON, we could pass arbitrary types w/o being required to specify a specific decoding structure. I think this is so. Types would be helpful. I found TJSON, which may be good, but no arbitrary types, still.
I notice JSON decoding is where magic happens and it requires full specification of what is expected. Perhaps there is a way for ANY & CHOICE with a CustomMapper. Contrarily, Fuel does all analysis on encoding, the front side, and encodes the found clusters on-stream so the materialized is lightweight. ASN1MODULES does a little of both with external types but also with ANY & CHOICE.
In looking at Fuel-Core, here are main classes..,
- HH
On Mon, Aug 14, 2017 at 13:44, henry <henry@callistohouse.club> wrote:
I find I do not necessarily have a specific question regarding a specific implementations, but a higher-level question of feasible solution. I need substitutable, type-relevant JSON encoding, perhaps based upon JSON Schemas.
I am looking at ASN1Modules & types, Fuel clusterings/substitutions and NeoJSON clusters & mappers. Is it feasible to bring all these together? I would use ASN1Types, Fuel graph walking/substitutions and NeoJSON encoding and cluster hooks.
Fuel also has Clusters but I see that Fuel encodes these clusters in each encoding. I am torn about this philosophically: good to have self-describing data encoding, but not suitable for JSON, unless just a type-name extension. Lightweight & robust descriptions? Well, this is all Fuel cluster specs within the encoding really is: a type-specification.
The aspect of NeoJSON that has me wondering is the lack of polymorphism in type-spec lookup and recursive substitution. This is where Fuel shines, I feel.
So what is the vision of the end state encoder? Use ASN1 Types as cluster descriptors, Fuel proxification on materializations and NeoJSON encoding and mappers.
Would you have for me guidances in this area?
- HH