- Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [Pharo-users] [ann] bloc & cairo+morphic

by Stephane Ducasse

Hi alain This is great to know that you can now work officially on Bloc and I really hope that your project with Thales will succeed. We need a strong Pharo. After more than 5 years of discussions with Thales I'm happy to see coming truth. People do not imagine the amount of energy we can spend sometimes :) And indeed you are right! I was sad to see that you are not even part of the github team after all the effort you spent on Bloc (and Miro and Miro2). I think that I was just the guy asking stupid questions but indeed this is true that it was in collaboration with Rmod, thank to mention it. I also agree with you that this is important that your work is respected and acknowledged and indeed I do not see how it relates to humane assessment. Stef On Thu, May 11, 2017 at 5:36 PM, Alain Plantec via Pharo-dev < pharo-dev(a)lists.pharo.org> wrote: > > > ---------- Forwarded message ---------- > From: Alain Plantec <alain.plantec(a)yahoo.com> > To: Pharo Development List <pharo-dev(a)lists.pharo.org> > Cc: Alain Plantec <alain.plantec(a)yahoo.com>, Moose-related development < > moose-dev(a)list.inf.unibe.ch>, Any question about pharo is welcome < > pharo-users(a)lists.pharo.org> > Bcc: > Date: Thu, 11 May 2017 17:36:36 +0200 > Subject: Re: [Pharo-users] [ann] bloc & cairo+morphic > Hello Doru, all, > > Iâm really happy to see Bloc progresses. > Even Iâm not active since more than one year, Bloc is still an important > project for me. > > but let me complete this short historical presentation a little bit. > > Bloc is a project that I initiated in 2013 in collaboration with RMOD > following experiments made around the ROME project. > The idea was to completely revisit the 2D framework of Pharo to address > Morphic limits. > Following an invitation of the Software Composition Group (thanks to Oscar > Nierstrasz and to Doru here), > I presented the first version of Bloc at Bern (March, 2015), then Doru and > Aliaksel joined the project. > One year ago, during his PhD at Brest, Glenn Cavarle produced a new > version of the Bloc infrastructure that is now the > one used together with the layouting system that was implemented by > Aliaksel. > > Please, do not use the humane assessment web site but the github project > one instead. > > I will restart working on Bloc/Brick soon in the context of a project that > we recently signed with the Thales company. > > Thanks, > Cheers > > Alain > > > > On 8 mai 2017, at 23:00, Tudor Girba <tudor(a)tudorgirba.com> wrote: > > > > Hi, > > > > We are happy to announce that based on the work of Glenn, Alex extended > Bloc (Sparta) to work directly in the Morphic world using Cairo as a > backend. > > > > Cairo is less powerful than Moz2D (see the screenshot below for an > example), but the implementation addresses a concern that the community > raised regarding a perceived increased liability due to the dependency to > Moz2D. Essentially this means that Bloc can be treated as another graphical > library that can coexist with Morphic without requiring any external VM > plugin. > > > > <bloc-two-backends-morphic-host-figures.png> > > > > I would also like to point out that adding a new backend and host was > possible because of the many iterations (including throwing away whole > implementations) that Alex and Glenn went through. I think they did an > amazing job. > > > > You can find a bit more details about Bloc here: > > http://www.humane-assessment.com/blog/bloc-flexible-backends-hosts/ > > > > Another issue raised regarding Bloc was that of the engineering effort > required to make it a reality. That is why I would also like to announce > that Alex joined feenk.com where he is primarily working on the graphical > stack for Pharo. > > > > Cheers, > > Doru > > > > > > -- > > www.tudorgirba.com > > www.feenk.com > > > > "To lead is not to demand things, it is to make them happen." > > > > > > > > > > > >

May 11, 2017

Re: [Pharo-dev] [Pharo-users] [ann] bloc & cairo+morphic

by Alexandre Bergel

Hi Alain! Thanks for the mail (even if the historial part has always been pretty clear to me). We miss you! Be back soon! Cheers, Alexandre -- _,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;: Alexandre Bergel http://www.bergel.eu ^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;._,.;:~^~:;. > On May 11, 2017, at 12:36 PM, Alain Plantec via Pharo-dev <pharo-dev(a)lists.pharo.org> wrote: > > > From: Alain Plantec <alain.plantec(a)yahoo.com> > Subject: Re: [Pharo-users] [ann] bloc & cairo+morphic > Date: May 11, 2017 at 12:36:36 PM GMT-3 > To: Pharo Development List <pharo-dev(a)lists.pharo.org> > Cc: Alain Plantec <alain.plantec(a)yahoo.com>, Moose-related development <moose-dev(a)list.inf.unibe.ch>, Any question about pharo is welcome <pharo-users(a)lists.pharo.org> > > > Hello Doru, all, > > Iâm really happy to see Bloc progresses. > Even Iâm not active since more than one year, Bloc is still an important project for me. > > but let me complete this short historical presentation a little bit. > > Bloc is a project that I initiated in 2013 in collaboration with RMOD following experiments made around the ROME project. > The idea was to completely revisit the 2D framework of Pharo to address Morphic limits. > Following an invitation of the Software Composition Group (thanks to Oscar Nierstrasz and to Doru here), > I presented the first version of Bloc at Bern (March, 2015), then Doru and Aliaksel joined the project. > One year ago, during his PhD at Brest, Glenn Cavarle produced a new version of the Bloc infrastructure that is now the > one used together with the layouting system that was implemented by Aliaksel. > > Please, do not use the humane assessment web site but the github project one instead. > > I will restart working on Bloc/Brick soon in the context of a project that we recently signed with the Thales company. > > Thanks, > Cheers > > Alain > > >> On 8 mai 2017, at 23:00, Tudor Girba <tudor(a)tudorgirba.com> wrote: >> >> Hi, >> >> We are happy to announce that based on the work of Glenn, Alex extended Bloc (Sparta) to work directly in the Morphic world using Cairo as a backend. >> >> Cairo is less powerful than Moz2D (see the screenshot below for an example), but the implementation addresses a concern that the community raised regarding a perceived increased liability due to the dependency to Moz2D. Essentially this means that Bloc can be treated as another graphical library that can coexist with Morphic without requiring any external VM plugin. >> >> <bloc-two-backends-morphic-host-figures.png> >> >> I would also like to point out that adding a new backend and host was possible because of the many iterations (including throwing away whole implementations) that Alex and Glenn went through. I think they did an amazing job. >> >> You can find a bit more details about Bloc here: >> http://www.humane-assessment.com/blog/bloc-flexible-backends-hosts/ >> >> Another issue raised regarding Bloc was that of the engineering effort required to make it a reality. That is why I would also like to announce that Alex joined feenk.com where he is primarily working on the graphical stack for Pharo. >> >> Cheers, >> Doru >> >> >> -- >> www.tudorgirba.com >> www.feenk.com >> >> "To lead is not to demand things, it is to make them happen." >> >> >> >> > > > >

May 11, 2017

Re: [Pharo-users] [ann] bloc & cairo+morphic

by Alain Plantec

Hello Doru, all, Iâm really happy to see Bloc progresses. Even Iâm not active since more than one year, Bloc is still an important project for me. but let me complete this short historical presentation a little bit. Bloc is a project that I initiated in 2013 in collaboration with RMOD following experiments made around the ROME project. The idea was to completely revisit the 2D framework of Pharo to address Morphic limits. Following an invitation of the Software Composition Group (thanks to Oscar Nierstrasz and to Doru here), I presented the first version of Bloc at Bern (March, 2015), then Doru and Aliaksel joined the project. One year ago, during his PhD at Brest, Glenn Cavarle produced a new version of the Bloc infrastructure that is now the one used together with the layouting system that was implemented by Aliaksel. Please, do not use the humane assessment web site but the github project one instead. I will restart working on Bloc/Brick soon in the context of a project that we recently signed with the Thales company. Thanks, Cheers Alain > On 8 mai 2017, at 23:00, Tudor Girba <tudor(a)tudorgirba.com> wrote: > > Hi, > > We are happy to announce that based on the work of Glenn, Alex extended Bloc (Sparta) to work directly in the Morphic world using Cairo as a backend. > > Cairo is less powerful than Moz2D (see the screenshot below for an example), but the implementation addresses a concern that the community raised regarding a perceived increased liability due to the dependency to Moz2D. Essentially this means that Bloc can be treated as another graphical library that can coexist with Morphic without requiring any external VM plugin. > > <bloc-two-backends-morphic-host-figures.png> > > I would also like to point out that adding a new backend and host was possible because of the many iterations (including throwing away whole implementations) that Alex and Glenn went through. I think they did an amazing job. > > You can find a bit more details about Bloc here: > http://www.humane-assessment.com/blog/bloc-flexible-backends-hosts/ > > Another issue raised regarding Bloc was that of the engineering effort required to make it a reality. That is why I would also like to announce that Alex joined feenk.com where he is primarily working on the graphical stack for Pharo. > > Cheers, > Doru > > > -- > www.tudorgirba.com > www.feenk.com > > "To lead is not to demand things, it is to make them happen." > > > >

May 11, 2017

Pharo-Jenkins github user

by Guillermo Polito

Hi everybody, I wanted to create a github user to use for ci integration (pharo-ci), and while configuring it I saw there is already a Pharo-Jenkins user. https://github.com/Pharo-Jenkins Does somebody know who is the user? Guille -- Guille Polito Research Engineer French National Center for Scientific Research - *http://www.cnrs.fr* <http://www.cnrs.fr> *Web:* *http://guillep.github.io* <http://guillep.github.io> *Phone: *+33 06 52 70 66 13

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Max Leske

> On 11 May 2017, at 14:37, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Can't you just set it to a suitable high limit then ? Like 10e6 or so ? > > The limit really exists for a reason, else I would not have added it in the first place. > > I can't really change how dynamic variables work, and I don't feel that this justifies hacking around that. Sure, I can do that. I just thought that since there is an #unlimited configuration option it should be possible to use it. But I'm happy enough with the way it is now. Thanks! > >> On 11 May 2017, at 14:24, Max Leske <maxleske(a)gmail.com> wrote: >> >> That's perfect, with one exception: it is still not possible to set the number of maximum dictionary entries to unlimited. In ZnMultiValueDictionary you have a nil check for the unlimited case but the dynamic variable will always default to the default value because DynamicVariable uses the default when the variable has a nil value. Not sure what the best solution is there... >> >> Max >> >> >>> On 11 May 2017, at 14:06, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> ZnZincServerAdaptor and subclasses expose a #server accessor that gives you access to the configured Zn server. You should be able to grab that and set additional options (#maximumEntitySize: #maximumNumberOfDictionaryEntries: #defaultEncoder:) in your setup code. >>> >>>> On 11 May 2017, at 14:00, Max Leske <maxleske(a)gmail.com> wrote: >>>> >>>> Hi Sven, >>>> >>>> I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? >>>> >>>> Cheers, >>>> Max >>>> >>>> >>>>> On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com> wrote: >>>>> >>>>>> >>>>>> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>>> >>>>>> Hi Max, >>>>>> >>>>>> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >>>>>> >>>>>> Still, I decided to take another road, implementation wise. >>>>>> >>>>>> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >>>>>> >>>>>> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >>>>>> >>>>>> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. >>>>> >>>>> I will. Thanks Sven! >>>>> >>>>>> >>>>>> Regards, >>>>>> >>>>>> Sven >>>>>> >>>>>> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >>>>>> >>>>>>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: >>>>>>> >>>>>>>> >>>>>>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>>>>> >>>>>>>> Max, >>>>>>>> >>>>>>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>>>>>> >>>>>>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>>>>>> >>>>>>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>>>>>> >>>>>>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>>>>>> >>>>>>>>> Thoughts? >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Max >>>>>>>> >>>>>>>> You are the first one to complain about this limit. >>>>>>>> >>>>>>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>>>>>> >>>>>>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>>>>>> >>>>>>>> >>>>>>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>>>>>> >>>>>>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>>>>>> >>>>>>> I've attached the patches. >>>>>>> >>>>>>> A couple of notes: >>>>>>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>>>>>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>>>>>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>>>>>> >>>>>>> >>>>>>> Cheers, >>>>>>> Max >>>>>>> >>>>>>>> >>>>>>>> Sven >>>> >>> >>> >> >> > >

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Sven Van Caekenberghe

Can't you just set it to a suitable high limit then ? Like 10e6 or so ? The limit really exists for a reason, else I would not have added it in the first place. I can't really change how dynamic variables work, and I don't feel that this justifies hacking around that. > On 11 May 2017, at 14:24, Max Leske <maxleske(a)gmail.com> wrote: > > That's perfect, with one exception: it is still not possible to set the number of maximum dictionary entries to unlimited. In ZnMultiValueDictionary you have a nil check for the unlimited case but the dynamic variable will always default to the default value because DynamicVariable uses the default when the variable has a nil value. Not sure what the best solution is there... > > Max > > >> On 11 May 2017, at 14:06, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> ZnZincServerAdaptor and subclasses expose a #server accessor that gives you access to the configured Zn server. You should be able to grab that and set additional options (#maximumEntitySize: #maximumNumberOfDictionaryEntries: #defaultEncoder:) in your setup code. >> >>> On 11 May 2017, at 14:00, Max Leske <maxleske(a)gmail.com> wrote: >>> >>> Hi Sven, >>> >>> I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? >>> >>> Cheers, >>> Max >>> >>> >>>> On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com> wrote: >>>> >>>>> >>>>> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>> >>>>> Hi Max, >>>>> >>>>> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >>>>> >>>>> Still, I decided to take another road, implementation wise. >>>>> >>>>> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >>>>> >>>>> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >>>>> >>>>> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. >>>> >>>> I will. Thanks Sven! >>>> >>>>> >>>>> Regards, >>>>> >>>>> Sven >>>>> >>>>> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >>>>> >>>>>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: >>>>>> >>>>>>> >>>>>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>>>> >>>>>>> Max, >>>>>>> >>>>>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>>>>> >>>>>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>>>>> >>>>>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>>>>> >>>>>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>>>>> >>>>>>>> Thoughts? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Max >>>>>>> >>>>>>> You are the first one to complain about this limit. >>>>>>> >>>>>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>>>>> >>>>>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>>>>> >>>>>>> >>>>>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>>>>> >>>>>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>>>>> >>>>>> I've attached the patches. >>>>>> >>>>>> A couple of notes: >>>>>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>>>>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>>>>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>>>>> >>>>>> >>>>>> Cheers, >>>>>> Max >>>>>> >>>>>>> >>>>>>> Sven >>> >> >> > >

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Max Leske

That's perfect, with one exception: it is still not possible to set the number of maximum dictionary entries to unlimited. In ZnMultiValueDictionary you have a nil check for the unlimited case but the dynamic variable will always default to the default value because DynamicVariable uses the default when the variable has a nil value. Not sure what the best solution is there... Max > On 11 May 2017, at 14:06, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > ZnZincServerAdaptor and subclasses expose a #server accessor that gives you access to the configured Zn server. You should be able to grab that and set additional options (#maximumEntitySize: #maximumNumberOfDictionaryEntries: #defaultEncoder:) in your setup code. > >> On 11 May 2017, at 14:00, Max Leske <maxleske(a)gmail.com> wrote: >> >> Hi Sven, >> >> I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? >> >> Cheers, >> Max >> >> >>> On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com> wrote: >>> >>>> >>>> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Hi Max, >>>> >>>> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >>>> >>>> Still, I decided to take another road, implementation wise. >>>> >>>> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >>>> >>>> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >>>> >>>> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. >>> >>> I will. Thanks Sven! >>> >>>> >>>> Regards, >>>> >>>> Sven >>>> >>>> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >>>> >>>>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: >>>>> >>>>>> >>>>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>>> >>>>>> Max, >>>>>> >>>>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>>>> >>>>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>>>> >>>>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>>>> >>>>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>>>> >>>>>>> Thoughts? >>>>>>> >>>>>>> Cheers, >>>>>>> Max >>>>>> >>>>>> You are the first one to complain about this limit. >>>>>> >>>>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>>>> >>>>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>>>> >>>>>> >>>>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>>>> >>>>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>>>> >>>>> I've attached the patches. >>>>> >>>>> A couple of notes: >>>>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>>>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>>>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>>>> >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>>> >>>>>> Sven >> > >

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Sven Van Caekenberghe

ZnZincServerAdaptor and subclasses expose a #server accessor that gives you access to the configured Zn server. You should be able to grab that and set additional options (#maximumEntitySize: #maximumNumberOfDictionaryEntries: #defaultEncoder:) in your setup code. > On 11 May 2017, at 14:00, Max Leske <maxleske(a)gmail.com> wrote: > > Hi Sven, > > I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? > > Cheers, > Max > > >> On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com> wrote: >> >>> >>> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Hi Max, >>> >>> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >>> >>> Still, I decided to take another road, implementation wise. >>> >>> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >>> >>> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >>> >>> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. >> >> I will. Thanks Sven! >> >>> >>> Regards, >>> >>> Sven >>> >>> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >>> >>>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: >>>> >>>>> >>>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>> >>>>> Max, >>>>> >>>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>>> >>>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>>> >>>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>>> >>>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>>> >>>>>> Thoughts? >>>>>> >>>>>> Cheers, >>>>>> Max >>>>> >>>>> You are the first one to complain about this limit. >>>>> >>>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>>> >>>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>>> >>>>> >>>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>>> >>>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>>> >>>> I've attached the patches. >>>> >>>> A couple of notes: >>>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>>> >>>> >>>> Cheers, >>>> Max >>>> >>>>> >>>>> Sven >

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Max Leske

Ignore that. I hadn't loaded all your code. > On 11 May 2017, at 14:00, Max Leske <maxleske(a)gmail.com> wrote: > > Hi Sven, > > I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? > > Cheers, > Max > > >> On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >> >>> >>> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu <mailto:sven@stfx.eu>> wrote: >>> >>> Hi Max, >>> >>> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >>> >>> Still, I decided to take another road, implementation wise. >>> >>> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >>> >>> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >>> >>> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. >> >> I will. Thanks Sven! >> >>> >>> Regards, >>> >>> Sven >>> >>> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >>> >>>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >>>> >>>>> >>>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu <mailto:sven@stfx.eu>> wrote: >>>>> >>>>> Max, >>>>> >>>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>>> >>>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>>> >>>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>>> >>>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>>> >>>>>> Thoughts? >>>>>> >>>>>> Cheers, >>>>>> Max >>>>> >>>>> You are the first one to complain about this limit. >>>>> >>>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>>> >>>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>>> >>>>> >>>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>>> >>>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>>> >>>> I've attached the patches. >>>> >>>> A couple of notes: >>>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>>> >>>> >>>> Cheers, >>>> Max >>>> >>>>> >>>>> Sven >

May 11, 2017

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/60485 Home: https://github.com/pharo-project/pharo-core

May 11, 2017