May 2017 - Pharo-dev - lists.pharo.org

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Sven Van Caekenberghe

Can't you just set it to a suitable high limit then ? Like 10e6 or so ? The limit really exists for a reason, else I would not have added it in the first place. I can't really change how dynamic variables work, and I don't feel that this justifies hacking around that. > On 11 May 2017, at 14:24, Max Leske <maxleske(a)gmail.com> wrote: > > That's perfect, with one exception: it is still not possible to set the number of maximum dictionary entries to unlimited. In ZnMultiValueDictionary you have a nil check for the unlimited case but the dynamic variable will always default to the default value because DynamicVariable uses the default when the variable has a nil value. Not sure what the best solution is there... > > Max > > >> On 11 May 2017, at 14:06, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> ZnZincServerAdaptor and subclasses expose a #server accessor that gives you access to the configured Zn server. You should be able to grab that and set additional options (#maximumEntitySize: #maximumNumberOfDictionaryEntries: #defaultEncoder:) in your setup code. >> >>> On 11 May 2017, at 14:00, Max Leske <maxleske(a)gmail.com> wrote: >>> >>> Hi Sven, >>> >>> I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? >>> >>> Cheers, >>> Max >>> >>> >>>> On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com> wrote: >>>> >>>>> >>>>> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>> >>>>> Hi Max, >>>>> >>>>> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >>>>> >>>>> Still, I decided to take another road, implementation wise. >>>>> >>>>> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >>>>> >>>>> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >>>>> >>>>> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. >>>> >>>> I will. Thanks Sven! >>>> >>>>> >>>>> Regards, >>>>> >>>>> Sven >>>>> >>>>> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >>>>> >>>>>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: >>>>>> >>>>>>> >>>>>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>>>> >>>>>>> Max, >>>>>>> >>>>>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>>>>> >>>>>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>>>>> >>>>>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>>>>> >>>>>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>>>>> >>>>>>>> Thoughts? >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Max >>>>>>> >>>>>>> You are the first one to complain about this limit. >>>>>>> >>>>>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>>>>> >>>>>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>>>>> >>>>>>> >>>>>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>>>>> >>>>>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>>>>> >>>>>> I've attached the patches. >>>>>> >>>>>> A couple of notes: >>>>>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>>>>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>>>>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>>>>> >>>>>> >>>>>> Cheers, >>>>>> Max >>>>>> >>>>>>> >>>>>>> Sven >>> >> >> > >

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Max Leske

That's perfect, with one exception: it is still not possible to set the number of maximum dictionary entries to unlimited. In ZnMultiValueDictionary you have a nil check for the unlimited case but the dynamic variable will always default to the default value because DynamicVariable uses the default when the variable has a nil value. Not sure what the best solution is there... Max > On 11 May 2017, at 14:06, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > ZnZincServerAdaptor and subclasses expose a #server accessor that gives you access to the configured Zn server. You should be able to grab that and set additional options (#maximumEntitySize: #maximumNumberOfDictionaryEntries: #defaultEncoder:) in your setup code. > >> On 11 May 2017, at 14:00, Max Leske <maxleske(a)gmail.com> wrote: >> >> Hi Sven, >> >> I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? >> >> Cheers, >> Max >> >> >>> On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com> wrote: >>> >>>> >>>> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Hi Max, >>>> >>>> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >>>> >>>> Still, I decided to take another road, implementation wise. >>>> >>>> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >>>> >>>> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >>>> >>>> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. >>> >>> I will. Thanks Sven! >>> >>>> >>>> Regards, >>>> >>>> Sven >>>> >>>> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >>>> >>>>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: >>>>> >>>>>> >>>>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>>> >>>>>> Max, >>>>>> >>>>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>>>> >>>>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>>>> >>>>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>>>> >>>>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>>>> >>>>>>> Thoughts? >>>>>>> >>>>>>> Cheers, >>>>>>> Max >>>>>> >>>>>> You are the first one to complain about this limit. >>>>>> >>>>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>>>> >>>>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>>>> >>>>>> >>>>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>>>> >>>>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>>>> >>>>> I've attached the patches. >>>>> >>>>> A couple of notes: >>>>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>>>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>>>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>>>> >>>>> >>>>> Cheers, >>>>> Max >>>>> >>>>>> >>>>>> Sven >> > >

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Sven Van Caekenberghe

ZnZincServerAdaptor and subclasses expose a #server accessor that gives you access to the configured Zn server. You should be able to grab that and set additional options (#maximumEntitySize: #maximumNumberOfDictionaryEntries: #defaultEncoder:) in your setup code. > On 11 May 2017, at 14:00, Max Leske <maxleske(a)gmail.com> wrote: > > Hi Sven, > > I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? > > Cheers, > Max > > >> On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com> wrote: >> >>> >>> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Hi Max, >>> >>> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >>> >>> Still, I decided to take another road, implementation wise. >>> >>> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >>> >>> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >>> >>> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. >> >> I will. Thanks Sven! >> >>> >>> Regards, >>> >>> Sven >>> >>> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >>> >>>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: >>>> >>>>> >>>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>> >>>>> Max, >>>>> >>>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>>> >>>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>>> >>>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>>> >>>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>>> >>>>>> Thoughts? >>>>>> >>>>>> Cheers, >>>>>> Max >>>>> >>>>> You are the first one to complain about this limit. >>>>> >>>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>>> >>>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>>> >>>>> >>>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>>> >>>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>>> >>>> I've attached the patches. >>>> >>>> A couple of notes: >>>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>>> >>>> >>>> Cheers, >>>> Max >>>> >>>>> >>>>> Sven >

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Max Leske

Ignore that. I hadn't loaded all your code. > On 11 May 2017, at 14:00, Max Leske <maxleske(a)gmail.com> wrote: > > Hi Sven, > > I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? > > Cheers, > Max > > >> On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >> >>> >>> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu <mailto:sven@stfx.eu>> wrote: >>> >>> Hi Max, >>> >>> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >>> >>> Still, I decided to take another road, implementation wise. >>> >>> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >>> >>> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >>> >>> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. >> >> I will. Thanks Sven! >> >>> >>> Regards, >>> >>> Sven >>> >>> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >>> >>>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >>>> >>>>> >>>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu <mailto:sven@stfx.eu>> wrote: >>>>> >>>>> Max, >>>>> >>>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com <mailto:maxleske@gmail.com>> wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>>> >>>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>>> >>>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>>> >>>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>>> >>>>>> Thoughts? >>>>>> >>>>>> Cheers, >>>>>> Max >>>>> >>>>> You are the first one to complain about this limit. >>>>> >>>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>>> >>>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>>> >>>>> >>>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>>> >>>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>>> >>>> I've attached the patches. >>>> >>>> A couple of notes: >>>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>>> >>>> >>>> Cheers, >>>> Max >>>> >>>>> >>>>> Sven >

May 11, 2017

[pharo-project/pharo-core]

by GitHub

Branch: refs/tags/60485 Home: https://github.com/pharo-project/pharo-core

May 11, 2017

[pharo-project/pharo-core] 618ef8: 60485

by GitHub

Branch: refs/heads/6.0 Home: https://github.com/pharo-project/pharo-core Commit: 618ef8b3bdc5576186f37dbbfb5d389406491be3 https://github.com/pharo-project/pharo-core/commit/618ef8b3bdc5576186f37dbb… Author: Jenkins Build Server <board(a)pharo-project.org> Date: 2017-05-11 (Thu, 11 May 2017) Changed paths: M ConfigurationOfEpicea.package/ConfigurationOfEpicea.class/instance/tags/stable_.st A ConfigurationOfEpicea.package/ConfigurationOfEpicea.class/instance/versions/version822_.st M Epicea.package/EpMonitor.class/instance/announcement handling/behaviorRemoved_.st M Epicea.package/EpMonitor.class/instance/private/subscribeToSystemAnnouncer.st M EpiceaBrowsers.package/EpApplyVisitor.class/instance/visitor/visitClassAddition_.st M EpiceaBrowsers.package/EpApplyVisitor.class/instance/visitor/visitClassModification_.st A EpiceaBrowsers.package/EpLostChangesDetector.class/class/instance creation/new.st A EpiceaBrowsers.package/EpLostChangesDetector.class/class/instance creation/newWithLog_.st M EpiceaBrowsers.package/EpLostChangesDetector.class/class/system startup/startUp_.st M EpiceaBrowsers.package/EpLostChangesDetector.class/definition.st R EpiceaBrowsers.package/EpLostChangesDetector.class/instance/accessing/browserIfLostChanges_.st M EpiceaBrowsers.package/EpLostChangesDetector.class/instance/accessing/lostChanges.st R EpiceaBrowsers.package/EpLostChangesDetector.class/instance/accessing/monitorLog.st R EpiceaBrowsers.package/EpLostChangesDetector.class/instance/accessing/monitorLog_.st M EpiceaBrowsers.package/EpLostChangesDetector.class/instance/accessing/openBrowserIfLostChanges.st A EpiceaBrowsers.package/EpLostChangesDetector.class/instance/initialization/initializeWithLog_.st A EpiceaBrowsers.package/EpLostChangesDetector.class/instance/testing/hasLostChanges.st A EpiceaBrowsersTests.package/EpApplyTest.class/instance/tests/testClassAdditionWithMetaclassInstanceVariablesChanged.st M EpiceaBrowsersTests.package/EpLostChangesDetectorTest.class/instance/tests/testDetectInEmptyLog.st M EpiceaBrowsersTests.package/EpLostChangesDetectorTest.class/instance/tests/testDetectNoChange.st M EpiceaBrowsersTests.package/EpLostChangesDetectorTest.class/instance/tests/testDetectOneChange.st M EpiceaBrowsersTests.package/EpRevertTest.class/instance/tests/testClassRemoval.st M Komitter.package/RemotesManager.class/instance/private/buildRoots.st R Ombu.package/OmBlockFileStore.class/class/as yet unclassified/example.st R Ombu.package/OmBlockFileStore.class/class/as yet unclassified/exampleOriginal.st A Ombu.package/OmCurrentGlobalName.class/README.md A Ombu.package/OmCurrentGlobalName.class/definition.st M Ombu.package/OmDeferrer.class/class/system startup/shutDown_.st A Ombu.package/OmEntryReader.class/class/instance creation/newForStore_.st M Ombu.package/OmEntryReader.class/definition.st A Ombu.package/OmEntryReader.class/instance/accessing/store.st A Ombu.package/OmEntryReader.class/instance/initialization/initializeWithStore_.st R Ombu.package/OmEntryReader.class/instance/reading/entryPositionsOn_do_.st R Ombu.package/OmEntryReader.class/instance/reading/nextEntryFrom_.st A Ombu.package/OmEntryWriter.class/class/instance creation/newForStore_.st M Ombu.package/OmEntryWriter.class/definition.st A Ombu.package/OmEntryWriter.class/instance/accessing/store.st A Ombu.package/OmEntryWriter.class/instance/initialization/initializeWithStore_.st M Ombu.package/OmFileStore.class/instance/private/newEntryReader.st M Ombu.package/OmFileStore.class/instance/private/newEntryWriter.st M Ombu.package/OmFileStore.class/instance/testing/isOutdated.st M Ombu.package/OmReference.class/class/ston persistence/fromSton_.st M Ombu.package/OmReference.class/instance/ston persistence/stonOn_.st M Ombu.package/OmSTONEntryReader.class/instance/reading/nextEntry.st M Ombu.package/OmSTONEntryWriter.class/instance/writing/on_nextEntryPut_.st M Ombu.package/OmSessionStore.class/instance/initialization/initializeWithBaseLocator_.st R Ombu.package/OmSessionStore.class/instance/initialization/subscribeToSnapshotDone.st A Ombu.package/OmStoreFactory.class/class/system startup/shutDown_.st R ScriptLoader60.package/ScriptLoader.class/instance/pharo - scripts/script60484.st A ScriptLoader60.package/ScriptLoader.class/instance/pharo - scripts/script60485.st R ScriptLoader60.package/ScriptLoader.class/instance/pharo - updates/update60484.st A ScriptLoader60.package/ScriptLoader.class/instance/pharo - updates/update60485.st M ScriptLoader60.package/ScriptLoader.class/instance/public/commentForCurrentUpdate.st Log Message: ----------- 60485 20047 in Versioner (or Komitter), RemotesManager needs to list just remotes https://pharo.fogbugz.com/f/cases/20047 20005 Epicea: Integrate release 8.2.2 https://pharo.fogbugz.com/f/cases/20005 http://files.pharo.org/image/60/60485.zip

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Max Leske

Hi Sven, I'm unsure about where to set the dynamic variables. I don't really want to do that in my WAApplication as it's not application logic. I could subclass ZnZincStreamingServerAdaptor... Do you have any suggestions? Cheers, Max > On 11 May 2017, at 13:05, Max Leske <maxleske(a)gmail.com> wrote: > >> >> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> Hi Max, >> >> First thank you for the feedback, the discussion so far and your patches. I studied them carefully. >> >> Still, I decided to take another road, implementation wise. >> >> You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. >> >> I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. >> >> Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. > > I will. Thanks Sven! > >> >> Regards, >> >> Sven >> >> PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). >> >>> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: >>> >>>> >>>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Max, >>>> >>>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>>>> >>>>> Hi, >>>>> >>>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>>> >>>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>>> >>>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>>> >>>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>>> >>>>> Thoughts? >>>>> >>>>> Cheers, >>>>> Max >>>> >>>> You are the first one to complain about this limit. >>>> >>>> This one and other resource limits exist to protect the client/server against abuse and attacks. >>> >>> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >>> >>>> >>>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >>> >>> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >>> >>> I've attached the patches. >>> >>> A couple of notes: >>> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >>> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >>> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >>> >>> >>> Cheers, >>> Max >>> >>>> >>>> Sven

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Max Leske

> On 11 May 2017, at 11:49, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Hi Max, > > First thank you for the feedback, the discussion so far and your patches. I studied them carefully. > > Still, I decided to take another road, implementation wise. > > You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. > > I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. > > Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. I will. Thanks Sven! > > Regards, > > Sven > > PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). > >> On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: >> >>> >>> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Max, >>> >>>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>>> >>>> Hi, >>>> >>>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>>> >>>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>>> >>>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>>> >>>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>>> >>>> Thoughts? >>>> >>>> Cheers, >>>> Max >>> >>> You are the first one to complain about this limit. >>> >>> This one and other resource limits exist to protect the client/server against abuse and attacks. >> >> I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. >> >>> >>> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? >> >> Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). >> >> I've attached the patches. >> >> A couple of notes: >> - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. >> - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation >> - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) >> >> >> Cheers, >> Max >> >>> >>> Sven > >

May 11, 2017

Re: [Pharo-dev] Missing plugins to make the vmProfiler work on Pharo

by Nicolas Cellier

2017-05-11 11:34 GMT+02:00 ClÃ©ment Bera <bera.clement(a)gmail.com>: > > > On Wed, May 10, 2017 at 8:12 PM, Esteban A. Maringolo < > emaringolo(a)gmail.com> wrote: > >> 2017-05-10 13:38 GMT-03:00 ClÃ©ment Bera <bera.clement(a)gmail.com>: >> > On Wed, May 10, 2017 at 5:28 PM, Esteban A. Maringolo < >> emaringolo(a)gmail.com> wrote: >> >> >> If it is a plug-in, why does it need VM recompilation? >> > >> > Internal plugins require VM recompilation. Only external plugins do not. >> >> It was more of a semantic question, I'd expect a plug-in to be that, >> pluggable, as you say external plugins are. >> Or what is the "pluggability" of such plugins? >> > > I would say there is 3 levels of pluggability: > 1 non optional internal plugin, pluggable because it can be replaced by > another internal plugin implementing the same API at VM compilation time. > Used for modularity of specific aspects of the VM. > 2 optional internal plugin, pluggable because it can be removed / added / > replaced at VM compilation time. Used to improve performance over external > plugin or to access VM functions not exposed to external plugins. > 3 external plugin, pluggable because it can be removed / added / replaced > after the VM is compiled. Used for all other plugins. > > So they're all pluggable in some way. > > External plugins have to use a well defined API for all interactions with VM. Internal plugins not only could have access to unexposed functions, but can also inline some of these functions as recently introduced by Eliot ( https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/7fa0184ef8c7c8badb…) so the performance might be a criterion of choice. >From a packaging POV, making a plugin external is a source of trouble versus embedded internal plugins. The number of possible combinations can rapidly explode (think of testing). It is essentially for letting user choose: - to not embed the plugin * because he knows he ain't gonna need it, and he wants to limit the memory footprint * or because the plugins expose some security issues - to replace a plugin with an alternative version - to produce his own plugins without bothering to recompile the whole VM (or just update the VM without the need to update the plugin because the major revision of API did not change) >> Disclaimer: I don't know about the VM internals (*), and this is not a >> complain of any sort. I just wonder it is called that way. >> >> Regards! >> >> Esteban A. Maringolo >> >> (*) I did compile a Squeak VM a decade ago (added two primitives) but >> nothing more than that. >> >> >

May 11, 2017

Re: [Pharo-dev] [Zinc] How do I set the entry limit for ZnMultiValueDictionary?

by Sven Van Caekenberghe

Hi Max, First thank you for the feedback, the discussion so far and your patches. I studied them carefully. Still, I decided to take another road, implementation wise. You see, the reason a global setting does not make sense is that there can (and are) multiple Zn clients and servers active in the same image, and each should be independent, not be influenced by configuration changes in others. This is why the use of dynamic variables fits so well. I refactored the current situation a bit and added the necessary high level hooks. I moved the default to each dynamic variable class itself, removed it from ZnConstants, and added options to both ZnClient and ZnServer. Please test (code committed in bleedingEdge) and let me know if it works for you (especially then #defaultEncoder part). Maybe we have to iterate more over this to fully support your use case. Regards, Sven PS: One resource limit not yet moved under the new scheme is the maximum line length (currently set to 4096). > On 6 May 2017, at 15:30, Max Leske <maxleske(a)gmail.com> wrote: > >> >> On 5 May 2017, at 17:11, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> Max, >> >>> On 5 May 2017, at 16:59, Max Leske <maxleske(a)gmail.com> wrote: >>> >>> Hi, >>> >>> I'm performing a legal request that has more than 4000 parameters. This causes the Zinc server to return 400: Bad Request because ZnMultiValueDictionary is limited to 256 entries by default. >>> >>> The dictionary has the option to remove the limit or to adjust it with a dynamic variable. Unfortunately, I don't see any way to properly configure this without monkey patching Zinc. Ideally, I'd like to remove the limit (which can't be done through the dynamic variable by the way because when the dynamic variable answers nil, the default will be set to 256). >>> >>> The first thing that comes to mind is to move this setting to ZnConstants, but then I don't see any way to configure ZnConstants either (ZnConstants is referenced directly by its users). Maybe ZnConstants could be changed to hold a concrete constants class (itself by default). >>> >>> In any case, I think this setting should be configurable and the configuration should be possible through one single entry point, together with options like #codec. >>> >>> Thoughts? >>> >>> Cheers, >>> Max >> >> You are the first one to complain about this limit. >> >> This one and other resource limits exist to protect the client/server against abuse and attacks. > > I'm aware of that but have not other way to do this right now. We have mod_security with a higher value configured for that. > >> >> I think what is needed is something like ZnServer>>#withMaximumEntitySizeDo: which uses the server option #maximumEntitySize. Would that work for you, you think ? > > Yes, it would. I've implemented the change, closely following the semantics of #withMaximumEntitySizeDo:. While I was working on that I discovered another problem I had which I also fixed by dispatching to ZnConstants. The problem being, that I set the codec on the server to GRNullCodec, but ZnPercentEncoder would still always use a ZnUTF8Encoder to decode requests. The result was an error when the server tried to write wide strings onto the stream (usually ZnUTF8Encoder: UTF-8 -> WideString, GRPharoUtf8Codec: WideString -> UTF-8). > > I've attached the patches. > > A couple of notes: > - #defaultMaximumNumberOfDictionaryEntries and #maximumDictionaryEntries: have the same implementation, which is unnecessary in my opinion but I've copied the code from #maximumEntitySize:. > - I'm not sure whether storing the option a second time on ZnServer makes sense, but, again, I stuck to the existing implementation > - #withMaximumNumberOfDictionaryEntriesDo: also checks the dynamic variable, which is overkill I think, as ZnConstants does the same thing. Again, I stuck to the existing implementation (which means that ZnConstants is actually thread agnostic and references to dynamic variables should probably all be on ZnConstants) > > > Cheers, > Max > >> >> Sven

May 11, 2017