November 2021 - Pharo-dev - lists.pharo.org

Re: Hardening Zinc's Core HTTP Server

by Sven Van Caekenberghe

Hi, > On 29 Oct 2021, at 20:42, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Here is yet another update: > > The instances > > - On Amazon AWS: http://34.245.183.130:1701 > > - On Microsoft Azure: http://51.137.72.94:8080 > > have now been running for 18+ days. > > Having observed the overall amounts of critical resources inside the Pharo images, things look good. > > Seaside WASSession cleanup goes slowly but does work over time. > > Eventually I will stop at least the Azure instance because it costs me personal money. I disabled the Microsoft Azure VM instance at http://51.137.72.94:8080 to save money. Final uptime was 51 days. Here is the final screenshot: I'll leave the Amazon AWS VM instance at http://34.245.183.130:1701 up for now. Sven >> On 11 Oct 2021, at 19:37, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> Here is an update on the status of both instances. >> >> A couple of people tried fiddling with invalid input which did result in exceptions, but everything was handled correctly so that the server kept on functioning. >> >> There were no spurious crashes. >> >> There was one attack by Levente Uzonyi that was successful though. >> >> He used the fact that Seaside sessions are long lived (cached for a certain time) combined with the fact that the Reddit.st app kept one GLORP database connection through P3 open to PostgreSQL and fired off a small denial of service (DOS) attack. Eventually either the VM ran out of space in the ExternalSemaphoreTable, making Socket creation, either for database connections or for handling new HTTP request impossible or the database's own connection limit was reached. At that point the HTTP server or the Pharo VM stopped functioning. >> >> Although Levente used a sequential number of requests, a concurrent number of request could cause similar problems (related, but differently). >> >> In looking at what he reported it also became clear that I accidentally left a debugging exception handler active, which is not good in a production image. >> >> Both instances are now redeployed with the following changes: >> >> - the Reddit.st code was changed to not keep the database connection open all the time, but instead connect/disconnect per request. this might be a bit slower, but it conserves resources much better and solves the original issue Levente reported >> >> https://github.com/svenvc/Reddit/commit/f2e0a0dc00b9cbb68cfa4fb007906365ae6… >> >> - a new feature was added to Zinc HTTP Components' ZnManagingMultiThreadedServer (the default) to enforce a maximum number of concurrent connections being allowed (the limit is 32 by default, but changeable if you know what you are doing). when the limit is reached, 503 Service Unavailable responses are sent to the excess clients and the connection is closed. this should help protect against concurrent connection DOS attacks >> >> https://github.com/svenvc/zinc/commit/ac0f06e74e7ab129610c466cb1d7ea9533d29… >> >> - the deploy script was changed to use the more primitive WAErrorHandler >> >> https://github.com/svenvc/Reddit/commit/874b631e6dc0c04c8c0b687ef770d00540d… >> >> Thanks again to Levente for taking the time to try an attack and for reporting it clearly. >> >> Sven >> >>> On 29 Sep 2021, at 17:10, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Both instances have been up for 5 days now, looking for more testers. >>> >>>> On 23 Sep 2021, at 17:03, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Hi, >>>> >>>> Zinc HTTP Components [https://github.com/svenvc/zinc] has been a part of Pharo since version 1.3 (2011). It is an open-source framework to deal with the HTTP networking protocol, modelling all aspects involved. It also offers both client and server functionality. >>>> >>>> The reliability of the code base has improved steadily over the years, thanks to virtually all Pharo developers using it, directly or indirectly. Over the summer a number of issues that popped up after Pharo 9 was released were resolved. >>>> >>>> The robustness of the core HTTP server is one important aspect. To put this quality further to the test, I deployed two servers with the same demo Seaside application, Reddit.st, open to the internet, without any further protections. >>>> >>>> - On Amazon AWS: http://34.245.183.130:1701 >>>> >>>> - On Microsoft Azure: http://51.137.72.94:8080 >>>> >>>> The application's source code can be found at [https://github.com/svenvc/Reddit] For the technically curious there are also deploy instructions at [https://github.com/svenvc/Reddit/blob/main/DEPLOY.md] The demo app itself is described in an older article [https://medium.com/@svenvc/reddit-st-in-10-cool-pharo-classes-1b5327ca0740] Note that, by definition, there is no HTTPS/TLS variant. >>>> >>>> If you manage to break this server with (a) malicious request(s) in such a way that you can explain what you did for others to confirm your approach, you not only help me/us improve the code, but earn eternal fame as well ;-) >>>> >>>> Sven >>>> >>>> PS: I hope I won't regret this, I am looking for constructive criticism. >>>> >>>> >>>> -- >>>> Sven Van Caekenberghe >>>> Proudly supporting Pharo >>>> http://pharo.org >>>> http://association.pharo.org >>>> http://consortium.pharo.org >>>> >>> >> >

Nov. 30, 2021

Re: Hardening Zinc's Core HTTP Server

by Sven Van Caekenberghe

Thx. > On 31 Oct 2021, at 10:09, Tim Mackinnon <tim(a)testit.works> wrote: > > Hey Sven - really appreciate your thorough approach to this, and itâs been interesting seeing the results (I did try a few script injection attempts but didnât get up to running burp suite on it, which I may try it while you have the instances). > > Kudos to Levente for taking a more reasoned approach and using inside knowledgeâ¦ this community is awesome! > > Tim > >> On 29 Oct 2021, at 19:42, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> ï»¿Here is yet another update: >> >> The instances >> >> - On Amazon AWS: http://34.245.183.130:1701 >> >> - On Microsoft Azure: http://51.137.72.94:8080 >> >> have now been running for 18+ days. >> >> Having observed the overall amounts of critical resources inside the Pharo images, things look good. >> >> Seaside WASSession cleanup goes slowly but does work over time. >> >> Eventually I will stop at least the Azure instance because it costs me personal money. >> >>> On 11 Oct 2021, at 19:37, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Here is an update on the status of both instances. >>> >>> A couple of people tried fiddling with invalid input which did result in exceptions, but everything was handled correctly so that the server kept on functioning. >>> >>> There were no spurious crashes. >>> >>> There was one attack by Levente Uzonyi that was successful though. >>> >>> He used the fact that Seaside sessions are long lived (cached for a certain time) combined with the fact that the Reddit.st app kept one GLORP database connection through P3 open to PostgreSQL and fired off a small denial of service (DOS) attack. Eventually either the VM ran out of space in the ExternalSemaphoreTable, making Socket creation, either for database connections or for handling new HTTP request impossible or the database's own connection limit was reached. At that point the HTTP server or the Pharo VM stopped functioning. >>> >>> Although Levente used a sequential number of requests, a concurrent number of request could cause similar problems (related, but differently). >>> >>> In looking at what he reported it also became clear that I accidentally left a debugging exception handler active, which is not good in a production image. >>> >>> Both instances are now redeployed with the following changes: >>> >>> - the Reddit.st code was changed to not keep the database connection open all the time, but instead connect/disconnect per request. this might be a bit slower, but it conserves resources much better and solves the original issue Levente reported >>> >>> https://github.com/svenvc/Reddit/commit/f2e0a0dc00b9cbb68cfa4fb007906365ae6… >>> >>> - a new feature was added to Zinc HTTP Components' ZnManagingMultiThreadedServer (the default) to enforce a maximum number of concurrent connections being allowed (the limit is 32 by default, but changeable if you know what you are doing). when the limit is reached, 503 Service Unavailable responses are sent to the excess clients and the connection is closed. this should help protect against concurrent connection DOS attacks >>> >>> https://github.com/svenvc/zinc/commit/ac0f06e74e7ab129610c466cb1d7ea9533d29… >>> >>> - the deploy script was changed to use the more primitive WAErrorHandler >>> >>> https://github.com/svenvc/Reddit/commit/874b631e6dc0c04c8c0b687ef770d00540d… >>> >>> Thanks again to Levente for taking the time to try an attack and for reporting it clearly. >>> >>> Sven >>> >>>>> On 29 Sep 2021, at 17:10, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Both instances have been up for 5 days now, looking for more testers. >>>> >>>>> On 23 Sep 2021, at 17:03, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>>> >>>>> Hi, >>>>> >>>>> Zinc HTTP Components [https://github.com/svenvc/zinc] has been a part of Pharo since version 1.3 (2011). It is an open-source framework to deal with the HTTP networking protocol, modelling all aspects involved. It also offers both client and server functionality. >>>>> >>>>> The reliability of the code base has improved steadily over the years, thanks to virtually all Pharo developers using it, directly or indirectly. Over the summer a number of issues that popped up after Pharo 9 was released were resolved. >>>>> >>>>> The robustness of the core HTTP server is one important aspect. To put this quality further to the test, I deployed two servers with the same demo Seaside application, Reddit.st, open to the internet, without any further protections. >>>>> >>>>> - On Amazon AWS: http://34.245.183.130:1701 >>>>> >>>>> - On Microsoft Azure: http://51.137.72.94:8080 >>>>> >>>>> The application's source code can be found at [https://github.com/svenvc/Reddit] For the technically curious there are also deploy instructions at [https://github.com/svenvc/Reddit/blob/main/DEPLOY.md] The demo app itself is described in an older article [https://medium.com/@svenvc/reddit-st-in-10-cool-pharo-classes-1b5327ca0740] Note that, by definition, there is no HTTPS/TLS variant. >>>>> >>>>> If you manage to break this server with (a) malicious request(s) in such a way that you can explain what you did for others to confirm your approach, you not only help me/us improve the code, but earn eternal fame as well ;-) >>>>> >>>>> Sven >>>>> >>>>> PS: I hope I won't regret this, I am looking for constructive criticism. >>>>> >>>>> >>>>> -- >>>>> Sven Van Caekenberghe >>>>> Proudly supporting Pharo >>>>> http://pharo.org >>>>> http://association.pharo.org >>>>> http://consortium.pharo.org >>>>> >>>> >>>

Nov. 30, 2021

This week (47/2021) on the Pharo Issue Tracker

by Marcus Denker

This week (47/2021) on the Pharo Issue Tracker We merged 40 PRs and closed 44 Issues. Speed ===== - 10384-Performance-Regression-in-SmallInteger-/ #10385 https://github.com/pharo-project/pharo/pull/10385 Fixes ===== - Added :ref to methodName for refactoring options #10389 https://github.com/pharo-project/pharo/pull/10389 - Fix completion of keyword selectors #10438 https://github.com/pharo-project/pharo/pull/10438 - Reflectivity-Fix-Reify-Node-DynamicArray #10435 https://github.com/pharo-project/pharo/pull/10435 Tests ===== - rescue testForTiltedStickyness #10351 https://github.com/pharo-project/pharo/pull/10351 - Fixes testTerminationShouldProceedAllEnsureBlocksIfSomeWasFailed withâ¦ #10464 https://github.com/pharo-project/pharo/pull/10464 - Adopt testNoUtilsMethods #10167 https://github.com/pharo-project/pharo/pull/10167 Cleanups ======== - Behavior>>#allCallsOnIn: sends unimplemented methods #10431 https://github.com/pharo-project/pharo/pull/10431 - Small cleanups CircleMorph and EllipseMorph #10393 https://github.com/pharo-project/pharo/pull/10393 - Cleanup UndefinedClassTest #10373 https://github.com/pharo-project/pharo/pull/10373 - 24 PRs: Cleanup newlines and blanks Pharo 9 ======= - [P9] 10208-Cannot-inline-method-if-the-code-to-inline-is-not-an-immediate-node #10379 https://github.com/pharo-project/pharo/pull/10379 - [P9] 10264-Text-Wrapping-is-not-working #10382 https://github.com/pharo-project/pharo/pull/10382 - P9-10384-Performance-Regression-in-SmallInteger-/ #10386 https://github.com/pharo-project/pharo/pull/10386

Nov. 26, 2021

This week (46/2021) on the Pharo Issue Tracker

by Marcus Denker

More infos at https://github.com/pharo-project/pharo/pulse Fixes ===== - 10334-Metaclass-should-be-able-to-answer-hasClassVarNamed #10335 https://github.com/pharo-project/pharo/pull/10335 - Completion index #10326 https://github.com/pharo-project/pharo/pull/10326 - 10319-OSSDL2BackendWindowfetchDPI-seems-broken #10325 https://github.com/pharo-project/pharo/pull/10325 - 10059-cannot-save-method-when-it-contains-a-breakpoint #10349 https://github.com/pharo-project/pharo/pull/10349 - 10196 class side fluid syntax does not return a builder #10321 https://github.com/pharo-project/pharo/pull/10321 - 9922-RenderBugsTesttestForward-checks-for-timing-leads-to-timeouts-on-slow-CI #10218 https://github.com/pharo-project/pharo/pull/10218 - 10375-Selecting-Packages-in-Calypso-produces-MNU #10376 https://github.com/pharo-project/pharo/pull/10376 - Fix STONZnUrl source code problem #10357 https://github.com/pharo-project/pharo/pull/10357 - 10208-Cannot-inline-method-if-the-code-to-inline-is-not-an-immediate-node-P10 #10380 https://github.com/pharo-project/pharo/pull/10380 - [P10] 10264-Text-Wrapping-is-not-working #10381 https://github.com/pharo-project/pharo/pull/10381 - Forward porting the removal for: OSSDL2BackendWindow>>#fetchDPI seems broken #10348 https://github.com/pharo-project/pharo/pull/10348 - Improving Code Completion #10346 https://github.com/pharo-project/pharo/pull/10346 - Remove class of the RBSelectorEnvirionment if there is no more selectors associated to the class #10209 https://github.com/pharo-project/pharo/pull/10209 Release Tests / Rules ===================== - Rule to check class side #reset methods are in 'class initialization' protocol https://github.com/pharo-project/pharo/pull/10182 - Release test for proper packages #10248 https://github.com/pharo-project/pharo/pull/10248 Cleanup ======= - Reduce usage of NativeBoost calls for UnifiedFFI-Legacy #10283 https://github.com/pharo-project/pharo/pull/10283 - 10343 cleanup opal compiler core extensions should use correct casing https://github.com/pharo-project/pharo/pull/10344 - Cleanup correct casing #10340 https://github.com/pharo-project/pharo/pull/10340 - 10337 cleanup metacello core extensions should use correct casing #10338 https://github.com/pharo-project/pharo/pull/10338 - Correct casing for STON-Core extensions #10342 https://github.com/pharo-project/pharo/pull/10342 - Cleanup rpackage core extensions should use correct casing #10359 https://github.com/pharo-project/pharo/pull/10359 - Fix Gofer-Core extensions should use correct casing / package name #10365 https://github.com/pharo-project/pharo/pull/10365 - Fix Metacello extensions should use correct casing / package name #10369 https://github.com/pharo-project/pharo/pull/10369 - Fix Manifest-Core extensions should use correct casing / package name https://github.com/pharo-project/pharo/pull/10367 - Fix Monticello extensions should use correct casing / package name #10363 https://github.com/pharo-project/pharo/pull/10363 Pharo9 ====== - 10350-Backport-Pharo9-10059-cannot-save-method-when-it-contains-a-breakpoint #10352 https://github.com/pharo-project/pharo/pull/10352 - 10353-Backport-Pharo9-CI-ZnHTTPSTesttestGmailEncrypted-started-failing-on-the-CI- #10355 https://github.com/pharo-project/pharo/pull/10355

Nov. 22, 2021

Re: [ANN] Fuel release 4.0.0

by sean＠clipperadams.com

Thanks. I appreciate your continued support. I use Fuel in almost every project via SimplePersistence. Iâm particularly eager to play with the talent support

Nov. 21, 2021

This week (45/2021) on the Pharo Issue Tracker

by Marcus Denker

This week we merged 7 PRs and closed 3 issue tracker entries Refactoring Tools ================== - Added a MakeAbstract command to execute a transformation to make a class abstract â¦ #10327 https://github.com/pharo-project/pharo/pull/10327 - Migrate RBMoveMethodToClassSide to refactoring2 #10291 https://github.com/pharo-project/pharo/pull/10291 Cleanup ===== - Microoptimization-MethodDictionary-scanFor #10309 https://github.com/pharo-project/pharo/pull/10309 - Small cleanups SystemDictionary #10317 https://github.com/pharo-project/pharo/pull/10317 - Reduce usage of NativeBoost calls for UnifiedFFI-Legacy #10283 https://github.com/pharo-project/pharo/pull/10283 Speed ===== - Speedup-SendersOf #10311 https://github.com/pharo-project/pharo/pull/10311

Nov. 12, 2021

One PhD and two internships offers on object-centric debugging at Inria Lille

by Steven Costiou

Hello, Inria Lille is recruiting one phd student and two interns to work on object-centric debugging within the OCRE project (ANR). DESCRIPTION OF THE PROJECT Debugging is difficult and costly. Object-centric debugging is a young technique arguing that focusing the scope of debugging on specific objects considerably eases the tracking and the understanding of hard bugs in Object-Oriented Programs (OOP). But it lacks fundamental bricks to be applicable in practice. Therefore, it has never been empirically evaluated. The objectives of the OCRE project are to study the fundamental and practical limits that hinder the implementation, the evaluation, and the adoption of object-centric debugging. We propose to build the first generation of object-centric debuggers, in order to identify and evaluate its real benefits to OOP debugging. We argue that these debuggers have the potential to drastically lower the cost (time and effort) of tracking and understanding hard bugs in OOP. PHD TOPIC The goal of this PhD is to understand and to achieve the full potential of object-centric debugging by: - identifying the precise scenarios for object-centric debugging, - studying means to obtain objects to debug, - studying the implementation requirements for object-centric debuggers and to implement prototypes, - evaluate empirically object-centric debugging using these prototypes. The full detailed offer is available here: https://jobs.inria.fr/public/classic/fr/offres/2021-04101 INTERNSHIPS These two internships aim at helping the design and the implementation of an object-centric debugger, by unveiling knowledge without which object-centric debuggers cannot be built and evaluated, or cannot be used in practice. Internship 1 - Identifying and acquiring objects to debug The objective of this internship is to study the practical problems of acquiring objects to debug. The intern will characterize the scenarios in which developers need to identify objects to debug and explore these scenarios by implementing prototypes. Internship 2 - Evaluating object-centric debugging scenarios The objective of this internship is to identify and to evaluate the debugging scenarios that are the most likely to benefit from object-centric debugging. The intern will interrogate real developers about object-centric debugging scenarios, then implement prototypes and evaluate them empirically with developers to validate these scenarios. ADMINISTRATIVE DETAILS Location: Lille, France Phd: - starting date: preferably october 2022, but it can start before. - duration: 3 years (fully funded) - the candidate must hold a Master or an equivalent degree Internships: - starting date: anytime - duration: 4-6 months - the candidate must be studying for a Master degree or equivalent (1st or 2nd year, equivalent to M1 or M2 in France)

Nov. 9, 2021

This week (44/2021) on the Pharo Issue Tracker

by Marcus Denker

We merged 16 PRs and closed 11 Issue tracker entries Speed + Features =============== - Added IdentitySet>>identityIncludes: for improved performance #10293 https://github.com/pharo-project/pharo/pull/10293 - Added offset-based copy method to ExternalAddress to parallel *offsetâ¦ #10285 https://github.com/pharo-project/pharo/pull/10285 Bugs ==== - 10280-Unimplemented-method-in-MemoryLogger-classwriteLoop #10310 https://github.com/pharo-project/pharo/pull/10310 Refactoring Engine =========== - Migrating RBRenamePackage into Refactoring2 #10164 https://github.com/pharo-project/pharo/pull/10164 - Small changes to make flow clearer. #10287 https://github.com/pharo-project/pharo/pull/10287 Cleanups ======== - 10150-obsoleteClasses--obsoleteBehaviors #10217 https://github.com/pharo-project/pharo/pull/10217 - Fix typos initialise => initialize #10305 https://github.com/pharo-project/pharo/pull/10305 - Typo in DiskFileSystemTest>>#testEntriesHaveAttributes #10303 https://github.com/pharo-project/pharo/pull/10303 - Consistently categorize #startUp: and shutDown: methods #10301 https://github.com/pharo-project/pharo/pull/10301 - Categorize TViewModel #10296 https://github.com/pharo-project/pharo/pull/10296 - More Small cleanups - class side initialize #10299 https://github.com/pharo-project/pharo/pull/10299 - Cleanup EndianDetector #10315 https://github.com/pharo-project/pharo/pull/10315 Tests ===== - Breakpoint-timeSpamp-unchanged-test #10324 https://github.com/pharo-project/pharo/pull/10324 - 10290-CI-ZnHTTPSTesttestGmailEncrypted-started-failing-on-the-CI #10308 https://github.com/pharo-project/pharo/pull/10308 - 10312-CI-Failing-test-NoUnusedVariablesLeftTest-due-to-RBPackageTransformation #10313 https://github.com/pharo-project/pharo/pull/10313 Pharo 9 backports ================== - P9 - Added offset-based copy method to ExternalAddress #10320 https://github.com/pharo-project/pharo/pull/10320

Nov. 8, 2021

[ANN] Pharo Consortium New Academic Member: Ivan Franko National University of Lviv

by Marcus Denker

The Pharo Consortium is very happy to announce that Ivan Franko National University of Lviv has joined the Consortium as an Academic Member, the platform of cooperation will be the Faculty of Applied Mathematics and Informatics. About - Faculty of Applied Mathematics and Informatics: https://ami.lnu.edu.ua/en/ - Pharo Consortium: http://consortium.pharo.org The goal of the Pharo Consortium is to allow companies and institutions to support the ongoing development and future of Pharo. Individuals can support Pharo via the Pharo Association: http://association.pharo.org/

Nov. 4, 2021

a little help for my daughter :)

by Jannik Laval

Dear pharoers, My daughter's school (2nd year of kindergarten, she is 4 years old) is organizing a project "Around the world". The goal is to discover the countries of the world by different means, one of which leads me to ask for your help :). The goal is to send a postcard from your country with the following text: "Bonjour l'Ã©cole des Petits Pas" (in English, "Hello School of Little Steps"), and in the language of your country. The card should be sent to : Ecole maternelle des Petits Pas, classe verte 42 rue GrassoliÃ¨re 38 790 Saint Georges d'EspÃ©ranche, France Thank you for your help, and I hope to see you at the next ESUG. See you soon, -- ~~Jannik Laval~~ Associate professor (HDR) - Information systems agility IUT LumiÃ¨re, UniversitÃ© LumiÃ¨re Lyon 2 DISP Lab - https://disp-lab.fr/ +33 4 78 77 43 06 http://www.jannik-laval.eu

Nov. 1, 2021