July 2021 - Pharo-users - lists.pharo.org

Re: Error in Pharo 9's Spec UI Framework Demo

by Esteban Lorenzano

Hi Tau, The filter that was bundled with the list components is deprecated and should not be used. Instead, we provide another component with that functionality: SpFilteringListPresenter. Take a loot there and its examples. Esteban On Jul 25 2021, at 5:52 am, tau <tau(a)cedalion.info> wrote: > > In the Spec2 examples included with Pharo9, in the example for SpListPresenter, when I try to input text in the list filter, I get an error: the debug window says: "Instance of OrderedCollection did not understand #hasElementAt:". The example works correctly in Pharo8. > > Is there a workaround to make the example work?

July 25, 2021

Error in Pharo 9's Spec UI Framework Demo

by tau

In the Spec2 examples included with Pharo9, in the example for SpListPresenter, when I try to input text in the list filter, I get an error: the debug window says: "Instance of OrderedCollection did not understand #hasElementAt:". The example works correctly in Pharo8. Is there a workaround to make the example work?

July 25, 2021

The Greatest Contributors to Smalltalk since 1980

by horrido.hobbies＠gmail.com

Iâm looking for a list of individuals who have contributed greatly to the advancement of Smalltalk, post Xerox PARC period (1972-1980). By advancement, I donât only mean on a technical basis but on an educational or public awareness basis (this could include books, podcasts, talk circuit, video instruction, etc.). Any basis that has made Smalltalk a success in the marketplace (including commercialization). I posted this question on LinkedIn and got one useful response: the late James Robertson. My personal nomination is Kent Beck. Iâm not that familiar with the deep history of Smalltalk, so Iâm looking for more nominations. Thanks.

July 24, 2021

Re: How do I remove old classes?

by Clacton Server

Good thinking. I will check that out. Thank you. David > On 23 Jul 2021, at 16:34, Richard Sargent <richard.sargent(a)gemtalksystems.com> wrote: > > > > On Fri, Jul 23, 2021, 06:57 Clacton Server <david(a)totallyobjects.com <mailto:david@totallyobjects.com>> wrote: > I have been having an enduring problem with Pharo "disappearing" on any machine I try it on. In an effort to resolve that it isn't my code, I want to remove the Package that contains all my Seaside code so that I can install a minimal home page under Seaside. This will, at least exonerate my code. However, I have removed the offending package, executed Smalltalk cleanUp: true but I can still get into the old code. A debugger comes up when I try the old code and it shows "AnObsoleteCNWMeetingsGalleries(AnObsoleteCNWBasic)" so the code is still there and all the bots that have the address will still be calling it. How do I finally remove the code? > > I would venture that you previously registered the class as a Seaside handler/application/whatever and the registry holds the class despite it being removed from the Smalltalk global dictionary. > > If so, you would need to remove it from the registry. > > > David Pennington

July 23, 2021

Re: How do I remove old classes?

by Richard Sargent

On Fri, Jul 23, 2021, 06:57 Clacton Server <david(a)totallyobjects.com> wrote: > I have been having an enduring problem with Pharo "disappearing" on any > machine I try it on. In an effort to resolve that it isn't my code, I want > to remove the Package that contains all my Seaside code so that I can > install a minimal home page under Seaside. This will, at least exonerate my > code. However, I have removed the offending package, executed Smalltalk > cleanUp: true but I can still get into the old code. A debugger comes up > when I try the old code and it shows > "AnObsoleteCNWMeetingsGalleries(AnObsoleteCNWBasic)" so the code is still > there and all the bots that have the address will still be calling it. How > do I finally remove the code? > I would venture that you previously registered the class as a Seaside handler/application/whatever and the registry holds the class despite it being removed from the Smalltalk global dictionary. If so, you would need to remove it from the registry. > David Pennington

July 23, 2021

Re: How do I remove old classes?

by Kasper Osterbye

Not knowing your specific usage, this tends to happen if you have an instance of class X, and then remove the class. To save the instance from going stale, Smalltalk has always made an Obsolete class. You can find the offending X instance by inspecting this: ReferenceFinder findAllPathsTo: X allInstances first This examines the first instance, you might want to print X allInstances size to see how big your issue is. Also, perhaps you need to force a garbage collect first: Smalltalk image garbageCollect Tracing the back-references can be a bit overwhelming the first times. For me the culprit has often been some global variable I thought I had under control. Best, Kasper > On 23 Jul 2021, at 15.56, Clacton Server <david(a)totallyobjects.com> wrote: > > I have been having an enduring problem with Pharo "disappearing" on any machine I try it on. In an effort to resolve that it isn't my code, I want to remove the Package that contains all my Seaside code so that I can install a minimal home page under Seaside. This will, at least exonerate my code. However, I have removed the offending package, executed Smalltalk cleanUp: true but I can still get into the old code. A debugger comes up when I try the old code and it shows "AnObsoleteCNWMeetingsGalleries(AnObsoleteCNWBasic)" so the code is still there and all the bots that have the address will still be calling it. How do I finally remove the code? > > David Pennington

July 23, 2021

How do I remove old classes?

by Clacton Server

I have been having an enduring problem with Pharo "disappearing" on any machine I try it on. In an effort to resolve that it isn't my code, I want to remove the Package that contains all my Seaside code so that I can install a minimal home page under Seaside. This will, at least exonerate my code. However, I have removed the offending package, executed Smalltalk cleanUp: true but I can still get into the old code. A debugger comes up when I try the old code and it shows "AnObsoleteCNWMeetingsGalleries(AnObsoleteCNWBasic)" so the code is still there and all the bots that have the address will still be calling it. How do I finally remove the code? David Pennington

July 23, 2021

[ANN] Buoy v6.1.0 [v6.1.0] released!

by Buenos Aires Smalltalk

July 23, 2021

Re: Pharo concurrency: select on socket or file IO

by vinref＠gmail.com

TaskIt is fantastic! I never realised this existed at all. This is exactly what I need. Thanks Pablo

July 22, 2021

Re: ZnServer on a cheap server: your experiences

by Esteban Maringolo

Yeap, not much more to add than what Sven said. What's important to mention is that they are really stable. I've been running Pharo servers in DO droplets of all sizes, without issues (some running for months), I only had to upscale one droplet to more memory, and it was because of a leak I introduced with PGSQL connections, otherwise, they're pretty lightweight for today's standards and a normal workload. I have deployments with nginx and Pharo images as upstreams, and I have one with Docker swarm and Traefik doing the load balancing among different Pharo workers and acting as the HTTPS endpoint. I'm removing this option though and moving back to nginx only. What I never tried was to host a docker container running Pharo, in some Docker hosting. It might be the best option as a quick start, but after doing the math, it's always more expensive than a Droplet. Best regards, Esteban A. Maringolo On Thu, Jul 22, 2021 at 3:21 AM Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Hi Vince, > > That is certainly possible and works well. I would recommend an instance with 1GB RAM, that leaves you some headroom. > > Deploying web applications is of course a broad subject, much of the required knowledge is not Pharo specific, but needed anyway. > > The last chapter in the Pharo Enterprise book (https://books.pharo.org/enterprise-pharo/) is a good starting point (Deployment). But there are other and different approaches. > > A plain HTTP demo instance running on a DO instance can be found here: http://zn.stfx.eu/welcome > > For production use you should front with NGINX or something similar to add HTTPS. > > May people on this mailing list deploy Pharo server applications, we have tens of them in day to day production doing real work. > > Good luck on your journey, you know where to ask questions. > > Regards, > > Sven > > > On 22 Jul 2021, at 07:56, vinref(a)gmail.com wrote: > > > > Anyone here run a web app using plain ZnServer (or subclasses) on a cheap VPS (i.e., $10/month DO droplet or equivalent). What are your experiences?, suggestions. > > > > I am planning a web app with just plain ZnServer, SQLite3, ATS or equivalent. > > > > Thanks, Vince > >

July 22, 2021

Re: Share package

by Renaud de Villemeur

Hi, You need to define a baseline, that will contain a definition of your dependencies, and everything needed to load and run your package with Metacello in Pharo. For a full documentation, look here:Â https://github.com/pharo-open-documentation/pharo-wiki/blob/master/General/… All package inside pharo are using a baseline, but some can be more intimidating than others to study. If you are looking for a simple, basic working example, you can have a look toÂ https://github.com/rvillemeur/SmallGameOfLife Renaud Jul 21, 2021, 11:10 by mspgate(a)gmail.com: > > Hello everybody, > > > sorry for the newbie question. I would like to know what is the fastest, lightweight way to share a package with other Pharo users. the package can be downloaded from my GitHub. > > > thanks. > > > d > >

July 22, 2021

Re: Pharo concurrency: select on socket or file IO

by tesonep＠gmail.com

Hi, Socket implementation in the VM already handles Async IO operations. The execution model in Pharo is intended to have many concurrent green threads (they are not OS threads, they are handled by Pharo VM). These green threads are called processes in Pharo. Pharo Green threads are quite cheap to create and use, and the context switch of them is quite cheap also. The programming model of Pharo is intended to have many concurrent green threads, without caring if they block in tIO operations. These block green threads are handled by the VM and the VM continue executing useful code. For orchestrating and communication between the green threads you can use low-level synchronization as semaphores and mutex, but also there is a rich library (Taskit) for using Asynchronous tasks, Callables and Futures. Cheers, Pablo On Thu, Jul 22, 2021 at 8:57 AM <vinref(a)gmail.com> wrote: > Can the Pharo VM or a library do a âselectâ on a collection of processes > waiting on a socket or file IO? Something like > https://man7.org/linux/man-pages/man2/select.2.html, or > https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-sel… > . > > Say I have a bunch of forked processes and some will get blocked waiting > for IO. I would put the blocked connections/processes in a collection and > non-blocked processes can continue to do useful work. At some appropriate I > would do a select on the blocked processes and resume those that are done. > > Vince > -- Pablo Tesone. tesonep(a)gmail.com

July 22, 2021

Re: Pharo concurrency: select on socket or file IO

by giorgio ferraris

Hi, Vince, I don't know exactly Pharo, but when you fork a process, it becomes an independent thread of work. So in that process you can send your async call and wait for a response and then do the work. All inside the forked process. It's the smallalk's scheduler that will take care of passing control to other available (green) threads So it's not clear from me what you need. If you have many forked processes, each one will do is work up to the end. No need to resume it from outside. hth ciao giorgio On Thu, Jul 22, 2021 at 8:57 AM <vinref(a)gmail.com> wrote: > Can the Pharo VM or a library do a âselectâ on a collection of processes > waiting on a socket or file IO? Something like > https://man7.org/linux/man-pages/man2/select.2.html, or > https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-sel… > . > > Say I have a bunch of forked processes and some will get blocked waiting > for IO. I would put the blocked connections/processes in a collection and > non-blocked processes can continue to do useful work. At some appropriate I > would do a select on the blocked processes and resume those that are done. > > Vince >

July 22, 2021

Pharo concurrency: select on socket or file IO

by vinref＠gmail.com

Can the Pharo VM or a library do a âselectâ on a collection of processes waiting on a socket or file IO? Something like https://man7.org/linux/man-pages/man2/select.2.html, or https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-sel…. Say I have a bunch of forked processes and some will get blocked waiting for IO. I would put the blocked connections/processes in a collection and non-blocked processes can continue to do useful work. At some appropriate I would do a select on the blocked processes and resume those that are done. Vince

July 22, 2021

Re: ZnServer on a cheap server: your experiences

by Sven Van Caekenberghe

Hi Vince, That is certainly possible and works well. I would recommend an instance with 1GB RAM, that leaves you some headroom. Deploying web applications is of course a broad subject, much of the required knowledge is not Pharo specific, but needed anyway. The last chapter in the Pharo Enterprise book (https://books.pharo.org/enterprise-pharo/) is a good starting point (Deployment). But there are other and different approaches. A plain HTTP demo instance running on a DO instance can be found here: http://zn.stfx.eu/welcome For production use you should front with NGINX or something similar to add HTTPS. May people on this mailing list deploy Pharo server applications, we have tens of them in day to day production doing real work. Good luck on your journey, you know where to ask questions. Regards, Sven > On 22 Jul 2021, at 07:56, vinref(a)gmail.com wrote: > > Anyone here run a web app using plain ZnServer (or subclasses) on a cheap VPS (i.e., $10/month DO droplet or equivalent). What are your experiences?, suggestions. > > I am planning a web app with just plain ZnServer, SQLite3, ATS or equivalent. > > Thanks, Vince >

July 22, 2021

ZnServer on a cheap server: your experiences

by vinref＠gmail.com

Anyone here run a web app using plain ZnServer (or subclasses) on a cheap VPS (i.e., $10/month DO droplet or equivalent). What are your experiences?, suggestions. I am planning a web app with just plain ZnServer, SQLite3, ATS or equivalent. Thanks, Vince

July 22, 2021

[ANN] smalltalkCI now supports Pharo 10

by Esteban Lorenzano

... this is more for Pharo developpers than Pharo users, but informing also users just in case :) https://github.com/hpi-swa/smalltalkCI/pull/538 Esteban

July 21, 2021

Share package

by mspgate＠gmail.com

Hello everybody, sorry for the newbie question. I would like to know what is the fastest, lightweight way to share a package with other Pharo users. the package can be downloaded from my GitHub. thanks. d

July 21, 2021

Re: How to handle (recover) from a ZnInvalidUTF8: Illegal continuation byte for utf-8 encoding error?

by Sven Van Caekenberghe

There is ZnCharacterEncoder knownEncodingIdentifiers. You either provide an identifier from this list (as string or symbol) or an instance (the argument gets sent #asZnCharacterEncoder if you want to know). Most text editors will tell you the encoding they are using to read your file and you can use that to inspect the contents. If you want, you can sent me such a file privately. Yes, you can access the encoder from the character read stream to configure it further. Or you do it upfront as an instance instead of an identifier. > On 20 Jul 2021, at 15:47, Tim Mackinnon <tim(a)testit.works> wrote: > > Hey thanks guys - so looking at readStreamEncoded: - how do I know what the valid encodings are? Skimming those docâs Sven referenced, I can start to pick out some - but is there a list? I see that method parameter says âanEncodingâ but the type hint on that is misleading as it seems like its a String or is it a Symbol? If I search for Encoder classes - I do find ZnCharacterEncoder - and it has class methods for latin1, utf8, ascii - so is this the definitive list? And should the encoding strings used in those methods be constants or something I can reference in my code? > > Gosh - this raises a whole host of things I just naively assumed happened for me. > > So it looks like the file giving me issues - seems to have characters like Â£ or Â¬ in it. So Iâm wondering how I know what the proper encoding format would be (I think these files were written out with some PHP app) - is it just a trial and error thing? > > I tried changing my code to: > > details parseStream: (firmEfs readStreamEncoded: 'iso-8859-1â). - and other variants like âASCIIâ and âlatin1â - and this then gives me another error: > "ZnCharacterEncodingError: Character Unicode code point outside encoder rangeâ > > So it does sound like I have a file that isnât conforming to known standards - and I guess I have to use #beLenient option. > > Sven - In the examples for using #beLenient - you seem to show something that assumes you will iterate with Do - as my existing code takes a stream, that it wants to do a #nextLine on - would it be bad to do something like this: > > efsStream := (firmEfs readStreamEncoded: 'latin1'). > efsStream encoder beLenient. > > details parsStream: efsStream. > > That is - get the endcoder from my Stream and make it lenient? > > Appreciate the pointers on this guys - Iâm definitely learning something new here. > > Tim > >> On 20 Jul 2021, at 12:11, Guillermo Polito <guillermopolito(a)gmail.com> wrote: >> >> >> >>> El 20 jul 2021, a las 11:45, Sven Van Caekenberghe <sven(a)stfx.eu> escribiÃ³: >>> >>> >>> >>>> On 20 Jul 2021, at 11:03, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>>> >>>> Hi Tim, >>>> >>>> An introduction to this part of the system is in https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf… [Character Encoding and Resource Meta Description] from the "Enterprise Pharo" book. >>>> >>>> The error means that a file that you try to read as UTF-8 does contain things that are invalid with respect to the UTF-8 standard. >>>> >>>> Are you sure the file is in UTF-8, maybe it is in ASCII, Latin-1 or something else ? >>>> >>>> It is possible to customise the encoding to something different than the default UTF-8. For non-UTF encoders, there is a strict/lenient option to disallow/allow illegal stuff (but then you will get these in your strings). >>>> >>>> I can show you how to do that if you want. >>> >>> '/var/log/system.log' asFileReference readStreamDo: [ :in | in upToEnd ]. >>> >>> '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | >>> (ZnCharacterReadStream on: in encoding: #ascii) upToEnd ]. >>> >>> '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | >>> (ZnCharacterReadStream on: in encoding: ZnCharacterEncoder ascii beLenient) upToEnd ]. >> >> There is also readStreamEncoded:[do:], which is a bit more concise but does the same :) >> >>> >>> HTH >>> >>>> Sven >>>> >>>>> On 20 Jul 2021, at 10:31, Tim Mackinnon <tim(a)testit.works> wrote: >>>>> >>>>> Hi - Iâm doing a bit of log file processing with Pharo - and Iâve hit an unexpected error and am wondering what the best way to approach it is. >>>>> >>>>> It seems that I have a log file that has unexpected characters, and so my readStream loop that reads lines gets an error: "ZnInvalidUTF8: Illegal continuation byte for utf-8 encodingâ. >>>>> >>>>> For some reason this file (unlike my others) seems to contain characters that it shouldnât - but what is the best way for me to continue processing? Should I be opening my files in a different way - or can I resume the error somehow- Iâm not familiar with this area of Pharo and am after a bit of advice. >>>>> >>>>> My code is like this (and I get the error when doing nextLine) >>>>> >>>>> >>>>> parseStream: aFileStream with: aBlock >>>>> | line items | >>>>> [ (line := aFileStream nextLine) isNil ] >>>>> whileFalse: [ >>>>> items := $/ split: line. >>>>> items size = 3 ifTrue: [aBlock value: items]] >>>>> >>>>> My stream is created like this: >>>>> >>>>> firmEfs := (pathName , '/' , firmName , '_files') asFileReference. >>>>> details parseStream: firmEfs readStream. >>>>> >>>>> >>>>> Should I be opening the stream a bit differently - or can I catch that encoding error and resume it with some safe character? >>>>> >>>>> Thanks for any help. >>>>> >>>>> Tim >

July 20, 2021

Re: How to handle (recover) from a ZnInvalidUTF8: Illegal continuation byte for utf-8 encoding error?

by Tim Mackinnon

Hey thanks guys - so looking at readStreamEncoded: - how do I know what the valid encodings are? Skimming those docâs Sven referenced, I can start to pick out some - but is there a list? I see that method parameter says âanEncodingâ but the type hint on that is misleading as it seems like its a String or is it a Symbol? If I search for Encoder classes - I do find ZnCharacterEncoder - and it has class methods for latin1, utf8, ascii - so is this the definitive list? And should the encoding strings used in those methods be constants or something I can reference in my code? Gosh - this raises a whole host of things I just naively assumed happened for me. So it looks like the file giving me issues - seems to have characters like Â£ or Â¬ in it. So Iâm wondering how I know what the proper encoding format would be (I think these files were written out with some PHP app) - is it just a trial and error thing? I tried changing my code to: details parseStream: (firmEfs readStreamEncoded: 'iso-8859-1â). - and other variants like âASCIIâ and âlatin1â - and this then gives me another error: "ZnCharacterEncodingError: Character Unicode code point outside encoder rangeâ So it does sound like I have a file that isnât conforming to known standards - and I guess I have to use #beLenient option. Sven - In the examples for using #beLenient - you seem to show something that assumes you will iterate with Do - as my existing code takes a stream, that it wants to do a #nextLine on - would it be bad to do something like this: efsStream := (firmEfs readStreamEncoded: 'latin1'). efsStream encoder beLenient. details parsStream: efsStream. That is - get the endcoder from my Stream and make it lenient? Appreciate the pointers on this guys - Iâm definitely learning something new here. Tim > On 20 Jul 2021, at 12:11, Guillermo Polito <guillermopolito(a)gmail.com <mailto:guillermopolito@gmail.com>> wrote: > > > >> El 20 jul 2021, a las 11:45, Sven Van Caekenberghe <sven(a)stfx.eu <mailto:sven@stfx.eu>> escribiÃ³: >> >> >> >>> On 20 Jul 2021, at 11:03, Sven Van Caekenberghe <sven(a)stfx.eu <mailto:sven@stfx.eu>> wrote: >>> >>> Hi Tim, >>> >>> An introduction to this part of the system is in https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf… <https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf…> [Character Encoding and Resource Meta Description] from the "Enterprise Pharo" book. >>> >>> The error means that a file that you try to read as UTF-8 does contain things that are invalid with respect to the UTF-8 standard. >>> >>> Are you sure the file is in UTF-8, maybe it is in ASCII, Latin-1 or something else ? >>> >>> It is possible to customise the encoding to something different than the default UTF-8. For non-UTF encoders, there is a strict/lenient option to disallow/allow illegal stuff (but then you will get these in your strings). >>> >>> I can show you how to do that if you want. >> >> '/var/log/system.log' asFileReference readStreamDo: [ :in | in upToEnd ]. >> >> '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | >> (ZnCharacterReadStream on: in encoding: #ascii) upToEnd ]. >> >> '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | >> (ZnCharacterReadStream on: in encoding: ZnCharacterEncoder ascii beLenient) upToEnd ]. > > There is also readStreamEncoded:[do:], which is a bit more concise but does the same :) > >> >> HTH >> >>> Sven >>> >>>> On 20 Jul 2021, at 10:31, Tim Mackinnon <tim(a)testit.works <mailto:tim@testit.works>> wrote: >>>> >>>> Hi - Iâm doing a bit of log file processing with Pharo - and Iâve hit an unexpected error and am wondering what the best way to approach it is. >>>> >>>> It seems that I have a log file that has unexpected characters, and so my readStream loop that reads lines gets an error: "ZnInvalidUTF8: Illegal continuation byte for utf-8 encodingâ. >>>> >>>> For some reason this file (unlike my others) seems to contain characters that it shouldnât - but what is the best way for me to continue processing? Should I be opening my files in a different way - or can I resume the error somehow- Iâm not familiar with this area of Pharo and am after a bit of advice. >>>> >>>> My code is like this (and I get the error when doing nextLine) >>>> >>>> >>>> parseStream: aFileStream with: aBlock >>>> | line items | >>>> [ (line := aFileStream nextLine) isNil ] >>>> whileFalse: [ >>>> items := $/ split: line. >>>> items size = 3 ifTrue: [aBlock value: items]] >>>> >>>> My stream is created like this: >>>> >>>> firmEfs := (pathName , '/' , firmName , '_files') asFileReference. >>>> details parseStream: firmEfs readStream. >>>> >>>> >>>> Should I be opening the stream a bit differently - or can I catch that encoding error and resume it with some safe character? >>>> >>>> Thanks for any help. >>>> >>>> Tim

July 20, 2021

Re: How to handle (recover) from a ZnInvalidUTF8: Illegal continuation byte for utf-8 encoding error?

by Sven Van Caekenberghe

> On 20 Jul 2021, at 12:11, Guillermo Polito <guillermopolito(a)gmail.com> wrote: > > > >> El 20 jul 2021, a las 11:45, Sven Van Caekenberghe <sven(a)stfx.eu> escribiÃ³: >> >> >> >>> On 20 Jul 2021, at 11:03, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >>> >>> Hi Tim, >>> >>> An introduction to this part of the system is in https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf… [Character Encoding and Resource Meta Description] from the "Enterprise Pharo" book. >>> >>> The error means that a file that you try to read as UTF-8 does contain things that are invalid with respect to the UTF-8 standard. >>> >>> Are you sure the file is in UTF-8, maybe it is in ASCII, Latin-1 or something else ? >>> >>> It is possible to customise the encoding to something different than the default UTF-8. For non-UTF encoders, there is a strict/lenient option to disallow/allow illegal stuff (but then you will get these in your strings). >>> >>> I can show you how to do that if you want. >> >> '/var/log/system.log' asFileReference readStreamDo: [ :in | in upToEnd ]. >> >> '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | >> (ZnCharacterReadStream on: in encoding: #ascii) upToEnd ]. >> >> '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | >> (ZnCharacterReadStream on: in encoding: ZnCharacterEncoder ascii beLenient) upToEnd ]. > > There is also readStreamEncoded:[do:], which is a bit more concise but does the same :) Yes indeed ! >> HTH >> >>> Sven >>> >>>> On 20 Jul 2021, at 10:31, Tim Mackinnon <tim(a)testit.works> wrote: >>>> >>>> Hi - Iâm doing a bit of log file processing with Pharo - and Iâve hit an unexpected error and am wondering what the best way to approach it is. >>>> >>>> It seems that I have a log file that has unexpected characters, and so my readStream loop that reads lines gets an error: "ZnInvalidUTF8: Illegal continuation byte for utf-8 encodingâ. >>>> >>>> For some reason this file (unlike my others) seems to contain characters that it shouldnât - but what is the best way for me to continue processing? Should I be opening my files in a different way - or can I resume the error somehow- Iâm not familiar with this area of Pharo and am after a bit of advice. >>>> >>>> My code is like this (and I get the error when doing nextLine) >>>> >>>> >>>> parseStream: aFileStream with: aBlock >>>> | line items | >>>> [ (line := aFileStream nextLine) isNil ] >>>> whileFalse: [ >>>> items := $/ split: line. >>>> items size = 3 ifTrue: [aBlock value: items]] >>>> >>>> My stream is created like this: >>>> >>>> firmEfs := (pathName , '/' , firmName , '_files') asFileReference. >>>> details parseStream: firmEfs readStream. >>>> >>>> >>>> Should I be opening the stream a bit differently - or can I catch that encoding error and resume it with some safe character? >>>> >>>> Thanks for any help. >>>> >>>> Tim

July 20, 2021

Re: How to handle (recover) from a ZnInvalidUTF8: Illegal continuation byte for utf-8 encoding error?

by Guillermo Polito

> El 20 jul 2021, a las 11:45, Sven Van Caekenberghe <sven(a)stfx.eu> escribiÃ³: > > > >> On 20 Jul 2021, at 11:03, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: >> >> Hi Tim, >> >> An introduction to this part of the system is in https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf… [Character Encoding and Resource Meta Description] from the "Enterprise Pharo" book. >> >> The error means that a file that you try to read as UTF-8 does contain things that are invalid with respect to the UTF-8 standard. >> >> Are you sure the file is in UTF-8, maybe it is in ASCII, Latin-1 or something else ? >> >> It is possible to customise the encoding to something different than the default UTF-8. For non-UTF encoders, there is a strict/lenient option to disallow/allow illegal stuff (but then you will get these in your strings). >> >> I can show you how to do that if you want. > > '/var/log/system.log' asFileReference readStreamDo: [ :in | in upToEnd ]. > > '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | > (ZnCharacterReadStream on: in encoding: #ascii) upToEnd ]. > > '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | > (ZnCharacterReadStream on: in encoding: ZnCharacterEncoder ascii beLenient) upToEnd ]. There is also readStreamEncoded:[do:], which is a bit more concise but does the same :) > > HTH > >> Sven >> >>> On 20 Jul 2021, at 10:31, Tim Mackinnon <tim(a)testit.works> wrote: >>> >>> Hi - Iâm doing a bit of log file processing with Pharo - and Iâve hit an unexpected error and am wondering what the best way to approach it is. >>> >>> It seems that I have a log file that has unexpected characters, and so my readStream loop that reads lines gets an error: "ZnInvalidUTF8: Illegal continuation byte for utf-8 encodingâ. >>> >>> For some reason this file (unlike my others) seems to contain characters that it shouldnât - but what is the best way for me to continue processing? Should I be opening my files in a different way - or can I resume the error somehow- Iâm not familiar with this area of Pharo and am after a bit of advice. >>> >>> My code is like this (and I get the error when doing nextLine) >>> >>> >>> parseStream: aFileStream with: aBlock >>> | line items | >>> [ (line := aFileStream nextLine) isNil ] >>> whileFalse: [ >>> items := $/ split: line. >>> items size = 3 ifTrue: [aBlock value: items]] >>> >>> My stream is created like this: >>> >>> firmEfs := (pathName , '/' , firmName , '_files') asFileReference. >>> details parseStream: firmEfs readStream. >>> >>> >>> Should I be opening the stream a bit differently - or can I catch that encoding error and resume it with some safe character? >>> >>> Thanks for any help. >>> >>> Tim

July 20, 2021

Re: How to handle (recover) from a ZnInvalidUTF8: Illegal continuation byte for utf-8 encoding error?

by Sven Van Caekenberghe

> On 20 Jul 2021, at 11:03, Sven Van Caekenberghe <sven(a)stfx.eu> wrote: > > Hi Tim, > > An introduction to this part of the system is in https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf… [Character Encoding and Resource Meta Description] from the "Enterprise Pharo" book. > > The error means that a file that you try to read as UTF-8 does contain things that are invalid with respect to the UTF-8 standard. > > Are you sure the file is in UTF-8, maybe it is in ASCII, Latin-1 or something else ? > > It is possible to customise the encoding to something different than the default UTF-8. For non-UTF encoders, there is a strict/lenient option to disallow/allow illegal stuff (but then you will get these in your strings). > > I can show you how to do that if you want. '/var/log/system.log' asFileReference readStreamDo: [ :in | in upToEnd ]. '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | (ZnCharacterReadStream on: in encoding: #ascii) upToEnd ]. '/var/log/system.log' asFileReference binaryReadStreamDo: [ :in | (ZnCharacterReadStream on: in encoding: ZnCharacterEncoder ascii beLenient) upToEnd ]. HTH > Sven > >> On 20 Jul 2021, at 10:31, Tim Mackinnon <tim(a)testit.works> wrote: >> >> Hi - Iâm doing a bit of log file processing with Pharo - and Iâve hit an unexpected error and am wondering what the best way to approach it is. >> >> It seems that I have a log file that has unexpected characters, and so my readStream loop that reads lines gets an error: "ZnInvalidUTF8: Illegal continuation byte for utf-8 encodingâ. >> >> For some reason this file (unlike my others) seems to contain characters that it shouldnât - but what is the best way for me to continue processing? Should I be opening my files in a different way - or can I resume the error somehow- Iâm not familiar with this area of Pharo and am after a bit of advice. >> >> My code is like this (and I get the error when doing nextLine) >> >> >> parseStream: aFileStream with: aBlock >> | line items | >> [ (line := aFileStream nextLine) isNil ] >> whileFalse: [ >> items := $/ split: line. >> items size = 3 ifTrue: [aBlock value: items]] >> >> My stream is created like this: >> >> firmEfs := (pathName , '/' , firmName , '_files') asFileReference. >> details parseStream: firmEfs readStream. >> >> >> Should I be opening the stream a bit differently - or can I catch that encoding error and resume it with some safe character? >> >> Thanks for any help. >> >> Tim >

July 20, 2021

Re: How to handle (recover) from a ZnInvalidUTF8: Illegal continuation byte for utf-8 encoding error?

by Sven Van Caekenberghe

Hi Tim, An introduction to this part of the system is in https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessf… [Character Encoding and Resource Meta Description] from the "Enterprise Pharo" book. The error means that a file that you try to read as UTF-8 does contain things that are invalid with respect to the UTF-8 standard. Are you sure the file is in UTF-8, maybe it is in ASCII, Latin-1 or something else ? It is possible to customise the encoding to something different than the default UTF-8. For non-UTF encoders, there is a strict/lenient option to disallow/allow illegal stuff (but then you will get these in your strings). I can show you how to do that if you want. Sven > On 20 Jul 2021, at 10:31, Tim Mackinnon <tim(a)testit.works> wrote: > > Hi - Iâm doing a bit of log file processing with Pharo - and Iâve hit an unexpected error and am wondering what the best way to approach it is. > > It seems that I have a log file that has unexpected characters, and so my readStream loop that reads lines gets an error: "ZnInvalidUTF8: Illegal continuation byte for utf-8 encodingâ. > > For some reason this file (unlike my others) seems to contain characters that it shouldnât - but what is the best way for me to continue processing? Should I be opening my files in a different way - or can I resume the error somehow- Iâm not familiar with this area of Pharo and am after a bit of advice. > > My code is like this (and I get the error when doing nextLine) > > > parseStream: aFileStream with: aBlock > | line items | > [ (line := aFileStream nextLine) isNil ] > whileFalse: [ > items := $/ split: line. > items size = 3 ifTrue: [aBlock value: items]] > > My stream is created like this: > > firmEfs := (pathName , '/' , firmName , '_files') asFileReference. > details parseStream: firmEfs readStream. > > > Should I be opening the stream a bit differently - or can I catch that encoding error and resume it with some safe character? > > Thanks for any help. > > Tim

July 20, 2021

How to handle (recover) from a ZnInvalidUTF8: Illegal continuation byte for utf-8 encoding error?

by Tim Mackinnon

Hi - Iâm doing a bit of log file processing with Pharo - and Iâve hit an unexpected error and am wondering what the best way to approach it is. It seems that I have a log file that has unexpected characters, and so my readStream loop that reads lines gets an error: "ZnInvalidUTF8: Illegal continuation byte for utf-8 encodingâ. For some reason this file (unlike my others) seems to contain characters that it shouldnât - but what is the best way for me to continue processing? Should I be opening my files in a different way - or can I resume the error somehow- Iâm not familiar with this area of Pharo and am after a bit of advice. My code is like this (and I get the error when doing nextLine) parseStream: aFileStream with: aBlock | line items | [ (line := aFileStream nextLine) isNil ] whileFalse: [ items := $/ split: line. items size = 3 ifTrue: [aBlock value: items]] My stream is created like this: firmEfs := (pathName , '/' , firmName , '_files') asFileReference. details parseStream: firmEfs readStream. Should I be opening the stream a bit differently - or can I catch that encoding error and resume it with some safe character? Thanks for any help. Tim

July 20, 2021