- Pharo-dev - lists.pharo.org

Re: [Pharo-project] Databases with full text search support

by Norbert Hartl

On 27.01.2011, at 20:48, laurent laffont wrote: > Hi, > > I need to have full text indexing and searching over a 3-5 million documents. Actually the products I work on are written in PHP - MySQL. > > What would you use (open source - mandatory - our products are open source) in Smalltalk world to do this ? > That topic troubled me some time in the last years. The first approach I took was postgresql+tsearch2 because I used Glorp back then. AFAIK tsearch2 has been incorporated into postgresql in the meantime so this is no extra weight. Stemming libraries for postgresql are quite good and available for some languages. If you would use Glorp you can even extend the query language to support it out of the box. I changed the setup as I moved from postgresql to gemstone. With the new setup I would have used postgresql only for the fulltext retrieval which I didn't like. I used SOLR [1] instead. It is a search server build upon lucene which supports faceted search and a lot of nifty stuff. I used magritte to generate the xml that I've put into SOLR. I was satisfied with this solution. But all solutions that need to cross the smalltalk boundary start to become cumbersome at some stage. If I find time I would take another approach. I found a smalltalk stemming library while observing the moose mailing list. If you have good stemming support it may be a reasonable effort to produce forward and inverted indexes of your objects with search capabilities. If you use magritte (or pragmas) then you could add the fields to index there. The rest would be automatic: Taking all fields that should be searchable. Tokenize and canonicalize (using stemming) the words and build forward and inverted index. The only thing that would be needed is a change notification system that updates the index but I think this is more of an easy task. Well, that is the sketch what I would try to do myself but don't find time right now. I personally think that the power you gain by search smalltalk objects directly outweighs all the external servers by some extent. If you use a special external database anyway that supports it, well, then this would be the easiest start. hope this helps, Norbert

Jan. 28, 2011

Re: [Pharo-project] About FileSystem (probably not for 1.3)

by StÃ©phane Ducasse

On Jan 28, 2011, at 12:00 AM, Lukas Renggli wrote: >> I started to merge some of your changes to FS. > > I already merged most of the changes of Max into the main code. > >> Now I was wondering about this one: >> ***for my work on GitFS I needed a resumable exception (alternatively an ifPresent block would have done the trick but that would have ment more meddling with Filesystem) to delete existing files when copying. The following changes are a result of introducing this exception to the copy methods. >> *added class FSFileExistsResumbale and its superclass FSFilesystemException to support the changes to the copy methods >> *#copy:to: signals FSFileExistsResumbale instead of FSFileExists >> *FSCopyVisitor>>copyDirectory uses #ensureDirectory instead of #createDirectory >> *needed to change FSFilesystemTest>>testCopyDestExists to expect FSFileExistsResumable >> *FSReference>>basicCopyTo: signals FSFileExistsResumable instead of FSFileExists >> >> Could you explain us a bit more? > > This is not something that should go into the main code. If I remember > correctly this was a quick hack to avoid some problems in GitFS. Thanks lukas this is what I thought too. I picked some of the other elements. And I'm continuing to add comments. I will take your blog as starting material for a chapter. Stef > > Lukas > > -- > Lukas Renggli > www.lukas-renggli.ch >

Jan. 28, 2011

Re: [Pharo-project] About FileSystem (probably not for 1.3)

by StÃ©phane Ducasse

On Jan 27, 2011, at 10:56 PM, Dale Henrichs wrote: > Sounds good ... I'll work through the changes that I've made and try to extract out the goodness and then provide suggested changes (in the form of an mcz file you can merge if the changes make sense to you)... > > BTW, I used the FileSystem port with a Monticello port (replacing all of the FileDirectory stuff with FileSystem calls) and the Monticello port was based on a fairly recent Pharo version of Monticello:) Excellent > > Dale > > On 01/27/2011 01:34 PM, StÃ©phane Ducasse wrote: >> I'm. :) >> I do not want differences for the shake for them. I want an excellent system. >> I'm working on PharoTaskForces so if you can merge step by step it would be great. >> >> Stef >> >> >> On Jan 27, 2011, at 10:31 PM, Dale Henrichs wrote: >> >>> Stef, >>> >>> I have ported FileSystem to GemStone, but I had to make a number of changes that were necessary to improve the portability (at least for GemStone) ... It would be a bit of work for me to extract the changes, but I'd be willing to do it if you were interested in incorporated some of them (if not all) ... >>> >>> Dale >>> >>> On 01/27/2011 12:58 PM, StÃ©phane Ducasse wrote: >>>> Hi max >>>> >>>> I started to merge some of your changes to FS. >>>> Now I was wondering about this one: >>>> >>>> ***for my work on GitFS I needed a resumable exception (alternatively an ifPresent block would have done the trick but that would have ment more meddling with Filesystem) to delete existing files when copying. The following changes are a result of introducing this exception to the copy methods. >>>> *added class FSFileExistsResumbale and its superclass FSFilesystemException to support the changes to the copy methods >>>> *#copy:to: signals FSFileExistsResumbale instead of FSFileExists >>>> *FSCopyVisitor>>copyDirectory uses #ensureDirectory instead of #createDirectory >>>> *needed to change FSFilesystemTest>>testCopyDestExists to expect FSFileExistsResumable >>>> *FSReference>>basicCopyTo: signals FSFileExistsResumable instead of FSFileExists >>>> >>>> Could you explain us a bit more? >>>> >>>> Stef >>>>> >>>> >>> >> >

Jan. 28, 2011

Re: [Pharo-project] CommandShell

by Sean P. DeNigris

After loading CommandShell-dtl.50 (and .45, which works in Squeak 4.1), when I evaluate: (PipeableOSProcess command: 'ls') output I get a NonBooleanReceiver error when AttachableFileStream>>upToEndOfFile calls #atEndOfFile, which returns nil. Sean p.s. I also tried loading the latest Pharo package from the CommandShell repo -- View this message in context: http://forum.world.st/CommandShell-tp2226990p3243890.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Jan. 28, 2011

[Pharo-project] Smalltalk at FOSDEM in Brussels in February

by Geert Claes

For those who have missed http://astares.blogspot.com/2011/01/seaside-at-fossdem.html Torsten's and http://www.jarober.com/blog/blogView?showComments=true&printTitle=Seaside_a… James' blog posts or the post below on the Seaside mailing list; Smalltalk/Seaside at FOSDEM in Brussels on Sunday Feb 6 at 15:00 in room AW1.120 Link to FOSDEM: http://fosdem.org/2011/schedule/event/cfgmgmtdev http://fosdem.org/2011/schedule/event/cfgmgmtdev Stephan Eggermont wrote: > > Sunday 2011-2-6 we'll show how we deployed our Seaside application. If > you're interested, please join us. > > Stephan > > Configuration & Systems Management devroom (room AW1.120) > â¢ 15:00 - 15:45: Configuration management for developers (Stephan > Eggermont, Willem van den Ende) > > We created a server environment so a small startup could deploy its web > application with all the services it needs. Well, environment is a big > word... bootstrapped startups don't have money to manage redundant > servers, so we set up just one production server. Still, when disaster > strikes, we want to be able to have another server up and running within > thirty minutes. For developers laziness is a virtue. Therefore we want to > apply DRY (don't repeat yourself) to systems administration. So for this > project we considered each time we log in to the production server a > 'systems administration smell'. Attend our case study to see how we used a > configuration managent system (puppet), application specific scripts and a > few practices to keep us sane. See how far we have gotten towards our goal > of DRYness and fast recovery, and learn from our mistakes. > -- View this message in context: http://forum.world.st/Smalltalk-at-FOSDEM-in-Brussels-in-February-tp3243873… Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Jan. 28, 2011

Re: [Pharo-project] CommandShell

by laurent laffont

On Fri, Jan 28, 2011 at 7:19 AM, Sean P. DeNigris <sean(a)clipperadams.com>wrote: > > > laurent laffont wrote: > > > > It works (I can publish a package when back at home). > > > > Did you ever publish this anywhere? I didn't see it on SqS and I need it! > It seems no. Laurent. > > Thanks. > Sean > -- > View this message in context: > http://forum.world.st/CommandShell-tp2226990p3243771.html > Sent from the Pharo Smalltalk mailing list archive at Nabble.com. > >

Jan. 28, 2011

Re: [Pharo-project] Databases with full text search support

by laurent laffont

Riak looks extremely erotic: facet queries, HTTP/JSON query interface... Thanks for the pointer. Laurent. 2011/1/28 GÃ¶ran Krampe <goran(a)krampe.se> > Hi! > > I think the absolutely most interesting NoSQL DB with this in mind is Riak > and its Riak Search. Google it. I have been itching to write a binding for > Riak but have had no time to do it. > > Riak is built for "crazy just add another node" scaling and is modelled > after Dynamo. Recently Riak Search was added, which is a Lucene compatible > "overlay" for Riak thus giving you a free text search db with insane > scalability and robustness. > > CouchDB does not have anything close to this scalability, unless you look > at Cloudant perhaps - but AFAIK it does not have any kind of free text > indexing. Yes, CouchDB has a Lucene integration (I have written a binding in > C# for CouchDB and the Lucene integration) but we are still talking "single > server" setup. > > MongoDB is fast as hell, but AFAIK also does not have free text engine nor > the same "add another box" scalability like Riak has. > > regards, GÃ¶ran > >

Jan. 28, 2011

Re: [Pharo-project] CommandShell

by Sean P. DeNigris

laurent laffont wrote: > > It works (I can publish a package when back at home). > Did you ever publish this anywhere? I didn't see it on SqS and I need it! Thanks. Sean -- View this message in context: http://forum.world.st/CommandShell-tp2226990p3243771.html Sent from the Pharo Smalltalk mailing list archive at Nabble.com.

Jan. 28, 2011

Re: [Pharo-project] SimpleButtonMorph openInWorld

by Fernando Olivero

You can also evaluate the following expresions in a wokspace, SimpleButtonMorph allInstances do:[:each| each delete]. or | buttons | buttons := World submorphs detect:[:each| each isKindOf: SimpleButtonMorph ] . buttons anyOne delete. But i also prefer Serge option, because Morphs are directly manipulable for a reason! Fernando On Fri, Jan 28, 2011 at 2:49 AM, Serge Stinckwich <serge.stinckwich(a)gmail.com> wrote: > On Fri, Jan 28, 2011 at 9:40 AM, DAE <daebiz(a)douglasedmunds.com> wrote: >> This code makes a square button appear in the topleft corner >> >> Â Â Â Â SimpleButtonMorph new >> Â Â Â Â Â Â Â Â target: Smalltalk; >> Â Â Â Â Â Â Â Â label: 'Beep!'; >> Â Â Â Â Â Â Â Â actionSelector: #beep; >> Â Â Â Â Â Â Â Â openInWorld >> >> >> How do I make it go away? >> >> Sorry if I posted to the wrong group. Â If so, please tell me where is correct. > > On my macintosh, CTRL+ right mouse button > and select the delete item. > > Regards, > > -- > Serge Stinckwich > UMI UMMISCO 209 (IRD/UPMC), Hanoi, Vietnam > Every DSL ends up being Smalltalk > http://doesnotunderstand.org/ > >

Jan. 28, 2011

Re: [Pharo-project] SimpleButtonMorph openInWorld

by Serge Stinckwich

On Fri, Jan 28, 2011 at 9:40 AM, DAE <daebiz(a)douglasedmunds.com> wrote: > This code makes a square button appear in the topleft corner > > Â Â Â Â SimpleButtonMorph new > Â Â Â Â Â Â Â Â target: Smalltalk; > Â Â Â Â Â Â Â Â label: 'Beep!'; > Â Â Â Â Â Â Â Â actionSelector: #beep; > Â Â Â Â Â Â Â Â openInWorld > > > How do I make it go away? > > Sorry if I posted to the wrong group. Â If so, please tell me where is correct. On my macintosh, CTRL+ right mouse button and select the delete item. Regards, -- Serge Stinckwich UMI UMMISCO 209 (IRD/UPMC), Hanoi, Vietnam Every DSL ends up being Smalltalk http://doesnotunderstand.org/

Jan. 28, 2011