Pharo-dev
By thread
pharo-dev@lists.pharo.org
By month
Messages by month
- ----- 2026 -----
- July
- June
- May
- April
- March
- February
- January
- ----- 2025 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2024 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
January 2015
- 1046 messages
Re: [Pharo-dev] Prepared Statements on PostgresV2?
by Alain Rastoul
It was just a thought, I also remembered that there was a problem with
the single threaded approach of the VM. With ODBC FFI, it is very easy
for a simple application to lock itself with two connections with Sql
server or Oracle.
I think it was a motivation for DbxTalk development : doing db calls in
another thread, the same as it is for socket IO (perhaps it could be
done with nativeboost/odbc ?).
Developing native drivers (like type 4 jdbc drivers in java) in Pharo is
probably a very good solution, but will require lot of work and multiple
specific skills (probably someone developing-maintaining driver-x/db-x,
another the driver-y/db-y and so on).
Regards,
Alain
Le 11/01/2015 09:29, stepharo a écrit :
>
> Le 10/1/15 14:47, Alain Rastoul a écrit :
>> Hi,
>>
>> Nice document, however, about ODBC support column,
>> looking at and old (Pharo 1.3) image I have on my pc, I can see:
>> ODBCLibray>>moduleName
>> "Return the name of the module for this library"
>> SmalltalkImage current platformName = 'Win32' ifTrue: [ ^ 'odbc32' ].
>> SmalltalkImage current platformName = 'unix' ifTrue: [ ^
>> 'libodbc.so' ].
>> ^ self error: 'Don''t know the ODBC library name'
>> I thought odbc was supported under linux in the FFI ODBC package.
>>
>> odbc rocks under windows, for non dotNet apps, (my opinion), and MS
>> has released an Mssql odbc driver for linux (They say they love linux
>> now :) ).
>>
>> http://www.microsoft.com/en-us/download/details.aspx?id=28160
>>
>> Oracle is also supported for 12 (I think at least since 10g)
>> https://docs.oracle.com/database/121/ODBCR/toc.htm#ODBCR101
>>
>> Could it be interesting to re-investigate ODBC support ?
>
> No idea :)
> I'm not expert
>> Or DbxTalk is the Pharo standard (and it doesn't mind) ?
>
> What oliier told me is that organizations reluctant to add extra
> librariesto deploy app so
> having some good native drivers would be better.
>
> Stef
>>
>> Regards,
>>
>> Alain
>>
>>
>> Le 10/01/2015 11:32, stepharo a écrit :
>>>
>>> Le 10/1/15 10:59, Stephan Eggermont a écrit :
>>>> Stef wrote:
>>>>> May be this is the time to improve this package :)
>>>>> I'm sorry I have no DB experience.
>>>>> As part of the consortium agenda for this year we have
>>>>> - better ffi
>>>>> - better db support
>>>> Moving to Levente's PostgreSQLv3 and migrating that
>>>> might be a better idea. He has been using it in production
>>>> systems for a long time AFAIK.
>>>
>>> when I meant db support it is not limited to one implementation.
>>> Olivier Auverlot started to do a document listing all the solutions and
>>> their status.
>>> https://docs.google.com/spreadsheets/d/1uD6UFmwCJxAkOC9tlvdJsHtt10C0KHJ6Zp8…
>>>
>>>
>>> Add the information avout SQLv3
>>>
>>> Now I do not get it.
>>> Did you port it to Pharo?
>>>>
>>>> The code mirror is very interesting. Porting it needs
>>>> someone with close access to Opal/old compiler knowledge.
>>>> The way it works now in Squeak looks ideal for someone
>>>> with a PostgreSQL background. Using a code browser
>>>> to edit plsql, and having a directly mirrored smalltalk
>>>> class for each table.
>>>>
>>>> http://forum.world.st/Status-of-PostgresV3-td4780110.html
>>>> I have verified that the connection works and that I can get
>>>> data out of postgres.
>>>> Missing:
>>>> - switch the code highlighter from smalltalk to one
>>>> for plsql in nautilus.
>>>> - connect to glorp
>>>> - (later) add a real plsql parser (petitsql?)
>>>>
>>>> Stephan
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>
>
>
Jan. 11, 2015
Re: [Pharo-dev] Prepared Statements on PostgresV2?
by stepharo
Le 10/1/15 14:47, Alain Rastoul a écrit :
> Hi,
>
> Nice document, however, about ODBC support column,
> looking at and old (Pharo 1.3) image I have on my pc, I can see:
> ODBCLibray>>moduleName
> "Return the name of the module for this library"
> SmalltalkImage current platformName = 'Win32' ifTrue: [ ^ 'odbc32' ].
> SmalltalkImage current platformName = 'unix' ifTrue: [ ^
> 'libodbc.so' ].
> ^ self error: 'Don''t know the ODBC library name'
> I thought odbc was supported under linux in the FFI ODBC package.
>
> odbc rocks under windows, for non dotNet apps, (my opinion), and MS
> has released an Mssql odbc driver for linux (They say they love linux
> now :) ).
>
> http://www.microsoft.com/en-us/download/details.aspx?id=28160
>
> Oracle is also supported for 12 (I think at least since 10g)
> https://docs.oracle.com/database/121/ODBCR/toc.htm#ODBCR101
>
> Could it be interesting to re-investigate ODBC support ?
No idea :)
I'm not expert
> Or DbxTalk is the Pharo standard (and it doesn't mind) ?
What oliier told me is that organizations reluctant to add extra
librariesto deploy app so
having some good native drivers would be better.
Stef
>
> Regards,
>
> Alain
>
>
> Le 10/01/2015 11:32, stepharo a écrit :
>>
>> Le 10/1/15 10:59, Stephan Eggermont a écrit :
>>> Stef wrote:
>>>> May be this is the time to improve this package :)
>>>> I'm sorry I have no DB experience.
>>>> As part of the consortium agenda for this year we have
>>>> - better ffi
>>>> - better db support
>>> Moving to Levente's PostgreSQLv3 and migrating that
>>> might be a better idea. He has been using it in production
>>> systems for a long time AFAIK.
>>
>> when I meant db support it is not limited to one implementation.
>> Olivier Auverlot started to do a document listing all the solutions and
>> their status.
>> https://docs.google.com/spreadsheets/d/1uD6UFmwCJxAkOC9tlvdJsHtt10C0KHJ6Zp8…
>>
>>
>> Add the information avout SQLv3
>>
>> Now I do not get it.
>> Did you port it to Pharo?
>>>
>>> The code mirror is very interesting. Porting it needs
>>> someone with close access to Opal/old compiler knowledge.
>>> The way it works now in Squeak looks ideal for someone
>>> with a PostgreSQL background. Using a code browser
>>> to edit plsql, and having a directly mirrored smalltalk
>>> class for each table.
>>>
>>> http://forum.world.st/Status-of-PostgresV3-td4780110.html
>>> I have verified that the connection works and that I can get
>>> data out of postgres.
>>> Missing:
>>> - switch the code highlighter from smalltalk to one
>>> for plsql in nautilus.
>>> - connect to glorp
>>> - (later) add a real plsql parser (petitsql?)
>>>
>>> Stephan
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>
>
Jan. 11, 2015
OSProcess stdErr issue on Windows
by Ben Coman
On Windows 7, trying to execute a non-existent program as follows...
OSProcess command: 'notepadXX.exe'
...causes an MNU in ExternalWindowsOSProcess>>value. This is due to writing
to stderr where "self initialStdErr" returns nil, seemingly due to
ExternalWindowsOSProcess missing the #initialize and #setDefaults of E
xternalUnixOSProcess.
Now I understand that stderr is problematic in general on Windows.
The following statements...
OSProcess thisOSProcess stdErr nextPutAll: 'test'.
FileStream stderr nextPutAll: 'test'.
...both work fine on OSX, but fail on Windows with the error
"FileWriteError: File stderr is closed"
The result is the same on both Pharo 4 and Squeak 4.5.
(Squeak also brings up a bottom message bar saying... "# To disable: F2 ->
'debug options' -> 'show console on errors' WARNING: Manufactured file
handle detected!")
Pharo has a workaround for the general issue by creating a disk file
named 'stderr'
as follows...
FileStream class >> standardIOStreamNamed: moniker forWrite: forWrite
self flag: #todo. "This is an ugly hack, while waiting for a real fix for
windows. There several problems with this approach, but it allow us to run
tests, etc."
Smalltalk os isWin32
ifTrue: [
[ ^ MultiByteFileStream forceNewFileNamed: moniker asString ]
on: CannotDeleteFileException do: [
"HACK: if the image is opened a second time windows barks about the already
opened locked file"
^ MultiByteFileStream forceNewFileNamed: moniker asString, '_', (Random new
nextInt: SmallInteger maxVal) asString ]].
So to address the MNU for OSProcess?
1. wrap the "self initialStdErr" with an #ifNil: check
2. Squeak adopt the Pharo workaround, then lean on that to define
#initialize and #setDefaults for ExternalWindowsOSProcess
3. real fix for stderr (outside of OSProcess?)
4. wont fix
5. other?
Note that this is not a burning issue for me. I am just cleaning out old
issues that I logged in the Pharo bug tracker. If the answer if (4.) I'll
just close the issue [1].
For both Squeak and Pharo, I
loaded ConfigurationOfOSProcess-ThierryGoubier.33 from...
MCHttpRepository
location: 'http://smalltalkhub.com/mc/Pharo/MetaRepoForPharo40/main'
(Perhaps someone with access can copy that to the main repo)
cheers -ben
[1] https://pharo.fogbugz.com/default.asp?11615
Jan. 11, 2015
Re: [Pharo-dev] Prepared Statements on PostgresV2?
by Stephan Eggermont
Stef wrote:
>Now I do not get it.
> Did you port it to Pharo?
I spend about a day evaluating it in 40272.
I looked at three packages (and their two test packages).
PostgresV3-Core handles the V3 protocol, makes a connection
using a socket, provides a result set with rows, columns and
text converters.
I got a few problems in the unit tests, looks like something in the
way floats are handled is different (PG3SocketReadStream>>parseFloatLength:
and there is a method missing PG3SocketReadStream>>parseScaledDecimal
PostgresV3-Pool provides a connection pool.
I was able to use that to connect to a database, execute queries and get
data out of it. Issues there are trivial:
PG3Connection>>md5HashMessage: aString
^Smalltalk globals
at: #MD5
ifPresent: [ :md5 | (md5 hashMessage: aString) hex asLowercase ]
ifAbsent: [ WebUtils md5Digest: aString ]
We don't have WebUtils, but we have MD5
Then there is PostgresV3-CodeMirror.
In PG3SchemaMirror there are some overrides to select a different
compiler (+parser) and code highlighter. That allows subclasses
to have plsql functions as methods, using the PG3FunctionMirror.
In Pharo, I couldn't easily find how to change the code highlighter in Nautilus.
The PG3Compiler just changes the parser used to PG3Parser.
The PG3Parser is a subclass from the squeak parser and so uses some things that work a bit different
PG3Parser>>checkSyntaxFor: uses a cue instance variable. It is not a full plsql parser, relying on the
database to check syntax.
PG3SchemaChanges uses Toolbuilder to show a window with the differences between the functions in the
database and those in the image and select which ones to change.
Stephan
Jan. 10, 2015
Re: [Pharo-dev] Prepared Statements on PostgresV2?
by Levente Uzonyi
There's no documentation yet, but the mail quoted by Stephan has detailed
description with examples:
http://forum.world.st/Status-of-PostgresV3-td4780110.html
Levente
On Sat, 10 Jan 2015, phil(a)highoctane.be wrote:
>
>
> Le 10 janv. 2015 20:04, "Levente Uzonyi" <leves(a)elte.hu> a écrit :
> >
> > The V2 version of the protocol doesn't support prepared statements.
> >
> > When we were writing the PostgresV3 package, we planned to implement prepared statements (via the extended query protocol), but later we decided to not do it.
> >
> > Why?
> >
> > The protocol is way more complex than the simple query protocol (which is used for regular queries), and involves more message sends, which can increase the network latency.
> >
> > The security that prepared statements give is equivalent to proper escaping. That can and should be implemented for standard queries too.
> >
> > The performance "improvement" what prepared statements give has two sides. You save the cost of parsing and planning, but the query is executed with the same plan, no matter what the arguments are. This can
> lead to sub-optimal performance.
> >
> > With the extended query protocol you can execute one statement per query. With the simple query protocol you can execute as many as you want. This can increase the network latency significantly for prepared
> statements.
> >
> > In PostgreSQL, calling functions using the simple query protocol behave similarly to prepared statements. You don't have to pass your SQL query to the server all the time, just the ame of the function and
> its arguments wrapped in a very simple SQL query.
> > This way the plan is created when the function is executed the first time (so it's even superior to prepared statements, because there's no per-connection parsing and planning cost).
> >
> > Functions also let you to use any procedural language PostgreSQL supports, which are currently SQL, PL/pgSQL, Pl/Tcl, PL/Perl, PL/Python. Using the high level languages makes it easier to write complex
> queries, or express complex logic, which wouldn't be possible with a single SQL query.
> >
> > So we decided to use functions instead. We added some image side tools to make it easier to write, save, load and debug functions. Currently only the PL/pgSQL language is supported by these tools, and there
> are some other limitations, but we'll implement new features as time permits.
> >
> > What about performance?
> >
> > Using our toolchain, a single process can execute 2500-3000 queries (using a modern CPU, and CogVM) per second, and an image can make about 5-6k queries per second overall.
> > This includes the overhead of using the connection pool, and the generation of the textual SQL query for each function call. Without these it's possible to go above 10k/second.
> >
>
> About connection pool: how does one uses it?
>
> Is there a doc somewhere ?
>
> Phil
>
> > Text or binary?
> >
> > We implemented the text-based protocol, but added the hooks for the binary protocol too. Why?
> > The binary protocol is undocumented (its documentation is the C source code), and it's subject to change with each release.
> > The text-based protocol is a bit better documented, and it should work with all versions of PostgreSQL starting from 7.4.
> >
> > Levente
> >
> >
> > On Sat, 10 Jan 2015, David Carlos Manuelda wrote:
> >
> >> I've installed and tested PostgresV2 under pharo with the following doits
> >>
> >> Gofer new
> >> Â Â smalltalkhubUser: 'PharoExtras' project: 'PostgresV2';
> >> Â Â configuration;
> >> Â Â load.
> >> (#ConfigurationOfPostgresV2 asClass project version: '2.4') load
> >>
> >> While it works good, it is missing a very important feature from both
> >> security and performance point of view: The prepared statements.
> >>
> >> As a brief, prepared statements are parameterized SQL statements that are
> >> loaded ONCE per connection instead of sending the whole query to DB every
> >> time, and also, they are parameterized, so it completelly prevents SQL
> >> injection, as the parameters are automatically 'detected' and scaped and/or
> >> handled accordingly without allowing in any case a parameter to alter the
> >> SQL meaning, which can happen by using regular SQL queries made by string
> >> concatenation.
> >>
> >> I browsed the class and did not find any prepare: method nor anything
> >> similar.
> >>
> >> Also, you can give a name to a SQL sentence, which makes the code much more
> >> readable without messing too much logic with SQL commands and string
> >> concatenation.
> >>
> >> An example is as follows: (supposing we have an instance variable
> >> connection, already initialized and connected via PGConnection class)
> >>
> >> Instead of:
> >>
> >> self connection execute: 'SELECT data FROM mytable WHERE name=''', anUser
> >> userName, ''';'.
> >>
> >> Would be something like this:
> >> self connection executePrepared: 'getUserData' with: anUser userName.
> >>
> >> And another suggestion could be something like:
> >> self connection prepare: 'getUserData' withSQL: 'SELECT data FROM mytable
> >> WHERE name=$1'
> >>
> >> Any plan for this to be implemented or any hint to other PostgreSQL class
> >> that already has it?
> >>
> >> David.
> >>
> >> P.S. I've written in my blog about this some time ago:
> >> http://stormbyte.blogspot.com.es/2012/06/programming-with-database-using.ht… if someone finds it useful.
> >>
> >>
> >>
> >
>
>
>
Jan. 10, 2015
Re: [Pharo-dev] Prepared Statements on PostgresV2?
by Levente Uzonyi
You can achieve the same with functions. Here's an example for a method
representing a function in PostgresV3:
ExampleSchemaMirror >> add: a and: b
<pg3Function: 'add'
arguments: #('a' integer 'b' integer)
returns: #integer
volatility: #volatile>
begin
return a + b;
end;
And here's how you call that function:
ExampleSchemaMirror default add: 3 and: 4. "==> 7"
Levente
On Sat, 10 Jan 2015, David Carlos Manuelda wrote:
>
> Well, I want to point a thing to be considered regarding prepared
> statements.
>
> It is not only about security/performance, using them also helps to have
> your code cleaner and read it better.
> * No need to use ' character for text elements inside SQL query
> Avoid the mess that: execute: 'INSERT INTO table VALUES (''', aUser
> id, ''', ''', Auser otherField, ''');'
> Confusing and hard to read '
> Instead, with prepared is something like 'INSERT INTO table
> VALUES ($1,$2)'
> * No need to filter data ( even more readable code )
> * Ability to name the SQL so you can use it later by name ( for comodity
> )
>
> Just take this in consideration if it is going to be evaluated for
> implementation :)
Jan. 10, 2015
Re: [Pharo-dev] Prepared Statements on PostgresV2?
by David Carlos Manuelda
Levente Uzonyi wrote:
> The V2 version of the protocol doesn't support prepared statements.
>
> When we were writing the PostgresV3 package, we planned to implement
> prepared statements (via the extended query protocol), but later we
> decided to not do it.
>
> Why?
>
> The protocol is way more complex than the simple query protocol (which is
> used for regular queries), and involves more message sends, which can
> increase the network latency.
>
> The security that prepared statements give is equivalent to proper
> escaping. That can and should be implemented for standard queries too.
>
> The performance "improvement" what prepared statements give has two sides.
> You save the cost of parsing and planning, but the query is executed with
> the same plan, no matter what the arguments are. This can lead to
> sub-optimal performance.
>
> With the extended query protocol you can execute one statement per query.
> With the simple query protocol you can execute as many as you want. This
> can increase the network latency significantly for prepared statements.
>
> In PostgreSQL, calling functions using the simple query protocol behave
> similarly to prepared statements. You don't have to pass your SQL query to
> the server all the time, just the ame of the function and its arguments
> wrapped in a very simple SQL query.
> This way the plan is created when the function is executed the first time
> (so it's even superior to prepared statements, because there's no
> per-connection parsing and planning cost).
>
> Functions also let you to use any procedural language PostgreSQL supports,
> which are currently SQL, PL/pgSQL, Pl/Tcl, PL/Perl, PL/Python. Using the
> high level languages makes it easier to write complex queries, or express
> complex logic, which wouldn't be possible with a single SQL query.
>
> So we decided to use functions instead. We added some image side tools to
> make it easier to write, save, load and debug functions. Currently only
> the PL/pgSQL language is supported by these tools, and there are some
> other limitations, but we'll implement new features as time permits.
>
> What about performance?
>
> Using our toolchain, a single process can execute 2500-3000 queries
> (using a modern CPU, and CogVM) per second, and an image can make about
> 5-6k queries per second overall.
> This includes the overhead of using the connection pool, and the
> generation of the textual SQL query for each function call. Without these
> it's possible to go above 10k/second.
>
> Text or binary?
>
> We implemented the text-based protocol, but added the hooks for the
> binary protocol too. Why?
> The binary protocol is undocumented (its documentation is the C source
> code), and it's subject to change with each release.
> The text-based protocol is a bit better documented, and it should work
> with all versions of PostgreSQL starting from 7.4.
>
> Levente
>
Well, I want to point a thing to be considered regarding prepared
statements.
It is not only about security/performance, using them also helps to have
your code cleaner and read it better.
* No need to use ' character for text elements inside SQL query
Avoid the mess that: execute: 'INSERT INTO table VALUES (''', aUser
id, ''', ''', Auser otherField, ''');'
Confusing and hard to read '
Instead, with prepared is something like 'INSERT INTO table
VALUES ($1,$2)'
* No need to filter data ( even more readable code )
* Ability to name the SQL so you can use it later by name ( for comodity
)
Just take this in consideration if it is going to be evaluated for
implementation :)
> On Sat, 10 Jan 2015, David Carlos Manuelda wrote:
>
>> I've installed and tested PostgresV2 under pharo with the following doits
>>
>> Gofer new
>> smalltalkhubUser: 'PharoExtras' project: 'PostgresV2';
>> configuration;
>> load.
>> (#ConfigurationOfPostgresV2 asClass project version: '2.4') load
>>
>> While it works good, it is missing a very important feature from both
>> security and performance point of view: The prepared statements.
>>
>> As a brief, prepared statements are parameterized SQL statements that are
>> loaded ONCE per connection instead of sending the whole query to DB every
>> time, and also, they are parameterized, so it completelly prevents SQL
>> injection, as the parameters are automatically 'detected' and scaped
>> and/or handled accordingly without allowing in any case a parameter to
>> alter the SQL meaning, which can happen by using regular SQL queries made
>> by string concatenation.
>>
>> I browsed the class and did not find any prepare: method nor anything
>> similar.
>>
>> Also, you can give a name to a SQL sentence, which makes the code much
>> more readable without messing too much logic with SQL commands and string
>> concatenation.
>>
>> An example is as follows: (supposing we have an instance variable
>> connection, already initialized and connected via PGConnection class)
>>
>> Instead of:
>>
>> self connection execute: 'SELECT data FROM mytable WHERE name=''', anUser
>> userName, ''';'.
>>
>> Would be something like this:
>> self connection executePrepared: 'getUserData' with: anUser userName.
>>
>> And another suggestion could be something like:
>> self connection prepare: 'getUserData' withSQL: 'SELECT data FROM mytable
>> WHERE name=$1'
>>
>> Any plan for this to be implemented or any hint to other PostgreSQL class
>> that already has it?
>>
>> David.
>>
>> P.S. I've written in my blog about this some time ago:
>> http://stormbyte.blogspot.com.es/2012/06/programming-with-database-using.ht…
>> if someone finds it useful.
>>
>>
>>
Jan. 10, 2015
[OT] Things people built with Smalltalk on Twitter: #thingsPeopleBuiltWithSmalltalk
by Torsten Bergmann
If you know something built in Smalltalk then twitter it:
https://twitter.com/hashtag/thingsPeopleBuiltWithSmalltalk?src=hash
Lets make 2015 the "Anybody knows about Smalltalk" year ;)
Jan. 10, 2015
Re: [Pharo-dev] PettitParser and Python
by kilon alios
there is also the python.ast model which gives the ability to parse the
python syntax ( I can use this model because of my communication bridge
with python) which I can use but if there is something already done with
petitparser I would love to take a look too.
The documentation of module is in this link together with the structure of
the python syntax
https://docs.python.org/2/library/ast.html
For now only python types concern me but I will need more as time passes.
On Sat, Jan 10, 2015 at 8:32 PM, Tudor Girba <tudor(a)tudorgirba.com> wrote:
> Hi,
>
> The challenge with Python parsing is dealing with whitespace properly.
>
> Jan Kurs was working on such a parser. I do not know how advanced it is.
> Jan?
>
> Cheers,
> Doru
>
>
>
> On Fri, Jan 9, 2015 at 7:30 PM, kilon alios <kilon.alios(a)gmail.com> wrote:
>
>> Hello guys I am learning PettitParser , I dowloaded the video tutorials
>> of Tudor Girba (very good tutorial Tudor , well done ) and other
>> documentation and I was wondering if a parser for Python already exists . I
>> hope I am not asking something obvious.
>>
>> For my project I want to parse python types , that will be given to me as
>> strings , to pharo objects. So that means parsing lists, dictionaries and
>> other usual candidates. For example a python list is like
>>
>> '[1, 3, 50, [45, 'hello', 0.2], 63]'
>>
>> etc
>>
>> Anyone already worked on such problem ?
>>
>>
>>
>
>
> --
> www.tudorgirba.com
>
> "Every thing has its own flow"
>
Jan. 10, 2015
Re: [Pharo-dev] Prepared Statements on PostgresV2?
by phil@highoctane.be
Le 10 janv. 2015 20:04, "Levente Uzonyi" <leves(a)elte.hu> a écrit :
>
> The V2 version of the protocol doesn't support prepared statements.
>
> When we were writing the PostgresV3 package, we planned to implement
prepared statements (via the extended query protocol), but later we decided
to not do it.
>
> Why?
>
> The protocol is way more complex than the simple query protocol (which is
used for regular queries), and involves more message sends, which can
increase the network latency.
>
> The security that prepared statements give is equivalent to proper
escaping. That can and should be implemented for standard queries too.
>
> The performance "improvement" what prepared statements give has two
sides. You save the cost of parsing and planning, but the query is executed
with the same plan, no matter what the arguments are. This can lead to
sub-optimal performance.
>
> With the extended query protocol you can execute one statement per query.
With the simple query protocol you can execute as many as you want. This
can increase the network latency significantly for prepared statements.
>
> In PostgreSQL, calling functions using the simple query protocol behave
similarly to prepared statements. You don't have to pass your SQL query to
the server all the time, just the ame of the function and its arguments
wrapped in a very simple SQL query.
> This way the plan is created when the function is executed the first time
(so it's even superior to prepared statements, because there's no
per-connection parsing and planning cost).
>
> Functions also let you to use any procedural language PostgreSQL
supports, which are currently SQL, PL/pgSQL, Pl/Tcl, PL/Perl, PL/Python.
Using the high level languages makes it easier to write complex queries, or
express complex logic, which wouldn't be possible with a single SQL query.
>
> So we decided to use functions instead. We added some image side tools to
make it easier to write, save, load and debug functions. Currently only the
PL/pgSQL language is supported by these tools, and there are some other
limitations, but we'll implement new features as time permits.
>
> What about performance?
>
> Using our toolchain, a single process can execute 2500-3000 queries
(using a modern CPU, and CogVM) per second, and an image can make about
5-6k queries per second overall.
> This includes the overhead of using the connection pool, and the
generation of the textual SQL query for each function call. Without these
it's possible to go above 10k/second.
>
About connection pool: how does one uses it?
Is there a doc somewhere ?
Phil
> Text or binary?
>
> We implemented the text-based protocol, but added the hooks for the
binary protocol too. Why?
> The binary protocol is undocumented (its documentation is the C source
code), and it's subject to change with each release.
> The text-based protocol is a bit better documented, and it should work
with all versions of PostgreSQL starting from 7.4.
>
> Levente
>
>
> On Sat, 10 Jan 2015, David Carlos Manuelda wrote:
>
>> I've installed and tested PostgresV2 under pharo with the following doits
>>
>> Gofer new
>> smalltalkhubUser: 'PharoExtras' project: 'PostgresV2';
>> configuration;
>> load.
>> (#ConfigurationOfPostgresV2 asClass project version: '2.4') load
>>
>> While it works good, it is missing a very important feature from both
>> security and performance point of view: The prepared statements.
>>
>> As a brief, prepared statements are parameterized SQL statements that are
>> loaded ONCE per connection instead of sending the whole query to DB every
>> time, and also, they are parameterized, so it completelly prevents SQL
>> injection, as the parameters are automatically 'detected' and scaped
and/or
>> handled accordingly without allowing in any case a parameter to alter the
>> SQL meaning, which can happen by using regular SQL queries made by string
>> concatenation.
>>
>> I browsed the class and did not find any prepare: method nor anything
>> similar.
>>
>> Also, you can give a name to a SQL sentence, which makes the code much
more
>> readable without messing too much logic with SQL commands and string
>> concatenation.
>>
>> An example is as follows: (supposing we have an instance variable
>> connection, already initialized and connected via PGConnection class)
>>
>> Instead of:
>>
>> self connection execute: 'SELECT data FROM mytable WHERE name=''', anUser
>> userName, ''';'.
>>
>> Would be something like this:
>> self connection executePrepared: 'getUserData' with: anUser userName.
>>
>> And another suggestion could be something like:
>> self connection prepare: 'getUserData' withSQL: 'SELECT data FROM mytable
>> WHERE name=$1'
>>
>> Any plan for this to be implemented or any hint to other PostgreSQL class
>> that already has it?
>>
>> David.
>>
>> P.S. I've written in my blog about this some time ago:
>>
http://stormbyte.blogspot.com.es/2012/06/programming-with-database-using.ht…
if someone finds it useful.
>>
>>
>>
>
Jan. 10, 2015