Follow up. Seems that playing with these COM/ADO objects in the playground messes with the VM; so I'm getting weird image crashes, where the image closes without any notice. Do you experience the same? Esteban A. Maringolo On Wed, Mar 11, 2020 at 4:11 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
I did a small change in ADORecordset to have the fields as an instVar and also in ADOFields to have each ADOField in an `items` instVar. It is still slow compared to an ODBC API, but I got a 20x speedup (~1000ms), that is spent mostly on the actual call to the dispatcher to fetch each cell value.
I look forward for an ODBC API, but at least to read any other data source, this seems to work pretty well.
I already sent a pull-request.
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 3:31 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
The profiling using the recordset was even worst. I changed the iteration [1] to add everything to a collection, but most of the time is spent in #calculateMethods and #calculateProperties. Which is performed everytime for each ADOField.
I wouldn't consider this a stress test, but maybe the methods and properties should be cached, or a different class of dispatcher should be used where the methods and properties are defined ahead of time instead of being discovered when first invoked.
Regards,
[1] "Modified code" rst := ADORecordset createInstance . rst open: 'SELECT * FROM PLAYER' activeConnection: conn cursorType: 2 lockType: 3 options: -1. results := OrderedCollection new. [rst eof] whileFalse: [ | row | row := Array new: rst fields count. 1 to: row size do: [ :idx | row at: idx put: (rst fields item: idx) value ]. results add: row. rst moveNext ].
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 3:02 PM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
Hi Esteban,
Thanks for reporting this issue. I haven't made any "stress" tests yet for ADOClient. Could you please try to use ADORecordset as described in the readme? There you have a direct access to each record at a time, and then you can create your own "loop" to process the query result. In this way you can avoid the loop in ADOClient>>query: method.
Best wishes, Tomaz
------ Original Message ------ From: "Esteban Maringolo" <emaringolo@gmail.com> To: "Tomaž Turk" <tomaz.turk@ef.uni-lj.si>; "Any question about pharo is welcome" <pharo-users@lists.pharo.org> Sent: 11. 03. 2020 18:25:51 Subject: Re: [Pharo-users] Generate class hierarchy from JSON Schema
As an additional reference, I attach the profile tally for the query I'm mentioning.
The culprit seems to be the calculation of the ADOField properties that might be calculated on every call.
Regards,
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 2:13 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
To feed a Pharo app I need to read from an old MDB file, via ODBC.
So I tested PharoADO and, after adding a missing Variant Type [1], I got my query working (at least read only).
However the time of ADOClient to return a query with 10K rows of 5 columns each is extremely slow (+65000ms) compared, e.g., with Dolphin Smalltalk via ODBC (62ms).
The time of the execute call to return is instantaneous, so I think the issue might be in the creation of the result set.
Is this a known issue?
Should I check something else?
Thanks in advance,
[1] https://github.com/tesonep/pharo-com/pull/10 Esteban A. Maringolo
On Fri, Jan 24, 2020 at 5:59 AM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
That would be amazing!
I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite.
I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests.
As for database introspection, I am relying on this new method and the result set format on DatabasePlatform.
printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk
-------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1
This is great news about ActiveRecord!
ADO is, unfortunately, a Windows library, so PharoADO is limited to Windows. Maybe as a next step we should focus into Garage and the addition of SQL Server and Oracle support. If I only had time ...
Stéphane, thanks for the support.
Best wishes, Tomaz