Re: [Pharo-users] Generate class hierarchy from JSON Schema
Todd, this is great, I didn't know that you are working on ActiveRecord. BTW, I have some minor additions to SQLServerPlatform (support for National Language Character Set) and OraclePlatform (Decimal data type) - is there a central git repository for Glorp so that I could PR to? Thanks and best wishes, Tomaz -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
The official is https://github.com/pharo-rdbms/glorp (The one I forked at pharo-db - a username I need to change as it is causing confusion). I've been mostly working with P3 and SQLite together. I've standardized the query results format for fetching field definitions so if you want to add support for another db, please note the format of the result set and structure yours appropriately. I would very much like all the databases to be hosted under pharo-rdbms and all of them to have glorp adaptors and the additional meta structure queries I'm adding to the GlorpPlatform classes to enable ActiveRecord. I feel like Pharo's RDBMS story has been lagging and the lack of ready to go database support keeps people from building apps on it. There are a zillion little business CRUD apps that could be knocked out with spec2 or Seaside. Please, dive in and make improvements.
On Jan 20, 2020, at 9:15 AM, eftomi <tomaz.turk@ef.uni-lj.si> wrote:
Todd, this is great, I didn't know that you are working on ActiveRecord.
BTW, I have some minor additions to SQLServerPlatform (support for National Language Character Set) and OraclePlatform (Decimal data type) - is there a central git repository for Glorp so that I could PR to?
Thanks and best wishes, Tomaz
-- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
Thanks, Todd - I'll prepare a PR for https://github.com/pharo-rdbms/glorp to update SQLServerPlatform and OraclePlatform. I already used the standardized query results format in PharoADO which works nicely with Glorp. You can find PharoADO here: https://github.com/eftomi/pharo-ado. Maybe it can be included into the pharo-rdbms collection if you feel that it's mature enough. PharoADO can be used standalone (as interface to ActiveX Data Objects) or with Glorp. However, there are some "high level" tests in Glorp package that are still red with Oracle and SQLServer, you can find the list on GitHub under "issues". In particular, I didn't have time yet to implement blobs - PharoADO uses PharoCOM (by Pablo) and firstly we have to settle the Variant SafeArray data type in that package. Now that we have stable Pharo 8.0 64 bit for Windows, PharoCOM should be checked for 64-bits, too. Anyway, I agree with you that the database support is one of the key success factors for a development platform. I'm focusing into this since I hope that one day I could teach the concepts of business solutions development fully in Pharo :-) Best wishes, Tomaz
The official is https://github.com/pharo-rdbms/glorp (The one I forked at pharo-db - a username I need to change as it is causing confusion).
I've been mostly working with P3 and SQLite together. I've standardized the query results format for fetching field definitions so if you want to add support for another db, please note the format of the result set and structure yours appropriately.
I would very much like all the databases to be hosted under pharo-rdbms and all of them to have glorp adaptors and the additional meta structure queries I'm adding to the GlorpPlatform classes to enable ActiveRecord. I feel like Pharo's RDBMS story has been lagging and the lack of ready to go database support keeps people from building apps on it.
There are a zillion little business CRUD apps that could be knocked out with spec2 or Seaside.
Please, dive in and make improvements.
That would be amazing! I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite. I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests. As for database introspection, I am relying on this new method and the result set format on DatabasePlatform. printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk -------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1 Then I can build a whole class model and descriptor system off the schema.
On Jan 22, 2020, at 9:55 AM, Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
Thanks, Todd - I'll prepare a PR for https://github.com/pharo-rdbms/glorp <https://github.com/pharo-rdbms/glorp> to update SQLServerPlatform and OraclePlatform.
I already used the standardized query results format in PharoADO which works nicely with Glorp. You can find PharoADO here: https://github.com/eftomi/pharo-ado <https://github.com/eftomi/pharo-ado>. Maybe it can be included into the pharo-rdbms collection if you feel that it's mature enough. PharoADO can be used standalone (as interface to ActiveX Data Objects) or with Glorp.
However, there are some "high level" tests in Glorp package that are still red with Oracle and SQLServer, you can find the list on GitHub under "issues". In particular, I didn't have time yet to implement blobs - PharoADO uses PharoCOM (by Pablo) and firstly we have to settle the Variant SafeArray data type in that package. Now that we have stable Pharo 8.0 64 bit for Windows, PharoCOM should be checked for 64-bits, too.
Anyway, I agree with you that the database support is one of the key success factors for a development platform. I'm focusing into this since I hope that one day I could teach the concepts of business solutions development fully in Pharo :-)
Best wishes, Tomaz
The official is https://github.com/pharo-rdbms/glorp (The one I forked at pharo-db - a username I need to change as it is causing confusion).
I've been mostly working with P3 and SQLite together. I've standardized the query results format for fetching field definitions so if you want to add support for another db, please note the format of the result set and structure yours appropriately.
I would very much like all the databases to be hosted under pharo-rdbms and all of them to have glorp adaptors and the additional meta structure queries I'm adding to the GlorpPlatform classes to enable ActiveRecord. I feel like Pharo's RDBMS story has been lagging and the lack of ready to go database support keeps people from building apps on it.
There are a zillion little business CRUD apps that could be knocked out with spec2 or Seaside.
Please, dive in and make improvements.
That would be amazing!
I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite.
I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests.
As for database introspection, I am relying on this new method and the result set format on DatabasePlatform.
printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk -------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1
This is great news about ActiveRecord! ADO is, unfortunately, a Windows library, so PharoADO is limited to Windows. Maybe as a next step we should focus into Garage and the addition of SQL Server and Oracle support. If I only had time ... Stéphane, thanks for the support. Best wishes, Tomaz
Hi, To feed a Pharo app I need to read from an old MDB file, via ODBC. So I tested PharoADO and, after adding a missing Variant Type [1], I got my query working (at least read only). However the time of ADOClient to return a query with 10K rows of 5 columns each is extremely slow (+65000ms) compared, e.g., with Dolphin Smalltalk via ODBC (62ms). The time of the execute call to return is instantaneous, so I think the issue might be in the creation of the result set. Is this a known issue? Should I check something else? Thanks in advance, [1] https://github.com/tesonep/pharo-com/pull/10 Esteban A. Maringolo On Fri, Jan 24, 2020 at 5:59 AM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
That would be amazing!
I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite.
I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests.
As for database introspection, I am relying on this new method and the result set format on DatabasePlatform.
printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk
-------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1
This is great news about ActiveRecord!
ADO is, unfortunately, a Windows library, so PharoADO is limited to Windows. Maybe as a next step we should focus into Garage and the addition of SQL Server and Oracle support. If I only had time ...
Stéphane, thanks for the support.
Best wishes, Tomaz
As an additional reference, I attach the profile tally for the query I'm mentioning. The culprit seems to be the calculation of the ADOField properties that might be calculated on every call. Regards, Esteban A. Maringolo On Wed, Mar 11, 2020 at 2:13 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
To feed a Pharo app I need to read from an old MDB file, via ODBC.
So I tested PharoADO and, after adding a missing Variant Type [1], I got my query working (at least read only).
However the time of ADOClient to return a query with 10K rows of 5 columns each is extremely slow (+65000ms) compared, e.g., with Dolphin Smalltalk via ODBC (62ms).
The time of the execute call to return is instantaneous, so I think the issue might be in the creation of the result set.
Is this a known issue?
Should I check something else?
Thanks in advance,
[1] https://github.com/tesonep/pharo-com/pull/10 Esteban A. Maringolo
On Fri, Jan 24, 2020 at 5:59 AM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
That would be amazing!
I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite.
I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests.
As for database introspection, I am relying on this new method and the result set format on DatabasePlatform.
printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk
-------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1
This is great news about ActiveRecord!
ADO is, unfortunately, a Windows library, so PharoADO is limited to Windows. Maybe as a next step we should focus into Garage and the addition of SQL Server and Oracle support. If I only had time ...
Stéphane, thanks for the support.
Best wishes, Tomaz
Hi Esteban, Thanks for reporting this issue. I haven't made any "stress" tests yet for ADOClient. Could you please try to use ADORecordset as described in the readme? There you have a direct access to each record at a time, and then you can create your own "loop" to process the query result. In this way you can avoid the loop in ADOClient>>query: method. Best wishes, Tomaz ------ Original Message ------ From: "Esteban Maringolo" <emaringolo@gmail.com> To: "Tomaž Turk" <tomaz.turk@ef.uni-lj.si>; "Any question about pharo is welcome" <pharo-users@lists.pharo.org> Sent: 11. 03. 2020 18:25:51 Subject: Re: [Pharo-users] Generate class hierarchy from JSON Schema
As an additional reference, I attach the profile tally for the query I'm mentioning.
The culprit seems to be the calculation of the ADOField properties that might be calculated on every call.
Regards,
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 2:13 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
To feed a Pharo app I need to read from an old MDB file, via ODBC.
So I tested PharoADO and, after adding a missing Variant Type [1], I got my query working (at least read only).
However the time of ADOClient to return a query with 10K rows of 5 columns each is extremely slow (+65000ms) compared, e.g., with Dolphin Smalltalk via ODBC (62ms).
The time of the execute call to return is instantaneous, so I think the issue might be in the creation of the result set.
Is this a known issue?
Should I check something else?
Thanks in advance,
[1] https://github.com/tesonep/pharo-com/pull/10 Esteban A. Maringolo
On Fri, Jan 24, 2020 at 5:59 AM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
That would be amazing!
I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite.
I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests.
As for database introspection, I am relying on this new method and the result set format on DatabasePlatform.
printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk -------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1
This is great news about ActiveRecord!
ADO is, unfortunately, a Windows library, so PharoADO is limited to Windows. Maybe as a next step we should focus into Garage and the addition of SQL Server and Oracle support. If I only had time ...
Stéphane, thanks for the support.
Best wishes, Tomaz
The profiling using the recordset was even worst. I changed the iteration [1] to add everything to a collection, but most of the time is spent in #calculateMethods and #calculateProperties. Which is performed everytime for each ADOField. I wouldn't consider this a stress test, but maybe the methods and properties should be cached, or a different class of dispatcher should be used where the methods and properties are defined ahead of time instead of being discovered when first invoked. Regards, [1] "Modified code" rst := ADORecordset createInstance . rst open: 'SELECT * FROM PLAYER' activeConnection: conn cursorType: 2 lockType: 3 options: -1. results := OrderedCollection new. [rst eof] whileFalse: [ | row | row := Array new: rst fields count. 1 to: row size do: [ :idx | row at: idx put: (rst fields item: idx) value ]. results add: row. rst moveNext ]. Esteban A. Maringolo On Wed, Mar 11, 2020 at 3:02 PM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
Hi Esteban,
Thanks for reporting this issue. I haven't made any "stress" tests yet for ADOClient. Could you please try to use ADORecordset as described in the readme? There you have a direct access to each record at a time, and then you can create your own "loop" to process the query result. In this way you can avoid the loop in ADOClient>>query: method.
Best wishes, Tomaz
------ Original Message ------ From: "Esteban Maringolo" <emaringolo@gmail.com> To: "Tomaž Turk" <tomaz.turk@ef.uni-lj.si>; "Any question about pharo is welcome" <pharo-users@lists.pharo.org> Sent: 11. 03. 2020 18:25:51 Subject: Re: [Pharo-users] Generate class hierarchy from JSON Schema
As an additional reference, I attach the profile tally for the query I'm mentioning.
The culprit seems to be the calculation of the ADOField properties that might be calculated on every call.
Regards,
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 2:13 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
To feed a Pharo app I need to read from an old MDB file, via ODBC.
So I tested PharoADO and, after adding a missing Variant Type [1], I got my query working (at least read only).
However the time of ADOClient to return a query with 10K rows of 5 columns each is extremely slow (+65000ms) compared, e.g., with Dolphin Smalltalk via ODBC (62ms).
The time of the execute call to return is instantaneous, so I think the issue might be in the creation of the result set.
Is this a known issue?
Should I check something else?
Thanks in advance,
[1] https://github.com/tesonep/pharo-com/pull/10 Esteban A. Maringolo
On Fri, Jan 24, 2020 at 5:59 AM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
That would be amazing!
I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite.
I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests.
As for database introspection, I am relying on this new method and the result set format on DatabasePlatform.
printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk
-------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1
This is great news about ActiveRecord!
ADO is, unfortunately, a Windows library, so PharoADO is limited to Windows. Maybe as a next step we should focus into Garage and the addition of SQL Server and Oracle support. If I only had time ...
Stéphane, thanks for the support.
Best wishes, Tomaz
I did a small change in ADORecordset to have the fields as an instVar and also in ADOFields to have each ADOField in an `items` instVar. It is still slow compared to an ODBC API, but I got a 20x speedup (~1000ms), that is spent mostly on the actual call to the dispatcher to fetch each cell value. I look forward for an ODBC API, but at least to read any other data source, this seems to work pretty well. I already sent a pull-request. Esteban A. Maringolo On Wed, Mar 11, 2020 at 3:31 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
The profiling using the recordset was even worst. I changed the iteration [1] to add everything to a collection, but most of the time is spent in #calculateMethods and #calculateProperties. Which is performed everytime for each ADOField.
I wouldn't consider this a stress test, but maybe the methods and properties should be cached, or a different class of dispatcher should be used where the methods and properties are defined ahead of time instead of being discovered when first invoked.
Regards,
[1] "Modified code" rst := ADORecordset createInstance . rst open: 'SELECT * FROM PLAYER' activeConnection: conn cursorType: 2 lockType: 3 options: -1. results := OrderedCollection new. [rst eof] whileFalse: [ | row | row := Array new: rst fields count. 1 to: row size do: [ :idx | row at: idx put: (rst fields item: idx) value ]. results add: row. rst moveNext ].
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 3:02 PM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
Hi Esteban,
Thanks for reporting this issue. I haven't made any "stress" tests yet for ADOClient. Could you please try to use ADORecordset as described in the readme? There you have a direct access to each record at a time, and then you can create your own "loop" to process the query result. In this way you can avoid the loop in ADOClient>>query: method.
Best wishes, Tomaz
------ Original Message ------ From: "Esteban Maringolo" <emaringolo@gmail.com> To: "Tomaž Turk" <tomaz.turk@ef.uni-lj.si>; "Any question about pharo is welcome" <pharo-users@lists.pharo.org> Sent: 11. 03. 2020 18:25:51 Subject: Re: [Pharo-users] Generate class hierarchy from JSON Schema
As an additional reference, I attach the profile tally for the query I'm mentioning.
The culprit seems to be the calculation of the ADOField properties that might be calculated on every call.
Regards,
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 2:13 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
To feed a Pharo app I need to read from an old MDB file, via ODBC.
So I tested PharoADO and, after adding a missing Variant Type [1], I got my query working (at least read only).
However the time of ADOClient to return a query with 10K rows of 5 columns each is extremely slow (+65000ms) compared, e.g., with Dolphin Smalltalk via ODBC (62ms).
The time of the execute call to return is instantaneous, so I think the issue might be in the creation of the result set.
Is this a known issue?
Should I check something else?
Thanks in advance,
[1] https://github.com/tesonep/pharo-com/pull/10 Esteban A. Maringolo
On Fri, Jan 24, 2020 at 5:59 AM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
That would be amazing!
I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite.
I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests.
As for database introspection, I am relying on this new method and the result set format on DatabasePlatform.
printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk
-------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1
This is great news about ActiveRecord!
ADO is, unfortunately, a Windows library, so PharoADO is limited to Windows. Maybe as a next step we should focus into Garage and the addition of SQL Server and Oracle support. If I only had time ...
Stéphane, thanks for the support.
Best wishes, Tomaz
Follow up. Seems that playing with these COM/ADO objects in the playground messes with the VM; so I'm getting weird image crashes, where the image closes without any notice. Do you experience the same? Esteban A. Maringolo On Wed, Mar 11, 2020 at 4:11 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
I did a small change in ADORecordset to have the fields as an instVar and also in ADOFields to have each ADOField in an `items` instVar. It is still slow compared to an ODBC API, but I got a 20x speedup (~1000ms), that is spent mostly on the actual call to the dispatcher to fetch each cell value.
I look forward for an ODBC API, but at least to read any other data source, this seems to work pretty well.
I already sent a pull-request.
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 3:31 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
The profiling using the recordset was even worst. I changed the iteration [1] to add everything to a collection, but most of the time is spent in #calculateMethods and #calculateProperties. Which is performed everytime for each ADOField.
I wouldn't consider this a stress test, but maybe the methods and properties should be cached, or a different class of dispatcher should be used where the methods and properties are defined ahead of time instead of being discovered when first invoked.
Regards,
[1] "Modified code" rst := ADORecordset createInstance . rst open: 'SELECT * FROM PLAYER' activeConnection: conn cursorType: 2 lockType: 3 options: -1. results := OrderedCollection new. [rst eof] whileFalse: [ | row | row := Array new: rst fields count. 1 to: row size do: [ :idx | row at: idx put: (rst fields item: idx) value ]. results add: row. rst moveNext ].
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 3:02 PM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
Hi Esteban,
Thanks for reporting this issue. I haven't made any "stress" tests yet for ADOClient. Could you please try to use ADORecordset as described in the readme? There you have a direct access to each record at a time, and then you can create your own "loop" to process the query result. In this way you can avoid the loop in ADOClient>>query: method.
Best wishes, Tomaz
------ Original Message ------ From: "Esteban Maringolo" <emaringolo@gmail.com> To: "Tomaž Turk" <tomaz.turk@ef.uni-lj.si>; "Any question about pharo is welcome" <pharo-users@lists.pharo.org> Sent: 11. 03. 2020 18:25:51 Subject: Re: [Pharo-users] Generate class hierarchy from JSON Schema
As an additional reference, I attach the profile tally for the query I'm mentioning.
The culprit seems to be the calculation of the ADOField properties that might be calculated on every call.
Regards,
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 2:13 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
To feed a Pharo app I need to read from an old MDB file, via ODBC.
So I tested PharoADO and, after adding a missing Variant Type [1], I got my query working (at least read only).
However the time of ADOClient to return a query with 10K rows of 5 columns each is extremely slow (+65000ms) compared, e.g., with Dolphin Smalltalk via ODBC (62ms).
The time of the execute call to return is instantaneous, so I think the issue might be in the creation of the result set.
Is this a known issue?
Should I check something else?
Thanks in advance,
[1] https://github.com/tesonep/pharo-com/pull/10 Esteban A. Maringolo
On Fri, Jan 24, 2020 at 5:59 AM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
That would be amazing!
I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite.
I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests.
As for database introspection, I am relying on this new method and the result set format on DatabasePlatform.
printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk
-------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1
This is great news about ActiveRecord!
ADO is, unfortunately, a Windows library, so PharoADO is limited to Windows. Maybe as a next step we should focus into Garage and the addition of SQL Server and Oracle support. If I only had time ...
Stéphane, thanks for the support.
Best wishes, Tomaz
Thanks for the PR! Maybe something can be done also on the PharoCOM level within calculateMethods and calculateProperties. I didn't experience any crashes, although I developed PharoADO on 32bits in Pharo 7 and Pablo upgraded PharoCOM to 64bit after Pharo 8 was released. Besides, I haven't managed yet to re-check if all the COM instances are properly released after usage. Best wishes Tomaz ------ Original Message ------ From: "Esteban Maringolo" <emaringolo@gmail.com> To: "Tomaž Turk" <tomaz.turk@ef.uni-lj.si> Cc: "Any question about pharo is welcome" <pharo-users@lists.pharo.org> Sent: 11. 03. 2020 20:35:10 Subject: Re: Re[2]: [Pharo-users] Generate class hierarchy from JSON Schema
Follow up.
Seems that playing with these COM/ADO objects in the playground messes with the VM; so I'm getting weird image crashes, where the image closes without any notice.
Do you experience the same?
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 4:11 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
I did a small change in ADORecordset to have the fields as an instVar and also in ADOFields to have each ADOField in an `items` instVar. It is still slow compared to an ODBC API, but I got a 20x speedup (~1000ms), that is spent mostly on the actual call to the dispatcher to fetch each cell value.
I look forward for an ODBC API, but at least to read any other data source, this seems to work pretty well.
I already sent a pull-request.
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 3:31 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
The profiling using the recordset was even worst. I changed the iteration [1] to add everything to a collection, but most of the time is spent in #calculateMethods and #calculateProperties. Which is performed everytime for each ADOField.
I wouldn't consider this a stress test, but maybe the methods and properties should be cached, or a different class of dispatcher should be used where the methods and properties are defined ahead of time instead of being discovered when first invoked.
Regards,
[1] "Modified code" rst := ADORecordset createInstance . rst open: 'SELECT * FROM PLAYER' activeConnection: conn cursorType: 2 lockType: 3 options: -1. results := OrderedCollection new. [rst eof] whileFalse: [ | row | row := Array new: rst fields count. 1 to: row size do: [ :idx | row at: idx put: (rst fields item: idx) value ]. results add: row. rst moveNext ].
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 3:02 PM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
Hi Esteban,
Thanks for reporting this issue. I haven't made any "stress" tests yet for ADOClient. Could you please try to use ADORecordset as described in the readme? There you have a direct access to each record at a time, and then you can create your own "loop" to process the query result. In this way you can avoid the loop in ADOClient>>query: method.
Best wishes, Tomaz
------ Original Message ------ From: "Esteban Maringolo" <emaringolo@gmail.com> To: "Tomaž Turk" <tomaz.turk@ef.uni-lj.si>; "Any question about pharo is welcome" <pharo-users@lists.pharo.org> Sent: 11. 03. 2020 18:25:51 Subject: Re: [Pharo-users] Generate class hierarchy from JSON Schema
As an additional reference, I attach the profile tally for the query I'm mentioning.
The culprit seems to be the calculation of the ADOField properties that might be calculated on every call.
Regards,
Esteban A. Maringolo
On Wed, Mar 11, 2020 at 2:13 PM Esteban Maringolo <emaringolo@gmail.com> wrote:
Hi,
To feed a Pharo app I need to read from an old MDB file, via ODBC.
So I tested PharoADO and, after adding a missing Variant Type [1], I got my query working (at least read only).
However the time of ADOClient to return a query with 10K rows of 5 columns each is extremely slow (+65000ms) compared, e.g., with Dolphin Smalltalk via ODBC (62ms).
The time of the execute call to return is instantaneous, so I think the issue might be in the creation of the result set.
Is this a known issue?
Should I check something else?
Thanks in advance,
[1] https://github.com/tesonep/pharo-com/pull/10 Esteban A. Maringolo
On Fri, Jan 24, 2020 at 5:59 AM Tomaž Turk <tomaz.turk@ef.uni-lj.si> wrote:
That would be amazing!
I'm a Mac/Unix guy so I don't have access to the other platforms (I suppose I could fire up an AWS Oracle). I can do mysql/mariadb, posgresql, and sqlite.
I'm pretty close to pushing my ActiveRecord extensions. I just need to get many to many with link tables done and it is good to go. I spent a few days "porting" the ruby inflector and its tests.
As for database introspection, I am relying on this new method and the result set format on DatabasePlatform.
printSqlStatementToListColumnsInTable: aDatabaseTable inSchema: schemaString on: aStream " Format: name | type | length | nullable | default_value | pk -------------------------+-------------------+--------+----------+-----------------------+---- id | character varying | 255 | 0 | ''::character varying | 1
This is great news about ActiveRecord!
ADO is, unfortunately, a Windows library, so PharoADO is limited to Windows. Maybe as a next step we should focus into Garage and the addition of SQL Server and Oracle support. If I only had time ...
Stéphane, thanks for the support.
Best wishes, Tomaz
Esteban, if time permits, could you please describe what situations lead to crashes, and with which images? Thanks, Tomaz -- Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
participants (4)
-
eftomi -
Esteban Maringolo -
Todd Blanchard -
Tomaž Turk