pharo-users@lists.pharo.org

Any question about pharo is welcome

View all threads

Question about Spec2 and SpTablePresenter

MO
Mark O'Donoghue
Sun, Jul 4, 2021 9:26 AM

Howdy all

I’ve got stuck trying to manage Tables – over 12 hours now and I’m out of ideas!. ☹

Any observations / suggestions  are most welcome…

I’ve been loading small external files of transactions using NeoCSV into Spec2 tables.

I am trying to use Fuel to persist the table contents so that my application will re-load the working state from where I finished in the last session.

(I’ve opted for Fuel as a simple alternative to having to do the whole object relational mapping thing.)

The idea is that transactions (and potentially some manual adjustments) will be processed over time.

(This is preferable to having to reload all files from the beginning evert time I run the application…)

The Spec2 tables have been working well until I tried to persist them.

I can’t seem to fully re-load them to a previously saved state.

For example - I can restore the essential contents of my table  in most circumstances using:

restoreObjects

            “filePresenter1 is a SpTablePresenter”



            | objects savedEntry |

            objects := CpPersist restoreObjectsFromFileNamed: 'E:\Me\zzzST-Test\demo.fuel'.

            

            recentFileList                     := objects at: 1.

            currFileFilter                       := objects at: 2.

            savedEntry                          := objects at: 3.

                                            

            self updateFilterButton: currFileFilter.

            

            filteredFileList := self filterFilesUsing: currFileFilter. 

            

            filePresenter1 items: filteredFileList.

            

            savedEntry 

ifNotNil: [filePresenter1 selectItem: savedEntry ].

However, if any of the table columns are re-sorted , the re-load operation gets confused and I can’t get the saved a saved selected item to become selected again.

(It seems to be confusing the index numbers of the sorted and unsorted lists – even when I match by contents rather than index.)

(I also created an equality test to ensure equivalent entries are recognised by the #= operation in the list of the underlying model ).

This all works fine - unless I sort a column!

Since this approach was going to be used on several screens I’d really like to find a solution.

Cheers

Mark

Perth, Western Australia

Howdy all I’ve got stuck trying to manage Tables – over 12 hours now and I’m out of ideas!. ☹ Any observations / suggestions are most welcome… I’ve been loading small external files of transactions using NeoCSV into Spec2 tables. I am trying to use Fuel to persist the table contents so that my application will re-load the working state from where I finished in the last session. (I’ve opted for Fuel as a simple alternative to having to do the whole object relational mapping thing.) The idea is that transactions (and potentially some manual adjustments) will be processed over time. (This is preferable to having to reload all files from the beginning evert time I run the application…) The Spec2 tables have been working well until I tried to persist them. I can’t seem to fully re-load them to a previously saved state. For example - I can restore the essential contents of my table in most circumstances using: restoreObjects “filePresenter1 is a SpTablePresenter” | objects savedEntry | objects := CpPersist restoreObjectsFromFileNamed: 'E:\Me\zzzST-Test\demo.fuel'. recentFileList := objects at: 1. currFileFilter := objects at: 2. savedEntry := objects at: 3. self updateFilterButton: currFileFilter. filteredFileList := self filterFilesUsing: currFileFilter. filePresenter1 items: filteredFileList. savedEntry ifNotNil: [filePresenter1 selectItem: savedEntry ]. However, if any of the table columns are re-sorted , the re-load operation gets confused and I can’t get the saved a saved selected item to become selected again. (It seems to be confusing the index numbers of the sorted and unsorted lists – even when I match by contents rather than index.) (I also created an equality test to ensure equivalent entries are recognised by the #= operation in the list of the underlying model ). This all works fine - unless I sort a column! Since this approach was going to be used on several screens I’d really like to find a solution. Cheers Mark Perth, Western Australia
KO
Kasper Osterbye
Mon, Jul 5, 2021 4:05 AM

Cheers,

From what code you provide I can not spot any error. I do not use your persistence strategy myself, so I first thought that it might be an issue related to that - for instance something with the identity of objects from the recentFileList and the savedEntry messing up. But that would not explain why it works unsorted.

I have a piece of code in my application which is doing pretty much the same:
selected := table selection selectedItem.
table items: self sheeps.
(selected notNil and: [ table items includes: selected ])
ifTrue: [
table selection selectItem: selected.
self announceNewSelection: selected ].
For some reason I work on the selection rather than the table, but it does not seem to make any difference when I just tried.

Is there a way in which you mock the “objects := ….” to something which do not use persistence to rule out that is the persistence causing the problems (or verify it). A method which returns a triple of known objects perhaps.

Sometimes the problem lies in the instantiation of the table, so perhaps that might help to see.

I am on EU time and on discord this morning, you are more than welcome to try to grab me. I have been using SpTablePresenter a lot (and is often baffled as well).

Best,

Kasper

On 4 Jul 2021, at 11.26, Mark O'Donoghue mark.odonoghue.2010@gmail.com wrote:

Howdy all

I’ve got stuck trying to manage Tables – over 12 hours now and I’m out of ideas!. ☹

Any observations / suggestions  are most welcome…

I’ve been loading small external files of transactions using NeoCSV into Spec2 tables.
I am trying to use Fuel to persist the table contents so that my application will re-load the working state from where I finished in the last session.
(I’ve opted for Fuel as a simple alternative to having to do the whole object relational mapping thing.)

The idea is that transactions (and potentially some manual adjustments) will be processed over time.
(This is preferable to having to reload all files from the beginning evert time I run the application…)

The Spec2 tables have been working well until I tried to persist them.
I can’t seem to fully re-load them to a previously saved state.

For example - I can restore the essential contents of my table  in most circumstances using:

restoreObjects
“filePresenter1 is a SpTablePresenter”

             | objects savedEntry |
             objects := CpPersist restoreObjectsFromFileNamed: 'E:\Me\zzzST-Test\demo.fuel'.
             
             recentFileList                     := objects at: 1.
             currFileFilter                       := objects at: 2.
             savedEntry                          := objects at: 3.
                                             
             self updateFilterButton: currFileFilter.
             
             filteredFileList := self filterFilesUsing: currFileFilter. 
             
             filePresenter1 items: filteredFileList.
             
             savedEntry 

ifNotNil: [filePresenter1 selectItem: savedEntry ].

However, if any of the table columns are re-sorted , the re-load operation gets confused and I can’t get the saved a saved selected item to become selected again.

(It seems to be confusing the index numbers of the sorted and unsorted lists – even when I match by contents rather than index.)
(I also created an equality test to ensure equivalent entries are recognised by the #= operation in the list of the underlying model ).

This all works fine - unless I sort a column!

Since this approach was going to be used on several screens I’d really like to find a solution.

Cheers
Mark
Perth, Western Australia

Cheers, From what code you provide I can not spot any error. I do not use your persistence strategy myself, so I first thought that it might be an issue related to that - for instance something with the identity of objects from the recentFileList and the savedEntry messing up. But that would not explain why it works unsorted. I have a piece of code in my application which is doing pretty much the same: selected := table selection selectedItem. table items: self sheeps. (selected notNil and: [ table items includes: selected ]) ifTrue: [ table selection selectItem: selected. self announceNewSelection: selected ]. For some reason I work on the selection rather than the table, but it does not seem to make any difference when I just tried. Is there a way in which you mock the “objects := ….” to something which do not use persistence to rule out that is the persistence causing the problems (or verify it). A method which returns a triple of known objects perhaps. Sometimes the problem lies in the instantiation of the table, so perhaps that might help to see. I am on EU time and on discord this morning, you are more than welcome to try to grab me. I have been using SpTablePresenter a lot (and is often baffled as well). Best, Kasper > On 4 Jul 2021, at 11.26, Mark O'Donoghue <mark.odonoghue.2010@gmail.com> wrote: > > Howdy all > > I’ve got stuck trying to manage Tables – over 12 hours now and I’m out of ideas!. ☹ > > Any observations / suggestions are most welcome… > > I’ve been loading small external files of transactions using NeoCSV into Spec2 tables. > I am trying to use Fuel to persist the table contents so that my application will re-load the working state from where I finished in the last session. > (I’ve opted for Fuel as a simple alternative to having to do the whole object relational mapping thing.) > > The idea is that transactions (and potentially some manual adjustments) will be processed over time. > (This is preferable to having to reload all files from the beginning evert time I run the application…) > > The Spec2 tables have been working well until I tried to persist them. > I can’t seem to fully re-load them to a previously saved state. > > For example - I can restore the essential contents of my table in most circumstances using: > > restoreObjects > “filePresenter1 is a SpTablePresenter” > > | objects savedEntry | > objects := CpPersist restoreObjectsFromFileNamed: 'E:\Me\zzzST-Test\demo.fuel'. > > recentFileList := objects at: 1. > currFileFilter := objects at: 2. > savedEntry := objects at: 3. > > self updateFilterButton: currFileFilter. > > filteredFileList := self filterFilesUsing: currFileFilter. > > filePresenter1 items: filteredFileList. > > savedEntry > ifNotNil: [filePresenter1 selectItem: savedEntry ]. > > > However, if any of the table columns are re-sorted , the re-load operation gets confused and I can’t get the saved a saved selected item to become selected again. > > (It seems to be confusing the index numbers of the sorted and unsorted lists – even when I match by contents rather than index.) > (I also created an equality test to ensure equivalent entries are recognised by the #= operation in the list of the underlying model ). > > This all works fine - unless I sort a column! > > Since this approach was going to be used on several screens I’d really like to find a solution. > > Cheers > Mark > Perth, Western Australia
MO
Mark O'Donoghue
Mon, Jul 5, 2021 7:30 AM

Thanks Kasper

Thanks for your thoughts.

(BTW - I have been using your sheep app as an informal reference – many thanks)

I’m satisfied that the restored objects are fine. Nothing strange arising from persisting and restoring when I inspect the restored objects.

Because I have my own equality test I don’t think there is anything strange happening with comparing object identities.

(I’ve verified that the underlying SpTablePresenter model collection is using my equality test to do the comparisons.)

= anObject

            ^ (self file name) = (anObject file name).

I have now stopped digging after a few days lost on this issue. Fortunately on my own time!

I had to go into the Morphic code that supports the Spec2 table and columns, but couldn’t find the answers.

It seems the built in column sort functionality happens down deep in Morphic and I couldn’t influence it with Spec methods (e.g. spTablePresenter>>sortingBlock: ).

The sort handles on the column headers seem to be something that the table presenter doesn’t control or know about.

My conclusion is that the TablePresenter items (the model) does not actually know what the visual representation looks like once the table has been re-sorted.

That sounds desirable in terms of coupling; but the model doesn’t actually know what item is selected, which is a problem…!

So I think there must be some kind of a disconnect between the model state and the visual representation.

Any connection is actually implicit, but is only valid when the table is in its original condition.

In my tests, using selectItem: newItem only works if the table has not been resorted.

Same for selectIndex: savedIndex too…

So my guess is that the select functionality is implemented at the visual level and hasn’t been fully represented at the model level.

The model level functionality for selections isn’t really complete – it is just a pass through to the visual level…

And of course it works most of the time, so the problem goes unnoticed…

In the end, my (brute force) work around was to:

  1. reload the table presenter items with the persisted entries ( filePresenter1 items: filteredFileLIst )
  2. scan the table items (from 1 to items size) - selecting each one in turn and then noting what the newly selected item was
  3. stopping when the newly selected item matches the entry that was persisted

The result is that the correct entry is selected and the table has the correct items. 😊

The only thing missing is that the table sorting state is not restored to its previous state (which is acceptable for now)

This workaround is inelegant and may be unusable when/if the main tables have many more entries (which is likely).

[ Another possibility was to persist the entire table presenter (which is very slow) and reconstruct my active layout using the restored table presenter.

I’m not sure if this is a good idea anyway? My attempts showed it was fiddly, and slow, and it really didn’t appeal to me. ]

Cheers, and thanks again

Mark

From: Kasper Osterbye kasper.osterbye@gmail.com
Sent: Monday, 5 July 2021 12:06 PM
To: Any question about pharo is welcome pharo-users@lists.pharo.org
Subject: [Pharo-users] Re: Question about Spec2 and SpTablePresenter

Cheers,

From what code you provide I can not spot any error. I do not use your persistence strategy myself, so I first thought that it might be an issue related to that - for instance something with the identity of objects from the recentFileList and the savedEntry messing up. But that would not explain why it works unsorted.

I have a piece of code in my application which is doing pretty much the same:

            selected := table selection selectedItem.

            table items: self sheeps.

            (selected notNil and: [ table items includes: selected ]) 

                            ifTrue: [ 

                                            table selection selectItem: selected.

                                            self announceNewSelection: selected ].

For some reason I work on the selection rather than the table, but it does not seem to make any difference when I just tried.

Is there a way in which you mock the “objects := ….” to something which do not use persistence to rule out that is the persistence causing the problems (or verify it). A method which returns a triple of known objects perhaps.

Sometimes the problem lies in the instantiation of the table, so perhaps that might help to see.

I am on EU time and on discord this morning, you are more than welcome to try to grab me. I have been using SpTablePresenter a lot (and is often baffled as well).

Best,

Kasper

On 4 Jul 2021, at 11.26, Mark O'Donoghue <mark.odonoghue.2010@gmail.com mailto:mark.odonoghue.2010@gmail.com > wrote:

Howdy all

I’ve got stuck trying to manage Tables – over 12 hours now and I’m out of ideas!. ☹

Any observations / suggestions  are most welcome…

I’ve been loading small external files of transactions using NeoCSV into Spec2 tables.

I am trying to use Fuel to persist the table contents so that my application will re-load the working state from where I finished in the last session.

(I’ve opted for Fuel as a simple alternative to having to do the whole object relational mapping thing.)

The idea is that transactions (and potentially some manual adjustments) will be processed over time.

(This is preferable to having to reload all files from the beginning evert time I run the application…)

The Spec2 tables have been working well until I tried to persist them.

I can’t seem to fully re-load them to a previously saved state.

For example - I can restore the essential contents of my table  in most circumstances using:

restoreObjects

            “filePresenter1 is a SpTablePresenter”



            | objects savedEntry |

            objects := CpPersist restoreObjectsFromFileNamed: 'E:\Me\zzzST-Test\demo.fuel'.

            

            recentFileList                     := objects at: 1.

            currFileFilter                       := objects at: 2.

            savedEntry                          := objects at: 3.

                                            

            self updateFilterButton: currFileFilter.

            

            filteredFileList := self filterFilesUsing: currFileFilter. 

            

            filePresenter1 items: filteredFileList.

            

            savedEntry 

ifNotNil: [filePresenter1 selectItem: savedEntry ].

However, if any of the table columns are re-sorted , the re-load operation gets confused and I can’t get the saved a saved selected item to become selected again.

(It seems to be confusing the index numbers of the sorted and unsorted lists – even when I match by contents rather than index.)

(I also created an equality test to ensure equivalent entries are recognised by the #= operation in the list of the underlying model ).

This all works fine - unless I sort a column!

Since this approach was going to be used on several screens I’d really like to find a solution.

Cheers

Mark

Perth, Western Australia

Thanks Kasper Thanks for your thoughts. (BTW - I have been using your sheep app as an informal reference – many thanks) I’m satisfied that the restored objects are fine. Nothing strange arising from persisting and restoring when I inspect the restored objects. Because I have my own equality test I don’t think there is anything strange happening with comparing object identities. (I’ve verified that the underlying SpTablePresenter model collection is using my equality test to do the comparisons.) = anObject ^ (self file name) = (anObject file name). I have now stopped digging after a few days lost on this issue. Fortunately on my own time! I had to go into the Morphic code that supports the Spec2 table and columns, but couldn’t find the answers. It seems the built in column sort functionality happens down deep in Morphic and I couldn’t influence it with Spec methods (e.g. spTablePresenter>>sortingBlock: ). The sort handles on the column headers seem to be something that the table presenter doesn’t control or know about. My conclusion is that the TablePresenter items (the model) does not actually know what the visual representation looks like once the table has been re-sorted. That sounds desirable in terms of coupling; but the model doesn’t actually know what item is selected, which is a problem…! So I think there must be some kind of a disconnect between the model state and the visual representation. Any connection is actually implicit, but is only valid when the table is in its original condition. In my tests, using selectItem: newItem only works if the table has not been resorted. Same for selectIndex: savedIndex too… So my guess is that the select functionality is implemented at the visual level and hasn’t been fully represented at the model level. The model level functionality for selections isn’t really complete – it is just a pass through to the visual level… And of course it works most of the time, so the problem goes unnoticed… In the end, my (brute force) work around was to: 1. reload the table presenter items with the persisted entries ( filePresenter1 items: filteredFileLIst ) 2. scan the table items (from 1 to items size) - selecting each one in turn and then noting what the newly selected item was 3. stopping when the newly selected item matches the entry that was persisted The result is that the correct entry is selected and the table has the correct items. 😊 The only thing missing is that the table sorting state is not restored to its previous state (which is acceptable for now) This workaround is inelegant and may be unusable when/if the main tables have many more entries (which is likely). [ Another possibility was to persist the entire table presenter (which is very slow) and reconstruct my active layout using the restored table presenter. I’m not sure if this is a good idea anyway? My attempts showed it was fiddly, and slow, and it really didn’t appeal to me. ] Cheers, and thanks again Mark From: Kasper Osterbye <kasper.osterbye@gmail.com> Sent: Monday, 5 July 2021 12:06 PM To: Any question about pharo is welcome <pharo-users@lists.pharo.org> Subject: [Pharo-users] Re: Question about Spec2 and SpTablePresenter Cheers, From what code you provide I can not spot any error. I do not use your persistence strategy myself, so I first thought that it might be an issue related to that - for instance something with the identity of objects from the recentFileList and the savedEntry messing up. But that would not explain why it works unsorted. I have a piece of code in my application which is doing pretty much the same: selected := table selection selectedItem. table items: self sheeps. (selected notNil and: [ table items includes: selected ]) ifTrue: [ table selection selectItem: selected. self announceNewSelection: selected ]. For some reason I work on the selection rather than the table, but it does not seem to make any difference when I just tried. Is there a way in which you mock the “objects := ….” to something which do not use persistence to rule out that is the persistence causing the problems (or verify it). A method which returns a triple of known objects perhaps. Sometimes the problem lies in the instantiation of the table, so perhaps that might help to see. I am on EU time and on discord this morning, you are more than welcome to try to grab me. I have been using SpTablePresenter a lot (and is often baffled as well). Best, Kasper On 4 Jul 2021, at 11.26, Mark O'Donoghue <mark.odonoghue.2010@gmail.com <mailto:mark.odonoghue.2010@gmail.com> > wrote: Howdy all I’ve got stuck trying to manage Tables – over 12 hours now and I’m out of ideas!. ☹ Any observations / suggestions are most welcome… I’ve been loading small external files of transactions using NeoCSV into Spec2 tables. I am trying to use Fuel to persist the table contents so that my application will re-load the working state from where I finished in the last session. (I’ve opted for Fuel as a simple alternative to having to do the whole object relational mapping thing.) The idea is that transactions (and potentially some manual adjustments) will be processed over time. (This is preferable to having to reload all files from the beginning evert time I run the application…) The Spec2 tables have been working well until I tried to persist them. I can’t seem to fully re-load them to a previously saved state. For example - I can restore the essential contents of my table in most circumstances using: restoreObjects “filePresenter1 is a SpTablePresenter” | objects savedEntry | objects := CpPersist restoreObjectsFromFileNamed: 'E:\Me\zzzST-Test\demo.fuel'. recentFileList := objects at: 1. currFileFilter := objects at: 2. savedEntry := objects at: 3. self updateFilterButton: currFileFilter. filteredFileList := self filterFilesUsing: currFileFilter. filePresenter1 items: filteredFileList. savedEntry ifNotNil: [filePresenter1 selectItem: savedEntry ]. However, if any of the table columns are re-sorted , the re-load operation gets confused and I can’t get the saved a saved selected item to become selected again. (It seems to be confusing the index numbers of the sorted and unsorted lists – even when I match by contents rather than index.) (I also created an equality test to ensure equivalent entries are recognised by the #= operation in the list of the underlying model ). This all works fine - unless I sort a column! Since this approach was going to be used on several screens I’d really like to find a solution. Cheers Mark Perth, Western Australia
KO
Kasper Osterbye
Mon, Jul 5, 2021 7:50 AM

On 5 Jul 2021, at 09.30, Mark O'Donoghue mark.odonoghue.2010@gmail.com wrote:

Thanks Kasper

Thanks for your thoughts.
(BTW - I have been using your sheep app as an informal reference – many thanks)

Your welcome.

It seems like SpTablePresenter is work in development in many ways. I run into its boundaries all the time (last was how to add a vertical scrollbar in case of wide tables).

I vaguely remember problems with sort from earlier, which is why I operate directly on the selection model (different from the items model). There was also some issues which had to do with threeway vs. binary comparisons, but it does not sound like that is your issue either.

Best,

Kasper

> On 5 Jul 2021, at 09.30, Mark O'Donoghue <mark.odonoghue.2010@gmail.com> wrote: > > Thanks Kasper > > Thanks for your thoughts. > (BTW - I have been using your sheep app as an informal reference – many thanks) Your welcome. It seems like SpTablePresenter is work in development in many ways. I run into its boundaries all the time (last was how to add a vertical scrollbar in case of wide tables). I vaguely remember problems with sort from earlier, which is why I operate directly on the selection model (different from the items model). There was also some issues which had to do with threeway vs. binary comparisons, but it does not sound like that is your issue either. Best, Kasper