Snapshot selected items from a list

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Snapshot selected items from a list

Kevin Day
Here's the use case:

We have a JTable backed by a GL sorted list.  Users can choose rows in the table (or the entire table) and perform an operation on the selected items.  These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...).  This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated).  This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing.  The problem is that  users may change their selected rows, or adjust sort order, while we are in the middle of processing.  Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is:  is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm  pretty sure I could figure it out).  The selected list, though, seems like a much harder nut to crack.  I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view.  And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Snapshot selected items from a list

robeden
Have you tried wrapping it in a FreezeList? If I understand what you’re trying to do, you could freeze it, make changes and then thaw.

Rob

On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:

Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

re[2]: Snapshot selected items from a list

Kevin Day
hmmmm - doesn't look like that is going to work.  Once the list is frozen, it's not possible to push changes upstream. (I get 'List ca.odell.glazedlists.FreezableList cannot be modified in the current state' ).  And I can't wait until the operations are all finished to unfreeze (the list would be totally non-responsive if I did that)...


Some ideas (in case this triggers anything for anyone):  create a static Set with all of the selected elements, then create a filtered list that uses the set membership to determine whether to include an element or not.  This filtered list could then be used for the work list.  The problem here is that the ordering doesn't get preserved.  So I'd need to also wrap it in a sorted list...  Ugh - nightmare.

ok - another option:  create the set like above, then loop through the parent list looking for each element.  Then capture the index in the source list for each element.  When I do my updates, I then write directly to the source list instead of writing to the selected/sorted list.  Maybe it would be possible to create a TransformList subclass that implements this sort of behavior...

I could maybe try to take advantage of the getSourceIndex() method in the selection list, but I think by that point that I may be 4 or 5 lists down from the BasicEventList, so there may be several index transformations between the select list and the root of the list chain.  Without having access to the parent list (and TransformedList doesn't appear to provide that) of each list in the chain, there doesn't seem to be any way to make it work...

Thoughts?

- K

----------------------- Original Message -----------------------

From: Rob Eden <[hidden email]>
To: Kevin Day <[hidden email]>, [hidden email]
Cc:
Date: Thu, 15 May 2014 12:11:15 -0500
Subject: Re: Snapshot selected items from a list

Have you tried wrapping it in a FreezeList? If I understand what youre trying to do, you could freeze it, make changes and then thaw.


Rob


On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:
Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: re[2]: Snapshot selected items from a list

James Lemieux
Hey Kevin,

   So, you'll definitely want to "snapshot" a private view of the selected items at the time that processing begins. I would do that in the simplest way possible:

final Set<Thing> snapshot = new HashSet<Thing>(yourDefaultEventSelectionModel.getSelected());

   Next, construct a FilterList from an appropriate upstream EventList that is sourcing your DefaultEventSelectionModel (something above all of the SortedLists and FilterLists that the user is free to alter). Make the Matcher of said FilterList take in the snapshot Set<Thing> from above and returns true if snapshot.contains(someThing). That Matcher should be fast to evaluate thanks to hashing. More importantly, it provides you with a live view into your pipeline that you can write through and that won't change as the user mutates their view of the table (re-sorts, re-filters, alters selection, etc) . e.g.:

for (ListIterator<Thing> li = theFilterList.listIterator(); li.hasNext();) {
   final Thing toProcess = li.next();
   final Thing processed = process(toProcess);
   li.set(processed);
}

alternatively, you can use an index loop and walk backwards through the FilterList if you want to reduce the ListEvent processing caused by each call to set. e.g.:

for (int i = theFilterList.size()-1; i >= 0; i--) {
   final Thing toProcess = theFilterList.get(i);
   final Thing processed = process(toProcess);
   theFilterList.set(i, processed);
}

Final notes:

a) remember to detach the FilterList from the pipeline after processing stops
b) take special care to ensure the Matcher and the snapshot isn't referenced by any long-lived objects, since the Set<Thing> in the Matcher in turn references a bunch of old Things that are out of date and no longer useful and would constitute a memory leak.

James


On Thu, May 15, 2014 at 11:39 AM, Kevin Day <[hidden email]> wrote:
hmmmm - doesn't look like that is going to work.  Once the list is frozen, it's not possible to push changes upstream. (I get 'List ca.odell.glazedlists.FreezableList cannot be modified in the current state' ).  And I can't wait until the operations are all finished to unfreeze (the list would be totally non-responsive if I did that)...


Some ideas (in case this triggers anything for anyone):  create a static Set with all of the selected elements, then create a filtered list that uses the set membership to determine whether to include an element or not.  This filtered list could then be used for the work list.  The problem here is that the ordering doesn't get preserved.  So I'd need to also wrap it in a sorted list...  Ugh - nightmare.

ok - another option:  create the set like above, then loop through the parent list looking for each element.  Then capture the index in the source list for each element.  When I do my updates, I then write directly to the source list instead of writing to the selected/sorted list.  Maybe it would be possible to create a TransformList subclass that implements this sort of behavior...

I could maybe try to take advantage of the getSourceIndex() method in the selection list, but I think by that point that I may be 4 or 5 lists down from the BasicEventList, so there may be several index transformations between the select list and the root of the list chain.  Without having access to the parent list (and TransformedList doesn't appear to provide that) of each list in the chain, there doesn't seem to be any way to make it work...

Thoughts?

- K

----------------------- Original Message -----------------------

From: Rob Eden <[hidden email]>
To: Kevin Day <[hidden email]>, [hidden email]
Cc:
Date: Thu, 15 May 2014 12:11:15 -0500
Subject: Re: Snapshot selected items from a list

Have you tried wrapping it in a FreezeList? If I understand what youre trying to do, you could freeze it, make changes and then thaw.


Rob


On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:
Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

re[4]: Snapshot selected items from a list

Kevin Day
OK - this works - thank you.  And yes - I see that calling dispose() on the filterlist is critical.


A couple of detail questions:

If I use an iterator like you have shown, does the iterator become invalid if the order of the list changes, or does that take care of preserving the iteration order at the time the iterator was created?  Ideally, I'd like to preserve the ordering of the selection model during our processing (not mandatory, but it would be nice).  I was thinking that I'd have to create a separate sorted list, but maybe that isn't necessary if the iterator is safe to use?  I'm using granular locks (ThreadSafeList), so I'm guessing that this would cause the iterator to fail, but I figured it would be worth asking.


I am curious about why walking the list backward would result in more efficient ListEvent processing - is that indeed the case?

Thanks,

- K

----------------------- Original Message -----------------------

From: James Lemieux <[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: Rob Eden <[hidden email]>
Date: Thu, 15 May 2014 14:05:24 -0700
Subject: Re: re[2]: Snapshot selected items from a list

Hey Kevin,


So, you'll definitely want to "snapshot" a private view of the selected items at the time that processing begins. I would do that in the simplest way possible:


final Set<Thing> snapshot = new HashSet<Thing>(yourDefaultEventSelectionModel.getSelected());


Next, construct a FilterList from an appropriate upstream EventList that is sourcing your DefaultEventSelectionModel (something above all of the SortedLists and FilterLists that the user is free to alter). Make the Matcher of said FilterList take in the snapshot Set<Thing> from above and returns true if snapshot.contains(someThing). That Matcher should be fast to evaluate thanks to hashing. More importantly, it provides you with a live view into your pipeline that you can write through and that won't change as the user mutates their view of the table (re-sorts, re-filters, alters selection, etc) . e.g.:


for (ListIterator<Thing> li = theFilterList.listIterator(); li.hasNext();) {
final Thing toProcess = li.next();
final Thing processed = process(toProcess);
li.set(processed);
}


alternatively, you can use an index loop and walk backwards through the FilterList if you want to reduce the ListEvent processing caused by each call to set. e.g.:


for (int i = theFilterList.size()-1; i >= 0; i--) {
final Thing toProcess = theFilterList.get(i);
final Thing processed = process(toProcess);
theFilterList.set(i, processed);
}



Final notes:


a) remember to detach the FilterList from the pipeline after processing stops
b) take special care to ensure the Matcher and the snapshot isn't referenced by any long-lived objects, since the Set<Thing> in the Matcher in turn references a bunch of old Things that are out of date and no longer useful and would constitute a memory leak.


James



On Thu, May 15, 2014 at 11:39 AM, Kevin Day <[hidden email]> wrote:

hmmmm - doesn't look like that is going to work.  Once the list is frozen, it's not possible to push changes upstream. (I get 'List ca.odell.glazedlists.FreezableList cannot be modified in the current state' ).  And I can't wait until the operations are all finished to unfreeze (the list would be totally non-responsive if I did that)...


Some ideas (in case this triggers anything for anyone):  create a static Set with all of the selected elements, then create a filtered list that uses the set membership to determine whether to include an element or not.  This filtered list could then be used for the work list.  The problem here is that the ordering doesn't get preserved.  So I'd need to also wrap it in a sorted list...  Ugh - nightmare.

ok - another option:  create the set like above, then loop through the parent list looking for each element.  Then capture the index in the source list for each element.  When I do my updates, I then write directly to the source list instead of writing to the selected/sorted list.  Maybe it would be possible to create a TransformList subclass that implements this sort of behavior...

I could maybe try to take advantage of the getSourceIndex() method in the selection list, but I think by that point that I may be 4 or 5 lists down from the BasicEventList, so there may be several index transformations between the select list and the root of the list chain.  Without having access to the parent list (and TransformedList doesn't appear to provide that) of each list in the chain, there doesn't seem to be any way to make it work...

Thoughts?

- K

----------------------- Original Message -----------------------

From: Rob Eden <[hidden email]>
To: Kevin Day <[hidden email]>, [hidden email]
Cc:
Date: Thu, 15 May 2014 12:11:15 -0500
Subject: Re: Snapshot selected items from a list

Have you tried wrapping it in a FreezeList? If I understand what youre trying to do, you could freeze it, make changes and then thaw.



Rob


On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:
Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: re[4]: Snapshot selected items from a list

James Lemieux
Hey Kevin,

   The ListIterator returned by AbstractEventList actually listens to the pipeline for changes and stays in sync (regardless of the origin of the change). i.e. Because of this, in theory, ConcurrentModificationException is impossible if you have proper pipeline locking. But, that also means you've got *one more listener* reacting to changes in your pipeline, and that listener is merely updating a data structure we intend to discard.

   So, if you want to avoid the cost of that "extra listener" you can loop with indices instead. But, the first time you call "theFilterList.set(i, processed);" you'll actually be REMOVING an item from "theFilterList", since the matcher will return false for the newly replaced item. If the newly removed item is the LAST item in the FilterList, then it means little to us because it doesn't disturb the indices of the remaining elements in the FilterList.

   By the time you process the last item in the FilterList, you'll have effectively emptied it, because none of the snapshots in your Matcher remain in the pipeline.

   So, if you'd like to preserve the order of the elements when the snapshot is taken, it's a bit more complicated. You'd want to stack a SortedList on top of the snapshot FilterList and you'd want it to have the same Comparator as the one that backed your JTable's SortedList. In essence, you're taking a snapshot of the sorting state this way. Since sorting is much more costly than your filtering, ensure you stack it AFTER the snapshot FilterList so it sorts the fewest elements possible. Remember to dispose() of the SortedList as well.

   As well, since it sounds like you want to process the selections "top down" according to their original order, you should use the ListIterator loop mentioned above.

Hope this helps,

James

   


On Thu, May 15, 2014 at 4:44 PM, Kevin Day <[hidden email]> wrote:
OK - this works - thank you.  And yes - I see that calling dispose() on the filterlist is critical.


A couple of detail questions:

If I use an iterator like you have shown, does the iterator become invalid if the order of the list changes, or does that take care of preserving the iteration order at the time the iterator was created?  Ideally, I'd like to preserve the ordering of the selection model during our processing (not mandatory, but it would be nice).  I was thinking that I'd have to create a separate sorted list, but maybe that isn't necessary if the iterator is safe to use?  I'm using granular locks (ThreadSafeList), so I'm guessing that this would cause the iterator to fail, but I figured it would be worth asking.


I am curious about why walking the list backward would result in more efficient ListEvent processing - is that indeed the case?

Thanks,

- K

----------------------- Original Message -----------------------

From: James Lemieux <[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: Rob Eden <[hidden email]>
Date: Thu, 15 May 2014 14:05:24 -0700
Subject: Re: re[2]: Snapshot selected items from a list

Hey Kevin,


So, you'll definitely want to "snapshot" a private view of the selected items at the time that processing begins. I would do that in the simplest way possible:


final Set<Thing> snapshot = new HashSet<Thing>(yourDefaultEventSelectionModel.getSelected());


Next, construct a FilterList from an appropriate upstream EventList that is sourcing your DefaultEventSelectionModel (something above all of the SortedLists and FilterLists that the user is free to alter). Make the Matcher of said FilterList take in the snapshot Set<Thing> from above and returns true if snapshot.contains(someThing). That Matcher should be fast to evaluate thanks to hashing. More importantly, it provides you with a live view into your pipeline that you can write through and that won't change as the user mutates their view of the table (re-sorts, re-filters, alters selection, etc) . e.g.:


for (ListIterator<Thing> li = theFilterList.listIterator(); li.hasNext();) {
final Thing toProcess = li.next();
final Thing processed = process(toProcess);
li.set(processed);
}


alternatively, you can use an index loop and walk backwards through the FilterList if you want to reduce the ListEvent processing caused by each call to set. e.g.:


for (int i = theFilterList.size()-1; i >= 0; i--) {
final Thing toProcess = theFilterList.get(i);
final Thing processed = process(toProcess);
theFilterList.set(i, processed);
}



Final notes:


a) remember to detach the FilterList from the pipeline after processing stops
b) take special care to ensure the Matcher and the snapshot isn't referenced by any long-lived objects, since the Set<Thing> in the Matcher in turn references a bunch of old Things that are out of date and no longer useful and would constitute a memory leak.


James



On Thu, May 15, 2014 at 11:39 AM, Kevin Day <[hidden email]> wrote:

hmmmm - doesn't look like that is going to work.  Once the list is frozen, it's not possible to push changes upstream. (I get 'List ca.odell.glazedlists.FreezableList cannot be modified in the current state' ).  And I can't wait until the operations are all finished to unfreeze (the list would be totally non-responsive if I did that)...


Some ideas (in case this triggers anything for anyone):  create a static Set with all of the selected elements, then create a filtered list that uses the set membership to determine whether to include an element or not.  This filtered list could then be used for the work list.  The problem here is that the ordering doesn't get preserved.  So I'd need to also wrap it in a sorted list...  Ugh - nightmare.

ok - another option:  create the set like above, then loop through the parent list looking for each element.  Then capture the index in the source list for each element.  When I do my updates, I then write directly to the source list instead of writing to the selected/sorted list.  Maybe it would be possible to create a TransformList subclass that implements this sort of behavior...

I could maybe try to take advantage of the getSourceIndex() method in the selection list, but I think by that point that I may be 4 or 5 lists down from the BasicEventList, so there may be several index transformations between the select list and the root of the list chain.  Without having access to the parent list (and TransformedList doesn't appear to provide that) of each list in the chain, there doesn't seem to be any way to make it work...

Thoughts?

- K

----------------------- Original Message -----------------------

From: Rob Eden <[hidden email]>
To: Kevin Day <[hidden email]>, [hidden email]
Cc:
Date: Thu, 15 May 2014 12:11:15 -0500
Subject: Re: Snapshot selected items from a list

Have you tried wrapping it in a FreezeList? If I understand what youre trying to do, you could freeze it, make changes and then thaw.



Rob


On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:
Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

re[6]: Snapshot selected items from a list

Kevin Day

This has been a huge, huge help - thank you.  I had originally worked around the issue with elements being removed from the filter by using a unique key (instead of Java ID based hashcode) - using the iterator like you describe is a far better solution.  I'll get cracking on that now.

Thanks again,

- K


----------------------- Original Message -----------------------
  
From: James Lemieux [hidden email]
Cc: 
Date: Thu, 15 May 2014 17:43:34 -0700
Subject: Re: re[4]: Snapshot selected items from a list
  
Hey Kevin,

   The ListIterator returned by AbstractEventList actually listens to the pipeline for changes and stays in sync (regardless of the origin of the change). i.e. Because of this, in theory, ConcurrentModificationException is impossible if you have proper pipeline locking. But, that also means you've got *one more listener* reacting to changes in your pipeline, and that listener is merely updating a data structure we intend to discard.

   So, if you want to avoid the cost of that "extra listener" you can loop with indices instead. But, the first time you call "theFilterList.set(i, processed);" you'll actually be REMOVING an item from "theFilterList", since the matcher will return false for the newly replaced item. If the newly removed item is the LAST item in the FilterList, then it means little to us because it doesn't disturb the indices of the remaining elements in the FilterList.

   By the time you process the last item in the FilterList, you'll have effectively emptied it, because none of the snapshots in your Matcher remain in the pipeline.

   So, if you'd like to preserve the order of the elements when the snapshot is taken, it's a bit more complicated. You'd want to stack a SortedList on top of the snapshot FilterList and you'd want it to have the same Comparator as the one that backed your JTable's SortedList. In essence, you're taking a snapshot of the sorting state this way. Since sorting is much more costly than your filtering, ensure you stack it AFTER the snapshot FilterList so it sorts the fewest elements possible. Remember to dispose() of the SortedList as well.

   As well, since it sounds like you want to process the selections "top down" according to their original order, you should use the ListIterator loop mentioned above.

Hope this helps,

James

   


On Thu, May 15, 2014 at 4:44 PM, Kevin Day <[hidden email]> wrote:
OK - this works - thank you.  And yes - I see that calling dispose() on the filterlist is critical.


A couple of detail questions:

If I use an iterator like you have shown, does the iterator become invalid if the order of the list changes, or does that take care of preserving the iteration order at the time the iterator was created?  Ideally, I'd like to preserve the ordering of the selection model during our processing (not mandatory, but it would be nice).  I was thinking that I'd have to create a separate sorted list, but maybe that isn't necessary if the iterator is safe to use?  I'm using granular locks (ThreadSafeList), so I'm guessing that this would cause the iterator to fail, but I figured it would be worth asking.


I am curious about why walking the list backward would result in more efficient ListEvent processing - is that indeed the case?

Thanks,

- K

----------------------- Original Message -----------------------

From: James Lemieux <[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: Rob Eden <[hidden email]>
Date: Thu, 15 May 2014 14:05:24 -0700
Subject: Re: re[2]: Snapshot selected items from a list

Hey Kevin,


So, you'll definitely want to "snapshot" a private view of the selected items at the time that processing begins. I would do that in the simplest way possible:


final Set<Thing> snapshot = new HashSet<Thing>(yourDefaultEventSelectionModel.getSelected());


Next, construct a FilterList from an appropriate upstream EventList that is sourcing your DefaultEventSelectionModel (something above all of the SortedLists and FilterLists that the user is free to alter). Make the Matcher of said FilterList take in the snapshot Set<Thing> from above and returns true if snapshot.contains(someThing). That Matcher should be fast to evaluate thanks to hashing. More importantly, it provides you with a live view into your pipeline that you can write through and that won't change as the user mutates their view of the table (re-sorts, re-filters, alters selection, etc) . e.g.:


for (ListIterator<Thing> li = theFilterList.listIterator(); li.hasNext();) {
final Thing toProcess = li.next();
final Thing processed = process(toProcess);
li.set(processed);
}


alternatively, you can use an index loop and walk backwards through the FilterList if you want to reduce the ListEvent processing caused by each call to set. e.g.:


for (int i = theFilterList.size()-1; i >= 0; i--) {
final Thing toProcess = theFilterList.get(i);
final Thing processed = process(toProcess);
theFilterList.set(i, processed);
}



Final notes:


a) remember to detach the FilterList from the pipeline after processing stops
b) take special care to ensure the Matcher and the snapshot isn't referenced by any long-lived objects, since the Set<Thing> in the Matcher in turn references a bunch of old Things that are out of date and no longer useful and would constitute a memory leak.


James



On Thu, May 15, 2014 at 11:39 AM, Kevin Day <[hidden email] > wrote:

hmmmm - doesn't look like that is going to work.  Once the list is frozen, it's not possible to push changes upstream. (I get 'List ca.odell.glazedlists.FreezableList cannot be modified in the current state' ).  And I can't wait until the operations are all finished to unfreeze (the list would be totally non-responsive if I did that)...


Some ideas (in case this triggers anything for anyone):  create a static Set with all of the selected elements, then create a filtered list that uses the set membership to determine whether to include an element or not.  This filtered list could then be used for the work list.  The problem here is that the ordering doesn't get preserved.  So I'd need to also wrap it in a sorted list...  Ugh - nightmare.

ok - another option:  create the set like above, then loop through the parent list looking for each element.  Then capture the index in the source list for each element.  When I do my updates, I then write directly to the source list instead of writing to the selected/sorted list.  Maybe it would be possible to create a TransformList subclass that implements this sort of behavior...

I could maybe try to take advantage of the getSourceIndex() method in the selection list, but I think by that point that I may be 4 or 5 lists down from the BasicEventList, so there may be several index transformations between the select list and the root of the list chain.  Without having access to the parent list (and TransformedList doesn't appear to provide that) of each list in the chain, there doesn't seem to be any way to make it work...

Thoughts?

- K

----------------------- Original Message -----------------------

From: Rob Eden <[hidden email]>
To: Kevin Day <[hidden email]>, [hidden email]
Cc:
Date: Thu, 15 May 2014 12:11:15 -0500
Subject: Re: Snapshot selected items from a list

Have you tried wrapping it in a FreezeList? If I understand what youre trying to do, you could freeze it, make changes and then thaw.



Rob


On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:
Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

re[6]: Snapshot selected items from a list

Kevin Day
In reply to this post by James Lemieux

oh - wait a sec.  If I use the iterator approach, then maybe I don't need the filter list at all??  Won't the iterator from the selection list model remain consistent?

- K


----------------------- Original Message -----------------------
  
From: James Lemieux [hidden email]
Cc: 
Date: Thu, 15 May 2014 17:43:34 -0700
Subject: Re: re[4]: Snapshot selected items from a list
  
Hey Kevin,

   The ListIterator returned by AbstractEventList actually listens to the pipeline for changes and stays in sync (regardless of the origin of the change). i.e. Because of this, in theory, ConcurrentModificationException is impossible if you have proper pipeline locking. But, that also means you've got *one more listener* reacting to changes in your pipeline, and that listener is merely updating a data structure we intend to discard.

   So, if you want to avoid the cost of that "extra listener" you can loop with indices instead. But, the first time you call "theFilterList.set(i, processed);" you'll actually be REMOVING an item from "theFilterList", since the matcher will return false for the newly replaced item. If the newly removed item is the LAST item in the FilterList, then it means little to us because it doesn't disturb the indices of the remaining elements in the FilterList.

   By the time you process the last item in the FilterList, you'll have effectively emptied it, because none of the snapshots in your Matcher remain in the pipeline.

   So, if you'd like to preserve the order of the elements when the snapshot is taken, it's a bit more complicated. You'd want to stack a SortedList on top of the snapshot FilterList and you'd want it to have the same Comparator as the one that backed your JTable's SortedList. In essence, you're taking a snapshot of the sorting state this way. Since sorting is much more costly than your filtering, ensure you stack it AFTER the snapshot FilterList so it sorts the fewest elements possible. Remember to dispose() of the SortedList as well.

   As well, since it sounds like you want to process the selections "top down" according to their original order, you should use the ListIterator loop mentioned above.

Hope this helps,

James

   


On Thu, May 15, 2014 at 4:44 PM, Kevin Day <[hidden email]> wrote:
OK - this works - thank you.  And yes - I see that calling dispose() on the filterlist is critical.


A couple of detail questions:

If I use an iterator like you have shown, does the iterator become invalid if the order of the list changes, or does that take care of preserving the iteration order at the time the iterator was created?  Ideally, I'd like to preserve the ordering of the selection model during our processing (not mandatory, but it would be nice).  I was thinking that I'd have to create a separate sorted list, but maybe that isn't necessary if the iterator is safe to use?  I'm using granular locks (ThreadSafeList), so I'm guessing that this would cause the iterator to fail, but I figured it would be worth asking.


I am curious about why walking the list backward would result in more efficient ListEvent processing - is that indeed the case?

Thanks,

- K

----------------------- Original Message -----------------------

From: James Lemieux <[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: Rob Eden <[hidden email]>
Date: Thu, 15 May 2014 14:05:24 -0700
Subject: Re: re[2]: Snapshot selected items from a list

Hey Kevin,


So, you'll definitely want to "snapshot" a private view of the selected items at the time that processing begins. I would do that in the simplest way possible:


final Set<Thing> snapshot = new HashSet<Thing>(yourDefaultEventSelectionModel.getSelected());


Next, construct a FilterList from an appropriate upstream EventList that is sourcing your DefaultEventSelectionModel (something above all of the SortedLists and FilterLists that the user is free to alter). Make the Matcher of said FilterList take in the snapshot Set<Thing> from above and returns true if snapshot.contains(someThing). That Matcher should be fast to evaluate thanks to hashing. More importantly, it provides you with a live view into your pipeline that you can write through and that won't change as the user mutates their view of the table (re-sorts, re-filters, alters selection, etc) . e.g.:


for (ListIterator<Thing> li = theFilterList.listIterator(); li.hasNext();) {
final Thing toProcess = li.next();
final Thing processed = process(toProcess);
li.set(processed);
}


alternatively, you can use an index loop and walk backwards through the FilterList if you want to reduce the ListEvent processing caused by each call to set. e.g.:


for (int i = theFilterList.size()-1; i >= 0; i--) {
final Thing toProcess = theFilterList.get(i);
final Thing processed = process(toProcess);
theFilterList.set(i, processed);
}



Final notes:


a) remember to detach the FilterList from the pipeline after processing stops
b) take special care to ensure the Matcher and the snapshot isn't referenced by any long-lived objects, since the Set<Thing> in the Matcher in turn references a bunch of old Things that are out of date and no longer useful and would constitute a memory leak.


James



On Thu, May 15, 2014 at 11:39 AM, Kevin Day <[hidden email] > wrote:

hmmmm - doesn't look like that is going to work.  Once the list is frozen, it's not possible to push changes upstream. (I get 'List ca.odell.glazedlists.FreezableList cannot be modified in the current state' ).  And I can't wait until the operations are all finished to unfreeze (the list would be totally non-responsive if I did that)...


Some ideas (in case this triggers anything for anyone):  create a static Set with all of the selected elements, then create a filtered list that uses the set membership to determine whether to include an element or not.  This filtered list could then be used for the work list.  The problem here is that the ordering doesn't get preserved.  So I'd need to also wrap it in a sorted list...  Ugh - nightmare.

ok - another option:  create the set like above, then loop through the parent list looking for each element.  Then capture the index in the source list for each element.  When I do my updates, I then write directly to the source list instead of writing to the selected/sorted list.  Maybe it would be possible to create a TransformList subclass that implements this sort of behavior...

I could maybe try to take advantage of the getSourceIndex() method in the selection list, but I think by that point that I may be 4 or 5 lists down from the BasicEventList, so there may be several index transformations between the select list and the root of the list chain.  Without having access to the parent list (and TransformedList doesn't appear to provide that) of each list in the chain, there doesn't seem to be any way to make it work...

Thoughts?

- K

----------------------- Original Message -----------------------

From: Rob Eden <[hidden email]>
To: Kevin Day <[hidden email]>, [hidden email]
Cc:
Date: Thu, 15 May 2014 12:11:15 -0500
Subject: Re: Snapshot selected items from a list

Have you tried wrapping it in a FreezeList? If I understand what youre trying to do, you could freeze it, make changes and then thaw.



Rob


On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:
Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

re[6]: Snapshot selected items from a list

Kevin Day
In reply to this post by James Lemieux

ok - I just tested, and the answer is 'no' - the iterator is still subject to the underlying entries in the selection list.  If we change selection, then things break.  So a filter list is going to be necessary to capture the current selection.

 

I'm digging a bit at the code, and I'm wondering:

Would a better approach be to enhance ListSelection so it can return a static copy of a SelectedList (getCurrentSelected() maybe), hard tied to the current barcode (instead of referring to the ListSelection barcode instance?

 

Thanks,

- K


----------------------- Original Message -----------------------
  
From: James Lemieux [hidden email]
Cc: 
Date: Thu, 15 May 2014 17:43:34 -0700
Subject: Re: re[4]: Snapshot selected items from a list
  
Hey Kevin,

   The ListIterator returned by AbstractEventList actually listens to the pipeline for changes and stays in sync (regardless of the origin of the change). i.e. Because of this, in theory, ConcurrentModificationException is impossible if you have proper pipeline locking. But, that also means you've got *one more listener* reacting to changes in your pipeline, and that listener is merely updating a data structure we intend to discard.

   So, if you want to avoid the cost of that "extra listener" you can loop with indices instead. But, the first time you call "theFilterList.set(i, processed);" you'll actually be REMOVING an item from "theFilterList", since the matcher will return false for the newly replaced item. If the newly removed item is the LAST item in the FilterList, then it means little to us because it doesn't disturb the indices of the remaining elements in the FilterList.

   By the time you process the last item in the FilterList, you'll have effectively emptied it, because none of the snapshots in your Matcher remain in the pipeline.

   So, if you'd like to preserve the order of the elements when the snapshot is taken, it's a bit more complicated. You'd want to stack a SortedList on top of the snapshot FilterList and you'd want it to have the same Comparator as the one that backed your JTable's SortedList. In essence, you're taking a snapshot of the sorting state this way. Since sorting is much more costly than your filtering, ensure you stack it AFTER the snapshot FilterList so it sorts the fewest elements possible. Remember to dispose() of the SortedList as well.

   As well, since it sounds like you want to process the selections "top down" according to their original order, you should use the ListIterator loop mentioned above.

Hope this helps,

James

   


On Thu, May 15, 2014 at 4:44 PM, Kevin Day <[hidden email]> wrote:
OK - this works - thank you.  And yes - I see that calling dispose() on the filterlist is critical.


A couple of detail questions:

If I use an iterator like you have shown, does the iterator become invalid if the order of the list changes, or does that take care of preserving the iteration order at the time the iterator was created?  Ideally, I'd like to preserve the ordering of the selection model during our processing (not mandatory, but it would be nice).  I was thinking that I'd have to create a separate sorted list, but maybe that isn't necessary if the iterator is safe to use?  I'm using granular locks (ThreadSafeList), so I'm guessing that this would cause the iterator to fail, but I figured it would be worth asking.


I am curious about why walking the list backward would result in more efficient ListEvent processing - is that indeed the case?

Thanks,

- K

----------------------- Original Message -----------------------

From: James Lemieux <[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: Rob Eden <[hidden email]>
Date: Thu, 15 May 2014 14:05:24 -0700
Subject: Re: re[2]: Snapshot selected items from a list

Hey Kevin,


So, you'll definitely want to "snapshot" a private view of the selected items at the time that processing begins. I would do that in the simplest way possible:


final Set<Thing> snapshot = new HashSet<Thing>(yourDefaultEventSelectionModel.getSelected());


Next, construct a FilterList from an appropriate upstream EventList that is sourcing your DefaultEventSelectionModel (something above all of the SortedLists and FilterLists that the user is free to alter). Make the Matcher of said FilterList take in the snapshot Set<Thing> from above and returns true if snapshot.contains(someThing). That Matcher should be fast to evaluate thanks to hashing. More importantly, it provides you with a live view into your pipeline that you can write through and that won't change as the user mutates their view of the table (re-sorts, re-filters, alters selection, etc) . e.g.:


for (ListIterator<Thing> li = theFilterList.listIterator(); li.hasNext();) {
final Thing toProcess = li.next();
final Thing processed = process(toProcess);
li.set(processed);
}


alternatively, you can use an index loop and walk backwards through the FilterList if you want to reduce the ListEvent processing caused by each call to set. e.g.:


for (int i = theFilterList.size()-1; i >= 0; i--) {
final Thing toProcess = theFilterList.get(i);
final Thing processed = process(toProcess);
theFilterList.set(i, processed);
}



Final notes:


a) remember to detach the FilterList from the pipeline after processing stops
b) take special care to ensure the Matcher and the snapshot isn't referenced by any long-lived objects, since the Set<Thing> in the Matcher in turn references a bunch of old Things that are out of date and no longer useful and would constitute a memory leak.


James



On Thu, May 15, 2014 at 11:39 AM, Kevin Day <[hidden email] > wrote:

hmmmm - doesn't look like that is going to work.  Once the list is frozen, it's not possible to push changes upstream. (I get 'List ca.odell.glazedlists.FreezableList cannot be modified in the current state' ).  And I can't wait until the operations are all finished to unfreeze (the list would be totally non-responsive if I did that)...


Some ideas (in case this triggers anything for anyone):  create a static Set with all of the selected elements, then create a filtered list that uses the set membership to determine whether to include an element or not.  This filtered list could then be used for the work list.  The problem here is that the ordering doesn't get preserved.  So I'd need to also wrap it in a sorted list...  Ugh - nightmare.

ok - another option:  create the set like above, then loop through the parent list looking for each element.  Then capture the index in the source list for each element.  When I do my updates, I then write directly to the source list instead of writing to the selected/sorted list.  Maybe it would be possible to create a TransformList subclass that implements this sort of behavior...

I could maybe try to take advantage of the getSourceIndex() method in the selection list, but I think by that point that I may be 4 or 5 lists down from the BasicEventList, so there may be several index transformations between the select list and the root of the list chain.  Without having access to the parent list (and TransformedList doesn't appear to provide that) of each list in the chain, there doesn't seem to be any way to make it work...

Thoughts?

- K

----------------------- Original Message -----------------------

From: Rob Eden <[hidden email]>
To: Kevin Day <[hidden email]>, [hidden email]
Cc:
Date: Thu, 15 May 2014 12:11:15 -0500
Subject: Re: Snapshot selected items from a list

Have you tried wrapping it in a FreezeList? If I understand what youre trying to do, you could freeze it, make changes and then thaw.



Rob


On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:
Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: re[6]: Snapshot selected items from a list

James Lemieux
A "snapshotting" mechanism built right into DefaultEventSelectionModel is possibly interesting. I'm not sure how often such a use case shows itself.

There would be a bunch of edge cases to it though. In the normal selection model, there are already strange cases where newly added items are auto-selected because of their proximity to existing selections. Stuff like that is tricky to tease out and test...

James


On Fri, May 16, 2014 at 10:51 AM, Kevin Day <[hidden email]> wrote:

ok - I just tested, and the answer is 'no' - the iterator is still subject to the underlying entries in the selection list.  If we change selection, then things break.  So a filter list is going to be necessary to capture the current selection.

 

I'm digging a bit at the code, and I'm wondering:

Would a better approach be to enhance ListSelection so it can return a static copy of a SelectedList (getCurrentSelected() maybe), hard tied to the current barcode (instead of referring to the ListSelection barcode instance?

 

Thanks,

- K


----------------------- Original Message -----------------------
  
From: James Lemieux [hidden email]
Cc: 
Date: Thu, 15 May 2014 17:43:34 -0700
Subject: Re: re[4]: Snapshot selected items from a list
  
Hey Kevin,

   The ListIterator returned by AbstractEventList actually listens to the pipeline for changes and stays in sync (regardless of the origin of the change). i.e. Because of this, in theory, ConcurrentModificationException is impossible if you have proper pipeline locking. But, that also means you've got *one more listener* reacting to changes in your pipeline, and that listener is merely updating a data structure we intend to discard.

   So, if you want to avoid the cost of that "extra listener" you can loop with indices instead. But, the first time you call "theFilterList.set(i, processed);" you'll actually be REMOVING an item from "theFilterList", since the matcher will return false for the newly replaced item. If the newly removed item is the LAST item in the FilterList, then it means little to us because it doesn't disturb the indices of the remaining elements in the FilterList.

   By the time you process the last item in the FilterList, you'll have effectively emptied it, because none of the snapshots in your Matcher remain in the pipeline.

   So, if you'd like to preserve the order of the elements when the snapshot is taken, it's a bit more complicated. You'd want to stack a SortedList on top of the snapshot FilterList and you'd want it to have the same Comparator as the one that backed your JTable's SortedList. In essence, you're taking a snapshot of the sorting state this way. Since sorting is much more costly than your filtering, ensure you stack it AFTER the snapshot FilterList so it sorts the fewest elements possible. Remember to dispose() of the SortedList as well.

   As well, since it sounds like you want to process the selections "top down" according to their original order, you should use the ListIterator loop mentioned above.

Hope this helps,

James

   


On Thu, May 15, 2014 at 4:44 PM, Kevin Day <[hidden email]> wrote:
OK - this works - thank you.  And yes - I see that calling dispose() on the filterlist is critical.


A couple of detail questions:

If I use an iterator like you have shown, does the iterator become invalid if the order of the list changes, or does that take care of preserving the iteration order at the time the iterator was created?  Ideally, I'd like to preserve the ordering of the selection model during our processing (not mandatory, but it would be nice).  I was thinking that I'd have to create a separate sorted list, but maybe that isn't necessary if the iterator is safe to use?  I'm using granular locks (ThreadSafeList), so I'm guessing that this would cause the iterator to fail, but I figured it would be worth asking.


I am curious about why walking the list backward would result in more efficient ListEvent processing - is that indeed the case?

Thanks,

- K

----------------------- Original Message -----------------------

From: James Lemieux <[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: Rob Eden <[hidden email]>
Date: Thu, 15 May 2014 14:05:24 -0700
Subject: Re: re[2]: Snapshot selected items from a list

Hey Kevin,


So, you'll definitely want to "snapshot" a private view of the selected items at the time that processing begins. I would do that in the simplest way possible:


final Set<Thing> snapshot = new HashSet<Thing>(yourDefaultEventSelectionModel.getSelected());


Next, construct a FilterList from an appropriate upstream EventList that is sourcing your DefaultEventSelectionModel (something above all of the SortedLists and FilterLists that the user is free to alter). Make the Matcher of said FilterList take in the snapshot Set<Thing> from above and returns true if snapshot.contains(someThing). That Matcher should be fast to evaluate thanks to hashing. More importantly, it provides you with a live view into your pipeline that you can write through and that won't change as the user mutates their view of the table (re-sorts, re-filters, alters selection, etc) . e.g.:


for (ListIterator<Thing> li = theFilterList.listIterator(); li.hasNext();) {
final Thing toProcess = li.next();
final Thing processed = process(toProcess);
li.set(processed);
}


alternatively, you can use an index loop and walk backwards through the FilterList if you want to reduce the ListEvent processing caused by each call to set. e.g.:


for (int i = theFilterList.size()-1; i >= 0; i--) {
final Thing toProcess = theFilterList.get(i);
final Thing processed = process(toProcess);
theFilterList.set(i, processed);
}



Final notes:


a) remember to detach the FilterList from the pipeline after processing stops
b) take special care to ensure the Matcher and the snapshot isn't referenced by any long-lived objects, since the Set<Thing> in the Matcher in turn references a bunch of old Things that are out of date and no longer useful and would constitute a memory leak.


James



On Thu, May 15, 2014 at 11:39 AM, Kevin Day <[hidden email] > wrote:

hmmmm - doesn't look like that is going to work.  Once the list is frozen, it's not possible to push changes upstream. (I get 'List ca.odell.glazedlists.FreezableList cannot be modified in the current state' ).  And I can't wait until the operations are all finished to unfreeze (the list would be totally non-responsive if I did that)...


Some ideas (in case this triggers anything for anyone):  create a static Set with all of the selected elements, then create a filtered list that uses the set membership to determine whether to include an element or not.  This filtered list could then be used for the work list.  The problem here is that the ordering doesn't get preserved.  So I'd need to also wrap it in a sorted list...  Ugh - nightmare.

ok - another option:  create the set like above, then loop through the parent list looking for each element.  Then capture the index in the source list for each element.  When I do my updates, I then write directly to the source list instead of writing to the selected/sorted list.  Maybe it would be possible to create a TransformList subclass that implements this sort of behavior...

I could maybe try to take advantage of the getSourceIndex() method in the selection list, but I think by that point that I may be 4 or 5 lists down from the BasicEventList, so there may be several index transformations between the select list and the root of the list chain.  Without having access to the parent list (and TransformedList doesn't appear to provide that) of each list in the chain, there doesn't seem to be any way to make it work...

Thoughts?

- K

----------------------- Original Message -----------------------

From: Rob Eden <[hidden email]>
To: Kevin Day <[hidden email]>, [hidden email]
Cc:
Date: Thu, 15 May 2014 12:11:15 -0500
Subject: Re: Snapshot selected items from a list

Have you tried wrapping it in a FreezeList? If I understand what youre trying to do, you could freeze it, make changes and then thaw.



Rob


On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:
Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

re[8]: Snapshot selected items from a list

Kevin Day

ah - ok.  I figured that if we copied the barcode, everything else would fall out.  Easy enough to just use the filterlist.

 

In terms of whether this use-case arises often or not, I can't imagine a UI that allows users to select items that wouldn't want to also be able to process the selected items - but I guess the funky part of my use-case is that the user wants to be able to continue to work with the table, even while processing is occuring...

Thanks,

- K


----------------------- Original Message -----------------------
  
From: James Lemieux [hidden email]
Cc: 
Date: Fri, 16 May 2014 11:49:00 -0700
Subject: Re: re[6]: Snapshot selected items from a list
  
A "snapshotting" mechanism built right into DefaultEventSelectionModel is possibly interesting. I'm not sure how often such a use case shows itself.

There would be a bunch of edge cases to it though. In the normal selection model, there are already strange cases where newly added items are auto-selected because of their proximity to existing selections. Stuff like that is tricky to tease out and test...

James


On Fri, May 16, 2014 at 10:51 AM, Kevin Day <[hidden email]> wrote:

ok - I just tested, and the answer is 'no' - the iterator is still subject to the underlying entries in the selection list.  If we change selection, then things break.  So a filter list is going to be necessary to capture the current selection.

 

I'm digging a bit at the code, and I'm wondering:

Would a better approach be to enhance ListSelection so it can return a static copy of a SelectedList (getCurrentSelected() maybe), hard tied to the current barcode (instead of referring to the ListSelection barcode instance?

 

Thanks,

- K


----------------------- Original Message -----------------------
  
From: James Lemieux [hidden email]
Cc: 
Date: Thu, 15 May 2014 17:43:34 -0700
Subject: Re: re[4]: Snapshot selected items from a list
  
Hey Kevin,

   The ListIterator returned by AbstractEventList actually listens to the pipeline for changes and stays in sync (regardless of the origin of the change). i.e. Because of this, in theory, ConcurrentModificationException is impossible if you have proper pipeline locking. But, that also means you've got *one more listener* reacting to changes in your pipeline, and that listener is merely updating a data structure we intend to discard.

   So, if you want to avoid the cost of that "extra listener" you can loop with indices instead. But, the first time you call "theFilterList.set(i, processed);" you'll actually be REMOVING an item from "theFilterList", since the matcher will return false for the newly replaced item. If the newly removed item is the LAST item in the FilterList, then it means little to us because it doesn't disturb the indices of the remaining elements in the FilterList.

   By the time you process the last item in the FilterList, you'll have effectively emptied it, because none of the snapshots in your Matcher remain in the pipeline.

   So, if you'd like to preserve the order of the elements when the snapshot is taken, it's a bit more complicated. You'd want to stack a SortedList on top of the snapshot FilterList and you'd want it to have the same Comparator as the one that backed your JTable's SortedList. In essence, you're taking a snapshot of the sorting state this way. Since sorting is much more costly than your filtering, ensure you stack it AFTER the snapshot FilterList so it sorts the fewest elements possible. Remember to dispose() of the SortedList as well.

   As well, since it sounds like you want to process the selections "top down" according to their original order, you should use the ListIterator loop mentioned above.

Hope this helps,

James

   


On Thu, May 15, 2014 at 4:44 PM, Kevin Day <[hidden email]> wrote:
OK - this works - thank you.  And yes - I see that calling dispose() on the filterlist is critical.


A couple of detail questions:

If I use an iterator like you have shown, does the iterator become invalid if the order of the list changes, or does that take care of preserving the iteration order at the time the iterator was created?  Ideally, I'd like to preserve the ordering of the selection model during our processing (not mandatory, but it would be nice).  I was thinking that I'd have to create a separate sorted list, but maybe that isn't necessary if the iterator is safe to use?  I'm using granular locks (ThreadSafeList), so I'm guessing that this would cause the iterator to fail, but I figured it would be worth asking.


I am curious about why walking the list backward would result in more efficient ListEvent processing - is that indeed the case?

Thanks,

- K

----------------------- Original Message -----------------------

From: James Lemieux <[hidden email]>
To: "[hidden email]" <[hidden email]>
Cc: Rob Eden <[hidden email]>
Date: Thu, 15 May 2014 14:05:24 -0700
Subject: Re: re[2]: Snapshot selected items from a list

Hey Kevin,


So, you'll definitely want to "snapshot" a private view of the selected items at the time that processing begins. I would do that in the simplest way possible:


final Set<Thing> snapshot = new HashSet<Thing>(yourDefaultEventSelectionModel.getSelected());


Next, construct a FilterList from an appropriate upstream EventList that is sourcing your DefaultEventSelectionModel (something above all of the SortedLists and FilterLists that the user is free to alter). Make the Matcher of said FilterList take in the snapshot Set<Thing> from above and returns true if snapshot.contains(someThing). That Matcher should be fast to evaluate thanks to hashing. More importantly, it provides you with a live view into your pipeline that you can write through and that won't change as the user mutates their view of the table (re-sorts, re-filters, alters selection, etc) . e.g.:


for (ListIterator<Thing> li = theFilterList.listIterator(); li.hasNext();) {
final Thing toProcess = li.next();
final Thing processed = process(toProcess);
li.set(processed);
}


alternatively, you can use an index loop and walk backwards through the FilterList if you want to reduce the ListEvent processing caused by each call to set. e.g.:


for (int i = theFilterList.size()-1; i >= 0; i--) {
final Thing toProcess = theFilterList.get(i);
final Thing processed = process(toProcess);
theFilterList.set(i, processed);
}



Final notes:


a) remember to detach the FilterList from the pipeline after processing stops
b) take special care to ensure the Matcher and the snapshot isn't referenced by any long-lived objects, since the Set<Thing> in the Matcher in turn references a bunch of old Things that are out of date and no longer useful and would constitute a memory leak.


James



On Thu, May 15, 2014 at 11:39 AM, Kevin Day <[hidden email] > wrote:

hmmmm - doesn't look like that is going to work.  Once the list is frozen, it's not possible to push changes upstream. (I get 'List ca.odell.glazedlists.FreezableList cannot be modified in the current state' ).  And I can't wait until the operations are all finished to unfreeze (the list would be totally non-responsive if I did that)...


Some ideas (in case this triggers anything for anyone):  create a static Set with all of the selected elements, then create a filtered list that uses the set membership to determine whether to include an element or not.  This filtered list could then be used for the work list.  The problem here is that the ordering doesn't get preserved.  So I'd need to also wrap it in a sorted list...  Ugh - nightmare.

ok - another option:  create the set like above, then loop through the parent list looking for each element.  Then capture the index in the source list for each element.  When I do my updates, I then write directly to the source list instead of writing to the selected/sorted list.  Maybe it would be possible to create a TransformList subclass that implements this sort of behavior...

I could maybe try to take advantage of the getSourceIndex() method in the selection list, but I think by that point that I may be 4 or 5 lists down from the BasicEventList, so there may be several index transformations between the select list and the root of the list chain.  Without having access to the parent list (and TransformedList doesn't appear to provide that) of each list in the chain, there doesn't seem to be any way to make it work...

Thoughts?

- K

----------------------- Original Message -----------------------

From: Rob Eden <[hidden email]>
To: Kevin Day <[hidden email]>, [hidden email]
Cc:
Date: Thu, 15 May 2014 12:11:15 -0500
Subject: Re: Snapshot selected items from a list

Have you tried wrapping it in a FreezeList? If I understand what youre trying to do, you could freeze it, make changes and then thaw.



Rob


On May 15, 2014 at 12:09:24 PM, Kevin Day ([hidden email]) wrote:
Here's the use case:

We have a JTable backed by a GL sorted list. Users can choose rows in the table (or the entire table) and perform an operation on the selected items. These operations can be long lived (~1 second per list entry, and there can be hundreds of thousands of entries that we are processing).

As we process the items, we need to replace the current element with an updated version (we use immutable objects in our lists).

We obtain the list of selected entries using DefaultEventSelectionModel.getSelected() (and if nothing is selected, we use the SortedList that backs the table).



Previously, we made the operation dialog modal - while it was running, users couldn't continue to click on the table (selecting different rows, changing sort order, etc...). This worked very well - we could loop through the list of elements to work on, get(i), do our work, then set(i, updated). This magically propogated the updated element back into the source list (presmuably b/c the child list maintains index information back to the parent).


We are now trying to make the table so it *can* be interacted with, even though the long operations are ongoing. The problem is that users may change their selected rows, or adjust sort order, while we are in the middle of processing. Because of this, the get(i) and set(i, updated) calls can't be counted on to remain stable throughout our run.



So... the question is: is there some way to take a snapshot of the list returned by getSelected() and/or of the SortedList - one that won't change underneath us as the user changes their selections and sort, but will still allow us to efficiently push updates back into the main list?

I could see making a new sorted list instance that has a copy of the current comparator (not sure how hard that is, but I'm pretty sure I could figure it out). The selected list, though, seems like a much harder nut to crack. I kind of want to grab a static view of the selected list at a point in time, and then prevent any future table selection changes from modifying the static view. And changes in that 'static' view need to propogate back into the primary list.

Thanks in advance,

- K




Loading...