ICollectionView's SourceCollection is null

I have a ViewModel with two ICollectionViews which are bound as ItemsSources to two different ListBoxes. Both wrap the same ObservableCollection, but with different filters. Everything works fine initially and both ListBoxes appear properly filled.

However when I change an item in the ObservableCollection and modify a property which is relevant for filtering, the ListBoxes don't get updated. In the debugger I found that SourceCollection for both ICollectionVIews is null although my ObservableCollection is still there.

This is how I modify an item making sure that the ICollectionViews are updated by removing and adding the same item:

private void UnassignTag(TagViewModel tag)
{
    TrackChangedTagOnCollectionViews(tag, t => t.IsAssigned = false);
}

private void TrackChangedTagOnCollectionViews(TagViewModel tag, Action<TagViewModel> changeTagAction)
{
    _tags.Remove(tag);

    changeTagAction.Invoke(tag);

    _tags.Add(tag);
}

The mechanism works in another context where I use the same class.

Also I realized that the problem disappears if I register listeners on the ICollectionViews' CollectionChanged events. I made sure that I create and modify them from the GUI thread and suspect that garbage collection is the problem, but currently I'm stuck... Ideas?

While debugging I realized that the SourceCollections are still there right before I call ShowDialog() on the WinForms Form in which my UserControl is hosted. When the dialog is shown they're gone.

I create the ICollectionViews like this:

AvailableTags = new CollectionViewSource { Source = _tags }.View;
AssignedTags = new CollectionViewSource { Source = _tags }.View;

Here's how I bind one of the two (the other one is pretty similar):

<ListBox Grid.Column="0"  ItemsSource="{Binding AvailableTags}" Style="{StaticResource ListBoxStyle}">
            <ListBox.ItemTemplate>
                <DataTemplate>
                    <Border Style="{StaticResource ListBoxItemBorderStyle}">
                        <DockPanel>
                            <Button DockPanel.Dock="Right" ToolTip="Assign" Style="{StaticResource IconButtonStyle}"
                                            Command="{Binding Path=DataContext.AssignSelectedTagCommand, RelativeSource={RelativeSource AncestorType={x:Type tags:TagsListView}}}"
                                            CommandParameter="{Binding}">
                                <Image Source="..."/>
                            </Button>

                            <TextBlock Text="{Binding Name}" Style="{StaticResource TagNameTextBlockStyle}"/>
                        </DockPanel>
                    </Border>
                </DataTemplate>
            </ListBox.ItemTemplate>
        </ListBox>

I use MvvmLight's RelayCommand<T> as ICommand implementation in my ViewModel:

AssignSelectedTagCommand = new RelayCommand<TagViewModel>(AssignTag);