When listview destroys the object during dragging, it causes the drag to fail. The source of drag should always be loaded.
Setting cacheBuffer to contentHeight ensures that the listview would not purge out of sight items.
This fixes the issues during scrolling and dragging items in the playqueue.
Having all the items loaded at all times increases average memory consumption, but it seems there is no alternative. I tried increasing cache buffer only when dragging is active, but it caused major stutters, especially when the item count was over 500.
VideoLAN code repository instance