Funny you would bring this up now. I just switched to tint2 taskbar specifically for this reason. I sometimes have 2 or 3 xterms open on a desktop and need a way to know what each is doing before I go to it. Tint2's previews are perfect for what I need and if Icewm had them, I would go back to the Icewm taskbar even though tint2 is more configurable than the Icewm taskbar.

People have been asking similar features before. Showing the actual contents of a window someplace else requires new technical code which isn't currently in icewm and for which I personally don't have an interest. But you can express your appreciation for icewm most convincingly and your desire for such new functionality as well by donating.