Better relationship between File Entries Tag and Channel Categories Tag?

I think it’s great that you can organize files by directories and categories, and then use the meta information about files in your templates.

However, the Channel Categories Tag seems to only be aware of channels, not files associated with it. Meaning, if I just want to use a category group to organize files there is NO WAY I can use the Channel Categories Tag in my templates to organize and display the related files because the Channel Categories Tag requires the channel parameter (unless you only have one channel.)

Is the only way to truly relate files to channels to create a redundant “files” channel so I can relate different channels to those files instead of just associating entries with the files themselves?

This is what I’d love to do:

{exp:channel:categories category_group="3" directory_id="2" show_empty="no"}
<h2>{category_name}</h2>
{exp:file:entries category="{category_id}" directory_id="2"}
<a href="http://{file_url}">{title}</a>
{/exp:file:entries}
{/exp:channel:categories}