Can someone who has tons of attachments/gallery images/files tell me if the content which the My Media button displays ever becomes paged or if it just dumps every item you have all at once in the appropriate tab.

Yes it has pagination over a certain number of items :)

Thanks!