Hi,
for non-production use and if it is just a catalog without custom coding running I see no problem in using one MDM server for multiple SRM systems (especially sharing one for sandboxes and dev). It is a setup I have seen multiple times and it worked always without problems.
You should set up different repositories for each SRM system so that you don't interfere with each others data (i.e. suppliers or other lookups tied to SRM).
You have to keep in mind that you have to upgrade all systems together. But you can always use separate MDM servers during the upgrade process and then consolidate onto one server when the upgrade is done.
If you are running custom coding with your MDM catalog I would recommend having a separate MDM system for the QA SRM in order to be closer to the production set-up when testing your custom solution.
Regards,
Martin