Let's say we had the ability to weapon swap. What even would we swap to? We have rifle/pistol/shield/hammer/sword, the latter two of which are mutually exclusive so can't use them together. Which leaves only rifle/hammer on scrapper which is actually a terrible combo. As you said holosmith does well with just a sword and photon forge and pistol is condi weapon anyway which makes it unsuitable for use with hammer or rifle.