Great question; one I've mulled-over for about ever. I, too, like to have the option of non-holster carry. My experience with this issue developed over the 20 years during which I carried Glocks as a uniformed/duty gun. As much as I love Glocks for that purpose, I found their light trigger pull to be an impediment to non-holster carry.
And, because of the Glock/striker-fired trigger pull, I relegated the Glock to carry only in a sturdy holster. I know that some take issue with not using a holster. However, I don't share that opinion, and find the issue to be a different topic for a different thread.
Also, even when carrying in a holster, I still have the need to unholster the carried gun and put it unholstered in a side pants pocket when preparing to use a urinal. Scary, to me at least, with a striker-fired gun.
Although I find a manual safety to be a distant second to a heavier, revolver-like DA trigger pull, it's better than nothing. However, I doubt that I'd ever rely on it for any long-term IWB carry.
However, the one exception is my Ruger LC9 (not LC9s). I purposely bought the hammer-fired old version just for it's DA trigger pull. The LC9's like mine do come with both a magazine and side mounted safety. To me, both are unnecessary on a urinal-friendly pistol like the old LC9; the newer LC9s's can be obtained without either. Wish I had the option.
Going back to the original question, I guess my opinion is that there is merit to either position, depending on the gun user's carry style. And that if a safety is present, it can either be used or not used. However, on a gun without a thumb safety, you have no choice.
Consequently, I think it's important for the gun user to develop a carry/ thumb safety protocol; use or not use, then stick to it to avoid a potential deadly conflict in procedure.