I had a very similar issue in my duplex.
I am also a big proponent of the mini split system. I have two Mitsubishi hyper heat systems in my house in northern Minnesota and they work down to -17 F. These systems are expensive but well worth the investment in my opinion. I have been very pleased with them. They have cut my heating bill by about 75 percent and the side benefit of having air conditioning which costs practically nothing to operate is a huge bonus.
One thing to consider is whether or not you will be having them run on off peak electric or not. If so, you will need to have some type of back up heating source, which you currently have with the current gas system in place, in other words use the mini-split as the new main heating and cooling and the old gas as a back up system, if they will allow you to keep in in place. You would also require a back up system whenever the temperature drops below the rated threshold of whatever heat pump that you are purchasing. It is important to do some research here.
The natural gas direct vent space heaters that are currently available on the market and one of the main issues is that they are only approximately 73-83% efficient and usually not available for tax credits or rebates. There are few units that are more efficient but are very cost prohibitive and frankly not worth the investment.