Disclaimer: I am not an HVAC professional, and I am very new to being a landlord.
In the short term, education may help. I have seen other landlords here post that if it's too hot inside, some tenants will open the window instead of turning down the furnace. Yes, even in the winter time.
Since you already have air ducts in the house, I'd tend to agree with your gas furnace idea. If you know that there is already a gas main in that street, it shouldn't be too expensive to get a line to the house and a meter. If it's an existing house that you're converting from electric heat, sometimes the gas company covers part of the cost of putting the line in from the street to the house.
If there isn't a gas main on that street, but there is one on the corner or otherwise nearby, it might be a little more expensive. If that is the case, talk to the owners of adjacent properties about splitting the cost of running the main down the street - they may be ready to cut down their electric bills, too.
For the gas furnace itself, you can get them in different efficiency levels - how much of the gas they turn into useful heat in the house. Right now, the cheapest kind you can buy are 80% efficient. The next step up is about 90%, and they go all the way up to about 97%. The more efficient ones cost more to buy, but are cheaper to run.
The 80% furnace will need a metal flue that goes outside. This is a piece of double-walled sheet metal pipe that is fairly large outside diameter, like 8" or so. It doesn't *have* to go straight up from the basement to the roof - it can go up, turn and run horizontally for a while, then turn and go up again - but finding a space for it in an existing house can be a challenge.
The 90% and up furnaces can be vented to the outside with PVC drain pipe. The pipe size depends on how big of a furnace you have and how many twists and turns are in the pipe, but usually, around a 3" pipe will do it. Sometimes it's easier to find a place to put a 3" PVC pipe than an 8" metal pipe. Often this pipe can go through the side wall of the house, rather than having to run all the way up to the roof. If it goes through the side wall, there are rules about how far away it has to be from doors, windows, etc.
The more efficient furnaces also need a fresh air pipe that comes from outside. This is the same size PVC pipe as the exhaust. It is often installed next to the exhaust pipe, but I don't think this is a requirement... if the furnace is in the basement, you can do things like have the fresh air pipe come in through the basement wall, and have the exhaust pipe go up through the house and out of the roof.
If you have a big, open, basement, some installers will skip the fresh air pipe and just let the furnace draw air from inside the basement. My personal feeling is that this is a bad idea, and in some places it might be prohibited by the building code. PVC pipe is pretty cheap and easy to install.
A 90% and up furnace will also need a drain line for the *furnace* - they pull enough heat out of the exhaust that the water vapor condenses out and doesn't go up the flue. This isn't a very big deal... if the furnace is in the basement, it's usually something like a 3/4" PVC pipe running to a floor drain in the basement. It doesn't have to be plumbed in like a sink; the end of the pipe from the furnace just sits on top of the floor drain and water drips out of the pipe into the drain. If you also have central air conditioning, the furnace installer can plumb the inside air conditioner drain (A-coil drain) and the furnace drain into the same pipe.
Sometimes there are federal, state, or local rebates or tax credits for adding insulation. Your electric or gas utility will probably know about them, as will people that install insulation. Sometimes adding insulation will let you use the next smaller size gas furnace, which helps offset some of the cost of the insulation.
Unless you know the wiring in the house is tip-top, I don't know if I would go with space heaters as a fix. They pull a *lot* of juice and are good at finding bad connections inside the wall. People are also not good at keeping blankets, curtains, etc off of them.
I hope this helps!
