You want to confirm rent, security deposit amount and all terms of the lease. 

Review their lease agreement and addenda.

The heat exchangers fail most frequently on the propane boilers due to the high acidity, low temperatures the propane condensing units run at, my friend who owns a 3rd Generation heating/plumbing company doubts those units will ever come close to outlasting a traditional 3 pass boiler.Agree the pros of having a propane boiler/furnace-fuel is cleaner, services tend to be cheaper, tanks can be bought or fuel company can supply tank, work great if you also have a tankless hot water heater with same fuel source.Efficiency of Maine does offer rebates as well for the high efficiency units, they don't mention this but, if you insulate all baseboard piping that is exposed in the basement, take photos and have a high efficiency oil boiler they will qualify as the same rebate as the propane unit.

Or I may just keep leasing it.

The switch is a PITA and a notorious problem with other well owners down here because the switch is outside and exposed to the elements and fireants and frequently fail.

As a beginner I would look at wholesaling or lease options. 

Subject To’s and Lease Options are going to be the next play for me.

Their lease is only for the square footage inside the walls, correct? 

Put it in writing that he’ll be out on xxx and will continue to pay $yyy via the lease dated xx.

Review their lease agreement and addendums.