@Jenni Moore, I'm speaking from a theoretical perspective, as I have not done this type of renovation, but here's what I understand. Improvements should be paid off by about 2 years worth of increased rents, based on what I've read. If it takes too much longer, its probably not worth doing.
That would put you at $7,200 to $9,600 to spend on this conversion. If the garage space is 22' x 10" (typical size), this would be about $38/s.f. A typical basic new build home is around $110/s.f and you are not building a roof or foundations, so you should be in range, depending on the specifics of your home.
One question is if your existing heating and cooling systems are capable of handling the increase load, or if you need to do a dedicated unit in the attic for this room. HVAC should run around $10-$15/s.f. as an average placeholder.
I'd make sure to have the floor levels are the same as the other bedrooms if possible, meaning you will have to frame a wood floor over the garage floor. Also make sure you plan for extra layers of sheet rock to keep a fire rated wall between the garage and the new bedroom.
Here's my rough estimate (keep in mind I'm not a contractor):
1. HVAC system new or upgrades ($3,500)
2. Raised floor structure ($2,000)
3. Gypsum board walls - about $5/s.f. of wall area ($2000)
4. Closet doors and bedroom door ($500)
5. Lighting and electrical ($600)
6. Removal and infill of garage door, new siding ($1,000)
7. New window where garage door was located ($500)
8. Carpeting ($650)
9. Paint ($750)
Subtotal $11,500
10% contingency ($1,150)
Total $12,650
Reduction to only adapt existing HVAC system ($2,500)
Revised Total $10,150
I think the HVAC system is a big determining factor of this decision. Hopefully there is enough existing tonnage to make this work. My opinion on this is that it is worth doing if you have some level of confidence of the increased rental numbers. Have you checked Craigslist or Rentometer.com to confirm your understanding of the difference of an extra bedroom?
On the basement renovation, I would not pursue that as I don't think you will get the returns unless you do a serious renovation to add a bedroom and bath with an egress window.
I hope that is helpful. Maybe other BP'ers can give some feedback on my numbers. Congratulations on your first big deal!
Rich