Hi @Paul Moore,
After more research and listening to the advice of others here I've realized cap rate is a poor comparison to ROI. The initial scenario I laid out (fully paid off property with cap rate of 10%) is a poor premise to begin with, because most real estate investors want to use leverage and therefore won't have a property that is fully paid off with no HELOC. Since cap rate doesn't factor in your initial investment it is hard to compare it to ROI.
Many people suggested looking at IRR for a more direct comparison with ROI. I found that the IRR formula was exceedingly complicated and while I could use a computer to find IRR, I dislike following methods I can't understand. Also, I found it uncommon for IRR to be applied to properties with fewer than 4 units ,which is the size I am examining.
I have my own method for making this comparison which I have found useful and is actually pretty simple. If I want to find the ROI of a duplex over 5 years let's say, first I estimate my equity gained. Houses tend to appreciate 2% per year so over five years the house will appreciate roughly 10%. If the house value was $500,000 when I bought it, it should now be worth about $550,000 and I will have gained $50,000 in equity. I will also have gained equity due to amortization. Using 5% interest rate over 30 years for this example I will pay off roughly 8% of the loan. Let's say the loan amount was $400,000. After 5 years I have paid off (and gained in equity) $32,000.
What all this math boils down do is after 5 years I will have gained about 16% (varying be the LTV) of the initial property value. To find ROI over 5 years I just have to take 16% of the initial home value divided by my down payment. ($500,000 / $100,000) * .16 = 80% ROI, which is a little better than 12% per year.
Now the big caveat is that all of this assumes neutral cash flow. In other words all expenses even cap ex are covered by rental income and there is no positive cash flow.
You can tweak the loan amount to whatever you feel is realistic for 0 net cash flow. All-in-all it is a pretty simple method but it is useful to get an idea of expected ROI.