First to get a better understanding of cap rate, flip the formula around. Purchase price x cap rate = NOI. Basically the cap rate is what % interest rate you are going to make on the property. So if you buy a $100,000 property at a 5 cap you in theory should be making 5%, after all expenses, on an all cash purchase. This is one reason that people have been saying cap rates would be going up because why would you buy RE at a 5 cap (5%) when you can take that money, in todays interest rate environment, and invest it in safe investments that take zero of your time and don't have unseen expenses, and get 5% or better? People are still buying the RE because of the other benefits such as value ad upside, leverage, tax benefits, etc. As Bradley pointed out they can be all over the place. From my experience small properties don't seem understand cap rates, or even care about cap rates as much. If small enough they seem to look at house values. The bigger you get, the more cap rates actually play into it. Example, in one of the markets that I am in there are two different trip plex properties. Both listed for around $499,000. However one has $3,700/mth in income while the other has $1,875/mth. Both have about the same expenses as they are similar properties in a similar area. The one has a 7.5 cap while the other has just over a 3 cap. When you start talking larger multi family properties and commercial buildings (office, retail, industrial) then cap rates become a much bigger factor.
As far as what cap rate is a fair cap rate, well that is up to each of you to decide if it is a fair cap rate and the buyer and seller may not agree on what is a fair rate. Best you can do is look at what cap rate similar properties in your area are selling for. Other things that impact the cap rate can be premium location, newer age for the property vs old building, quality of tenants, etc.