Having a storefront in a primary residence

[Jessie Rowe]

Hi there, 
We're subdividing our primary residence into a 4-unit multifamily, which is allowed under our zoning and terms of our mortgage. However, we live in a tiny town (not Denver but town is in Colorado) and our property is on a busy street. 

Arguably the best and highest use of the street-facing unit is a storefront. I'm hearing that violates our primary residence mortgage. If you were super-set on the idea of having a storefront, how would you do it? Is there a way to get around this mortgage easily? 

We were really hoping to avoid refinancing (we closed 2 years ago with a low rate by today's standards)