Each property has insurance on it, and in addition we have a $2 million umbrella policy.

I also carry a $2M umbrella policy that around $1,200 / year.

On top of that you can purchase umbrella policy.

There are many different insurance options you can look into, generally you would look into an umbrella policy if you are in the property with other tenants, if you have a single family home as a rental usually you would only have what they call a "fire policy" to cover the building but not the tenants belongings.

We specifically did this in order to get it to have its own tax return (when your rental income gets large enough it starts to make sense to not have it as a schedule on your personal return, so now we get a K1.The master LLC pays joint expenses like umbrella insurance etc. 

Do I also need umbrella insurance?

Adding an umbrella policy doesn't cost very much but gives you quite a bit more protection.  

They each have a mortgage with a rate between 3.5% and 4.0%, so I don't want to refinance them into an umbrella loan or anything like that.  

Do I need a separate Umbrella for every state I buy property in?  

And I opened a $2 million umbrella policy for myself personally.