Anyone ever built their own prefab home?

[Maria Ferre]

I really want to invest in a vacational property near a lake in CT. To use for myself and then rent out short term the days I'm not using it. 

I'm thinking of buying a small piece of land, and putting a pre-fab home on it.

Has anyone ever done this before? Is this a crazy idea? Can I get financing to do this?  Welcome any thoughts!