Big Data in Real Estate

[Nicholas Varner]

In today's post, I want to use experience I've had in two fields that I really love: databases and real estate. Specifically, with databases I want to go over new techniques using more data and varieties of data as it pertains to real estate. First of all, many companies store data using Excel spreadsheets, flat files, or Quickbooks and there's nothing wrong with these techniques, until there's something wrong with them. These tools suffer to scale very well, which means that as you grow your business you will find it increasingly difficult to manually input sales leads, employees, contractors, accounting, business processes, and so on. However, you have to start somewhere and you can migrate this data to more robust solutions later on. Deciding on what tool to use is important, whether to rent using SaaS or buy is another important decision, but deciding what kind of company you want to be is the most important question to ask yourself.

What kind of data is important to store? Maybe what you think is important today won't be important tomorrow or vice versa. Not all the data you want to know or store are numbers that easily fit into spreadsheets. Are there competitive advantages that can be gained from crunching numbers in real estate? Yes. Does it tell the whole story or the most important part of the story? No. I argue that the numbers, or the "cash flow" is less important than most investors believe it is. With some effort, anyone can collect public available data like: listing prices, sold prices, square feet, year built, rent amount, taxes, etc. What kinds of data are less accessible? How about perspective tenants phone calls in that area? Their jobs? Their criminal records? Their rental history? How do you measure all of that?

What about a home inspection report? I would argue that the condition of the home is far more important than the cash flow. Although, I disagree with the gloom and doom nature of many home inspection reports, i.e. the house is not falling over tomorrow, the condition of an individual home matters far more than the numbers you put into a software program. Mother nature and individual properties do not care about your numbers, nor do your tenants. All of whom, can and will alter those numbers and distort your cash flow.

I just provided some examples of a variety of data that cannot be captured in Relational Databases, like Excel. Pictures, text files, videos have to be stored in NoSQL databases. Quantifying non-structured data is possible using metadata and analysts in order to make analysis easier and more meaningful for decision-makers. Why is any of this important?

You can definitely do business without any of this and many do business without any of these tools. However, you are doing business without capturing a more complete picture of the surrounding landscape. Are you just guessing? For example, what if one company established a tenant call line and they could accept or deny the tenant just based on their voice? What if you could store video for every single property and never show the property to tenants or buyers? What if you could measure air temperature every 10 seconds to minimize heating and cooling costs? What if you could measure water use every 10 seconds and identify tenants that are abusing water? What if you could know the condition of the property without getting off your sofa? Aggregating structured and non-structured data and knowing what to do with it has been the same challenge we faced since the beginning of time. Now, it's just about harnessing our capabilities, tools, and the data to make great decisions.

Data can choke you and drown you if you do not understand it, cause you to pay attention to wrong kind of data, or ignore the most important kinds of data. I think of each of us as investors as ships (some bigger, some smaller) sailing along in a sea of data with all of its variety, volume, and velocity and we have to make decisions along the way. Make the right decisions and you end up on a beautiful island drinking Pina Coladas (I'd prefer Scotch, actually) or make the wrong ones and end up in Davey Jones' locker. Happy Investing! 

[Michael Verkruyse]

If you truly wanted to get into RE big data mining, you could consider natural language classification techniques to see what patterns exist and if there are words/groups of words that predict good deals. 

You could also look at time series data for listing prices to make a statistical prediction on pricing bands which would help you make an optimal winning offer before a price drop causes an influx of competitor buyers. 

There's truly a lot out there in the realm of big data. My day job is in a field where traditional big data solutions (Hadoop on computer clusters) isn't even efficient enough, which has put the focus on resilient data structures (RDD's) and Apache Spark as a foundation for alternative solutions. A bit overkill for what you were mentioning, but it's interesting to think about!

Happy to chat more, and as always feel free to connect!