Anyone use APIs to get good information?

Question

I just finished a program to pull data from the Zillow API and it works but was wondering if anyone knows of any APIs with better data that are free or cheap...

Jon Schwartz · Answer

@Kevin Keck, I'm not a programmer, so I've been downloading and importing into Excel a ton of data by hand. I've worked out some good functions for analyzing supply-and-demand trends, growth volatility, etc. I'm not a programmer, though! Maybe we can help each other out? I'd love to be able to pull data directly via APIs!

Here are my top free resources:

US Census Bureau American Community Survey 1-Year Data API:

https://www.census.gov/data/developers/data-sets/acs-1year.html

This'll give you annual MSA-level data for so many metrics: industry diversity, household income, household makeup, educational attainment, unit counts, vacancy, median rents. So much value here!

Bureau of Labor Statistics API:

https://www.bls.gov/developers/home.htm

Here's where you get monthly unemployment data. For us, the growth is jobs is a more important metric than the unemployment rate.

Great Schools API:

https://www.greatschools.org/api/request-api-key

This'll give you school ratings for the assigned schools near a property. Priceless! Except, not entirely free: there's a per-call cost after 3000 daily calls (so an individual can basically use it for free).

SpotCrime unofficial API:

https://github.com/skyline-ai/spotcrime

I don't actually understand what's on this page, but if you can use this resource to pull calls from SpotCrime, it'll be a godsend.

Let me know what you think! Are these useable?

Trevor Ewen · Answer

@Jonathan Schwartz

Thanks for this! I am working on a project now using the Bureau of Labor Statistics API. However, I was not aware of the greatschools one.

Hard to tell usability without integrating with them. I found the BLS API to be a bit cumbersome, but it solved my needs for my current project.

Account Closed · Answer

For those that may still be looking into this... First American and Corelogic are going to be the best bet for any data building models. That is, if you want 'accuracy'. They are the pretty much the main source fueling the rest of the aggregators, when 'quality' counts.

Corelogic prolly has the edge on anything pertaining to 'valuations' and AVM's...but First American is right there chin-to-chin with them. They both have been collecting property data since time immemorial. There is also a history of the two joining as one company, swapping data (swapping spit) and something will happen where First American will spin off on its own again. This has happened a couple of times.

Corelogic's tool 'Realist' is baked into 80% of the MLSs nationwide, providing the tax/sales/people data fueling the comps/CMA many realtors use for a quick look.

Attom bought Realty Trac. It's Largely welfare data from Corelogic https://www.ftc.gov/news-event... . They just sued Corelogic again, within the last two years or so, because the welfare data they get was largely crap, if that tells you anything of what was being sent to them. There was a distinct difference in hat Corelogic sent Attom under the court-ordered welfare agreement, and what Corelogic supplied directly to consumers. Buying from Corelogic direct is the way to go.

I will say Attom has upped its game in the last few years concerning ML and AI.... so who knows, they might actually have cleaner data now, and overall it might be cheaper to buy from them as a reseller.

First American are quality people, and they are scrappy... they have a huuuge database of scanned doc images that they have run ML against for the juicy tidbits of data a county/city does not collect or load onto public-facing .gov sites.

Most of the data you are looking for, like accurate monthly parcel/assessor/people ect data can be had just by googling your county name with 'open data GIS' in the search string. A modern county GIS dept will make a variety of datasets available as .csv, and some manner of GIS file like a .shapefile or KML, geofile ect. The bonus of that? It's a staggering amount of data, countywide, that is simple to load into a PostgreSQL-PostGIS install. It's stupid simple to do, and most realtors are oblivious that they can look at data on that scale, let alone outside of their MLS system. Go to your local county site, look for the nifty 'search for a property in the county' map, and look for the download button. If they do not have online, simply put in a Open Records Data request to the county.

For anyone looking to do this in Florida, MN, AZ, VA, or AL, https://propertykey.com/compan... is prolly your best bet... they're likely THE BEST dataset in use by realtors for maps/parcel/assessor/sales/people data itjustkeepsgoingonandon what they have. I could arrange an introduction with the team behind the data, I know them well, and they are just terrific people. Total 'makers' when it comes to property/GIS data for 25 years.... and most likely far less expensive as a reseller than some of the options out there.

MLS data? Scrape it. Scrape ALL of it. NAR and it's outdated approach to MLS data is OV/er https://www.eff.org/deeplinks/... . Just make sure to use rotate your Ip's on the regular, or set up some sort of proxy...

The County parcel datasets are the easy thing to obtain... It's the fresh sales data (what sold today/yesterday, to 'who', from 'who') is what's critical for any ML models you are creating. That's where the API will come into play for fresh sales.

Hope this helps~

Mary Gansallo · Answer

Hi Kevin, I stumbled upon this post via searching around. Wondering if you have a github or something similar for this program? Would love to check it out.

Thanks!

Kevin Keck · Answer

@Steve B. That's true but it's 1000 a day so plenty for me and most people. There data is definitely questionable, but probably the best that's out there for free. @Michelle Phelan I keep hearing about that one thanks I'll check it out.

Michelle Phelan · Answer

@Kevin Keck

Redfin has some good data in NY at least. I can pull actual rents and statuses of rentals. However Redfin isn’t everywhere yet. Also I find VT to have very limited data.

Steve B. · Answer

@Kevin Keck pretty sure Zillow caps GET requests on their APIs.  I don’t know if the have a paid option for more requests.  You could screen scrap with Python but the new Zillow format makes that way harder.  They really have made the website much worse in general with their goofy focus screens.

Kevin Keck · Answer

@Peter Jaglom Yeah I'm still working on this program's front end but its almost done. I have some economics coursework so I can analyze a rental property pretty well, but I don't know much beyond that as far as analyzing an entire market. I'll try to give anyone access to it when it's finished. If you're interested in hiring someone for your software idea, a good friend of mine is getting his MBA in data analysis from Duke University and just started a consulting LLC.

Peter Jaglom · Answer

@Kevin Keck, I was just thinking about writing algorithm descriptions for some API-sourced programs to 1) analyze one or many specific properties, and 2) analyze a market to determine suitability for rental/BRRRR. I'm also not a programmer so I was planning to hire someone online to turn my descriptions into actual code. I'm also trying to figure out if mixnode (https://www.mixnode.com) could be used to glean some good info from sites that might not have an API for the data I want.

PM me if you have any interest in trying to collaborate on a program, and either way I'd also be very interested in your zillow API program if you're willing to share!

Kevin Keck · Answer

@Jonathan Schwartz Ok

Anyone use APIs to get good information?

Most Popular Reply