Web Scraping is the process of automatically extracting data from websites. This technique can be employed on real estate websites like Zillow, Realtor.com and Trulia, to get property listings data, agent or owner contact details etc. This data can be used in a variety of ways by realtors and companies in the real estate sector.
Software for Web Scraping
There are several software tools using which data scraping from websites can be easily performed. If you are a software developer, there are various APIs and libraries which helps you develop web scraping software.
In this article, we are using WebHarvy to scrape data from Realtor.com. WebHarvy is very easy to use and can scrape data from any website, including real estate websites.
How to use WebHarvy to scrape Realtor.com listings?
1. Loading the property listings page
First, download and install WebHarvy in your computer. Open WebHarvy and load the website from which you need to scrape data.
WebHarvy contains a built-in browser using which you can load and navigate websites. Navigate to the page from which you wish to scrape data and click on the Start button in the Configuration section of the Home menu.
2. Starting configuration and selecting data
Once you click the Start button, WebHarvy switches to configuration mode, where you can click and select data which you need to scrape from a web page. Clicking on any data item (text or image) on the page will bring up a Capture window with various options. To scrape the text of an item (for example, property address), select the Capture Text option.
WebHarvy will identify and select all similar items on the page and display them in the Captured Data Preview pane. You can select data like property address, price, beds, baths, thumbnail image, property details page URL etc. from the listings page in this manner.
3. Handling pagination
Since property listings span across multiple pages, you should teach WebHarvy how to load the subsequent listings pages. For this, scroll down the page and click on the link/button to load the next page.
From the resulting Capture window, select the Set as Next Page Link option.
4. Following links
You can also follow the link of each property listed in the starting page, to load the property details page. To follow a link from the starting page, click on it and select the Follow this link option from the Capture window.
Once the property details page is loaded, you can click and select more data for extraction, just like how it was done in the starting page.
Video Demo : Scraping Realtor.com
The following video shows how WebHarvy can be configured to scrape property data from Realtor.com real estate listings. Codes used in the video can be found in the video description.
Try WebHarvy
Download the free 15 days trial version of WebHarvy and try scraping Realtor.com property listings. You may refer the Getting started guide
Need Help? | Have Questions?
In case you need assistance or have any questions, please do not hesitate to contact our technical support.