How to scrape data from YellowPages.com.au using WebHarvy?
WebHarvy contains a built-in browser using which you can navigate to the web page from which you need to scrape data. For scraping Yellow Pages Australia, load the listings page from which you need to extract data and click on the Start configuration button.
You can now click and select any required data item from the page for extraction. To scrape the name of the listing, click on the name of the first non-ad (non sponsored) listing and from the resulting Capture window displayed, select the Capture Text option.
You can select other details like phone number, website, address etc. from the listing page in this manner. But before we proceed to select other details, if we scroll down the page, we can see that beyond a limit, the listings are in collapsed mode. We need to expand them. For this, click on the More Info link and select the Click option from the Capture window displayed.
Once this is done, you can select the remaining data from the listings page, namely the phone number, website address, business address etc., by directly clicking over the text and by selecting the Capture Text option. WebHarvy will automatically parse and select all similar items from the page and display them in the Captured Data Preview pane.
Since the listings span across multiple pages, in order to teach WebHarvy how to load and scrape data from all subsequent pages, scroll down the page and click on the next page link and select the Set as next page link option from the Capture window.
To follow each listing link to load its details page and scrape further data, click on the title/link of the first listing and select the Follow this link option from the Capture window.
Once you have selected all required data, you can stop configuration by clicking on the Stop button in the top menu. You may now optionally save the configuration so that it can be run later. Saved configurations can also be edited in case you wish to add or remove data fields from the configuration. Click the Start Mine button to start mining data using the configuration. In the resulting Miner window displayed, click the Start button and WebHarvy will start to scrape data from the website for which we created the configuration.
Scraped data can be saved to a file or exported to a database. WebHarvy also allows you to schedule web scraping tasks so that they run without user intervention and fetch the latest data to a file or database.
The following video shows how WebHarvy can be used to scrape details like name, website, email, phone number and address from yellowpages.com.au listings.
We recommend that you try the free evaluation version of WebHarvy available for download.
If you have any questions or need assistance in setting up WebHarvy for Yellow Pages scraping, please do not hesitate to contact our technical support team by providing details regarding your requirement or problem faced.