How to scrape TripAdvisor reviews and ratings ?

WebHarvy can be used to scrape data from TripAdvisor website. In this article we will be see how WebHarvy can be configured to scrape reviews and ratings from multiple listings at TripAdvisor website.

By default, TripAdvisor does not display the complete review text in its listings pages. You will have to click a ‘Read more’ link at the end of each partially displayed review, to view the complete review. This can be automated using WebHarvy as shown in the following video.

 

Regular expression strings are used to correctly select the date of review, and also the rating numerical value. The rating value is selected from the HTML source of the rating stars displayed by the website. The RegEx strings used are copied below.

wrote a review (.*)

rating bubble_([^”]*)

We have several videos in our YouTube channel related to TripAdvisor data extraction. You may watch them at the following link.

TripAdvisor Scraping Videos using WebHarvy

Try WebHarvy

We recommend that you download and try the free evaluation version of WebHarvy. To know more please follow the link below.

Getting started with data scraping using WebHarvy

How to scrape data from eBay product listings ? (price, images, specification, seller description etc.)

WebHarvy can be used to easily scrape product data from listings at eCommerce websites like Amazon, eBay etc. We have an entire playlist of demonstration videos related to eCommerce data extraction in our YouTube channel.

eBay Data Scraping

In this article we will see how WebHarvy can be used to extract product data from eBay listings. Details like product name, price, product URL, item specifications (condition, weight, UPC/MPN etc.), seller description etc. can be extracted. WebHarvy can also extract product images (thumbnail as well as high resolution images) from eBay product listings. The following video shows the steps involved.

The JavaScript code used in the above video to open seller description as a separate page is copied below.

location.href = document.getElementById(‘desc_ifr’).getAttribute(‘src’);

More videos related to eBay data extraction

Try WebHarvy

In case you are interested in exploring more, we highly recommend that you download and try using our free evaluation version. To get started, please follow the link given below.

Getting started with web data scraping using WebHarvy

 

Scraping data from paginasamarillas.es | extracción paginas amarillas

In this article we will see how WebHarvy can be used to extract data from Spanish Yellow Pages website – paginasamarillas.es

Paginas Amarillas Data Extraction

WebHarvy can extract data like business name, address, website, email and phone numbers from paginasamarillas.es listings. The following video shows how this can be done. Most of the details except email address (which is not directly displayed by the website) can be selected by directly by clicking on them during configuration. Email address can be selected from the HTML source of the business details page by applying regular expressions. The Regular Expression string used to extract email address is copied below.

customerMail[^;]*;[^;]*;([^\&]*)

Try WebHarvy

To know more we highly recommend that you download and try the free evaluation version of WebHarvy. To get started, please follow the link below.

Getting started with web scraping using WebHarvy

 

Build an Yellow Pages Australia Scraper

WebHarvy is a visual web scraping software which can be easily configured to scrape data from any website. In this article we will see how WebHarvy can be used to scrape data from Yellow Pages Australia listings.

Yellow Pages Australia Scraper

The built-in browser of WebHarvy allows you to load any website and build a web scraper for that website. Data which you need to scrape can be selected using mouse clicks. WebHarvy will automatically identify and scrape repeating data of the same kind from a web page. It can also automatically scrape data from multiple pages of listings and also by following links to fetch detailed data.

The following video shows how WebHarvy can be configured to build an Yellow Pages Australia Scraper. A special technique is employed to select data correctly and consistently from yellowpages.com.au listings. This is mainly because the layout of boxes of listings vary from one listing to another – some has header with their logo/image, some does not etc.

The regular expression strings used in the above video can be found in the video description.

Know More

We highly recommend that you download and try using the free evaluation version of WebHarvy available in our website. To get started, please follow the link below.

Getting started with web scraping using WebHarvy

Scraping Zillow for real estate data and agent phone numbers

WebHarvy can be used to easily scrape data from real estate websites like Zillow, Realtor, Trulia, RedFin etc. In this article we will see how real estate data including agent/owner contact details (phone numbers) can be extracted using WebHarvy.

Scraping Real Estate Data from Zillow

The following video shows the steps involved. You can see that data like property address, price, zestimate, beds/baths, area, property facts and features (like type, year built, parking etc.), pricing history, tax history, neighborhood details etc. can be easily selected for extraction using a point and click interface. WebHarvy will automatically scrape the data which you select from multiple properties listed across multiple pages in Zillow.

Update (June 2021) : Due to recent changes in Zillow website, a new technique has to be used to scrape all 40 properties which are displayed on each page. Please watch this video to know more.

Scraping agent phone numbers from Zillow

The following video shows how agent phone numbers can be scraped from Zillow property listings. The ‘contact agent’ button needs to be clicked in each property details page to get the agent contact details.

Try WebHarvy

We recommend that you download and try with the free evaluation version of WebHarvy available in our website and avail our free technical assistance for your first data scraping project. To get started please follow the link below.

Getting started with web scraping using WebHarvy

Scraping Flashscore – Statistics of all matches in a league

The following video shows how match statistics (possession, goal attempts, shots on goal, blocked shots, corners, off-sides  etc.) of all matches in a league from FlashScore website can be extracted using WebHarvy.

In addition to FlashScore, WebHarvy can also be used to extract sports betting odds from many other betting sites like BetExplorer, OddsPortal etc.

The regular expression string used in the video to get match ID is :

id=”([^”]*)

To form the URL, the following string is replaced :

g_1_

with

https://www.flashscore.com/match/

Know More

We recommend that you download and try using the free evaluation version of WebHarvy available in our website. To get started, please visit the link below.

Getting started with web scraping using WebHarvy

Scraping latitude, longitude from Yellow Pages listings – GPS Coordinates Extaction

Yellow Pages business listings often display the location (Map Direction) of the business. The location details are displayed on a map interface. But the latitude, longitude values (GPS coordinates) are not displayed on page. However, this information is present inside the HTML code behind the map interface.

Extracting latitude, longitude values

The Capture HTML feature along with Apply RegEx feature of WebHarvy can be used to extract the map coordinates from the HTML code of the page. The following video shows how this can be done. The Regular Expression strings used in the video are copied below.

data-lat=”([^”]*)

data-lng=”([^”]*)

Try WebHarvy

We recommend that you download and try using the free evaluation version of WebHarvy available in our website. To get started, please follow the link below.

Getting started with web scraping using WebHarvy

Yellow Pages Scraper – Scrape Email, Website, Phone Numbers and Addresses

WebHarvy is a visual web scraping software which can be easily configured to scrape data from any website including Yellow Pages. There are various flavors of Yellow Pages websites. In this article we see how WebHarvy can be configured as an Yellow Pages Scraper (www.yellowpages.com).

YellowPages.com Data Scraping

Yellow Pages websites are the go to place for contact details related to any business. And for the same reason it is one of the greatest sources of business/professional contact details. The following video shows how easy it is to use WebHarvy to scrape details like phone number, website address, address etc. from yellow pages listings.

Scrape search results for multiple keywords

The above video also shows how you can automatically submit multiple search keywords at yellow pages website and scrape the resulting data. This feature is called Keyword based Scraping and is explained in the following link.

WebHarvy Keyword based Scraping Explained

Multiple lists of keywords can be provided (example: one list for search and another for location) and WebHarvy will automatically submit all combinations of input keyword lists and scrape the resulting data.

We recommend that you download and try using the free evaluation version of WebHarvy to know more. Please follow the link below to get started.

Getting started with web scraping using WebHarvy

Scraping Amazon by submitting a list of ISBN numbers

The Keyword Scraping feature of WebHarvy lets you submit a list of keywords (search terms, ASIN, ISBN etc.) at Amazon and extract the resulting data displayed. WebHarvy supports submitting multiple lists of keywords to multiple search fields (ex: search query + location) in a website and scrape results for all combinations of submitted keywords. To know more please follow the link given below.

Keyword based Scraping Explained

The following video shows how this feature can be used to extract data from Amazon for a list of ISBN numbers. Details like book title, author, reviews, publisher, cover image etc. can be extracted. The same technique can be used to extract product data corresponding to a list of ASINs.

Know more about Amazon product data extraction using WebHarvy

We recommend that you download and try using the free evaluation version of WebHarvy. For more details please follow the link below.

Getting started with web scraping using WebHarvy

Scraping data from a list of URLs

WebHarvy can scrape data from a list of URLs, provided that they all belong to the same website/domain and share the same layout/page design. This technique is explained in the following link.

How to scrape a list of URLs using a single configuration ?

To any WebHarvy configuration (built to extract data from a page / website), you can add additional URLs as explained here. This can be done while creating the configuration, or while editing it later.

The following video shows how a list of Amazon product page URLs can be scraped using WebHarvy.

To know more, please visit the link below.

Getting started with web scraping using WebHarvy