Job Details Extraction
WebHarvy can be used to scrape data from job listings at various job search websites. You can find a list of demonstration videos related to this topic at the following link.
Extracting Job Details from various websites using WebHarvy
Google Job Listings Extraction
In this article we will see how WebHarvy can be configured to extract job details from Google job listings. When you perform a general Google search for a specific job, Google will display a card with openings relevant to your search in your area as shown below.
If you click on the ‘more jobs‘ link at the bottom of the card, the following page will be displayed.
The following video shows how WebHarvy can be configured to extract data from job listings in such pages.
The job listings page loads more jobs in the same page as you scroll down the left hand side list of jobs. So to configure pagination (to extract data from multiple pages by repeatedly scrolling down the page), we will have to use JavaScript. This is explained in the following link.
Configuring pagination using JavaScript
Once the required list of jobs are loaded by repeatedly scrolling down the page, we will need to do another task, before we can start extraction. The job listings newly displayed while we scrolled down the page are loaded in a different level inside the HTML source of the page. So to bring all listings to the same level we will need to run another JS code snippet.
You may find both the above JS codes here.
Such JS codes are not required to scrape data from majority of websites. Data can be selected from multiple pages using the simple point and click interface as shown in our demonstration videos here.
But in cases where the simple approach does not work, WebHarvy offers powerful under the hood features like capability to run custom JavaScript code on page to perform DOM manipulation and other actions like loading multiple pages.
Try WebHarvy
If you are new to WebHarvy, we highly recommend that you visit the following link to know more and get started with data extraction using WebHarvy.