Indeed is a job listings website, available in over 60 countries and close to 30 languages. It is one of the most popular job listing / job search websites globally. Job details like role, salary, description, application link, company details etc. can be scraped from Indeed.com using various web scraping tools or APIs.
Benefits of scraping job listings
For recruiters and job consultancy/placement companies, there are several benefits of web scraping job listings data. Web Scraping provides latest data related to new job postings, aggregated from multiple job listings websites at the click of a button. Having the entire data in your own spreadsheet or database makes it very easy to search and analyze it. Analysis can be used to make intelligent data-driven hiring decisions.
Which tool to use to web scrape job listings data?
There are various web scraping software tools (both online and locally installable) which you can use to scrape job listings data. You may also develop own custom web scraping software using various web scraping libraries or APIs. In this article, we are going to use WebHarvy to scrape data from Indeed.com.
WebHarvy is a visual web scraping software which can be used to scrape data from any website, including job listings websites. WebHarvy can be also used to scrape job listings data from Jooble, Monster, Google Jobs, LinkedIn etc. WebHarvy allows you to select the data which you need to scrape via simple mouse clicks.
Scraping Indeed Job Listings using WebHarvy
WebHarvy contains a built-in browser within which you can load any web page. For scraping Indeed, first load the job listings page from which data is to be extracted. (watch video below)
To start selecting data to scrape, click the Start Configuration menu button. You can then click and select the details which you need. When you click over text or images displayed on page, WebHarvy will display a Capture window with various options. Click on the ‘Capture Text’ option to scrape the text of the item. Details like Job title, salary, category etc. can be selected from the starting page using this method.
WebHarvy can scrape data from multiple pages of listings. For this scroll down to the bottom of the page and click on the link/button to load the second page of listings. From the resulting Capture window, select the ‘Set as Next Page Link‘ option.
To follow each job listing link to load its details page, click on the first listing title/link and select the Follow this link option from the resulting Capture window.
Once the job details page is loaded, you can click and select data from it, just like from the starting page. After selecting all required data, stop configuration. You may now optionally save the configuration. Click the Start Mine button to start mining using the configuration. This will bring up the Miner window. Click the Start button to start mining. As mining proceeds, you should be able to see the data pulled from Indeed.com in the data table.
Once mining finishes, you can save the scraped data to a file or database.
Video : Scraping Indeed.com Job Listings
The following video shows how WebHarvy can be configured to scrape Indeed Job Listings data.
Download WebHarvy Free Trial
You may download and try the 15 days free evaluation version of WebHarvy from our website. Getting Started Guide.
If you have any questions, please do not hesitate to contact our technical support team.