We have introduced support for scraping multiple level categories (main categories, sub categories tree) and support for multiple input keyword lists in this release. The main features are:-
True multi-level Category Scraping
WebHarvy now supports automatically navigating category/subcategory lists of a website to extract data from the final listing pages. Know More
[vimeo 171059540 w=640 h=480]
Support for multiple input keywords
Any number of input text fields can be populated with lists of strings/keywords during configuration. WebHarvy will automatically apply all combinations of provided keywords during the mining phase. Know More.
[vimeo 171062404 w=640 h=480]
Capture window with new options
Run JavaScript on Page
Run specified Java Script code on page – know more. This option can be used to load elements on a page which cannot be done using the default navigation options (link-follow, click) provided by WebHarvy.
Input strings to text input fields
Strings to be input to text fields can now be made a part of the configuration. Know More. Earlier such parameters were automatically taken from the PostData of the configuration. But sometimes, with some websites, the PostData will not contain the input strings submitted and this option helps to correctly load the page displaying data during mining phase.
Extract data from Popups
Know More. Helps to extract data by clicking each listing link/button and get data from a popup window or a view in the same page populated by data. This is different from ‘Follow this link’ option because here the data is loaded on the same page (no page navigation) and different from ‘Click’ option because after clicking each link data has to be extracted from page before clicking the next link.
Option to smoothly scroll page during mining to load all contents (lazy loading)
Smooth scroll to page end to load elements which are loaded (for example lazy loading of images) only when the elements are made visible by scrolling down. Know More.
Select drop-down/list-box options
Select drop-down/list-box/combo-box options during configuration and mining. Again this option allows navigation to result pages when normal configuration is unable to make these selections and load the result page. Know More.
Other Minor Additions Include :-
- Improvements in automatic scraping of multiple product images
- Support for loading keyword lists directly from file
- ‘Capture Image’ option automatically enabled via HTML/RegEx method in applicable cases.
- Name downloaded image files by value obtained from a column/cell in miner data table. More.
- Allows applying ‘Capture More Content’ after selecting ‘Capture HTML’.
- Quick access to items under ‘More Options’ in Capture window via toolbar buttons.
- Minor bug fixes.
You may please download and try the latest version from https://www.webharvy.com/download.html.