Product Tour

Loading Web Pages & Starting Configuration

Selecting Data to Scrape

Following a link

Capturing data from multiple pages

Saving Configuration

Editing Configuration

Scraping Data

Export captured data

Category Scraping

Keyword based Scraping

Scrape via Proxy Server

Miner Settings

Scheduler & Command line options

How to register ?

• You can configure WebHarvy to scrape websites via Proxy Servers. To edit Proxy Settings click the Settings option from the Edit menu. To add a proxy to the 'Proxy List', provide the proxy details in the 'Add proxy' box and press the '+' button. Proxy details include proxy server address, port and authentication details (if required).

  
  
  
• To import a list of proxy addresses from a file (CSV or Text), click the Import button (the button below the + button). The proxy list file should have the following format.


• Each line of the file will describe a proxy in the following format :-
  
  proxy-address:port username password
  
  As shown above, a blank space must be used to separate proxy address, username and password. The username and password fields are optional. So in case they are absent only the proxy IP address will be present per line. Each line describes a proxy server. You may also separate proxy server information by commas (,) or semi-colons (;) instead of new-line (line by line format).


• Example Proxy List File :-
  
  http://89.199.75.41:8080 w343sa pwd123
  http://203.162.168.89:80
  http://195.144.128.14:3128
  


• In the above example, the first proxy has login credentials (username and password), while the last two are open.


• Either a single proxy server or a list of proxy servers can be used for web scraping. In case you select the Rotate proxies option, WebHarvy will automatically rotate proxies in the list periodically. Otherwise, the first proxy in the list will be used.



• Read Article: Anonymously Scrape Data

© Copyright SysNucleus − www.sysnucleus.com (contact@sysnucleus.com)