SPIDERING Website using Burp Spider

Burp Spider is a tool for automatically crawling web applications. While it is generally preferable to map applications manually, you can use Burp Spider to partially automate this process for very large applications, or when you are short of time. - http://portswigger.net/burp/help/spider_gettingstarted.html

Note: Using Burp Spider may result in unexpected effects in some applications. Until you are fully familiar with its functionality and settings, you should only use Burp Spider against non-production systems.


Before performing this, ensure that the proxy has been set and configured properly. Click here to know how to set the proxy up: http://securityg33k.blogspot.sg/2013/07/data-manipulation-intercepting-http.html

Click on Spider and Tick the 'spider running' option

In this example, we are not going to perform any traffic interception. Go to Proxy and ensure that the 'Intercept is Off'

In this example we are going to use a testing site by Acunetix: http://testphp.vulnweb.com

Now go to the 'Target' tab in Burp Suite and you will notice a set of links. Right click on the link that we want to spider and choose 'spider this host'

Now go to the 'spider' tab and you will notice the spidering action being performed.

Once done, go back to the 'Target' tab and expand the list under the site you Spidered. This will give us the list of items in the website for analysis and review.

Download Burp Suite here: http://www.portswigger.net/burp/download.html