When assessing large environments to include Content Delivery Networks (CDN), you will find that you will be identifying hundreds of open web ports. Most of these web ports have no active web applications deployed on those ports, so you need to either visit each page or request the web page header. This can simply be done by executing a HEAD request to both the http:// and https:// versions of the site. A Python script that uses urllib2 can execute this very easily. This script simply takes a file of the host Internet Protocol (IP) addresses, which then builds the strings that create the relevant Uniform Resource Locator (URL). As each site is requested, if it receives a successful request, the data is written to a file:

#!/usr/bin/env python
import urllib2, argparse, sys
defhost_test(filename):
    file = "headrequests.log"
    bufsize = 0
    e = open(file, 'a', bufsize)
    print("[*] Reading file %s") % (file)
    with open(filename) as...