Shodan is essentially a vulnerability search engine. By providing it with a name, an IP address, or even a port, it returns all the systems in its databases that match. This makes it one of the most effective sources for intelligence when it comes to infrastructure. It's like Google for internet-connected devices. Shodan constantly scans the Internet and saves the results into a public database. Whilst this database is searchable from the Shodan website (https://www.shodan.io), the results and services reported on are limited, unless you access it through the Application Programming Interface (API).
Our task for this section will be to gain information about the Packt Publishing website by using the Shodan API.
At the time of writing this, Shodan membership is $49, and this is needed to get an API key. If you're serious about security, access to Shodan is invaluable.
If you don't already have an API key for Shodan, visit www.shodan.io/store/member and sign up for it. Shodan has a really nice Python library, which is also well documented at https://shodan.readthedocs.org/en/latest/.
To get your Python environment set up to work with Shodan, all you need to do is simply install the library using cheeseshop:
$ easy_install shodan
Here's the script that we are going to use for this task:
import shodan
import requests
SHODAN_API_KEY = "{Insert your Shodan API key}"
api = shodan.Shodan(SHODAN_API_KEY)
target = 'www.packtpub.com'
dnsResolve = 'https://api.shodan.io/dns/resolve?hostnames=' + target + '&key=' + SHODAN_API_KEY
try:
# First we need to resolve our targets domain to an IP
resolved = requests.get(dnsResolve)
hostIP = resolved.json()[target]
# Then we need to do a Shodan search on that IP
host = api.host(hostIP)
print "IP: %s" % host['ip_str']
print "Organization: %s" % host.get('org', 'n/a')
print "Operating System: %s" % host.get('os', 'n/a')
# Print all banners
for item in host['data']:
print "Port: %s" % item['port']
print "Banner: %s" % item['data']
# Print vuln information
for item in host['vulns']:
CVE = item.replace('!','')
print 'Vulns: %s' % item
exploits = api.exploits.search(CVE)
for item in exploits['matches']:
if item.get('cve')[0] == CVE:
print item.get('description')
except:
'An error occured'The preceding script should produce an output similar to the following:
IP: 83.166.169.231 Organization: Node4 Limited Operating System: None Port: 443 Banner: HTTP/1.0 200 OK Server: nginx/1.4.5 Date: Thu, 05 Feb 2015 15:29:35 GMT Content-Type: text/html; charset=utf-8 Transfer-Encoding: chunked Connection: keep-alive Expires: Sun, 19 Nov 1978 05:00:00 GMT Cache-Control: public, s-maxage=172800 Age: 1765 Via: 1.1 varnish X-Country-Code: US Port: 80 Banner: HTTP/1.0 301 https://www.packtpub.com/ Location: https://www.packtpub.com/ Accept-Ranges: bytes Date: Fri, 09 Jan 2015 12:08:05 GMT Age: 0 Via: 1.1 varnish Connection: close X-Country-Code: US Server: packt Vulns: !CVE-2014-0160 The (1) TLS and (2) DTLS implementations in OpenSSL 1.0.1 before 1.0.1g do not properly handle Heartbeat Extension packets, which allows remote attackers to obtain sensitive information from process memory via crafted packets that trigger a buffer over-read, as demonstrated by reading private keys, related to d1_both.c and t1_lib.c, aka the Heartbleed bug.
I've just chosen a few of the available data items that Shodan returns, but you can see that we get a fair bit of information back. In this particular instance, we can see that there is a potential vulnerability identified. We also see that this server is listening on ports 80 and 443 and that according to the banner information, it appears to be running nginx as the HTTP server.
Firstly, we set up our static strings within the code; this includes our API key:
SHODAN_API_KEY = "{Insert your Shodan API key}" target = 'www.packtpub.com' dnsResolve = 'https://api.shodan.io/dns/resolve?hostnames=' + target + '&key=' + SHODAN_API_KEYThe next step is to create our API object:
api = shodan.Shodan(SHODAN_API_KEY)
In order to search for information on a host using the API, we need to know the host's IP address. Shodan has a DNS resolver but it's not included in the Python library. To use Shodan's DNS resolver, we simply have to make a GET request to the Shodan DNS Resolver URL and pass it the domain (or domains) we are interested in:
resolved = requests.get(dnsResolve) hostIP = resolved.json()[target]
The returned JSON data will be a dictionary of domains to IP addresses; as we only have one target in our case, we can simply pull out the IP address of our host using the
targetstring as the key for the dictionary. If you were searching on multiple domains, you would probably want to iterate over this list to obtain all the IP addresses.Now, we have the host's IP address, we can use the Shodan libraries
hostfunction to obtain information on our host. The returned JSON data contains a wealth of information about the host, though in our case we will just pull out the IP address, organization, and if possible the operating system that is running. Then we will loop over all of the ports that were found to be open and their respective banners:host = api.host(hostIP) print "IP: %s" % host['ip_str'] print "Organization: %s" % host.get('org', 'n/a') print "Operating System: %s" % host.get('os', 'n/a') # Print all banners for item in host['data']: print "Port: %s" % item['port'] print "Banner: %s" % item['data']The returned data may also contain potential Common Vulnerabilities and Exposures (CVE) numbers for vulnerabilities that Shodan thinks the server may be susceptible to. This could be really beneficial to us, so we will iterate over the list of these (if there are any) and use another function from the Shodan library to get information on the exploit:
for item in host['vulns']: CVE = item.replace('!','') print 'Vulns: %s' % item exploits = api.exploits.search(CVE) for item in exploits['matches']: if item.get('cve')[0] == CVE: print item.get('description')That's it for our script. Try running it against your own server.
We've only really scratched the surface of the Shodan Python library with our script. It is well worth reading through the Shodan API reference documentation and playing around with the other search options. You can filter results based on "facets" to narrow down your searches. You can even use searches that other users have saved using the "tags" search.
Tip
Downloading the example code
You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.