Now that we've covered ground on scraping, crawling, and parsing, it's time for another interesting work that we can do with Python, which is working with third-party APIs. I'd assume many of us are aware and might have a basic understanding of REST API. So, let's get started!
To demonstrate the understanding, we take the case of GitHub gists. Gists in GitHub are the best way to share your work, a small code snippet that helps your colleague or a small app with multiple files that gives an understanding of a concept. GitHub allows the creation, listing, deleting, and updating of gists, and it presents a classical case of working with GitHub REST APIs.
So, in this section, we use our very own requests
module to make HTTP requests to GitHub REST API to create, update, list, or delete gists.
The following steps will show you how to work with GitHub REST APIs using Python.
To work with GitHub REST APIs, we need to create a Personal access token. For doing that, log in to https://github.com/ and browse to https://github.com/settings/tokens and click on Generate new token:
You'll now be taken to the New personal access token page. Enter a description at the top of the page and check the gists option among the scopes given out. Note that scope represents the access for your token. For instance, if you just select gists, you can use GitHub APIs to work on the gists resource but not on other resources such as repo or users. For this recipe, the gists scope is just what we need:
Once you click on Generate token, you'd be presented with a screen containing your personal access token. Keep this token confidential with you.
With the access token available, let's start working with APIs and create a new gist. With create, we add a new resource, and for doing this, we make an HTTP
POST
request on GitHub APIs, such as in the following code:import requests import json BASE_URL = 'https://api.github.com' Link_URL = 'https://gist.github.com' username = '<username>' ## Fill in your github username api_token = '<api_token>' ## Fill in your token header = { 'X-Github-Username': '%s' % username, 'Content-Type': 'application/json', 'Authorization': 'token %s' % api_token, } url = "/gists" data ={ "description": "the description for this gist", "public": True, "files": { "file1.txt": { "content": "String file contents" } } } r = requests.post('%s%s' % (BASE_URL, url), headers=header, data=json.dumps(data)) print r.json()['url']
If I now go to my
gists
page on GitHub, I should see the newly created gist. And voila, it's available!Hey, we were successful in creating the gist with the GitHub APIs. That's cool, but can we now view this
gist
? In the preceding example, we also printed the URL of the newly created gist. It will be in the format,https://gist.github.com/<username>/<gist_id>
. We now use this gist_id to get the details of the gist, which means we make a HTTPGET
request on the gist_id:import requests import json BASE_URL = 'https://api.github.com' Link_URL = 'https://gist.github.com' username = '<username>' api_token = '<api_token>' gist_id = '<gist id>' header = { 'X-Github-Username': '%s' % username, 'Content-Type': 'application/json', 'Authorization': 'token %s' % api_token, } url = "/gists/%s" % gist_id r = requests.get('%s%s' % (BASE_URL, url), headers=header) print r.json()
We created a new gist with the HTTP
POST
request and got the details of the gist with the HTTPGET
request in the previous steps. Now, let's update this gist with the HTTPPATCH
request.Note
Many third-party libraries choose to use the
PUT
request to update a resource, but HTTPPATCH
can also be used for this operation, as chosen by GitHub.The following code demonstrates updating the gist:
import requests import json BASE_URL = 'https://api.github.com' Link_URL = 'https://gist.github.com' username = '<username>' api_token = '<api_token>' gist_id = '<gist_id>' header = { 'X-Github-Username': '%s' % username, 'Content-Type': 'application/json', 'Authorization': 'token %s' % api_token, } data = { "description": "Updating the description for this gist", "files": { "file1.txt": { "content": "Updating file contents.." } } } url = "/gists/%s" % gist_id r = requests.patch('%s%s' %(BASE_URL, url), headers=header, data=json.dumps(data)) print r.json()
Now, if I look at my GitHub login and browse to this gist, the contents of the gist have been updated. Awesome! Don't forget to see the Revisions in the screenshot--see it got updated to revision 2:
Now comes the most destructive API operation--yes deleting the gist. GitHub provides an API for removing the gist by making use of the HTTP
DELETE
/gists/<gist_id>
resource. The following code helps us delete thegist
:import requests import json BASE_URL = 'https://api.github.com' Link_URL = 'https://gist.github.com' username = '<username>' api_token = '<api_token>' gist_id = '<gist_id>' header = { 'X-Github-Username': '%s' % username, 'Content-Type': 'application/json', 'Authorization': 'token %s' % api_token, } url = "/gists/%s" % gist_id r = requests.delete('%s%s' %(BASE_URL, url), headers=header, )
Let's quickly find out if the gist is now available on the GitHub website? We can do that by browsing the gist URL on any web browser. And what does the browser say? It says 404 resource not found, so we have successfully deleted the gist! Refer to the following screenshot:
Finally, let's list all the gists in your account. For this we make an HTTP
GET
API call on the/users/<username>/gists
resource:
import requests BASE_URL = 'https://api.github.com' Link_URL = 'https://gist.github.com' username = '<username>' ## Fill in your github username api_token = '<api_token>' ## Fill in your token header = { 'X-Github-Username': '%s' % username, 'Content-Type': 'application/json', 'Authorization': 'token %s' % api_token, } url = "/users/%s/gists" % username r = requests.get('%s%s' % (BASE_URL, url), headers=header) gists = r.json() for gist in gists: data = gist['files'].values()[0] print data['filename'], data['raw_url'], data['language']
The output of the preceding code for my account is as follows:
Python's requests
module helps in making HTTP GET
/POST
/PUT
/PATCH
and DELETE
API calls on GitHub's resources. These operations, also known as HTTP verbs in the REST terminology, are responsible for taking certain actions on the URL resources.
As we saw in the examples, the HTTP GET
request helps in listing the gists, POST
creates a new gist, PATCH
updates a gist, and DELETE
completely removes the gist. Thus, in this recipe, you learnt how to work with third-party REST APIs--an essential part of WWW today--using Python.