A POST request is another very common HTTP server request used by many APIs. We will be mining the University of Virginia directory search. When sending a POST request for a search query, the Lightweight Directory Access Protocol (LDAP) server replies with a web page of search results.
For this recipe, access to the Internet is necessary.
Install the HandsomeSoup CSS selector package, and also install the HXT library if it is not already installed:
$ cabal install HandsomeSoup $ cabal install hxt
Import the following libraries:
import Network.HTTP import Network.URI (parseURI) import Text.XML.HXT.Core import Text.HandsomeSoup import Data.Maybe (fromJust)
Define the POST request specified by the directory search website. Depending on the server, the following POST request details would be different. Refer to the following code snippet:
myRequestURL = "http://www.virginia.edu/cgi-local/ldapweb" myRequest :: String -> Request_String myRequest query = Request { rqURI = fromJust $ parseURI myRequestURL , rqMethod = POST , rqHeaders = [ mkHeader HdrContentType "text/html" , mkHeader HdrContentLength $ show $ length body ] , rqBody = body } where body = "whitepages=" ++ queryDefine and implement
mainto run the POST request on a query as follows:main :: IO () main = do response <- simpleHTTP $ myRequest "poon"
Gather the HTML and parse it:
html <- getResponseBody response let doc = readString [withParseHTML yes, withWarnings no] html
Find the table rows and print it out using the following:
rows <- runX $ doc >>> css "td" //> getText print rows
Running the code will display all search results relating to "poon", such as "Poonam" or "Witherspoon".
A POST request needs the specified URI, headers, and body. By filling out a Request data type, it can be used to establish a server request.