Advanced Lesson 3
HTTP Requests
Chapter 2: HTTP
Making an HTTP request
Let’s try retrieving the Department of Computing webpage in Python using the requests package. Try the following yourself!
>>> import requests
>>> response = requests.get("http://www.doc.ic.ac.uk")
>>> type(response)
<class 'requests.models.Response'>
>>> help(response) # Read the documentation!
>>> print(response.status_code) # Successful!
200
>>> print(response.ok) # Status code is less than 400 (no error)
True
>>> print(response.headers) # Get the response headers
{'Cache-Control': 'private', 'Content-Type': 'text/html; charset=UTF-8', 'Content-Encoding': 'gzip', 'Vary': 'Accept-Encoding', 'Server': 'Microsoft-IIS/10.0', 'X-Powered-By': 'ASP.NET, ARR/3.0', 'X-Web-Server': 'ICWWW7', 'Set-Cookie': 'ARRAffinity-WindowsWF=1e7adbsadsada93fcb3f9d30c4d15863f5d0e6dcc34ee20fdda09204db2c94644;Path=/;Domain=www.imperial.ac.uk', 'X-ARR-Server': 'ICRP10', 'Content-Security-Policy': "frame-ancestors 'self' *.imperial.ac.uk *.ic.ac.uk", 'Date': 'Mon, 15 Nov 2021 13:39:27 GMT', 'Content-Length': '23876'}
>>> print(response.headers["Content-Type"])
'text/html; charset=UTF-8'
>>> print(response.text) # The HTML source code of the webpage
...
>>> type(response.text)
<class 'str'>
>>> print(response.request.headers) # Check the headers that YOU sent in your request
{'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive'}
Read the official documentation for more details about the available attributes/methods for the Response instance.
And that’s it! You have made a simple GET request and returned an HTTP response from the server. It gives you the HTML source code for the webpage which you can then for example render into a webpage for your users (we are not going to go that far though!)