Python – HTTP Requests

The http or Hyper Text Transfer Protocol works on client server model. Usually the web browser is the client and the computer hosting the website is the
server. IN python we use the requests module for creating the http requests. It is a very powerful module which can handle many aspects of http communication beyond the
simple request and response data. It can handle authentication, compression/decompression, chunked requests etc.

An HTTP client sends an HTTP request to a server in the form of a request message which includes following format:

  • A Request-line
  • Zero or more header (General|Request|Entity) fields followed by CRLF
  • An empty line (i.e., a line with nothing preceding the CRLF)
    indicating the end of the header fields
  • Optionally a message-body

    • The following sections explain each of the entities used in an HTTP request message.

    Request-Line

    The Request-Line begins with a method token, followed by the Request-URI and the protocol version, and ending with CRLF. The elements are separated by space SP characters.

    >Request-Line = Method SP Request-URI SP HTTP-Version CRLF

    Let’s discuss each of the parts mentioned in the Request-Line.

    Request Method

    The request method indicates the method to be performed on the resource identified by the given Request-URI. The method is case-sensitive and should always be mentioned in uppercase. The following table lists all the supported methods in HTTP/1.1.

    S.N. Method and Description
    1 GET

    The GET method is used to retrieve information from the given server using a given URI. Requests using GET should only retrieve data and should have no other effect on the data.
    2 HEAD

    Same as GET, but it transfers the status line and the header section only.
    3 POST

    A POST request is used to send data to the server, for example, customer information, file upload, etc. using HTML forms.
    4 PUT

    Replaces all the current representations of the target resource with the uploaded content.
    5 DELETE

    Removes all the current representations of the target resource given by URI.
    6 CONNECT

    Establishes a tunnel to the server identified by a given URI.
    7 OPTIONS

    Describe the communication options for the target resource.
    8 TRACE

    Performs a message loop back test along with the path to the target resource.

    Request-URI

    The Request-URI is a Uniform Resource Identifier and identifies the resource upon which to apply the request. Following are the most commonly used forms to specify an URI:

    >Request-URI = "*" | absoluteURI | abs_path | authority
     
    S.N. Method and Description
    1 The asterisk * is used when an HTTP request does not apply to a particular resource, but to the server itself, and is only allowed when the method used does not necessarily apply to a resource. For example:

    OPTIONS * HTTP/1.1
    2 The absoluteURI is used when an HTTP request is being made to a proxy. The proxy is requested to forward the request or service from a valid cache, and return the response. For example:

    GET http://www.w3.org/pub/WWW/TheProject.html HTTP/1.1
    3 The most common form of Request-URI is that used to identify a resource on an origin server or gateway. For example, a client wishing to retrieve a resource directly from the origin server would create a TCP connection to port 80 of the host “www.w3.org” and send the following lines:

    GET /pub/WWW/TheProject.html HTTP/1.1

    Host: www.w3.org

    Note that the absolute path cannot be empty; if none is present in the original URI, it MUST be given as “/” (the server root).

    Using Python requests

    We will use the module requests for learning about http request.

    pip install requests

    In the below example we see a case of simple GET request annd print out the result of the response. We choose to print only the first 300 characters.

    # How to make http request
import requests as req
r = req.get('http://www.tutorialspoint.com/python/')
print(r.text)[0:300]

    When we run the above program, we get the following output −

    ><!DOCTYPE html>
<!--[if IE 8]><html class="ie ie8"> <![endif]-->
<!--[if IE 9]><html class="ie ie9"> <![endif]-->
<!--[if gt IE 9]><!-->	<html> <!--<![endif]-->
<head>
<!-- Basic -->
<meta charset="utf-8">
<title>Python Tutorial</title>
<meta name="description" content="Python Tutorial