Using proxies with cURL in php

If you are new to cURL you may want to refer to the PHP cURL tutorial. It covers the basic working of cURL.

Sometimes we have to use proxies with our curl requests. Proxy servers are buffers between the requesting client and the web server and are used for a variety of reasons including companies restricting web access to people wanting to appear anonymous.

To use proxies with cURL we need to set 3 additional cURL options.

CURLOPT_HTTPPROXYTUNNEL
CURLOPT_PROXYTYPE
CURLOPT_PROXY

First to enable use of a proxy in curl with CURLOPT_HTTPPROXYTUNNEL.

CURLOPT_PROXYTYPE automatically defaults to CURLPROXY_HTTP. You can Skip this option if you are using http proxy.

OR it should be set to either CURLPROXY_SOCKS5 or CURLPROXY_SOCKS4, depending on the type of proxy you are using.

CURLOPT_PROXY should be set to the IP address and port of the proxy you are using.

A simple curl request with a single proxy

  
  $url = "http://example.com";
  $agent = "Mozilla/5.0 (X11; U; Linux i686; en-US) 
            AppleWebKit/532.4 (KHTML, like Gecko) 
            Chrome/4.0.233.0 Safari/532.4";
  $referer = "http://www.google.com/";

  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
  curl_setopt($ch, CURLOPT_PROXY, '202.95.141.129:8080');
  curl_setopt($ch, CURLOPT_REFERER, $referer);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
  curl_setopt($ch, CURLOPT_MAXREDIRS, 2);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
  curl_setopt($ch, CURLOPT_USERAGENT, $agent);
  
  $data = curl_exec($ch);
  curl_close($ch);
  echo $data;



Note : Use a valid working proxy.

That's it. You are done if you are using only one proxy. But you don't always use only one proxy for your scripts. Let us move one step further and use random proxies.

You need to use multiple proxies and one proxy at a time. So you need to create a text file and store all your proxies in it and take one random proxy at a time while using curl. The format of the proxy will be

[IP]:[port]

One proxy per line. The text file looks like...

  202.95.141.129:8080
  58.67.147.203:8080
  202.95.141.129:8080


Note: You have to use your own Valid Proxies in this list

Further now we have to create a function to take one random proxy from the text file.

The whole script is here

  
  function get_random_proxy(){
      srand ((double)microtime()*1000000);
      $f_contents = file ("proxy.txt");
      $line = $f_contents[array_rand ($f_contents)];
      return $line;
  }
  
  function get_curl_proxy($url){
  
  $proxy_ip = get_random_proxy();
  $agent = "Mozilla/5.0 (X11; U; Linux i686; en-US) 
            AppleWebKit/532.4 (KHTML, like Gecko) 
            Chrome/4.0.233.0 Safari/532.4";
  $referer = "http://www.google.com/";

  $ch = curl_init();
  curl_setopt($ch, CURLOPT_URL, $url);
  curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
  curl_setopt($ch, CURLOPT_PROXY, $proxy_ip);
  curl_setopt($ch, CURLOPT_REFERER, $referer);
  curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
  curl_setopt($ch, CURLOPT_MAXREDIRS, 2);
  curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
  curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
  curl_setopt($ch, CURLOPT_USERAGENT, $agent);
  
  $data = curl_exec($ch);
  curl_close($ch);
  
  return $data;
  }
  
  $u="http://example.com";
  
  echo get_curl_proxy($u);  


The get_random_proxy() function takes one random proxy from the file proxy.txt and the function get_curl_proxy() makes a curl request using that proxy.
Using proxies with cURL in php Using proxies with cURL in php Reviewed by JS Pixels on June 30, 2012 Rating: 5

9 comments:

  1. nice code so usefull..

    ReplyDelete
  2. sir ihave this error @ ur example code



    Parse error: syntax error, unexpected T_DNUMBER in /home/content/65/11169265/html/test2/index.php on line 9...........plzzzzzzz help



    my mail id is minded@live.in

    ReplyDelete
    Replies
    1. Provide the proxy inside quotes. e.g
      curl_setopt($ch, CURLOPT_PROXY, '202.95.141.129:8080');

      Delete
    2. Thank you for making me aware. Updated it

      Delete
  3. returning a blank page

    ReplyDelete
  4. what if I don't know if there is a proxy ? my code runs on a pc. sometimes a proxy is configured, some times it doesnt.
    how can I tell ?

    ReplyDelete
  5. what purpose we are using the proxy in cURL call ??
    i have doubt in CURLOPT_PROXY, so please explain

    ReplyDelete
  6. This code is not working when it is kept in Live server... Any Help To Sort it Out???
    Thanks In Advance

    ReplyDelete
    Replies
    1. Check if cURL is enabled on your server !!

      By the way what is the issue/error?

      Delete

Altaf Web. Powered by Blogger.