Using proxies with cURL in php
If you are new to cURL you may want to refer to the PHP cURL tutorial. It covers the basic working of cURL.
Sometimes we have to use proxies with our curl requests. Proxy servers are buffers between the requesting client and the web server and are used for a variety of reasons including companies restricting web access to people wanting to appear anonymous.
To use proxies with cURL we need to set 3 additional cURL options.
CURLOPT_HTTPPROXYTUNNEL
CURLOPT_PROXYTYPE
CURLOPT_PROXY
First to enable use of a proxy in curl with CURLOPT_HTTPPROXYTUNNEL.
CURLOPT_PROXYTYPE automatically defaults to CURLPROXY_HTTP. You can Skip this option if you are using http proxy.
OR it should be set to either CURLPROXY_SOCKS5 or CURLPROXY_SOCKS4, depending on the type of proxy you are using.
CURLOPT_PROXY should be set to the IP address and port of the proxy you are using.
A simple curl request with a single proxy
Note : Use a valid working proxy.
That's it. You are done if you are using only one proxy. But you don't always use only one proxy for your scripts. Let us move one step further and use random proxies.
You need to use multiple proxies and one proxy at a time. So you need to create a text file and store all your proxies in it and take one random proxy at a time while using curl. The format of the proxy will be
[IP]:[port]
One proxy per line. The text file looks like...
Note: You have to use your own Valid Proxies in this list
Further now we have to create a function to take one random proxy from the text file.
The whole script is here
The get_random_proxy() function takes one random proxy from the file proxy.txt and the function get_curl_proxy() makes a curl request using that proxy.
Sometimes we have to use proxies with our curl requests. Proxy servers are buffers between the requesting client and the web server and are used for a variety of reasons including companies restricting web access to people wanting to appear anonymous.
To use proxies with cURL we need to set 3 additional cURL options.
CURLOPT_HTTPPROXYTUNNEL
CURLOPT_PROXYTYPE
CURLOPT_PROXY
First to enable use of a proxy in curl with CURLOPT_HTTPPROXYTUNNEL.
CURLOPT_PROXYTYPE automatically defaults to CURLPROXY_HTTP. You can Skip this option if you are using http proxy.
OR it should be set to either CURLPROXY_SOCKS5 or CURLPROXY_SOCKS4, depending on the type of proxy you are using.
CURLOPT_PROXY should be set to the IP address and port of the proxy you are using.
A simple curl request with a single proxy
$url = "http://example.com"; $agent = "Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.4 (KHTML, like Gecko) Chrome/4.0.233.0 Safari/532.4"; $referer = "http://www.google.com/"; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1); curl_setopt($ch, CURLOPT_PROXY, '202.95.141.129:8080'); curl_setopt($ch, CURLOPT_REFERER, $referer); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_MAXREDIRS, 2); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10); curl_setopt($ch, CURLOPT_USERAGENT, $agent); $data = curl_exec($ch); curl_close($ch); echo $data;
Note : Use a valid working proxy.
That's it. You are done if you are using only one proxy. But you don't always use only one proxy for your scripts. Let us move one step further and use random proxies.
You need to use multiple proxies and one proxy at a time. So you need to create a text file and store all your proxies in it and take one random proxy at a time while using curl. The format of the proxy will be
[IP]:[port]
One proxy per line. The text file looks like...
202.95.141.129:8080 58.67.147.203:8080 202.95.141.129:8080
Note: You have to use your own Valid Proxies in this list
Further now we have to create a function to take one random proxy from the text file.
The whole script is here
function get_random_proxy(){ srand ((double)microtime()*1000000); $f_contents = file ("proxy.txt"); $line = $f_contents[array_rand ($f_contents)]; return $line; } function get_curl_proxy($url){ $proxy_ip = get_random_proxy(); $agent = "Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.4 (KHTML, like Gecko) Chrome/4.0.233.0 Safari/532.4"; $referer = "http://www.google.com/"; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1); curl_setopt($ch, CURLOPT_PROXY, $proxy_ip); curl_setopt($ch, CURLOPT_REFERER, $referer); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); curl_setopt($ch, CURLOPT_MAXREDIRS, 2); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10); curl_setopt($ch, CURLOPT_USERAGENT, $agent); $data = curl_exec($ch); curl_close($ch); return $data; } $u="http://example.com"; echo get_curl_proxy($u);
The get_random_proxy() function takes one random proxy from the file proxy.txt and the function get_curl_proxy() makes a curl request using that proxy.
Using proxies with cURL in php
Reviewed by JS Pixels
on
June 30, 2012
Rating:
nice code so usefull..
ReplyDeletesir ihave this error @ ur example code
ReplyDeleteParse error: syntax error, unexpected T_DNUMBER in /home/content/65/11169265/html/test2/index.php on line 9...........plzzzzzzz help
my mail id is minded@live.in
Provide the proxy inside quotes. e.g
Deletecurl_setopt($ch, CURLOPT_PROXY, '202.95.141.129:8080');
Thank you for making me aware. Updated it
Deletereturning a blank page
ReplyDeletewhat if I don't know if there is a proxy ? my code runs on a pc. sometimes a proxy is configured, some times it doesnt.
ReplyDeletehow can I tell ?
what purpose we are using the proxy in cURL call ??
ReplyDeletei have doubt in CURLOPT_PROXY, so please explain
This code is not working when it is kept in Live server... Any Help To Sort it Out???
ReplyDeleteThanks In Advance
Check if cURL is enabled on your server !!
DeleteBy the way what is the issue/error?