Web Development Insight

I just had a harrowing experience: some of the custom 404 pages I set up were returning the wrong HTTP status code. A nonexistent page should return a 404 status code and be redirected to a custom 404 page (the structure is another story in itself). But…the nonexistent pages were returning a 200 status code, which means everything is ok…and it’s not.

This is the way I was given by my host (actually one of the ways) to redirect a 404 to a custom 404 page:

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) /custom404.php

This does redirect a nonexistent web page to the custom404.php page but it causes the bad page to return a 200 instead of a 404.

ErrorDocument 404 /custom404.php

This is what I replaced the last three lines with. Now the errant pages return a 404 rather than a 200.

BTW, I use Fiddler to check the return codes.

Doug

This only works for an Apache server. Note that the folder and its contents will not be indexed since it is passworded.

1. create .htaccess and .htpasswd as ordinary text files. Make sure there is no .txt at the end.

here is an .htaccess example

AuthUserFile /home/content/f/r/e/fred/html/myfolder/.htpasswd
AuthGroupFile /dev/null
AuthName "Access for Admin"
AuthType Basic

<limit GET POST>
require user myfriends
</limit>

where myfriends (replace with your own) is the login id and the folder to protect is myfolder (replace with your own directory path)

line 1 of .htaccess needs to be replaced with the full path on your server to your .htpasswd folder. This is dependent on your ISP. The example shown below is from GoDaddy. The path to the root of your GoDaddy account looks like:

/home/content/f/r/e/fred/html/

This is known as the DOCUMENT_ROOT in the world of PHP. Since most hosts support php you can create a file called showinfo.php and put the following into it:

<? phpinfo(); ?>

Upload this to your website’s home directory and invoke it:

http://www.mydomain.com/showinfo.php

Look for the entry called “DOCUMENT_ROOT”.
for example if DOCUMENT_ROOT was /home/content/f/r/e/fred/html/ and the folder you are trying to protect is myfolder then the entry for AuthUserFile is:

/home/content/f/r/e/fred/html/myfolder/.htaccess

Ask your ISP if you cannot find the full path.

Here is an .htpasswd example

myfriends:bZTGwg.9OWALY

• where myfriends is the login id and the stuff after the colon is the encrypted password.
• make sure .htpasswd is one line only with no breaks at the end

to get the .htpasswd entry you need to generate a password. Try this link:

Generate .htpasswd

It will generate a cut-and-paste line for .htpasswd. Remember no breaks at the end of the line.

If you have access to Perl, try this from the command line:

perl -e "print crypt('myfriends', 'abc')"

where the ‘myfriends’ is the login id and ‘abc’ can be any sequence of letters or numbers. It provides a starting point for the password.

2. upload .htaccess and .htpasswd to the folder you want passworded. If you have a problem uploading try this link:

Uploading the .htaccess file

3. you are done. Every time someone tries to access the folder or any of its subfolders they will be prompted for a login and password.

Doug

I am surprised at how many websites do not have custom 404 pages.
Try entering a nonexistent web page for your website.
And what is a custom 404 page? A response to a web page not found. If a web page is not found on your site, the server returns a 404 error. Normally the web host has a default web page in response to a 404 error. Most web hosts allow a custom 404 page which you can put just about anything you want on it. And a link to the homepage can be put on; something a default 404 page does not have.

Check with your web host on how to redirect a 404 to your custom 404 page. Each host has its own peculiarities.

The .htaccess file can be modified to point to a custom 404 page (note .htaccess file is only for Apache servers). Add the following line:

ErrorDocument 404 http://www.mydomain.com/404-error-page.php

Replace mydomain with your domain. Note the error page can be an html file instead of a php file. This worked fine if the error page was an html page but did not work if it was a php page.

A better way is this:

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule (.*) /errordocument.htm

where /errordocument.htm is the custom 404 page for your site. This can also be a php script such as custom404.php. Note this goes in the root directory. Just add directories like this: /dir1/dir2/errordocument.php

This method worked fine for nonexistent html and php pages.
Doug

Sometimes uploading the .htaccess file to the Apache server does not work; especially in the Windows environment. Try this:

1. Rename the .htaccess file to htaccess.txt
2. Upload it to the server
3. Rename htaccess.txt to .htaccess

Step 3 can be tricky. If your FTP program or web designer program (such as FrontPage) won’t allow you to rename the htaccess.txt file try using the FTP program supplied by your host. Once renamed it may take a few minutes for the .htaccess file to take effect.

Another way around this dilemma is to use a 3rd party ftp program. I like FileZilla which is free and easy to use. The .htaccess and .htpasswd files can directly be uploaded without changing their names.

Doug