When choosing a test framework, ask yourself the following questions:
What do I want to test? You cannot test everything and developing tests takes time.
How easy is it to write and maintain the tests? This question is just as relevant for developing tests as it is for developing code in general.
How much effort is needed to perform the tests? If it is easy to automate the tests, they can, for example, be run as part of the deployment as an extra check.
Just for Python alone there are quite a few testing frameworks available, but we will choose the unittest module distributed with Python. Note that although we choose to write only automated test for the Python parts of the applications, this doesn't mean we have not tested the JavaScript parts, but user interactions tend to lend themselves less to an automated way of testing so we do not address that in this book.
For the Python unit tests, we restrict ourselves to the unittest module that is distributed with Python, as this will not introduce any new dependencies on external tools but also because:
It is fairly simple to learn and use.
It produces clear messages if a test fails.
It is easy to automate and may easily be integrated with, for example, a setup script.
A version management tool is normally not part of a web application and not strictly required to develop one. However, when you want to keep track of changes in your code, especially when the number of files keeps on growing, a version management tool is invaluable.
Most come with integrated functionality to show the differences between versions and all have the possibility to annotate a version or revision in order to clearly mark it. Widely used open source solutions are git and svn.
Both may operate as a server that can be accessed through a web browser but command-line tools are available as well and svn even has a very user-friendly integration within Windows' file explorer. Both have their strengths and weaknesses and it is hard to declare a clear winner. This book and its accompanying examples were all maintained in svn, primarily because of the ease of use of the Windows client.
Web applications are built for end users, not for developers. It is not always easy to design an interface that is easy to use. In fact, designing really good interfaces is difficult and takes considerable skill and knowledge. However, this does not mean that there aren't any rules of thumb that can help you prevent usability disasters. We look at some of them in the following sections.
Applications are easier to use if the interface components are already familiar. Therefore, it is generally a good idea to look at applications that are successful and used by many people.
A common concern in many applications is the need to present a lot of information in a small amount of space. It is therefore no wonder that many modern applications use accordion menus and/or a tabbed interface to structure that data, such as the following screenshots:
An accordion menu is great for displaying a fair amount of information in a side bar but even more information can be presented in tabs:
Examples are found in all recent editions of common office productivity software, web browser, and CRM applications. Having a good look at the ones you like working with yourself might be a good start. In the larger applications developed in this book, we will certainly refer to some key applications that may be used as an inspiration.
Choosing a consistent and pleasing color scheme and font makes an application more coherent and therefore more pleasurable to use. An overload of information can baffle people and using a wild color scheme or many different fonts will not help in getting an overview of the data that is presented.
But whether your user interface supports the concept of a theme that is easy to change plays an important role in other areas as well. You probably want your web application to blend in well with the rest of your website or to convey some sort of company or brand identity. Using a consistent color scheme will help. It might even be desirable to offer a choice of themes to the end user, for example, to provide people with visual impairments with high contrast themes for better legibility. The library fully supports the use of themes and makes it simple to extend this themability to widgets we design ourselves.
Web applications are often geared to a specific audience, so it might be possible that the requirements specify only a single browser, but in general, we don't want to deny the user his/her favorite browser. jQuery takes away most of the pain in supporting more than one browser. Our apps are designed for Internet Explorer 8, Firefox 3.x, and Google Chrome, but probably will run on most other browsers as well. Note that 'probably' might not be good enough and it is always a good idea to test your application specifically on any required platform!
Client-side, the web browser is the key component in our chain to watch out for and therefore, the operating system it is running on will quite likely not be a source of problems.
Server-side, we want to keep our options open as well. Fortunately, Python is a cross platform solution, so any Python program that runs on Windows will normally run on GNU/Linux as well and vice versa.
We should be careful though when using modules that are not distributed with Python and are not pure Python. These might be available on every platform but it is better to check beforehand. The applications in this book use only modules provided in the standard Python distribution, with the exception of CherryPy, which is a pure Python module and should run on every platform.
Writing code is hard work, maintaining it can be even harder. We briefly touched upon this subject earlier when we discussed the use of a testing framework, but maintaining code is more than being able to test it.
An important concept in creating code that is easy to maintain is being standards compliant. Adhering to standards means that other people stand a greater chance in understanding your code.
SQL, for example, is a query language that most database engines understand. Therefore, it is less relevant which engine we use for people maintaining the code as they do not have to learn an obscure query language.
Another example is communication between client and server. We can devise our own protocol to construct requests in JavaScript and respond to those requests in Python, but it is a lot less error prone to use documented standards like AJAX to communicate and JSON to encode data. It also saves on documentation as people can be referred to any number of books, if they want to learn more about those standards.
Note
Standard does not necessarily mean 'approved by some independent organization'. Many standards are informal but work because everybody uses them and writes about them. Both AJAX and JSON are examples of that. Also the Python programming language is a de facto standard but JavaScript enjoys a formal standard (which doesn't mean all implementations adhere to the standard).
Security is often regarded as an obscure or arcane subject, but security covers many practical issues that play a role in even the smallest web application. We wouldn't want anyone to access a paid-for web application, for example. However, security is more than just access control and we touch briefly on some aspects of security in the next sections.
A web application should be reliable in its use. Nothing is more annoying than being presented with a server-side error halfway in the process of filling in a mortgage application, for example. As a developer and tester, you take care of testing the software thoroughly in the hope of catching any bugs but before implementing the application, the reliability of the software and libraries it uses should be taken into consideration.
You should especially be wary of using the latest and greatest nifty feature of some library in production software. This might be fun when whipping up some mock up or concept application, but do ask yourself if your customer really needs this bleeding edge feature and if he's/she's not better off with a tried and tested version.
Many open source projects (including Python) develop and maintain both a so called stable branch and a development branch to show off new features. The former you should use in production applications and the latter should be tried elsewhere.
Applications should not only be as bug-free as possible, but should also perform nicely under stress as well. The performance should be as high as possible under load, but just as important you should know what to expect when the load reaches some threshold.
Unfortunately, tuning for performance is one of the trickiest jobs imaginable because all components in the chain may play a role. Server-side considerations are the performance of the database engine used, the scripting language, and the web server.
Client-side, the quality of the presentation framework and the overall performance of the web browser are important and in between the server and client is the great unknown of the characteristics of the underlying network.
With so many variables, it is not easy to design an optimal solution in advance. However, we can test the performance of individual components and see if the component is a bottle neck. For example, if it takes three seconds to refresh a page provided by a web application you can rule out the database engine as a bottleneck if you can time the database access independently. The knowledge gained creating unit tests can be reused here because we already know how to isolate some functionality, and adding a timer and asserting that the response for a query is fast enough can be made a test itself.
It is also quite feasible to separately measure the time it takes to fetch a web component and to render it in the browser with a tool like Firebug and get an idea whether the client or the server is the bottleneck. (Firebug is a Firefox extension and can be found at http://getfirebug.com/).
In almost every application that we develop in this book, we implement some sort of authentication scheme. Most of the time, we will use a simple username/password combination to verify that the user is who he/she claims to be. Once the user is authenticated, we can then decide to serve only certain information, for example, just a list of the tasks belonging to him/her, but no tasks of any other user.
However, whether access to information is allowed, isn't always that basic. Even in simple applications, there might be a user who should be allowed more than others, for example, adding new users or resetting passwords. If the number of different things a user is allowed to do is small, this is straightforward to implement, but if the situation is more complex, it is not that easy to implement, let alone to maintain.
In the more elaborate applications featured in the later chapters of this book, we will therefore adopt the concept of role based access. The idea is to define roles that describe which actions are allowed when assuming a role. In a customer relations management application, for example, there might be three roles: a sales person, who is only allowed to access information for his customers, the sales manager who may access all information, and an administrator who may not access any information, but is allowed to back up and restore information, for example.
Once the rights of these roles are clear, we can associate any or all of these roles with specific persons. A small organization, for example, may have a technically savvy sales person who can also assume the admin role, yet still be unable to access information about customers other than his own this way.
If rights associated with a certain role are changed, we do not have to repeat this information for each and every person that may assume that role, thus making administration that much simpler.
In some applications, we may want to make sure no one is listening in on the data transferred between the browser and web server. After all, in general you do not know which path your data takes, as it is routed across the Internet and at any point there might be someone who can intercept your data.
The easiest way to ensure confidentiality is to use connection level encryption and the HTTPS protocol does just that. The web server we use, CherryPy, is certainly capable of serving requests over HTTPS and configuring it to do so is quite simple but it involves creating signed certificates which is a bit out of the scope of this book. Refer to http://www.cherrypy.org/wiki/ServerObject for more information.
The last aspect of security we talk about in this context is data integrity. Corruption of data may not always be prevented, but wholesale destruction may be guarded against with proper backup and restore protocols.
However, data corruption lurks in very small corners too. One of the trickiest things that can happen is the possibility of inserting data that is wrong. For example, if it is possible to input a date with a month outside the range 1-12, very strange things might happen if the application relies elsewhere on dates having the correct format.
It is, therefore, important to prevent the user entering wrong data by building in some sort of client-side validation. An excellent example is jQuery UI's datepicker widget that we will encounter in Chapter 3, Tasklist I: Persistence. If a text input field is adorned with a datepicker, the user can only enter dates by selecting dates from the datepicker. This is a great aid to the end-user, but we should never rely on client-side validation because our client-side validation might be inadequate (because it contains a bug or doesn't check all cases) and certainly cannot prevent malicious users from connecting to the server and actively inserting wrong data. We do need server-side input validation as well to prevent this and we will encounter some examples of it.
The key thing is to provide both: server-side validation as a last resort and client-side as an aid to the user.
Security is complex and tricky and details may be overlooked easily. You might know you have a front door made of 10 centimeter oak with state of the art steel locks, but if you forget to lock the backdoor all that oak and steel serves no purpose. Of all the subjects touched upon in this book, security is the one that you should always talk over with an expert. Even an expert cannot give you guarantees but taking a fresh look at the security requirements might keep you out of trouble. Make sure that you run the sample applications provided in this book in a secure environment behind a well managed firewall.