I don’t care if my database grows into a giant data-monster, ready to ravage my server’s hard disk and slow queries down to a crawl; I’ll deal with it.  The key here is to not allow any important history of data to get lost.

Why Deleting is Bad

Here’s an example, to get us started:

Let’s say you have a simple online store, and you’re selling important consumer items to important people.  You have a database that keeps track of your customers, your products, and your transactions.  It may look something like this:

So, you’ve been in business for a while, and you have a bunch of customers in the system, quite a few products, and a lengthy transaction history.  Recently, you’ve decided to stop selling your Jell-O Brain Mold and you’re ready to remove it from your online store.  Common sense says that you would go to your database, find the record referring to your product, and delete it.

But wait!  If you do this, then the entire history of transactions linked to those products become orphaned (theproducts_id field will point to a non-existent record).  So, if you want to go back in at a future date to see the history of how many of those items that you sold, you will be in trouble.  You haven’t just deleted the item from your website per se, you’ve actually deleted the record of the item ever existing in the first place.

And, you don’t want to do that.

Flag It!

The useful alternative would be to add a flag to the data, so that instead of deleting the record, you could mark it as inactive.  This way, it wouldn’t appear on your website, but it would still exist in the database for future reference.

Your tables will now look something like:

Now you can tell your application to only show the products with `active` = “TRUE” on your website, but refer to products with `active` = “FALSE” when viewing older transaction histories.

This works pretty well, and if you program your User Interface correctly, your website users should never actually be deleting records; instead they should be marking them inactive so they “go away” but never truly die.

Audit History

Okay, now it gets a bit more complex.  In some cases, you don’t just want to mark a record “active” or “inactive” (that is, give it the ability to have two states), you also want to keep a complete revision history of that record, including what change was made, who made it, and when.  This way you can see what a record looked like at a particular moment in time.

This is not a new concept.  Think about your bank account.  It’s pretty much a bunch of numbers stored in a database somewhere.  Do you think that your checking account balance is simply a single field in a single record stored in a database?  Heck no!  Every single debit and credit is stored as a separate transaction, showing a complete history of changes to the account.

There are many ways to accomplish this, really, and for me to put up a tutorial, would be to reinvent the wheel.  The key is to think about which data you want to have complete histories on, and which data is not expendable.  Design your application around these constraints.

Sometimes you Just Have to Kill It

I wrote an application that contains patient records.  When an administrator clicks “Delete”, the patient doesn’t really go away; all of the information remains in the database.  Sometimes, however, we must delete the record permanently. Usually this is because a patient opts out of the program and requests to be removed from the study.  I use a special word for this kind of delete: Purge.

But, before the record is purged, the user has to click a checkbox, and go through two “Are You Sure?” screens informing him or her that not only will the patient record be deleted, but also the complete history of that record, and all data associated with it.  I try to have the application show the data in question on the screen the user has to click yes to.

…well, that’s enough for now.  Hope you’re thinking about data consistency, so you too can become a digital pack rat.

Recently, a major department right here at FSU suffered catastrophic loss to their servers, with scant means of quick recovery.  Their site was down literally for weeks as staff from all over the University scrambled to assist them in recreating the critical pieces.  They lost of a lot of time and money.

I’m absolutely sure they’ll be super-prepared next time, but you don’t have to wait until something tragic happens to be prepared!

Disaster Recovery Enterprise Planning = Big Words (Scary!)

When I was an IT Manager n00b, I thought that planning for disasters was intimidating and complicated.  I thought it took a team of folks working around the clock, military-style, to prepare for incidents.  I was encouraged to believe this by colleagues constantly talking in business-speak, conference sessions talking about things like “DR evaluation metrics”, and by big ugly books that take days to even read through.

In the meantime, while I was being intimidated by all this “enterprise” stuff, we went without reliable backups.  We would have been Scre-ewed if something happened.

1. Embracing Common Sense

Okay, just ignore all that business-speak.  Let’s start with the basics:  If you’re not backing up your data, put down the big ugly DR books right now and go setup a backup.

No, seriously, do it right now.

This does not have to be complicated.  Go find an external hard drive, then find out where your data is on your computers or servers.  Plug the hard-drive into the back of the computers and copy the folders with your data.  Do this every day, and you have a basic disaster recovery strategy!

Pro Tip: If you’re using Windows, I’m a big fan of using Teracopy when I have to copy lots of data at a time.

You get bonus if you copy the data to two hard-drives and take one off-site every day.  Now your data is in two geographic locations on three devices.

See? Easy.  Now that you’re off and running, you can make your life even easier by automating your backups.  If you’re not excited about doing a bunch of geeky tech stuff to get your automated backups working, go convince your boss to spend a few bucks a month on cheap, easy-to-setup online backup services.  If he doesn’t want to spend the money, you can scare him with many horrifying stories of data loss on the Internet.

2. Being Prepared

Next Step: Just sit down in a quiet place for a few minutes and think about the things that could go wrong:

• “My server could suffer a catastrophic hardware failure”
• “My backup data could be corrupt”
• “The building could burn down”
• “We could get robbed”
• “An employee could go apeshit and format the C: drive of the server”
• etc..

Now consider which three or four of those scenarios are most likely to happen in your department. You’ve just considered your risks.  Now, for each one of those scenarios, think about (a) what you can easily do to guard against them and (b) what you would have to do to recover from them:

• Buy a second, backup server (or set an old computer aside to act as server temporarily in case of disaster).
• Keep two backup copies instead of one and periodically test opening files on the backup.
• Take backups offsite so that the data doesn’t burn down with the rest of the building.
• Lock the servers in a room without windows.
• Keep backups (already done), and make sure employees don’t have too much access to the server.
• etc..

If you write all of this stuff down somewhere and tell your boss about it, you have a basic DR plan!  Go have a beer.

3. Building Your Kit

Now that you are creating backups and thinking about our risks/responses, it’s time to build your Disaster Recovery Kit! Put this kit somewhere out of harm’s way, and have it handy in case you need it.  Here’s mine:

1. A camcorder bag — I like this, because it’s easy to carry and has a lot of compartments for all the stuff below:
2. Operating System Boot discs – In my case, Ubuntu Server 8.04
3. GParted Boot disc — For formatting and partitioning systems on-the-fly.
4. Cables – Standard power cable and USB cable.
5. Backup hard drives — Two.  They are my offsite copies of all our data.  I rotate them and keep them encrypted.
6. Flash light – Small and compact, for poking around inside computers or navigating server rooms in the dark
7. Thumb drive — Includes a backup of my server setup procedures/changelogs, systems passwords (in KeePass), and a bunch of portable apps.  Updated weekly.
8. Printed copy of contact information for key stakeholders — My IT colleagues, clients, and my boss, so I can keep them updated.
9. Geek tool kit –  Something like this.  Includes screwdrivers, etc.
10. External SATA Hard Disk Reader – Something small and compact.  In case I need to rip a hard drive from a computer and get the data off quickly.
11. A big sheet of paper that says “Don’t Panic“
12. Music CD with high-energy power music on it.
13. Energy supplement — Or a flask of whiskey, depending on how you roll.
14. “Getting started” guide – A piece of paper, right on top, that explains briefly how to use the kit.
15. Checklist — With all of these items on it, and with weekly DR Kit maintenance procedures.

There, now I can sleep at night much better.  What’s in your DR Toolkit?

Oh, and I also fixed the links…

..and I put them in their very own Google Code pages, so you can peruse the source code, download from Google servers, and leave comments and tickets for yours truly.

AND.. as your Hanukkah/New Years/Participation gift, I also have added a brand new CodeIgniter project that I’m working on called CI Headstart with all kinds of helpers, libraries, and other goodies to help speed up development even more.  Roll on over to the Code page to check it all out.

Now that Christmas break is upon us, and I since I work for a University that gives employees like a bazillion days off each December, I hope to clean these up even more and provide some quality documentation.

Happy Holidays from me to you,

PS >> If anybody is interested in contributing code to these projects, let me know.

It required me to change the whole way that my office operated, though.  See, the way that we were doing things before seemed so simple; we just edited files and shouted at each other if we collided in our editing.  As it turns out, this was a bad thing, and there have been better alternatives around way longer than I’ve even been in this business.

But like most crotchety, over-egotistical developer-types it took a while for me to come around.  Here’s a brief history of my flirtations and eventual commitment to version control.

Our Blissful, Youthful Ignorance

This wasn’t really that long ago…

1..2..3... Easy Peasy Japanesey.. and stupid.

We would all work directly on the files directly on our development server.  We would sit down, SSH into the server using our favorite editor and code away, never thinking of change management, reverting back to old versions, or who would be responsible for what.  Conflict resolution seemed easy… if I was editing a file named welcome.php, I would simply shout across the room, “Hey Mike–don’t work on welcome.php right now, or I’ll have to beat you!”.

Our revision control system consisted of me creating and storing weekly backups of the entire database and file system.  If we made a change that killed the system, we’d simply either lose that week’s work or spend hours debugging it.  It was all gloriously simple.

Version Control Puberty — The Angsty Years

One day, I decided to implement Subversion, mainly because other developers were sneering at me and telling me I was a dummy.  And while I didn’t want to be a dummy, the whole thing seemed overly complicated.  Why would I want to increase complication when everything was working so simple?  I also had some fundamental issues with this new version control idea:

• First off, everybody has to have to get their own copy of the entire file set.  Bad!  Doom! Insecure! I have to trust my developers with the codebase? ::Gasp::
• Secondly, what do you mean that developers can’t see their changes on the server immediately upon commit without some hacked hook system?  This means they’ll have to run LAMP on their local boxes, too!  Geez.. now we have to have local dev environment setup procedures, too..
• Finally, SVN requires management of its own.  You know.. backing up, securing access, and generally managing.  More overhead and more complication!

Grudgingly, in order to remove the “dummy” label from my operations, I implemented Subversion.  It was difficult at first to understand how it would fit easily into our procedures, but with some very helpful free online material, we started using it.

And sure, I did have to write procedures for setting up local dev environments AND install a subversion clients AND manage a repository server, but you know what?

All of that stuff was way worth it.

All Grown-up Now

Let me repeat that: whatever frustrations version control causes, they are all worth it.

After being SVN disciples for a while now, our development procedures have matured.  Here’s how we do things:

The way we do things now.. Much better!

I’ve since tried other VCS systems like Mercurial, Git, and Bazaar, but have not found any compelling reason to switch.   Either way, using version control has saved our sorry butts on more than one occasion and has made life easier in about 1,001 ways.

As a bonus, it has also forced us to use better procedures and generate better development documentation, which is amazing, because we’re pretty lazy.  Now if only we could find a tool to better force us to behave when it comes to unit testing…

Well, that’s my story.  If I’ve inspired you to use VCS, hooray!  Here’s a helpful place to get started.  If not, you’re a dummy.

Any questions, comments, or suggestions are always welcome.  Also, if anybody has tried this plugin on CI 1.7.2, let me know how it works.

I know they must be pretty motivated, because they’re signing up for the class despite the fact that it starts at 8 am.

This Fall, I will  be teaching a course in Web Application Development at FSU.  The topic will be server-side scripting, and the focus will be PHP.  Since I’ve taught this course before, developing the curriculum won’t take months, but I am nevertheless spending a fair amount of time updating and tweaking for the next go-around.

A quick Google Search for “Teaching PHP” brings up only a few relevant links, most message board discussions.  “Server Side Scripting Curriculum” doesn’t return much either.  Likewise, WaSP have not yet published their recommended server-side scripting course recommendations.

So, I’ve had to come up with a lot of the course unilaterally.  The learning objectives are as follows:

By the end of the course, students will understand:

• Basic programming skills
• Web application development principles
• How to find and use PHP resources on the Internet
• How to manage tasks, bugs, and revision control mechanisms for web projects

By the completion of this course, students will be able to:

• Program and deploy simple scripts in PHP
• Process web-based input and output in PHP
• Build multi-file PHP applications
• Connect PHP applications to a database
• Design a simple dynamic website using PHP, MySQL, HTML, and CSS

After a few iterations of course development, here are the topics I have planned to discuss, in order, thus far:

• The current “State of the Internet”
• How the technology works – Web servers and clients
• Eclipse setup and good  practices for personal workflow
• PHP basics – Syntax, variables, arrays, functions, iterators, etc.
• Form processing (GET/POST)
• Database integration
• Multi-file application organization
• Classes and objects
• MVC frameworks
• Revision control and associated workflow
• Basic management: Ticket and bug tracking
• Web Services – SOAP and RPC
• XML integration
• Comprehensive application organization

That’s a lot of information.  Fortunately the students will have object-oriented experience AND database experience.  So, we should be able to leap off the cliff quickly and into the world of iterators, encapsulation, and inheritance.

I’ll post the week-by-week calendar when I finalize it, which necessity dictates will be very soon.  Feel free to grab it and use it for your own courses, or tell me what I can improve on.

Note: This article assumes that you already have a working CodeIgniter application setup.  If you are interested in getting started with CodeIgniter, there are some really great tutorials on the web.  If you’re not using CodeIgniter, skip to the “What to Check” section below for a general idea of how to apply this to PHP apps in general.

Setting up the Hook

The best way to setup a preflight checker is to create a hook.  Make sure that you enable hooks in your main configuration file before getting started.  We’ll be setting up a pre-system hook, so add the following code to your application/config/hooks.php file:

/*Preflight Check Hook*/
$hook['pre_system'] = array('class' => '',
'function' => 'preflight_check',
'filename' => 'preflight_check_hook.php',
'filepath' => 'hooks');

This will tell CodeIgniter to run this hook every time the system starts.  As you can infer from the above code, we’ll now have to create a file in the application/hooks/ folder (if it doesn’t exist, create it) named preflight_check_hook.php and add a function to that file named preflight_check():

/**
 * Preflight Check checks all of the prerequistes before the first
 * pageview for each session.
 *
 * If a problem is found, the program halts execution.
 */
function preflight_check()
{
  //all your checks go here
}

Now we’re ready to add checks so we can test the server environment.

What to Check

Technically, you can check whatever you want here, but I tend to check the things that are likely to be different from one server environment to the next.  Some of the more important ones are:

• Whether the version of PHP or CodeIgniter is new enough to support your application.  You can check this by using the phpversion() function
• Necessary PHP Extensions installed?  You can check this by using the function_exists() function.
• PHP.INI variables set correctly?  You can check this by using the ini_get() function.
• Whether certain folders are really writable on the server or not
• Whether files exist or not.

Of course, you can check for most anything you want to in the preflight check, and even try to tweak the environment a little using ini_set().  What you can’t do is access most of the functionality in CodeIgniter, since this hook will run before most of your libraries are loaded.  There are a few useful CodeIgniter variables and functions that you will have access to, though:

• show_error() — For printing out errors (non CodeIgniter users should use the die() function)
• is_really_writable() — For testing if a file is writable or not (non CodeIgniter users should use the is_readable() function)
• $application_folder and $system_folder

Rather than bore you with lengthy explanations of each type of check, it will be easier to simply show you an example of how I typically do it.  Below is an example preflight check hook that I ripped almost verbatim from one of my applications:

function preflight_check()
{
	global $application_folder;
	global $system_folder;
 
	//Ensure proper version of PHP
	if (phpversion() < '5.1')
	  show_error("This application requires at least PHP 5.1 to run properly!");
 
	//Ensure SQLITE extension is installed
	if ( ! function_exists('mysql_connect'))
	  show_error("The sqlite PHP extension is required to run this application!  Refer to <a href="http://us.php.net/manual/en/book.sqlite.php">the PHP documentation for this extension</a> for more information.");
 
	//Ensure GD2 is is installed
	if ( ! function_exists("gd_info"))
		show_error("The PHP GD2 image library is required to run this software.  Check with your server administrator or hosting provider, or <a title="Installation Instructions for GD2>refer to the PHP documentation for installation instructions</a>.&quot;); 	//Check if the system cache directory is writable 	if ( ! is_really_writable($system_folder . " href="http://us2.php.net/manual/en/intro.image.php"></a><a title="JSON extension installation instructions" href="http://us3.php.net/manual/en/book.json.php">manually install it</a>.)");
 
	//Check memory limit and attempt to set it to what this application needs
	if (get_bytes(ini_get('php_value memory_limit')) < 33554432)
	{
		if( ! ini_set('php_value_memory_limit', '32m')
			show_error("The memory limit in your PHP.INI file must be set to at least 32 megabytes (32m).");
	}
}

You can copy this code into your application/hooks/preflight_check_hook.php and try it out.  Basically, it will halt execution (via the show_error() function) and show an error on the screen that alerts the user to something not being setup correctly if any one of these checks fails.

What’s the Advantage over just letting the program fail on its own?

Technically, nothing; your preflight checker will check things that would normally cause your application to fail at some point in the execution anyway.  This is simply a way for you to alert your admins/developers/users that they need to fix their environment right off the bat.  Plus it’s more graceful and clean than PHP error messages.

Practically, however, this may save a lot of headaches, especially if you distribute the application or expect people to download a copy onto their own desktops for development.  Oh, and if your users/developers/admins have PHP error reporting turned off on their system, then a carefully coded preflight checker will save them from getting a mysterious blank screen when the environment doesn’t match the requirements.

Keeping this thing from running for every page view

You probably won’t want this script to run every time a  page is loaded on your site, especially if the script includes file operations, lest your server take a performance hit.  Typically, you will want to run this preflight checker only once per session.  By adding a few lines of code to the top and bottom of this function, you can make it run only once per session (unless it fails).

function preflight_check()
{
	//See if this has already been run during the current session
	session_start();
	if (isset($_SESSION['preflight_check_pass']) && $_SESSION['preflight_check_pass'] == 'pass')
		return TRUE;
 
	global $application_folder;
	global $system_folder;
 
	//Ensure proper version of PHP
	if (phpversion() < '5.1')
	  show_error("This application requires at least PHP 5.1 to run properly!");
 
	// All of the
	// rest of your checks
	// go in here...
 
	// ...
 
	//Woo hoo.. the check passed!
	$_SESSION['preflight_check_pass'] = 'pass';
}

You’ll notice that this example uses PHP’s built-in session system rather than CodeIgniter’s session library.  This is because CodeIgniter’s will not have been loaded yet when this hook runs, and because we’re not storing any particularly sensitive information in this session, so we don’t have to take advantage of CI’s advanced session security features.

Well, that’s about it for now.  If you have any comments or see any issues with the examples in today’s post, feel free to share by leaving a comment.

Ensuring portability is an absolute must for programs that are packaged and distributed for public consumption, like Wordpress or MediaWiki.  But, you should aim for portability in all of your applications.  The ideal is a one or two-step installation process and only two or three system requirements:  1. Webserver (IIS/Apache/etc), 2. PHP v5.X+, 3. MySQL 4+.

Why Make it Portable?

For one, it keeps things simple.  The concept of portability meshes well with the KISS concept that I value so highly.

Secondly, it makes deploying your application to a new server simple if the need ever arises.

Thirdly, it enables you to easily distribute your package to the world if that is your intention.  I, for one, simply hate installing web applications on my server with long lists of prerequisites and a lengthy install process, especially if one of those steps is to install another set of libraries (e.g. from PEAR) before installing the application itself.

Finally, portability makes distributed development easier.  Many applications are developed on Windows but run on Linux.  Portable applications eliminate the headaches involved in trying to get the to two environments (development & production) to match.

How to Make Your Application Portable

A truly portable application will work out-of-the-box, and require minimal configuration.  It will also automatically check the environment for specific configuration or prerequisites that do happen to be required.  Below are five things to try and avoid, and one thing to try and do to ensure your application is portable:

1. Don’t rely on exotic PHP extensions

I’m talking about extensions that are not bundled with PHP or found pre-installed on most hosting providers.  If you do require a specific PHP extension that is not bundled with PHP, you should check for the existence of the extension at the beginning of the program’s execution.

For example, many of my applications require JSON libraries, which some versions of PHP5 do not include, so I check for the existence of the json_encode function when the application first runs.

2. Don’t rely on separate installation of PEAR Libraries

Don’t get me wrong; PEAR is great.  But, you shouldn’t predicate the successful execution of your application on your users installing external libraries.  Most PEAR libraries are dependent on other PEAR libraries, and require the user to figure out and install multiple packages.  Your application should not require this pre-setup, and it should run with all of the required libraries and classes included.

3. Don’t rely on command-line configuration

If you want your application to be truly portable, you have to cater to the crowd who has only FTP access to their servers.  Yes, these folks are few-and-far-between, but coding with them in mind ensures that your application setup procedure is as simple as possible.  This means that your installation instructions should be completely void of anything that says “login to the command line and run xyz command”.

Definitely don’t rely on the user to have administrative privileges on the server.  This means to try to avoide the necessity for external software or packages (although this is unavoidable sometimes).

4. Don’t rely on the program to be run on a specific operating system

This one is  a stretch for some people, but truly portable applications can run on Windows, Linux, or MAC servers without fuss.  I make all my applications portable enough to run on any OS simply because we use SVN for development, which requires my developers to download fully working copies of the project to their desktops.  Since my developers use Windows & Mac, and my servers run Linux, my applications have to be cross-platform compatible.

5. Don’t rely on PHP.INI customizations

A lot of PHP applications I have tried won’t work with magic quotes (or vice versa), or won’t work without the user increasing the amount of memory or upload file size for PHP.   Before making these nice-to-have’s required, ask yourself if you can find a different way to solve the problem.

For situations where it is unavoidable to ensure that a specific directive is set, your application should automatically attempt to override the default PHP.INI customization via ini_set() and then inform the user in plain English that the ini value needs to be changed.

6. Do include a mechanism in the application to test for required prerequisites at the very start of execution (or as part of the installation process)

Rather than have your application spit out a generic PHP error message when it comes across an environment setting that it doesn’t like, you should include a function that performs a “pre-flight checklist” at the beginning of runtime for each session, or at least the very first time your application is run.

Next week, I will show you how to do exactly this by creating a hook in CodeIgniter that thoroughly tests the environment, and informs the user how to remedy issues.

For Further Reading, I recommend:

• Writing Portable PHP Code from PHPwact.org
• Making Your PHP Code Portable from mtdev.com

It’s been exactly two years, six months, 1 day, and 12 minutes since I last published a post on my website.  You could say that I took a vacation from blogging got a real job, so coming back to the activity of blogging feels like running into an old friend that I’ve hardly talked to in years.  Do you feel the tension?!  Recently, my sister asked me to dig up some old blog posts, and you know what?  Reading my old posts, I remembered just how much fun it is to write.

The problem is that my writing skills have grown rusty, since I have spent the past eight years in a “business” environment writing:

• mostly bullet
• statements directed
• at managers and deans
• who “don’t have time” to
• appreciate literary wit…

…so I’ve been trolling the Internet looking for good tips for blogging.  Here’s a smattering of what I’ve found:

1. Offer the Reader Something

So, I can tell you right now that this post will offer you absolutely no rewarding value.  There it is.  In fact, I wrote it just in order to have something besides the Wordpress default post to start off with.  So, now that we have gotten that out of the way, the next tip I found was:

2. Include Bulleted Point Lists

Done!  See my bulleted list above.  Actually, I’m a pro at this.  One thing I’ve learned over the past few years is that nobody in my department has an inclination for reading.  Especially not emails.  Especially not from me.  So, I’ve learned to avoid complete thoughts, because writing bullets is just way more effective.  But anyhow, I also read that you should…

3. Make Headlines Snappy

What could be better than “Hello World!”  And, I’ve included a nice “hello”-themed picture to go along with it.

4. Write with Passion

You can’t see me right now, but believe me, I am writing with much passion and vigor.  The reason for this is that I have had at least eight cups of coffee in the past three hours.  My right eye is pulsating like crazy and my hands are like paint shakers.  But, as Paul Erdős has taught us, without amphetamines, there can be no creativity.

5. Publish Often

Well, if you consider every two years to be “often”, strictly speaking, then I am good.

Seriously, I do plan to put a lot of useful stuff here.  The problem most people have with blogging is thinking up good content to share.  Some people spend weeks tinkering with their blog’s design and playing with plugins but then forget to, y’know, write something.  I, on the other hand, have a laundry-list of topics queued up to be posted here.  So, now that we’ve cut the tension and gotten introductions out of the way, I look forward to reacquainting myself with my blog.

It’s been a long time, old friend.