There's an article in the latest eWeek magazine, entitled "Scaling Ruby on Rails", that's instructive for those of us in the ColdFusion community. RoR is going through that time-honored rite of passage experienced by any programming language that gains wide adoption. Namely, so-called critics are questioning its scalability and its suitability for mission-critical applications. Sound familiar?

One supporter of Rails notes the following:

"The critiques we hear about Rails is it's not scalable, that it's not well-suited for mission-critical applications. I think those critiques are similar in nature to what we heard about Java in the mid-90s."

And my favorite retort to the scalability questions comes from Rails creator David Heinemeier:

"This is the known as the 'last stance' defense. When you have nothing left of substance to argue with, you draw the 'but does it scale?' card."

Awesome. Amen, my brother. I couldn't have said it better myself. And the exact same response applies to ColdFusion, just as it did to Java and many others over the years. Once you've tuned a few hundred queries in your career from 3000ms to 30ms, you come to know that the scalability question is almost always a red herring. To paraphrase a quip from politics, "It's the code, stupid!"

Our co-founder and former CTO, Mike Brunt, tells a great story about when he was onsite doing some of our famed tuning work some years ago. He had been applying our tuning efforts on a system well into the night during an engagement, and around 6am the next morning, when user traffic ramped up dramatically every day, he got a frantic call from the CEO, breathlessly saying that the servers were all down. And why did he think this? Because the traffic monitoring graphs had all dropped to sub-1-second response times, such that they looked so different than "normal" that that he concluded there must not be any traffic on the servers. In true British form, Mike said something along the lines of "Bollox to you and your bad code--the servers are fine!" Again, "It's the code, stupid!"

So, to all ColdFusion compatriots--don't fall for this nonsense. Ever. If you hear it, smile. It's just a reflection of the prominence of your platform. And if you're stuck on performance problems, contact us--we've never NOT tuned a system we've been engaged to tune.

To our brethren in the Rails community--welcome to the party. Come on in, the water's warm. Now that you're hearing all the same "last stance" questions that ColdFusion has been dispatching for years, we wish you all the growth that ColdFusion has experienced!

Here's another section from my presentation at CFUnited:

//

This segues into the question of what to do when the customer needs an "estimate" or requires a "fixed bid"? I think I just answered that one - you write a thorough spec doc, break it down into tasks and provide an estimate based on your anticipated schedule. Submitting a bid without a spec doc in hand is, frankly, suicide. If you submit a bid at, let's imagine, $1000 for a project, and oh-my-gosh, you actually come in at that budget, you weren't right. You were wrong in your estimate since you didn't know what you were estimating and just happened to come in at the same amount for the development. It's like back in math class when you had to write out your solution on tests because the teacher wanted to know you did the problem right, not just guessed at the answer. It's the process of breaking out a spec doc into tasks that lets you come close in your estimates. (Estimating is another gigantic topic in its own right, and I'm not going to get into it here.)

There is one little snafu with this plan. Many customers want that fixed bid before they're willing to drop any coin. They won't accept a ballpark range/time-and-materials budget for the development of the spec doc without any idea how much their total outlay is going to be, including the coding. This is a total catch-22. I can't commit to a project cost until I know what the scope is, but the customer won't let me determine the scope until I commit to a project cost.

Sometimes in the "real world", I can't always convince a client that they need blueprints and house plans plans before they can get a bid on the construction of a house. What do I do in this situation? Quite simply, I must determine the scope before committing to a price. Anything else is a guess. Here are some tricks I try:

1. Walk away. There are other customers out there, waiting for me, who know that demanding a fixed price without a spec doc is foolish. These are the customers who have been burned in the past.
2. Hire a fancy salesman in a $1000 suit to smooth-talk the customer into waiting for a fixed bid until a "discovery phase" is complete. Don't call it a "spec doc", call it "discovery". Maybe I eat the time spent on the spec doc and jack up the prices on the coding phase to cover your losses.
3. Offer the customer a fixed bid based on just the spec doc process. When you're building a house, you first pay just the architect, right? When your car is broken, you take it to the mechanic for a flat-rate diagnostic fee, right? In order to swing this solution, I craft the customer relationship into an ongoing series of agreements to get from one phase to the next. At the end of every phase, the customer can pull the plug and still have value with the deliverables provided by the previous stage. After all, the final software is just the deliverable from the coding phase, right?
4. Insist that we never work on fixed bids. This is more of a meta-solution. Since we never entertain the idea of working on a fixed bid, we never have to figure out what to do about a customer who needs one.
5. Employ the used-car-salesman approach. When a customer walks onto the lot, a good used-car salesman will sell them something, anything. He takes their money and makes sure they drive off the lot. If the customer must have a total price firmly fixed before we do any work, then I can certainly guarantee that something will be delivered within that budget. What that something is, is the scope of the project, something no one yet knows, but we get to determine it by working until the customer's budget is dried up. This is roughly what the iterative development folks do, but can create big troubles because of unrealistic customer expectations.
6. Educate the customer by encouraging them to perform their own initial discovery phase. I show them a template of a previously-completed Statement of Work or Preliminary Project Plan and have them create the first draft of it. Some customers are capable of producing this kind of document. We may end up giving away some services assisting them with creating the document, but when I have it in hand, I have a good idea of how much work it will take to perform the spec doc process.

In all these cases, what I'm trying to do is nail down the scope of the application. By determining the scope, I answering many of the big, hard questions and provide a direction and some details to the project. This is the only way to produce a fixed bid without wildly guessing.

There's a pretty good article about job titles in the current ColdFusion Developer's Journal:

Are the Job Titles "Web Designer" and "Web Developer" Too General? - There are a lot of professions that have emerged from the web: designers, developers, strategists, search engine optimists, information architects, usability and accessibility consultants, the list goes on... Today, I'd like to talk about the first two. I wouldn't go so far to say that the titles should be considered harmful by any means, rather we have just outgrown our job titles!

My favorite part is where the author, at his most recent job, gave himself the title "Front End Engineer", because it seemed to fit what he was doing. As it happens, just yesterday Nat (Papovich, Lead Architect here) and I were having a discussion about how, with the recent evolution of rich applications in the Web development realm, it's time for a more-or-less official title of UI Developer. There's no doubt now that the proliferation of development tools/languages/frameworks/etc. that are solely focused on producing better UIs means that there's a whole lot more programming that happens just in the UI layer of an application. Just taking one tool as an example--Flex--requires proficiency in a range of skills, including OO development (ideally, but not required), CSS, Actionscript, MXML, and more.

In our development practice, we certainly see this playing out. We typically have several developers working solely on the UI layer, be it AJAX (in its million different forms), Flex, etc.

On a related note, I recently read about the new workflow integrations between Flex and design tools, which is a great thing. What I find interesting about the "UI Developer" role is that, on average, I think it entails developers learning better graphic design, usability, and other "designer" skills more so than it entails designers learning to program. But that's a side note to the whole store. The bottom line is, the fact that we have Front End Engineers, or even the possibility of them, is great for users, and it's another sign of the maturing of the Web as a platform for business computing.

I was reading with some amusement Doug Boude's retort to a Java purist that was dismissing ColdFusion for being too... simple? to be considered a "real" language, and I was reminded of a topic I commonly rant on. It's what I call the Three Phases of Programmer Development.

Phase One

A novice programmer is happy to get anything to work. Eventually, the novice programmer becomes a journeyman programmer, and is able to not just get things to work, but to start to create some truly complicated programs.

Phase Two

Here, a journeyman programmer starts to develop their chops. They can take a complicated problem and build applications with a level of complexity equal to (or greater than) the problem at hand.

Phase Three

Programmers usually find Phase Three when they've been asked to go back and maintain some Phase Two code they wrote six months or two years ago... and all they can think was "WTF was I doing /here/?" "What's /this/, now?" "Geez, did /I/ write this? Wow."

The epiphany that leads a programmer to Phase Three is the realization that the hallmark of the professional programmer is not in the complexity of the code they produce, but rather in the simplicity. When you can take a complex problem, and solve it with simple code, you are a professional programmer. A lot of people that are die-hard ColdFusion fans are Phase Three programmers, who use it because it allows them to produce simpler, more maintainable code. And a lot of ColdFusion's detractors are Phase Two programmers, who think that anything /that/ simple must be for novices/children.

Note that frameworks of all types can often be categorized into Phase Two or Phase Three frameworks. The question to ask is, "in the long run, does this framework (or design pattern) make this application simpler, or more complex?" Or, "Will this framework make troubleshooting errors or performance problems nightmarish?" Or, "do I have to use this whole bloated framework for this problem, or can I just use a few of tastier bits to address this issue at hand?" If the use of a framework or design pattern makes the solution simpler, or at least more maintainable, great. If not, it may be good to evaluate whether you're using it for Phase Two or for Phase Three reasons. I suppose it bears mentioning that "resume-building" is never a valid reason for the use of a pattern or framework. That being said, having at least a passing familiarity with many patterns and frameworks is good, so that when you start designing something that does fit an existing solution well, you know to use it.

[n.b. I often recommend that people new to ColdFusion or Java (or any other language) read the description of *every* function or JDK library method. Not for the sake of memorization, but so when you run into a problem that's already solved, the voice in back of your head can say, "Hey, I think I saw something that does that..."]

Code complexity is like an infestation; it tends to spread to all areas that touch it. When you find code for a problem growing overly complex, STOP. Take a moment to understand where the complexity is coming from, and refactor it to something more simple; don't let complexity sprawl and infest the rest of the system. I tend to dislike ORM implementations, because they tend to reduce object cohesion and often require you to write more code in support of the ORM than writing simple loadFromDB(id)/loadFromResultSet() methods would require, and I've found performance (for other than basic CRUD applications) to be anywhere from unpredictable to abysmal. I know this makes me something of a heretic, and there are bound to be people that insist I was doing something wrong or "just don't GET it", but when I recently ripped Hibernate out of a Java application, the code size shrunk by about 30%, and the performance improved by an order of magnitude. (For the record, I was the one who put it in there in the first place, because I bought into the hype. Now I understand better what it's good for, and what it's less good for.)

It bears repeating: the hallmark of the professional programmer is not the complexity of the code they produce, but rather the simplicity. Whenever you see code getting complex, stop and ask yourself if there's a way to do it simpler/better.

I've been working on my CFUnited presentation titled "Reviving the Lost Craft of Writing Specifications" and want to publish another teaser that might make it into the final version. This is raw, rough, uncut and unedited so beware.

//

What About Data-Centric Applications? Early in the project, you need to determine whether this is a data- centric application or a UI-driven application. If it's a data- centric application, then you should get into the existing data, or the creation of the ERD very early in the process. I believe that "version 2 rewrites" are data-driven applications. These kinds of applications may require some kind of data importer or massager if the database schema changes and importers can be one of the most costly parts of a software application and can, in some extreme examples, make requirements on the rest of the application by highlighting relationships among data objects. Also, if your data-centric application has a poorly-organized database schema which is unable to handle and manipulate the data set, the entire application (UI and all) will fall apart very quickly and may generally be unrecoverable. But the opposite is not true - a UI-driven application with a crappy database isn't necessarily a recipe for instant failure. There are many large commercial applications with atrocious underlying data-storage mechanisms.

If the application is a UI-driven application or is a version 1 application or has no existing data to import, the database schema can be ignored until later in the project. But in either case, the prototype/wireframe must be created early and often. These two classes of applications are the only types of applications I'm talking about here. I don't know anything about writing device drivers or scientific analysis programs, but I'll bet dollars to donuts that those classes of applications also make use of spec docs in some format.

I've been working on my CFUnited presentation titled "Reviving the Lost Craft of Writing Specifications" and thought I'd publish a teaser that might make it into the final version. This is raw, rough, uncut and unedited so beware.

//

Sometimes, a client knows what they want in the end, but really truly can't tell you all the component pieces to get to that goal. One day, you get around to reading Robinson Crusoe and fall in love with living in a tree house. You know that a tree house is exactly what you need to make yourself happy forever and ever. So you hire an architect, buy them a copy of Robinson Crusoe and tell them that you want a tree house like the one in the book, only modernized. The iterative programming folks would have you climb up in the tree every day with the work crews while they saw and nail and screw and prune, asking you questions while you're in the tree about attachment points, wind-shear factors and various things you know that Robinson didn't think about when he was building his tree house because they weren't in the book. After the structural builders are done, the trim guys are up in the house every day putting up crown molding and running pipes and lights and they keep asking you questions like, "Would you prefer the 15 watt halogen soft-white bulbs or the 45 watt incandescent natural glow bulbs in these pendant lights?" If you wanted to be wrapped up in that level of detail, you'd have built the damn thing yourself. Sometimes you just want someone else to execute your vision. You pick a dynamite tree-house builder, someone with dozens of houses under their belt, especially of the shipwrecked-looking-variety, you read them some passages from Robinson Crusoe and then tell them to call you when it's done. And if you pick a really good builder, it'll be just like you dreamed it would be.

To be the software developer equivalent of this latter type of builder, you need to do two things well: learn and communicate. The learning comes from your ability to rapidly acquire a deep and wide understanding of the business domain at hand. If it's the futures trading market, you need to learn the difference between a pork belly and a cheddar barrel, a short sell and a long put. If it's healthcare billing, you need to learn the difference between CPTs, DMEs, and CCIs. If it's electronics manufacturing, you need to learn the difference between servo motors, teach pendants, PLBs and weld tips. In order to get that understanding of the domain, you need to be a good communicator, and get buddy-buddy with another good communicator who already is an expert of the domain you're learning (generally called the "subject matter expert"). If you become a domain expert, you can translate the fuzzy, ambiguous ideas that your customer spouts out into a workable software design, and the whole time, poke holes in his harebrained schemes that won't work because of domain rules.

Once you're a really good builder, you can make customers feel involved in a process in which they are ultimately not involved by offering them lipstick-on-a-pig options, like with the style sheet. Should the logo be more "swooshy"? How about adding a horizontal rule between the sub-navs? Some customers dig this kind of interaction. Some don't want to be bothered.

Getting to the point where you are a really good builder also has a drawback: You are opinionated and must express those opinions in the face of potential scorn. It goes something like this:

A not-so-experienced builder will dutifully take notes during meetings, distill the essence of a decision into the spec doc, send out the spec doc for review and ensure that the finished application follows the guidelines set out in the spec doc. If you're building a social-networking app to let users watch paint drying, and your customer requires that all users log in just to view other people's paint drying, you'll spec it as such and build it as such.

But an experienced builder would be able to cross over into the business-decision making arena and question why the customer wants to force viewers of drying paint to log in. What's the overriding business need for that? Won't it place an onerous burden on the casual viewer who wants to check out some drying paint? The natural answer from the customer would be that they want to capture email addresses and some other contact information to sell other complimentary services (like watching grass grow) to people who watch paint dry. The experienced builder suggests a compromise: Let any visitor view drying paint, say, twice on the first day that they visit the site. If they want to view a third, they are gently prompted to register with just their email address. Or if they come back tomorrow and watch some more paint dry, we have a pretty good idea that they're habitual boredom-seekers, so they certainly might be interested in watching grass grow or waiting for kettles to boil. Through a slight change in the requirements of the system, you're collecting a fraction of the email addresses that the customer's proposed way would have collected, but all of the emails collected in your way are pre-qualified. That is, they're actually boredom junkies as opposed to just people who StumbleUpon the site. That, along with the fact that pre-registration is proven to turn off users, so some of those genuinely bored visitors might not have even viewed that FIRST drying paint if they were forced to enter their email address in the beginning. This kind of critical thinking and collaboration on your part is what separates the merely adequate from the really good application builders.

At Webapper, we've always described our tuning & troubleshooting consulting as a "wire-to-wire" service. This means that we find and fix performance and/or stability problems wherever they are, even if they're in the network layer (e.g., the TCP "silly window" problem). And beyond troubleshooting production systems, we've always found that HTTP sniffers are useful during development, too. We just heard about a free tool from Microsoft, named Fiddler, so I thought I'd blog it here along with some others that we use regularly:

• Fiddler: It's a free tool written by Eric Lawrence at Microsoft. It's really technically an HTTP proxy, and thus is a specialized tool for sniffing Web requests.
• Wireshark (formerly Ethereal): This is one we've used for a long time. It's a general purpose packet sniffer, and sees all packets, not just HTTP.
• ServiceCapture: Written by Kevin Langdon, we've got copies of this laying all over the virtual office. The killer feature in this tool is its decoding of service requests. It'll decode AMF/remoting packets, for example, and show you their full data structure. Very, very useful.

This Christmas, Webapper was kind enough to shower us each with a Nabaztag/tag wifi rabbit. If you haven't seen these things yet, they're pretty neat... It's got an API (along with prebuilt services) that you can call to make it do various things, like move it's ears, light up, and speak.

Now, the API looks quite simple, it's just URL calls, but building out any request with a choreography just takes a long time. I instantly wanted to create an "Explorer" type application in Flex to make building an API call easier. After a couple hours of playing around, I decided to go all-out, and do an open-sourced API wrapper to power the explorer application.

I did release a couple beta versions over Christmas vacation, but today I'm happy to announce the 1.0 release. Just hop on over to RIAForge to download the source/swc, or view the live demo.

I used to hate version control. Visual Source Safe was slow, obnoxious, and all but unusable over a VPN. CVS was a pain to set up, and I wasn't fond of the command line or GUI tools available at the time. Subversion seemed like a step in the right direction, but again as someone relatively new to the Unix/Linux world, it seemed to be a tad over my head.

I tried setting up a subversion server a couple times, and even succeeded on my Mac Mini, but getting a daemon to start on boot on the Mac is not a simple task, and eventually I stopped using it because I was too lazy.

Eventually, I came across DreamHost. While their shared hosting is probably not a good choice for a medium-to-high traffic website, it turned out to be perfect for me to use as off-site backup and Subversion repositories. With generous bandwidth and storage policies, it was perfect.

After you sign up for an account, they have the typical shared hosting control panel. If you click on Goodies, you are able to create as many Subversion repositories as your heart desires.

Once you've created your repositories, you're going to want to install TortoiseSVN. It's the perfect Windows client, and has integration with Flex Builder/Eclipse (Free) and Dreamweaver (the better plugins are Commercial).

Trac is a very simple project management tool that has integration with SVN. Set up your milestones, add tickets, and it does the rest. You're able to make everything public or private, including browsing the source code right from the site.

Now, the true power of version control lies in a few key areas:

First off, it's a great backup tool, in a sense. You get to keep a copy every revision of every file in your project. And, it's easy to compare any previous version to any other version of the file.

Secondly, it's critical if your project tends to have versions. If you follow the best practices for version control, you can have branches (and tags) of your code for each version of your project, which are easy to switch between with just one click. This way, you're able to maintain the current release while also working on a future version. Yes, occasionally you'll have to make the same code changes twice, but with the version compare tools built-in, it's pretty easy. If you want to learn more about best practices, I recommend this free article/book from openCOLLABNET.

Also, it allows for team projects. Even when you try to avoid it, multiple people will end up editing the same file. Try as you might, it will happen. SVN allows for this, and alerts you when there's a conflict. It has built in tools to handle this, but you'll probably want to manually double-check what needs to be done.

Hopefully, with as easy as it's become, you will embrace version control as I have.

I've been hammering on the Firefox 3 beta since it was released, and I can say that it's definitely snappier than Firefox 2, and hasn't crashed yet. I like that the UI hasn't been drastically changed, but the subtle tweaks are well worth it.

Pros: Snappier, more intuitive UI, no longer dog-slow on my Mac Mini, either.

Cons: All my plugins are not yet compatible, as expected.

I recommend you install it on your development machine today!