clog

When running the 64bit kernel in Snow Leopard, TextMate will fail to execute most scripts.

This produces the following error for me (in the case of a Ruby script):

/Applications/TextMate.app/Contents/SharedSupport/Support/lib/tm/process.rb:169:in `fcntl': Inappropriate ioctl for device (Errno::ENOTTY) from /Applications/TextMate.app/Contents/SharedSupport/Support/lib/tm/process.rb:169:in `run' from /Applications/TextMate.app/Contents/SharedSupport/Support/lib/tm/executor.rb:211:in `parse_version' from /Applications/TextMate.app/Contents/SharedSupport/Support/lib/tm/executor.rb:98:in `run' from /Applications/TextMate.app/Contents/SharedSupport/Bundles/Ruby.tmbundle/Support/RubyMate/run_script.rb:93

I was able to "fix" it by commenting out line 169 in:

/Applications/TextMate.app/Contents/SharedSupport/Support/lib/tm/process.rb

I haven't noticed any adverse affects (yet).

There are several schools of thought regarding how to best write user stories. There also seems to be a great deal of confusion regarding how stories are set up - probably because any research into the topic yields such a variety of opinions. Most of us (like myself) also come from a background that emphasizes technical requirements. Stories should emphasize "business success criteria" rather than technical requirements.

What is Business Success Criteria?

I know this sounds like marketing-speak but there's a reason for the semantics. A requirement can exist for any number of reasons. They may be politically motivated or part of another agenda. A requirement may be good or bad. We shouldn't assume that doing what we're "required" to do will lead to success. This doesn't mean anarchy is the answer either, the point is to succeed through collaboration.

You can call it Business Success Criteria, Success Criteria or Business Criteria (sometimes also called Acceptance Criteria). The point is always the same - to describe a path to success. A direction the project can go. There can be several ways to succeed but all "Success Criteria" should point to a consistent goal. If that goal needs to change then the criteria will also need to change. Stories should describe what it will take for a particular piece of functionality to contribute to the success of the business. Usually this is on a small scale at the story level. These small successes add up to successful features which add up to successful products which add up to successful businesses.

Breaking it into Pieces

Story writing usually starts at the feature level. With a set of large stories (sometimes epics) that are usually fully completed features. Most stories should not be fully complete features. These features should be broken up into several stories. Each story can be described as a minimal sub-component that still retains business value (meeting a small but essential set of business success criteria). Creating a new method for a class for example does not measure progress towards meeting business needs. The ability to create a new user, or the ability to modify that user for an application may both provide small yet essential functionality to the product.

Notice that these two items are mentioned separately. Rather than create a "User Manipulation" story there is a story for "Create User" and a story for "Edit User." You can think of these as contributing to a "User Manipulation" feature. Creating a user provides a small amount of business value, editing a user adds a little more. The code paths related to creating and editing are usually very similar (most likely with several shared pieces). They may even be identical, or use fancy metaprogramming for improved architecture. The best architectural solution is the way to go but from the standpoint of describing the business criteria these still are separate scenarios (or sets of scenarios) each providing their own value.

Duplication of Scenario Steps is not Necessarily a Problem Here

The steps involved with validating these scenarios may be nearly identical. That's OK! Duplication in (story) scenario steps is not such a big deal and many argue that it's actually a good thing. Scenarios should be as self-descriptive as possible, so copy/pasted steps may be just what you need. They shouldn't be superfluous however.

If stories are integrated with your acceptance testing framework you should avoid excessive use of metapgrogramming. Being DRY is still good but scenarios should be easy to understand. Scenarios should be clear and concise. The balance doesn't have to be perfect. Remember, you could spend all of your time architecting your test framework but keep in mind the point is to architect the application. At some point the essential specs and scenarios are in place so move on. If you're following Behavior Driven Development then your story scenarios and specs validate your success criteria and communicates how your system works.

As a side note. If you're writing effective tests, compare the amount of time you spend writing tests to the amount of time other projects spend adding bug fixes and the amount of time they require for integration. Which do you think is a more valuable way to spend your time? Would you rather write tests or create emergency patches? One approach requires more discipline but may be more enjoyable in the long run.

Human-Readable Specs and Scenarios

Your stories serve as a communication point between business and development. They should make sense to both developers and the business. Story Acceptance Tests should document the interface of all of the system components. The "Specs" (unit tests for the pre-agile) should describe the inner workings. Think of this as API documentation that a business person could make some sense out of, but more likely to be read by a developer that is new to the system. Sometimes (especially for large/difficult features) you need to describe a component that lies in the middle. This is similar to functional testing. In these cases you can validate functional components using your story testing. These are story tests that are "gray box." You peak into the internals a little but no more than possible. You should not use stubs or mocks for these. These stories should be used with care because they are describing technical aspects of the project. They should be relatively rare.

Tasks or Sub-Stories?

If you follow standard Scrum as defined in works such as "From the Trenches" then your stories may be more like the features I describe above which become broken into several separate tasks.

Advantages of doing it this way are that stories stay "pure." You don't have to worry about technical leakage into stories because these become technical tasks (but then you need to ensure that these tasks aren't superfluous). You can also have separate individuals working on different technical aspects of the story. This may or may not be a good thing. I usually think of it as a bad thing - preferring pairing and approaches involving more collaboration. Sometimes it's an unavoidable project requirement. Tasking may be a better approach for these projects. It also takes less time to write the stories themselves since you don't need to worry about many details. Downsides of tasking are that stories tend to be too large, making estimation less accurate and producing less detailed documentation. Tasks also tend to be redundant, with many smaller stories requiring the same tasks (create a model, create a controller etc). Large/difficult stories involve extensive time in the tasking phase, which can make IPMs drag on for a very long time.

Breaking stories up into sub-stories (creating the feature/story relationship) addresses the disadvantages of tasking but introduces some of its own. Note that the tasking method still involves the breaking up of "epics" just not to the same level of detail. When using this method, almost every story ends up being broken into smaller sub-stories. Story writing takes more time up front since there is more detail at the story level. It's hard to assign more than one person to a small story unless they pair. Stories are usually easier to estimate and track. Less time is spent in analysis during IPMs. Testing is usually easier since they are more targeted (despite the need to test more stories). There is a point of diminishing returns, however, regarding the reasonable lower limit of a story size. I prefer a typical story size that is either a single work-day or half a work-day to complete. Sometimes exceptions are necessary but they should be rare.

Start at the Beginning

If you are on a project that is completely new to Agile then I recommend starting with the reference scrum standard. Only start modifying your workflow/practices as you begin to understand the core concepts of Agile Development, which take much longer to sink in than you think. Most people tend to think in terms of processes rather than practices. This is the wrong approach. Often, when I outline a development workflow to management they don't see it as a workflow at all due to the exceptionally low process. It is much better when good practice keeps the process in check, rather than the other way around. An experienced agile coach can help a project or organization be as agile as possible while still recognizing business realities.

Nothing is Perfect

There is no perfect way to define success criteria, prepare for development, to ensure 100% success or revenue return on the product itself. Inevitably you will run into epics that are difficult to break apart or success criteria that are a little too technical. At some point the rubber needs to meet the road. Give it your best and fix problems along the way. The idea is to have efficient, sustainable and enjoyable development.

For an explanation of how stories should be narrated and scenarios written, the following blog by Dan North is a good description. There are some slightly different approaches out there. Cucumber adopts the "In order to x" style of narrative which puts the emphases on the reason where Dan North describes the "As a x" style of narrative which emphasizes the user. Since the concept of "all stories need a reason" is so important to agile techniques I prefer the "In order to" syntax. The essentials are the same though. There's also a stronger emphasis on shorter stories like I described above now - but the rest should help fill in the blanks.

http://dannorth.net/whats-in-a-story

Well OK maybe it wasn't quite THAT bad, but it was certainly difficult enough that the Ruby novice would have really had trouble. I upgraded from Typo 4.1 to 5.3 on DreamHost. It turned out to be quite a hassle. There were several obstacles. Before upgrading I read over the Typo upgrade checklist. It was pretty straightforward other than I interpreted removing my sidebar items to mean just reprogramming them. Apparently you should remove (at least some of them) from the database altogether. First challenge was getting migrations working. Dreamhost kept killing my migration in progress due to the high memory usage. So I ended up doing a mysql dump:

mysqldump -u USER -p PASS -h HOST > clog.mysql

I copied that over to my macbook where I loaded the typo gem, created a skeleton blog app and loaded the db for the migrations. Still no luck, I could not get typo to load at all on my mac. I kept getting this error:

[BUG] cross-thread violation of rb_gc()
ruby 1.8.6 (2008-08-11) [universal-darwin9.0]

I have ruby 1.8.7 installed via ports so this was definitely incorrectly linking to the original ruby install on my mac. Even after renaming ruby binaries, libs and reinstalling all gems I still had this problem. Others had luck with this but not me.. I decided the path of least resistance would be to use Ubuntu. So I copied my dump file over to my Ubuntu machine, did the migration (it worked - YAY!), made a new dump (clognew.mysql) and uploaded back to my Dreamhost account. There I loaded it directly into the database and restarted Typo. It worked! Well... almost... My sidebars wouldn't work, I would get the following error and could not find a way to fix it: It seems something went wrong. Maybe some of your sidebars are actually missing and you should either reinstall them or remove them manually So I started a Rails console:

script/console production

And tried to pull all sidebar models:

>> Sidebar.all
ActiveRecord::SubclassNotFound: The single-table inheritance mechanism failed to locate the subclass: 'AimpresenceSidebar'. This error is raised because the column 'type' is reserved for storing the class in case of inheritance. Please rename this column if you didn't intend it to be used for storing the inheritance class or overwrite Sidebar.inheritance_column to use another column for that information.

So my problem is that this Aimpresence model no longer exists. To fix the problem I actually created it in the console

>> class AimpresenceSidebar < Sidebar
>> end
=> nil

And finally I found the model to delete (querying by name would probably be better, but for me just pulling the 'fourth' worked) NOTE that your Sidebar probably will be in a different position

Sidebar.find(:all, :order => 'active_position ASC', :limit => 4).last.destroy

Now finally querying for all sidebars works and the production error is gone.. hopefully this will help others that find themselves in this situation

Following directions I found here:

http://mkoga.wordpress.com/2008/09/09/zsh-tab-completion-fix-for-subversion-15/

I was finally able to fix the extremely annoying ZSH+SVN tab completion bug.

I have zsh installed via ports so I copied the subversion file to: /opt/local/share/zsh/4.2.7/functions/subversion

Now I finally have tab completion back.. happiness

I’ve come across a number of web applications recently that cross-link between other applications to steal their functionality.

This sucks!

It leads to poor application design as sooner or later developers come along and are naturally confused about what should go where. They then increase the amount of cross-stitching between the applications. Eventually you end up with one large application that’s not so great at doing two things. You have to get both applications working independently when you just want to use one. It wastes time and it creates a mess. The more applications you connect this way, the worse it becomes. It lowers the quality of both applications and kills code velocity.

Stop the madness!

For those of us that are Rails developers there is a pre-determined place to put outside dependent functionality - the ‘vendor’ path. If you put all of your external dependencies there you have a nice self-contained application that can stand on its own. Functionality shared between applications can be broken out into mutual dependencies and be pulled into the respective applications as plugins, frameworks, or gems that are stored in the vendor path.

If you have a framework, put it into “vendor/framework.” This is where rails goes for instance (vendor/rails).

If you have a plugin, put it into “vendor/plugins.” It’s the standard plugin location for Rails.

If you have a gem, install gemsonrails and put it into “vendor/gems.”

Now everything is nice and tidy. All you have to do to get your application ready for development or deployment is to check it out. No gem installations or framework installations. The dependencies are already met as soon as you get the code.



I started phasing into a TDD workflow about year and a half ago. It took awhile to learn. Fixtures and test:unit were the tools of the time (so 2006). I didn’t see much about mocking, stubbing, or even factories until I started getting into Rspec about 5 months ago. TDD definitely improved the quality of my code but it slowed me down. I started programming with BASIC in 1984 after all (TI-99 ftw) so I have over twenty years worth of habits to overcome. Even now the amount of time I have spent doing functional programming dwarfs everything else.

Developers in particular must learn how to constantly adapt. Software Engineering has a long way to go and phase-gate/serial cycles aren’t the direction it needs to go in. Unfortunately there is a lot of resistance to changes in programming languages and methodologies. Most people cling to the old ways, the ways they were thought. A professor they respected told them to UML model every detail before starting to code so that’s what they do. Most people don’t want to constantly re-learn how to do their job. If they’re a developer then they need to find a new career path. We need to learn better ways of doing things and TDD is one of the most important improvements.

These days I’m doing 100% TDD and will never go back. I finally have the work flow down and not only has my code quality dramatically improved, so has velocity. My team is doing all TDD now and we burn through development at amazing speed. Our QA team almost never finds any defects because the developers are just as thorough during the TDD process. It took awhile to get to this point, but TDD doesn’t slow me down anymore - it helps catapult me ahead.

How many times have you received a hunk of completely untested code? Yeah I lost count too. I imagine your experiences were like mine, I bunch of mangled code that made little or no sense that really just needed to be tossed. Instead you have to black box it and put in a bunch of wrapper functionality. It’s legacy code. We’ve all experienced legacy code and it sucks. The really sad part is that we all wrote the same type of code that others are now wrestling with.

Untested code is legacy code. Everyone seems to assume that OLD code is legacy code. This isn’t true. C now isn’t much different from what C was 30 years ago. If code written in C a long time ago came with a complete testing suite, then all you would need to do is get it compiling and running. After that you can gradually make changes and rely on the tests to help lead the way. Inheriting Java code from 2 years ago is no different. If it doesn’t have tests, it’s almost certainly going to be legacy code. That means black boxes, wrappers and pain.

TDD ensures maintainability, provides documentation, helps in the design process, exposes architectural flaws early (you might think you’re an awesome coder but, trust me - you still have them) and (once you get it down) actually speeds up development. It also gives your code a life support system. Someone that comes along several years later should be able to pick up where you left off.

Give it a try and stick with it (if you haven’t already). It’s the only way to roll.

I always seem to be on the losing side of arguments that debate the finer points of the Universe. Sometimes the details are important but most of the time they’re just noise.

In these cases, I tend to follow the convention unless I have a strong reason not to. I don’t necessarily know how the convention came to be in all cases.

Unfortunately it’s surprisingly hard to convince others to do the same. Everyone likes to think they’re an expert. Expert is a subjective term though. As a developer, I picked the Ruby and Rails camps. I don’t necessarily blindly agree with everything that comes out of the mouths of Dave Thomas, Jim Weirich, and DHH, but I have learned to trust their judgment.

Finding mentors makes life easier. If you know you can trust the words of someone 90+% of the time, then you can build on what they know and not bother repeating the same arguments they have gone through. This is how “experts” work. Experts have a network of peers that they trust. That’s why they advance so much more quickly. They don’t bother repeating the details.

In order to become an expert you have to assume that you don’t already know everything.

I’ve encountered this situation once again. I’ve been following the RESTful Rails naming convention of using URIs where others use URLs. I did this because I trusted the judgment of the Rails Core team that this was the proper term. I always hated the interchangeability of the two so I was fine with just using one.

The debates came. There were inappropriate URIs strewn all over my code! These are supposed to be URLs I was told. I was asked WHY?! Why oh why have you strewn sloppy terminology all over our codes? My answer of that’s the Rails convention was not good enough. My peers wanted to replace all of these, I said “fine - if you really want to take the time on that.”

So… migrations are added, tests cases modified , source, documentation - all changed to reflect the “proper” naming scheme.

The truly ironic part. The initial naming scheme was right. That’s correct. All of that changed things to the WRONG terminology. It turns out that the Rails core team was right (imagine that). While it’s true that a URL must have an http, ftp, etc protocol at the beginning - it ALSO has to have an extension at the end that says how the resource should be presented. The concept is completely outdated now - which is why URL is considered obsolete terminology (just Google around for that).

I found the below… URI.. which describes the fine points of how the two differ.

http://ajaxian.com/archives/uri-vs-url-whats-the-difference

The lesson is to not assume you know more than everyone else. When a group of experts agree on something, maybe they’re right! Investigate the issue if you like, but don’t assume they’re wrong and you’re right.

Many who have worked with me in the past know that I always resisted including search servers (ferret/lucene/solr) into my projects. I argued that they added unnecessary complexity and didn’t provide much that some fancy database searching couldn’t do.

I was wrong.

While I suppose that I still consider the above arguments to be valid, there is one more important argument on the side of using search servers - scalability. I share many rubyists’ disdain for the “S” word. It gets thrown around a lot and typically results in overbloated software that still doesn’t perform well even when “scaled.” There really isn’t a better word in this case though.

HTTP is incredible. As the recent RESTful advocates have pointed out, the protocol is infinitely scalable, simple, and is (in a large way) responsible for the explosive growth of the web. Web servers getting bogged down? No problem, just add another (as long as you’re not trying to do any - or at least not much - state tracking). You can keep adding servers to your heart’s content. A good admin can have this process down to a five minute setup.

Eventually all of those database calls add up though and your database starts straining under load. What do you do now? You can start optimizing your queries, do some fancy caching, add a slave database and keep a read-only connection open to it or (the mother of all evils) create a database cluster.

All of these options suck.

All of that beloved simplicity and scalability of HTTP is gone and replaced by infrastructure that’s extremely expensive and hard to maintain. Sometimes you just can’t get around the database cluster - but it should be a weapon of last resort. Query optimization isn’t so bad if done correctly, but it will only get you so far. Query optimization can also take nice clean code and turn it into a mangled nightmare. Caching is also good in moderation, but too much of it is also a code mangler and can make your application’s usability suffer (waiting 10 minutes for a user’s change to appear is often not usable). Database slaves are fine as failover/backup options but reading from one db and writing to the other also produces mangled code.

The solution? Search servers. Search servers can grow infinitely. You can run one on each web server or you can also have a centralized search server if you prefer. Whenever you pull results from the search server your database has avoided the blow. You’ll still probably need to hit the database occasionally, but your load will be dramatically lower. The load is on the web server end and can be shared across the web cluster without any fancy configuration. Once your search server is in place, use it for as much querying as you possibly can. The growth of your database load will be a nice slowly climbing linear line that will probably keep pace with processor speeds. If done right, you can avoid the extreme overhead involved with a database cluster.

The TIOBE index tracks the popularity of programming languages. It’s been around for quite some time now and seems to be a fairly accurate representation of the relative popularity of programming languages.

Very rarely you will see a language from the lower 50 suddenly leap into the top 20. I only saw this happen once in the past actually, with Ruby. Around the time Rails reached 1.0 status Ruby suddenly leaped from near the bottom into the top 20. Since then it has continued a strong upward trend. It’s likely to pass C# very soon, which is a major accomplishment (take that Mono!).

I had been doing Ruby for about a year when that happened and since then there’s been far from any shortage in work. I’ve always tried to push the edge in programming languages and usually achieved quite a bit of criticism in the process. I’ve heard the “well it’s way too unstable and dangerous to use in production” argument time after time. Stability is all in your testing friends… but that’s another topic.

The language of the month for November is unquestionably Lua. It has moved up 32 spots over the past year and still continues a strong upward trend. It has gone from 48th position to 16. Surprisingly this was done without all of the incessant hype that was associated with Ruby on Rails.

I noticed Lua a few years ago, when I started hearing about it being used in game development by Blizzard and others. The use of a scripting language in gaming piqued my curiosity. I never had the chance to learn much of it though (Ruby/Rails has dominated my development career over the past few years). Maybe now is the time.

So what makes Lua so great? It’s small (tiny really), extremely flexible, and exceptionally fast. It has an elegant implementation that is paradigm agnostic. Lua was designed to be used as an extension to C (Lua is attached to C code) but can run through an independent interpreter.

In contrast, Ruby is a very large, extremely flexible, and exceptionally slow programming language. It’s implementation is purely object oriented (although functional programming is possible) . Ruby was designed to allow C extensions (C is attached to Ruby code).

Speed isn’t everything. Choosing sides for a paradigm isn’t necessarily bad either. What’s interesting to me is how well the two complement each other. All languages have strengths and weaknesses. Lua may be a good answer for those times where Ruby is the sledgehammer when all you need is a scalpel. Especially for those rare performance critical applications.

One thing is certain, both are here to stay.

You may have noticed a lot of errors popping up recently around here. The problem has been how triggers work with Typo. It seems that publishing articles in the future will make Typo go kaput if you’re on Dreamhost. It doesn’t seem to happen anywhere else - there must be something unique to the Dreamhost environment.

I was able to fix this problem by deleting the content items in question, along with the related articles_tags and triggers. This required a combination of work in the typo console and mysql cli client to do. You have to find the id of the items in question. Delete the rows matching that id in the contents table, delete the rows that have that id for articles_id in the articles_tags table and delete the rows that have that id for pending_item_id in the triggers table.

What a great way to start the weekend! :)

Obviously this means that I had some articles coming that will be delayed now. I’ve read elsewhere that you can get around this by marking the publish date in the future but leaving the “publish” item checked.