clog

I posted one way to get rid of the TextMate beachball earlier. It just keeps coming back though. My current project is heavy on themes, meaning I have a lot of cache and images in a theme directory that also contains views and stylesheets that I need to edit. This really began to slow things down for me, and there’s no way to remove subfolders from the project. I finally figured out a way to put the beachball to rest for good though (hopefully).

First follow my previous advice and create a brand new TextMate project and add all folders that contain files that you will edit (and no others). Don’t add the log or tmp directories. Be thorough though - don’t leave out folders that MIGHT contain editable files in the future either.

TextMate will automatically scan all of these folders for filesystem changes, but there is a way to tell it NOT to scan certain subfolders. To do this you select the top-level folder that contains subfolders that don’t need editing (usually image and large static content areas). Right click and select the “Show information” option.

The “Folder pattern” textbox contains a regular expression that filters out unwanted folders. All you need to do is add the name of the folder(s) you want left out. Just add it to the existing list of names that are | separated. Make sure you have a single | on both ends of the folder’s name. You don’t have to put the entire path, just the name.

For example, I don’t want to check image subfolders, so I change this: !./(.[^/]|CVS|_dar to this: !./(.[^/]|images|CVS|_dar

Now my massive images folder is hidden. After doing this for all applicable folders, TextMate is faster than ever! Plus I still get the benefits of the live filesystem check that other methods don’t have.

There’s always a need for importing, exporting, and synchronizing data between various applications, especially in corporate environments.

As data has grown by orders of magnitude in my databases, the amount of resources required to run these processes has shot up dramatically. Many databases are now 10 times larger and exports that once took 2 or 3 minutes now take a good hour or more.

Think about that for a second and you’ll notice something strange. Shouldn’t 10X more data only take 20-30 minutes to export rather than an hour.

I’ve recently had to go back and make changes to some (of my own) legacy code doing such an export and was so aggravated by the excessive run times, I decided to do something about it.

After looking into it, I figured out that the problem I was running into is the excessive use of garbage collection. By pushing gigabytes of data into memory, not only was the OS working overtime to reallocate all of that, the garbage collector was having to constantly run in an effort to keep memory use down. When I tried running the process on my macbook that “only” has 2 gig of ram, the process ate up all 2 gig and virtually starved itself to death. I finally gave up and killed it.

The solution may seem obvious, but I really haven’t come across it before:

Page all data that gets pulled.

I always thought of data paging as something done to improve the user experience and not a method of improving performance when you need to work on all of the data (quite the opposite really). While it does slow things down for a small number of results, it really makes a HUGE difference as your application grows.

As a Rubyist, I now rely on my relatively new friend paginating_find to meet my paging needs. I set up an enumerated version of my export process to pull 1000 record pages at a time. Experiment with results-per-page until you get your desired performance. A lower number of results_per_page will tax your database, while a higher number will tax your web server and increase your memory requirements. I found that at 1000 records the database would carry about 15% of the load, and about 100MB of memory would be used.

What was the performance difference? The process has gone from 60+ minutes per run to just over 7 minutes. Wow!

Vendor lock-in is an ugly thing. I try to avoid any platform that uses a lock-in style of marketing, which pretty much means I avoid Microsoft products. I’m not religiously against them though. I use them as needed and have some installed. If there’s a reasonable alternative though, I’ll use it - even if it’s not quite as good.

Now we have a new form of lock-in. It’s the I-can’t-live-without-the-interface lock-in, and it’s far more dangerous. I’ve been using Mac OS X for two years now and although initially learning it was very frustrating, I’ve become so addicted to the UI niceties that I almost can’t function without it. This is particularly surprising considering the amount of time I spend using a terminal.

So why then am I addicted to OS X? Reason #1 - the shortcuts. OS X shortcuts are well thought out and are based on a combination of classical UNIX/Emacs shortcuts and PC based shortcuts. Now I have no idea whether Command+s came before Ctrl+s and I don’t care - I’m just calling them PC based shortcuts. PC shortcuts ARE sensible shortcuts. In Emacs the Ctrl+s equivalent would be Ctrl+x Ctrl+s - Not quite so friendly (and that’s an easy one for Emacs). Movement shortcuts resemble Emacs shortcuts though. These ARE handy. Ctrl+n, Ctrl+f, Ctrl+p, and Ctrl+b will move you around almost everything. Whether I’m checking email, browsing the web, or writing code - I can ALWAYS count on these shortcuts working as expected. Another less known shortcut is the Escape key. Type a word and hit escape and OS X will auto-complete it - ANYWHERE. It’s universal and only happens on command - unlike the annoying pop-up style auto-complete Windows and Linux apps use.

Let’s not forget the other reasons…

Reason #2 - It’s beautiful. Call me a romantic, but I much rather look at a visually appealing OS all day rather than something that looks like Playskool’s My First Small Business Kit.

Reason #3 - TextMate. I write code. This is the best editor in the Universe. Period.

Reason #4 - Expose. Ultimate window navigation.

Reason #5 - Spotlight. Dump your crap in just the one relevant folder and forget about it. Spotlight will always find it for you down the line.

Reason #6 - Quicksilver. The awesomely effective application launcher. Who needs a stupid Program Files menu when you have magic. I wish it was more stable on Intel though..

Reason #7 - Growl. I know what your email is about because I saw the first line in the growl cloud. And I already know I don’t want to read it, so I’m not opening my email. Also.. I know I’m getting a critical server error because I saw it pop up in the growl cloud. That looks important…

Reason #8 - It all “just works.” This is the only OS where downloading an application is almost always synonymous with installing it. No stupid dialogs with useless questions and no need to make sure your gcc flags are correct.

Reason #9 - Dashboard. Weather/Documentation/well anything you want at the touch of a button. Highly useful once you get used to it.

Reason #10 - Finder. Seriously, I’m not joking. Finder is great. It does NOTHING but the bare minimum and that’s all it should do. Why? Reasons 4, 5, 6, and 9. I used to hate finder, but now I love it’s simplicity. The real productivity gains come from not having to use finder in the first place. When you do need it, it’s there and it’s very easy to get from point a to point b. If you find yourself hating finder, then you’re using it for the wrong things.

Take those things away and my productivity sucks. Working without them these days is like going back to the stone age.

The thing that surprises me the most is the huge number of Mac users that don’t learn ANY of the above. LEARN THEM!! You’re brain (and fingers) will thank you.

I installed the Ruby GD2 gem on my Mac last night and found myself getting the following error:

ruby/gems/1.8/gems/gd2-1.1.1/lib/gd2.rb:59:in `initialize’:RuntimeError: dlopen(libgd.2.dylib, 9): image not found

It didn’t take very long to figure out that OSX doesn’t seem to come with any version of libgd. After bouncing around with trying to get RMagick installed instead, and finding very little documentation on setting up the Ruby GD2 library, I decided to install the gd2 package via ports and reinstalled the gem with the same results. It still wasn’t finding libgd because the ports library path (/opt/local/lib) is not set in my path. I don’t set this because I want anything self-compiled to only use the Mac libraries. After some digging, I found that I could specify the path in the Ruby GD2 source.

To fix it I changed line 33 in “ruby/gems/1.8/gems/gd2-1.1.1/lib/gd2.rb” from:

case Config::CONFIG[‘arch’]
when /darwin/
‘libgd.2.dylib’

to:

case Config::CONFIG[‘arch’]
when /darwin/
‘/opt/local/lib/libgd.2.dylib’

It’s not the cleanest fix, but it’s good enough for development purposes. Hopefully this saves a few people from some of the circles I went through figuring this out. It does seem that this is a least a more simple fix than what’s required to get the RMagick library running on OSX.

Getting delays while working in TextMate from the beach ball of death?

If you only add files and directories that you work on in a project, this is greatly minimized. Also make sure that you don’t have the log directory in your textmate project. TextMate scans the filesystem for changes, and the log directory is always doing that (slowing things down).

If you are using an existing project (or not), create a new one then:

Go to your rails path in finder and open TextMate and choose File -> New Project

Drag over only the directories that you will need to edit. Avoid log and tmp, since there is nothing to edit there. You may want to avoid public if there are no html files in it that need editing (sometimes it contains a lot of images).

Save the project using “Save As” Overwrite the original if necessary.. Save doesn’t pick these changes up for some reason.

This should reduce the size of the project and make that alt+tab magic nice and speedy.