NOTE: There is now a relevant answer on StackOverflow. Unfortunately, this answer wasn’t there when I needed it.

The problem

Recently, while adding iCloud support to the Day One iOS app, I ran into an issue copying files to and from the application’s Documents directory and its corresponding iCloud directory. Day One can run in one of three modes:

1. Sync entries via iCloud
2. Sync entries via Dropbox
3. Don’t sync entries

If you’ve developed for iCloud at all, you know that iCloud documents live within specific, “ubiquitous” directories on the user’s device (discovered via NSFileManager’s URLForUbiquityContainerIdentifier: method). So in order to enable iCloud sync, we needed to copy all files into the appropriate ubiquitous container. Conversely, if the user elects to disable iCloud then we need to copy all files from the ubiquitous container into the application’s Documents directory.

The assumption

We wrote a simple method to copy files from one directory to another, either going into or coming out of iCloud. It uses a basic NSDirectoryEnumerator to find all the files in the current source directory, check if they are present in the destination directory, and copy them there if they aren’t. Here’s a simplified example:

NSURL *sourceDir = [self currentWorkingDir]; // current directory, ie ubiquitous iCloud directory
NSURL *destDir = [self destDirForMovingFromDir:sourceDir]; // where should we copy to?
NSFileManager *fm = [NSFileManager defaultManager];
NSDirectoryEnumerator *dirEnum = [fm enumeratorAtPath:[sourceDir path]];
 
NSError *mergeError;
while ((sourceFile = [dirEnum nextObject])) {
    NSURL *sourceFileURL = [sourceDir URLByAppendingPathComponent:sourceFile];
    NSURL *destFileURL = [destDir URLByAppendingPathComponent:[sourceFile lastPathComponent]];
    if (![fm fileExistsAtPath:[destFileURL path]]) {
        if (![fm copyItemAtURL:sourceFileURL toURL:destFileURL error:&mergeError]) {
            NSLog(@"ERROR (copy error): %@ -> %@ - %@", sourceFileURL, destFileURL, mergeError);
        }
    }
}

I frequently found “bad file descriptor” errors being logged out. We were assuming that the NSDirectoryEnumerator would only list files that were actually present on the device. As it turns out, that is not the case when enumerating a ubiquitous directory.

The solution

Once you realize that the NSDirectoryEnumerator will list all ubiquitous files that exist on any device, regardless of whether or not they have been downloaded to the current device, the solution becomes pretty obvious:

You can’t copy a file that hasn’t been downloaded.

Adding that check is fairly straight forward. Since our method doesn’t know if we are going into or out of iCloud, we implemented a pretty general solution. Modifying the above example, it looks something like this:

...
while ((sourceFile = [dirEnum nextObject])) {
    ...
    // By default, assume all files are able to be copied.
    NSNumber *ableToBeCopied = [NSNumber numberWithBool:YES];
 
    // However, ubiquitous items that are not downloaded CANNOT be copied
    if ([fm isUbiquitousItemAtURL:sourceFileURL]) {
        [sourceFileURL getResourceValue:&ableToBeCopied forKey:NSURLUbiquitousItemIsDownloadedKey error:nil];
    }
 
    // Now copy the file if it doesn't exist and is able to be copied
    if (![fm fileExistsAtPath:[destFileURL path]] && [ableToBeCopied boolValue]) {
        if (![fm copyItemAtURL:sourceFileURL toURL:destFileURL error:&mergeError]) {
            NSLog(@"ERROR (copy error): %@ -> %@ - %@", sourceFileURL, destFileURL, mergeError);
        }
    }
}

Conclusion

See what I mean? Pretty obvious, and I can’t believe I lost so much time tracking this down. If the source file is ubiquitous, it might not be downloaded, and thus you might not be able to copy it. So check it first.

• TotalTerminal has replaced Visor
• Mac OS X Lion has been released

TotalTerminal changes up the way it launches Terminal — instead of being automatically injected via SIMBL, it manually injects the plugin into Terminal.app. Well, I wasn’t going to let this deter me.

Getting back to Nirvana after installing TotalTerminal is not all that bad. Most of the steps haven’t changed from my original post so I will only go into detail about the new steps here. Briefly, here are the steps I followed.

1. Remove TerminalColours.bundle SIMBL plugin — per the TotalTerminal FAQ, it injects its own version and having TerminalColours.bundle installed creates a conflict. (~/Library/Application\ Support/SIMBL/Plugins)
2. Fix the Home / End keys (see original post)
3. Copy Terminal.app to VisorTerminal.app (see original post)
4. Install TotalTerminal
5. Modify TotalTerminal to use VisorTerminal.app instead of Terminal.app

That’s right, there’s really only one thing to change, and it’s very simple

TotalTerminal.app has a simple, compiled AppleScript which launches Terminal and injects itself. The compiled file is located at /Applications/TotalTerminal.app/Contents/Resources/Scripts/main.scpt. But, we don’t want the compiled binary version, we want the original so we can make it use VisorTerminal.app.

Fortunately the uncompiled source file is available in the TotalTerminal repo on Github.

Only one change is necessary.

--- TotalTerminal.app/Contents/Resources/Scripts/main.applescript
+++ TotalTerminal.app/Contents/Resources/Scripts/main.applescript
@@ -1,4 +1,4 @@
-tell application "Terminal"
+tell application "VisorTerminal"
        delay 1 -- this delay is important to prevent random "Connection is Invalid -609" AppleScript errors
        try
                event BATTinit

Once that is done, you just need to compile the script and replace the one that shipped with TotalTerminal.

$ osacompile -o main.scpt main.applescript
$ mv main.scpt /Applications/TotalTerminal.app/Contents/Resources/Scripts

That’s it.

You should be good to go. Put TotalTerminal.app into your user startup applications if you wish (I do), or launch it manually by launching TotalTerminal.app.

Enjoy the restoration of terminal nirvana.

For the lazy…

My modified and compiled main.applescript is available for download.

We needed something easier. We did not want to go to a workflow where there is one git user on the system, every employee authenticates with their own public key to that user, and everyone with a key has access to everything. We wanted more granular control, but also an easier management workflow. While gitolite does use a single system user under the covers, it provides per-user access control (governed by public keys), easy repo management, and a couple other nice features like wildcard repositories and branch-level access restrictions.

However, one of our developers is in the habit of accessing a random server, cloning a repo (using his password), and pushing from environments other than his development environment. Obviously, his private key is not going to be on each of the machines he does this from. Gitolite does not allow password-based access for obvious reasons — it needs to know who you are to determine permissions — so gitolite was cramping his style.

There are a couple ways around this:

1. Add a public key for each system he pushes changes from. Since he often logs in as “administrator” or “root” on individual systems, this obviously is not a good solution.
2. Have the developer enable SSH Agent Forwarding from his machine. This works well, as long as the user does not chain sessions (connect to server A, from there to server B, and clone on B) or is willing / able / allowed to enable Agent Forwarding on each system in the chain. (NOTE: on Mac OS X Snow Leopard, enabling SSH Agent Forwarding is as simple as running ssh-add and adding two lines to your ~/.ssh/config file:

   # add key to agent (I do this in ~/.profile)
   ssh-add $HOME/.ssh/id_rsa
    
   # ~/.ssh/config
   Host *
       ForwardAgent yes

3. Come up with something else.

What we really wanted was a way to allow password-based access to the repositories. Since gitolite does not support this itself, we had to come up with another way. As it turns out, doing so is not too bad, though it does involve a little bash shell black magic.

What we came up with is creating a system user on the git server for each developer who wants password access to git repositories, using a custom script as their defined shell (instead of /bin/sh or /bin/bash). Don’t forget to make your new script executable.

This script needs to look at what the user is trying to do (git action, gitolite ADC, some random shell command, nothing…) and act accordingly. Here’s the script we came up with, which will be dissected below to explain what each part does in more detail than the inline doc (gist).

#!/bin/bash
shift # get rid of -c
 
# if no commands, just open a shell
if [[ $# -eq 0 ]]; then
        /bin/bash -l
 
# if the first arg is a git- command, that means it is something like git-push, etc... so forward it
elif [[ $1 == git-* ]]; then
        ssh -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no git@localhost $*
 
# if the first arg is SET_ENV_ONLY, we sourced this file in order to set up the gitolite function
elif [[ $1 == "SET_ENV_ONLY" ]]; then
        gitolite () {
                ssh -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no git@localhost $*
        }
 
# if there is at least one non-git command, source this file in a new shell and create the 'gitolite' function
else
        /bin/bash -c "source $0 shiftme SET_ENV_ONLY; $*"
fi

If the user is not trying to do anything (just wants to open a shell), then there are no arguments and we should just open a bash prompt:

if [[ $# -eq 0 ]]; then
        /bin/bash -l

Okay, the simple bit is out of the way. Now on to the black magic.

If the user is trying to perform a git command (via git push, etc.) then we create a new SSH session to localhost (remember, this is running on the same server as gitolite) and pass along the commands ($*). Since the user has already logged in, this ssh command will run from their account using their private / public key. The git commands are things like git-receive-pack, git-upload-pack, etc, so we match the first script argument against “git-”.

Finally, we don’t care about validating the host’s fingerprint (we are already on the host) and there can be no man in the middle (since we are going to localhost), so we can turn off StrictHostKeyChecking. We also don’t want to print out the various fingerprint messages that SSH will generate when you ignore host keys, so we make the new SSH connection quiet. That’s it for this block:

elif [[ $1 == git-* ]]; then
        ssh -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no git@localhost $*

I am going to go over the last two pieces in reverse order, because it is easier to follow.

If the user sent some SSH commands that were not native git commands, we need to process those commands. So, we will execute the remaining commands in a new bash process.

BUT, what if the user is trying to perform a gitolite command (such as expand or an ADC), what should we do? We can’t just trust that all commands the user sends will either be a git command or a gitolite command, so we need to set up some more black magic to handle that for us. So the first thing we will do when we open a new bash process to handle the remaining commands is source this custom script ($0) with a custom value we made up (SET_ENV_ONLY). That way, if this script executes again we will know it was done by ourselves and we can set up some more magic.

else
        /bin/bash -c "source $0 shiftme SET_ENV_ONLY; $*"

Finally for the last block.

If our new script is being executed, and the first argument is SET_ENV_ONLY (after shifting), then it is because we sourced it from our new bash process from the previous step. What we do now is set up a gitolite command which does something very similar to what happens if the user is performing a native git command: it opens an SSH connection to git@host and performs the gitolite commands.

After this has been done, “gitolite expand” is the same as “ssh git@git.company.com expand”. This makes it so the developer can still run gitolite commands and ADCs, he just has to prefix each command with gitolite. Something like this:

$ ssh [DEVELOPER]@git.company.com "gitolite expand; echo $PATH; gitolite info"

is equivalent to this:

$ ssh git@git.company.com expand
$ ssh [DEVELOPER]@git.company.com echo $PATH
$ ssh git@git.company.com info

Now you just need to create accounts that use this script as their shell. Each user can add whatever public keys they want to their authorized_keys file, or they can authenticate solely with their password. The only requirement is that their server account has a private / public key generated, and that the public key is added to gitolite.

$ ssh root@git.company.com
$ useradd -s /usr/local/bin/custom_gitbash.sh -m [DEVELOPER]
$ su [DEVELOPER]
$ ssh-keygen -t rsa
$ cp ~/.ssh/id_rsa.pub ~/[DEVELOPER]@git.company.com.pub
# add the new public key to the gitolite keydir

I also ran into a problem with words being tokenized with their surrounding punctuation, rather than simply the word — searching for “something” was definitely NOT the same thing as searching for “something.” — note the included punctuation in the second search.

The bottom line:

• make sure stopwords is configured the same way for the fields you are searching (or at least be sure you understand the ramifications)
• don’t be afraid to play with the “minimum-match” (mm) default configuration
• when it makes sense, strip surrounding punctuation using PatternReplaceFilterFactory using an index tokenizer

For the full, detailed write-up, head over to OpenSky’s engineering blog and read the original blog post.

The Goal

Create HTML elements dynamically which transition appropriately when the view controller appears.

The Problem

The shipping version of TuneKit.js doesn’t give you a chance to append elements to the DOM before viewDidAppear is invoked (which is after transitionToController).

Initial Attempts

I first tried appending the child elements (using TKUtils.buildElement in a loop over appData.songs) inside the viewDidAppear callback. Obviously, the problem here is that the elements don’t exist the first time the view controller appears, so can’t nicely fade in via your defined CSS animations.

Next, I tried moving the logic that create the HTML elements into viewWillAppear. The problem here was that this method was called before the desired view controller is added to the DOM — so the element I need to append the children to doesn’t exist.

The Solution

The solution was actually pretty simple, and works like a charm; you just need to be willing to edit TuneKit.js.

Before:

// pre-transition methods
controller._viewWillAppear();
controller.viewWillAppear();
 
// add it to the tree
this.view.appendChild(controller.view);
 
// transition
if (previous_controller !== null) {
  this.transitionToController(previous_controller, controller);
}

After:

// add it to the tree
this.view.appendChild(controller.view);
 
// pre-transition methods
controller._viewWillAppear();
controller.viewWillAppear();
 
// transition
if (previous_controller !== null) {
  this.transitionToController(previous_controller, controller);
}

Conclusion

Now that TuneKit appends the view controller to the DOM before calling the viewWillAppear callback, your dynamic elements can be created before the view appears and transitionToController is invoked.

Done.

Note 1: The method described here should also work in Mac OS X Leopard or Tiger, just get the appropriate versions of SIMBL / Visor.
Note 2: If you don’t want everything on my bullet list, just follow the steps appropriate for what you want to accomplish.
Note 3: I have provided .patch files, etc. at the bottom of the post for those who, like me, are a bit lazy.

What is Terminal Nirvana?

As a developer, Terminal is one of my best friends. Maybe you can relate. I have constantly been in search of the ultimate Terminal configuration, and it has ever eluded me… until now.

But what is Terminal Nirvana? Here are my criteria:

1. I want the Home / End keys to move the cursor to the front / end of the line. I grew up on Windows, and I like the keys to function this way. To me, it just makes sense. Unfortunately, KeyFixer doesn’t work in Terminal (but I highly recommend it for changing the behavior in most other apps!).
2. I don’t want Home / End to behave differently when I am in a screen session. I use screen all the time, and Home / End need to behave consistently.
3. I don’t want Home / End to do unexpected things in VIM – ever. Pressing Home / End in VIM should, at the very least, not do weird things like delete lines, whether I am in a screen session or not.
4. I want my terminal window available at all times. With as much as I use the terminal, opening Terminal.app all the time feels wasteful. There should be a better way. This is where SIMBL and Visor come in.
5. I don’t want it to show up in the Dock, or in my Command + Tab list. As useful as Terminal is, there are a lot of other useful applications around, and I use a lot of them. Since I always want Terminal open anyway, it is annoying to constantly avoid / ignore / tab over it when I am using Command + Tab to switch to another running application. Oh, and it should stay away from my Dock, too.
6. I want good, configurable colors in my shell. Fortunately, Visor takes care of this one by including TerminalColours as part of the package, so you get this one “for free.”

Sound like a tall order? It took me a few hours crawling around the web and experimenting to come up with a good solution, but I managed it in the end. Hopefully my experience can help save a few people some time in the future.

This is how I did it.

First things first

Before you go any further, make sure you have installed SIMBL and Visor. The latest versions work great in Snow Leopard with a 64-bit Terminal.app, so knock yourself out.

Home / End keys

To change the Home / End keys in Terminal.app, you are going to need to go to Terminal -> Preferences -> [Settings] -> [Keyboard]. Remember, settings are profile-specific, so you will need to make the changes to any profiles you care about. (Note: Visor uses the “Visor” profile, regardless of what you set as your default profile. What I do is rename the automatically-created “Visor” profile, configure my default profile, and then duplicate it with the name “Visor”.)

The first thing I tried was changing the keybindings to “\001″ and “\005″ per the instructions I found on Ricky Campbell’s blog. This seemed to make sense; CTRL+A and CTRL+E move the cursor to the front / end of the line in Terminal. And I thought it was working perfectly, for a few glorious moments…

Unfortunately, CTRL+A is also the key combination to start a command in screen. So when I launched screen, suddenly my home key didn’t work anymore, and I randomly started sending screen commands when I used it.

I also tried using a ~/.inputrc file and the keybindings “\033[1~” and “\033[4~” as discussed on two blogs I stumbled across while looking for an alternate solution. Now, I really thought I had the problem licked. Except, Home / End did weird things in VIM running within a screen session, and the necessity of using an additional .inputrc file was just a bit of a bummer.

Turns out Shift + Home and Shift + End  already did what I wanted, so the simplest solution of all is to just copy those keybindings already present in the Terminal settings. For good measure, I copied the “shift page up” and “shift page down” bindings to “page up” and “page down.” (Note: you can either copy-paste the existing values, or hit ESC to get the \033 value and then type the other characters manually.)

Terminal key bindings:

• home
  • Value: \033[H
• end
  • Value: \033[F
• page up
  • Value: \033[5~
• page down
  • Value: \033[6~

Fixing VIM in screen

Okay, almost there with the keybindings. If you run VIM inside a screen session, you will notice that Home deletes the current line, and using the arrow keys while in insert mode inserts A / B / C / D rather than moving the cursor. Not good.

A little tweak to your existing ~/.vimrc file will fix it right up (or, if you don’t have an existing ~/.vimrc, create it). (Note: the easiest way to get the proper escape characters is to enter insert mode, and hit CTRL+V followed by the Home or End key. This will add the ^[ escape character, as well as the [F or [H characters. Do the same thing for CTRL + O to get the ^O escape character.) By using CTRL+O on the imap lines, we make the Home / End keys work the same way in insert mode or command mode.

As an added bonus, this fixes the arrow keys while in insert mode. Though, to be honest, I have no idea why…

""" ~/.vimrc
 
""" Fix home and end keybindings for screen
map ^[[F $
imap ^[[F ^O$
map ^[[H g0
imap ^[[H ^Og0

Making Terminal always available

This one was actually pretty simple. Using SIMBL and Visor (both compatible with 64-bit Terminal at the time of this writing) you can get Terminal to dock to the top / side / bottom of your screen, and configure it to open with a keystroke you can define.

My keystroke is Command + Esc (turn off Front Row first, if you hate it as much as I do). Once Terminal has been launched for the first time (thus creating the Visor window), this keystroke gets you in from anywhere, or hides it when you are done.

Hiding the Dock icon, and removing Terminal from the Command + Tab list

This is the most involved step, but it also provides one of the biggest benefits in my mind. Most of the information here comes from a MetaSkills blog article.

In order to accomplish this, the LSUIElement key in Terminal’s Info.plist must be set to “1″. This disables the Dock icon and removes the application from the Command + Tab list. Don’t want to completely prevent Terminal from ever functioning like a normal application? The quick solution is to create new application “VisorTerminal.app” which is used for Visor, while leaving Terminal.app alone to function as normal. (This is actually a good thing, because another side-effect of setting LSUIElement : 1 is you no longer have anything change in the menubar when you are in your application… ie, you can’t click on VisorTerminal next to the Apple icon, and go into Preferences.)

Create a new VisorTerminal.app

Duplicate the existing Terminal.app.

cp -r /Applications/Utilities/Terminal.app /Applications/Utilities/VisorTerminal.app

Once you have duplicated Terminal.app, you need to change a few bundle identifiers in your new application so the two appear as distinct entities to Mac OS X. In other words, change “Terminal” to “VisorTerminal” in a few places. This is also when you add LSUIElements : 1.

<!-- /Applications/Utilities/VisorTerminal.app/Contents/Info.plist -->
...
<key>CFBundleDisplayName</key>
<string>VisorTerminal</string>
...
<key>CFBundleIdentifier</key>
<string>com.apple.VisorTerminal</string>
...
<key>CFBundleName</key>
<string>VisorTerminal</string>
...
<!-- Note: Put this right before the closing </dict></plist>
     at the end of the file -->
<key>LSUIElement</key>
<string>1</string>
...

At this point, rather than entering my preferences all over again, I elected to copy my Terminal.app preferences. Repeat this step any time you make preference changes that you want to copy over. (Note: using a symlink does not seem to work. If you make changes while in VisorTerminal, such as changing the Visor hotkey, the modified file is saved over your symlink.)

cp ~/Library/Preferences/com.apple.Terminal.plist ~/Library/Preferences/com.apple.VisorTerminal.plist

Modify Visor to use VisorTerminal.app

Once your new application is created and ready to go, you need to tell Visor to activate for that application rather than Terminal.app. Apply the following very familiar tweaks to Visor’s Info.plist:

<!-- ~/Library/Application\ Support/SIMBL/Plugins/Visor.bundle/Contents/Info.plist -->
...
<key>BundleIdentifier</key>
<string>com.apple.VisorTerminal</string>
...
<key>ExecPattern</key>
<string>*/VisorTerminal.app/Contents/MacOS/Terminal</string>
...
<key>BundleIdentifier</key>
<string>com.apple.VisorTerminal</string>
...

Once this last step is done, you are almost there. Make sure Terminal.app and VisorTerminal.app are both closed. Now, launch VisorTerminal.app and behold its wonder!

open /Applications/Utilities/VisorTerminal.app

What? It didn’t work?

Let me guess. A Terminal window opened, but Visor didn’t take effect. What gives? (You may have also noticed that you don’t have an entry in your menubar… see above.)

Turns out newer versions of SIMBL don’t activate in applications with LSUIElement : 1. Apparently this is expected behavior due to the Cocoa event mechanism. But all is not lost!

You can manually inject SIMBL into an application using Apple Script.

tell application "VisorTerminal"
	inject SIMBL into Snow Leopard
end tell

Great, but I don’t want to do something manually every time I want a Visor window. I just want it to always be there, remember?

Use the AppleScript Editor to save this snippet as an application, and then include that application in your user startup items. Voila! When you log in, the AppleScript helper executes, which launches VisorTerminal with SIMBL injected. Visor activates, your terminal automatically hides itself, yet is always there when you want it with the push of a button. It doesn’t clutter up your Dock, your application switching, and you can still use Terminal.app as usual whenever it makes sense to do so.

Final thoughts

First off, note that this tutorial was written using SIMBL 0.9.7a, Visor 2.2, and Terminal 2.1.1 (273) on Mac OS X Snow Leopard 10.6.4. YMMV.

Secondly, my thanks to all the people who are smarter than me and figured all this out. All I did was put things together.

Third, see below for patch files and my AppleScript application.

Finally, does anyone have the following issue, and know a way around it? After setting my key bindings, when I launch Terminal for the first time, they don’t take effect. Home does nothing special. If I open a new Terminal window, the home key starts working as expected, both in the new window as well as the original window. This happens to me whether I use \001 for the control character (and this without a ~/.inputrc file) or if I use the solution described above. Slightly bothersome, but since I almost always use my Visor terminal (which doesn’t exhibit the same problem… nor did it using Visor sans any modifications discussed here) I don’t notice it too often.

Patch files: http://gist.github.com/484082
AppleScript app: HiddenVisorTerminal.zip