When I highlight a script and click the Edit button, nothing seems to happen. I'm using Firefox 3.0b4 and Greasemonkey 0.7.20080121.0.
I've tried different values for greasemonkey.editor in about:config and, while troubleshooting, I ended up with a shell script that records it's command line and output to a text file for me to look at. The script is never executed.
I tried a few variations of using strace -p <firefox-bin pid> 2>&1 | grep <my editor> while clicking on the Edit button and I learned a couple of things:
Even if the editor isn't set to gvim, it always looks for and opens gvim.desktop, in ~/.local/share/applications, /usr/local/share/applications, and finally in /usr/share/applications. I suppose it would stop at my home directory if it found it there.
It does try to access (ref: http://linux.die.net/man/2/access) the program, checking modes of F_OK and X_OK. A guess would be that it's looking for the file and making sure it can run it, but someone forgot to put in the line of code that actually calls the editor.