Speeeeeeed!

Today's release of Gigantt is all about performance. Navigating inside plans, zooming in and out - it's all real snappy now. Try it out: http://www.gigantt.com

Flash Optimization

So what did we actually do? Programmers, read on. Humans, feel free to skip the rest of this post and move straight on to trying it out.

Our recent releases have been focused on adding functionality and the whole thing just got very heavy and bloated. Memory was leaking because apparently Adobe Flex isn't really meant for creating thousands of recursive controls over and over again. The worst part was the latency - if you've navigated into a task that contained many sub-tasks it would take them a while to load which was very frustrating and it slowed you down. That dog won't hunt. We learned that one of the slowest things you can do in Flex is to add and remove elements from the display list. So now we don't. We recycle and hide unused elements instead of adding/removing them. That also took care of the memory leaks, since apparently we cannot rely on Flex to properly dispose of unused objects (even [sigh] when the Flex Profiler itself shows no paths to those objects from the GC root). 

Rendering tiny zoomed-out sub-tasks was rewritten in lower-level Flash graphics (as a C++ developer it sounds odd to me - "low level Flash"... but I'm getting used to it). This approach worked great when we implemented Team View so we adopted in the other views as well.

So how noticeable is the difference? Quite noticeable. We measured how long it takes to drill down from the top of a large plan to the bottom-most "leaf" task and it took about one third of the time. So roughly 3x as fast. But that's mainly because the built-in animation simply takes its time (about 400ms) - without it navigation just seems jumpy and weird. When we profiled just rendering time we saw a 10x improvement. 

Hopefully you'll notice this too. 

What are we doing next month?

I'm happy to announce the latest version of Gigantt. The long anticipated and much requested "Team View" is now live. Finally you can see what every team member is doing at any given time.


You can see Team View in action in the following two-minute video: 

Learn more on our help wiki.


If you think Team View looks awesome you can sign-up for the private beta program. More invites will be going out in about two weeks.

Screenshots Straight into Imgur.com

If you tend to use lots of screen-shots (like me) you should probably know SnagIt. It's the number-one screen-shot application for Windows. It takes over of your PrintScreen keyboard button and when you click it you can then choose the screen area to capture.

If you create lots and lots of screen-shots you can configure SnagIt to automatically write the files to some directory. But what if you always need to upload these image files to some public location?


That's where imgur comes into play. They'll gladly host your image files and give you permanent links to them. The cool thing about this is that they have an API. So, yes, you've guessed it, you can connect SnagIt to imgur!

Here's how:

1. You need Python (if you're not a programmer, this isn't for you, sorry...)
2. Save this script somewhere in your machine:  https://gist.github.com/1828787
3. Then configure SnagIt to output into this script as a program. You might need to wrap it with a CMD file, depending on your local python setup.

That's it. Now when you snag a new screen-shot it will automatically be uploaded to imgur and the URL will be copied into your clipboard. PrintScreen -> Paste URL into your blog or Facebook status -> Bang, you're done.

A Programmer's Guide to Successful Outsourcing Online

Gigantt outsources much of its work online through sub-contractors. If it's not part of our core intellectual property then we get someone else to do it. It's much more cost-effective. Every full-time employee gets a discretionary outsourcing budget used to get work done, no questions asked as long as it's within the following guidelines.


1. It's amazing how much you can actually outsource effectively once you do the mental switch. It ain't about the jobs you can't or don't want to be doing, it's about the jobs you shouldn't be doing.
Let's say you need to learn about a new technology and whether or not integrating it into the product is a smart move. The auto-didactic hacker approach would automatically lead you through the path of reading manuals, searching forums, asking questions online and generally satisfying your intellectual curiosity through hours of reading and playing around with new toys. Now let's say you value your time as a developer at $100/hour (it can actually be much more). If you spend only three hours figuring out a new technology - that's $300. But why not pay someone who's already an expert at this technology to tutor you on Skype for one hour? That person probably values their time roughly the same as you do. You just got the best tutorial you could get and saved $100 in the process. 
Notice this particular example only really works if people don't need to get their boss's approval for every job they wish to outsource. Not many people would be comfortable approaching their boss and asking him to pay for an hour's worth of tutoring, but often that's the most cost-effective thing to do.
The point is to be mindful of the possibility of outsourcing and really think twice before deciding to do something on your own. 


2. Get to know oDesk. It's probably the smoothest running outsourcing engine out there, with more than a million sub-contractors. Posting a new contract takes us no more than 5 minutes now. And, if you know what you're doing, deciding on a candidate and hiring can take about 10 additional minutes. That's about 15 minutes we can put down as cost-of-transaction. At the above rate that's $25 overhead. Naturally this means if you can do the job yourself faster than that then you should. But most jobs take more than 15 minutes.


3. Insist on superb English communications skills. That is, of course, assuming English is your work language. This cannot be stressed enough. 90% of outsourcing success and failure is attributable to communication quality. If you aren't speaking exactly the same language, literally, it's not going to work. Be very selective when evaluating candidate's CVs and cover letters. We automatically disqualify based on the tiniest spelling, punctuation, grammatical and style errors. If a candidates can't be bothered to proofread his own profile then you're probably not going to get a 100% perfectionist performance our of them.


4. Plant some non-obvious requirement into your job post. You need to be able to identify (and disqualify) those that send out applications en-masse without really investing time into their cover letter. We sometimes ask for a sample of their English writing work in the job description, even if the job itself is completely unrelated to writing. If candidates don't attach such a link they obviously either didn't read the job description (don't care enough) or can't follow simple instructions.


5. If it's a long-term job or requires any amount of talking over phone or Skype then be sure you can understand the person's accent. Many sub-contractors, especially in Asia, are just not that easy for westerners to understand. It's a matter of accent and also a matter of connection quality - Skype can really a problem in this respect. So do a Skype interview. It can be as short as five minutes. Again, this often does not apply because some jobs require no Skyping to get done at all.


6. Use an hourly rate instead of a fixed-price for the job. Don't worry so much about people billing too many hours. That never happened to us and we've employed dozens of people. Using an hourly rate even for stuff that initially seems like a very clear cut and properly estimated job has a few advantages. Unless your job description is superbly clear and comprehensive, realize that you might not get exactly what you asked for. People will misunderstand and then have to repeat work for you, and if they start losing money while doing so you'll get hurried, resentful work. Unless you've worked with the particular sub-contractor before and he's doing the same exact job again you can't really trust their fixed-price bid to be spot on. The point of outsourcing is to not worry about the job, and this means you have to employ people who are willing to redo and improve their work up to your standards, which may be higher than theirs.


7. Be very clear about I/O. It's just like design by contract. Specify exactly what your input is going to be and how you expect to get their output. It can be trivial stuff like which file formats or applications to use. But it can also be about progress updates - how often do you expect them to occur, etc. Our job descriptions often ask sub-contractors to spend not more than X hours initially on the job, then stop working and report to us with their progress so we can evaluate it and adjust expectations.* The clearer you are about breaking down the work into milestones and receivables the less you'll be disappointed.


8. Coordinate expectations about the amount of work that needs to be done and realize that sub-contractors usually work on more than one job at a time. Specify exactly how many hours per day or week you expect them to work. If the work is occasional in nature then coordinate expectations and realize they might not always be available on a day's notice.


9. Consider over-hiring. Need some data-entry work to be done? Hire two or three people for the same job (usually $2-$3 an hour - very cheap in western standards) and see which one performs best on a smaller initial milestone - that's the one you keep for the long term. Be honest and upfront about the fact that work will start on a smaller evaluation basis. Let people prove themselves to you by working (and getting paid). The cost of hiring and firing is very, very low and we developers are accustomed to the very opposite (recruiting takes forever and mistakes are very costly).
If the work is repetitive and error prone, people will become fatigued and make mistakes, especially the 17th time they basically do the same thing. In such cases it's perfectly reasonable to hire more people for the job than strictly necessary. Redundancy is your friend because it's unlikely two people will do make the exact same mistake. It also helps with the issue of availability mentioned earlier.


10. Finally, be respectful and courteous. Yes, hiring and firing on oDesk can take five minutes and have little implications for you, but the people working for you deserve your respect, even when they occasionally don't live up to your high expectations. It sounds silly to have to even mention this as a tip... Outsourcing online usually happens across cultural boundaries and not everyone has the same work attitude, conversational style and politeness (Israelis, I'm looking at you!). So be extra nice.




---------------------------------------------------------
* This is actually a tip by Tim Ferris. Read his book, or at least the chapter on outsourcing. Another great chapter on outsourcing can be found on http://www.startupbook.net/.

Clear Browser Cache - a Saturday Project

UPDATE: Because of embedded iframes in this post I highly recommend you read it out of your RSS reader.


Woke up today with an idea that must have its day. So, you know how web applications sometimes ask you to clear your browser's cache. It's the equivalent of "turning it off an on again". A general cure for everything.

But how do you explain to very simple users the steps to clearing their browser cache? Most users probably don't even know what sort of browser they're using. You need to detect it for them and present them with step-by-step instructions (with screen-shots), and practically nobody does this. It's just too much work.


We need a website that does just this - it detects your browser and shows nothing but the steps to clear your cache. Web developers could then just refer to it instead of recreating it on their own over and over again. They could even embed it as an iframe...


So this is my Saturday project. Let's see how far I can get today.


09:30 - Wrote the into above and checking online that nobody has already done this.


09:31 - This can be a totally static website with just Javascript to detect the browser. So it can be hosted in its entirety from S3. I always wanted to do that. Let's read up on how.


09:38 - Seems simple enough. Just create a HTML page and upload it to a new bucket on S3. Then turn the "website" switch in the bucket properties. 
Created a bucket called "clearcache" (I already have a Amazon account).
Created a new local directory for this project on the best source control system - DropBox :)
Created a hello world html file called index.html
Now just s3cmd sync . s3://clearcache


Oops! Amazon says access denied when I tried to view the website at http://clearcache.s3-website-us-east-1.amazonaws.com

Giving "View Permissions" to the bucket doesn't seem to solve the problem.. but making the index.html file public does. But that's no good - would I need to do this for every file I upload?


09:56 - Time to RTFM...
09:57 - Argh.. they ask you to create a "bucket policy" and upload it. Common, Amazon, couldn't this have been a checkbox?
Ok, so the site is up.


10:10 - Let's give it a nice name like clearcache.gigantt.com. Amazon suggest adding a cname in the domain DNS records.
That doesn't seem to work. Gives a permission error..
The manual says the bucket name has to be the same as the domain name.. rats! 
Ok, let's rename the bucket to clearcache.gigantt.com
Great, now I get to wait 30 minutes for the dns cache to expire. I guess that's a good a time as any to eat breakfast.


10:42 - And we're back. And the new cname works: http://clearcache.gigantt.com/
Time for a little design. 
This website should be dead simple. You visit it and all you see is instruction on how to clear your browser. So we need some javascript that can detect browser, OS (because screen-shots will look different on each OS) and browser version. Probably also language, but let's start with just English for now.
I'm thinking the index page should just contain enough js to determine the right version and then fetch the appropriate html. So it's basically a dictionary of browsers identifiers to HTML manuals. Which also means we need to show a default in case we don't have a manual for the specific browser at hand. That can probably be some 3rd party web site that explains the same steps but doesn't detect the version.


10:53 - Cool. There's a JS library that does browser detection pretty well. We'll make our index.html file link to it and also add JQuery just for good measure.


10:58 - Here's our skeleton HTML. We'll worry about validation later:




It prints:


Your Browser is Chrome 16

11:06 - Let's figure out the dictionary we're going to use. I want it to degrade gracefully, so if we have instructions for FireFox but maybe not the specific version in question then it's probably better to show instructions for some other version than to fallback to the options of last resort (linking to an external page). So the keys need to be Language / Browser / Version / OS - in that order. This order makes sense because the same instructions on different OSs are much more similar to each other than instructions for different browsers on the same OS....


First let's shove all the JS into one main file:






12:10 - Here's main.js. It looks for the best match and tries to load the relevant instruction manual:




12:23 - Now I just need to populate the directory structure lang/browser/version/os.html with a few initial manuals. Let's create the first one for Chrome, then style it a bit.


12:48 - First manual for Chrome is ready. The html page itself has no styling at all:




And it looks like this:

So obviously it needs some nicer styling.


12:53 - Let's add CSS to this thing. First, we want it to be mobile friendly from the get-go. So we'll keep the snapshots as narrow as possible and the design itself should be dead simple. Just a centered narrow page with ordered instructions.

Nothing too fancy. 


13:55 - Let's DRY things up. I really don't feel like writing the same html template for each manual. Only three things change, basically: the logo, the instruction text and the image that goes with each instruction. Everything else can and should be generated automatically.
We want the entire manual HTML file above to be reduced to the following JS snippet:





14:34 - Here's the code that generates our HTML from the above JS dictionary:




Even the image names are auto-generated (image1.png, image2.png, ...) and each browser has its logo specified just once. Here is the resulting directory structure:

14:39 - Lunch time. Afterwards I'll add Firefox and IE.


15:48 - Firefox 10 and IE 9 are done. Now it's just a matter of adding more browsers and OSs. But I think I'll offload this to an outsourcing contract on oDesk.


That was fun. Visit http://clearcache.gigantt.com to see the finished product.


Sources: main.js, index.html, manual.css

How to Shampoo Your Employees

I joined the Israeli Defense Force in 1997 and served the mandatory three year period as a software developer. Army service usually consists of three* phases: first year - you learn the ropes, second year - you get things done, third year - you teach the new guys.

The Hebrew term for this period where the veterans teach the newcomers is called "the overlap", which, incidentally, is exactly the same Hebrew word for "shampooing". Don't ask me why. It's weird. So the veterans shampoo the newcomers and thus the wheels keep rolling from year to year.

Something similar happens in software companies. The term "training period" doesn't really capture the essence of what really goes on. Newcomers are often extremely well trained at their job already and all they really have to go though is a getting-to-know phase. Getting to know the product, the team, how things usually work. You know, shampooing... English speakers, I suggest you adopt this term.

One problem that plagues many software teams is knowledge hoarding. You all know someone like that. Someone who really loves being the most informed. Some veteran to whom everyone comes with questions. I guess some people think that makes them more valuable. I think they should be fired as quickly as possible because I've yet to see someone like that actually mend his ways. 

The perfect team player, on the other hand, is someone who's powerful, effective, but also completely replaceable. Like a CPU core. Your machine has four of them. They're expensive and they get things done. But lose one of them and your computer will keep on going, just a little slower. There's nothing it won't be able to do with just three cores, and when you plug in a new one things will go back to normal. 

Now here's the secret for ultimate shampooing. 
Ready?
Take the third year and the first year and squish them together. In other words, when a newcomer arrives on the scene his first order of business is to write the shampooing plan for his replacement. If one was already written, he can improve and augment it as he picks things up. 

That's a great general principle, and it applies not only to newcomers, because in our line of work we constantly learn new things. Keep nothing solely in your head. Be mindful of what you learn, and document it in a discover-able, public way. Not only the idiosyncratic quirks of your project, but also tricks of the trade, theoretical knowledge of the problem domain, mapping out who knows what and where to find stuff. Everything. 

Gigantt has recently recruited a QA manager. Until now the QA team had to deal without one. His first job is to write our QA Manager Handbook, which ought to include 100% of what his successor would have to read in order to get going. Being completely replaceable as a team member almost makes you irreplaceable, because so few of us are actually that minded towards knowledge sharing.

_________________________________________________________

* Actually everything in the army consists of three parts. It's... exhausting.

Handwritten Signatures in MS Word

How many times have you had to print a Word document just so you can sign it and then fax/scan it back to someone?

Ideally you should be able to add your signature directly to the Word document, and then print it to PDF or just send it. This turns out to be really tricky to do, but today I've cracked the problem.

Typically, you get a document looking like this:

How do we add a signature? Well, we could sign our name on a white piece of paper, scan it  and just plant that image in Word, right? Not quite. When you scan an image (or take a photo with your iPhone for that matter) you get a JPEG image, and JPEG images do not support transparency.

Here's what a scanned signature looks like right off the scanner:

First thing you'll notice is it's big. If you want your signature to look legit when the document is printed you have to scan it at at least 300 DPI. Your computer screen, however, has a resolution of about 75 DPI normally. So if you add this photo to Word you'll have to re-size it down. The end result is something that looks like this when printed:

Terrible.

First of all, the white background of the signature photo is opaque. As a result the image hides the "Signature" text and the line we're supposed to sign on. We want to somehow make the white areas transparent, so that when we plant the signature onto the page it looks as if it was drawn on by hand, not plastered on with a computer. This can be done with image editing software, such as Paint.NET (free), if you know what you're doing. The resulting image must be saved in a more appropriate image format - one that supports transparency. Best is PNG. 

But even that's not enough. The end result, when printed, still looks terribly fake. Here's a close-up version to illustrate:

See how pixelated it looks? When you print this on a piece of paper you definitely see the difference. Why is this happening? Didn't we take a hi-res scan of the signature?

Why? Because Word is dumb. When you scan an image in high resolution (e.g. 300 or DPI, same as a good printer) the image you get on your computer is big. When you then resize the image in Word to fit the signature space, Word just prints the picture as it appears on screen (i.e. in 75 DPI). It's not smart enough to say "this picture is scaled down, but I do have the original high-resolution at hand, so I can send it to the printer at 300 DPI".

Also, what if we want to print at 600 DPI, but our scanned image is just in 300 DPI? Ideally we'd want a vector image of our signature, not a raster image. Vector graphics nowadays is usually saved in the popular (and open) SVG format. And you can find services online that will try to convert a raster image (e.g. jpg) to SVG. Only one problem with this: Word can't handle SVG. So there's one more hoop to jump through.

Here's what you need to do to take a scanned image of a signature and convert it to something you can actually add to Word. You need just one application - it's called Inkscape and it's a totally free vector graphics application. 

1. Take a white A4 paper and put your signature on it using a nice, thick pen.
2. Scan it at 300 DPI (grey-scale is fine, too).
3. Crop out everything but the signature. This can usually be done in the scanner program itself if you do a pre-scan. But you can also do it with MS Paint.
4. Save it as JPG.
5. Open Inkscape.
6. Drag and drop your scanned JPG into Inkscape.
7. Select the image in Inkscape and choose Path -> Trace Bitmap from the menu.
8. Use "Brightness Cutoff" with a high threshold (over 0.9) and press Ok. 
9. It will create a vector version of the signature and place it directly above the image. Drag it away to see the difference between the two.
10. Feel free to play with the threshold until you get a good reproduction of the original image.
11. Now select the original image and delete it (DEL).
12. Select the good vector version and do File -> Document Properties -> Fit page to selection.
13. Now save as EMF. 
14. You can now drag and drop this EMF into Word. Make sure the Text Wrap property of the image in Word is set to "In Front of Text" and then just place it over the signature space.

Here's a close-up of the raster vs. vector images side by side in Inkscape:

Here's just one letter scaled way up, just so you see how good the vector version is:

And here's what it looks like in Word:

Keep that EMF file at hand. No more printing, faxing and scanning.