Whilst there are some historical reasons for them both existing (KDE was originally built with a toolkit that wasn’t free, so GNOME was created), they’re both now based on OSS toolkits.
Considering they both serve the same purpose, they’re surprisingly different.
GNOME takes a very simple approach. One app for each job (no overlapping apps), a simple interface etc..
KDE on the other hand is aimed at people who like to tweak every little detail.
Usually I’d prefer the option to tweak, but, in my case, I’ve always found KDE to be hideous. I hated their ‘start menu’ and disliked that konqueror was the file manager (not any more though) and the web browser. I also don’t like that their releases are not scheduled or consistent.
Recently(ish) KDE 4 came out. A major redesign from the ground up that’s intended to fix many of the shortcomings. In theory, it’s a fantastic release, but in practise, it’s not ready for prime-time.
Since I have a laptop that I don’t really need, I occasionally install OS’s on it to test them out (without ruining my productivity).
I have Arch Linux on it right now, so I decided to install KDE 4.1 (the kdemod version, from the ‘kdemod.ath.cx’ repo). The same thing is true now. It’s theoretically fantastic, has some brilliant ideas, and is gorgeous. The problem is, the usability is worse, meaning that I’m less productive when using KDE then GNOME. A large part of this is due to my not being as experienced in KDE, and another part is the fact that every app is different.
The way desktop environments (DE) work in linux is a little more complicated than you’d think. Each DE is built with its own toolkit. Because of that, each DE comes with its own set of libraries. For an application to run well (and look good), it has to also use those same libraries (if each app used different libraries, you’d soon run out of memory!).
The upside of this is that a distribution of linux usually comes with either KDE or GNOME. And, with either the qt (for KDE) or GTK+ (for GNOME) libraries. The downside is that if you ever wanted to switch, you’d have to adjust to all different apps.
A different web browser (firefox GUI in linux is GTK+), different IM Client, Different file Manager. Everything! This can be very jarring because despite still using linux, a switch in DE will change almost every app you use. It’s also annoying because you cant really mix and match apps unless you’re willing to have an app run more slowly, and look ugly. In many cases one DE may have a better web browser, but the other might have a better IM client.
This is a little frustrating, but unavoidable.
Fortunately, there’s enough good quality open source software (OSS) that you can usually find an app (built with your prefered toolkit) to fit your needs.
So, now back to which is the right DE to use.
Right now, I believe that GNOME is the best choice. It’s more stable, more supported by the main distros, and receives regular bi-annual updates.
Unfortunately, I worry about its future. They seem to be doing good releases every 6 months, but the improvements are minor at best. Their recent announcement of releasing GNOME 3.0 seems lame too, as it’s just a rename of 2.30. I’m not sure they really have a vision of the sorts of major innovations required to stay ahead in the next few years.
KDE on the other hand has a very clear vision. While I don’t believe it’s completely stable right now, I don’t doubt that it soon will be. When it is, I’ll consider switching, but not until then (hopefully GNOME will step it up so I won’t have to switch.)
Due a decision by KDE’s board, they decided to release KDE much too early. Before it was ready. Their idea was to release it early so it could be improved through user feedback. They officially announced that people requiring a stable system shold not use 4.0, and wait for 4.1
In my opinion this was a mistake. They lost a lot of community trust. Even though they warned people, it wasn’t enough. People still switched and got burned. It’s unprofessional for a company to release a a product that they know isn’t ready (despite giving warnings, it was given a 4.0 stable release name). The flip side is that more people used it, and that probably helped them through feedback)
Despite people’s opinions on whether they should have waited or not, what’s certain is that they’ve lost community trust. It’s sort of like what Microsoft has done with Vista. Even though Vista’s service pack ‘may’ have fixed some stuff (though that’s debatable), they lost trust. People who got burned with the original release may not re-try it now.