Search Engines: Technology, Society, and Business
2007-12-18 01:55PM PST/Home
Here's a very interesting class at Berkeley on Search Engines: INFO 141 Search Engines: Technology, Society, and Business. They have the lectures online as a free podcast. If you're at all interested in Search, this is a great one.
If you listen to Yahoo's Jan Pedersen's lead-off talk, at about an hour or so, he describes a particular experimental methodology for web pages, one of many, which is pretty much what I do for a living currently in a nutshell. I loved how he neatly captured the limitations of what Yahoo calls bucket tests (I really hate that name by the way).
At 01:02:58 into Jan's talk, if only I could get these points across to more people:
"There are some strong limits to this kind of methodology. Because the only things you can test this way are first, things that you are willing to expose users to. So if you're doing something very experimental you probably don't want to do a live experiment because you don't know what the impact is. It's really only good for minor incremental variations on what you are doing. And the second thing is that you have to have a behavioral number that you can interpret correctly."
I've only listened to the first talk so far, I'll post some of my other favorite quotes as I work my way through.
Update: Google has released a number of their introductory training lectures Google: Cluster Computing and MapReduce, certainly worth a look.