Cutest Cutset Conditioning

Well, I am delving deeper and deeper into the realm of machine learning and inference/learning algorithms, and even a bit of information and graph theory. One book I’ve found to be really helpful so far is Information Theory, Inference, and Learning Algorithms by David MacKay. Basically, besides being a really cool textbook (how many authors actually include a dependency chart in their table of contents?!) it is available for free online!

I also stumbled upon a useful introduction to the world of unsupervised learning in the form of a tutorial by Zoubin Ghahramani. The coolest part of this overview of the different methods for unsupervised learning was the section on exact inference in graphs. A technique new to me for graph-based inference is a method known as “cutset conditioning.” I don’t think I fully understand why it works, the idea is based on “reasoning by assumptions.” Basically you find a smallest set of nodes in the graph that when known form a singly connected graph. Based upon this, as well as a small typo in the paper, I have created my own inference technique for graphs, “cutest conditioning.”

The principle behind this is even simpler than cutset conditioning. Here, we are searching for the minimal set of nodes that when removed form the cutest graph possible. I’m not sure what sort of implications this could have for the field of graph theory and mathematical punnery in general. I have included a diagram of this:

Based solely on the included graph, I can infer that I should find more productive things to pursue in my free time.

I got 99 – 97 problems…

As I am three weeks into my REU, I can definitely post exactly what I’m doing since I have a pretty good feeling for it.

So basically I am working on two main things:

1. Help design and implement unsupervised learning into the current model for DIVA, the professor’s NN.
2. Figure out exactly what sort of dimensional reduction occurs within the hidden node architecture of DIVA. Is it similar to principle component analysis? Or maybe even some other form of feature space dimensionality reduction? So determine what this is, and hopefully, with a little luck, be able to formalize it.

What I’d say the coolest implication of this is right now, for me anyway, besides the epistemological values, is seeing what type of implications this has for machine learning. To do this, we are currently talking about taking on an insanely challenging dataset, the netflix dataset.

We shall see where that ends up! Realistically the top contenders (BellKor and BigChaos) are ridiculously close and their progress seems to be slowing of late. BellKor is particularly impressive, with a 9.15% improvement (10% is the prize) over Netflix’s current algorithm.

I would be ecstatic to even submit an entry. Realistically speaking… well I don’t have a clue how realistic this is, but you have to shoot for the stars to hit the moon, eh?

Summertime and the REU is easy…

So, this is my first attempt at a blog and is largely an attempt to both improve my writing as well as provide some accountability for my summer goals! I am spending my summer in Binghamton, NY, participating in a Research Experience for Undergrads (REU). The basic idea of an REU is to provide little undergrads, like myself, an opportunity to check out what research and life in academia is all about.

The main question now is… well what am I doing? Well, I’m researching under Dr. Ken Kurtz. We are working on his artificial neural network, DIVA. It’s a blast. It’s something totally unique for me, and I’m really grateful that I was able to do a project that spans across both computer science as well as cognitive science (and not to mention touches on information theory, statistics, and machine learning).

I will post more as things progress!