No cs234 Project

It was with a mix of disappointment and relief that I learned cs234, the final course for my Artificial Intelligence Graduate Certificate from Stanford, cancelled the project for this quarter. With the course going fully online due to the pandemic, the rational was to give everyone a bit of a break. That is perfectly reasonable given the work projects often entail, but it would have been pretty cool.

As such, development is once again suspended. (These Stanford courses are a lot of work, and there is a family and a job.) During the December sprint, however, I am happy to say that quite a bit of progress was made:

2020 GitHub Contributions
2020 GitHub Contributions
  1. A line of code was written – Just doing anything after 4 years is an accomplishment.
  2. Automated combat with enhanced messaging and logging – When I do get around to building a deep RL model to train the agents, this will be handy.
  3. Created the wizard class with wizard spells, including touch spell attacks – This one rounds out the fighters, clerics and rouges. I won’t be adding any new classes for a while.
  4. Group combat – Previously it was the ‘hero’ versus the monsters, but now two groups of combatants can fight each other.
  5. Charging, flanking and cover – These add a little more dynamic to the combat.
  6. Ferocity – Orcs just became much more challenging.
  7. Fixed a bunch of bugs – Always a decent idea.

I’ll try to pick it up again in April, but we have to see how that goes.

A line of code has been written

The title’s deliberately passive voice is intended to comically understate the fact that it has been four years since I last worked on this project. Four years! I previously wrote about the delay, but the long story short is that life got busy and I got distracted with, arguably, much more important things. That being said, I am giving it another go. The renewed motivation comes from an unlikely source: Stanford University.

Over the past several years I have become quite passionate about AI, taking a number if different MOOCs, reading various book and pursuing some personal projects to explore the domain and learn the technology. One day in late 2018, gazing out of my office window at PARC, I spied the spire at the top of Hoover Tower and thought to myself, “I bet they have got some AI classes over there.”

Turns out they do. More specifically, turns out they have a Graduate Certificate in AI which offers a 4-course, 16-unit program of master’s-level instruction in AI. What could be better than that? A statement of motivation and a couple transcripts later, I was in. With a family and demanding full-time job I knew that pursuing graduate coursework would be challenging, but if I took only one course at a time, how hard could it be?

I quickly discovered that it could be very, very hard. Deep Learning (cs230) with Prof. Andrew Ng was the first course I took, which was probably a good choice. It was not a ton of work, until the final project, but the midterm was shockingly difficult. AI Principals and Techniques (cs221) is the only required course for the Certificate, so I took that next. It wasn’t too difficult, and I was more prepared for the midterm, but it was a lot of work and our final project was challenging. Then I took Machine Learning (cs229), again with Prof. Ng. It is notoriously one of the most difficult courses in the department and it lived up to its reputation. It was also an amazing amount of work and really tested my family life, but I survived that, too. In fact, we all did.

So what is this all about? For my last class I am signed up to take Reinforcement Learning (cs234) with Prof. Emma Brunskill. What does that have to do with Mortal Wayfare? Well, like all the courses, there is a final project. I therefore thought it could be really cool, for the final project, to implement RL within the game to ‘teach’ the actors (i.e. monsters and NPCs) to fight optimally. You can tell from game play that most RPGs use a simple rules-based decision engines to control enemies during combat. With a complex D&D-based system, however, constructing those rules can be complicated and the end result is often flat. I am curious to know what sort of dynamic and exciting combat I might be able to get by leveraging the materials from the course. To be able to do that I need to get the game back in working order as well as build a mechanism for fully-automated, repeating combat. After only a week, which I’m chronically on Twitter, I am getting closer, but we will see how it goes. The course starts in January, so I am running out of time. Whatever happens, however, I will report back here.

In any event, unwilling to stop after course number four, I have already submitted my application for the MSCS program. Wish me luck!