No cs234 project

It was with a mix of disappointment and relief that I learned cs234, the final course for my Artificial Intelligence Graduate Certificate from Stanford, cancelled the project for this quarter. With the course going fully online due to the pandemic, the rational was to give everyone a bit of a break. That is perfectly reasonable given the work projects often entail, but it would have been pretty cool.

As such, development is once again suspended. (These Stanford courses are a lot of work, and there is a family and a job.) During the December sprint, however, I am happy to say that quite a bit of progress was made:

2020 GitHub Contributions
2020 GitHub Contributions
  1. A line of code was written – Just doing anything after 4 years is an accomplishment.
  2. Automated combat with enhanced messaging and logging – When I do get around to building a deep RL model to train the agents, this will be handy.
  3. Created the wizard class with wizard spells, including touch spell attacks – This one rounds out the fighters, clerics and rouges. I won’t be adding any new classes for a while.
  4. Group combat – Previously it was the ‘hero’ versus the monsters, but now two groups of combatants can fight each other.
  5. Charging, flanking and cover – These add a little more dynamic to the combat.
  6. Ferocity – Orcs just became much more challenging.
  7. Fixed a bunch of bugs – Always a decent idea.

So, what does all this progress look like? Here’s a little demo video set to O Fortuna from Carmina Burana by the MIT Concert Choir.

I’ll try to pick it up again in April, but we have to see how that goes.

A line of code has been written

The title’s deliberately passive voice is intended to comically understate the fact that it has been four years since I last worked on this project. Four years! I previously wrote about the delay, but the long story short is that life got busy and I got distracted with, arguably, much more important things. That being said, I am giving it another go. The renewed motivation comes from an unlikely source: Stanford University.

Over the past several years I have become quite passionate about AI, taking a number if different MOOCs, reading various book and pursuing some personal projects to explore the domain and learn the technology. One day in late 2018, gazing out of my office window at PARC, I spied the spire at the top of Hoover Tower and thought to myself, “I bet they have got some AI classes over there.”

Turns out they do. More specifically, turns out they have a Graduate Certificate in AI which offers a 4-course, 16-unit program of master’s-level instruction in AI. What could be better than that? A statement of motivation and a couple transcripts later, I was in. With a family and demanding full-time job I knew that pursuing graduate coursework would be challenging, but if I took only one course at a time, how hard could it be?

I quickly discovered that it could be very, very hard. Deep Learning (cs230) with Prof. Andrew Ng was the first course I took, which was probably a good choice. It was not a ton of work, until the final project, but the midterm was shockingly difficult. AI Principals and Techniques (cs221) is the only required course for the Certificate, so I took that next. It wasn’t too difficult, and I was more prepared for the midterm, but it was a lot of work and our final project was challenging. Then I took Machine Learning (cs229), again with Prof. Ng. It is notoriously one of the most difficult courses in the department and it lived up to its reputation. It was also an amazing amount of work and really tested my family life, but I survived that, too. In fact, we all did.

So what is this all about? For my last class I am signed up to take Reinforcement Learning (cs234) with Prof. Emma Brunskill. What does that have to do with Mortal Wayfare? Well, like all the courses, there is a final project. I therefore thought it could be really cool, for the final project, to implement RL within the game to ‘teach’ the actors (i.e. monsters and NPCs) to fight optimally. You can tell from game play that most RPGs use a simple rules-based decision engines to control enemies during combat. With a complex D&D-based system, however, constructing those rules can be complicated and the end result is often flat. I am curious to know what sort of dynamic and exciting combat I might be able to get by leveraging the materials from the course. To be able to do that I need to get the game back in working order as well as build a mechanism for fully-automated, repeating combat. After only a week, which I’m chronicling on Twitter, I am getting closer, but we will see how it goes. The course starts in January, so I am running out of time. Whatever happens, however, I will report back here.

In any event, unwilling to stop after course number four, I have already submitted my application for the MSCS program. Wish me luck!

Holy hiatus

I’m sure nobody noticed because nobody is actually following this blog, which is fine. That being said, I just looked up my last commit, which was Oct 31th, 2016 – 671 days ago! During this time I have been busy with full-time employment, taking classes in ML and AI, playing around with deep learning, volunteering for a fantastic AI conference, running the books and making a presentation for an amazing non-profit and otherwise life.

In the back of my mind, however, there has always been Mortal Wayfare. I’ve fantasized about finding time to do everything from creating the wizard class (there are already fighters, clerics and rogues) to using reinforcement learning to ‘teach’ the enemies and NPCs policies for optimal decision-making. So why this post now?

Yesterday, in a burst of get-it-done-idness, I fixed the MW email and upgraded the phpBB to v3.2.2. It still needs some tweaks, but I’m now going to try to figure out a system to deflect the relentless spammers, who, for the moment, are the only visitors to this site. Then I’m going to download the repo and try to figure out a plan to, painful inch by inch, move this project forward until, one day, it’s the most amazing retro 2D Android turn-based RPG on planet Earth. If not, I hope to at least have something that is playable. That sounds like a lot, but actually, I don’t think I’m too far off.

Wish me luck, nobody.