Investigating "Texas Hold'em Heads Up Poker"

by Nick Christenson

This article originally appeared in the November 2010 issue of TwoPlusTwo Magazine.

Background

Recently there was a discussion in the TwoPlusTwo "News, Views, and Gossip" forum, now closed, about a new video casino game that might be of interest to poker players. Called, "Texas Hold'em Heads Up Poker", it's a slot machine style console that contains a computer that plays regular old heads-up limit Texas hold 'em against players who care to put up their money.

What's especially intriguing is that there's no rake of any kind charged. Assuming it's not cheating, which seems unlikely for a machine licensed in Nevada and built by a reputable manufacturer, the only way it can win in the long term is if it plays better than its opponents. That's an intriguing proposition to anyone interested in the game of poker.

I have a background in software development, game theory, and have been writing an article series for TwoPlusTwo Magazine reviewing research on developing effective poker playing software, so investigating a game such as this one is right up my alley.

The Game

The game console is manufactured and marketed by IGT, the largest manufacturer of slot and video poker machines in the world. The console indicates that G2 Game Design. was the developer. The game is described on p.15 of an IGT product procure (PDF). I tried to contact folks at G2 Game Design through their web site for this article, but I didn't receive a response as deadline approached.

The game is available at the Bellagio and Red Rock in Las Vegas, and possibly other casinos. In both cases I've personally verified, the machines are set up a short distance from the poker room, which is only logical. The brochure referenced above mentions that the game is approved for use in most major U.S. jurisdictions.

Game play itself is pretty straightforward with video poker-style buttons to check, call, fold or bet/raise. These are pretty easy to work and there's relatively little chance of making a bet that wasn't intended. A side bet is available called "bonus bet". It allows you to make a bet that a hand will have a certain value on the flop turn or river, much like the "pair plus" bet in Three Card Poker or in video poker. I haven't run the numbers, but I can't imagine this bet is +EV, so I didn't play it.

I wish the game were more clear about whether the amounts of money listed on the console refer to the size of the blinds or the amount of the limit bet on early vs. later betting rounds. The bottom line is that the game I played offers limits of $0.50/$1.00 with $0.25/$0.50 blinds, $1/$2 limits with $0.50/$1 blinds, and $2/$4 limits with $1/$2 blinds. Presumably, the game could be tuned to offer other denominations if desired. As is usual in heads-up poker, the small blind is on the button, and begins the pre-flop betting action, and the big blind acts first on all other betting rounds. When you start a session the computer gets the button first.

The one aspect of game play that I found confusing was the Bonus Bet button. It's not a binary "on/off" thing. With repeated presses it cycles through various sizes of bets (nothing, $0.25, $0.50, $0.75, $1, $1.25.) So, if the previous player of the game has turned it on, you can't just toggle it off. Additionally, it's not very clear when this bet is in effect and when it's not. The player needs to look carefully at the screen by the bonus bet pay table to determine if this money is being deducted or not. Players should be cautious of this.

The manufacturers are confident enough in the non-exploitability of the game to allow its human players to "peek" at the game's hole cards at the conclusion of the hand. Needless to say, this makes it much easier to analyze how it plays.

At the Bellagio there is a receptacle for a slot club card, but the machine is clear that at the present time slot benefits don't accrue from playing this game. At Red Rock you do accrue slot club points for playing this game.

How It Works

The brochure claims its poker acumen comes from "neural net technology". A neural net, or, more properly, an "artificial neural network", is a programming technique that is based on the structure of a biological brain that adapts the pathways of information flow through the system based upon positive results.

A neural net can be set up to continually learn and improve its strategies over time. Again, the brochure states that it does not do this. That is, it's telling us that a neural net was employed to learn how to play poker, but once its strategy was felt to be "good enough", this was frozen and deployed in the machines in question. That doesn't mean that the strategies used in these machines couldn't be updated some day with a superior algorithm, but it wouldn't be trivial to do so.

A side effect of this design decision, and again, a feature claimed by the manufacturer, is that because its playing a fixed strategy, it isn't able to change its tactics to take advantage of the bad play of a novice player. A skilled human poker player or a computer algorithm with opponent modeling capability should expect to win at a higher rate than this game against especially weak opponents. This isn't to say that an unskilled poker player won't lose more against this game than a skilled poker player, that's absolutely what one should expect. It's just that the difference in EV won't be nearly as dramatic as it would have been if the algorithm were designed to successfully exploit weak play.

There are two other reasons why I think it was wise for the designers to not incorporate opponent modeling into their software. First, if the computer algorithm adjusts its play away from its approximation of game theoretic optimal strategy to take advantage of opponent mistakes, then it itself is playing less optimally and may be counter-exploited. This gets one in to a multiply layered strategic battle, and while it's certainly possible for a computer to play these sorts of games well, it adds a great deal of complexity to the software for relatively little payoff.

Second, how does the computer know when one player gets up and another leaves? Does the game algorithm have ties into the bill acceptor? How about the slot card reader? I seriously doubt that Nevada Gaming Control would be all that comfortable with this. In any case, both of these measures be easily thwarted by a player who was aware of them.

How It Plays

I've played against it for several hours with an eye toward examining how it plays. While I'm not a recognized name in the world of heads-up online limit poker, I do have experience playing short handed and heads-up poker successfully over more than a decade. Note that even with several hours of playing, this is still a small sample size and the fact that I'm playing and taking hand-written notes almost certainly leads to inaccuracies, so my impressions should be taken with a grain of salt.

First, pre-flop I'd describe it as hyper-LAG, and if someone wanted to categorize it as a maniac, I wouldn't put up much of a protest. From the SBB (Small Blind on the Button), it would raise almost every time. From the BB, if I just called in the small blind, it would raise over 80% of the time. I've observed the machine three bet with hands as weak as T5off, although I can't say that it does so often.

Pre flop, it does occasionally fold in the small blind or when raised in the big blind, but, again, it doesn't do this often. Leading from the SBB position, I approximate its folding range as T3o, 94o, 84o, 74o, 63o, 53o, 42o, 32s, or worse. From the BB facing a raise with a small sample size, I've seen it fold 92o, 72o, 62o, and 42o. Note that it folds more hands from the small blind in position on most streets than it does from the big blind facing a raise despite the pot odds being the same. The reason for this is that calling a raise from the big blind pre-flop closes the betting for that round, and that's more important than positional issues. This is consistent with game theoretic solutions to restricted poker games. See for example my commentary on Alex Selby's classic analysis of pre-flop limit hold 'em.

On the flop I'd still categorize its play as very LAG. I've seen it 3-bet the flop with hands as bad as one weak overcard, no straight, flush or backdoor possibilities, something like a J4 on a T82 flop, although this is very rare. These aren't bluffs. It's clear to me that the algorithm "knows" that these are hands that have value in catching a free card, so it saves its pure bluffs for hands with no showdown value, which is appropriate. From a limited sample size, the game is much more likely to fold on the flop than pre-flop, but it doesn't need much of a hand to continue at this point.

On the turn the game tightens up. It's much less likely to raise with a weak hand. On the flop, it would routinely raise a bet with middle pair and a weak kicker. On the turn with a similar strength hand it will seem to go into check-and-call mode. Of course, like all good heads-up players it still plays very loose. Getting hyper-aggressive against it at this point as a counter-strategy is not going to be a winning strategy.

On the river if it has a piece of the board, it's calling. If it has ace high, unless the board is very scary that's enough for it to call down the whole way. If it has king high, that's likely enough for it to call on the end. On some boards, especially those where the opponent's hand strength is likely to be bimodal (a monster or a complete miss), it will call down with even weaker hands. I've seen it make calls on the end with J high on a one pair board, for example.

Players whose immediate reaction to this strategy is to go into check-and-call mode trying to trap it are probably not used to playing a lot of heads-up poker. Hyper-LAG on the early betting rounds is a very strong way to play. You have to play very loose to survive, and making heroic folds on the end in heads-up limit poker is going to get you crushed. I found the game to be a very challenging opponent.

One aspect of programming learning algorithms, such as artificial neural networks, is that during their training phase it's possible for them to experience some fortunate success with non-optimal strategies that then take a long time to unlearn. I suspect that the game's hyper-LAG style early on might be an artifact of this. One aspect of that strategy is that if you play very aggressively early on, strategy for the later betting rounds becomes somewhat simplified. If you're always putting your opponent to the test, then if they keep coming they must have something, and because folding is rarely a good option in heads-up limit hold 'em, this constricts the range of options on the big bet streets.

However, from my limited experience it does seem to do some things pretty well that other poker software doesn't. For example, it seems to do a decent job of adjusting its expectation of what an opponent might hold based on board composition. It seems to play top pair/top kicker more slowly on a board with flushes and straights possible. It also seems to play high card/no pair more aggressively on paired flops. It bluffs, check-raises, and will do the occasional check-raise bluff.

It also seems to do a reasonable job of adjusting its opponent hand range based upon previous street action. That is, if it spikes top pair on the river, it's likely to raise a bet if there has been little action earlier in the hand, whereas it will just call if its opponent has shown a great deal of strength on previous streets.

Similarly, it also seems to have a multi-street model for its betting plan in a hand. It seems more likely to go for a check-raise if its opponent put in the last bet on the previous street. These aren't easy aspects to add to a heuristic computer program, and it shows the game's strength, as well as the benefit of employing neural net programming. If it is properly parameterized and has effective training, then you don't need to explicitly program in a Baysian opponent model or deal with the abstraction of single street betting models.

How People React To It

One thing I find even more interesting than the game itself is how people react to it, both in person and in the forums. A dear departed friend and colleague of mine was fond of saying, "Don't anthropomorphize the computers. They don't like it." I can't help but think of that when I hear other players talk about this game.

Many people seem convinced that the game is adaptive even though the designers of the game insist that it's not. The human brain is so good at pattern matching that it tends to find patterns even when they're not there. The game may get a certain class of hand more often at one point in a session than in another, and our brains compel us to believe that it has switched up its game.

In reference to the game I've heard every variation of "it plays great"/"it plays awful" in combination with "I won/I lost". Everyone is talking about the same machines, so some of them have to be wrong. I find public sentiment on the quality of the play of this game to be all over map. Part of that is due to the fact that few poker players have any idea what good heads-up play looks like, but it just goes to show you how unreliable the consensus is when evaluating poker talent in general, especially with small sample sets. I think this applies equally to poker players ability to evaluate human talent levels.

I think the game plays strong, but it's not unbeatable. What's my EV against it? I have no idea, but if it's positive, I don't think it's going to make me rich. I'll bet on the machine against most denizens of a given Las Vegas poker room, but that's largely a reflection of how poorly I think the way most players play translates to a heads-up game.

I didn't see it making huge mistakes. The big mistakes in heads-up limit come in folding too often, and it didn't do too much of that. Of course, as I gain more data, my opinion might very well change.

Conclusion

I found Texas Holdem Heads Up Poker to be an interesting game. I'd like to learn more about it, and for the time being, I'm willing to play against it. I'd be very interested in talking to the developers about whether my suppositions about the game are correct or not. If I learn more about its internals, I'd also be interested in writing a follow-up to this article with additional analysis.

I'd also like to encourage the poker community to embrace this game. It has several aspects that are very rare for casino games, and I'd like to encourage these aspects in casino operators and game manufacturers.

1. Even though I expect it will be rare to find many people who can both play better than it can and are interested in doing so for an extensive period of time, it's a casino game that if you play well enough you can break even with or even beat it.
2. The basic strategy is complicated, so it's a casino game that actually makes you think. Maybe that doesn't appeal to many casino patrons, but I hope it appeals to enough patrons to justify keeping a bank of these machines around many poker rooms.
3. It's something unobjectionable to do while waiting for you seat in the poker room. This makes me more likely to wait around for a seat, which should be good for card rooms.
4. In playing this game, I can actually learn something. If I'm better than it is, then I can learn how to better recognize weaknesses in my opponents in order to exploit them. If I'm worse than it is, I can learn a superior heads-up strategy. When's the last time anyone has said this about a casino game?

So, I encourage folks to give it a try and spread the word.