Showing posts with label machine. Show all posts
Showing posts with label machine. Show all posts

Monday, November 3, 2014

Perfect Heads-Up Limit Hold' Em Strategy part 1

STUDYING THE STRATEGIES OF THE FOUR ALL-TIME BEST HULHE PLAYERS, MAN OR MACHINE, TO CREATE YOUR OWN


The best way to find the Perfect Heads-Up Fixed Limit Hold' Em Strategy is to study the strategies used by the great ones.  For me the greats in HULHE are two humans plus two machines.  My two human players are Phil Ivey and Andy Beal.  My two poker bots are Polaris and Texas Hold ‘Em Heads Up Poker.  I will try to enter the brain of those four world-class experts to find and analyze their winning strategies.





Phil Ivey's HUHU FL HE strategy:

Step 1. Raise pre-flop in position

Raise pre-flop in the big blind with (1:1)
Ax, Kx, Q3s, Q6, J8, T7s, T9, 98s, 33+
Step 2. Continuation bet on the flop and turn 100% of the time
Step 3. Leaving after losing one buy-in

Keep playing when winning

Why is Phil Ivey so good?
a) hyper aggressive
b) incredible focus and concentration
c) uncanny ability to detect and exploit opponents's weaknesses and betting patterns
d) zero tilt factor
e) intimidating and fearsome reputation
f) unlimited bankroll





Andy Beal High Stakes Heads-Up Limit Hold' Em strategy
Step 1. Play nearly every hands
Raise (most of the time) every pot he enters
Step 2. Check-call all the way down to the river with any A-high hand
8-bets the turn with two pair
Step 3. Obsession with preventing tells

Why is Andy Beal so good?
a) Obsession
b) Specialization
c) Mathematical approach
d) High stakes
e) Ultra-aggressive
f) Impossible to read




Phil Ivey vs The Brain




In NVGtard fantasy land (12-01-2010) :

"The Brain" heads up poker machine proves to be remarkably resilient, beating the best minds in poker over a large sample. When the limits are raised to 200/400 Hoss tbf flys to vegas and enters the Bellagio and plays it days on end only to break even until he tires and begins to lose. And so the machine sits outside the poker room unplayed and unscathed save a few suckers from time to time. The stakes are raised to any that the tourist wishes to play, from .50/1 to 500000/1000000. A billionaire is reported to have lost 4 million on may 23, 2011 during a break from highstakes craps. One day at the poker table Ivey is asked if he would play heads up nl omaha by a young upcoming susperstar from norway and Ivey responds that he would play anybody. The kid says I bet you wouldn't play the brain smugly. "Who's that?" asks Ivey. The kid points to the machine. Ivey says sure and spends the next 45 minutes inserting a giant wad of hundreds and over the course of the next 28 hours he beats it for 300 million dollars infront of the ever growing massive crowd of spectators. Finally after being down 324 million dollars and 28 hours of poker, The Brain ask Phil Ivey if he would let him play on credit. A fat executive hobbles out of the crowd red faced and screams hell no! and pulls the plug.


That's exactly what could happen if Phil Ivey was met to play Texas Hold' Em Heads Up Poker machine "The Brain".  This funny story was written by spaceman Bryce on the twoplustwo poker forum.
http://forumserver.twoplustwo.com/29/news-views-gossip/heads-up-poker-machines-raise-limits-929205/index2.html



It's nice to see that Phil Ivey's dominance made him a mythological figure just like Chuck Norris. 

Now you can say that Phil Ivey is the Chuck Norris of Poker.


Originally published in: http://www.heads-up-fixed-limit-texas-hold-em-poker.com/2013/10/phil-ivey-vs-brain.html 

Polaris

Polaris is the Deep Blue of poker.


The patience of a monk or the fierce aggression of a tiger, changing gears in a single heartbeat. Polaris can make a pro's head spin. Bluff, trap, check-raise bluff, big lay-down -- name your poison.


Polaris is a Texas hold 'em poker playing program developed by the computer poker research group at the University of Alberta, a project that has been under way for 16 years as of 2007. Polaris is a composite program consisting of a number of bots, including Hyperborean08, the winner of the limit equilibrium series in the 2008 Association for the Advancement of Artificial Intelligence (AAAI) Computer Poker Competition. Polaris also contains a number of other fixed strategies, and chooses between these strategies during a match. Polaris requires little computational power at match time, so it is run on an Apple MacBook Pro laptop during competitions. Polaris plays only heads-up (two player) Limit Texas hold'em.



Limit Hold'em (LH) Heads-up Duplicate poker


On July 23–24, 2007, Polaris played against poker professionals Phil Laak and Ali Eslami at the Hyatt Regency Hotel in Vancouver, B.C.  The competition consisted of four duplicate matches, with 500 hands per match. In each duplicate match, the same cards were dealt to both pairs of players, human and bot, but with the seating reversed.  After roughly 16 hours of play over two days, Polaris tied the first round, won the second and lost the last two.

Final result -- Polaris record: 1-2-1


On July 3–6, 2008, Polaris competed against six human professional poker players in the Second Man-Machine Poker Championship, held in Las Vegas at the 2008 Gaming Life Expo. Polaris defeated the human players with three wins, two losses and one tie. Each of the six sessions was a duplicate match of 500 hands against two different players, resulting in six thousand hands played.

Across all six sessions, Polaris won 195 big blinds. The version of Polaris used in the 2008 match was much stronger than the 2007 version, both in the quality of the component strategies and in its ability to learn which component strategy to use.

Final result -- Polaris record: 3-2-1

Overall Polaris's record vs. humans: 4-4-2


Polaris and Deep Blue

Comparing Polaris results with IBM chess computer Deep Blue versus reigning world chess champion Garry Kasparov in two games under tournament regulations.

Result: Kasparov–Deep Blue (4–2)

Result: Deep Blue–Kasparov (3½–2½)

Overall Deep Blue record versus Kasparov (5½–6½)

Polaris (4-4-2 or 5-5) vs. Deep Blue (5½–6½)

So with that being said. We can see that Polaris had better results versus humans than Deep Blue.




Polaris's Strategies:

Polaris have a total of 5 different strategies
The Nash Equilibrium plus 4 other to use against different types of opponents.  Polaris identifies which common poker strategy a human is using and switches its own strategy to counter.

1- The first approach is to approximate a Nash equilibrium strategy which is robust against any opponent.
2- The second approach is to find an exploitive counter-strategy to an opponent. We will show that these counter-strategies are brittle: they can lose to arbitrary other opponents.

3- The third approach is a compromise of the first two, to find robust counter-strategies.
4- The four approach is to combine several of these agents into a team, and learn during a game which to use.



Why is Polaris so good?

a) Adaptation. Polaris doesn't have a "best way" to play; it has a "best way" to adapt.
b) Specialization
c) No emotion
d) Aggression





"The Brain"





TEXAS HOLD'EM HEADS UP Poker

"Texas Hold ‘Em Heads Up Poker" machine play the limit version of the most popular poker game around.  The casino owners obviously think that no human opponent could be able to defeat "The Brain" on a regular basis.   However, the manufacturer estimated that only 100 around the world will be able to beat it on a regular basis.  But that remain to be seen.  Some accomplished world class poker players and poker theorists think otherwise. 

I’ve seen about 10 different [heads-up] pros playing the bot, all with their own special strategy which they are convinced is the one that beats the bot. I feel less than 95 percent certain that they are all losing players against it, and at least one of their assumptions has always been wrong. I feel after 100 hours of play against it I can break even against it. I think I’d rather play craps, though.”  ---  Anthony Rivera, high-stakes limit pro and World Series of Poker bracelet winner


You vs "The Brain"


Non adapting strategy "artificial neural network" 

"Texas Hold ‘Em Heads Up Poker" machine was originally designed by Dahl to learn while playing and adjust his strategy to the opposition. If, for example, an opponent folded a lot, it played aggressively; if it faced aggressive play, it tried to trapHowever casino commissions mandate that a gaming machine cannot change its playing style in response to particular opponents. 

Playing a fixed strategy, "The Brain" isn't able to change its tactics to take advantage of the bad play of a novice player. A skilled human poker player or a computer algorithm with opponent modeling capability should expect to win at a higher rate than this game against especially weak opponents.  So the poker game must play a World Series of Poker champion the same way it does a neophyte.  Nonetheless an unskilled player will lose more against "The Brain" than a skilled player. 


Perfectly defensive game

What seem to be a disadvantage turned into a huge advantage. Due to the fact that in theory a non adapting strategy could be unbeatable.  If its play away from its approximation of game theoretic optimal strategy to take advantage of opponent mistakes, then it itself is playing less optimally and may be counter-exploited.
With that being said, since the machine bot needs to be stable and can't figure out weaknesses in the opponent and find ways to exploit those weaknesses.  So instead, the effort were put on defense. The basic idea behind its play technique is "to prevent itself from being exploited." "The theory behind it is almost paranoid."  So that it would teach itself to play a perfectly defensive game. Rather than steer it to study its opponent and try to capitalize on weaknesses, the net was directed to make itself as hard to beat as possible.

"If it "adapted" it would open itself up for explotation. In theory a non adapting strategy could be unbeatable. I played it for a few minutes and it was quite aggressive and tricky. It even check raised bluffed the river once. At first glance it appears to me that only very good players will beat it, they won't beat it for too much, and that the correct strategy to beat it is a bit different than against typical humans." -- David Sklansky, respected poker author and mathematician




Texas Hold ‘Em Heads Up Poker "The Brain" strategy

Pre-flop
Hyper-LAG / maniac
+ Raise almost every time from the Button
+ Raise called button from over 75% of the time from the BB.
... Sometime 3-bet weak hands like T5off (negligible)
... Occasionally fold on the Button or when raised in the BB (negligible)
- Fold more hands from the SB on most streets than from the BB

Flop
Very LAG
+ Routinely raise with middle pair and a weak kicker
+ Call most of the time a check-raise+ Rarely 3-bet the flop with weak overcard, no straight, flush or backdoor possibilities
+ Often fold to a bet if a high card comes on the flop
+ But he will bet the high card on the flop
+ Most likely to fold on the flop

Turn
Tightens up
+ Check-call with middle pair and weak kicker
+ Check the turn most of the time 

River
+ Call with a piece of the board
+ Call down with Ace high unless the board is scary
+ Likely to call with King high
+ Sometimes call down with weaker hands (J high) on a bimodal (monster or miss) board.  Like a pair on the board.

Overall
+ Reasonable adjustment to what the opponent might hold based on the board.
+ Seem to play top pair / top kicker more slowly on board with flushes and straights possible
+ Seem to play high card / no pair more aggressively on paired flops.  Check-raise bluff
+ More likely to check-raise if opponent put the last bet on the previous street.
+ Doesn't fold too often.



Why is "The Brain" so good?

a) Defense. The basic idea behind its play technique is to do everything it can "to prevent itself from being exploited".  "The theory behind it is almost paranoid," 

b) Unpredictable. The pokerbot use knowledge gained from billions of staged rounds of poker fed through neural networks, and the result is an unpredictable poker player that can win almost every time.  Three different banks of knowledge are used depending on the gameplay scenario, but the basic idea behind its play technique is, as I said above,  "to prevent itself from being exploited." "The theory behind it is almost paranoid." So "The Brain" unpredictibility and three different game plays are part of his perfectly defensive game. 
   
Here are his 3 tag-team fighters alternating against an opponent.
    * The first is a neural net with optimal number of bluffs and can do anything in anyone hand.
    *The second play a slightly different style.
    *The third come into play when the opponent has a reduced stack.

Aggression level might change at random moment

c) Aggression.  "The Brain" is the aggressor. Almost never check-calls or simply matches an opponent's bet without a raise. But give credit to your hand when you raise and reraise.  Far too aggressive and steals far too many pots to get beat on a regular basis.




http://www.lvrevealed.com/articles/research/investigating-texas-holdem-heads-up-poker.html
http://www.nytimes.com/2013/09/08/magazine/poker-computer.html?pagewanted=1&_r=1&
http://forumserver.twoplustwo.com/109/heads-up-limit/beating-heads-up-limit-poker-machine-1170564/


Originally published in: http://www.heads-up-fixed-limit-texas-hold-em-poker.com/p/blog-page.html