Part 2: Texas Holdem Poker, Human vs. AI
The competition was set up by the Carnegie Mellon University School of Computer Science, who created a poker-playing program named ‘Claudico’ and were looking for a significant test. The program, with the ability to “learn” as it played and thus is considered an “artificial” intelligence, is the first-of-its-kind in that it was created to play No Limit Texas Hold’em; every other poker playing computer created played the more-statistical Limit version of the game. Once the Carnegie Mellon staff nailed down the players – and the management of the Rivers Casino in Pittsburgh offered an exciting venue to play – the subjects set out on the 14-day competition.
From the start, the representatives of the human race – World Series of Poker bracelet winner and online wunderkindDoug ‘WCGRider’ Polk, Dong Kim, Jason Les and Bjorn Li – moved out to a financial edge that they wouldn’t relinquish. Playing a total of 80,000 hands of $50/$100 Heads Up No Limit Hold’em over the two-week period, the four men built up a $587,231 edge only a week into the play. They would seemingly ride that advantage over the last half of the competition and, once the results were announced on Friday, both sides crowed about their achievement.
When the final tallies were completed, the “Brains” in the competition had vanquished their “Artificial Intelligence” foe by the sizeable figure of $732,713. Leading the way was Li, who accounted for an astounding $528,033 of the total winnings amassed by the humans. Polk didn’t do badly either, racking up $213,671 in winnings and Kim slipped by ‘Claudico’ in taking slightly more than $70,000 in earnings. Only Les would disappoint the human race, dropping $80,482 to be the only one to lose to ‘Claudico’ by the money counts. (The four men divvied up a $100,000 prize provided by the Rivers Casino for their two weeks of work.)
The human players were a bit surprised at the skill that ‘Claudico’ demonstrated. “We know theoretically that artificial intelligence is going to overtake us one day,” Li said during the post-match celebration. “At the end of the day, the most important thing is that the humans remain on top for now.” Les, who had seen a previous version of ‘Claudico’ when it defeated professional players just last year, was stunned by the developers’ skills.
“The advances made in Claudico in just eight months were huge,” Les said, indicating that, at that rate of improvement, an Artificial Intelligence system might need only another year before it clearly plays better than professionals.
Polk seemed to be the only player who critiqued the playing of ‘Claudico’ during the finale. “There are spots where it plays well and others where I just don’t understand it,” Polk noted, stating that some of its bets were highly unusual. Polk saw instances that, where a human might place a bet worth half or three-quarters of the pot, Claudico would sometimes bet a miserly 10% or an over-the-top all-in move. “Betting $19,000 to win a $700 pot just isn’t something that a person would do,” Polk observed.
So who won the event? While the overall numbers would suggest that the “Brains” crushed the “Artificial Intelligence,” a closer look at those figures is necessary. As individuals, the humans once again take a 3-1 winning edge, but the actual analysis of the figures that the players put up indicate that the score might have been closer to 1-0-3, with Li the only outright winner and the remainder of the human team within the statistical range of calling their matches a tie. It wasn’t a point that was missed by the professor who helped to develop ‘Claudico.’
“We knew Claudico was the strongest computer poker program in the world, but we had no idea before this competition how it would fare against four Top 10 poker players,” said Dr. Tuomas Sandholm, the Carnegie Mellon University professor of computer science who helped to create ‘Claudico.’ “It would have been no shame for Claudico to lose to a set of such talented pros, so even pulling off a statistical tie with them is a tremendous achievement.” In replying to Polk’s stab at the unorthodox play during the event, the Carnegie Mellon team admitted they were just as puzzled as to why ‘Claudico’ made the decisions he made.
After the completion of the interesting competition, there have been no indications that there will be another event on the horizon. The Carnegie Mellon team will no doubt head back to the laboratory to tweak on ‘Claudico’ (or potentially a more-potent creation?), while the human race will wait for the next challenge to their ‘superiority’ in this world.
This may turn out to be the latest installment in a grand tradition of computers beating us at our own games. In 1997, IBM's Deep Blue computer famously beat chess great Garry Kasparov. Four years ago, IBM's Watson took part in the TV quiz show Jeopardy! and crushed two contestants with a strong track record. AI has even mastered the popular smartphone game 2048.
Still, poker is a tough nut to crack. In a game like chess, everyone knows where all the pieces are on the board. By contrast, poker is a game of imperfect information: players don't know for sure what cards the others hold or what will come up next in the deck. That makes it a challenge for any player, human or computer, to choose the right play.
(This blog entry was compiled from various sources, and some attribution is lacking. I apologize and will correct this if I can.)
Painkiller Overdoses On the Rise
This news isn’t shocking to those of us who encounter addiction daily, but it’s in the headline today at USA Today. Other items of note from today’s paper:
With nearly 44,000 deaths a year, more Americans today die from drug overdoses than from car accidents or any other type of injury. Many of these deaths could be prevented if patients had better access to substance abuse therapy, experts say. Yet people battling addiction say that treatment often is unavailable or unaffordable.
Only 11% of the 22.7 million Americans who needed drug or alcohol treatment in 2013 actually got it, according to the Substance Abuse and Mental Health Services Administration. While some of those who went without care did so by choice, at least 316,000 tried and failed to get treatment.
"We know addiction treatment saves lives, reduces drug use, reduces criminal activity and improves employment," says Paul Samuels, president and director of the Legal Action Center, which advocates on behalf of people with HIV or addiction. "The data is there, the evidence is in, but our public policy has not caught up with the science."
Meanwhile, the crisis is getting worse, says Thomas Frieden, director of the Centers for Disease Control and Prevention. The death rate from drug overdoses more than doubled from 1999 to 2013, according to the CDC. The bulk of these deaths involve opiates, a class of pain killers that includes morphine and Oxycontin.
Injection drug use has fueled an outbreak of HIV in rural Indiana, a nationwide surge in hepatitis C infections, and an increase in the number of babies born addicted to drugs. States have responded to the surge in overdose deaths by expanding access to naloxone, a fast-acting rescue drug that can reverse the effects of an opiate overdose. Indiana lawmakers also voted to allow needle exchange programs in communities facing a public health crisis related to injection drug use. While those approaches are welcome, they don't treat the underlying addiction.
The wait for a spot in a detoxification program ranges from days to weeks, and it can be very expensive. As I say elsewhere here on my site, it's also true that not all treatment is really good treatment. Consumers are disadvantaged twice. Not only is treatment hard to find in the first place, but it's hard to know what constitutes a good treatment program. I have some comments about this on my FAQ page and on my Philosophy page.
Part 1: Texas Holdem Poker, Human vs AI Computer
HUMAN VS. MACHINE AS TOP POKER PROS TAKE ON AI
Fifteen years ago I, Dr. John Talmadge, became a serious poker player. I have a winning record at the poker table, and although I am (probably) not good enough to quit my day job, I have held my own with professionals in Las Vegas, and I finished in the top third of the field in the World Series of Poker Senior Event in 2009. Although I finished out of the money, I came in ahead of about 2000 other players in that event. In the years since, I have consulted with professional players on "the mental game," and I remain a serious student of Texas Holdem, the world's greatest card game.
Now comes Aviva Rutkin, writing in New Scientist that human poker professionals are taking on artificial intelligence (and the strongest poker playing computer built to date), and they are playing for real money. Here's the story.
Computer scientists have already made some progress, at least with simpler forms of the game. But the version being played at the Pittsburgh tournament, called Heads Up No Limit Texas Hold 'em, is "a completely different beast", says pro player Vanessa Selbst. "There's much more human elements and game strategies to employ, so it's a much more complex game." What's more, there are no betting limits, so the computer also has to take into account how much players might stake on each game.
Competing in Pittsburgh is Claudico, a program created at Carnegie Mellon University. Claudico taught itself poker skills by playing trillions of games in search of some kind of optimal strategy. Whatever it has picked is pretty good: last year, Claudico beat all 13 other computer competitors at no-limit poker in the annual contest run by the Association for the Advancement of Artificial Intelligence.
Computers have a few edges over humans, says graduate student Noam Brown, part of the team behind Claudico. For example, a computer can switch randomly between various betting strategies, which may confuse human opponents.
On the other hand, Claudico is slow to pick up on and adapt to people's playing styles – something that many pro players do with ease. "One of our big concerns is that the human will be able to identify weaknesses that Claudico has and exploit them," says Brown.
Because Claudico taught itself to play, even the team that built it don't quite know how it picks its moves. "We're putting our faith in Claudico. It knows much better than we do what it's doing."
Algorithms like those used to play poker could be valuable for other kinds of problems characterised by imperfect information. They could suggest optimal locations for military resources in a war, for example. Rival AIs could also be tasked to negotiate with each other over insurance rates or handle legal squabbles. "In society, sometimes you see one side getting screwed over because someone has more lawyers or more information or more resources at their disposal," says Brown. "Something like this can really level the playing field."
The winner of the poker tournament won't be crowned until the event ends on 7 May. Eric Jackson, a software engineer who creates poker bots as a hobby, is cautiously optimistic that Claudico can win. As we went to press, the pros and Claudico were neck and neck.
Even if AI triumphs, it won't mean programmers have conquered the game. "Beating humans decisively would be a landmark, but it wouldn't mean the end of work on poker," says Jackson. "We still don't know what the perfect strategy is."
The original article by Aviva Rutkin can be found online here.
Traumatic Brain Injury: Progress
"Gist reasoning characterizes a meaningful, complex cognitive capacity. Assessing how well one understands and expresses big ideas from information they are exposed, commonly known as an ability to "get the gist", is window into real life functionality. I do not know of any other paper and pencil test that can tell us both," explained Asha Vas, Ph.D., research scientist at the Center for BrainHealth and lead study author. "Although performance on traditional cognitive tests is informative, widely-used measures do not paint the full picture. Adults with TBI often fare average or above on these structured measures. All too often, adults with brain injury have been told that they ought to be fine; in reality, they are not doing and thinking like they used to prior to the injury and struggle managing everyday life responsibilities years after the injury. Gist reasoning could be a sensitive tool to connect some of those dots as to why they are having trouble with real-life functionality despite falling into the range of "normal" on other cognitive tests." For more details and the longer article, click here.
Kevin McCauley, M.D. on Addiction
Aging Brain? Not So Bad...
At middle age, the brain begins to draw on more of its capacity for improved judgment and decision making.
If you forget a name or two, take longer to finish the crossword, or find it hard to manage two tasks at once, you’re not on the road to dementia.
What you’re experiencing is your brain changing the way it works as you get older. And in many ways it’s actually working better. Studies have shown that older people have better judgment, are better at making rational decisions, and are better able to screen out negativity than their juniors are.
Although it may take you a little longer to get to the solution, you’re probably better at inductive and spatial reasoning at middle age than you were in your youth.
The brain changes as we get older, and in some ways it works better as we get older.
How is it possible for older people to function better even as their brains slow? “The brain begins to compensate by using more of itself,” explains Dr. Bruce Yankner, professor of genetics and co-director of the Paul F. Glenn Laboratories for the Biological Mechanisms of Aging at Harvard Medical School. He notes that MRIs taken of a teenager working through a problem show a lot of activity on one side of the prefrontal cortex, the region we use for conscious reasoning. In middle age, the other side of the brain begins to pitch in a little. In seniors, both sides of the brain share the task equally. The cooperative effort has a payoff. “Several studies suggest that seniors who can activate both sides of the brain actually do better on tasks, while those who can’t do worse,” Dr. Yankner says.
If you’ve found that it’s a little harder to carry on a conversation while searching your bag for your keys, MRI studies offer some clues. They show that in younger people, the area of the brain used to do a task goes dark immediately once the task is completed, while in older people it takes longer to shut down. As a result, it’s harder for the older brain to take on several tasks, because not only do you need to use more of the brain for any single task, but the brain also has a harder time letting go of a task. So even after you fish out your keys, you may have trouble getting back into the conversation.
What about the moments when you find yourself driving down the street without any recollection of having passed the last few blocks? Or the times you’ve locked the car door with your keys in the ignition? On those occasions your brain may have slipped into the default mode, which controls processes like remembering and daydreaming that are not required for a directed task. Imaging studies show that interconnected regions of the brain dubbed the “default network” grow more active with age, indicating that as we age we spend more time daydreaming.
THE GOOD NEWS ABOUT "THE MORE MATURE" BRAIN
At The Center for BrainHealth, as at Harvard, we are discovering that the more mature brain actually has advantages over its younger counterpart. These findings came as a surprise to many people, who were accustomed to seeing “senior moments”—groping for the right word or taking longer to articulate your thoughts—as a sign that the brain was on the skids. Yet even in professions where youth is valued, testing has shown that maturity has advantages.
For example, in a study of air-traffic controllers and airline pilots, those between ages 50 and 69 took longer than those under 50 to master new equipment, but once they had, they made fewer mistakes using it. (Keep this in mind when you’re trying to conquer a new computer program or adapt to a new car!) The mastery that comes with maturity is due to changes in your glands as well as your brain. Declining levels of testosterone—even in women—result in better impulse control. The end of the hormonal roller coaster of menopause may also contribute to emotional stability. After midlife, people are less likely to have emotional issues like mood swings and neuroses that interfere with cognitive function.
Most importantly, the wealth of knowledge from decades of learning and life experience enables you to better assess new situations. At midlife, most people are more adept at making financial decisions and getting to the heart of issues than they were when they were younger.
In most people, these abilities improve with age:
Inductive reasoning. Older people are less likely to rush to judgment and more likely to reach the right conclusion based on the information. This is an enormous help in everyday problem solving, from planning the most efficient way to do your errands to managing your staff at work.
Verbal abilities. In middle age, you continue to expand your vocabulary and hone your ability to express yourself.
Spatial reasoning. Remember those quizzes that require you to identify an object that has been turned around? You are likely to score better on them in your 50s and 60s than you did in your teens. And you may be better at some aspects of driving, too, because you are better able to assess the distance between your car and other objects on the road.
Basic math. You may be better at splitting the check and figuring the tip when you’re lunching with friends, simply because you’ve been doing it for so many years.
Accentuating the positive. The amygdala, the area of the brain that consolidates emotion and memory, is less responsive to negatively charged situations in older people than in younger ones, which may explain why studies have shown that people over 60 tend to brood less.
Attaining contentment. Years ago, researchers were surprised to find that people seem to be more satisfied with their lives as they age, despite the losses that accumulate with passing years. This is probably because they tend to minimize the negative, accept their limitations and use their experience to compensate for them, and set reasonable goals for the future. Dr. Yankner notes that this trait may be innate, because it is prevalent even in the United States and other Western nations, which tend to value youth over age.
Network Adequacy: Not Adequate
Deceptive insurance company practices
According to Mark Moran, writing in the April 3, 2015 issue of Psychiatric News, access to psychiatry appears to be an area in which many health plans are offering deceptive products. This problem has been studied for many years in Maryland, near the nation's capital, where access to care has long been a troubled business. Previous studies done over the last 25 years by MHAMD and other organizations found there were long delays for individuals to access psychiatric care. In 1988, MHAMD published, “Study of Mental Health Coverage Provided by Maryland HMOs.”
This study sought to provide a comprehensive picture of the impact of HMOs on access to mental health care. The anecdotes from mental health professionals in 1988 illustrated the long wait times their patients faced when trying to secure an appointment with a psychiatrist. As a follow-up, in 2002, the Mental Health Coalition of Maryland conducted a survey of mental health professionals to ascertain how the managed care system affected an individual’s ability to access mental health care.
Psychiatrists dropping our of private insurance networks
Many respondents reported dropping out of private insurance networks, resulting in more consumers having to pay out of pocket for mental health care. In 2007, the Maryland Psychological Association published a white paper titled, “Access to Care in the State of Maryland.” Their survey found that 44% of mental health professionals listed in the managed care networks were unreachable, and that the average wait time for an appointment with a psychiatrist was 25 days.
In the same April 2015 article, attorney Colleen Coyle, general counsel for the American Psychiatric Association, said that "in many instances health plans advertise provider networks that appear to offer users of behavioral health services a wide range of option for access to care, but in reality those networks may be much "thinner" than consumers are led to believe.
Insurance false advertising is fraud
"In my view, that is fraud," Coyle said. The insurer is promoting a provider list that it knows is not reflective of the access to care that patients will really have. I don't think it's innocent at all."
She goes on to say that insurance plans are constantly looking at their providers and and claims "because that's how they target which providers to audit and whether they are going to challenge the claim."
John McIntyre, M.D., a past president of the American Medical Association, said: "Many insurers, in an attempt to hold down costs, employ very narrow or shallow networks that are inadequate to provide necessary medical care. But patients don't know that when they sign up."
The detailed report from The Maryland Mental Health Association is available online here.
Best Therapists = Best Outcomes (by far)
Treatment: Know What to Ask
SEEKING TREATMENT: KNOW WHAT TO ASK
My goal in helping people includes educating them about what questions to ask.
Finding the right treatment for a person’s specific needs is critical. And finding the right treatment is not easy. Drug and alcohol addiction treatment is not “one size fits all.”
Treatment outcomes depend upon:
- the extent and nature of the person’s problems;
• the appropriateness of treatment; - the competence and skill of clinical staff;
• the availability of additional services; and
• the quality of interaction between the person and the treatment providers.
Family and friends play important roles in motivating people with drug problems to enter and remain in treatment. However, trying to identify the right treatment programs for a loved one can be a difficult process.
The National Institute on Drug Abuse (NIDA) has sound advice about the five questions to ask when searching for a treatment program:
1. Does the program use treatments backed by scientific evidence?
In the internet era, answering this question has become increasingly difficult. Many programs offer flashy "treatments" that are not scientific at all, despite claims made on beautiful web pages. According to Thomas McLellen, professor of psychiatry at the University of Pennsylvania, 90% of patients who enter addiction treatment programs don’t receive evidence-based treatment. Many current programs actually reject scientific evidence. For example, they prohibit the use of addiction medications, even though they’ve been shown to be more effective treating some addictions (specifically, the range of opiates like Oxycontin to heroin) than anything else. My concern is a bit different from Dr. McLellan's view (for example, he is not friendly toward 12 Step work). My major concern is that there are dozens of quack therapies, ranging from nutritional "cures" to "new age" approaches, to potentially harmful "trauma therapy."
A great website is not a guarantee that the advertised program is any good at all. Many programs today, using clever marketing, take advantage of opposition to Alcoholics Anonymous and 12 Step Programs by advertising that says, "We are NOT a 12 Step Program!" What these programs fail to do, in almost every example, is to state clearly what they actually do in their approach. Many other websites—and I have reviewed hundreds of them—claim to offer "evidence based treatment," when in fact the program is not evidence based at all. The average person has no idea whether or not claims of scientific evidence are true. Above all, beware of "testimonials" and celebrity endorsements. In selecting a treatment program, you have to use at least as much good judgment as you would use if you were buying a new car or having heart surgery.
Effective alcoholism and addiction treatments can include cognitive behavioral therapy, medications, or, ideally, the combination of both.
Key elements include:
• addressing a patient’s motivation to change;
• providing incentives to stop drinking or using drugs;
• building skills to resist alcohol/drug use;
• replacing addiction related activities with constructive and rewarding activities;
• improving problem-solving skills; and
• building better personal relationships.
Any and every good addiction treatment program will include competent assessment by a qualified addiction psychiatrist. Some programs will team an addiction medicine doctor with a well-qualified clinical psychologist, and that can work well. The point is that everything begins with getting the right assessment. Medications are an important part of treatment for many patients, especially when combined with counseling and other behavioral therapies. Different types of medications may be useful at different stages of treatment: to stop alcohol and drug abuse, to stay in treatment, and to avoid relapse.
2. Does the program tailor treatment to the needs of each patient?
No single treatment is right for everyone. The best treatment addresses a person’s various needs, not just his or her alcohol and drug abuse. Matching treatment settings, programs, and services to a person’s unique problems and level of need is key to his or her ultimate success in returning to a productive life. It is important for the treatment approach to be broad in scope, taking into account a person’s age, gender, ethnicity, and culture. The severity of addiction and previous efforts to stop using drugs can also influence a treatment approach.
The best programs provide a combination of therapies and other services to meet a patient’s needs. In addition to addiction treatment, a patient may require other medical services, family therapy, parenting support, job training, and social and legal services.
Finally, because addictive disorders and other mental disorders often occur together, a person with one of these conditions should be assessed for the other. And when these problems co-occur, treatment should address both (or all conditions), including use of medications, as appropriate.
Medical detoxification is a necessary first step in the treatment of certain addictions, but by itself does little to change long-term drug use.
3. Does the program adapt treatment as the patient’s needs change?
Individual treatment and service plans must be assessed and modified as needed to meet changing needs.
A person in treatment may require varying combinations of services during its course, including ongoing assessment. For instance, the program should build in drug monitoring so the treatment plan can be adjusted if relapse occurs. For most people, a continuing care approach provides the best results, with treatment level adapted to a person’s changing needs. A patient’s needs for support services, such as day care or transportation, should also be met during treatment.
4. Is the duration of treatment sufficient?
Remaining in treatment for the right period of time is critical. Appropriate time in treatment depends on the type and degree of a person’s problems and needs. People argue about this point all the time, and I don't have the patience or space on this blog to cite the references, but research tells us that most addicted people need at least three months in treatment to really reduce or stop their drug use and that longer treatment times result in better outcomes. The best programs will measure progress and suggest plans for maintaining recovery. Recovery from drug addiction is a long-term process that often requires several episodes of treatment and ongoing support from family or community. If you have read this far and are starving for the references, contact me.
Relapse does not mean treatment failure. The chronic nature of addiction means that relapsing to drug abuse is not only possible, but likely, similar to what happens with other chronic medical illnesses—such as diabetes, hypertension, and asthma—that have both physical and behavioral components. And like these illnesses, addiction also requires continual evaluation and treatment modification if necessary. A relapse to drug use indicates a need to re-instate or adjust treatment strategy; it does not mean treatment has failed.
5. How do 12-step or similar recovery programs fit into drug addiction treatment?
Self-help groups can complement and extend the effects of professional treatment. The most well-known programs are Alcoholics Anonymous (AA), Narcotics Anonymous (NA), and Cocaine Anonymous (CA), all of which are based on the 12-step model. This group therapy model draws on the social support offered by peer discussion to help promote and sustain drug-free lifestyles.
Most drug addiction treatment programs encourage patients to participate in supportive therapy during and after formal treatment. These groups offer an added layer of community-level social support to help people in recovery with abstinence and other healthy lifestyle goals.
To order NIDA materials, please go to: http://drugpubs.drugabuse.gov.