PC Beats Lawyers in Legal Prediction Contest

Computers can beat lawyers at predicting the outcome of cases according to a recent test. Organizers say an artificial intelligence program might even have uncovered insight into what influences case decisions.

The test involved a computer program that has proven more sophisticated than intended. "Case Cruncher" was originally designed as a chatbot - a program that simulates a text conversation with a user - that would answer questions about the law. It was created and 'taught' by four law students in the UK.

The team later developed it into a tool called Case Cruncher. Each variant of Case Cruncher is custom designed for a client and deals with a specific problem or issue, with the idea being to save time in analyzing legal options.

Computer 'Out-Predicts' 100 Lawyers

To test its effectiveness, it challenged 100 lawyers to a week-long contest. Both the humans and Case Cruncher were given facts about real cases submitted to the Financial Ombudsman and asked to guess what the outcome was.

The Financial Ombudsman is independent organization in the UK that investigates complaints about the behavior of financial companies. Although it isn't a court, its decisions have legal force and can order companies to pay compensation. The cases all covered the same subject: alleged mis-selling of payment protection insurance (PPI), a financial product that used to be offered to credit card and loan customers.

In total, the lawyers made a total of 775 predictions, of which 66.3 percent proved correct. Assessing the same cases, the computer was correct 86.6 percent of the time. (Source: bbc.co.uk)

Non-Legal Factors Could Be Key

There are some reasons the computer system had an advantage with this specific test. The subject is relatively simple in legal terms, with established guidelines for the Financial Ombudsman to follow when making a decision. Meanwhile, the lawyers were experts in commercial law, but weren't specifically experienced in the narrow topic of PPI claims.

That aside, it may be that the system picked up on unexpected patterns in the real case decisions. Case Cruncher said that "this experiment also suggests that there may be aspects other than legal factors contributing to the outcome of cases. Further research is necessary to establish this proposition beyond the specific parameters of this experiment." (Source: case-crunch.com)

What's Your Opinion?

Do you think this was a fair test? Would you expect the technology to work with wider legal issues? Would it be valuable for consumers and lawyers to get a quick and affordable prediction of whether taking legal action in a specific case would be likely to succeed?