If you thought HAL 9000 was scary, wait until you meet the robot customer service AI at SQUARE. Their mutual lack of humanity makes both of them stunning examples of badly programmed systems.
HAL 9000 is a fictional artificial intelligence character and the main antagonist in Arthur C. Clarke’s Space Odyssey series. First appearing in the 1968 film 2001: A Space Odyssey, HAL (Heuristically programmed ALgorithmic computer) is a sentient artificial general intelligence computer that controls the systems of the Discovery One spacecraft and interacts with the ship’s astronaut crew.
Both of them are examples of simplistic money-saving solutions with no access to human override. This kind of robot “intelligence” fails spectacularly when presented with data that does not meet the guidelines set by programmers with limited real-life experience.
The robot customer service system at Square was installed to save money on human customer service staff and it does everything possible to keep you from contacting a live human being. It has a low tolerance for input data errors and a simple error in entering a CVV code can lock up your funds and mark you as a high-risk customer.
Once the Square Customer Service System has marked you as a high-risk customer, it effectively screens you from contacting a live customer service agent by presenting you with a canned set of recordings when you call for help. Text messages sent to the help center get canned replies that refer you to emails sent previously and are no longer available.
I went to Square in mid-October looking for an invoicing and virtual terminal service to replace the one I have been using for the past ten years. I ran test transactions through their system to see how reliable the system was, but I did not test it thoroughly enough. I invoiced a customer and that transaction was completed with no problem. I used the virtual terminal system for another customer and the transaction was denied twice with no reason for the error message. (The customer had given the CVV code in the wrong order.) When I entered the correct CVV code the transaction went through, but Square froze my funds and indicated I was a high-risk customer. Square has indicated that my money will be sent to my bank account, but I am unable to get a date for this transfer.
If you enjoy interacting with a robot system that can freeze your account and deny access to a human counterpart, Square will give you that experience in spades. You can relive the experience of astronaut Dave being locked out of the spaceship airlock with nobody to hear you plead for help.
I have the feeling that I am not the only customer who has been locked out of the Square airlock because the system they have erected to keep customers from contacting real humans is carefully designed to keep us from registering complaints with responsible people working there.
The one human contact I made in the early stages of this catastrophe was a soft-spoken person of foreign extract who apologized because he could not help me straighten this matter out by simply checking with my customer and verifying the transaction. He said I would have to prove to the system that I was a real corporation.
Unfortunately, the system has already given me its answer, “I’m sorry Dave, I’m afraid I can’t do that”
If you wish to trust your business to the Square version of HAL, have a backup plan ready. You will need it.
If you would like to see what an AI expert says about these systems, see this presentation:
The Real Reason to be Afraid of Artificial Intelligence | Peter Haas | TEDxDirigo