Trustworthy Robots in the AI Era
Times Have Changed...
The era of ROS1/2 is coming to an end. They were of vital importance back to the age when we needed to efficiently glue different modules and sensors together, therefore, we could stand all of its drawbacks such as intrinsic performance issues and lack of structural design. However, the situation of robotics research has changed since the renaissance of neural-network-based technologies.
Today's emphasis on robotics research has transferred to more complicated tasks and systems where AI infrastructures are usually involved. For researchers with a focus on the software part of robotics, it is no longer necessary to build customized robots. Customized robots are hard to be competitive compared to these successful commercial products, such as the legged robot Boston Dynamics Spot and the humanoid robot Unitree G1. The decreasing price trend of these commercial robot platforms has made these products affordable to more and more research labs, and they all provide their own software development kits that are not built upon ROS.
[Talk on International GameDev Conference, 2024]
Modern languages don't help solve real hard problems. ... People talk all the time about it's more important not to leak memory, so you got all these quite complex systems put into place to make sure you free your memory. ... For us, the much harder problem than freeing memory is to know how the game should behave and what this system should be like. ... What are the real problems? I don't understand my program, it's too big and complicated. Sometimes when I go to debug it, it takes too long, and I don't know what the problem is. Sometimes I have to write big systems that are a little bit error-prone because the languages don't have features to help me out.
As the talk of Jonathan, the same story happens in robotics: ROS1/2 do not help solve real hard problems of current robotics research focus.
For robot manufactures, the ability of ROS1/2 to adapt a wide range of sensors is not needed. Sensors and modules of their robot platforms have a limited number and range. The philosophy of ROS1/2 is to treat every robot system as designed for academic research, hence many underlying infrastructures are developed with a lack of consideration, which, of course, cannot meet the requirement for commercial products. As for these robot manufactures, they cannot benefit from the advantage of ROS1/2 to easily develop quick yet thoughtless prototypes, but they have to be legally responsible for internal bugs and security issues within ROS1/2. To gain full control over the whole system platform and also to better optimize the software for their hardware, these famous manufactures choose to gradually adapt to their exclusive SDKs after they have enough loyal customers. Despite the intention of tightly binding customers to their products, ROS1/2 cannot help robot manufactures with a commercial standard to meet.
For current researchers, the days of easily publishing papers on assembling robots have gone forever. Spending weeks on assembling a robot whose functions are provided by most commercial robot platforms will do no good for robotics researchers of software or algorithms. Also, the implicitness of support for messages which are usually predefined as broader than usually needed in third-party ROS1/2 nodes will cause more mental burden for researchers. There are still fierce debates about the pros and cons of distributed microservice structure, but it is widely acknowledged that a communication interface cannot guarantee microservices (or nodes) to be robust and reusable: it depends on how the whole system is split into these services (or nodes), and it is certain that distributed structure will take more tools such as log gathering and more effort to debug. ROS1/2 seems not to be helpful at all for the real difficulties in complicated systems, where AI infrastructures such complicated decision-making systems are used. And it only adds the difficulties to design and debug into the development-validation loops.
...But Not That Much.
It is undeniable that the technology researchers used in these days is much more advanced than previous; however, the ultimate problem that we try to solve remains still: to make robots more widely used in society. Robots have already embraced huge success in industry, and incoming success in academic. According to a government report of China, in the year of 2020, more than 212 thousand sets of industrial robots had been produced, and the robot application density had increased to 24.6 sets per thousand of people. People have witnessed the success of industrial robot arms, especially in the field of high-end automobile manufacturing, but no one can guarantee that it would be the same story for robots in other fields.
Overbuilding things the world doesn't have use for, or is not ready for, typically ends badly.
Both of the societies and robots are probably not ready for the large-scale deployment of autonomous robots in non-high-end fields. No one would argue against the future when we will have a tremendous number of robots working for us, but the only question is when. For developers who work with robot software systems, there is an eternal question: who will buy your robot (software systems) for what purpose? Unitree gives an answer: laboratories, for research. Here is a joke about this:
The current business of legged robots or humanoid robots is like a Ponzi scheme: many students rush into robotics, for they think there is a huge business opportunity awaiting them; and after they graduate and start their companies, the only customer is students who recently rushed into robotics.
This joke is not completely true, and judging from what is going on in China, where more than 5.461 billion CNY is publicly disclosed as invested in robotics since 2023, there are many investment scams. According to real cases happened among my friends, many people in the 'old money' class in China are recruiting oversea PhDs with high annual salaries to start companies quickly. Their only goal is not to develop useful robots but to go public as soon as possible. These oversea PhDs will be their CEO, or technical leader, but are actually used as golden brands to convince investors to buy their stocks. Their target is not the potential undeveloped market, but the money of their investors — if they cannot answer the question of 'who and for what', how can they see the market?
The situation has gradually gotten out of control. Here is an example. According to recent news, a 23-year-old PhD candidate young boy created a robot company U***x AI (name is hidden). He is a second-year PhD student, and there are still three years left for him to graduate. His research focus is visual-touch in his bachelor and PhD study, but his company is manufacturing whole robots. Its 100% capacity of 10 million CNY comes from the company Bo***e Intelligent Manufacturing (founded in June 2023), whose 100% of capacity comes from the company An***n Company Management (founded in July 2022), whose boss is also a member of the board of directors of U***x, Bo***e, and another fully controlled company Ag***a Intelligent Manufacturing (founded in October 2023).
How can this affect researchers and developers? The most direct influence is that after this large-scale farce is subsided, researchers or developers will have much more difficulty getting investment for macro goals; they must specifically answer the eternal question of 'who will buy your robots and for what purpose'.
Who and for What
The essence of the robot’s mission is to replace humans. This sounds terrifying to irrelevant people. However, what is more terrifying to some researchers is the doubt that whether their robots are capable and competitive enough to substitute humans, and for now, the answer is usually a stuttering and hesitant yet undeniable no.
Household service robot is a huge potential market, but the fact is that it is unreachable in recent years. Everyone would like a robot to do the housework and cook delicious meals for them, but no one would expect a robot, which may fall down if it steps on children's toy and could possibly hit into furniture and break the glass of television while it is trying to restore its balance; nor a robot which may drop vegetables on the ground or get on fire as it is trying to cook. This story sounds pessimistic, but probably it is an optimistic expectation for robots soon.
Despite functions, there is also a vital factor that cannot be ignored: price. Months ago, a visitor to our laboratory summarized the goal of their laboratory as 'making robots faster, higher, and stronger.' I personally held a negative attitude towards this kind of 'Robot Olympics'. Even though many robotics researchers disdain engineering, but robotics itself is actually a subject of engineering rather than science: if they seriously consider conventional science, they would find that there is no 'undiscovered ultimate truth of the universe' within robots, which are fully artificial creations, designed, built, and improved by humans. To think about this, even mathematics is not categorized as 'science'. As a subject of engineering, customers nor the society cannot really benefit from 'Robot Olympics'. Because, if we learned something from the tragic failure of Concorde supersonic airplane, then we would know: engineering is about weighing the pros and cons, and it is not enough to get accepted by the public for just being 'faster'; customers always care about more aspects, such as price.
After taking the factor of price into consideration, currently there are not too many areas where robots are competitive enough compared to human labor. Even though the price of robots is keeping going down, there are many geographic areas where human labor is at a much lower price. An example is Guangzhou, China, where workers in electronic assembling factories usually are paid 0.98~1.96 USD (about 140~280 JPY) per hour. It is nearly impossible for robots to be efficient and inexpensive enough to function at such a low price soon. Therefore, robots can only embrace success in geographic areas or business areas with high enough salaries.
Though human labor is cheap somewhere, but human lives are expensive everywhere. A soldier in Russia has a month salary of 1,402 USD and gets compensation money of 137,030 USD if he or she sacrificed for duty; meanwhile, the whole-procedure price of an Unitree legged robot Go2 is 1,403 USD, which is only the salary of a month for a human soldier. Even if we skip the discussion in these sensitive areas, people will not be hesitated to pay for affordable things which can truly guarantee their safety, especially in areas with a special social situation, such as America. Currently, there are already attempts of selling robots as police equipment. However, that is more like a financial support from the local government to the young robot companies whose orders have a characteristic of 'low demand on numbers and high price per robot'. In contrast, in rural areas, a large number of families are willing to replace 'guns under pillows' with security robots in the yard, no matter if they are legged or humanoids, only if their prices are affordable. And the functions required for robots are just patrolling, identifying, alarming, and using non-lethal equipment such as pepper spray, or simply calling the police, which are already proved as fully doable in laboratories.
In conclusion, there are profitable yet untouched areas awaiting challengers in robotics, even for the technology level of today.
Overlooked Reliability.
It takes 10 years of Baidu to put autonomous driving from laboratories to society as Apollo Autonomous-Driving Taxi. However, for now, these robot taxis still cannot understand the commands of point-policemen and cause chaos when they encounter bicycle riders who do not obey traffic rules. Usually, we do not see too much research applied in society
because making these systems resilient enough to function under a social context usually requires more effort than that to develop them. The reliability of robots is their ability to perform proper and necessary operations to handle unexpected incidents that hinder them from carrying out their tasks, especially when they are stuck in unrecoverable states. In the past decades, researchers have put enough efforts on recovering robot states, yet there is a usually ignored principle: the solution should not be more harmful than the problem itself. While handling unexpected incidents, the solutions taken by robots should also be predictable and in line with a minimal subset of common sense. At least the solutions should not be against these laws:
- Law 1: A robot may not injure a human being or, through inaction, allow a human being to come to harm.
- Law 2: A robot must obey the orders given to it by human beings except where such orders would conflict with the Law 1.
- Law 3: A robot must protect its own existence as long as such protection does not conflict with the Law 1 or Law 2.
Indeed, they are Asimov's Three Laws of Robotics. It is very intuitive that trustworthy robots should obey these laws. Actually, there can be more local laws made by their owners, but these mentioned are fundamental laws for the whole society. Sadly, current robot systems cannot guarantee to obey these laws; and what is sadder is the fact that judging from the topics of published papers, robot researchers are not interested in this direction in recent years.
In conclusion, which sounds awful, current robots are not trustworthy, and most of the current robots researchers do not mean to make them trustworthy. On November 8th 2023, a 40-year-old worker in South Korea was crushed by an industrial robot arm, which confused him as a box of vegetables and crushed his face and chest against the conveyor belt. This incident did not affect the operation of the company who developed this robot system, but that night was surely a long night for the family awaiting this worker at home; and at that night, many researchers were still too busy with their 'Robot Olympics', researching on how to make their robots 'faster, higher, and stronger' (to injure people). It is fair that we cannot expect too much from them, for they cannot easily publish papers on the topic of trustworthy robots (or caring about 'human lives'). Reliability is a vital yet ignored prerequisite.
AI Makes Robots More Unreliable.
Probability-theory-based neural networks cannot guarantee the same output for the same input. Some people may expect to use AI to regulate robots, but according to the behavior of current LLMs and other neural network-based AI services (such as object detection), AI is also a nondeterministic factor. On July 18th, news was reported that LLMs including ChatGPT-4o, Gemini Advanced, Claude 3.5, and more, could not give the correct answer of 'which is bigger, 9.9, or 9.11'. And someone also noticed that if positions of these two numbers are switched, then these LLMs could output the correct answer. This uncovered the disappointing fact that LLMs only recognized these two numbers as two tokens and never learned to do math. There is a bad trend of worshiping LLMs as some kind of 'digital gods', and regard any research where LLMs are involved as only a wrapper of LLMs based on the false belief that LLMs have unleashed all the magic. In fact, according to my experiment, even the 'common sense' within LLMs is not always as stable and correct as we usually believe; levels of details of the answers vary according to the language that the question is asked in; and sometimes they are confident to forgery answers to questions they do not understand. Many of the researchers take this as 'natural and normal', yet it is 'unnatural and abnormal' to see them still worshiping LLMs as 'digital gods' who know everything and can do everything right.
Therefore, when it comes to AI (especially LLM) embedded robot systems, more measures must be taken to ensure their reliability. These systems should not only obey Asimov's Three Laws of Robotics, but also ignore the commands from the AI which violate these laws.