The capabilities of the artificial intelligence of an unmanned vehicle on the roads approached human ones, although in some cases insignificant, but surpass them, a study by Cognitive Technologies showed.
As RIA Novosti was told in the company, the purpose of the research was to determine who at the current stage of creating unmanned vehicles better copes with the task of understanding individual traffic situations – artificial intelligence (AI) or natural (EI) – that is, a person. “In other words: does the man’s artificial mind win already in the problem of control over the drone or not?”, The company noted.
Cognitive Technologies specialists compared the capabilities that AI has already achieved. In the tests to assess the quality and speed of detection (detection, identification – ed.) Road signs, cars and pedestrians conducted on public roads in Moscow and the Moscow region, 17 volunteers participated.
A number of tests took place in difficult weather conditions (dark time, rain), tests were also conducted in such difficult conditions for recognition as the transition from light to shadow and blinding by the sun. The volunteer, in accordance with the task, sounded certain objects of the road scene, noticed by him, in parallel recording of the results of recognition of the AI of the same line of objects. As a result of the trip, the data was processed and entered into a general test table.
The research was carried out in September – November 2017, while one video camera (mono camera) was used.
Tests were prepared in such a way that the number of objects that appear simultaneously did not exceed three, since otherwise it would be technically difficult for a person to voice the results of a detection. On each route, a person was given not more than 30-35 objects to recognize.
In total, testing was conducted on 27 different routes in settlements at speeds of 50-60 kilometers per hour.
Robot vs. Human
The study showed that the possibilities of artificial intelligence are already close to natural, although in some cases the robot “defeated” a man, albeit with a slight margin.
The first result of the study – determining the quality of the detection of objects in the road scene in good weather and road conditions – has become approximately equal in the results of AI and AI, both in terms of speed and accuracy. According to the latest indicator for road signs, cars and pedestrians, they exceeded 99%.
AI showed itself better in tests when individual objects were not completely visible (behind trees parked by cars and other obstacles).
And the advantage of AI was not in the quality of recognition, but in the speed of detection. For example, in tests on road signs, AI was able to see partly obscured objects for a fraction of a second. “The test showed that under more difficult conditions, the subjects (volunteers – ed.) Often noticed road signs for a moment later than the AI does, but this time gives the control system an additional advantage for processing and analyzing information about the traffic situation as a whole,” – said the head of the department of unmanned vehicles Cognitive Technologies Yuri Minkin.
The exact time of the reaction of AI and human under the conditions of this experiment was not fixed, its value fluctuated from about 0.5 seconds and did not exceed 1 second. The accuracy of recognition of EI and AI in these tests exceeded 99%.
In tests with partially blocked cars and pedestrians, the advantage of AI in speed was preserved, but it could already be considered as a statistical error (on the order of 0.5 seconds or less). The accuracy of recognition of EI and AI was also more than 99%.
A slight advantage of AI in the speed and quality of detection was also noted in the rain conditions when recognizing traffic signs – it was 98.3% against 97% in the EI. When recognizing cars, the results were approximately equal (of the order of 98% with a difference of less than 0.5% in favor of AI). “Such a result can be explained by the greater stability of the AI for such interference when detecting objects of relatively small dimensions,” the company explained.
In addition, AI won in border situations: in situations of transition from light to shadow and blindness, AI more quickly coped with the task of recognition. The reaction time in humans was greater in almost all tests, but did not exceed 1 second. The accuracy of recognition of both EI and AI was more than 98%.
The tests showed that the person did a little better job of recognizing pedestrians in difficult conditions (night, rain). The accuracy of the EI recognition was approximately 99.2% versus 98% for AI at night and in rain. “A person is a rather complex object for recognition: pedestrians do not have a permanent form, they can walk, embracing, holding hands, carrying a load, etc. And if in good conditions AI has not yielded to a person, then in difficult conditions, were slightly better, “Minkin said.
conclusions
The main result of the study was the approximation of AI capabilities to the EI. Moreover, as the test results showed, the advantage of AI was manifested in the complication of road and weather conditions, the company noted. “It is to be expected that with the increase in the computing power, as well as the quality of the sensors and software, the advantage of AI will become obvious, as it was in chess,” said a representative of Cognitive Technologies.
“Of course, many aspects of the organization and conduct of the research can not claim perfection, and we will work together with the expert community to develop both the methodology and the tests themselves, but it must be understood that this is the first attempt in history to compare AI and human capabilities. in the first approximation we got real results that can and should be taken into account, on the basis of which we can predict the development of the direction of creating AI for autobots, “said Olga Uskova, president of Cognitive Technologies.
According to her, the company expects to conduct the following tests, analyzing the quality of understanding of various traffic situations by man and AI.
The company noted that initially it was not intended to compare the capabilities of artificial intelligence and human.
“We initially did not intend to compare the capabilities of a human being and AI, we had plans to test our own AI system for C-Pilot unmanned vehicles.” The idea of comparing AI with a person came already in the course of testing, but we got excited with this idea, it became very interesting for us to get evaluation of the comparison of artificial intelligence with humans, “said Yuri Minkin, head of the department of unmanned vehicles Cognitive Technologies.
