‘Science in Humanities’ with Platon Academy③ SNU Prof. Dongjun Lee on Drones

“China’s DJI Has Monopoly on Drone Market … 7,000 Researchers are Working Day and Night”

● American drone that killed Iranian commander is an impressive technological feat
● ‘Drone boom’ made possible by breakthroughs in batteries and material
● Rules of the game must be changed to beat DJI
● A falling propeller can lead to a devastating accident
● Drone delivery? Virtually impossible in Seoul

_{<Editor’s note> Shindonga, partnered with Platon Academy, will be publishing a series titled ‘Science in Humanities.’ Founded in November 2010, Platon Academy is the first foundation in Korea to exclusively support the humanities. Their projects include research funding, public lectures, and online websites. The series aims to bridge the gap between the humanities and sciences which, upon first glance, seem to be standing at opposite ends of a spectrum, but in fact are deeply interrelated in our lives.}

[by Do-kyun Kim]

On January 3^rd, the deadly power of drones surprised the world. A United States drone strike killed Major General Qasem Soleimani of Iran’s Islamic Revolutionary Guard Corps at the Baghdad Airport in Iraq.

What exactly are drones? How far will they advance?

Professor Dongjun Lee (47) of SNU Dept. of Mechanical and Aerospace Engineering is a drone expert. He received his Bachelor’s and Master’s degree in Mechanical Engineering from KAIST. After working at Kia Motors, he went on to receive his Ph.D. from the University of Illinois, then taught at the University of Tennessee for five years. Prof. Lee has been teaching at SNU College of Engineering since 2011.

He is a recipient of the 2009 CAREER Award, conferred by the U.S. National Science Foundation. He has also served as Associate Editor for the international robotics journal T-RO. The interview took place on January 30^th in his lab at SNU College of Engineering.

January 13th (local time), American military personnel and journalists stand next to the damage at the Ayn al Asad AirBase in Iraq. It went under an Iranian missile strike on January 8th, which was in response to the killing of Qasem Soleimani.

- The dictionary definition of the word ‘drone’ is ‘a low humming sound,’ or ‘to make a low humming sound’ if used as a verb. Is it true that the word’s usage as a proper noun originated from military technology?

“Yes. The Cambridge Dictionary defines a drone as ‘an aircraft that does not have a pilot but is controlled by someone on the ground, used especially for dropping bombs or for surveillance.’ Then, as drones became commercialized, they added another definition about 5 years ago: ‘an aircraft used especially as a hobby.’

Simply put, a drone is a ‘flying robot.’ It has two types—military and civilian, depending on its purpose. 90% of drones that currently exist are for military purposes.”

- I was very shocked to see a drone killing a person.  

“Actually, military drones have a history of over 100 years. The electrical engineer Nikola Tesla invented a remote-controlled airplane in 1918, the world’s first drone. They were put to use in World War II. In the 1980s, with dramatic technological advancements in avionics and computers, drone using satellite technology began to be produced.

The US is the biggest investor in developing and using military drones. They completed the technology during the Gulf War in 1990, and have been implementing drones in military operations since 2001. The drone that killed the Iranian commander is called the ‘MQ-9 Reaper,’ a next-generation military drone. It’s also a fixed wing drone, which is different from multirotor drones typically used as a hobby.”

“The basic technology used in MQ-9 is not that groundbreaking,” he says. “but the scary thing is that it was unable to be detected by sound or radar.”

“As per the dictionary definition, drones are extremely loud. The Reaper drone seem to have been flying at 200m above the ground with the wings being 20m wide. If it was a helicopter, we could have certainly heard the noise from the ground. If you lower the sound of a drone, its power also decreases, so it can’t fly or hover in the air. So they must have used a very quiet engine, so quiet that the Iranian military couldn’t detect. What’s also amazing is the technology of ‘Hellfire R9X,’ a guided missile equipped on Reaper. Once it reaches the target, a total of 6 blades pop out and completely rips apart the target. The fact that the missile was able to distinguish between the driver’s seat and the passenger seat proves how superior its sensing and control technology is.”

- So there are military and civilian drones. Do you also categorize them by their shape or form?

“Yes, depending on whether the wings are fixed or not. Airplane drones like Reaper are called fixed drones. They are powered by propellers or engines. Civilian drones are usually powered by rotors (motor + propeller), but more recent ones have a turbine engine so they shoot off with a blast of fire.

Drones must be able to withstand wind and gravity. The fixed drone can fly at a fast speed, and at much longer distances. The one that flew from North Korea a few years ago was also a fixed drone. But the fixed drone cannot hover in the air. If you want to take surveillance photos with it, the drone must be able to hover in the air at a certain spot and not move around. The fixed drone can’t do that. If its speed decreases, it falls. So it can’t work as a camera drone.

One of the types that is able to hover is the helicopter drone. But these are harder to control during takeoff and flight. Also, since their propeller is attached to the fuselage, the rotating part is weak and it’s difficult to salvage after a fall. And the propeller rotates at a slower speed, so the fuselage cannot absorb the vibration, shaking the whole thing, which isn’t helpful when you’re taking photos. This is why remote-controlled airplanes and helicopters have not been widely used, even though they’ve been around for a while.”

- Who invented the multirotor drone with 4 propellers?

“The first commercialized product was made by a French company named Parrot. After China’s DJI, they are the second largest commercial drone company. Since the early 2000s, France has been researching drones as a national research project. At first, it was regarded as an expensive toy, but Parrot’s first product was called ‘AR.Drone.’ So they are already thinking of augmented reality. Augmented reality is like when you’re wearing something like Google Glass, you look at a table, and an image appears on the table. They’re really onto something, implementing AR on drones.”

- Drones do look like a toy. How did they become a symbol of the 4^th Industrial Revolution?

“There are a variety of advanced technologies in a drone. As different fields reached breakthroughs, drone technology made a leap as well. The batteries, for example. With the development of lithium-ion batteries, now commonly used in mobile phones, the weight of the drone came to be much lighter than before. But the most important breakthrough was the material.”

- Meaning?

“In order to fly, the drone must be light and stable, it can’t be deformed by the force of the wind. The development of carbon fibers made it possible or drones to hover in the air without vibrations. It was incredibly expensive just 10 years ago, but now it costs only a couple thousand won per drone. There are several other technological developments that made the drone technology the way it is today: high efficiency and durability of the motor, the small, inexpensive navigation sensor, manufacturing technologies such as laser cutting which allow a variation in drone shapes. In addition, we also have AI technology, as well as miniaturization of inertia sensors, cameras, GPS, and other computing technology. Every bit of technology used for the mobile phone also goes into the drone.”


A video of drone delivery by Amazon. [YouTube]

- There’s a lot of fields that drones can be used. Aerial photography seems to be the principal usage.

“Indeed, the most successful application so far has been aerial photography, whether it’s a photo or a video. It’s also used in surveillance. The human eye captures a lot of data in a very vivid way. Camera drones are like an extension of that.”

- There’s also drone delivery. I’ve watched a video made by Amazon.

“You shouldn’t believe all the videos you see on YouTube (laughs). It’s not just Amazon, there’s a lot of demo videos around the Internet, but it’s fairly easy to tell which is fake and which is real even with a basic understanding of drone technology. One of the drone videos on YouTube features a selfie drone, which is the size of a mobile phone. It’s fake. In reality, it can’t fly at all or even if it does take off, it will fall to the ground in a minute. The aluminum material too heavy and the propeller is too small. When the propeller is twice as small, its power decreases to just one eighth of the initial magnitude.”

- Why are there so many fake videos? To attract investors?

“That’s right – which is why investors need to understand the technology. A lot of robot videos are fake. They’re like, we can make a video of this first then fully develop it later. I’m sure most of them have good intentions, but it doesn’t hurt to be careful when there’s money involved. Us researchers have a hard time with all these exaggerated technological depictions going around. It’s very difficult to design and actually implement a technology, but when we say we want to do this and this, the typical reaction is ‘That’s already on YouTube.’ But if you take a closer look, most of what you see is not actual technology that can be utilized.”

He proceeds to show a video of a firefighter drone on YouTube.

“This is a video of a firefighter drone approaching the burning building horizontally. But a drone that large can’t actually approach the building in a straight line. All the rotors are facing up, so the body of the drone has to be tilted in order to move sideways. But the windows might break, which is dangerous. It’s easy to be fooled if you don’t know the basic principles of how a drone moves.”

- Let’s go back to drone delivery.

“We have a long way to go before that happens. There’s one common thing in all the YouTube videos. Do you know what that is?”

“The demonstration takes place in an open area. No obstacles, no barriers. It might be possible to deliver light documents or small packages in a place like the US where the houses are far apart with gardens in between. But drone delivery in urban areas is highly unlikely. Especially in Seoul. It’d be virtually impossible.”

- That’s surprising, since we’ve been hearing that drone deliveries will be a reality in just a few years. Why is it especially more difficult in Seoul?

“It’s a simple problem but extremely difficult to solve: power lines of telephone poles. The camera of a drone can’t detect them. The camera sensors are not as good as the human eye. The human eye perceives the surroundings as a panorama, but the camera is limited by the resolution and screen size. Currently, the best high-performance camera out there can capture 20 images per second in a 600x400 screen, but can’t detect anything smaller that this pixel size. Power lines are very thin. You can’t distinguish it within a single pixel. Making a database is also difficult. So drones are bound to fly into them. Drones can’t also detect them when it’s dark or the color of the power lines are similar to that of the background.”

- Can’t we set up a headquarter to control them?

“For military drones, absolutely. But it’d cost too much for individuals to control every single delivery drone like that. Much better to have people do it. Remote control is essentially done via screens looking through a camera, which would not provide a rich enough visual data like the human eye in an urban environment.

Another issue we have is localization. For autonomous flight, the drone needs to know its current location and where it must go for deliveries. This is localization. It would need a GPS-based location setting. But cities like Seoul are filled with high-rise buildings, which interrupt satellite signals, so GPS doesn’t work. You get the same problem when you’re in a narrow alley or in the mountains and your navigation doesn’t work.

Safety is an important issue. The skyscraper wind effect could disrupt the drone. Interruptions in the GPS signal could cause it to slam into a building, or fall on passersby and cars.

Because of its propellers, a drone falling can lead to a critical accident. It can rotate at a speed of 6000RPM, which is the same speed as a car engine rotating.

The propeller of a drone is very thin and solid, almost like a blade. When that falls, you don’t get hit. You get cut. Drones can cause serious injuries, you must be very careful. It’s why we have regulations banning flying drones in crowded areas. Researchers all wear helmets in a lab, too. There’s also the noise problem. The intel drone show at the PyeongChang Olympics took place outside, so that was okay, but having drones flying around neighborhoods would be hugely irritating.”

‘Mavic Mini’ created by China’s DJI. [Newsis]

- Let’s talk about the market. The global market for commercial drones is dominated by the Chinese company, DJI. They account for 70% of the total market.

“They are so, so far ahead. Even American companies are falling behind in technology and scale. Practically all the commercial drones and their parts are manufactured by DJI. Their new products show an impressive rate of technological advancement. The drones get smaller and can fly for a longer amount of time. The recent ‘Mavic Mini’ is the smallest in the market (245g, excluding the battery) and can fly for about 30 minutes (previously, 15 minutes) with a fully charged battery. You fold all the propellers, you can hold it in your palm. It means they are using a control technology that is extremely precise. And it’s not just hardware like electric motors and gimbals for cameras. The US and Europe used to lead software technology, but DJI is way ahead of them in terms of sensor combination and control algorithms. Every new product they put out is equipped with new high-level control technology.”

- What was behind DJI’s rise to success?

“They got a head start, and the founder Frank Wang was very committed to it, but also the timing was fortunate. Like I said before, breakthroughs in material and battery made possible what used to be impossible just 10 years ago. They also had a lot of money for investments. It seems like no other company in the world will be able to beat DJI for the time being in camera drones.”

- Why is that? 

“In 2015, I had a chance to attend a seminar held at the DJI headquarters in Shenzhen, China. The average age of their engineers was 24. The main engineer, a position that corresponds to head of department in Korea, was in their late 20s and had 50 employees under them. Now, five years later, that person would be in their 30s, supervising on sites and leading research projects. And they would be working for at least 20 years, 30 years. The number of employees and depth of their research are unmatched. DJI has a total of about 7000 employees, while Korean companies have under 30. They have 1500 employees just for R&D, 50 of them conducting research solely on control technology. The quality of their research is impressive as well. In 2015, the topic of the seminar was about a new drone platform, and one of the field engineers asked me an in-depth question that I’d never been asked before. I was shocked.”

“DJI’s management style is like a mix of capitalism and communism,” he adds.

“Layoffs are very common there. It’s very unstable for the employees, so they have intense competition. They just keep putting out new product after product. It’s almost like they don’t have a concept of marketing. Even when the product they currently have is selling very well, they instantly put out a new product. I hear it’s due to internal competition between different teams.

From Phantom 1 in 2013 till Phantom 4, they’ve shown a dramatic progress in technology, and just 6 months ago they released Mavic Mini with half the price and advanced technology. Releasing a new product into the market that’s already dominated by another one of your products is, from a business perspective, a crazy move (laughs). The quality of their drone parts are about 20% better than those of other companies—which is a huge difference.

My students have told me that they feel like they’re unboxing an Apple product when they open up a DJI product. The design is very sophisticated and detailed as well.”

- It’s almost scary. 

“I sound like a brand ambassador for DJI (laughs), but I simply wanted to illustrate just how intense the global competition is. Another thing that surprised me in their labs was that, on top of everything, they even work hard. Everyone was so concentrated on coding, with a cot next to them. It looked like they didn’t go home and just stayed in the labs to work.”

- How far has the drone technology come?

“Recently, drones are being developed that can be flown indoors where GPS doesn’t work. This is another big progress. Not just indoors, but in places like the mountains as well. So when following a person in such places, they decide where to follow and where not to, and they learn to do so via deep learning technology. As you know, the perception field in AI technology is exceptionally advanced. You walk, disappear behind a tree, come out and walk again—and the drone will not miss you. When you hide behind a tree, it hovers and continues following once you show up.”

- How is that possible?

“The drone runs a ton of simulations with a variety of data on human movements. It decides, based on probability, when the person would move.”



- What if the person jumps out of an unexpected place?

“That’s also resolved by probability. When the person jumps out from a totally unexpected direction, it’s too different from the location calculated from its simulation so it just disregards the data. It sounds easy, but the AI technology needs to be quite advanced. There’s a lot of open-source materials for drones on the Internet. In Europe and the US, teenagers can create drones (laughs). But what distinguishes between an amateur and a professional is not whether the drone flies, but the algorithm technology. The most important factor is sensor combination. How to extract best information by combining a variety of sensors in the most optimal way. Drones these days do not have much difference between control, propellers, or motors; their performance depends on sensors and perception technology. Naturally, you can’t develop drones by yourself. You need to collaborate.”

Prof. Lee offers an example of agricultural drones which are currently commercialized.

“The agricultural drone must address 2 problems. First, it needs to distinguish between the owner’s land and someone else’s land. It should also be easy to control. And since it’s not a clean environment, it needs to automatically clean up any dirt that sticks to the camera.

For the agricultural drone, it can likely cause errors if only the GPS is used. There needs to be other sensors. If we use an inertia sensor called IMU(Inertial measurement unit) and a compass together, we can detect not only the acceleration but also the angular velocity of where it’s about to shift directions. Combining this with GPS leads to higher accuracy. With this kind of locating technology, you can get a full view of your entire field, then you press the button of your controller and your drone will automatically fly over the field, spraying pesticides and whatnot, covering the whole area. Currently the best agricultural drones are also coming out of DJI.

Drones are also used for maintenance of sewer pipes or dams, and for safety inspections of places that are difficult for people to approach, such as the top of a bridge. In these cases, we can’t use GPS. Instead we attach the LIDAR sensor, commonly used in self-driving automobiles, to the drone.”

He mentions that “In order to beat DJI, the rules of the game have to be changed altogether.”

- What does that mean? 

“‘Aerial manipulation’ is one of such research areas. If the camera drone is an extension of the human eye, the aerial manipulation drone is an extension of the human hand. Performing tasks at large bridges or high-voltage transmission towers is immensely difficult for humans, but it could be done by a drone. There’s still a long way to go. The fundamental limit of drones is their flight time. Even the best battery can only last up to 30 minutes at maximum. You may wonder why the drone can’t just fly over there and get it done quick, but it doesn’t work like that. You need to perform the task by looking at the camera feed from the drone, but you get a limited view and it’s quite confusing. The drone may fail to move according to your commands. It takes a lot of time. There are also unpredictable factors such as the wind. When the wind blows, only the propellers of the drone must be moving while the body keeps still, which is difficult with current technology. Since the rotors face up, the drone has to be tilted sideways when there’s wind blowing. We have to come up with a completely new design, something that is drastically different. Areas such as aerial manipulation and AR technology are the ones that have the potential to change the rules of the game.”

Shindonga, Mar 2020

By Moon-myeong Huh angelhuh@donga.com

Link to article

Shindonga https://shindonga.donga.com/3/all/13/1994414/1

Naver Tech https://m.post.naver.com/viewer/postView.nhn?volumeNo=27648870&memberNo=39094895&navigationType=push