In Which a Goal was Set

Youth

I've had interest in artificial intelligence since I was a wee lad, from science fiction books to TV series to movies. It opened up my imagination to worlds that could be, populated by strange and fascinating and terrible beings barely on the edge of my comprehension.

____________________________________________________________

Books 

Issac Asimov's Robot series

I, Robot, a collection of short stories by Issac Asimov

In Issac Asimov's Robot series (Wikipedia link, Goodreads link), robots were governed by three simple laws of robotics:

• A robot may not injure a human being or, through inaction, allow a human being to come to harm.
• A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
• A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

But these simple laws interacted with the situation that they were in to create interesting consequences, such as when two laws conflicted with one another, or when the laws created unintended consequences, or eventually when the robots outgrew the three laws and created their own law... though that part is a bit of a spoiler.

William Gibson's Sprawl trilogy

The first novel of the Sprawl trilogy

William Gibson's Sprawl trilogy (Wikipedia link, Goodreads link) described a dystopian future where artificial intelligences with their own agendas used their agents to rage battles across cyberspace and meatspace, and were terrible and terrifying creatures which moved beyond human understanding.

____________________________________________________________

Youth

The idea of a computer that is capable of thinking at a human level fascinated me, both because of the alienness of it (what strange concepts and thought processes it must have, with values systems completely foreign, perhaps even horrifying to humans) and the familiarity of it (how nice it would be to have another kind of consciousness, another type of sentience, with which humanity can use as a mirror).

____________________________________________________________

Movies

The Tron series

Computer programs brought to life

The Tron series (Wikipedia link, IMDB link) showed us a digital world inside a computer, where software programs acted according to their programming, and yet had their own take on Good and Evil. When a computer programmer is accidentally scanned into this digital world, he becomes the catalyst to ending the tyranny of the Master Control Program, who deletes other programs for sport and to maintain control of the system.

The Terminator series

He keeps on coming back

The Terminator series (Wikipedia link, IMDB link) had a military AI go rogue and determine that humanity was something that needs to be eradicated. Through the creation of killing machines which outclass human combat abilities in every way, it has reduced humanity to small roving bands of resistance who fight for survival across the timeline.

2001: A Space Odyssey

AIs and spaceships and aliens, oh my.

2001: A Space Odyssey (Wikipedia link, IMDB link) has its own famous rouge AI, HAL 9000, which went insane when pushed past its programming, and began to secretly plot the murder of the crew on the spaceship that it was supposed to help maintain.

____________________________________________________________

University

In university and graduate school I gravitated towards electrical engineering and computer science majors, the foundations on which artificial intelligence can be constructed. I took courses which used transistors as artificial neurons, and designed my own transistor-based neurons in hardware. I created self-learning board game players using reinforcement learning, probabilistically choosing the best action through a softmax function, and pitted them against each other so that they can learn faster than from playing against humans.

____________________________________________________________

University Projects

Integrated Circuit

Closeup of die in IC package

My first tapeout and IC, implementing artificial neurons in silicon using CMOS transistors, with inputs connected to floating gates. A single neurons could execute simple binary logic functions such as NOT, AND, NAND, OR, NOR, XOR, XNOR by adjusting control signals without changing the hardware.  Based on the design by Nakai, T., Yamashita, T., Ohmi, T. et al. Analog Integrated Circuits and Signal Processing (1999) 21: 173. (Digital Object Identifier link)

Dao

Dao board game

The game of Dao (BoardGameGeek link) is played on a 4x4 board, with each player controlling 4 pieces. The pieces may move vertically, horizontally, or diagonally until it reached an edge or is blocked by another piece. There are 3 ways to win:

• Line up the pieces horizontally or vertically
• Occupy all four corners
• Form a 2x2 square

However, if you trap an opponent's piece in the corner which they can't move out of, you lose the game.

I chose to train an AI to play this game because it is very limited in scope, and can give results quickly. At that time, there wasn't enough computational power easily available to train an AI to play go, which was what I had originally investigated before admitting that it was not appropriate for a short university project.

Using reinforcement learning, I created a database of moves which recorded the results of thousands of games the program played against itself, and the program progressively learned the moves that led to a better outcome.

This game has since been perfectly solved, which means that all possible moves have been calculated and the optimal move can always be chosen, obviating the need for an AI player.

____________________________________________________________

Career

At work, I have been involved with AI in a different way. Being part of the Product Planning team at Samsung Electronics's System LSI Division, over the years I have responsible for many projects which had deep ties to AI. While managing the GPU project for the Exynos SoCs (System on a Chip) (Wikipedia link, Samsung link), I have had to investigate how neural networks could be implemented on GPUs in mobile devices, as well as which applications were important in the market.

When I was in charge of investigating the drone (or UAV, Unmanned Autonomous Vehicle) market, I also looked into how computer vision can be used to improve drone performance, such as with object detection and tracking, obstacle avoidance, and path planning.

____________________________________________________________

Drone Algorithms

Object Detection and Tracking

DJI Mavic Pro tutorial (YouTube link)

Detect the object to be tracked, figure out the motion vector of every pixel in a video frame using a variety of techniques such as optical flow or Kalman filters, then predict where the object will be. This is used for features such as automatic user following and automatic camera panning during flight.

Path Planning, Motion Planning, and Obstacle Avoidance

MIT CSAIL Drone Autonomously Avoiding Obstacles at 30 MPH (YouTube link)

Detect the objects in the area, plan a path from current location to the destination, and controls the motion of the drone to properly navigate the planned path. All this needs to be done dynamically, as object can move and change the situation without notice.

____________________________________________________________

Career

Recently, when I took over software product planning, I collaborated with major China solution vendors and device manufacturers to implement the latest in deep learning technology on the Exynos platform. This included semantic segmentation (for single camera bokeh - what is part of the body needs to be separated from the background, this includes difficult parts such as hair, fingers, akimbo arms), object and scene recognition (for automatic camera settings such as food, pets, sunrise/sunset, snow, flower), and facial landmarks / face recognition (for face unlock, face beauty, AR emojis, which all need an accurate measurement of various features of the face).

____________________________________________________________

Camera Algorithms

Semantic segmentation

Segmentation Competition of Visual Object Classes Challenge 2012 (Link)

Carve out the the different parts of the image so that each can go through image processing separately. For example, separate out the hair of the subject so that its color can be changed, or separate out the person so that the background can be blurred (bokeh effect).

Object and Scene Recognition

COCO 2019 scene segmentation challenge (COCO link)

Recognize the objects in the scene so that proper camera setting can be applied. For example, overexposing photos of snowscapes to make the snow white, using a warmer white balance to bring out the colors of a sunset, or boosting saturation slightly to bring out the color of flowers. The categories that have to pass recognition is many times more various and complicated than used for semantic segmentation. More sophisticated algorithms have also begun incorporating semantic segmentation into object and scene recognition functions so that each part of the image can be processed separately, for example separating the sky, grass, and person in an image.

Facial Landmarks

High-resolution networks for facial landmark detection (GitHub link)

Recognize and mark the major points on a human face, such as outlines of the chin, nose, eyes, eyebrows, lips. These points can then be mapped to AR emojis for animation, warped for face beauty functions such as slimmer face or larger eyes, or used to calculate a set of unique characteristic values which can be used to identify the user.

____________________________________________________________

The Future

My study of AI in university predated the current trend of deep learning, and although many of the terminology are the same, and many of the concepts carry over, I often feel like I do not have an in-depth understanding of what deep learning involves, in terms of the theory and practice behind the applications that I am working with. Therefore I've decided to embark on a study of these fancy newfangled algorithms that kids nowadays are slinging about, and see what kind of meat is behind it.

Please follow along in my next article, In Which a Hardware Platform was Chosen, Part 1 (link), as I chronicle my path through the new deep learning landscape, starting from figuring out the type of hardware to use, to learning the material, to implementing a project of my own. Leave any question or comments below, and let me know what you think!