What Are Neural Networks?

Image result for brain neural network

Intelligence can be described as “the ability to perceive information, retain it as knowledge, and apply it to adaptive behaviors within an environment or context” [1]. The ability to exercise intelligence has been a key differentiator between animals and computers. The prior has a complex brain that is able to simultaneously digest various bits of data and provide an informed solution or reaction. Conversely, a computer is traditionally confined to executing a set of instructions based on confined and user-dependent instructions. For example, in tweeting using a hashtag is necessary to tell the software how and where to classify it.

While the algorithms for artificial intelligence are based on research conducted some 50 years ago, advancements in technology (for example research into self-driving cars) and computing power have put AI in the spotlight. This blog post will discuss neural networks, which is just one classification of AI that attempts to create a decision-making model based on the human brain.

Image result for artificial neural network
Diagram of artificial neuron network. Source:http://futurehumanevolution.com/artificial-intelligence-future-human-evolution/artificial-neural-networks

Artifical neural networks (ANN) are comprised of artificial neurons (units) which can hold and transmit values of 0, 1, or something in-between. These units are interconnected and fire different signals when fed information. Each neuron receives multiple inputs simultaneously which have different weightings. This combination of input signals determines whether an ‘activation threshold’ has been crossed, which then passes information downstream to other neurons. A series of these interactions (that occur simultaneously) allows the computation of various tasks, rather than a uni-dimensional framework of processing. This system of processing information is analogous to how we perform many of our bodily functions. For example, consider how our body differentiates between the different substances we ingest. When we put something in our mouth our body receives input data like how it feels (texture), its temperature, its smell, and weight. This information is then processed to output different reactions: like blocking the gullet, swallowing and coughing.

A pre-requisite for these networks to function is prior training. This is done by feeding the system with data and expected outputs. For example, some Android phones have the ability to search for pictures in the gallery by typing text. In order for the software to be able to recognize what a cat is, it requires initial data to be introduced in a database with pictures that have cats and do not include cats. Note that it’s impossible to feed the algorithm every conceivable picture of a cat. Instead, algorithms recognize various patterns in its past experience and use these features to calculate a probability for the appropriate outcome. AI’s pattern recognition abilities are what have allowed it to master more complex problems, like playing chess and more recently GO. By pitting Google’s AlphaGo against computers in billions of games, a computer is now able to create winning strategies that are far beyond the reach of any current Grand Master [2].

The applications of these powerful algorithms continue to grow in fields like autonomous driving, speech and image recognition, fraud detection and data mining [3]. However, the neural network’s biggest strength also contributes to a weakness. In order to function effectively, vast amounts of training and data are required for training. When dealing with scenarios that are completely new, faults in decision making become more evident. Furthermore, it’s possible for trend changes to occur, resulting in biased decision making since the AI uses backdated or redundant data to make decisions. While there is a long way before AI reaches our ability to process complex sets of data and reach conclusions, it has the potential and is already beginning to replace some professional jobs, like junior lawyers, that we once thought were untouchable.

 

Reference

[1] https://www.innoarchitech.com/artificial-intelligence-deep-learning-neural-networks-explained/

[2] https://techcrunch.com/2017/05/24/alphago-beats-planets-best-human-go-player-ke-jie/

[3] http://www.alyuda.com/products/forecaster/neural-network-applications.htm

[4] https://www.ft.com/content/f809870c-26a1-11e7-8691-d5f7e0cd0a16

1+

Users who have LIKED this post:

  • avatar

6 comments on “What Are Neural Networks?”

  1. Thanks for this interesting post.
    In your article, you mentioned the problem of training neural networks with numerous inputs and using thousands of CPU cores to process the vast amount data it receives.
    What’s your point of view regarding the recent breakthrough in this sector?
    New software frameworks now allow us to increase the scale and speed of deep neural networks training. Image recognition softwares are a good example of such progress as the computer’s accuracy is now similar to that of humans.

    2+

    Users who have LIKED this comment:

    • avatar
  2. Hi Victor,
    Thanks for your comment 🙂 Recent progression means that this technology is able to challenge us on the execution of relatively simple, repeated tasks. Much like comparing quantum computing to regular computing, improving just the computational power isn’t sufficient. Rather there will need to be new methods created for training AI that are more effective with less data.

    0
  3. Thank you for your post Sidharth!

    In my experience, neural networks can make relationships between variables difficult to explain. For a linear regression, for instance, it is very easy to explain that one variable relates to the other by some slope, and that the independent variable predicts some percent of the variance in the dependent variable. Explaining the types of relationships discovered by (especially deep, multi-layer) neural networks can be much more difficult.

    Can you comment on this? Do you feel the responsibility is on the technical folks or the decision-makers to make these relationships easier to understand?

    1+

    Users who have LIKED this comment:

    • avatar
  4. Thanks for your comment Emys, you raised some really valid points.

    Model misspecification is absolutely a huge risk in creating these neural networks, particularly when you start introducing a lot of new variables. While I’m not 100% sure, I think that the implementation of neural networks partly reduces issues related to incorrectly capturing cross-correlations and estimator bias when compared to your normal multi-parameter statistical regression models. By feeding systems with massive amounts of data the AI changes the weighting of different nodes to help tackle the issues you noted above. Naturally, when we extrapolate from our data-set you are bound to make errors. The neural networks implemented probably maximize some sort of user-specified MLE which is inherently non-linear, I think this would be pretty difficult to explain to people without a base knowledge of statistics. A good way of improving clarity to users could be instead explaining what to do with Type 1 or Type 2 errors.

    I think I’m beginning to stretch my knowledge on stats and AI so hopefully the above made sense!

    0
  5. Hey Sidharth, well articulated summary of a ANN. The amount of research in this field is what has made Deep Learning such an industrial success, with advanced neural networks like Convolution Neural networks (used extensively in image recognition) and Recurrent neural networks. If people reading this comment are aware of a term called over fitting in machine learning, then researchers have come up with very innovative methods to overcome over fitting in neural networks. These methods have proven to be very successful and I list one example here : https://www.cs.toronto.edu/~hinton/absps/JMLRdropout.pdf

    PS: My section is MS&E 238A

    1+

    Users who have LIKED this comment:

    • avatar
  6. Thanks for such an informative post, Sidharth! I agree this intersection of neurobiology and machine learning is really a big breakthrough. It is also true that the current capability of algorithms is limited when the new decision is not related to past events. I am hopeful though to see more innovations in this field.

    1+

    Users who have LIKED this comment:

    • avatar

Comments are closed.