Reference 1
Early stage of AI: problems can be described by a list of formal, mathematical rules: Relatively easy for computers but hard for humans.
Now Challenging Task: that are easy for people to perform but hard for people to describe formally. ==> LLM: but not impossible to describe formally, yes???
Deep Learning: a solution to solve tasks that easy for people but hard to describe formally.
This solution is to allow computers to learn from experience and understand the world in terms of a hierarchy of concepts, with each concept defined in terms of its relation to simpler concepts.
By gathering knowlege from experience, this approach avoids the need for human operators to formally specify all of the knowledge that the computer needs.
The hierarchy of concepts allows the computer to learn complicated concepts by building them out of simpler ones. If we draw a graph showing how these concepts are built on top of each other, the graph is deep, with many layers. For this reason, we call this approach to AI deep learning.
sterile and formal environments, and did not require computer to have much knowlege about the world.
IBM’s Deep Blue chess-playing system. Chess can be completely described by a very brief list of completely formal rules, easily provided ahead of time by the programmer.
A person’s everyday life requires an immense amount of knowlege about the world. Much of this knowledge is subjective and intuitive, and therefore difficult to articulate in a formal way.
Hardcode knowledge does not work:
FredWhileShaving==> AI systems need the ability to their own knowledge.
==> Machine learning:, find their own knowledge by extracting patterns from raw data.
Algorithms to discover knowledge:
==> dependence on representations: a general phenomenon that appears throughout computer science and even everyday life.
Many artificial intelligence tasks can be solved by designing the right set of features to extract for that task, then providing these features to a simple machine learn algorithm. For example, a useful feature for speaker identification from sound is an estimate of the size of speaker’s vocal tract. It therefore gives a strong clue as to whether the speaker is a man, woman, or child.
For many tasks, it is difficult to know what features should be extracted.
For example, a program to detect cars/wheels in photographs. Wheels has a simple geometric shape but its image may be complicated by shadows falling on the wheel, the sun glaring off the metal parts of the wheel, the fender of the car or an object in the forground obscuring part of the wheel, and so on.
==> LLM: a much easier approach: add new features for us to detect more easily??? In this example, we can put a sensor in the car/wheel, it will tell you whether its a car/wheel, or more things about itself you want to know. (the picture taken is also a feature of car/wheel, but is badly choosen feature that is hard to learn something from it)
Examples: Atomic fluorescence (原子荧光用于物体跟踪的技术)
If you could revise
the fundmental principles of
computer system design
to improve security...
... what would you change?