Hi,
I am looking to increase my functional understanding of Artificial Intelligence (AI) but am having a hard time understanding the following terms, despite a lot of reading. They seem to be used inconsistently in the literature so I am looking for any input and perspectives.
• Annotated database
• Knowledge graph
• Ontology
I think I understand ontology the best – a way to describe things or concepts and how they relate to each other. For example, an ontology for team sports would have entities/concepts such as team, player, position, etc. and how they relate to one another. My interpretation of an annotated database and knowledge graph is they are essentially the same thing, and are a defined ontology for a specific domain that computer systems/AI can reference to provide contextually correct information and answers or actions/responses.
Is this generally correct? If so, what is the real magic in AI? It seems many product and service offerings rely on these concepts (meaning some actual intelligence/person had to already define and curate information and organize it in a way that a computer can process).
Keep in mind, that artificial intelligence is a very broad domain and so encompasses a great many things.
In this context, to best answer your question, AI could be talking about a couple of different things. I’d need to see a specific example of a product or service to answer more accurately.
One possibility is optimization. So in this context, given a set of knowledge, an AI algorithm can provide some kind of optimal planning.
Another possibility is to do analytics. So having a lot of information doesn’t necessarily mean that much, so different AI algorithms can analyze the information to learn thing that might not be evident to a human because say the knowledge graph is huge.
Finally, another possibilty is learning (this would be pretty related to analytics). So given some set of information, it is possible to then learn something new about it. Alternatively, a set of information could act as a core from which an algorithm learns so that it can add to the knowledge base.
I wish I could answer more specifically but AI is a very broad term. If you can be more specific, I’ll try to answer more completely.
Not an expert here, but I have read a fair amount about it. There seem to be two general approaches to it. The first is to have an enormous database and complicated rules suggested by experts. That seems to be the OP’s picture. It is how IBM programmed Deep Blue. The other one is what are called neural networks. They come in layers, 3 or 4 (or maybe more of them). The circuits at one layer are programmed to connect to circuits at the next; like nerves, the connections are weighted by positive or negative numbers. It is set up so that when a circuit’s total input from below exceeds a threshold it “fires”, meaning it sends a signal to the circuits in the next layer it connects to through the assigned weights. If the weight is negative, it inhibits the circuit in the next layer; if positive, it helps incites it.
Imagine you would like to get the network to learn the game of Go. Then it plays an enormous number of games (with itself, in fact) and whenever it wins it strengthens the weighting factors, more or less at random and when it loses it weakens them. This continues as it gets stronger and stronger until it can beat a human expert.
Similarly if you want it to learn to recognize dogs, say, you start with random weights and feed it photos of both dogs and non-dogs, both wolves and coyotes and things nothing like dogs. It starts to guess at random, but every time it guesses right, the connections are strengthened and when it is wrong they are weakened. And this actually works.
There are a lot of different ways to attempt AI and AI-like systems, but if a human has to pre-organise the information, that might not be much more than an ‘expert system’ - these things have a lot of uses in domains with finite answers and clean inputs (such as, I dunno, prescribed medication).
AFAIK, most of the more promising attempts at AI involve self-organising systems where the product isn’t ‘programmed’ as such, but rather, is ‘taught’. These systems have more chance of producing a meaningful output when the inputs are not exactly as expected - Neural Nets are the most obvious example of this, but not the only approach.
The stuff the OP refers to is often lumped under the banner of knowledge representation. As noted above, AI means a lot of different techniques, although over time there are a few dominant ideas that seem to have gained a lot of traction. And there are hybrids. You can also use an AI system to optimise another AI system.
There is a huge difference between say: processing a business database looking for trends, versus identifying a person in a set of images, to working out the optimal chess move. The fundamentals of these systems are totally different.
But there are times when you have some domain knowledge about something, and you have a lot of facts in the domain. The problem is how to represent this in a form that is automatically processable with only blind rules. You might say “team consists of players”, but from the point of view of the inference system, all you have is “A consists of more than one B”. So you need to build the definition very carefully. But as you add knowledge it fills out. Team must have a coach. Coach is not a player. The KR system allows you to express these rules in a manner that allows inference. (Often starting with the basics of: is-a, has-a.) This is your ontology. The inference engine is the trick. Rules that are expressed in the right form can be chained together, reduced, and searched over. The search works towards goals that are also expressible in the same system. One way of doing this is backward chaining, where you express your goal, and the system tries to find sets of rules and facts that can deliver the goal. This may involve finding intermediate rules and facts, that are themselves inferable from others, and so on, until you find a set of facts in your knowledge base that allow the goal to be satisfied with the application of the rules. Forward chaining goes the opposite way, and tries to match the goal by transforming existing facts and rules with other rules and facts until the goal is found. Obviously to perform an exhaustive combinatorial search of all possible facts and application of rules in infeasible except for toy systems. So there is a lot of magic in the inference engine to allow searches to be truncated early and most paths eliminated without spending too much time. Additional rules may help in this, ways of defining the quality of the intermediate goal, and so on. The better your ontology represents facts and the better your constraints are expressed will make a huge difference. There needs to be a match of ontological form and function with the problem. The basics of this go back to the very early days of AI. The modern capabilities of machines and the huge wealth of accumulated experience in how make it work are the big game changers.
The current key “magic” is that:
1 - neural networks are generalized function approximators. Meaning they can approximate pretty much any function, with the right number of neurons, layers and connections.
2 - some smart people have figured out how to train multi-layered neural networks so they approximate the desired function
Meaning, for example, to perform the image recognition function, images are run through the network, the resulting “answer” is compared to the correct answer and if it’s not correct, the network is adjusted to get closer to the correct answer. Lather rinse repeat using lots of test data until the answers are almost always correct and at that point the network is trained to perform that specific function (e.g. identifying a cat or a hot dog).
This is a powerful technique because the functions being approximated are not functions we could really figure out using any other methods.
While this technique can be used to solve many problems, it’s not really obvious at this point how to use it to create the type of advanced abstract thinking type of intelligence that most people think of when we think “intelligence”, but it’s certainly a foundational element to the type of processing we do when dealing so flexibly with our environment.
Thank you for the detailed answers. These concepts are still pretty abstract for me (as were neural networks as they were explained in grad school 18 years ago), so I have a few clarifying questions:
I don’t have a particular domain in in mind, but am trying to relate these concepts to “Input A results in Output B” based on “knowledge”. Meaning, can an AI system
a) infer a person’s intent (pretty sure yes, based on NLP advances and the fact that a language can be expressed in an ontology such as a noun is a person, place or thing and an adjective is a property, descriptor, or attribute of a noun, etc and we have curated ontologies (such as dictionaries that define these concepts); and
b) based on an input, produce an answer or output informed by a body of knowledge expressed in similar ways (unstructured data sources such as news articles, laws or regulations, etc - all language that can be parsed as nouns, adjectives, verbs, etc).
So two examples (and I am wondering what is truly feasible here):
My federal tax return. The inputs are my financial information and qualitative aspects of my situation to which certain tax breaks might apply. The are all easy “inputs”. The body of knowledge is the tax code. Can an AI complete my tax return (a desired/targeted output) with only these as inputs? I believe the answer is “no”, because it’s not been done yet. Programs like TurboTax are built on complex IF THEN ELSE rules that a human has created - you get asked a bunch of questions, and based on your answers you get asked some n number of iterations of additional questions, and then you get your completed tax return. To me, there is nothing “artificially intelligent” about that - someone had to understand the decision tree and program it into an application.
More generally, I picture a spreadsheet where column A has a question, and column Z has the answer. There may be many “right” answers to one question based on facts and circumstances that require additional information. I can model those paths in columns B-Y (again IF THEN ELSE), but if I have to do this work, then what is the “magic” of AI if I have to model the decision path?
More generally, can I ask a computer system an open ended question and get a valid answer based on the information it has access to? I tend to think not - this is fundamentally a structured vs. unstructured data issue, and information systems are not advanced to the point where they can derive a valid answer from unstructured data. My perception is that creating ontologies and taxonomies is not fundamentally different from building a relational database based on content that “you” know because it’s “you” (a person) that has the understanding of how data/information/facts relate to one another and how that that information may be queried.
Neural networks are perhaps an abstraction of this - if I understand them correctly, they need to see many inputs and iterations of patterns, and can derive a “correct” response based on past experience. This is how structured machine learning works but that does not seem “AI” - you can either invest a lot of time anticipating questions and decision trees (programming an IF THEN ELSE system) or invest a lot of time/data in ML/neural networks with an expectation that with enough time and pattering it might get to a sensible result. I can see applications of both but am still missing the magic of “AI” - it just not seem to be there (ML, yes…but that is statistics and correlations and not much else.
b) based on an input, produce an answer or output informed by a body of knowledge expressed in similar ways (unstructured data sources such as news articles, laws or regulations, etc - all language that can be parsed as nouns, adjectives, verbs, etc).
[/quote]
I’m not going to answer this one because I have very little expertise in NLP. To the best of my knowledge, the answer is probably yes with caveats. See IBM’s Watson.
The answer is almost certainly yes, in the sense that could you train an AI to complete your tax return. An AI given proper samples could almost certainly infer the proper rules. The reason it hasn’t been done is because there’s not really much point in doing so. The rules are already well known so there’s no real reason to have an AI develop a process for completing the tax return, it is easier (and more certain) to simply code the rules using traditional programming techniques. AI is particularly valuable when what to do isn’t well-known, isn’t obvious or is so complex that coding the rules is difficult. This is what you see, for example, with self-driving cars. It is easier (but not easy) to have an AI learn how to drive a car then try to encode how to drive a car.
The “magic” is more in the case where you don’t know the rules because the data is too complex to decipher by human ability alone. I’ve said a few times on this board, that the best way to think of AI, as it currently exists, is as a computational aide to human abilities, more so than anything truly intelligence.
If you mean ask in the sense of English question, a la Star Trek. Probably not (but maybe, again see IBM’s Watson) but this is to some degree the point in the applications of AI to analytics. Is this perfect? No. There’s a lot of problems with AI and analytics esp. when you get into Big Data but there is much that can be done. This also touches on knowledge representation, which is where you get into ontologies (there are other approaches to knowledge representation).
This is true to a large degree, that much of the intelligence in AI is still coming from humans. As I said above, AI is currently best viewed as a computational aide to human abilities. That being said, it isn’t strictly true either. The whole point of using an AI algorithm is that you don’t need to be too explicit but again AI is a very broad domain. In my research, I don’t have to tell my AIs much of anything (that’s kind of the point of my research, but I don’t want to get into too much of a tangent) for them to do their processing. But I still do have to provide some front end and back end intelligence. Again, I don’t want to get into a tangent too much, but in essence the intelligence I provide is intelligent design of the problem encoding and interpretation of the results. But the AI does all the heavy lifting in terms of the computations without really “knowing” that much.
I hope that helps. If not, feel free to keep asking, and I’ll answer as best as I can.
This is true to an extent. The thing about a conventional database is that a human has to create the queries. A big part of the magic of a database is the human with the domain knowledge and experience who know how to craft the query to find what it is you might want. You can think of an inference engine as something that uses additional rules and facts to automatically craft the query.
You can describe a huge amount of AI as nothing more than a massive search space. A search space where given enough time a solution to the problem can be found. A big part of AI is working out how to find things in the space in a feasible amount of time. Chess is the usual example. We know it is deterministic. But nobody has the resources to search the space of games. Your database may be so large,and the rules so complex that no query will complete in any useful time. AI techniques try to get you an answer.
While I agree with you that the term “AI” feels overstated when you analyze how these systems work. The systems don’t seems to be operating the way humans probably feel like we operate: creating an internal model of the world over time, basing decisions on the model and the current input, AND being aware that this is what’s going on.
On the other hand, a computer system that can learn the rules of GO and become a master just by playing the game many times? That one seems intelligent. I was personally never impressed with chess playing games due to brute force nature, but the flexible and learning based approach for AlphaGo moves the needle a bit towards intelligent, IMO.
I have to disagree with this. AlphaGo is extremely impressive, but it doesn’t really move the needle of intelligence at all, in my view. It is, afterall, just a Monte Carlo search on an algorithmic graph composed by a deep learning network.
And before somebody says “Yeah but it is a deep learning network”, Deep Learning is just PR. It is no different than dynamic programming from the 1950s. It sounds much cooler than it really is. If I say that AlphaGo uses a convolutional 13-layer neural network, well, it loses something doesn’t it?
AlphaGo is still nothing more* than computational trickery with the intelligence provided by the developers on the front end via intelligent design. Anyway, just my point of view on the matter.
And by nothing more, let’s be clear, AlphaGo is amazing and a stellar breakthrough in using AI. Period. Full stop.
I would disagree that Deep Learning is just PR. Prior to Hinton (and probably others but he seems to get primary credit), there was no formulaic method of adjusting weights in a many layered network other than trial and error (e.g. evolution style algorithms).
And, it’s been known for some time that many layered neural networks outperformed all other approaches to things like image recognition, so the ability to train them using a backprop type algorithm seems like a significant development.
I mean the term “Deep Learning” is just PR. It doesn’t mean anything more than “dynamic programming” does. Unfortunately (or fortunately?) it sounds cool. It sounds like it learns deeper, but it doesn’t in any meaningful way. There is nothing more intelligent about a deep learning network than any other neural network. That doesn’t mean it isn’t impressive or isn’t significant (it clearly is), or it cannot be used for different or more complex things, but it is not more intelligent.
I don’t disagree that there is nothing more “intelligent” about the underlying tools. But I do think that AlphaGo represents an application of the tools in a way that produces something that moves the needle “a bit” towards intelligent.
This is firmly in subjective territory (and I did use the qualifier of “a bit”), but basically, the ability of the system to acquire the rules through learning, and the ability to improve it’s play to expert level for a complex game feels more intelligent than chess brute force (for example).
A generalized system that was able to do that (assuming some sort of proper drive and/or motivation towards the goal) for arbitrary domains would move the needle more.
My usage of “intelligent” does not mean conscious or aware, it’s more focused on flexible and adaptable to complex environments.
I was thinking about applying the Turing Test to humor. I imagined a conversation going like this:
–AI, tell me the funniest joke possible.
–I can’t.
–Why not?
–Because it would contain elements that are politically incorrect.
–OK, then, tell me the funniest joke that is politically correct.
–There are none.
So, we have an interesting conundrum. Shall we build political correctness into AI? Or should we ask AI if political correctness has any empirical truth? Are we prepared for the answer, and a social order guided by that answer?
This is not something remotely discussed in the AI community. What is discussed is how to ensure AIs don’t act contrary to the best interests of humanity.
Not sure if you’re saying that Ethics is, or is not, in the best interests of humanity. I don’t mean AI acting ethically, but AI defining ethics for its own purposes of evaluating ethical actions. The difference seems, to me, to be significant.
For exampke, is using animal testing for medical research in the best interests of humanity"? How does AI judge that?
I think I understand your question. Generally, speaking fitness (or how to value something) is provided by a human. This is called a supervised learning model. So if you were training an AI that is responsible for driving a car, then every time it ignores a pedestrian crossing a street (simulated of course), you smack it on the virtual wrist and say “No, bad” and every time it stops properly you say “Good boy”.
In the unsupervised learning model, you generally tell the AI to look at some metric. The classic is example is an automated taxi. You could tell it to consider the tips provided, under the assumption that the higher the tip the better the service provided and therefore it did things well (note, this isn’t actually a good way to do things because you could get very bad behaviour this way, such as speeding, it is just an example).
Currently, AI has no contextual awareness so it has no means to really judge whether something like political correctness or animal testing is good or bad. It is an open question right now as to whether AlphaGo knows or to what degree is knows it is playing Go. What does it mean to know you’re playing Go? At what point are you aware or that? What is the threshold?
To take animal testing further, you could certainly code some kind of fitness function that weighs human lives vs animal lives under different scenarios to come up with an answer. But those weights are either going to be human supplied (under a supervised learning model) or certainly human influenced under an unsupervised learning model.