Now that you have seen a basic example of how to detect faces, it is time to learn a bit about what else Cognitive Services can do for you. When using Cognitive Services, you have 21 different APIs to hand. These are, in turn, separated into five top-level domains according to what they do. They are vision, speech, language, knowledge, and search. Let's learn more about them in the following sections.
APIs under the Vision flags allows your apps to understand images and video content. It allows you to retrieve information about faces, feelings, and other visual content. You can stabilize videos and recognize celebrities. You can read text in images and generate thumbnails from videos and images.
There are four APIs contained in the Vision area, which we will look at now.
Using the Computer Vision API, you can retrieve actionable information from images. This means that you can identify content (such as image format, image size, colors, faces, and more). You can detect whether or not an image is adult/racy. This API can recognize text in images and extract it to machine-readable words. It can detect celebrities from a variety of areas. Lastly it can generate storage-efficient thumbnails with smart cropping functionality.
We will look into Computer Vision in Chapter 2, Analyzing Images to Recognize a Face.
The Emotion API allows you to recognize emotions, both in images and in videos. This can allow for more personalized experiences in applications. Emotions detected are cross-cultural emotions: anger, contempt, disgust, fear, happiness, neutral, sadness, and surprise.
We will cover Emotion API over two chapters: Chapter 2, Analyzing Images to Recognize a Face, for image-based emotions, and Chapter 3, Analyzing Videos, for video-based emotions.
We have already seen a very basic example of what the Face API can do. The rest of the API revolves around this detecting, identifying, organizing, and tagging faces in photos. Apart from face detection, you can see how likely it is that two faces belong to the same person. You can identify faces and also find similar-looking faces.
We will dive further into Face API in Chapter 2, Analyzing Images to Recognize a Face.
The Video API is about the analyzing, editing, and processing of videos in your app. If you have a video that is shaky, the API allows you to stabilize it. You can detect and track faces in videos. If a video contains a stationary background, you can detect motion. The API lets you generate thumbnail summaries for videos, which allows users to see previews or snapshots quickly.
Video will be covered in Chapter 3, Analyzing Videos.
Using the Video Indexer API, one can start indexing videos immediately upon upload. This means you can get video insights without using experts or custom code. Content discovery can be improved, utilizing the powerful artificial intelligence of this API. This allows you to make your content more discoverable.
Video indexer will be covered in Chapter 3, Analyzing Videos.
The Content Moderator API utilizes machine learning to automatically moderate content. It can detect potentially offensive and unwanted images, videos, and text for over 100 languages. In addition, it allows you to review detected material to improve the service.
Content Moderator will be covered in Chapter 2, Analyzing Images to Recognize a Face.
Adding one of the Speech APIs allows your application to hear and speak to your users. The APIs can filter noise and identify speakers. Based on the recognized intent, they can drive further actions in your application.
Speech contains three APIs that are discussed as follows.
Adding the Bing Speech API to your application allows you to convert speech to text and vice versa. You can convert spoken audio to text, either by utilizing a microphone or other sources in real time or by converting audio from files. The API also offers speech intent recognition, which is trained by Language Understanding Intelligent Service (LUIS) to understand the intent.
The SpeakerRecognition API gives your application the ability to know who is talking. By using this API, you can verify that the person speaking is who they claim to be. You can also determine who an unknown speaker is based on a group of selected speakers.
To improve speech recognition, you can use the Custom Recognition API. This allows you to fine-tune speech recognition operations for anyone, anywhere. By using this API, the speech recognition model can be tailored to the vocabulary and speaking style of the user. In addition, the model can be customized to match the expected environment of the application.
The Translator Speech API is a cloud-based automatic translation service for spoken audio. Using this API, you can add end-to-end translation across web apps, mobile apps, and desktop applications. Depending on your use cases, it can provide you with partial translations, full translations, and transcripts of the translations.
We will cover all speech related APIs in Chapter 5, Speak with Your Application.
APIs related to language allow your application to process natural language and learn how to recognize what users want. You can add textual and linguistic analysis to your application, as well as natural language understanding.
The following five APIs can be found in the Language area.
The Bing Spell Check API allows you to add advanced spell checking to your application.
This API will be covered in Chapter 6, Understanding Text.
LUIS is an API that can help your application understand commands from your users. Using this API, you can create language models that understand intents. By using models from Bing and Cortana, you can make these models recognize common requests and entities (such as places, times, and numbers). You can add conversational intelligence to your applications.
LUIS will be covered in Chapter 4, Let Applications Understand Commands.
The Linguistic Analysis API lets you parse complex text to explore the structure of text. By using this API, you can find nouns, verbs, and more in text, which allows your application to understand who is doing what to whom.
We will see more of Linguistic Analysis in Chapter 6, Understanding Text.
The Text Analysis API will help you in extracting information from text. You can find the sentiment of a text (whether the text is positive or negative). You will be able to detect language, topic, and key phrases used throughout the text.
We will also cover Text Analysis in Chapter 6, Understanding Text.
By using the Web Language Model (WebLM) API, you are able to leverage the power of language models trained on web-scale data. You can use this API to predict which words or sequences follow a given sequence or word.
Web Language Model API will be covered in Chapter 6, Understanding Text.
By adding the Translator Text API, you can get textual translations for over 60 languages. It can detect languages automatically, and you can customize the API to your needs. In addition, you can improve translations by creating user groups, utilizing the power of crowd-sourcing.
Translator Text API will not be covered in this book.
When talking about Knowledge APIs, we are talking about APIs that allow you to tap into rich knowledge. This may be knowledge from the web. It may be from academia or it may be your own data. Using these APIs, you will be able to explore different nuances of knowledge.
The following four APIs are contained in the Knowledge API area.
Using the Academic API, you can explore relationships among academic papers, journals, and authors. This API allows you to interpret natural language user query strings, which allow your application to anticipate what the user is typing. It will evaluate said expression and return academic knowledge entities.
This API will be covered more in Chapter 8, Query Structured Data in a Natural Way.
Entity Linking is the API you would use to extend knowledge of people, places, and events based on the context. As you may know, a single word may be used differently based on the context. Using this API allows you to recognize and identify each separate entity within a paragraph, based on the context.
We will go through Entity Linking API in Chapter 7, Extending Knowledge Based on Context.
The Knowledge Exploration API will let you add the possibility of using interactive search for structured data in your projects. It interprets natural language queries and offers auto-completions to minimize user effort. Based on the query expression received, it will retrieve detailed information about matching objects.
Details on this API will be covered in Chapter 8, Query Structured Data in a Natural Way.
The Recommendations API allows you to provide personalized product recommendations for your customers. You can use this API to add a frequently bought together functionality to your application. Another feature you can add is item-to-item recommendations, which allows customers to see what other customers like. This API will also allow you to add recommendations based on the prior activity of the customer.
We will go through this API in Chapter 7, Extending Knowledge Based on Context.
The QnA Maker is a service to distill information for Frequently Asked Questions (FAQ). Using existing FAQs, either online or per document, you can create question and answer pairs. Pairs can be edited, removed, and modified, and you can add several similar questions to match a given pair.
We will cover QnA Maker in Chapter 8, Query Structured Data in a Natural Way.
Search APIs give you the ability to make your applications more intelligent with the power of Bing. Using these APIs, you can use a single call to access data from billions of web pages, images, videos, and news.
The following five APIs are in the search domain.
With Bing Web Search, you can search for details in billions of web documents indexed by Bing. All the results can be arranged and ordered according to a layout you specify, and the results are customized to the location of the end user.
Using the Bing Image Search API, you can add an advanced image and metadata search to your application. Results include URL to images, thumbnails, and metadata. You will also be able to get machine-generated captions, similar images, and more. This API allows you to filter the results based on image type, layout, freshness (how new the image is), and license.
Bing Video Search will allow you to search for videos and return rich results. The results contain metadata from the videos, static or motion-based thumbnails, and the video itself. You can add filters to the result based on freshness, video length, resolution, and price.
If you add Bing News Search to your application, you can search for news articles. Results can include authoritative images, related news and categories, information on the provider, URL, and more. To be more specific, you can filter news based on topics.
The Bing Autosuggest API is a small, but powerful one. It will allow your users to search faster with search suggestions, allowing you to connect a powerful search to your apps.
All Search APIs will be covered in Chapter 9, Adding Specialized Search.