Using Amazon Polly to Provide Real-Time Home Monitoring Alerts

Read More...

Build a Voice Kit with Amazon Lex and a Raspberry Pi

Read More...

Two New Courses are Now Available for Machine Learning and Deep Learning on AWS

AWS Training and Certification helps you advance your knowledge with practical skills so you can get more out of the AWS Cloud. We now have two new courses to help you learn about how to leverage artificial intelligence (AI) solutions using AWS: Introduction to Machine Learning web-based training and Deep Learning on AWS instructor-led training. If you are looking to learn more about how you can put AI capabilities to use, we recommend that you start with the web-based training. Developers looking to learn more should then attend the one-day instructor-led training.

Here’s a bit more about each of these new training courses:

Introduction to Machine Learning is a free 40 minute web-based training intended for developers, solutions architects, and IT decision makers who already know the foundations of working with AWS. This online course gives an overview of machine learning (ML), walks through an example use case, teaches relevant terminology, and walks through the process for incorporating ML solutions into a business or product. Specifically, this course teaches you how to do the following:

• Approach ML as a business problem and work toward a technical solution.
• Frame your business problem as a ML problem.
• Use ML terminology and describe techniques in real-world business use cases.
• Understand the end-to-end process of building ML models correctly, from posing the question/problem, collecting data, and building a model to evaluating model performance and integrating it into your application.

The course also includes knowledge checks to help validate understanding.

Deep Learning on AWS is a one-day instructor-led training for developers who are interested in learning more about AWS solutions for deep learning. This course teaches developers about the deep learning model and gives them a roadmap for understanding what challenges deep learning can solve. Solutions related to image recognition, speech recognition, and speech translation are covered. The training includes how to run your models on the cloud using an Amazon Elastic Compute Cloud (Amazon EC2)-based Deep Learning Amazon Machine Image (AMI) and the MXNet framework.

The topics covered in this course include the following:

• Introduction to Machine Learning
• Introduction to Deep Learning
• Lab 1: Setting up a Deep Learning AMI instance and running a multi-layer perceptron model
• Introduction to MXNet on AWS
• Lab 2: Running a convolutional neural network model and predicting images on a CIFAR-10 dataset
• Deploying Deep Learning Workloads on AWS
• Lab 3: Deploying a Deep Learning model for predicting images using AWS Lambda

We recommend that you take Introduction to Machine Learning before attending Deep Learning on AWS, but it is not required. If you want to learn more about how to leverage data for Deep Learning, consider taking Big Data Technology Fundamentals and Building a Serverless Data Lake.

You can sign in to the AWS Training and Certification Portal to browse all of our training offerings and get started.

Powered by WPeMatico

The post Two New Courses are Now Available for Machine Learning and Deep Learning on AWS appeared first on Artificial Intelligence Solutions.

Read More...

Create a Question and Answer Bot with Amazon Lex and Amazon Alexa

Your users have questions and you have answers, but you need a better way for your users to ask their questions and get the right answers. They often call your help desk, or post to your support forum, but over time this adds stress and cost to your organization. Could a chat bot add value for your customers? Interestingly, a recent poll shows that 44% of people would rather talk to a chat bot than to a human!

In this post we provide a sample solution, called QnABot (pronounced “Q and A Bot”). The QnABot uses Amazon Lex and Amazon Alexa to provide a conversational interface for your “Questions and Answers.” This allows your users to ask their questions and get quick and relevant answers.

Amazon Lex lets you integrate both voice and text chat access into your existing applications. Amazon Alexa allows you to offer a hands-free voice interface to your users in their homes or workplaces using Amazon Echo or any Alexa Voice Service enabled device. QnABot offers the best of both worlds.

QnABot uses the Amazon Elasticsearch Service (Amazon ES) to make your questions and answers searchable. When a user asks a question, the Amazon ES powerful full-text search engine is used behind the scenes to find the answer that is the best match for that question.

In the sections that follow we’ll show you how to do the following:

• Deploy QnABot to your AWS account. This blog assumes that you are already an AWS customer. If you want to open an account, choose the Create an AWS Account button on the AWS home page.
• Populate the QnABot with your questions and answers using the Content Designer UI.
• Ask questions using voice or chat with the Web Client UI.
• Ask questions hands-free using the latest Amazon Echo devices.
• Troubleshoot and tune QnABot content to minimize the chances of getting wrong answers.
• Enrich your answers with images and web links.

We’ll also take a look under the hood to show you how QnABot works, and give you some ideas for how you can enhance it to suit your needs.

Deploying QnABot

Use this AWS CloudFormation button to launch QnABot in AWS Region us-east-1:

Note: At the time of writing, only the us-east-1 Region is supported.

The source code is available in our GitHub repository.

In the AWS Management Console, on the CloudFormation Parameters page, enter a password to assign to the QnABot Content Designer Admin user. The password must have a length of at least 8 characters, and contain upper-case and lower-case characters, plus numbers and special characters.

Choose Next twice to display the Review page.

Select the acknowledgement checkbox, and choose Create to deploy QnABot.

The master CloudFormation stack uses nested stacks to create the following resources in your AWS account:

• Amazon Elasticsearch Service domain to store and search your content (1 x t2.small.elasticsearch). Cost approx. $0.036/hour – free tier eligible.
  NOTE: For production use, increase node count to ensure your Amazon ES domain is fault tolerant. For more information, see Configuring Amazon ES Domains.
• Amazon S3 buckets to host the QnABot web UIs.
• Amazon Lex Bot to provide natural language processing.
• AWS Lambda functions used to administer QnABot content and to find answers to questions.
• Amazon API Gateway REST APIs used by the web UIs.
• Amazon Cognito configured to authenticate clients and provide temporary AWS credentials.
• Amazon CloudFront distribution used for websites and API endpoints.

When the deployment is complete (after about 60 minutes), the master stack Output tab shows the following:

• ContentDesignerURL: URL to launch the Content Designer UI.
• ClientURL: URL to launch the user client webpage.
• DashboardUrl: URL to launch the CloudWatch dashboard for monitoring.

Creating initial content in QnABot

You need to load some question and answer data using the Content Designer, before turning QnABot over to your users. Your data is stored in Amazon ES behind the scenes. This way, the data can be searched later when users ask questions using either a Lex client UI or an Alexa hands-free device. The steps that follow show you how to load sample questions used to illustrate the features of the QnABot. You can easily substitute your own content later.

1. In the CloudFormation console, launch the Content Designer webpage using the ContentDesignerURL from the Outputs tab of the master CloudFormation stack.
   
2. Log in with username Admin, and the password you set when you launched the stack.
   
3. Choose Add.
   
4. Enter the id: QnaBot.001
   NOTE: Use a naming convention to identify your items within categories.
5. Enter the question: What is Q and A Bot
6. Enter the answer: The Q and A Bot uses Amazon Lex and Alexa to provide a natural language interface for your FAQ knowledge base, so your users can just ask a question and get a quick and relevant answer.
7. Choose the Save tool (upper right).
   
8. Repeat steps 3-7, entering the items from the table below.
   Alternatively, to avoid the copy/paste process, you can import the items directly from a file. Choose Import/Export, then Import URL, and enter the URL: http://ift.tt/2ygoURI

Id	Question	Answer
QnABot.002	How do I use Q and A bot	Create and administer your questions and answers using the Q and A Bot Content Designer UI. End users ask questions using the Lex web UI which supports voice or chat, or using Alexa devices for hands free voice interaction.
Admin.001	How do I modify Q and A Bot content	Use the Content Designer Question and Test tools to find your existing documents and edit them directly in the console. You can also export existing documents as a JSON file, make changes to the file, and re-import.
Admin.002	Can I back up Q and A Bot content	Yes. Use the Content Designer to export your content as a JSON file. Maintain this file in your version control system or in an S3 bucket. Use the Designer UI Import feature to restore content from the JSON file.
Admin.003	Can I import Q and A Bot content from a file	Yes, the Content Designer has an import function that lets you load items from a formatted JSON file. You can create JSON files using the Export feature, or you can write your own tools to create JSON files from existing content such as a website FAQ page.
Admin.004	How do I troubleshoot and fix problems with Q and A Bot.	Use the Content Designer test tool to test a question, and check what items are returned, ranked in order of score. If the desired item does not have the highest score, then add the question to the item and run the test again. The desired item should now have the highest score. Take care not to create items with duplicate questions, to avoid unpredictable responses.
Admin.005	How can I find specific Q and A items in the Designer UI	Use the Filter feature in the Questions tab to filter the items list based on the ID field. Or use the Test tab to list all the items that match a question.
Media.001	How can I include pictures in Q and A Bot answers	Add an image attachment to the item using the Content Designer.

Getting answers using a Lex Web Client UI

You can launch QnABot from a Chrome, Firefox, or Microsoft Edge browser on your PC, Mac, Chromebook, or Android tablet.

1. On the CloudFormation console, select the main QnABot stack, choose Output, and then choose the link to the ClientURL.
2. When your browser requests access to the microphone on behalf of the web application, allow it. You’ll see the QnABot chat window:
   
   You can interact with QnABot using either text chat or voice.
3. Try it! Choose the microphone icon (bottom right) and say, “What is QnABot?”
   The bot will respond with the answer you previously entered for this question.

Getting answers using Alexa

QnABot also works with Alexa, allowing your users to get answers from your content via any Alexa enabled device, including Amazon FireTV, Amazon Dash Wand, and any of the Amazon Echo family of devices.

To enable Alexa, first create an Alexa skill for QnABot using the Amazon Developer Console. QnABot can’t automatically create the Alexa skill at this time, so we’ve provided instructions. Log in to the QnABot Content Designer (as explained earlier), choose Lex/Alexa and then Alexa Instructions:

Follow the displayed instructions, step by step, to create an Alexa skill in your Amazon account. It will take you just a few minutes.

If you want to publish your QnA skill to the Alexa skills store so other users can access it, see Submitting an Alexa Skill for Certification. Unpublished skills are accessible only to Alexa devices registered to your Amazon account, while published skills are available to anyone.

Troubleshooting and tuning

Tuning answers using the Content Designer

QnABot attempts to match a user’s question to the list of questions and answers stored in Amazon ES by the Content Designer. QnABot uses  Amazon ES full text search to find the item that is the best match for the question asked. Words that are used infrequently score higher than words that are used often, so sentence constructs such as prepositions have lower weighting than unique keywords. The closer the alignment between a question associated with an item and a question asked by the user, the greater the probability that QnABot will choose that item as the most relevant answer.

QnABot tries to find the best answer to questions by matching the user’s question to both the question and the answer fields of the stored items, giving more weight to any matches found in the question field.

You might find that users ask questions in ways that you haven’t anticipated, resulting in unexpected answers being returned by QnABot. When this happens, use the Content Designer to troubleshoot and fix the problem.

To illustrate an example troubleshooting / tuning scenario, let’s use the item we entered in the previous section – Admin.001 “How do I modify Q and A Bot content”:

1. Use the Lex Web UI (voice or text chat) to ask the question in a different way: How can I edit items?
2. Ideally you want the answer for item Admin.001, but in this case, since the question is phrased very differently than the question associated with this item, you don’t get the desired answer.
3. Log in to the Content Designer, and choose Test:
   
4. Enter the same question you asked, “How can I edit items”, into the question field, and choose test. Observe that the highest scoring answer is not Admin.001.
5. Now find and choose the lesser ranked Admin.001. This is our preferred answer.
6. Let’s tune this item, so that next time you ask the same question this item will have the top score. Choose the item to expand it, choose the edit tool (upper right), and then the add question (+) tool under the first question on the left:
   
7. Enter the new question: How can I edit items, and choose the save tool (upper right).
8. Try the same question again, by choosing test (top of the page).
9. Success! This time Admin.001 has the highest score.
   Try asking the question again in the Lex Web UI. This time you’ll get the desired answer.

Tuning the Bot’s Natural Language Processing

When you ask QnABot a question, it’s processed and transcribed by either Lex or Alexa using a Natural Language Processing (NLP) engine. QnABot initially trains the NLP to match a wide variety of possible questions and statements, so that our Lex bot and Alexa skill will accept just about any question a user might ask. (You can see the default training questions listed in the file default-utterances.js.)

However, occasionally you might find that you want to say things to QnABot that it doesn’t understand and is unable to process. You might notice that the transcription shown in the web client or the Alexa app isn’t accurate, or that QnABot asks you to repeat what you just said.

If this happens, it is time to re-train the NLP. QnABot can generate additional training data automatically using questions from all the items you have added.

Retrain Lex

To illustrate, let’s use the item we entered in the previous section, item Admin.001 “How do I modify Q and A Bot content”:

1. Using the Lex Web Client UI (voice or text chat), state: “I need to change some answers”
2. In the last exercise, QnABot returned the wrong answer, but this time, it simply replies: “Sorry, can you please repeat that.” If you repeat the same statement, you will get an error message like, “Sorry, I could not understand. Goodbye.”
   This is a symptom of the fact that the default bot is not trained to understand a statement in this form.
3. Log in to the Content Designer, open item “Admin.001,” choose the Edit tool, choose the add question (+) tool, and add the statement as a new question: I need to change some answers.
   Save the update using the Save tool.
   
4. Choose Lex/Alexa, then choose Rebuild Lex Bot. Wait for the rebuild to complete.
5. Now try again, using the Lex Web Client UI to state “I need to change some answers.” Since the new statement has been added to the QnABot training set, you’ll see that it is no longer rejected, and that QnABot now responds with an appropriate answer.

Retrain Alexa

Update the training data set for Alexa, using the Amazon Developer console to edit the Custom Slot Type values for your skill.

1. Log in to the Content Designer UI webpage, choose Lex/Alexa, and then choose Alexa Instructions.
2. Find Custom Slot Type: EXAMPLE_QUESTIONS, and choose Copy to Clip Board:
   
3. Sign in to the Amazon Developer Console, open your QnABot Alexa skill, go to InteractionModel, and replace the values for the EXAMPLE_QUESTIONS Custom Slot Type by pasting the new values from your clipboard. Choose Save to save the updates.

Using CloudWatch to monitor and troubleshoot

QnABot metrics and logs are available using an Amazon CloudWatch dashboard.

1. Launch the dashboard using the DashboardUrl on the Outputs tab of the master CloudFormation stack.
   
2. When troubleshooting QnABot responses to your questions, trace the request and response using the logs created by the Fulfilment Lambda function.
   1. Choose the menu tool in the upper right of the Fulfilment Lambda widget, choose View logs, and choose the AWSLambda function.
      
   2. Inspect the log messages. Each interaction with QnABot is delimited by START and END messages. Between these messages you’ll be able to find insights into how QnABot processes the question.
      

Exporting and importing answers

The QnABot Content Designer allows you to export and import your content using JSON files.

Use the export feature to create backup versions of your content that you can use to restore if you accidentally delete items or need to go back to a previous version. You can also use exported JSON files to load content into another instance of your bot to help with testing or blue/green deployments.

Or, you might want to automatically create JSON content files using scripts which scrape and reformat content using existing sources like FAQ web pages or documents.

Try it! Let’s export all the items that are in the QnABot category (items whose ID starts with “QnABot”).

1. Log in to the Content Designer, choose Questions, and enter QnABot in the Filter to display just the items with IDs that start with the characters “QnABot”:
   

2. Now, choose Import /Export, and Export Filtered to generate and save a JSON file containing the filtered items:
3. Open the exported file in a text editor and inspect the JSON structure.
   

4. Add a new item to the qna list, as shown in the following example, and save the file.

   {
   "qid": "QnABot.003",
   "q": [
                           "What can Q and A bot do"
   ],
   "a": "You can integrate it with your website to provide quick and easy access to frequently asked questions. Use it with Alexa to provide hands free answers in the kitchen, in the factory or in the car. Since it can display images too, use it to provide illustrations and photographs to enrich your answers.",
   "r": {
   "title": "",
   "imageUrl": ""
   }
   },
   

5. In the Content Designer, choose Import/Export, and Import Local. Import your modified JSON file from your local computer. NOTE: Importing items with the same ID as an existing item will overwrite the existing item with the definition contained in the JSON file.
6. In the Content Designer, enter QnABot in the filter field, and inspect the newly imported item, QnABot.003.

Adding Images to your answers

You can easily augment your answers with image attachments that can be displayed on a user’s Lex Web Client UI, Amazon Alexa smartphone app, or new Amazon Echo Show device touch screen. Use images to display maps, diagrams, or photographs to depict places and products relevant to the question.

1. Log in to the Content Designer, and choose Add
2. Enter ID: Alexa.001
3. Enter question: What is an Amazon Echo Show
4. Enter answer: Echo Show brings you everything you love about Alexa, and now she can show you things. She is the perfect companion for Q and A Bot.
5. Enter attachment:
   1. Title: Echo Show
   2. ImageUrl: http://ift.tt/2xlWAy2
6. Save the new item.
7. Use the Web UI to ask: “What is an Echo Show?”
   You’ll see the photograph displayed in the web UI chat.
   
8. Or, use an Amazon Echo or Echo Dot to say: “Ask Q and A, What is an Echo Show?” The card shown in the Alexa smartphone app shows the photo attachment.
   
9. Or, best yet, use an Amazon Echo Show to say: “Ask Q and A, What is an Echo Show?”
   The photo attachment is displayed on the Echo Show’s touch screen.
   

Use Amazon S3 or a photo sharing service to host your images (see hosting images).

Adding style to QnABot

As you define your content, be sure to make your answers light and engaging. Try to adapt a consistent style in your answers. Use images to augment verbal answers. Add flourish to make your bot friendly and stylish!

For example, teach it how to respond to chit-chat like ‘Hello,’ Bye-bye,’ ‘Thanks’ and more.

This is easy to try:

1. Log in to the Content Designer, and choose Add.
2. Enter ID: BotStyle.001
3. Enter question: Hello
4. Enter answer: Greetings friendly human! Ask me a question. Try to stump me.
5. Save the new item.
6. Use the Web UI, or any Alexa device to say “Hello” to your bot, and listen to it respond.

You should also use this technique to teach QnABot to respond with some brief instructions when your users say “Help.”

Looking Under the Hood

Let’s take a quick look at how QnABot works. Here’s a diagram of the main architectural components and how they fit together at a high level.

The following sections explain how all these architectural components work together for different modes of interacting with QnABot. For additional detail, see the source code and READMEs in the GitHub repository, and use the AWS console to explore the configuration of each of the resources.

Lex Web Client UI

Amazon Lex allows conversational interfaces to be integrated into applications like our Lex Web Client. A Lex bot uses intents to encapsulate the purpose of an interaction, and slots to capture elements of information from the interaction. Since QnABot has a single purpose, to answer a user’s question, it defines just one intent. This intent has a single slot which is trained to capture the text of the question (see the discussion of “Tuning the Bot’s Natural Language Processing” earlier). To learn more about how Amazon Lex bots work, and to understand the concepts of intents, slots, sample values, fulfillment functions, and more, see the Amazon Lex Developer Guide.

The QnABot Lex web client is deployed to an Amazon S3 bucket in your account, and accessed via an Amazon CloudFront distribution.

An Amazon API Gateway endpoint sitting behind the same CloudFront distribution provides run time configuration. Using this configuration, the web client connects to Amazon Cognito to obtain temporary AWS credentials, and then connects with the Amazon Lex service.

When you use the Lex web client to ask QnABot a question, for example, “How can I include pictures in Q and A Bot answers?”, the web client interacts with the QnABot Lex bot. The Lex bot then invokes the Bot fulfillment Lambda function in your AWS account with the transcribed question captured as a slot.

The Bot fulfillment Lambda function generates an Amazon ES query containing the transcribed question. The following example shows the query. The query attempts to find the best match from all the questions and answers you’ve previously provided, using Amazon ES relevance scoring. Matches found in the text of stored questions are given greater weight than matches found in the text of the stored answers.

{
    "size": 10,
    "from": 0,
    "query": {
        "multi_match": {
            "query": "How can I include pictures in Q and A Bot answers",
            "fields": [
                "q^2",
                "a"
            ]
        }
    }
}

The Bot fulfillment Lambda function uses the answer field from the item with the highest score to create a response to send back to Lex. If the item also contains an image attachment, a response card is also sent back in the response. The Lex web client responds to the user with either voice or text depending on the mode of operation, and also displays any images attached to the answer.

Alexa Enabled Devices

Alexa devices interact with QnABot using an Alexa skill. Like a Lex bot, an Alexa skill also uses intents to encapsulate the purpose of an interaction, and slots to capture elements of information from the interaction. The QnABot Alexa skill mirrors the setup of the Lex bot described above, defining a single intent with a single slot.

The Alexa QnA skill uses the same Bot Fulfillment Lambda function as the Lex bot. When you ask a question, for example, “Alexa, ask Q and A, How can I include pictures in Q and A Bot answers?”, your Alexa device interacts with the skill you created, which in turn invokes the Bot fulfillment Lambda function in your AWS account, passing the transcribed question as a parameter.

The Bot fulfillment Lambda function queries Amazon ES items as described earlier. The Bot fulfillment Lambda function uses the answer field from the item with the highest score to create a response to send back to Alexa. If the item also contains an image attachment, a card is also sent back in the response, causing the image to be displayed in the Alexa smartphone app, or, if an Echo Show is used, on the touchscreen.

Content Designer UI

The QnABot Content Designer UI is also deployed to an Amazon S3 bucket accessed via the same Amazon CloudFront distribution as the Client UI, and it too retrieves configuration from an API Gateway endpoint.  The Content Designer website requires the user to sign in with credentials defined in a Cognito user pool.

Using temporary AWS credentials from Cognito, the Content Designer UI interacts with secure API Gateway endpoints backed by the Content Designer Lambda functions. All interactions with Amazon ES and Amazon Lex are handled by these Lambda functions.

Customizing and enhancing QnABot

Fork the QnABot GitHub repository, enhance the code, and send us pull requests so we can incorporate and share your improvements!

Here are a few suggestions for features you might want to implement:

• Usage metrics: Implement custom metrics to track which questions and topics are accessed most frequently.
• Capture user feedback: Allow the user to report errors in answers, or to report when a given answer didn’t address the question that was asked.
• Multiple answers: Allow the user to request a different answer by saying “Try again,” or “Next” to instruct QnABot to reply with lower ranked answers. Useful when questions are ambiguous or when there is more than one correct answer.
• Topic context memory: Add support for follow-up questions. For example, after asking, “What is the Echo Show?” the follow-up question, “How much does it cost?” would be assumed to apply to the Echo Show context.
• Scrape existing FAQ pages or documents: Write scripts to capture content from existing suitable sources. Automate, or semi-automate the creation of the QnABot JSON format files for importing.
• Extend QnABot to access public knowledge base services: Tap into Wikipedia, Evi, WolframAlpha, or other public knowledge servers by extending the Bot fulfillment Lambda function to access additional external information sources.
• Integrate QnABot with Amazon Connect: Lower your support center call wait times by automating answers to frequently asked questions, using QnABot within an Amazon Connect contact flow. (See Amazon Connect and Amazon Lex Integration.) You can also use popular messaging platforms to interact with QnABot.

About the Amazon Lex web UI

The QnABot web UI uses the aws-lex-web-ui GitHub repository. This repository provides the code that enables the web page to interact with the Amazon Lex bot, and handles audio capture and playback on webRTC-supported  web browsers.

You can use the Lex web UI to integrate QnABot as a component into your own website:

For more information, review the README file.

Uninstalling QnABot

When you’re done experimenting, in the CloudFormation console, choose and delete the master QnABot stack. This removes all of the AWS resources associated with the bot. Be sure to first export any QnA items that you want to reuse later.

Summary

This blog shares our sample Question and Answer bot, QnABot. We explain how you can use QnABot to create a natural language interface that works with both text and voice using Amazon Lex via your web browser or using any Alexa-enabled device.

Now you can try it with your own questions and answers! Please use the comments link to share your experiences with the QnABot – we’d love to hear from you!

---

Additional Reading

• Building Better Bots Using Amazon Lex
• Build Skills with the Alexa Skills Kit
• Sample Amazon Lex Web Interface

---

About the Authors

Bob Strahan is a Senior Consultant for AWS Professional Services. John Calhoun is an Associate Solutions Architect for the AWS Public Sector Partners team. They work with our customers and partners to provide leadership on a variety of projects, helping them shorten their time to value when using AWS.

 

 

 

 

Powered by WPeMatico

The post Create a Question and Answer Bot with Amazon Lex and Amazon Alexa appeared first on Artificial Intelligence Solutions.

Read More...

Get Started with Deep Learning Using the AWS Deep Learning AMI

Read More...

Enhancements to the Amazon Lex Console Let You Test Your Bot for Better Troubleshooting

Building your chatbot in the Amazon Lex console takes just a few steps, and testing your bot is just as easy. We’ve made enhancements to the Test window of the Amazon Lex console which now provides you more details during testing and enables easier bot troubleshooting. Once you’ve built a bot to test, the Test window is enabled, located to the far right of the Amazon Lex console. As you chat with your bot from within the test window, the parsed service response is dynamically displayed on each turn of the conversation.

In the Summary view, you can review the slot values for the intent being fulfilled. This allows you to keep track of the conversation flow.

In the Detail view, you can review the parsed response from the chatbot. The detail gives you deeper visibility into the bot interaction and the current state of the dialog as you troubleshoot and debug your chatbot.

For more information on updates to the Test window, please refer to the Amazon Lex documentation.

---

Next Steps

What does it take to develop a chatbot using Amazon Lex? Learn how to build better bots with our two part tutorial!

 

Powered by WPeMatico

The post Enhancements to the Amazon Lex Console Let You Test Your Bot for Better Troubleshooting appeared first on Artificial Intelligence Solutions.

Read More...

NIPS 2017 Challenge Pushes Deep Learning to Improve Surgical Outcomes

AWS, NVIDIA, Stanford, EPFL, and UC Berkeley have joined forces to tackle clinical problems in biomechanics. The NIPS 2017 Learning to Run challenge brings together over 300 researchers, engineers, and enthusiasts from around the world to apply deep learning to medical research. The challenge will culminate at the international AI conference Neural Information Processing Systems (NIPS) 2017 in Long Beach, CA.

Through this collaboration, an open source environment was built to simulate adaptation of human motor control for new biomechanical settings. Algorithms solving this problem can be used for modeling how we respond to musculoskeletal surgeries for different kinds of prosthetics and exoskeletons.

With recent advancements in deep learning and deep reinforcement learning, and great successes in applying it to gaming and autonomous driving, researchers are investigating what role these methods can play in medical research. To spur participation in the challenge, AWS has offered $30,000 (USD) in cloud credits and $300 (USD) each to 100 top performing participants. Plus, you still have a chance to compete for an NVIDIA DGX Station!

Simulating surgical outcomes with deep learning

Prediction of surgical outcomes has always been a great challenge in medicine. In the age of big data this problem can finally be approached with quantitative methods. However, looking at large datasets of patients (their history and responses), tuning the parameters of surgeries, or experimenting with new ideas is difficult in practice and unethical.

Simulations provide a potential alternative to real surgeries. Software that is already available and widely adopted, such as OpenSim, greatly simplifies decision making. Practitioners can use the software to understand which muscles or bone deformities affect gait pathologies in individual cases. This helps doctors to tune surgery parameters and systematically improve outcomes.

While simulations can help in very specific and simple cases, they don’t allow us to predict the outcome holistically, since this would require understanding how the human brain adapts to the outcome of the surgery. However, with recent advancements in artificial intelligence, we can finally approach this problem.

Use of reinforcement learning in healthcare

Using deep learning we can tell if a mark on the skin is skin cancer as well as a doctor can. Deep learning can detect breast cancer earlier than ever, and can better understand gene expression data.

These and many other results motivate further exploration of deep learning methods in healthcare. Most of the approaches are based on a similar principle: collect the data, ask experts to annotate the data, and try to imitate these annotations using machine learning algorithms.

Although this approach works great in practice, its human-centric design only allows us to reach human accuracy. To go beyond these frameworks, we need to employ stochastic perturbations in the space of possible solutions. This insight allowed researchers from DeepMind to beat the champion of Go. Through our NIPS challenge, we want to understand if the same approach can help us predict outcomes of a treatment better than a doctor could.

The results so far…

Control of the human body is a very complex and sophisticated process due to the complexity of our muscles, nonlinear actuation, noise in the neural signals and many other problems. Participants of the challenge have been struggling with the problem of control, and many of them have already succeeded in making the biomechanical model run at around 3m per second!

While the results are not yet directly applicable in medical practice, we can clearly see that the approach can be useful in the near future. With two more months to go, we are confident that our crowd-sourced research will lead to great innovation in biomechanics.

Check out the top solution so far from Jgeek. He trained a controller that can not only pass the obstacles, but also synthesize movement close to a slow human run. Visit the leaderboard to see how the competition is shaping up. Have an idea for how you could use deep learning to tackle problems in medical research? There’s still time, so enter the NIPS 2017 Learning to Run challenge for your chance to get free AWS credits, win an NVIDIADGX Station and put deep learning to the test.

Powered by WPeMatico

The post NIPS 2017 Challenge Pushes Deep Learning to Improve Surgical Outcomes appeared first on Artificial Intelligence Solutions.

Read More...

Export your Amazon Lex bot schema to the Alexa Skills Kit

You can now export your Amazon Lex chatbot schema into the Alexa Skills Kit to simplify the process of creating an Alexa skill.

Amazon Lex now provides the ability to export your Amazon Lex chatbot definition as a JSON file that can be added to the Alexa Skills Kit (ASK). Once you add the bot schema file into ASK, you can build an Alexa skill with it, for use on the Amazon Echo, Amazon Dot, Amazon Look, Amazon Tap, Amazon Echo Show and 3^rd party Alexa enabled devices. The JSON configuration file contains the structure of your Amazon Lex chatbot, including the intent schema with utterances, slots, prompts and slot-types. The export functionality simplifies the process of creating an Alexa skill from an Amazon Lex chatbot.

To create an Alexa skill, you can use your Amazon Lex bot definition file from within the ASK portal, using the following steps.

From the Amazon Lex console

1. Once you have created, built, and published your bot, navigate back to the page containing the list of bots which you have currently developed.
2. Using the radio buttons, select the bot you would like to export. The Actions tab will become enabled at the top of the page once a bot is selected.
   
3. From the Actions tab, select Export from the drop-down menu items, and following the modal dialogs, select Alexa Skills Kit as your Platform, and choose your version.
4. Click Create. A zip file will be generating containing your JSON file of your bot schema.

Your Amazon Lex bot schema is now ready to be used in an Amazon Alexa skill!

From the Amazon Alexa Developer Portal

1. Log in to the Alexa developer portal and navigate to the Interaction Model tab, to Launch the Alexa Skill Builder (currently in Beta).
   
2. From the Skill Builder, paste the unzipped bot schema file into the Code Editor of your Alexa skill. (You can also drag-drop the JSON file.)
   
3. Create a custom back-end for your skill in AWS Lambda.
4. Connect the Lambda Function to your Interaction Model.
5. From there, you can select “Build Model” from the top menu and complete any other updates to your skill information, interaction model, or configuration. Then Submit your skill for publication!

For more information on exporting your chatbot schema from Amazon Lex to simplify skill creation through the Alexa Skills Kit, visit the Amazon Lex documentation and FAQs page for more details. For more information about creating an Alexa skill, see the Alexa Skills Kit.

 

Powered by WPeMatico

The post Export your Amazon Lex bot schema to the Alexa Skills Kit appeared first on Artificial Intelligence Solutions.

Read More...

Bring Machine Learning to iOS apps using Apache MXNet and Apple Core ML

With the release of Core ML by Apple at WWDC 2017, iOS, macOS, watchOS and tvOS developers can now easily integrate a machine learning model into their app. This enables developers to bring intelligent new features to users with just a few lines of code. Core ML makes machine learning more accessible to mobile developers. It also enables rapid prototyping and the use of different sensors (like the camera, GPS, etc.) to create more powerful apps than ever.

Members of the MXNet community, including contributors from Apple and Amazon Web Services (AWS), have collaborated to produce a tool that converts machine learning models built using MXNet to Core ML format. This tool makes it easy for developers to build apps powered by machine learning for Apple devices. With this conversion tool, you now have a fast pipeline for your deep-learning-enabled applications. You can move from scalable and efficient distributed model training in the AWS Cloud using MXNet to fast run time inference on Apple devices.

To support the release of the converter tool we decided to build a cool iOS app. We were inspired by a previous AWS AI Blog post, Estimating the Location of Images Using MXNet and Multimedia Commons Dataset on AWS EC2, that showcases the LocationNet model to predict the location where pictures were taken.

In this blog post, we explain how to set up an environment to convert MXNet models into Core ML, convert an existing model, and then import it into a sample iOS app written in Swift. The iOS app feeds a picture to the model, which predicts the location where that image was taken, and shows it on an interactive map. For performance, we recommend that you run the app on a physical iOS device (e.g., an iPhone) installed with iOS 11 beta, but you can also try it on the Simulator that comes with the Xcode 9.0 beta.

Note that at the time of writing Xcode 9, iOS 11, and Core ML are still in beta and you need an Apple Developer Program account to download Xcode and iOS. However, after they are all released publicly later this year, you can use the App Store on your Mac and Software Update on your iOS device to obtain them.

Installation of MXNet and the converter tool

The tool was installed and tested on the macOS High Sierra 10.13 beta 8. However, as long as you don’t run inferences on a Core ML model on your Mac, you can run the converter on macOS El Capitan (10.11) and later.

To run the converter tool, you need to have Python 2.7 installed.

Run the following command to install the MXNet framework, and the mxnet-to-coreml tool:

pip install mxnet-to-coreml

Conversion of the MXNet model

The LocationNet model was trained using MXNet on a single p2.16xlarge Amazon EC2 instance with geo-tagged images from the AWS Multimedia Commons dataset. It is shared publicly on the MXNet Model Zoo.

As with any MXNet model, LocationNet has two parts:

• A JSON file containing the model definition
• A binary file containing the parameters

Go ahead and download the .json model definition and the .params model parameters files stored on Amazon S3.

In addition, you will need to download the classes file grids.txt from the GitHub repository, which contains the geographic cells used for training the model. It was created with the training data using Google’s S2 Geometry Library. Each line in this text file is in the form of S2 Cell Token, Latitude, Longitude (e.g., 8644b594 30.2835162512 -97.7271641272). The Swift code in the iOS app will drop the S2 Cell Token information and only use the coordinates.

As explained on the GitHub repository for the conversion tool, we will now convert the model.

After you have everything downloaded in the same directory, run this command:

mxnet_coreml_converter.py --model-prefix='RN101-5k500' --epoch=12 --input-shape='{"data":"3,224,224"}' --mode=classifier --pre-processing-arguments='{"image_input_names":"data"}' --class-labels grids.txt --output-file="RN1015k500.mlmodel"

Internally, the model is first loaded by MXNet recreating the entire symbolic graph in memory. The converter walks through this symbolic graph converting each operator into its Core ML equivalent. Some of the supplied arguments to the converter are used by MXNet to generate the graph, while others are used by Core ML either to pre-process the input (before passing it to the neural network) or to process the output of the neural network in a particular way.

You should see the converter tool processing the multiple layers of the model, and then confirm SUCCESS with the name of the file generated. You will import the resulting file RN1015k500.mlmodel into your Xcode project in later stages.

With Xcode installed, if you double-click this model file, you can get more information about it, such as its size, name, and parameters, which would usually be used within your Swift code:

Download and configure the code for the iOS app

The sample iOS app was written in Swift using Xcode 9 beta 6 on a Mac running macOS Sierra 10.12.6. The app was tested on an iPhone 7 running iOS 11 beta 8.

We decided to use Apple’s new Vision framework to facilitate the use of Core ML with images because it automatically converts an image to the format and size that the Core ML model expects. Vision provides solutions to computer vision challenges through a consistent interface, and its features include face tracking, face detection, landmarks, text detection, rectangle detection, barcode detection, object tracking, and image registration.

We used the following resources to get started:

• Integrating a Core ML Model into Your App
• Apple Vision framework code samples from Matthijs Hollemans’ CoreMLHelpers

Now let’s get going with building the app!

Go ahead and download the iOS sample app source code from this GitHub repository: MXNet2CoreML_iOS_sample_app.

Open MXNet2CoreML.xcodeproj with Xcode.

Drag and drop the file RN1015k500.mlmodel that you generated earlier into your Xcode project navigator as shown on the right in the following picture, and make sure to tick the Target Membership checkbox for the current project.

In case you didn’t install the converter tool and you just want to try the iOS app, we uploaded the Core ML model RN1015k500.mlmodel here. Download the file then drag and drop it to the Xcode project navigator.

Run the app, see some magic

As stated previously, we recommend that you test the app on a physical device running iOS 11 (still in beta at the time of writing).

You can also run it in the Xcode Simulator but the performance and animations will not be great, especially if you pan or zoom in the map area.

Remember to sign the app with your Team account if you decide to run it on a physical iOS device, as shown in the following screenshot.

As we said in our preliminary notes, you will need an Apple Developer account for this to work.

Press play to build your app and run it on the iPhone.

The app will install on the iPhone and you should see the following screen.

It contains 3 sections:

• The top section displays a picture taken somewhere in the world. Swipe left or right on the picture to display one of the 3 built-in images. It’s relatively easy for the human eye to recognize these locations, but it is impressive to see that the model predicts really accurate locations when there is no GPS data embedded in these images!
• The middle section displays 3 real-time predictions with “1” being the most probable location with a higher percentage of probability. We purposely decided to only display the top 3 predictions out of the hundreds of predictions that the model produces.
• The bottom section displays an interactive map with pins for each of the 3 locations that were predicted by the model. You can zoom and pan at your leisure to explore the area where the pins are located.

Screenshot 1:

Screenshot 2:

Summary

As this blog post demonstrates, you can now build and train sophisticated machine learning models on AWS using MXNet, convert them to Core ML format with the MXNet to Core ML converter tool, and quickly import them into Xcode. Now you can start experimenting and creating new amazing capabilities in your apps for iOS or other Apple devices.

What’s next?

If you want to try the app with your own picture saved on your computer, just rename your picture 1.jpg, delete the existing file from the Xcode project navigator, and drag and drop it. We discussed how to do this in the section on the Core ML model.

You could also develop the sample app a bit further by implementing a camera function that allows you to take pictures within the app or load from the camera roll, and perform real-time location prediction on images you have already taken or would take on the spot.

We are excited to discover the different ways this blog post will inspire you.  If you have questions, comments or suggestions, please post them in the Comments section below.

Have fun!

---

Additional Reading

Learn how to use a pre-trained model created with Apache MXNet to estimate the location of images on AWS EC2.

---

About the Authors

Sebastien Menant is a Solutions Architect for Amazon Web Services. He started working with enterprise customers in Sydney and now helps customers in the San Francisco Bay Area innovate on AWS. He previously worked at Apple where he grew his interest for iOS software development and the Swift programming language.

 

 

 

Pracheer Gupta is a member of the AWS Deep Learning team in Palo Alto. He is currently a technical lead for Apache MXNet applications, helping customers build intelligent applications on top of MXNet. In the past, he worked on building a Paxos-based distributed lock manager, which became the foundational building block for much of AWS. In his spare time, he loves reading up on history and psychology.

 

 

 

 

 

Powered by WPeMatico

The post Bring Machine Learning to iOS apps using Apache MXNet and Apple Core ML appeared first on Artificial Intelligence Solutions.

Read More...

Apple Core ML and Keras Support Now Available for Apache MXNet

We’re excited about the availability of Apache MXNet version 0.11. With this release, MXNet hit major milestones, both in terms of community development and as an incubating Apache project. Contributors—including developers from Apple, Samsung and Microsoft—committed code to this release. There are over 400 contributors on the project so far. The project has now fully migrated its codebase to Apache and has made its first official release as an incubating project. We discussed some of the important features of this release in a prior blog. This blog recaps the highlights.

Use MXNet models to build machine learning into apps for iOS, macOS, watchOS and tvOS

With the release of Core ML by Apple at WWDC 2017 developers can now easily integrate a machine learning model into their app, enabling them to bring intelligent new features to users with just a few lines of code. We’re already beginning to see capabilities such as augmented reality transform the way we experience our surroundings. As capabilities in the fast-moving AI space expand, developers will have access to new machine learning models that unlock new capabilities that enhance our experience.

Apple has contributed code to the Apache MXNet project to facilitate the availability of cutting edge models for app developers. MXNet now works with Core ML that enables developers to build and train machine learning models in the cloud with MXNet, and import them right into Xcode so you can easily build intelligent new features in your apps. You can choose from MXNet’s model zoo of pre-trained models for various applications or build your own model. This release provides you with a tool (in preview) to convert MXNet models to the Core ML format. To import an MXNet model into Apple’s Core ML format, you’ll need to install the converter and run the Python command to convert a trained model. Installing the converter is just one simple command:

pip install mxnet-to-coreml

Follow this tutorial to learn how to build a simple iOS application powered by machine learning that identifies the geo-location of places in pictures. For instructions and an end to end example, please visit this GitHub repo.

MXNet for Keras v1.2

Keras is a popular high-level library for building neural networks. It provides users with an easy-to-use frontend interface that can drive various machine learning framework backends. MXNet is now available as a backend choice for users running Keras v1.2. MXNet gives you a simple way to set up multi-GPUs for training and superior performance that scales near-linearly. To read more about the benefits of using MXNet as a backend for Keras, refer to NVIDIA’s blog. You can install Keras v1.2 with the MXNet interface with the following command:

pip install keras-mxnet

Detailed instructions for setting up a training job with multiple GPU’s can be found at this GitHub repo.

MXNet’s 0.11 release adds several key features, API enhancements, performance improvements, and bug fixes. This blog post only covered some of the major features in this release. You can find more details in the full release notes.

We are excited to see the growth and momentum of this community-owned project. These advancements wouldn’t be possible without the hard work of the project’s contributors!

---

Next Steps

Learn how to set up an environment to convert Apache MXNet models into Apple’s Core ML, convert an existing model, and then import it into a sample iOS app.

 

Powered by WPeMatico

The post Apple Core ML and Keras Support Now Available for Apache MXNet appeared first on Artificial Intelligence Solutions.

Read More...

Use Synonyms and Slot Value Validation in your Amazon Lex Chatbots

You can now provide synonyms for slot values in Amazon Lex. With the synonym functionality, you can specify multiple synonyms for a slot value in your chatbot. The synonyms specified are resolved to the corresponding slot values. For example, if the slot value is “comedy”, with “funny” and “humorous” specified as synonyms, then user input of “funny” will now be resolved to “comedy”. You can optimize your business logic to implement code for just the slot value instead of entire set of synonyms. In addition, the chatbot automatically resolves minor variants of the slot value to the original value without the user having to explicitly specify the synonym. For example, if “pineapple” is one of the slot values and the user value is “pineapples” then the slot is resolved to the value “pineapple”.

You can also now validate user input and restrict it to a fixed set of values. You can enable this by configuring the slot resolution strategy so that the slot is resolved only if the user value is the same as one of the slot values or the synonym.

These new features are available with Amazon Lex in the US East (N. Virginia) AWS region. Visit the Amazon Lex documentation to learn more about these features.

---

Additional Reading

Learn how to build better bots using Amazon Lex – a two part series.

 

Powered by WPeMatico

The post Use Synonyms and Slot Value Validation in your Amazon Lex Chatbots appeared first on Artificial Intelligence Solutions.

Read More...