|
Voice interfaces are essential to increase customer experience in different areas, such as automation of customer support calls, game games, interactive education and language teaching. However, there are challenges when creating voice permits.
Traditional approaches to creating voice applications require complex orchestrations of multiple models, such as speech recognition to convert speeches, language models to understand and generate centers and text to language to convert the text back to sound.
This fragmented approval not only leads to the AM developmental force, but also to maintain a key language context, such as the tone, the proslum and the style of speaking that are essential for natural conversations. This can affect AI conversational applications that require low latency and nuanced understanding of verbal and non -verbal stimuli to handle fluid dialogue and natural fat.
In order to make speech applications more efficient, we introduced Amazon Nova Sonic today, a new addition to Amazon Nova Family of Foundation Models (FMS) available in Amazon Bedrock.
Amazon Nova Sonic unifies speech and generation understanding into a single model that developers can use to create natural conversational experiences with people similar to human conversational AI with low latency and top price performance. This integrated approach strengthens and reduces the complexity in creating conversational applications.
Its unified model architecture provides expressive speech generation and real -time text transcription without required to require a separate model. The result is an adaptive reaction of speech, which dynamically its supply based on the temporary, such as pace and Timbre, input speech.
When using Amazon Nova Sonic, developers have access to function calls (also known as the use of a tool) and agent work flows for interaction with external services and APIs and to perform tasks in the customer’s environment, including business data by searching (RAG).
At the opening of Amazon Nova Sonic, it provides robust speech understanding for American and British English across various speaking styles and acoustic conditions, with other Cuming languages soon.
Amazon Nova Sonic is developed with a responsible AI in the forefront of innovation and represents the protection of the assembly for moderation of content and watermark.
Amazon Nova Sonic in Action
The scenario of this demo is a contact center in the telecommunications industry. If you want to be tailored, it will appeal to improve your prescription plan, and Amazon Nova Sonic processes conversation.
Using the tool, the model can interact with other systems and use agent rags with knowledge of Amazon Bedrock to collect updated customer information such as account details, subscription plans and pricing information.
The demo shows streaming of speech input transcription and shows streaming speech centers as text. The feeling of conversation is displayed in two ways: illustration of the time chart as evolves, and the PIUS graph represents the overall distribution. There is also an AI Insights section providing context tips for the Call Center agent. Other metrics of the interests listed in the web interface are the overall division of the call time between the customer and the agent and the average response time.
During an interview with a support agent, you can observe through metrics and hear in your voice how the customer improves feeling.
The video included the example of Howon Nova Sonic processes the interruption smoothly, stopping to listen and then naturally continuing the conversation.
Now let’s explore how you can integrate voice skills into your applications.
Using Amazon Nova Sonic
If you want to start with the Amazon Nova Sonic, you must first turn on the access of the Amazon Bedrock model, as in how you would allow other FMS. Scroll to Access to the model Part of the navigation pane, find Amazon Nova Sonic under Amazon Models and enable it for your account.
Amazon Bedrock provides a new two -way streaming interface (InvokeModelWithBidirectionalStream) to help you implement conversation experience with low latency at the peak of the HTTP/2 protocol. With this API you can stream the audio input into the model and receive sound output in real time, so the conversation flows naturally.
With this ID model you can use Amazon Nova Sonic with the new API: amazon.nova-sonic-v1:0
After the initialization session, where you can configure the inference parameters, the model works through an architecture of events on the input and output tors.
There are three key types of events in the input current:
Quick system – you want to set the overall system prompt for conversation
Streaming audio inputs -Proofings continuous sound input in real time
Tool results processing – You want to send the result of using the tool back to the model (after using the tool is required at output events)
Similarly, there are three groups of events in the output currents:
Streaming of automatic speech recognition (ASR) The -speech-to-text transcript is generated, which contains the result of real-time speech recognition.
Handling the use of the tool – If there are events of the use of tools, they need to be processed here and the results feel back as input events.
Streaming the audio output -For real -time output audio playback, a buffer is required because the Amazon Nova Sonic generates sound faster than real -time playback.
You can find examples of using Amazon Nova Sonic in Amazon Nova Model Cookbook Restites.
Fast engineering for speech
When Crafting calls for Amazon Nova Sonic, your challenges should optimize the content for auditory understanding rather than visual reading and focus on conversational flow and clarity when you can hear it rather than see.
When defining roles for your assistant, focus on conversational attributes (such as warm, patient, brief) than text attributes (detailed, understanding, systematic). A good challenge for the baseline can be:
You are a friend. The user and you will engage in a spoken dialog exchanging the transcripts of a natural real-time conversation. Keep your responses short, generally two or three sentences for chatty scenarios.
More generally, when creating challenges for speech models, avoid visual formatting (such as reflecting points, tables, gold code blocks), voice characteristic adjustments (accent, age, gold singing) or sound effects.
What to know
Amazon Nova Sonic is available today at the AWS USA (N. Virginia). Visit Amazon Bedrock Awards and see price models.
Amazon Nova Sonic can understand speech in various speaking styles and create speech with expressive voices, including male sounding and female sounding voices, in various English accents, including Americans and British. Support for other languages will soon appear.
Amazon Nova Sonic processes the interruption of users elegantly without dropping the conversational context and is robust for background noise. The model supports the 32K 32K window for sound with rolling window for longer conversations and has a default session limit of 8 minutes.
The following AWS SDKS supports the new two -way API:
Python developers can use this new SDK experience that makes it easier to use two -way streaming capacity Amazon Nova Sonic. We are working to add support to other AWS SDKs.
I like to thank Reilly Manton and Chad Hendren, who created a demo with a contact center in the telecommunications industry, and Anuj Jauhari, who helped me understand the rich landscape in which speech models are deployed.
More examples can be found in Java, Node.js and Python in Amazon Nova Model Cookbook Repo, including common integration formulas, such as rag using Amazon Bedrock or Langchain.
If you want to know more, these articles that enter details on how to use a new two -way streaming API with convincing demonstrations:
Whether you are creating customer service solutions, language learning or other conversational experiences, Amazon Nova Sonic provides the basis for natural and involving voice interactions. If you want to get started, visit Amazon Bedrock today. If you want to learn more, visit the Amazon Nova User Guide.
– Danilo
How’s the Blog of news? Take this 1 minute survey!
(This survey is hosted by an external company. AWS processes your information as described in the AWS Privacy Notice. AWS will own data collected via this survey and will not share the collection of Lissel survey.)