Using responsible AI principles with Amazon Bedrock Batch Inference | AWS Machine Learning Blog

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

The recent announcement of batch inference in Amazon Bedrock enables organizations to process large volumes of data efficiently at 50% less cost compared to On-Demand pricing. It’s especially useful when the use case is not latency sensitive and you don’t need real-time inference. However, as we embrace these powerful capabilities, we must also address a critical challenge: implementing responsible AI practices in batch processing scenarios.

In this post, we explore a practical, cost-effective approach for incorporating responsible AI guardrails into Amazon Bedrock Batch Inference workflows. Although we use a call center’s transcript summarization as our primary example, the methods we discuss are broadly applicable to a variety of batch inference use cases where responsible considerations and data protection are a top priority.

Our approach combines two key elements:

This two-step process offers several advantages:

Throughout this post, we address several key challenges in responsible AI implementation for batch inference. These include safeguarding sensitive information, providing accuracy and relevance of AI-generated content, mitigating biases, maintaining transparency, and adhering to data protection regulations. By tackling these challenges, we aim to provide a comprehensive approach to responsible AI use in batch processing.

To illustrate these concepts, we provide practical step-by-step guidance on implementing this technique.

This solution uses Amazon Bedrock for batch inference to summarize call center transcripts, coupled with the following two-step approach to maintain responsible AI practices. The method is designed to be cost-effective, flexible, and maintain high responsible standards.

This two-step approach combines the efficiency of batch processing with robust responsible safeguards, providing a comprehensive solution for responsible AI implementation in scenarios involving sensitive data at scale.

In the following sections, we walk you through the key components of implementing responsible AI practices in batch inference workflows using Amazon Bedrock, with a focus on responsible prompting techniques and guardrails.

To implement the proposed solution, make sure you have satisfied the following requirements:

When setting up your batch inference job, it’s crucial to incorporate responsible guidelines into your prompts. The following is a concise example of how you might structure your prompt:

This prompt sets the stage for responsible summarization by explicitly instructing the model to protect privacy, minimize bias, and focus on relevant information.

For detailed instructions on how to set up and run a batch inference job using Amazon Bedrock, refer to Enhance call center efficiency using batch inference for transcript summarization with Amazon Bedrock. It provides detailed instructions for the following steps:

By following the instructions in our previous post and incorporating the responsible prompt provided in the preceding section, you’ll be well-equipped to set up batch inference jobs.

After the batch inference job has run successfully, apply Amazon Bedrock Guardrails as a postprocessing step. This provides an additional layer of protection against potential responsible violations or sensitive information disclosure. The following is a simple implementation, but you can update this based on your data volume and SLA requirements:

Key points about this implementation:

This approach allows you to benefit from the efficiency of batch processing while still maintaining strict control over the AI’s outputs and protecting sensitive information. By addressing responsible considerations at both the input (prompting) and output (guardrails) stages, you’ll have a comprehensive approach to responsible AI in batch inference workflows.

Although this example focuses on call center transcript summarization, you can adapt the principles and methods discussed in this post to various batch inference scenarios across different industries, always prioritizing responsible AI practices and data protection.

Although the prompt in the previous section provides a basic framework, there are many responsible considerations you can incorporate depending on your specific use case. The following is a more comprehensive list of responsible guidelines:

When implementing responsible AI practices in your batch inference workflows, consider which of these guidelines are most relevant to your specific use case. You may need to add, remove, or modify instructions based on your industry, target audience, and specific responsible considerations. Remember to regularly review and update your responsible guidelines as new challenges and considerations emerge in the field of AI ethics.

To delete the guardrail you created, follow the steps in Delete a guardrail.

Implementing responsible AI practices, regardless of the specific feature or method, requires a thoughtful balance of privacy protection, cost-effectiveness, and responsible considerations. In our exploration of batch inference with Amazon Bedrock, we’ve demonstrated how these principles can be applied to create a system that not only efficiently processes large volumes of data, but does so in a manner that respects privacy, avoids bias, and provides actionable insights.

We encourage you to adopt this approach in your own generative AI implementations. Start by incorporating responsible guidelines into your prompts and applying guardrails to your outputs. Responsible AI is an ongoing commitment—continuously monitor, gather feedback, and adapt your approach to align with the highest standards of responsible AI use. By prioritizing ethics alongside technological advancement, we can create AI systems that not only meet business needs, but also contribute positively to society.

Ishan Singh is a Generative AI Data Scientist at Amazon Web Services, where he helps customers build innovative and responsible generative AI solutions and products. With a strong background in AI/ML, Ishan specializes in building Generative AI solutions that drive business value. Outside of work, he enjoys playing volleyball, exploring local bike trails, and spending time with his wife and dog, Beau.

Yanyan Zhang is a Senior Generative AI Data Scientist at Amazon Web Services, where she has been working on cutting-edge AI/ML technologies as a Generative AI Specialist, helping customers use generative AI to achieve their desired outcomes. Yanyan graduated from Texas A&M University with a PhD in Electrical Engineering. Outside of work, she loves traveling, working out, and exploring new things.

Loading comments…