Amazon SageMaker Inference for Custom Amazon Nova Models

Amazon SageMaker Inference now supports custom Amazon Nova models, providing a production-grade, configurable, and cost-efficient managed inference service to deploy and scale full-rank customized Nova models.

Introduction to Custom Nova Models

Since the launch of Amazon Nova customization in Amazon SageMaker AI at AWS NY Summit 2025, customers have been asking for the same capabilities with Amazon Nova as they do when they customize open weights models in Amazon SageMaker Inference.

Features of Amazon SageMaker Inference for Custom Nova Models

With Amazon SageMaker Inference for custom Nova models, you can reduce inference cost through optimized GPU utilization using Amazon Elastic Compute Cloud (Amazon EC2) G5 and G6 instances over P5 instances, auto-scaling based on 5-minute usage patterns, and configurable inference parameters.

Deploying Custom Nova Models on SageMaker Inference

To deploy custom Nova models on SageMaker Inference, you can use the SageMaker Studio or SageMaker AI SDK. In the SageMaker Studio, choose a trained Nova model in Models in your models in the Models menu. You can deploy the model by choosing Deploy button, SageMaker AI and Create new endpoint.

Configuring Inference Parameters

You can configure inference parameters such as instance type, instance count, and max instance count. You can also set advanced configurations about context length, concurrency, and batch size for optimizing the latency-cost-accuracy tradeoff for your specific workloads.

Invoking Custom Nova Models

After the endpoint is created, you can send inference requests to generate predictions from your custom Nova model. Amazon SageMaker AI supports synchronous endpoints for real-time with streaming/non-streaming modes and asynchronous endpoints for batch processing.

Example Code

The following code sample creates a SageMaker AI model that references your Nova model artifacts and an endpoint configuration that defines how the model will be deployed.

# Create a SageMaker AI model
model_response = sagemaker.create_model(
    ModelName= 'Nova-micro-ml-g5-12xlarge',
    PrimaryContainer={
        'Image': '708977205387.dkr.ecr.us-east-1.amazonaws.com/nova-inference-repo:v1.0.0',
        'ModelDataSource': {
           'S3DataSource': {
               'S3Uri': 's3://your-bucket-name/path/to/model/artifacts/',
               'S3DataType': 'S3Prefix',
               'CompressionType': 'None'
           }
        },
        # Model Parameters
        'Environment': {
            'CONTEXT_LENGTH': 8000,
            'MAX_CONCURRENCY': 16,
            'DEFAULT_TEMPERATURE': 0.0,
            'DEFAULT_TOP_P': 1.0
        }
    },
    ExecutionRoleArn=SAGEMAKER_EXECUTION_ROLE_ARN,
    EnableNetworkIsolation=True
)
print("Model created successfully!")

Conclusion

Amazon SageMaker Inference for custom Amazon Nova models provides a production-grade, configurable, and cost-efficient managed inference service to deploy and scale full-rank customized Nova models. With this feature, you can reduce inference cost through optimized GPU utilization and configurable inference parameters.

Frequently Asked Questions

• What is Amazon SageMaker Inference for custom Amazon Nova models?
• How do I deploy custom Nova models on SageMaker Inference?
• What are the features of Amazon SageMaker Inference for custom Nova models?
• How do I configure inference parameters for custom Nova models?
• How do I invoke custom Nova models?