Sold by: HolboxAI
Deployed on AWS
Deploy an OpenAI-compatible vLLM endpoint serving Qwen4B directly into your own AWS account using AWS CloudFormation. No manual GPU, Docker, or vLLM setup required - receive an endpoint URL and API key in minutes while your model, compute, and data remain in your AWS environment.
Overview
Qwen4B OpenAI-Compatible Inference Endpoint lets you deploy a production-ready vLLM server serving Qwen4B directly into your own AWS account. Subscribe through AWS Marketplace, launch the included CloudFormation stack, and automatically provision a GPU EC2 instance, configure the inference service, and receive an OpenAI-compatible endpoint URL and API key through CloudFormation outputs. No manual EC2 provisioning, Docker configuration, or vLLM installation is required.
The deployment runs entirely within your AWS environment, so your model, compute resources, API traffic, and data never leave your account. The endpoint is compatible with the OpenAI Chat Completions API, making it easy to integrate with existing applications and SDKs. API keys are securely managed using AWS Secrets Manager, and network access can be restricted using security groups and CIDR controls.
Built for developers, data scientists, and enterprises, this solution provides a simple way to deploy private LLM inference on AWS using infrastructure as code. Features include automated CloudFormation deployment, GPU-optimized vLLM serving, OpenAI-compatible APIs, support for custom Hugging Face tokens for gated models, and easy lifecycle management by creating or deleting the CloudFormation stack. Keywords: Qwen4B, Qwen, vLLM, OpenAI-compatible API, LLM inference, Generative AI, GPU inference, AWS Marketplace, CloudFormation, self-hosted AI, private AI, machine learning.
Highlights
- Deploy a private, OpenAI-compatible Qwen4B inference endpoint in your own AWS account using AWS CloudFormation - no manual GPU, Docker, or vLLM setup required.
- Powered by vLLM for high-performance LLM inference with an OpenAI-compatible API, making it easy to integrate with existing applications and SDKs.
- Your model, compute resources, API traffic, and data remain entirely within your AWS environment, with secure API key management and configurable network access.
Details
Sold by
Categories
Delivery method
Delivery option
Qwen4B OpenAI-compatible Inference Endpoint
Latest version
Operating system
Ubuntu 22.04
Deployed on AWS
New
Introducing multi-product solutions
You can now purchase comprehensive solutions tailored to use cases and industries.
Features and programs
Financing for AWS Marketplace purchases
AWS Marketplace now accepts line of credit payments through the PNC Vendor Finance program. This program is available to select AWS customers in the US, excluding NV, NC, ND, TN, & VT.
Pricing
This product is available free of charge. Free subscriptions have no end date and may be canceled any time.
Additional AWS infrastructure costs may apply. Use the AWS Pricing Calculator to estimate your infrastructure costs.
Vendor refund policy
This product is provided free of charge through AWS Marketplace. No software fees are collected by the seller, therefore no refunds are applicable. Customers remain responsible for any AWS infrastructure charges (such as EC2, EBS, Elastic IP, or other AWS services) incurred in their own AWS account.
Legal
Vendor terms and conditions
Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .
Content disclaimer
Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.
Delivery details
Qwen4B OpenAI-compatible Inference Endpoint
Qwen4B Inference Endpoint deploys a private, OpenAI-compatible LLM endpoint directly into your AWS account.
A single CloudFormation stack provisions a GPU EC2 instance running Qwen4B with vLLM and exposes the standard OpenAI Chat Completions API on port 8000, secured with a bearer API key.
Your model, GPU, prompts, and data remain entirely within your AWS account and VPC. You control network access using a configurable source CIDR.
The stack creates:
- GPU EC2 instance (default: g5.xlarge) running vLLM with Qwen4B
- Security group for port 8000
- Elastic IP for a stable endpoint
- AWS Secrets Manager secret for the API key (auto-generated or user-supplied)
- IAM role with least-privilege access and AWS Systems Manager support
On first launch, the model downloads automatically (typically 8-15 minutes). CloudFormation Outputs provide the endpoint URL, API key retrieval command, health check command, and a sample API request. Existing OpenAI SDKs work by setting the endpoint URL and using the model name qwen4b.
Delete the CloudFormation stack to terminate the instance, release the Elastic IP, delete the API key secret, and stop charges.
Note: Standard AWS infrastructure charges (EC2, EBS, Elastic IP, Secrets Manager) apply in addition to any Marketplace software charges.
Usage
-
Subscribe and choose Launch -> CloudFormation.
-
Configure: VpcId / SubnetId: Public subnet with internet access. AllowedCidr: Restrict endpoint access (default 0.0.0.0/0). ApiKey: Leave blank to auto-generate or provide your own. InstanceType: g5.xlarge recommended. AmiId: Automatically provided by AWS Marketplace.
-
Wait for the stack to reach CREATE_COMPLETE, then allow 8-15 minutes for the model download. Poll StatusCommand until it returns HTTP 200.
-
Retrieve EndpointUrl and ApiKeyCommand from the stack outputs.
-
Call the API: curl $ENDPOINT/chat/completions
-H "Authorization: Bearer $API_KEY"
-H "Content-Type: application/json"
-d '{"model":"qwen4b","messages":[{"role":"user","content":"Hello"}]}'
Any OpenAI-compatible SDK can be used by setting the base URL to EndpointUrl.
Troubleshooting
- Connection refused: Wait until StatusCommand returns HTTP 200.
- Unsupported instance type: Select a subnet in another Availability Zone.
- Cannot connect: Verify your IP matches AllowedCidr.
CloudFormation Template (CFT)
AWS CloudFormation templates are JSON or YAML-formatted text files that simplify provisioning and management on AWS. The templates describe the service or application architecture you want to deploy, and AWS CloudFormation uses those templates to provision and configure the required services (such as Amazon EC2 instances or Amazon RDS DB instances). The deployed application and associated resources are called a "stack."
Version release notes
Initial release of the Qwen4B OpenAI-Compatible Inference Endpoint for AWS Marketplace.
Features included in this release:
- Deploy a production-ready Qwen4B inference endpoint into your own AWS account using AWS CloudFormation.
- OpenAI-compatible REST API powered by vLLM.
- Automated provisioning of a GPU EC2 instance from a preconfigured AMI.
- Automatic API key generation and secure storage in AWS Secrets Manager.
- Configurable networking, VPC, subnet, and GPU instance type.
- Support for custom Hugging Face access tokens for gated models.
- Simple lifecycle management through CloudFormation stack creation and deletion.
Additional details
Usage instructions
-
Subscribe to the product and choose Launch -> CloudFormation.
-
Configure the stack parameters: Select a VPC and public subnet with internet access. Set AllowedCidr to restrict access to the endpoint. Optionally provide an API key or leave it blank to auto-generate one. Use the recommended instance type (g5.xlarge) or another supported GPU instance. Leave the AMI parameter unchanged; AWS Marketplace supplies the correct AMI automatically.
-
Launch the stack and wait for CREATE_COMPLETE. The first startup downloads the model and typically takes 8-15 minutes. Use the stack's StatusCommand output to verify the endpoint is healthy.
-
Retrieve the EndpointUrl and ApiKeyCommand from the CloudFormation outputs. Run the API key command to retrieve the bearer token from AWS Secrets Manager.
-
Send requests using any OpenAI-compatible SDK or HTTP client by setting the base URL to EndpointUrl and using the model name qwen4b.
Example:
curl $ENDPOINT/chat/completions
-H "Authorization: Bearer $API_KEY"
-H "Content-Type: application/json"
-d '{"model":"qwen4b","messages":[{"role":"user","content":"Hello"}]}'
- To stop all charges, delete the CloudFormation stack. This terminates the EC2 instance, releases the Elastic IP, and deletes the API key secret.
Support
Vendor support
Support email: daxrajsinh.jadeja@holbox.ai
Support description: Support is provided via email for product installation, deployment, configuration, and troubleshooting. Assistance includes CloudFormation deployment issues, AMI startup problems, endpoint availability, and product-specific questions. Please include your AWS Region, CloudFormation stack events, and relevant logs when contacting support to help us investigate issues more efficiently.
AWS infrastructure support
AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.
Similar products
Enterprise-grade JSON structured outputs for Deepseek-R1-Distill-Qwen-32B
A self-hosted, production-ready Qwen 2.5 7B model deployed into your AWS environment with a single click. Because everything runs entirely within your private cloud, your data stays secure, isolated, and fully under your control. Best of all, unlimited tokens.
Enterprise-grade JSON structured outputs for Qwen2.5-VL-32B-Instruct
A self-hosted production-ready DeepSeek-R1-Distill-Qwen-1.5B model (https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) running seamlessly in your private AWS cloud! With an easy single-click installation, set up all the essential infrastructure in your own cloud environment hassle-free. Plus, you will have quick access to an API endpoint that is ready for your queries and scales automatically based on your needs. Best of all, with the service operating solely in your cloud, your data remains completely secure and confidential, never leaving your private space. Experience peace of mind and unleash the full potential of DeepSeek R1 models today!
A self-hosted, production-ready Qwen 3.6 35B model, with 3 billion active parameters deployed into your AWS environment with a single click. Because everything runs entirely within your private cloud, your data stays secure, isolated, and fully under your control. Best of all, unlimited tokens.
Customer reviews
No customer reviews yet
Be the first to review this product . We've partnered with PeerSpot to gather customer feedback. You can share your experience by writing or recording a review, or scheduling a call with a PeerSpot analyst.