Endpoints
Start with a popular model.
Each runs on dedicated GPUs with an OpenAI-compatible API, billed by the minute while it runs.
Let your agent manage endpoints
Connect a cloud
optional · InfraOptimal cloud needs none
AWSnot connectedMore options
The stack creates one role. It can see which GPU instance types your account can launch, file quota requests with AWS, and launch, stop and terminate instances it tags velobyte=1; it cannot read your data or touch anything else. Delete the stack and every permission goes with it. Read the template.
Lambdano keyHow we use your cloud credentials
InfraOptimal keeps it 30 days, then asks for it again, and uses it only to start, check and stop what you launch on your Lambda account.
Modalno tokenHow we use your cloud credentials
InfraOptimal keeps it 30 days, then asks for it again, and uses it only to start, check and stop what you launch in your Modal workspace.
Or make a deploy file and run it yourself
- In a terminal, run
pip install modal && modal setup. It opens modal.com, makes a token and keeps it on your computer. - Enter a model, pick the GPUs and make the file.
- Save the file and run the commands shown with it. The last one prints the endpoint's URL.
A file you deploy yourself runs on your computer's Modal token. InfraOptimal cannot start, watch or stop it, and it does not appear under Endpoints. Modal bills your workspace, credits included.
Connect your project with short-lived credentials. No key file required.
