The modelled monthly GPU rent for Llama-3.1-8B-Instruct using 1 replica of 1 x NVIDIA B200 180 GB is $4,302.64 at 50,000 requests/day.

Scroll horizontally to read every volume point.

500 requests per day, 7,609,375 output tokens per month: self-host $4,302.64, cheapest comparable API $0.46. 1,500 requests per day, 22,828,125 output tokens per month: self-host $4,302.64, cheapest comparable API $1.37. 5,000 requests per day, 76,093,750 output tokens per month: self-host $4,302.64, cheapest comparable API $4.57. 15,000 requests per day, 228,281,250 output tokens per month: self-host $4,302.64, cheapest comparable API $13.70. 50,000 requests per day, 760,937,500 output tokens per month: self-host $4,302.64, cheapest comparable API $45.66. 84,912 requests per day, 1,292,250,810 output tokens per month: self-host $4,302.64, cheapest comparable API $77.54. 86,627 requests per day, 1,318,356,887 output tokens per month: self-host $8,605.29, cheapest comparable API $79.10. 150,000 requests per day, 2,282,812,500 output tokens per month: self-host $8,605.29, cheapest comparable API $136.97. 500,000 requests per day, 7,609,375,000 output tokens per month: self-host $25,815.87, cheapest comparable API $456.56. 1,500,000 requests per day, 22,828,125,000 output tokens per month: self-host $77,447.61, cheapest comparable API $1,369.69. 5,000,000 requests per day, 76,093,750,000 output tokens per month: self-host $253,856.06, cheapest comparable API $4,565.63.

Scroll horizontally to read every chart label and value.

Busy GPU-hours: 425.9, Engine-reported share doing token work. Idle GPU-hours billed: 304.6, Engine-reported share paid while idle.

Scroll horizontally to read every volume point.

500 requests per day, 7,609,375 output tokens per month: self-host $4,302.64, cheapest comparable API $0.46. 1,500 requests per day, 22,828,125 output tokens per month: self-host $4,302.64, cheapest comparable API $1.37. 5,000 requests per day, 76,093,750 output tokens per month: self-host $4,302.64, cheapest comparable API $4.57. 15,000 requests per day, 228,281,250 output tokens per month: self-host $4,302.64, cheapest comparable API $13.70. 50,000 requests per day, 760,937,500 output tokens per month: self-host $4,302.64, cheapest comparable API $45.66. 84,912 requests per day, 1,292,250,810 output tokens per month: self-host $4,302.64, cheapest comparable API $77.54. 86,627 requests per day, 1,318,356,887 output tokens per month: self-host $8,605.29, cheapest comparable API $79.10. 150,000 requests per day, 2,282,812,500 output tokens per month: self-host $8,605.29, cheapest comparable API $136.97. 500,000 requests per day, 7,609,375,000 output tokens per month: self-host $25,815.87, cheapest comparable API $456.56. 1,500,000 requests per day, 22,828,125,000 output tokens per month: self-host $77,447.61, cheapest comparable API $1,369.69. 5,000,000 requests per day, 76,093,750,000 output tokens per month: self-host $253,856.06, cheapest comparable API $4,565.63.

Scroll horizontally to read every chart label and value.

Busy GPU-hours: 425.9, Engine-reported share doing token work. Idle GPU-hours billed: 304.6, Engine-reported share paid while idle.

Scroll horizontally to read every volume point.

500 requests per day, 7,609,375 output tokens per month: self-host $4,302.64, cheapest comparable API $0.46. 1,500 requests per day, 22,828,125 output tokens per month: self-host $4,302.64, cheapest comparable API $1.37. 5,000 requests per day, 76,093,750 output tokens per month: self-host $4,302.64, cheapest comparable API $4.57. 15,000 requests per day, 228,281,250 output tokens per month: self-host $4,302.64, cheapest comparable API $13.70. 50,000 requests per day, 760,937,500 output tokens per month: self-host $4,302.64, cheapest comparable API $45.66. 84,912 requests per day, 1,292,250,810 output tokens per month: self-host $4,302.64, cheapest comparable API $77.54. 86,627 requests per day, 1,318,356,887 output tokens per month: self-host $8,605.29, cheapest comparable API $79.10. 150,000 requests per day, 2,282,812,500 output tokens per month: self-host $8,605.29, cheapest comparable API $136.97. 500,000 requests per day, 7,609,375,000 output tokens per month: self-host $25,815.87, cheapest comparable API $456.56. 1,500,000 requests per day, 22,828,125,000 output tokens per month: self-host $77,447.61, cheapest comparable API $1,369.69. 5,000,000 requests per day, 76,093,750,000 output tokens per month: self-host $253,856.06, cheapest comparable API $4,565.63.

Scroll horizontally to read every chart label and value.

Busy GPU-hours: 425.9, Engine-reported share doing token work. Idle GPU-hours billed: 304.6, Engine-reported share paid while idle.

Scroll horizontally to read every volume point.

500 requests per day, 7,609,375 output tokens per month: self-host $4,302.64, cheapest comparable API $0.46. 1,500 requests per day, 22,828,125 output tokens per month: self-host $4,302.64, cheapest comparable API $1.37. 5,000 requests per day, 76,093,750 output tokens per month: self-host $4,302.64, cheapest comparable API $4.57. 15,000 requests per day, 228,281,250 output tokens per month: self-host $4,302.64, cheapest comparable API $13.70. 50,000 requests per day, 760,937,500 output tokens per month: self-host $4,302.64, cheapest comparable API $45.66. 84,912 requests per day, 1,292,250,810 output tokens per month: self-host $4,302.64, cheapest comparable API $77.54. 86,627 requests per day, 1,318,356,887 output tokens per month: self-host $8,605.29, cheapest comparable API $79.10. 150,000 requests per day, 2,282,812,500 output tokens per month: self-host $8,605.29, cheapest comparable API $136.97. 500,000 requests per day, 7,609,375,000 output tokens per month: self-host $25,815.87, cheapest comparable API $456.56. 1,500,000 requests per day, 22,828,125,000 output tokens per month: self-host $77,447.61, cheapest comparable API $1,369.69. 5,000,000 requests per day, 76,093,750,000 output tokens per month: self-host $253,856.06, cheapest comparable API $4,565.63.

Scroll horizontally to read every chart label and value.

Busy GPU-hours: 425.9, Engine-reported share doing token work. Idle GPU-hours billed: 304.6, Engine-reported share paid while idle.