Skip to content

Commit

Permalink
fix: info update in async baseten notebook
Browse files Browse the repository at this point in the history
  • Loading branch information
minaamshahid committed Oct 22, 2024
1 parent 1d1eb77 commit 9e361f4
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 243 deletions.
15 changes: 11 additions & 4 deletions examples/7_baseten_async_quickstart.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
"source": [
"### Additional Requirements\n",
"\n",
"Set your `Baseten API key` and `Webhook secret` in the environment."
"Set your `Baseten API key`, `Webhook secret` and `GPU` option in the environment."
]
},
{
Expand All @@ -46,7 +46,14 @@
"import os\n",
"\n",
"os.environ[\"BASETEN_WEBHOOK_SECRET\"] = \"your_baseten_webhook_secret\"\n",
"os.environ[\"BASETEN_API_KEY\"] = \"your_baseten_api_key\""
"os.environ[\"BASETEN_API_KEY\"] = \"your_baseten_api_key\"\n",
"\n",
"# You can optionally switch the GPU to H100.\n",
"# This will deploy the FlowJudge model on H100 40GB\n",
"# A10G deployment is Flow-Judge-v0.1-AWQ\n",
"# H100 deployment is Flow-Judge-v0.1-FP8\n",
"# !! Manually changing the hardware on Baseten's UI may cause compatibility issues !!\n",
"os.environ[\"BASETEN_GPU\"] = \"A10G\""
]
},
{
Expand Down Expand Up @@ -93,7 +100,7 @@
"\n",
"Let's test batched evaluations with our example csr data on the faithfulness 5 point likert.\n",
"\n",
"We use the `async_batch_evaluate` method from the FlowJudge class. Underneath this uses batched processing using the batch_size set with the `async_batch_size` argument of the Baseten model class."
"We use the `async_batch_evaluate` method from the FlowJudge class. Underneath this uses batched processing utilizing the batch_size set with the `async_batch_size` argument of the Baseten model class. If there are failures, for example with networking, the batch will process and errors will be propagated as log outputs. The output would include the successful responses."
]
},
{
Expand Down Expand Up @@ -144,7 +151,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Similarly you can run a single evaluation task given an input using the `async_evaluate` method on the Flow Judge class. Under the hood this will process a single async request and attach listeners to the webhook for the response."
"Similarly you can run a single evaluation task using the `async_evaluate` method on the `AsyncFlowJudge` class. Under the hood, this will process a single async request and attach listeners to the webhook for the response."
]
},
{
Expand Down
Loading

0 comments on commit 9e361f4

Please sign in to comment.