Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix azure client Add new client paramters: allowed_roles, default_role, finish_reason_allow_list, finish_reason_deny_list #1209

Merged
merged 7 commits into from
Dec 4, 2024

Conversation

hellovai
Copy link
Contributor

@hellovai hellovai commented Dec 3, 2024

TODO: add more tests, but screenshots show it working


Important

This PR adds role selection and finish reason filtering to clients, updates error handling, and modifies tests for new parameters.

  • Behavior:
    • Adds role_selection and finish_reason_filter parameters to clients in anthropic.rs, aws_bedrock.rs, and google_ai.rs.
    • Implements allowed_roles() and default_role() methods for role selection.
    • Introduces finish_reason_filter to manage finish reasons.
  • Error Handling:
    • Adds FinishReasonError in errors.rs for Python and TypeScript clients.
    • Updates error handling in internal_monkeypatch.py and errors.rs for Python.
    • Updates error handling in errors.rs for TypeScript.
  • Testing:
    • Updates tests in test_cli.rs, test_runtime.rs, and test_file_manager.rs for new parameters.
    • Modifies test utilities in harness.rs for new client parameters.
  • Misc:
    • Adds log-once dependency in Cargo.toml and Cargo.lock.
    • Minor updates to README.md and logging in app.py.

This description was created by Ellipsis for 8652179. It will automatically update as commits are pushed.

Copy link

vercel bot commented Dec 3, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
baml ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 4, 2024 0:10am

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 8b20396 in 1 minute and 16 seconds

More details
  • Looked at 2686 lines of code in 38 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. typescript/playground-common/src/baml_wasm_web/test_uis/test_result.tsx:58
  • Draft comment:
    Add a filter button for 'Finish Reason Failed' to maintain consistency with other test statuses.
<FilterButton
  selected={filter.has('finish_reason_failed')}
  name='Finish Reason Failed'
  count={statusCounts.done.finish_reason_failed}
  onClick={() => toggleFilter('finish_reason_failed')}
/>
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_kqODY7jE0XvTefys


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Add new client paramters: allowed_roles, default_role, finish_reason_allow_list, finish_reason_deny_list

TODO: add more tests, but screenshots show it working
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on aa6d118 in 22 seconds

More details
  • Looked at 18 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_Bjvo46lfC84P1WT4


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 7732f8b in 1 minute and 1 seconds

More details
  • Looked at 544 lines of code in 11 files
  • Skipped 0 files when reviewing.
  • Skipped posting 5 drafted comments based on config settings.
1. engine/baml-lib/jinja-runtime/src/lib.rs:246
  • Draft comment:
    Inconsistent usage of allowed_roles. Consider using a consistent approach to access allowed_roles across the codebase to avoid confusion and potential bugs.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR introduces a new parameter allowed_roles in multiple files, but the usage of this parameter is inconsistent. In some places, it is used directly, while in others, it is accessed through a method. This inconsistency can lead to confusion and potential bugs.
2. engine/baml-lib/llm-client/src/clients/anthropic.rs:66
  • Draft comment:
    Inconsistent usage of allowed_roles. Consider using a consistent approach to access allowed_roles across the codebase to avoid confusion and potential bugs.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR introduces a new parameter allowed_roles in multiple files, but the usage of this parameter is inconsistent. In some places, it is used directly, while in others, it is accessed through a method. This inconsistency can lead to confusion and potential bugs.
3. engine/baml-lib/llm-client/src/clients/aws_bedrock.rs:81
  • Draft comment:
    Inconsistent usage of allowed_roles. Consider using a consistent approach to access allowed_roles across the codebase to avoid confusion and potential bugs.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR introduces a new parameter allowed_roles in multiple files, but the usage of this parameter is inconsistent. In some places, it is used directly, while in others, it is accessed through a method. This inconsistency can lead to confusion and potential bugs.
4. engine/baml-lib/llm-client/src/clients/google_ai.rs:72
  • Draft comment:
    Inconsistent usage of allowed_roles. Consider using a consistent approach to access allowed_roles across the codebase to avoid confusion and potential bugs.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR introduces a new parameter allowed_roles in multiple files, but the usage of this parameter is inconsistent. In some places, it is used directly, while in others, it is accessed through a method. This inconsistency can lead to confusion and potential bugs.
5. engine/baml-lib/llm-client/src/clients/vertex.rs:152
  • Draft comment:
    Inconsistent usage of allowed_roles. Consider using a consistent approach to access allowed_roles across the codebase to avoid confusion and potential bugs.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The PR introduces a new parameter allowed_roles in multiple files, but the usage of this parameter is inconsistent. In some places, it is used directly, while in others, it is accessed through a method. This inconsistency can lead to confusion and potential bugs.

Workflow ID: wflow_2wt735gToeb23n6M


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 2039f19 in 22 seconds

More details
  • Looked at 114 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. engine/baml-lib/jinja-runtime/src/lib.rs:1118
  • Draft comment:
    Consider adding a test case to verify that when a role is not in the allowed roles, it defaults to the default role. This will ensure the role fallback mechanism works as expected.
  • Reason this comment was not posted:
    Confidence changes required: 50%
    The test render_with_kwargs_default_role is missing a check for the role fallback mechanism. It should verify that when a role is not in the allowed roles, it defaults to the default role.

Workflow ID: wflow_ApS04GJytR4B1lGo


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Incremental review on 0fe2c37 in 57 seconds

More details
  • Looked at 173 lines of code in 4 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. engine/language_client_typescript/index.js:86
  • Draft comment:
    Consider returning the original error instead of undefined if parsing fails in the from method. This ensures the error is not lost and can be handled appropriately.
  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_55jfIiT0O62Hj2EC


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

engine/language_client_typescript/index.js Show resolved Hide resolved
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Incremental review on 8652179 in 29 seconds

More details
  • Looked at 105 lines of code in 1 files
  • Skipped 0 files when reviewing.
  • Skipped posting 1 drafted comments based on config settings.
1. engine/baml-lib/llm-client/src/clients/helpers.rs:199
  • Draft comment:
    The function ensure_default_role is defined twice. Remove the redundant definition to avoid confusion.
  • Reason this comment was not posted:
    Comment was on unchanged code.

Workflow ID: wflow_N6EyTEkGhAvk3zIb


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@hellovai hellovai added this pull request to the merge queue Dec 4, 2024
Merged via the queue into canary with commit 9b57395 Dec 4, 2024
10 checks passed
@hellovai hellovai deleted the client-fixes branch December 4, 2024 06:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants