Skip to content

Commit

Permalink
Fix typo in prompt.
Browse files Browse the repository at this point in the history
  • Loading branch information
vinaysetty committed Feb 1, 2024
1 parent 1807b2d commit e21f8a6
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 5 deletions.
2 changes: 1 addition & 1 deletion data/llm_prompts/default/intent.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
What is user's intent with the following statement? The options are:

ADD - Statement of fact or preference, typically contains the verbs "is", "likes", "prefers", "told me", "admires", "went to", "am", "has seen",
ADD - Statement of fact or preference, typically contains the verbs "is", "likes", "prefers", "told me", "admires", "went to", "am", "has seen".

GET - Asking for information, typically it starts with "Do I", "Have I" etc.
DELETE - Request to delete or remove an item, typically contains the verbs "remove", "delete" or "discard"
Expand Down
8 changes: 4 additions & 4 deletions pkg_api/nl_to_pkg/eval_nl_to_pkg.py
Original file line number Diff line number Diff line change
Expand Up @@ -92,21 +92,21 @@ def get_preference_f1_scores(
Returns:
Tuple of macro and micro F1 scores for the preferences.
"""
true_preference = []
true_preferences = []
predicted_preferenes = []
for (_, _, _, _, _, true_pref), (_, pkg_data) in zip(
groundtruth_data, annotations
):
true_preference.append(true_pref)
true_preferences.append(true_pref)
if pkg_data.preference:
predicted_preferenes.append(str(int(pkg_data.preference.weight)))
else:
predicted_preferenes.append("")
preference_macro_f1 = f1_score(
true_preference, predicted_preferenes, average="macro"
true_preferences, predicted_preferenes, average="macro"
)
preference_micro_f1 = f1_score(
true_preference, predicted_preferenes, average="micro"
true_preferences, predicted_preferenes, average="micro"
)
return preference_macro_f1, preference_micro_f1

Expand Down

0 comments on commit e21f8a6

Please sign in to comment.