Skip to content
This repository has been archived by the owner on Feb 17, 2022. It is now read-only.

Add support for multiple attachments #8

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
28 changes: 24 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ concise, simple test files with strong flexibility and extensibility.
- Parallel execution of tests
- Test against multiple acceptable answers
- Wildcard matching (`<NUMBER>`, `<WORD>`, `<*>`)
- Support for testing rich attachments (`<IMAGE>`, `<CARDS>`)
- Support for testing rich attachments (`<IMAGE>`, `<CARDS>`, `<OTHER>`)
- Support for regular expressions (`<(st|nt|nd)>`)
- Show diffs on error
- Conversation branches
Expand Down Expand Up @@ -71,19 +71,39 @@ Dialogue:
- Bot: <WORD>
```

Handling multiple attachments

```YAML
Title: Conversation with rich messages
Dialogue:
- Bot: Hi
- Human: Hello
- Human: How are we doing this month?
- Bot: "Month-to-date: <IMAGE> <CARDS>"
- Human: Show me ad spend compared to last month
- Bot: <IMAGE> <IMAGE>
```

## Special Tags

You can use the following tags in your test dialogue files:
You can use the following tags anywhere in the bot’s response:

Tag | Meaning
--- | ---
`<*>` | Matches anything, including whitespaces
`<WORD>` | A single word without whitespaces
`<IMAGE>` | An image attachment
`<CARDS>` | A card attachment
`<(REGEX)>` | Any regex expression, i.e. `<([0-9]{2})>`
`<$VARNAME>` | An expression from the locale/translation file

You can match one or more attachments **on their own, or at the end** of the
response:

Tag | Meaning
--- | ---
`<CARDS>` | Any card attachment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the docs here. I would just add a comment here that includes buttons, as this might not be obvious to them.

`<IMAGE>` | Any image attachment
`<OTHER>` | Any other type of attachment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some examples below here.


## Install

To install from npm
Expand Down
16 changes: 7 additions & 9 deletions src/clients/botframework_client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import {
Activity,
Message as BotMessage } from 'botframework-directlinejs';
import { Client } from './client_interface';
import { Message, MessageType } from '../spec/message';
import { Message, Attachment } from '../spec/message';
import * as program from 'commander';
import * as chalk from 'chalk';

Expand Down Expand Up @@ -78,16 +78,14 @@ export class BotFrameworkClient extends Client {
}
const message: Message = {
user,
messageTypes: [
dlMessage.text && MessageType.Text,
dlMessage.attachments && dlMessage.attachments.some(a =>
a.contentType.includes('image')) && MessageType.Image,
dlMessage.attachments && dlMessage.attachments.some(a =>
a.contentType.includes('hero')) && MessageType.Card,
].filter(m => m),
attachments: dlMessage.attachments.map((a) => {
if (a.contentType.includes('image')) return Attachment.Image;
if (a.contentType.includes('hero')) return Attachment.Cards;
return Attachment.Other;
}),
text: dlMessage.text || null,
};
if (message.messageTypes.length) {
if (message.text || message.attachments.length) {
callback(message);
return;
}
Expand Down
6 changes: 3 additions & 3 deletions src/runner.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
'use strict';
import { Client } from './clients/client_interface';
import { Dialogue } from './spec/dialogue';
import { Message, MessageType } from './spec/message';
import { Message, Attachment } from './spec/message';
import { Response } from './spec/response';
import { Turn, TurnType } from './spec/turn';
import { Config } from './config';
Expand Down Expand Up @@ -156,7 +156,7 @@ export class Runner {
if (response !== null) {
if (program.verbose) {
const timing = (new Date().getTime() - this.timing) || 0;
if (response.messageTypes.includes(MessageType.Text)) {
if (response.attachments.length === 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems wrong? Should be if (response.text). Also, for printing, we should probably print the attachments if they exist like:

Bot: Here you go: <IMAGE> <BUTTON> <BUTTON>

I know this is not how it is before but this would bring it into consistency with the new formatting style.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah good catch, thanks.

const texts = response.text.split('\n');
const truncatedText = texts.length > 5
? texts.slice(0, 5).join('\n') + '...' : response.text;
Expand Down Expand Up @@ -261,7 +261,7 @@ export class Runner {
}
this.client.send({
user,
messageTypes: [MessageType.Text],
attachments: [],
text: next.query,
});
}
Expand Down
17 changes: 12 additions & 5 deletions src/spec/message.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,18 @@
export enum MessageType {
Text = 'Text',
Image = 'Image',
Card = 'Card',
/*
* An enumeration of the possible attachment types.
*
* `Other` is a catch-all for any attachment type that is not (yet) explicitly
* supported. The enum values are used to build the regexes for the attachment
* wildcards, e.g. <IMAGE> is derived from 'IMAGE'.
*/
export enum Attachment {
Image = 'IMAGE',
Cards = 'CARDS',
Other = 'OTHER',
}

export interface Message {
messageTypes: MessageType[];
attachments: Attachment[];
text: string;
user: string;
}
72 changes: 51 additions & 21 deletions src/spec/response.ts
Original file line number Diff line number Diff line change
@@ -1,50 +1,80 @@
import { Message, MessageType } from './message';
import { Message, Attachment } from './message';
import { Translator } from '../translator';
import * as program from 'commander';

const wildcardRegex: RegExp = /<\*>/g;
const wordRegex: RegExp = /<WORD>/g;
const numberRegex: RegExp = /<NUMBER>/g;
const imageRegex: RegExp = /<IMAGE>/g;
const cardsRegex: RegExp = /<CARDS>/g;
const regexRegex: RegExp = /<\((.+?)\)>/g;

// Matches an attachment wildcard (e.g. <IMAGE>) at the _end_ of responseData.
//
// The matched part is just the word in the brackets ('IMAGE'), which can be
// directly compared against the Attachment enum (from which it was generated).
const attachmentAlternatives = Object.values(Attachment).join('|');
const attachmentsRegex = new RegExp(`\\s*<(${attachmentAlternatives})>$`);

export class Response {
responseType: MessageType;
attachments: Attachment[];
private textMatchChecker: RegExp;
private bodyText: string;
original: string;

constructor(responseData: string) {
this.attachments = [];
this.original = responseData.trim();
switch (this.original) {
case '<IMAGE>':
this.responseType = MessageType.Image;
break;
case '<CARDS>':
this.responseType = MessageType.Card;
break;
default:
this.responseType = MessageType.Text;
this.textMatchChecker = new RegExp(Response.transformTags(this.original));
this.bodyText = this.original;

// Construct a list of attachments from the attachment wildcards at the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this solution, it's nice. Though I wonder why you constrain it to happening at the end of the line? Seems a bit artificial to me?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle they could go anywhere, but I’m not sure what the use case would be. For example:

Dialogue:
  - Human: How is my campaign doing?
  - Bot: According to <IMAGE> and <IMAGE>, it’s going great!

This doesn’t make any sense in bot framework per se, because it doesn’t define a mechanism for embedding attachments inline with the text. Presumably you would expect attachments to be [Attachment.Image, Attachment.Image], but is text supposed to be 'According to', or 'According to and , it’s going great!, or something else?

You can imagine a client which does support this (e.g. one that uses $0, $1, … to reference attachments in the attachments array), in which case you could check that text === 'According to $0 and $1, it’s going great!', and that attachments[0] === Attachment.Image and attachments[1] === Attachment.Image; but in the absence of an established convention I’m not sure how useful this would be.

Copy link
Contributor

@lingz lingz Jun 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually you are right that the botbuilder platform specifically limits this ordering. However, for other platforms like Slack, it is allowed to have intermingled text + attachments, so for that reason it might be good to allow the tags anywhere, and just concatenate the text.

It could lead to this edge case where someone sends

Hi <IMAGE> there <IMAGE>

and it validates against:

HI there <IMAGE> <IMAGE>

However at the same time, we could introduce this fix as a separate PR as you don't have any example connector right now, so it would be hard to test. For now we can concatenate and it will work for most practical use cases I think.

So to summarize I think for now a good compromise is to just drop the ordering condition (remove the $ on the regex parser).

// end of the response data. The end result is that `this.bodyText`
// contains only the body text and potentially some inline wildcards;
// and `this.attachments` contains a list of attachment types, in the
// same order as they originally were in the supplied data.
let match;
while ((match = this.bodyText.match(attachmentsRegex)) != null) {
this.attachments.push(match[1]);
this.bodyText = this.bodyText.slice(0, match.index);
}
this.attachments.reverse();

// Construct a regex to match the remaining body text in terms of its
// inline wildcards.
this.textMatchChecker = new RegExp(Response.transformTags(this.bodyText));
}

matches(message: Message): boolean {
if (!message.messageTypes.includes(this.responseType)) {

// If we are expecting a message body, check that we have one...
if (this.bodyText && !message.text) {
return false;
} else if (this.responseType === MessageType.Text) {
return message.text.trim().match(this.textMatchChecker) !== null;
} else {
return true;
}

// ...and check that the content is the same.
//
// Note that we ignore the message body altogether if the spec doesn't
// include one. This is for backwards compatibility with when <IMAGE>
// and <CARDS> could only be used in place of the entire message.
if (this.bodyText && message.text.trim().match(this.textMatchChecker) === null) {
return false;
}

// Check that we have the same number, kind, and order of attachments.
// We can use JSON.stringify here since we know attachments is just a
// simple array of strings.

// Check that we have the right number of attachments.
if (JSON.stringify(this.attachments) !== JSON.stringify(message.attachments)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stringify has this tricky behavior where ordering of keys is only guaranteed in Node environment IF the object is the same (same pointer). Even if two objects look the same, they will not necessarily get stringified in the same way.

Use https://github.com/substack/json-stable-stringify to do a sort before the stringify.

Copy link
Author

@ecridge ecridge Jun 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. I assumed that stringifying an Array<String> would be safe. If you like I can just revert ac0dd3b to avoid bringing in another library?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that sounds like a good idea

return false;
}

// Everything appears to be in order.
return true;
}

static transformTags(text: string): string {
regexRegex.lastIndex = 0;

const taggedText = text
.replace(imageRegex, '')
.replace(cardsRegex, '')
.replace(wildcardRegex, '<([\\s\\S]*?)>')
.replace(wordRegex, '<([^ ]+?)>')
.replace(numberRegex, '<([0-9\.,-]+)>');
Expand Down
Loading