Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi profiles #19

Open
wants to merge 19 commits into
base: master
Choose a base branch
from
Open

Multi profiles #19

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
da7273a
adding multi Jira profiles
paul-denning Dec 12, 2018
a0c8d49
fixed issues with default profile
paul-denning Dec 12, 2018
536e08d
Added profiles to all actions
paul-denning Dec 13, 2018
e382643
added profile section to readme and add changes to changes.md and bum…
paul-denning Dec 13, 2018
272123e
fixed readme
paul-denning Dec 13, 2018
6d7bff3
fixed whitespace issue
paul-denning Dec 13, 2018
4a85140
fixed readme
paul-denning Dec 13, 2018
b4c5803
fixed whitepsaces on blank lines
Ragnra Dec 13, 2018
3078b85
fixed liniting
Ragnra Dec 13, 2018
bda1233
fixed up example to include new config options
Ragnra Dec 13, 2018
3a6897e
removed unrequired second config example file
Ragnra Dec 13, 2018
5ef48cb
suggest changes from pull request
paul-denning Dec 17, 2018
21886d6
Merge pull request #1 from pdenning/multi-profiles-pr
pdenning Dec 17, 2018
d89c994
Fixed issues
paul-denning Dec 17, 2018
36ca67e
Fixed liniting issues
paul-denning Dec 17, 2018
680b306
Merge pull request #2 from pdenning/master
pdenning Dec 17, 2018
84193f6
fixed rebase issues
paul-denning Feb 19, 2019
96301b0
aligned config example with actual config issues?
paul-denning Feb 19, 2019
cc14c71
Merge branch 'master' into multi-profiles
paul-denning Feb 19, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Change Log

## 0.9.0

- Added config profile support for actions. See profile section in readme.
- Add new config option ``profiles`` this mimicks the jira configuration options to allow for multi jira connections.
- Add new config option ``default_profile`` sets a default profile. If left blank will use the current config method
- Add new action option ``config_profile`` to all actions. This will tell the action to use a different config profile then the default profile.


## 0.8.0

- Adding support for BASIC authentication
Expand Down
33 changes: 33 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,39 @@ You can also use dynamic values from the datastore. See the
remember to tell StackStorm to load these new values by running
`st2ctl reload --register-configs`

### Config Profiles
The configuration allows for multi jira definitions. This will allow for the automation to multiple Jira instances from a single stackstorm instance.

Profiles are defined within the ``profiles`` section of the configuration and they accept the same parameters as the inline options.
This option takes in an array of profile options.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using an array of profile dictionaries:

profiles:
- name: profile_one
  # ...
- name: profile_two
  # ...

is kind of clunky. Using a dictionary of profile dictionaries would be cleaner:

profiles:
  profile_one:
    # ...
  profile_two:
    # ...

Using a dictionary also means that you don't have to specify a default_profile - the inline values function as the default profile values.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I looked into this, but i wasn't sure how to define a dynamic key within the config schema file.
Could you point me in the right direction on how to define that and i'll look into changing it

Copy link
Contributor

@blag blag Dec 17, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh gotcha. Dictionaries are called "objects" in JSONSchema.

My review comment later on contains a good example of what you can/should do. All you should need to do is:

  1. add the <<: &root_anchor after the --- line, before all of the existing keys
  2. indent the existing keys
  3. copy the profiles key from my example into config.schema.yaml

Let me know if you have any further questions - this is easily the most confusing part of a pack.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw your example below, i did implement it. However it appears there is a bug.

If i try and use an encrypted key store, it ignores the secret value and doesn't decrypt the password.
This happens when using anchors or not.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have merged my changes. However there still appears to be a bug with secret key values. Maybe i am missing something you'll noticed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does st2kv.system.jira_prod_pass get decrypted?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@blag
Yes the jira_prod_pass gets decrypted, however jira_dev_pass doesnt.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pdenning Sorry I don't have more to update, but at this point I believe this isn't working due to an undiscovered bug in StackStorm. I haven't had time to reproduce the issue or add a similar config to StackStorm's test fixtures yet, but I haven't forgotten about it.

If you managed to get it working, then please resolve this conversation, but if not I'd like to fix the bug in StackStorm itself (st2 repo) before we merge this PR.

Copy link
Member

@cognifloyd cognifloyd Apr 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found the bug. In order to decrypt the key, the schema needs to declare secret: true, BUT, the config loader does not look at any schemas under additionalProperties, so the profiles object is effectively schema-less:
https://github.com/StackStorm/st2/blob/8496bb2407b969f0937431992172b98b545f6756/st2common/st2common/util/config_loader.py#L133-L137

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm working on a fix for this here: StackStorm/st2#5225


General Options
- ``name``
- ``url``
- ``verify``
- ``project``
- ``auth_method``

OAuth Options
- ``rsa_cert_file``
- ``oauth_token``
- ``oauth_secret``
- ``consumer_key``

Basic Auth Options
- ``username``
- ``password``

The ``name`` defines the name of the profile. This option is used to define the default profile and used within actions to define with config profile to use.

By default it will use the settings outside of the ``profiles`` section.

See [jira.yaml.example](./jira.yaml.example) for an example of how to use profiles.

To enable the the use of a profile within an action use the ``config_profile`` action option and set it to the same name as the profile you wish to use.

**Note** : Sensors still use the main config profile. Therefore it will still need to be defined.

### OAuth

### Disclaimer
Expand Down
5 changes: 4 additions & 1 deletion actions/attach_file_to_issue.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,10 @@

class AttachFileToJiraIssueAction(BaseJiraAction):

def run(self, issue_key, file_path, file_name=None):
def run(self, issue_key, file_path, file_name=None, config_profile=None):

super(AttachFileToJiraIssueAction, self)._run(config_profile)

if not file_name:
file_name = None

Expand Down
4 changes: 4 additions & 0 deletions actions/attach_file_to_issue.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,7 @@ parameters:
type: string
description: Issue key (e.g. PROJECT-1000).
required: true
config_profile:
type: string
description: Select which jira config profile to use.
required: false
4 changes: 3 additions & 1 deletion actions/attach_files_to_issue.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,11 @@

class AttachFilesToJiraIssueAction(BaseJiraAction):

def run(self, issue_key, file_paths):
def run(self, issue_key, file_paths, config_profile=None):
result = []

super(AttachFilesToJiraIssueAction, self)._run(config_profile)

for file_path in file_paths:
with open(file_path, 'rb') as fp:
attachment = self._client.add_attachment(
Expand Down
4 changes: 4 additions & 0 deletions actions/attach_files_to_issue.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,7 @@ parameters:
type: string
description: Issue key (e.g. PROJECT-1000).
required: true
config_profile:
type: string
description: Select which jira config profile to use.
required: false
5 changes: 4 additions & 1 deletion actions/comment_issue.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@

class CommentJiraIssueAction(BaseJiraAction):

def run(self, issue_key, comment_text):
def run(self, issue_key, comment_text, config_profile=None):

super(CommentJiraIssueAction, self)._run(config_profile)

comment = self._client.add_comment(issue_key, comment_text)
result = to_comment_dict(comment)
return result
4 changes: 4 additions & 0 deletions actions/comment_issue.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,7 @@ parameters:
type: string
description: Issue key (e.g. PROJECT-1000).
required: true
config_profile:
type: string
description: Select which jira config profile to use.
required: false
8 changes: 6 additions & 2 deletions actions/create_issue.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,12 @@
class CreateJiraIssueAction(BaseJiraAction):

def run(self, summary, type, description=None,
project=None, extra_fields=None):
project = project or self.config['project']
project=None, extra_fields=None, config_profile=None):

super(CreateJiraIssueAction, self)._run(config_profile)

# project = project or self.config['project']
project = project or self.project
data = {
'project': {'key': project},
'summary': summary,
Expand Down
4 changes: 4 additions & 0 deletions actions/create_issue.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,7 @@ parameters:
type: object
description: "extra fields like priority, labels, custom fields, etc"
required: false
config_profile:
type: string
description: Select which jira config profile to use.
required: false
5 changes: 4 additions & 1 deletion actions/get_issue.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@

class GetJiraIssueAction(BaseJiraAction):
def run(self, issue_key, include_comments=False, include_attachments=False,
include_customfields=False):
include_customfields=False, config_profile=None):

super(GetJiraIssueAction, self)._run(config_profile)

issue = self._client.issue(issue_key)
result = to_issue_dict(issue=issue, include_comments=include_comments,
include_attachments=include_attachments,
Expand Down
4 changes: 4 additions & 0 deletions actions/get_issue.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,7 @@ parameters:
description: True to include custom fields.
required: true
default: false
config_profile:
type: string
description: Select which jira config profile to use.
required: false
5 changes: 4 additions & 1 deletion actions/get_issue_attachments.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,10 @@


class GetJiraIssueAttachmentsAction(BaseJiraAction):
def run(self, issue_key):
def run(self, issue_key, config_profile=None):

super(GetJiraIssueAttachmentsAction, self)._run(config_profile)

issue = self._client.issue(issue_key)

result = []
Expand Down
5 changes: 5 additions & 0 deletions actions/get_issue_attachments.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,8 @@ parameters:
type: string
description: Issue key (e.g. PROJECT-1000).
required: true
config_profile:
type: string
description: Select which jira config profile to use.
required: false

5 changes: 4 additions & 1 deletion actions/get_issue_comments.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,10 @@


class GetJiraIssueCommentsAction(BaseJiraAction):
def run(self, issue_key):
def run(self, issue_key, config_profile=None):

super(GetJiraIssueCommentsAction, self)._run(config_profile)

issue = self._client.issue(issue_key)

result = []
Expand Down
4 changes: 4 additions & 0 deletions actions/get_issue_comments.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,3 +9,7 @@ parameters:
type: string
description: Issue key (e.g. PROJECT-1000).
required: true
config_profile:
type: string
description: Select which jira config profile to use.
required: false
34 changes: 24 additions & 10 deletions actions/lib/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,30 +14,34 @@ def __init__(self, config):
class BaseJiraAction(Action):
def __init__(self, config):
super(BaseJiraAction, self).__init__(config=config)
self._client = self._get_client()
self.project = ""

def _get_client(self):
config = self.config
def _run(self, profile=None):
self._client = self._get_client(profile)

def _get_client(self, profile=None):

options = {'server': config['url'], 'verify': config['verify']}
profile = self._build_profile(profile)

auth_method = config['auth_method']
options = {'server': profile['url'], 'verify': profile['verify']}

auth_method = profile['auth_method']

if auth_method == 'oauth':
rsa_cert_file = config['rsa_cert_file']
rsa_cert_file = profile['rsa_cert_file']
rsa_key_content = self._get_file_content(file_path=rsa_cert_file)

oauth_creds = {
'access_token': config['oauth_token'],
'access_token_secret': config['oauth_secret'],
'consumer_key': config['consumer_key'],
'access_token': profile['oauth_token'],
'access_token_secret': profile['oauth_secret'],
'consumer_key': profile['consumer_key'],
'key_cert': rsa_key_content
}

client = JIRA(options=options, oauth=oauth_creds)

elif auth_method == 'basic':
basic_creds = (config['username'], config['password'])
basic_creds = (profile['username'], profile['password'])
client = JIRA(options=options, basic_auth=basic_creds)

else:
Expand All @@ -47,6 +51,16 @@ def _get_client(self):

return client

def _build_profile(self, profile_name):
pdenning marked this conversation as resolved.
Show resolved Hide resolved
config = self.config

profiles = config.pop('profiles', {})
profile = profiles.get(profile_name, {})
if profile.get('url', None) is None:
profile = config

return profile

def _get_file_content(self, file_path):
with open(file_path, 'r') as fp:
content = fp.read()
Expand Down
5 changes: 4 additions & 1 deletion actions/search_issues.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@
class SearchJiraIssuesAction(BaseJiraAction):
def run(self, query, start_at=0, max_results=50,
include_comments=False, include_attachments=False,
include_customfields=False):
include_customfields=False, config_profile=None):

super(SearchJiraIssuesAction, self)._run(config_profile)

issues = self._client.search_issues(query, startAt=start_at,
maxResults=max_results)
results = []
Expand Down
4 changes: 4 additions & 0 deletions actions/search_issues.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,7 @@ parameters:
description: True to include custom fields.
required: true
default: false
config_profile:
type: string
description: Select which jira config profile to use.
required: false
5 changes: 4 additions & 1 deletion actions/transition_issue.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,9 @@

class TransitionJiraIssueAction(BaseJiraAction):

def run(self, issue_key, transition):
def run(self, issue_key, transition, config_profile=None):

super(TransitionJiraIssueAction, self)._run(config_profile)

result = self._client.transition_issue(issue_key, transition)
return result
4 changes: 4 additions & 0 deletions actions/transition_issue.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,7 @@ parameters:
type: string
description: Name of transition (e.g. Close, Start Progress, etc).
required: true
config_profile:
type: string
description: Select which jira config profile to use.
required: false
13 changes: 12 additions & 1 deletion config.schema.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
---
<<: &profile_options
url:
description: "URL of the JIRA instance (e.g. ``https://myproject.atlassian.net``)"
type: "string"
Expand Down Expand Up @@ -57,4 +58,14 @@
description: "Project to be used as default for actions that don't require or allow a project"
type: "string"
secret: false
required: true
required: false
profiles:
description: "Jira Connection profile."
type: object
required: false
properties: {}
additionalProperties:
type: object
additionalProperties: false
properties:
<<: *profile_options
37 changes: 28 additions & 9 deletions jira.yaml.example
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,29 @@
---
url: "https://company.atlassian.net"
rsa_cert_file: "/home/vagrant/jira.pem"
auth_method: "oauth"
oauth_token: ""
oauth_secret: ""
consumer_key: ""
poll_interval: 30
project: "MY_PROJECT"
verify: True
url: "https://company.atlassian.net"
rsa_cert_file: "/home/vagrant/jira.pem"
auth_method: "oauth"
oauth_token: ""
oauth_secret: ""
consumer_key: ""
poll_interval: 30
project: "MY_PROJECT"
verify: True
default_profile: "dev"
profiles:
- name: "dev"
pdenning marked this conversation as resolved.
Show resolved Hide resolved
url: "https://dev.atlassian.net"
rsa_cert_file: "/home/vagrant/jira.pem"
auth_method: "basic"
username: "dev-user"
password: "mypas"
project: "MY_PROJECT"
verify: True
- name: "prod"
url: "https://prod.atlassian.net"
rsa_cert_file: "/home/vagrant/jira.pem"
auth_method: "oauth"
oauth_token: ""
oauth_secret: ""
consumer_key: ""
project: "MY_PROJECT"
verify: True
2 changes: 1 addition & 1 deletion pack.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,6 @@ keywords:
- issues
- ticket management
- project management
version: 0.8.0
version: 0.9.0
author : StackStorm, Inc.
email : info@stackstorm.com