Keycloak Realm Installation is not creating user #157

daveanvit · 2022-09-13T10:42:56Z

daveanvit
Sep 13, 2022

I had followed this discussion #77 ,however I am not able to see other keycloak user @keshavprasadms can help in this
release tag:- release-4.10.1_RC1

Jenkins logs:-

Started by user [admin](http://34.138.204.52:8080/user/admin)
Rebuilds build #[40](http://34.138.204.52:8080/job/Deploy/job/dev/job/Kubernetes/job/KeycloakRealm/40)
Checking out git https://github.com/project-sunbird/sunbird-devops.git into /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm@script/7ef7a8c56617eaf0ab553d210db263f0667890b3b60cc5d0a492293898e109c7 to read pipelines/deploy/keycloak-realm/Jenkinsfile
The recommended git tool is: NONE
No credentials specified
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm@script/7ef7a8c56617eaf0ab553d210db263f0667890b3b60cc5d0a492293898e109c7/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/project-sunbird/sunbird-devops.git # timeout=10
Using shallow fetch with depth 1
Fetching upstream changes from https://github.com/project-sunbird/sunbird-devops.git
 > git --version # timeout=10
 > git --version # 'git version 2.37.3'
 > git fetch --tags --force --progress --depth=1 -- https://github.com/project-sunbird/sunbird-devops.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse origin/release-4.10.1_RC1^{commit} # timeout=10
 > git rev-parse release-4.10.1_RC1^{commit} # timeout=10
Checking out Revision aa8374b047b1e18e7ccf0655e31eba49d4384496 (release-4.10.1_RC1)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f aa8374b047b1e18e7ccf0655e31eba49d4384496 # timeout=10
Commit message: "Merge pull request #3403 from project-sunbird/revert-3402-release-4.10.0"
First time build. Skipping changelog.
Loading library deploy-conf@heads/shared-lib
Attempting to resolve heads/shared-lib from remote references...
 > git --version # timeout=10
 > git --version # 'git version 2.37.3'
 > git ls-remote -- https://github.com/project-sunbird/sunbird-devops.git # timeout=10
The recommended git tool is: NONE
No credentials specified
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm@libs/deploy-conf/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/project-sunbird/sunbird-devops.git # timeout=10
Using shallow fetch with depth 1
Fetching upstream changes from https://github.com/project-sunbird/sunbird-devops.git
 > git --version # timeout=10
 > git --version # 'git version 2.37.3'
 > git fetch --no-tags --force --progress --depth=1 -- https://github.com/project-sunbird/sunbird-devops.git +refs/heads/*:refs/remotes/origin/* refs/heads/shared-lib # timeout=10
Checking out Revision 778da7ce7f97efd003eed98071921a837d209343 (778da7ce7f97efd003eed98071921a837d209343)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 778da7ce7f97efd003eed98071921a837d209343 # timeout=10
Commit message: "fix: move variables to be accessible across both try blocks"
 > git rev-list --no-walk 778da7ce7f97efd003eed98071921a837d209343 # timeout=10
[Pipeline] Start of Pipeline
[Pipeline] node
Running on [Jenkins](http://34.138.204.52:8080/computer/(built-in)/) in /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] {
[Pipeline] stage
[Pipeline] { (checkout public repo)
[Pipeline] echo
Found .git folder. Clearing it..
[Pipeline] sh
+ git clean -fxd
Removing ansible/inventory/env/hosts
Removing ansible/inventory/env/secrets.yml
Removing private@tmp/
[Pipeline] checkout
The recommended git tool is: NONE
No credentials specified
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/project-sunbird/sunbird-devops.git # timeout=10
Using shallow fetch with depth 1
Fetching upstream changes from https://github.com/project-sunbird/sunbird-devops.git
 > git --version # timeout=10
 > git --version # 'git version 2.37.3'
 > git fetch --tags --force --progress --depth=1 -- https://github.com/project-sunbird/sunbird-devops.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse origin/release-4.10.1_RC1^{commit} # timeout=10
 > git rev-parse release-4.10.1_RC1^{commit} # timeout=10
Checking out Revision aa8374b047b1e18e7ccf0655e31eba49d4384496 (release-4.10.1_RC1)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f aa8374b047b1e18e7ccf0655e31eba49d4384496 # timeout=10
Commit message: "Merge pull request #3403 from project-sunbird/revert-3402-release-4.10.0"
[Pipeline] }
[Pipeline] // stage
[Pipeline] ansiColor
[Pipeline] {

[Pipeline] stage
[Pipeline] { (deploy)
[Pipeline] sh
+ pwd
[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] echo
{currentWs=/var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm, env=dev, module=Kubernetes, jobName=KeycloakRealm, ansiblePlaybook=/var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/ansible/keycloak.yml, ansibleExtraArgs=--extra-vars "host=keycloak" --tags bootstrap --vault-password-file  /var/lib/jenkins/secrets/vault-pass}
[Pipeline] ansiColor
[Pipeline] {

[Pipeline] stage
[Pipeline] { (ansible-run)
[Pipeline] echo
{currentWs=/var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm, env=dev, module=Kubernetes, jobName=KeycloakRealm, ansiblePlaybook=/var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/ansible/keycloak.yml, ansibleExtraArgs=--extra-vars "host=keycloak" --tags bootstrap --vault-password-file  /var/lib/jenkins/secrets/vault-pass}
[Pipeline] dir
Running in /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/private
[Pipeline] {
[Pipeline] echo
Info: Branch override is enabled
[Pipeline] checkout
The recommended git tool is: NONE
using credential github-cred
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/private/.git # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/daveanvit/searce-diksha-sunbird.git # timeout=10
Using shallow fetch with depth 1
Fetching upstream changes from https://github.com/daveanvit/searce-diksha-sunbird.git
 > git --version # timeout=10
 > git --version # 'git version 2.37.3'
using GIT_ASKPASS to set credentials 
 > git fetch --no-tags --force --progress --depth=1 -- https://github.com/daveanvit/searce-diksha-sunbird.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git rev-parse origin/main^{commit} # timeout=10
Checking out Revision 7b6d5f6e1561c25af11ae11de8a3a4c49527b0b0 (origin/main)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 7b6d5f6e1561c25af11ae11de8a3a4c49527b0b0 # timeout=10
Commit message: "Update common.yml"
 > git rev-list --no-walk 7b6d5f6e1561c25af11ae11de8a3a4c49527b0b0 # timeout=10
[Pipeline] }
[Pipeline] // dir
[Pipeline] sh
+ rsync -Lkr /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/private/ansible/inventory/dev/Kubernetes/common.yml /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/private/ansible/inventory/dev/Kubernetes/hosts /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/private/ansible/inventory/dev/Kubernetes/secrets.yml /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/ansible/inventory/env/
+ [ -f /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/ansible/inventory/env/kubernetes.yaml ]
+ ansible-playbook -i /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/ansible/inventory/env/hosts /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/ansible/keycloak.yml --extra-vars host=keycloak --tags bootstrap --vault-password-file /var/lib/jenkins/secrets/vault-pass
[WARNING]: Invalid characters were found in group names but not replaced, use
-vvvv to see details

[WARNING]:  * Failed to parse /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/
KeycloakRealm/ansible/inventory/env/hosts with yaml plugin: Syntax Error while
loading YAML.   did not find expected <document start>  The error appears to be
in '/var/lib/jenkins/workspace/Deploy/dev/Kubernetes/KeycloakRealm/ansible/inve
ntory/env/hosts': line 3, column 1, but may be elsewhere in the file depending
on the exact syntax problem.  The offending line appears to be:  # Replace the
value deployer to your server login user name, for example ubuntu
ansible_ssh_user=deployer ^ here

[WARNING]:  * Failed to parse /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/
KeycloakRealm/ansible/inventory/env/hosts with ini plugin: /var/lib/jenkins/wor
kspace/Deploy/dev/Kubernetes/KeycloakRealm/ansible/inventory/env/hosts:143:
Section [core:children] includes undefined group: postgresql-slave

[WARNING]: Unable to parse /var/lib/jenkins/workspace/Deploy/dev/Kubernetes/Key
cloakRealm/ansible/inventory/env/hosts as an inventory source

[WARNING]: No inventory was parsed, only implicit localhost is available

[WARNING]: provided hosts list is empty, only localhost is available. Note that
the implicit localhost does not match 'all'


PLAY [keycloak] ****************************************************************
/usr/lib/python3/dist-packages/requests/__init__.py:80: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
[WARNING]: While constructing a mapping from /var/lib/jenkins/workspace/Deploy/
dev/Kubernetes/KeycloakRealm/ansible/inventory/env/group_vars/all.yml, line 4,
column 1, found a duplicate dict key (app_es_snapshot_host). Using last defined
value only.

[WARNING]: While constructing a mapping from /var/lib/jenkins/workspace/Deploy/
dev/Kubernetes/KeycloakRealm/ansible/inventory/env/group_vars/all.yml, line 4,
column 1, found a duplicate dict key (sunbird_es_host). Using last defined
value only.

[WARNING]: While constructing a mapping from /var/lib/jenkins/workspace/Deploy/
dev/Kubernetes/KeycloakRealm/ansible/inventory/env/group_vars/all.yml, line 4,
column 1, found a duplicate dict key (log_es_snapshot_host). Using last defined
value only.


TASK [Gathering Facts] *********************************************************
ok: [192.170.0.6]

TASK [keycloak-deploy : start keycloak service] ********************************
changed: [192.170.0.6] => {}

MSG:

starting keycloak


RUNNING HANDLER [keycloak-deploy : start service] ******************************
fatal: [192.170.0.6]: FAILED! => {
    "changed": true,
    "cmd": [
        "/etc/init.d/keycloak",
        "start"
    ],
    "delta": "0:00:00.005606",
    "end": "2022-09-13 09:45:28.404501",
    "rc": 1,
    "start": "2022-09-13 09:45:28.398895"
}

STDOUT:

/etc/init.d/keycloak already running.
[FAILED]


MSG:

non-zero return code

...ignoring

RUNNING HANDLER [keycloak-deploy : wait for keycloak to start] *****************
ok: [192.170.0.6]

TASK [keycloak-deploy : Create admin user] *************************************
changed: [192.170.0.6]

TASK [keycloak-deploy : stop keycloak service] *********************************
changed: [192.170.0.6] => {}

MSG:

stopping keycloak


RUNNING HANDLER [keycloak-deploy : stop service] *******************************
changed: [192.170.0.6]

RUNNING HANDLER [keycloak-deploy : wait for keycloak to stop] ******************
ok: [192.170.0.6]

TASK [keycloak-deploy : start keycloak with script upload feature temporarily] ***
changed: [192.170.0.6]

RUNNING HANDLER [keycloak-deploy : wait for keycloak to start] *****************
ok: [192.170.0.6]

TASK [keycloak-deploy : Install dependencies for keycloak] *********************
ok: [192.170.0.6]

TASK [keycloak-deploy : Ensure python packages are installed] ******************
skipping: [192.170.0.6]

TASK [keycloak-deploy : Ensure python packages are installed] ******************
ok: [192.170.0.6]

TASK [keycloak-deploy : Copying python libraries] ******************************
changed: [192.170.0.6]

TASK [keycloak-deploy : Initialize python library to run keycloak bootstrap script] ***
changed: [192.170.0.6]

TASK [keycloak-deploy : Save keycloak vars to json] ****************************
ok: [192.170.0.6]

TASK [keycloak-deploy : Copy realm json file to tmp location] ******************
ok: [192.170.0.6]

TASK [keycloak-deploy : Copy user manager roles file to tmp location] **********
ok: [192.170.0.6]

TASK [keycloak-deploy : Copy the keycloak bootstrap script] ********************
ok: [192.170.0.6]

TASK [keycloak-deploy : Run the keycloak bootstrap script] *********************
changed: [192.170.0.6]

TASK [keycloak-deploy : stop keycloak service] *********************************
changed: [192.170.0.6] => {}

MSG:

restarting keycloak


RUNNING HANDLER [keycloak-deploy : stop service] *******************************
changed: [192.170.0.6]

RUNNING HANDLER [keycloak-deploy : wait for keycloak to stop] ******************
ok: [192.170.0.6]

RUNNING HANDLER [keycloak-deploy : start service] ******************************
changed: [192.170.0.6]

RUNNING HANDLER [keycloak-deploy : wait for keycloak to start] *****************
FAILED - RETRYING: wait for keycloak to start (60 retries left).
ok: [192.170.0.6]

PLAY RECAP *********************************************************************
192.170.0.6                : ok=24   changed=12   unreachable=0    failed=0    skipped=1    rescued=0    ignored=1   

[Pipeline] }
[Pipeline] // stage
[Pipeline] }

[Pipeline] // ansiColor
[Pipeline] }
[Pipeline] // stage
[Pipeline] }

[Pipeline] // ansiColor
[Pipeline] ansiColor
[Pipeline] {

[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] stage
[Pipeline] { (Write data)
[Pipeline] sh
+ mkdir -p /var/lib/jenkins/summary/dev
+ touch -a /var/lib/jenkins/summary/dev/summary.txt
[Pipeline] sh
+ [ -f metadata.json ]
+ echo null
[Pipeline] sh
+ [ -f metadata.json ]
+ echo null
[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm, imagetag: null, artifact_version: null, privatebranch: main, publicbranch: release-4.10.1_RC1
[Pipeline] }
[Pipeline] // stage
[Pipeline] }

[Pipeline] // ansiColor
[Pipeline] ansiColor
[Pipeline] {

[Pipeline] stage
[Pipeline] { (slack_notify)
[Pipeline] sh
+ git rev-parse --short HEAD
[Pipeline] sh
+ rev
+ cut -d / -f1
+ git name-rev --name-only HEAD
+ rev
[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] echo
Could not find env specific Slack channel. Check for global slack channel..
[Pipeline] echo
Could not find slack environment variable. Skipping slack notification..
[Pipeline] }
[Pipeline] // stage
[Pipeline] }

[Pipeline] // ansiColor
[Pipeline] ansiColor
[Pipeline] {

[Pipeline] stage
[Pipeline] { (email_notify)
[Pipeline] sh
+ echo Deploy/dev/Kubernetes/KeycloakRealm
[Pipeline] echo
Could not find env specific email group. Check for global email group..
[Pipeline] echo
Could not find global email group variable. Skipping email notification..
[Pipeline] }
[Pipeline] // stage
[Pipeline] }

[Pipeline] // ansiColor
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

Answered by beepdot

Sep 13, 2022

https://github.com/orgs/Sunbird-Ed/discussions/123#discussioncomment-3587559

View full answer

beepdot · 2022-09-13T11:15:00Z

beepdot
Sep 13, 2022

https://github.com/orgs/Sunbird-Ed/discussions/123#discussioncomment-3587559

10 replies

beepdot Sep 14, 2022

LMS is a client. So it would be under clients page. But it looks like you don't have it from the earlier screenshot. Has the Keycloak Realm job successfully run? Any errors?

daveanvit Sep 14, 2022
Author

Yes, it was successful, following is keycloak relm bootstrap.yaml

---
- name: start keycloak service
  debug: msg="starting keycloak"
  notify:
    - start service
    - wait for keycloak to start
  changed_when: true

- meta: flush_handlers

- name: Create admin user
  become_user: "{{ wildfly_user }}"
  command: "{{ wildfly_dir }}/bin/add-user-keycloak.sh -r master -u {{ keycloak_management_user }} -p {{ keycloak_management_password }}"
  when: keycloak_management_user is defined and
        keycloak_management_password is defined
  ignore_errors: true

- name: stop keycloak service
  debug: msg="stopping keycloak"
  notify:
    - stop service
    - wait for keycloak to stop
  changed_when: true

- meta: flush_handlers

- name: start keycloak with script upload feature temporarily
  shell: "nohup {{keycloak_home}}/bin/standalone.sh -Dkeycloak.profile.feature.upload_scripts=enabled -b={{ansible_default_ipv4.address}} -bprivate={{ansible_default_ipv4.address}} --server-config standalone-ha.xml &"
  become: yes
  become_user: "{{ wildfly_user }}"
  run_once: true
  notify: wait for keycloak to start

- meta: flush_handlers

- name: Install dependencies for keycloak
  apt:
  name: ["python-setuptools"]
  update_cache: true
  run_once: true

- name: Ensure python packages are installed
  apt: 
  name: ["python-pip", "python-dev"]
  update_cache: yes
  when: ansible_os_family == 'Debian' and ansible_distribution_version | float < 18

- name: Ensure python packages are installed
  apt:
  name: ["python3-pip", "python3-dev"]
  update_cache: yes
  when: ansible_os_family == 'Debian' and ansible_distribution_version | float > 18

# - name: Copying python libraries
#   copy:
#     src: "{{ role_path }}/files/python-keycloak-0.12.0"
#     dest: /tmp/
#   run_once: true

- name: Initialize python library to run keycloak bootstrap script
  shell: cd /tmp/python-keycloak-0.12.0 && python setup.py install
  run_once: true

- name: Save keycloak vars to json
  template:
    src: "keycloak-bootstrap.conf.j2"
    dest: "/tmp/keycloak-bootstrap.conf.json"
    mode: "0644"
  run_once: true

- name: Copy realm json file to tmp location
  template:
    src: "keycloak-realm.j2"
    dest: "/tmp/keycloak-realm.json"
    mode: "0644"
  run_once: true

- name: Copy user manager roles file to tmp location
  copy:
    src: "files/python-keycloak-0.12.0/roles.json"
    dest: "/tmp/roles.json"
    mode: "0644"
  run_once: true

- name: Copy the keycloak bootstrap script
  copy:
    src: "{{ role_path }}/files/python-keycloak-0.12.0/keycloak"
    dest: /tmp
  run_once: true

- name: Run the keycloak bootstrap script
  shell: cd /tmp/keycloak/ && python keycloak_main.py /tmp/keycloak-bootstrap.conf.json
  register: out
  until: '"404" not in out.stderr'
  retries: 2
  delay: 10
  run_once: true

- name: stop keycloak service
  debug: msg="restarting keycloak"
  notify:
    - stop service
    - wait for keycloak to stop
    - start service
    - wait for keycloak to start
  changed_when: true

- meta: flush_handlers

beepdot Sep 14, 2022

I am not 100% sure what has happened. Please check the jenkins job log once and see there are no errors. If the realm Sunbird got created, the clients should have also been created since all these are present in the same ansible template.
https://github.com/project-sunbird/sunbird-devops/blob/master/ansible/roles/keycloak-deploy/templates/keycloak-realm.j2#L1813

You could also delete the Sunbird realm and recreate it if required in case you have not progressed yet to creating users, orgs etc.
Or you could also create the clients manually.

beepdot Sep 14, 2022

Or did you create the Sunbird realm manually?

daveanvit Sep 15, 2022
Author

@keshavprasadms yes I had setup relm manually, however i had configured it again, and it worked closing this issue.
Thanks @keshavprasadms

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sunbird-ED (Sunbird for Education)

Keycloak Realm Installation is not creating user #157

{{title}}

Replies: 1 comment 10 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Sunbird-ED (Sunbird for Education)

Keycloak Realm Installation is not creating user #157

daveanvit Sep 13, 2022

Replies: 1 comment · 10 replies

beepdot Sep 13, 2022

beepdot Sep 14, 2022

daveanvit Sep 14, 2022 Author

beepdot Sep 14, 2022

beepdot Sep 14, 2022

daveanvit Sep 15, 2022 Author

daveanvit
Sep 13, 2022

Replies: 1 comment 10 replies

beepdot
Sep 13, 2022

daveanvit Sep 14, 2022
Author

daveanvit Sep 15, 2022
Author