Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 Shinyproxy Delivers Incorrect App Instances (Potential Race Condition or Caching Issue) #539

Open
AccsoAndreBuesgen opened this issue Dec 11, 2024 · 2 comments
Labels

Comments

@AccsoAndreBuesgen
Copy link

🐛 Shinyproxy Delivers Incorrect App Instances (Potential Race Condition or Caching Issue)

Hello Shinyproxy Team,
Recently we found some strange behaviour when using shinyproxy and we suspect
that there may be a caching related bug. I tried to gather all relevant information down below.
I hope that you can help us resolve the issue. If there is anything I can do to
assist, let me know!

Description

When using Shinyproxy with Shinyproxy-Operator and a Redis Sentinel cluster to
serve R-Shiny apps without user login, navigating quickly and randomly between
multiple apps can cause Shinyproxy to deliver the wrong application instance.
For example, when requesting https://url/app2, the user may see the content
of app1. Once this incorrect mapping occurs, it persists for the duration of
the browser session and only resets when the session cookie or browser cache is
cleared.

Through browser developer tools, it was confirmed that the endpoint in
ProxyController.java (Line 141) returns the wrong application ID, indicating
that the issue originates server-side. Additionally, tests have shown that
disabling caching entirely resolves the issue, but doing so also disables
essential Shinyproxy-Operator functionality.

Steps to Reproduce

  1. Open the Shinyproxy landing page (e.g., https://url/).
  2. Rapidly navigate between multiple Shiny apps (e.g., https://url/app1 then
    https://url/app2, and so on).
  3. Observe that occasionally, the requested URL displays the wrong application.
  4. Clear the browser’s session cookie or cache to temporarily resolve the
    incorrect mapping.

Expected Behavior

Each requested URL should consistently display the correct app:

  • https://url/app1 should always show app1.
  • https://url/app2 should always show app2.

Actual Behavior

Intermittently, the displayed app at a requested URL is incorrect. For example,
https://url/app2 might show app1. Once the mis-mapping occurs, it persists
until the session cookie is cleared. Meanwhile, the incorrectly launched app
continues running on the server, consuming resources unnecessarily.

Environment Details

  • Cloud Platform: AWS (Amazon EKS)

  • Shinyproxy-Operator:

    • Version: 2.1.0

    • Configuration:

      env:
        - name: SPO_MODE
          value: namespaced
        - name: SPO_PROBE_TIMEOUT
          value: "3"
  • Shinyproxy:

    • Version: 3.1.1

    • Configuration:

      apiVersion: openanalytics.eu/v1
      kind: ShinyProxy
      metadata:
        name: shinyproxy
        namespace: rshiny
      spec:
        server:
          secureCookies: true
          frameOptions: sameorigin
          forward-headers-strategy: native
        spring:
          session:
            store-type: redis
          data:
            redis:
              password: ${REDIS_PASSWORD}
              sentinel:
                master: shinyproxy
                password: ${REDIS_PASSWORD}
                nodes: <redacted>
        proxy:
          store-mode: Redis
          stop-proxies-on-shutdown: false
          title: testproxy
          landingPage: /
          containerBackend: kubernetes
          hide-navbar: true
          kubernetes:
            namespace: rshiny
            internal-networking: true
            image-pull-policy: Always
            image-pull-secret: r-shiny-cloud-images-docker-secret
            privileged: false
          authentication: none
          admin-groups: admins
          specs:
            - id: example-app-1
              container-image: <some-image>
              minimum-seats-available: 2
            - id: example-app-2
              container-image: <some-different-image>
              minimum-seats-available: 2
        kubernetesPodTemplateSpecPatches: |
          - op: add
            path: /spec/containers/0/env/-
            value:
              name: REDIS_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: redis-secret
                  key: redis-password
          - op: add
            path: /spec/containers/0/resources
            value:
              limits:
                cpu: 2
                memory: 2Gi
              requests:
                cpu: 1
                memory: 2Gi
          - op: add
            path: /spec/serviceAccountName
            value: shinyproxy-sa
        imagePullPolicy: Always
        fqdn: <redacted>
  • Database: Redis Sentinel Cluster

  • Browsers: Multiple modern browsers (e.g., Chrome, Firefox)

  • Platform: R-Shiny Applications

  • User Authentication: None (no login required)

Cause and Findings

Here is what we found:

  • Session Handling: Each user session depends on a session cookie mapping
    active apps in the Redis database. Apps remain active until the session ends.

  • Possible Race Condition: When multiple apps or their containers are
    requested in quick succession, a server-side race condition may cause the wrong
    app ID (and corresponding container) to be linked to a requested URL.

  • Persistence of Incorrect Mapping: Once established, the incorrect mapping
    persists for the session’s duration. Clearing the session cookie resolves the
    issue on the client side, but the previously mis-assigned app continues to run,
    consuming server resources.

  • Verified Source: Debugging with the Browser Developer tools we could confirm
    that the endpoint defined in
    ProxyController.java (Line 141)
    returns the wrong app ID, indicating a server-side origin of the issue.

Impact

  • Users may interact with the wrong application, leading to confusion and
    potential data inconsistencies.
  • Unnecessary server resource consumption occurs due to perpetually running,
    incorrectly mapped apps.
  • Overall system performance may degrade due to wasted resources.

I hope that the provided Information will be helpful for you.
Do you know anything we could do to resolve this?

@LEDfan LEDfan added the bug label Dec 11, 2024
@LEDfan
Copy link
Member

LEDfan commented Dec 11, 2024

Hi, thanks for the report. I do have a few questions:

disabling caching entirely

Which caching do you exactly mean?

Does this only happen when the user does not yet have any apps opened, i.e. only for new apps?
Can you provide some logs of ShinyProxy when this happens. It should specify which seat was claimed by which users, which helps debug the issue.

@AccsoAndreBuesgen
Copy link
Author

Hey @LEDfan,
thank you for your fast answer!
Sorry for the confusion: by "disabling caching entirely" I meant using a Shinyproxy without a redis cache configured.

I will try to get some logs and come back to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants