Can Caret-qualified MDX elements in views be stripped?

[IcanMorris]

The TM1 Views users generate often contain the fully qualified path in the raw MDX, for example:
[_test].[_test].[C^D]

When tm1py tries to execute a view that contains an element with a full path that is no longer valid, it immediately stops with an error

TM1: 11.8.01700.1
tm1py: 1.11.3
(in my case called via TM1Service.power_bi.execute_view)

  File "C:\Program Files\Python312\Lib\site-packages\TM1py\Services\RestService.py", line 538, in verify_response
    raise TM1pyRestException(response.text,
TM1py.Exceptions.Exceptions.TM1pyRestException: Text: '{"error":{"code":"501","message":"\"C^D\" : member not found (rte 81)"}}' - Status Code: 400 - Reason: 'Bad Request' - Headers: {'Content-Length': '91', 'Connection': 'keep-alive', 'Content-Encoding': 'gzip', 'Cache-Control': 'no-cache', 'Content-Type': 'application/json; charset=utf-8', 'OData-Version': '4.0'}  

I can see this in the tm1 message log occurring as well.

When PAfE executes the same view, I still see the attempt to resolve the bad-pathed element in the message log, but it manages to strip the broken path and silently continue. When PAfE (well, technically PAW?) modifies a broken path, only the broken path is removed. Other elements containing their path are left alone.
Thus a PAfE user doesn't even notice many structural changes, but a tm1py user notices them immediately.

Except for having them log on with PAfE and save their tm1py processed view, is there already something available that emulates the PAfE behavior?
It looks like the returned error message isn't really great and needs quite some text parsing to automate that implicit dropping of the path qualifier. I can probably just grab all the path-parts with regexp and drop all of them preventative, but that would be different from how PAfE handles it.

Note: For me it was rather hard to produce a path-containing element, this seems to happen when you work on the right half of the subset editor, open your consolidations and shift-select what you like from there. If you did the same on the left, this would not add the unique pathed names. I told them to let me know when they managed to break it on the left part of the window, but every so often someone will accidentally use the right half to sub-select and years from now when something is restructured, this will hit again.

[MariusWirtz]

Interesting question.

I think you could make the view retrieval more robust with some custom logic.

The first solution that comes to mind is:
Instead of execution the view immediately, you could turn it into an MDX and then substitute the MemberUniqueNames with ElementUniqueNames in the MDX string. Then execute the MDX. The data in the resulting dataframe would be the same.

[IcanMorris]

Maybe this helps someone else, so far there are no problems with simply stripping all the unique-ness in all element names. This uniqueness is also only introduced if a node has multiple parents, and it's only a fully qualified path if all elements except the root have more than one parent, otherwise it stops early. In TM1 we're not aware of what ancestry-path an element has and it doesn't change value depending on which parent contains it, so it should be fine to always remove all occurrences. Someone else or IBM themselves clearly ran into this problem before but solved it implicitly in PAW for the one use-case and do not write the corrected MDX back without it being prompted explicitly.
I believe most of you won't run into this if the users stay on the left side of the subset editor as this seems to never produce unnecessary uniqueness. As soon as someone works on the right half, and affected elements have more than one parent, and at a later point their structure is affected, your tm1py MDX query fails.

I went with the following regular expression:

import os
import pandas as pd
import numpy as np
from TM1py import TM1Service
from TM1py import View
from TM1py import Cube
from TM1py import Server
import re

regex = r"((?:\[[^\]]+\]\.){2})\[(?:(?:[^\]]|\]\])*\^)+((?:[^\]]|\]\])*)\]"
subst = "\\1[\\2]"

with TM1Service(address="1.1.1.1", port=8888, user="", password="", ssl=True) as tm1:
    mdx_view = tm1.views.get_mdx_view(cube_name="cube", view_name="view", private=False)
    new_mdx = re.sub(regex, subst, mdx_view.MDX, 0, re.MULTILINE)
    data_frame_result = tm1.power_bi.execute_mdx(new_mdx)
    print(data_frame_result)
    tm1.logout()

https://regex101.com/r/COdMWU/1

PAfE cannot handle the way closed square bracket is encoded (duplication) in dimension/hierarchy names, so I did not consider it. Elements however can (and do... don't ask) contain square brackets and PAfE can handle this as expected, so the regular expression is a little bit more complex.
I'm not super happy with this, but it is a solution that does not rely on people saving the views not producing edge-cases.

I believe the best way would be to not rely on views but execute MDX, but my people using tm1py do not want to bother with MDX and want to just call a named MDX result (a view) while having someone else in charge of updating this view.