beamng error

[rsun0013]

hi, when i run the label-tests command with the example config, i get info.json missing error in beamng once simulator is opened, can i know how to fix it. using main branch of repo, with beamng.tech v 0.25.4.0.

[ChristianBirchler]

Hi @rsun0013,

thanks for your comment. Unfortunately, SDC-Scissor is tested with BeamNG.tech v0.24 as written in the requirements: https://sdc-scissor.readthedocs.io/en/latest/software_guidebook/deployment.html#requirements

The goal is to upgrade SDC-Scissor to the new version of BeamNG, but it seems that for a specific version of BeamNG.tech, there is also a specific version of beamngpy.

If you see the same bug with the v0.24 then feel free to open a bug report.

Thanks for your comment. I hope I could help you. If not then do not hesitate to post your issues.

Best,
Christian

[rsun0013]

hi, this issue is fixed(looks like a error when copying info.json), for generating tests, is there a way to specify any features or attributes we want or is that done randomly, also id like to know what the evaluate models command does exactly. thanks

[ChristianBirchler]

Hi @rsun0013,

evaluate-models applies cross-validation to identify the best models for predicting test scenarios. Are you talking about features for ML models? If so, then SDC-Scissor uses currently a hard-coded set of features as described for the features attribute of the ModelEvaluator class (https://github.com/ChristianBirchler/sdc-scissor/blob/main/sdc_scissor/machine_learning_api/model_evaluator.py). If you want to have user-specific features, then you have to extend the FeatureExtractor class.

For future work, it is indeed an excellent idea to provide also a more useful API for implementing user-specific features. I opened issue #69 for that purpose. Feel free to follow this issue.

Best,
Christian

[rsun0013]

i'm referring to the road features when generating the test cases such as min and max straight road distance, min and max angle and total road length and map size etc, which are currently specified in the config.py file in the ambiegen test generator folder, but when i chnage those values, they dont reflect any change in the features of the tests generated. also, ambiegen test gen doesnt seem to use the test case count argument, is there a way to specify number of test cases we need when using the ambiegen generator?

[ChristianBirchler]

Hi @rsun0013,

The integration of the recent test generators of the SBST2022 is not completely tested since other features had higher priority. Of course, specifying parameters for the test generators should be also done for SDC-Scissor. The Frenetic test generator is the most used one for my research, and therefore all the configurations are mainly for this generator. I will check the adaption also for the other test generators and open an issue for this. In case this feature is urgent for your purposes, feel free to open a pull request with your code.

Best,
Christian

[ChristianBirchler]

I opened issue #74. Please describe your requirements there if it is not complete yet.